mirror of
https://github.com/ckaczor/azuredatastudio.git
synced 2026-01-27 09:35:37 -05:00
Hook up Notebook execution edits (#17943)
* Start rerouting VSCode cell execution APIs. * Add more conversion code. * Convert VSCode notebook registrations into ADS equivalents. * Update vscode notebook provider kernels when notebook controller's supportedLanguages are set. * Update an error message. * Add another session argument. * Add base classes for converting notebook serializers. * Disable some vscode notebook methods. * Disable more vscode APIs. * Disable more stuff. * Start implementing serializer notebook data conversions. * Use direct references to extension host notebook methods, rather than azdata ones. * Add a comment. * Remove a space. * Use import type to fix module loading errors. * Use internal cancellation token class. * Start adding cell output conversion. * Convert data from byte array to a string. * More output work. * Use a Set for proxy filtering. * Start adding tests. * Include metadata in cell conversion. Fix other test failures. * Fix serialize tests. * Add more tests. * Remove wildcard characters from vscode filenames. * Start implementing session details. * Add more kernel info. * Add kernel spec. * Add Future callback wrapper class. * Start implementing execute conversion. * Pass notebook URI to requestExecute. * Start working on CellExecution methods. * Move some code around to fix layering issues. * Use proxy to access browser code, rather than direct imports. * Move files around to fix layering issues. * Remove unused imports. * Start implementing some notebook cell execution behaviors. * Revert some unnecessary extHost API changes. * Check for nbformat. * Also handle nbformat in serialize case. * Active notebook extensions when resolving NotebookInput. * Fix nbformat handling. * Disable VSCode notebooks code. * Filter out notebook services from registration assertion. * Wait for providers to load before calling canResolve. * Use controller's viewType for notebook provider ID, instead of controller ID. * Start adding extHostNotebook tests for new APIs. * Re-order proxy calls. * Remove commented code. * Move vscode provider files to browser folder. Fix RPC serialization issues by using readonly field instead of getter for providerId. * Add a comment. * Remove unnecessary dispose call. * Handle disposable from registerExecuteProvider. * Remove a comment. * Remove unnecessary provider fields. * Remove reference to notebook service to fix circular reference issue in stringify. * Add object types for methods in ADSNotebookController. * Wait for controller languages to be ready before marking session manager as ready. * Add correct promise. * Add undefined return type for optional supportedLanguages property. * Refine promise logic. * Move vscode functionality back to ExtHostNotebook, since the NotebookService can't be passed back over RPC (some kind of circular reference error). * Fix remaining issues from last commit. * Replace "not implemented" methods with placeholder return types in order to enable testing. * Also wait for execution handler to be set before marking session manager as ready. * Fix usage of NotebookRegistry when updating provider description languages. * Refine file extension conversion. * Fix file extension conversion to match ADS extension behavior. * Emit new provider registration event when adding supported languages. * Remove checks for duplicate file providers and kernels. * Fix a test failure. * Fix file extension parsing. * Use default executeManager if one isn't defined for provider in notebookModel. * Add descriptors for waiting on standardKernels registration. * Increase timeout * Add an error message. * Start working on retrieving default kernel from registered providers, rather than always falling back to SQL. * Revert "Start working on retrieving default kernel from registered providers, rather than always falling back to SQL." This reverts commit 1916ea1ce3a0072f51bec683116dc7bb6c7aefdc. * Emit activation events after provider registration. * Wait on standard kernels availability when getting an execute provider. * Throw an error if session manager isn't ready yet. * Actually resolve language promise correctly. * Add some checks for undefined notebook data objects. * Create kernel spec data correctly. * Add extension changes for local testing only. * Clean up test class. * Add a reminder comment. * Undo commented out notebook stuff * Temporarily hard code default kernel. * Retrieve default kernel in notebookModel if it's not already provided. * Revert an import change. * Remove unnecessary method from extHostNotebook. * Move an interface around. * wip * Check for proposed API for some VSCode extHost methods. * Remove a comment. * Fix notebookUtils tests. * Fix notebookModel tests. * Fix notebookFindModel tests. * Fix notebookViewsExtension tests. * Fix remaining notebookView tests. * Refactor output conversion functionality into separate methods. * Update some unit tests for output conversion. * Move a method. * Rename conversion methods to fit acronym styling. * Add another conversion test case. * Revert local testing changes. * Remove old method. * cleanup * Remove some comments. * Move localized string to locConstants. * Add a space to loc string. * Add comments to new SQL Carbon Edit tags. * Create constants for nbformat and nbformat_minor. * Move some vscode-only fields to proposed APIs. * Check for valid state * Properly null check * Adding logging for provider wait timeouts. * wip update * Fix compile * Switch to cell edits * Update docs * Remove custom output type * cleanup * fix * cleanup * more cleanup * Fixes * Fix tests and lint errors Co-authored-by: Cory Rivera <corivera@microsoft.com>
This commit is contained in:
Charles Gagnon
@@ -12,7 +12,7 @@ import { localize } from 'vs/nls';
|
||||
import * as notebookUtils from 'sql/workbench/services/notebook/browser/models/notebookUtils';
|
||||
import { CellTypes, CellType, NotebookChangeType, TextCellEditModes } from 'sql/workbench/services/notebook/common/contracts';
|
||||
import { NotebookModel } from 'sql/workbench/services/notebook/browser/models/notebookModel';
|
||||
import { ICellModel, IOutputChangedEvent, CellExecutionState, ICellModelOptions, ITableUpdatedEvent, CellEditModes, ICaretPosition } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
|
||||
import { ICellModel, IOutputChangedEvent, CellExecutionState, ICellModelOptions, ITableUpdatedEvent, CellEditModes, ICaretPosition, ICellEdit, CellEditType } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
|
||||
import { IConnectionManagementService } from 'sql/platform/connection/common/connectionManagement';
|
||||
import { IConnectionProfile } from 'sql/platform/connection/common/interfaces';
|
||||
import { INotificationService, Severity } from 'vs/platform/notification/common/notification';
|
||||
@@ -31,6 +31,8 @@ import { Disposable } from 'vs/base/common/lifecycle';
|
||||
import * as TelemetryKeys from 'sql/platform/telemetry/common/telemetryKeys';
|
||||
import { IInsightOptions } from 'sql/workbench/common/editor/query/chartState';
|
||||
import { IPosition } from 'vs/editor/common/core/position';
|
||||
import { CellOutputEdit, CellOutputDataEdit } from 'sql/workbench/services/notebook/browser/models/cellEdit';
|
||||
import { ILogService } from 'vs/platform/log/common/log';
|
||||
|
||||
let modelId = 0;
|
||||
const ads_execute_command = 'ads_execute_command';
|
||||
@@ -93,6 +95,7 @@ export class CellModel extends Disposable implements ICellModel {
|
||||
@optional(INotebookService) private _notebookService?: INotebookService,
|
||||
@optional(ICommandService) private _commandService?: ICommandService,
|
||||
@optional(IConfigurationService) private _configurationService?: IConfigurationService,
|
||||
@optional(ILogService) private _logService?: ILogService
|
||||
) {
|
||||
super();
|
||||
this.id = `${modelId++}`;
|
||||
@@ -611,6 +614,7 @@ export class CellModel extends Disposable implements ICellModel {
|
||||
if (tryMatchCellMagic(this.source[0]) !== ads_execute_command || !this._isCommandExecutionSettingEnabled) {
|
||||
const future = kernel.requestExecute({
|
||||
code: content,
|
||||
cellIndex: this.notebookModel.findCellIndex(this),
|
||||
stop_on_error: true,
|
||||
notebookUri: this.notebookModel.notebookUri
|
||||
}, false);
|
||||
@@ -1014,6 +1018,39 @@ export class CellModel extends Disposable implements ICellModel {
|
||||
return CellEditModes.WYSIWYG;
|
||||
}
|
||||
|
||||
public processEdits(edits: ICellEdit[]): void {
|
||||
for (const edit of edits) {
|
||||
switch (edit.type) {
|
||||
case CellEditType.Output:
|
||||
const outputEdit = edit as CellOutputEdit;
|
||||
if (outputEdit.append) {
|
||||
this._outputs.push(...outputEdit.outputs);
|
||||
} else {
|
||||
this._outputs = outputEdit.outputs;
|
||||
}
|
||||
|
||||
break;
|
||||
case CellEditType.OutputData:
|
||||
const outputDataEdit = edit as CellOutputDataEdit;
|
||||
const outputIndex = this._outputs.findIndex(o => outputDataEdit.outputId === o.id);
|
||||
if (outputIndex > -1) {
|
||||
const output = this._outputs[outputIndex] as nb.IExecuteResult;
|
||||
// TODO: Append overwrites existing mime types currently
|
||||
const newData = (edit as CellOutputDataEdit).append ?
|
||||
Object.assign(output.data, outputDataEdit.data) :
|
||||
outputDataEdit.data;
|
||||
output.data = newData;
|
||||
// We create a new object so that angular detects that the content has changed
|
||||
this._outputs[outputIndex] = Object.assign({}, output);
|
||||
} else {
|
||||
this._logService.warn(`Unable to find output with ID ${outputDataEdit.outputId} when processing ReplaceOutputData`);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
this.fireOutputsChanged(false);
|
||||
}
|
||||
|
||||
private setLanguageFromContents(cell: nb.ICellContents): void {
|
||||
if (cell.cell_type === CellTypes.Markdown) {
|
||||
this._language = 'markdown';
|
||||
|
||||
@@ -3,8 +3,9 @@
|
||||
* Licensed under the Source EULA. See License.txt in the project root for license information.
|
||||
*--------------------------------------------------------------------------------------------*/
|
||||
|
||||
import type * as azdata from 'azdata';
|
||||
import { IResourceUndoRedoElement, UndoRedoElementType } from 'vs/platform/undoRedo/common/undoRedo';
|
||||
import { ICellModel, MoveDirection } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
|
||||
import { CellEditType, ICellEdit, ICellModel, MoveDirection } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
|
||||
import { NotebookModel, SplitCell } from 'sql/workbench/services/notebook/browser/models/notebookModel';
|
||||
import * as TelemetryKeys from 'sql/platform/telemetry/common/telemetryKeys';
|
||||
import { deepClone } from 'vs/base/common/objects';
|
||||
@@ -112,3 +113,19 @@ export class ConvertCellTypeEdit implements IResourceUndoRedoElement {
|
||||
this.model.sendNotebookTelemetryActionEvent(TelemetryKeys.NbTelemetryAction.RedoCell, this.cellOperation);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Edit for modifying the outputs of a cell.
|
||||
*/
|
||||
export class CellOutputEdit implements ICellEdit {
|
||||
type = CellEditType.Output;
|
||||
public constructor(public readonly outputs: azdata.nb.ICellOutput[], public readonly append: boolean) { }
|
||||
}
|
||||
|
||||
/**
|
||||
* Edit for modifying the data of a specific output of a cell.
|
||||
*/
|
||||
export class CellOutputDataEdit implements ICellEdit {
|
||||
type = CellEditType.OutputData;
|
||||
public constructor(public readonly outputId: string, public readonly data: azdata.nb.DisplayResultData, public readonly append: boolean) { }
|
||||
}
|
||||
|
||||
@@ -26,7 +26,7 @@ import { QueryResultId } from 'sql/workbench/services/notebook/browser/models/ce
|
||||
import { IPosition } from 'vs/editor/common/core/position';
|
||||
import * as TelemetryKeys from 'sql/platform/telemetry/common/telemetryKeys';
|
||||
import { ITelemetryEventProperties } from 'sql/platform/telemetry/common/telemetry';
|
||||
import { INotebookEditOperation } from 'sql/workbench/api/common/extHostNotebookEditor';
|
||||
import { INotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
|
||||
|
||||
|
||||
export enum ViewMode {
|
||||
@@ -554,6 +554,20 @@ export interface ICellModel {
|
||||
addAttachment(mimeType: string, base64Encoding: string, name: string): string;
|
||||
richTextCursorPosition: ICaretPosition;
|
||||
markdownCursorPosition: IPosition;
|
||||
/**
|
||||
* Processes a list of edits for the cell
|
||||
* @param edits List of edits to apply to the cell
|
||||
*/
|
||||
processEdits(edits: ICellEdit[]): void;
|
||||
}
|
||||
|
||||
export const enum CellEditType {
|
||||
Output,
|
||||
OutputData
|
||||
}
|
||||
|
||||
export interface ICellEdit {
|
||||
readonly type: CellEditType
|
||||
}
|
||||
|
||||
export interface ICaretPosition {
|
||||
|
||||
@@ -9,7 +9,7 @@ import { localize } from 'vs/nls';
|
||||
import { Event, Emitter } from 'vs/base/common/event';
|
||||
import { Disposable } from 'vs/base/common/lifecycle';
|
||||
|
||||
import { IClientSession, INotebookModel, INotebookModelOptions, ICellModel, NotebookContentChange, MoveDirection, ViewMode } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
|
||||
import { IClientSession, INotebookModel, INotebookModelOptions, ICellModel, NotebookContentChange, MoveDirection, ViewMode, ICellEdit } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
|
||||
import { NotebookChangeType, CellType, CellTypes } from 'sql/workbench/services/notebook/common/contracts';
|
||||
import { KernelsLanguage, nbversion } from 'sql/workbench/services/notebook/common/notebookConstants';
|
||||
import * as notebookUtils from 'sql/workbench/services/notebook/browser/models/notebookUtils';
|
||||
@@ -19,7 +19,7 @@ import { NotebookContexts } from 'sql/workbench/services/notebook/browser/models
|
||||
import { IConnectionProfile } from 'sql/platform/connection/common/interfaces';
|
||||
import { INotification, Severity, INotificationService } from 'vs/platform/notification/common/notification';
|
||||
import { URI } from 'vs/base/common/uri';
|
||||
import { ISingleNotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
|
||||
import { INotebookEditOperation, NotebookEditOperationType } from 'sql/workbench/api/common/sqlExtHostTypes';
|
||||
import { ConnectionProfile } from 'sql/platform/connection/common/connectionProfile';
|
||||
import { uriPrefixes } from 'sql/platform/connection/common/utils';
|
||||
import { ILogService } from 'vs/platform/log/common/log';
|
||||
@@ -35,7 +35,7 @@ import { isUUID } from 'vs/base/common/uuid';
|
||||
import { TextModel } from 'vs/editor/common/model/textModel';
|
||||
import { QueryTextEditor } from 'sql/workbench/browser/modelComponents/queryTextEditor';
|
||||
import { CodeEditorWidget } from 'vs/editor/browser/widget/codeEditorWidget';
|
||||
import { AddCellEdit, ConvertCellTypeEdit, DeleteCellEdit, MoveCellEdit, SplitCellEdit } from 'sql/workbench/services/notebook/browser/models/cellEdit';
|
||||
import { AddCellEdit, CellOutputEdit, ConvertCellTypeEdit, DeleteCellEdit, MoveCellEdit, CellOutputDataEdit, SplitCellEdit } from 'sql/workbench/services/notebook/browser/models/cellEdit';
|
||||
import { IUndoRedoService } from 'vs/platform/undoRedo/common/undoRedo';
|
||||
import { deepClone } from 'vs/base/common/objects';
|
||||
|
||||
@@ -870,27 +870,59 @@ export class NotebookModel extends Disposable implements INotebookModel {
|
||||
}
|
||||
}
|
||||
|
||||
pushEditOperations(edits: ISingleNotebookEditOperation[]): void {
|
||||
pushEditOperations(edits: INotebookEditOperation[]): void {
|
||||
if (this.inErrorState || !this._cells) {
|
||||
return;
|
||||
}
|
||||
|
||||
for (let edit of edits) {
|
||||
let newCells: ICellModel[] = [];
|
||||
if (edit.cell) {
|
||||
// TODO: should we validate and complete required missing parameters?
|
||||
let contents: nb.ICellContents = edit.cell as nb.ICellContents;
|
||||
newCells.push(this._notebookOptions.factory.createCell(contents, { notebook: this, isTrusted: this._trustedMode }));
|
||||
this.undoService.pushElement(new AddCellEdit(this, newCells[0], edit.range.start));
|
||||
for (const edit of edits) {
|
||||
const startCell = this.cells[edit.range.start];
|
||||
switch (edit.type) {
|
||||
case NotebookEditOperationType.UpdateCell:
|
||||
if (!startCell) {
|
||||
this.logService.warn(`Did not receive a valid starting cell when processing edit type ${edit.type}`);
|
||||
continue;
|
||||
}
|
||||
startCell.processEdits([
|
||||
new CellOutputEdit(edit.cell.outputs ?? [], !!edit.append)
|
||||
]);
|
||||
break;
|
||||
case NotebookEditOperationType.UpdateCellOutput:
|
||||
if (!startCell) {
|
||||
this.logService.warn(`Did not receive a valid starting cell when processing edit type ${edit.type}`);
|
||||
continue;
|
||||
}
|
||||
const cellEdits: ICellEdit[] = [];
|
||||
edit.cell.outputs?.forEach(o => {
|
||||
const targetOutput = startCell.outputs.find(o2 => o.id === o2.id);
|
||||
if (!targetOutput) {
|
||||
this.logService.warn(`Could not find target output with ID ${o.id} when updating cell output`);
|
||||
return;
|
||||
}
|
||||
cellEdits.push(new CellOutputDataEdit(targetOutput.id, (o as nb.IDisplayData).data, !!edit.append));
|
||||
});
|
||||
startCell.processEdits(cellEdits);
|
||||
break;
|
||||
case NotebookEditOperationType.InsertCell:
|
||||
case NotebookEditOperationType.ReplaceCells:
|
||||
case NotebookEditOperationType.DeleteCell:
|
||||
let newCells: ICellModel[] = [];
|
||||
if (edit.cell) {
|
||||
// TODO: should we validate and complete required missing parameters?
|
||||
let contents: nb.ICellContents = edit.cell as nb.ICellContents;
|
||||
newCells.push(this._notebookOptions.factory.createCell(contents, { notebook: this, isTrusted: this._trustedMode }));
|
||||
this.undoService.pushElement(new AddCellEdit(this, newCells[0], edit.range.start));
|
||||
}
|
||||
this._cells.splice(edit.range.start, edit.range.end - edit.range.start, ...newCells);
|
||||
if (newCells.length > 0) {
|
||||
this.updateActiveCell(newCells[0]);
|
||||
}
|
||||
this._contentChangedEmitter.fire({
|
||||
changeType: NotebookChangeType.CellsModified,
|
||||
isDirty: true
|
||||
});
|
||||
break;
|
||||
}
|
||||
this._cells.splice(edit.range.start, edit.range.end - edit.range.start, ...newCells);
|
||||
if (newCells.length > 0) {
|
||||
this.updateActiveCell(newCells[0]);
|
||||
}
|
||||
this._contentChangedEmitter.fire({
|
||||
changeType: NotebookChangeType.CellsModified,
|
||||
isDirty: true
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ import { URI, UriComponents } from 'vs/base/common/uri';
|
||||
import { RenderMimeRegistry } from 'sql/workbench/services/notebook/browser/outputs/registry';
|
||||
import { ModelFactory } from 'sql/workbench/services/notebook/browser/models/modelFactory';
|
||||
import { IConnectionProfile } from 'sql/platform/connection/common/interfaces';
|
||||
import { ISingleNotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
|
||||
import { INotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
|
||||
import { ICellModel, INotebookModel } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
|
||||
import { NotebookChangeType, CellType } from 'sql/workbench/services/notebook/common/contracts';
|
||||
import { IBootstrapParams } from 'sql/workbench/services/bootstrap/common/bootstrapParams';
|
||||
@@ -221,7 +221,7 @@ export interface INotebookEditor {
|
||||
isDirty(): boolean;
|
||||
isActive(): boolean;
|
||||
isVisible(): boolean;
|
||||
executeEdits(edits: ISingleNotebookEditOperation[]): boolean;
|
||||
executeEdits(edits: INotebookEditOperation[]): boolean;
|
||||
runCell(cell: ICellModel): Promise<boolean>;
|
||||
runAllCells(startCell?: ICellModel, endCell?: ICellModel): Promise<boolean>;
|
||||
clearOutput(cell: ICellModel): Promise<boolean>;
|
||||
|
||||
Reference in New Issue
Block a user