Hook up Notebook execution edits (#17943)

* Start rerouting VSCode cell execution APIs.

* Add more conversion code.

* Convert VSCode notebook registrations into ADS equivalents.

* Update vscode notebook provider kernels when notebook controller's supportedLanguages are set.

* Update an error message.

* Add another session argument.

* Add base classes for converting notebook serializers.

* Disable some vscode notebook methods.

* Disable more vscode APIs.

* Disable more stuff.

* Start implementing serializer notebook data conversions.

* Use direct references to extension host notebook methods, rather than azdata ones.

* Add a comment.

* Remove a space.

* Use import type to fix module loading errors.

* Use internal cancellation token class.

* Start adding cell output conversion.

* Convert data from byte array to a string.

* More output work.

* Use a Set for proxy filtering.

* Start adding tests.

* Include metadata in cell conversion. Fix other test failures.

* Fix serialize tests.

* Add more tests.

* Remove wildcard characters from vscode filenames.

* Start implementing session details.

* Add more kernel info.

* Add kernel spec.

* Add Future callback wrapper class.

* Start implementing execute conversion.

* Pass notebook URI to requestExecute.

* Start working on CellExecution methods.

* Move some code around to fix layering issues.

* Use proxy to access browser code, rather than direct imports.

* Move files around to fix layering issues.

* Remove unused imports.

* Start implementing some notebook cell execution behaviors.

* Revert some unnecessary extHost API changes.

* Check for nbformat.

* Also handle nbformat in serialize case.

* Active notebook extensions when resolving NotebookInput.

* Fix nbformat handling.

* Disable VSCode notebooks code.

* Filter out notebook services from registration assertion.

* Wait for providers to load before calling canResolve.

* Use controller's viewType for notebook provider ID, instead of controller ID.

* Start adding extHostNotebook tests for new APIs.

* Re-order proxy calls.

* Remove commented code.

* Move vscode provider files to browser folder. Fix RPC serialization issues by using readonly field instead of getter for providerId.

* Add a comment.

* Remove unnecessary dispose call.

* Handle disposable from registerExecuteProvider.

* Remove a comment.

* Remove unnecessary provider fields.

* Remove reference to notebook service to fix circular reference issue in stringify.

* Add object types for methods in ADSNotebookController.

* Wait for controller languages to be ready before marking session manager as ready.

* Add correct promise.

* Add undefined return type for optional supportedLanguages property.

* Refine promise logic.

* Move vscode functionality back to ExtHostNotebook, since the NotebookService can't be passed back over RPC (some kind of circular reference error).

* Fix remaining issues from last commit.

* Replace "not implemented" methods with placeholder return types in order to enable testing.

* Also wait for execution handler to be set before marking session manager as ready.

* Fix usage of NotebookRegistry when updating provider description languages.

* Refine file extension conversion.

* Fix file extension conversion to match ADS extension behavior.

* Emit new provider registration event when adding supported languages.

* Remove checks for duplicate file providers and kernels.

* Fix a test failure.

* Fix file extension parsing.

* Use default executeManager if one isn't defined for provider in notebookModel.

* Add descriptors for waiting on standardKernels registration.

* Increase timeout

* Add an error message.

* Start working on retrieving default kernel from registered providers, rather than always falling back to SQL.

* Revert "Start working on retrieving default kernel from registered providers, rather than always falling back to SQL."

This reverts commit 1916ea1ce3a0072f51bec683116dc7bb6c7aefdc.

* Emit activation events after provider registration.

* Wait on standard kernels availability when getting an execute provider.

* Throw an error if session manager isn't ready yet.

* Actually resolve language promise correctly.

* Add some checks for undefined notebook data objects.

* Create kernel spec data correctly.

* Add extension changes for local testing only.

* Clean up test class.

* Add a reminder comment.

* Undo commented out notebook stuff

* Temporarily hard code default kernel.

* Retrieve default kernel in notebookModel if it's not already provided.

* Revert an import change.

* Remove unnecessary method from extHostNotebook.

* Move an interface around.

* wip

* Check for proposed API for some VSCode extHost methods.

* Remove a comment.

* Fix notebookUtils tests.

* Fix notebookModel tests.

* Fix notebookFindModel tests.

* Fix notebookViewsExtension tests.

* Fix remaining notebookView tests.

* Refactor output conversion functionality into separate methods.

* Update some unit tests for output conversion.

* Move a method.

* Rename conversion methods to fit acronym styling.

* Add another conversion test case.

* Revert local testing changes.

* Remove old method.

* cleanup

* Remove some comments.

* Move localized string to locConstants.

* Add a space to loc string.

* Add comments to new SQL Carbon Edit tags.

* Create constants for nbformat and nbformat_minor.

* Move some vscode-only fields to proposed APIs.

* Check for valid state

* Properly null check

* Adding logging for provider wait timeouts.

* wip update

* Fix compile

* Switch to cell edits

* Update docs

* Remove custom output type

* cleanup

* fix

* cleanup

* more cleanup

* Fixes

* Fix tests and lint errors

Co-authored-by: Cory Rivera <corivera@microsoft.com>
This commit is contained in:
Charles Gagnon
2022-01-04 16:35:16 -08:00
committed by GitHub
parent c7bb58ceb9
commit 4a2b31f3ba
24 changed files with 304 additions and 97 deletions

View File

@@ -90,7 +90,7 @@ export class JupyterKernel implements nb.IKernel {
requestExecute(content: nb.IExecuteRequest, disposeOnDone?: boolean): nb.IFuture {
content.code = Array.isArray(content.code) ? content.code.join('') : content.code;
content.code = content.code.replace(/\r+\n/gm, '\n'); // Remove \r (if it exists) from newlines
let futureImpl = this.kernelImpl.requestExecute(content as KernelMessage.IExecuteRequest, disposeOnDone);
let futureImpl = this.kernelImpl.requestExecute(content as KernelMessage.IExecuteRequest & { cellIndex: number }, disposeOnDone);
return new JupyterFuture(futureImpl);
}

View File

@@ -97,7 +97,8 @@ describe('Jupyter Session', function (): void {
// When I request execute
let future = kernel.requestExecute({
code: code
code: code,
cellIndex: 0
}, true);
// Then expect wrapper to be returned

11
src/sql/azdata.d.ts vendored
View File

@@ -5129,12 +5129,17 @@ declare module 'azdata' {
*/
text: MultilineString;
}
/**
* Mime type -> contents mappings
*/
export type DisplayResultData = { [key: string]: any };
export interface IDisplayResult extends ICellOutput {
/**
* Mime bundle expected to contain mime type -> contents mappings.
* This is dynamic and is controlled by kernels, so cannot be more specific
* The output data to display as a mapping object of mime type to contents
*/
data: { [key: string]: any };
data: DisplayResultData;
}
export interface IDisplayData extends IDisplayResult {
output_type: 'display_data';

View File

@@ -39,7 +39,10 @@ declare module 'azdata' {
}
export interface ICellOutput {
id?: string; // Unique identifier for this cell output
/**
* Unique identifier for this cell output.
*/
id?: string;
}
export interface IExecuteResult {
@@ -53,7 +56,14 @@ declare module 'azdata' {
}
export interface IExecuteRequest {
notebookUri?: vscode.Uri; // URI of the notebook document that is sending this execute request
/**
* URI of the notebook document that is sending this execute request.
*/
notebookUri?: vscode.Uri;
/**
* The index of the cell which the code being executed is from.
*/
cellIndex: number;
}
export interface INotebookMetadata {

View File

@@ -19,7 +19,7 @@ import {
} from 'sql/workbench/api/common/sqlExtHost.protocol';
import { NotebookInput } from 'sql/workbench/contrib/notebook/browser/models/notebookInput';
import { INotebookService, INotebookEditor } from 'sql/workbench/services/notebook/browser/notebookService';
import { ISingleNotebookEditOperation, NotebookChangeKind } from 'sql/workbench/api/common/sqlExtHostTypes';
import { INotebookEditOperation, NotebookChangeKind } from 'sql/workbench/api/common/sqlExtHostTypes';
import { disposed } from 'vs/base/common/errors';
import { ICellModel, NotebookContentChange, INotebookModel } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { NotebookChangeType, CellTypes } from 'sql/workbench/services/notebook/common/contracts';
@@ -94,7 +94,7 @@ class MainThreadNotebookEditor extends Disposable {
return input.notebookUri.toString() === this.editor.notebookParams.input.notebookUri.toString();
}
public applyEdits(versionIdCheck: number, edits: ISingleNotebookEditOperation[], opts: IUndoStopOptions): boolean {
public applyEdits(versionIdCheck: number, edits: INotebookEditOperation[], opts: IUndoStopOptions): boolean {
// TODO Handle version tracking
// if (this._model.getVersionId() !== versionIdCheck) {
// // throw new Error('Model has changed in the meantime!');
@@ -351,7 +351,7 @@ export class MainThreadNotebookDocumentsAndEditors extends Disposable implements
return this._notebookService.setTrusted(uri, isTrusted);
}
$tryApplyEdits(id: string, modelVersionId: number, edits: ISingleNotebookEditOperation[], opts: IUndoStopOptions): Promise<boolean> {
$tryApplyEdits(id: string, modelVersionId: number, edits: INotebookEditOperation[], opts: IUndoStopOptions): Promise<boolean> {
let editor = this.getEditor(id);
if (!editor) {
return Promise.reject(disposed(`TextEditor(${id})`));

View File

@@ -9,6 +9,11 @@ import { INotebookKernelDto2 } from 'vs/workbench/api/common/extHost.protocol';
import { Emitter, Event } from 'vs/base/common/event';
import * as extHostTypeConverters from 'vs/workbench/api/common/extHostTypeConverters';
import { Deferred } from 'sql/base/common/promise';
import { ExtHostNotebookDocumentsAndEditors } from 'sql/workbench/api/common/extHostNotebookDocumentsAndEditors';
import { URI } from 'vs/base/common/uri';
import { VSCodeContentManager } from 'sql/workbench/api/common/vscodeSerializationProvider';
import { NotebookCellExecutionTaskState } from 'vs/workbench/api/common/extHostNotebookKernels';
import { asArray } from 'vs/base/common/arrays';
type SelectionChangedEvent = { selected: boolean, notebook: vscode.NotebookDocument; };
type MessageReceivedEvent = { editor: vscode.NotebookEditor, message: any; };
@@ -34,6 +39,7 @@ export class ADSNotebookController implements vscode.NotebookController {
private _viewType: string,
private _label: string,
private _addLanguagesHandler: (providerId, languages) => void,
private _extHostNotebookDocumentsAndEditors: ExtHostNotebookDocumentsAndEditors,
private _handler?: ExecutionHandler,
preloads?: vscode.NotebookRendererScript[]
) {
@@ -135,7 +141,7 @@ export class ADSNotebookController implements vscode.NotebookController {
}
public createNotebookCellExecution(cell: vscode.NotebookCell): vscode.NotebookCellExecution {
return new ADSNotebookCellExecution(cell);
return new ADSNotebookCellExecution(cell, this._extHostNotebookDocumentsAndEditors);
}
public dispose(): void {
@@ -157,7 +163,8 @@ export class ADSNotebookController implements vscode.NotebookController {
class ADSNotebookCellExecution implements vscode.NotebookCellExecution {
private _executionOrder: number;
constructor(private readonly _cell: vscode.NotebookCell) {
private _state = NotebookCellExecutionTaskState.Init;
constructor(private readonly _cell: vscode.NotebookCell, private readonly _extHostNotebookDocumentsAndEditors: ExtHostNotebookDocumentsAndEditors) {
this._executionOrder = this._cell.executionSummary?.executionOrder ?? -1;
}
@@ -178,30 +185,69 @@ class ADSNotebookCellExecution implements vscode.NotebookCellExecution {
}
public start(startTime?: number): void {
// No-op
this._state = NotebookCellExecutionTaskState.Started;
}
public end(success: boolean, endTime?: number): void {
// No-op
this._state = NotebookCellExecutionTaskState.Resolved;
}
public async clearOutput(cell?: vscode.NotebookCell): Promise<void> {
// No-op
this.verifyStateForOutput();
const targetCell = typeof cell === 'number' ? this._cell.notebook.cellAt(cell) : (cell ?? this._cell);
const editor = this._extHostNotebookDocumentsAndEditors.getEditor(URI.from(targetCell.notebook.uri).toString());
await editor.clearOutput(editor.document.cells[targetCell.index]);
}
public async replaceOutput(out: vscode.NotebookCellOutput | vscode.NotebookCellOutput[], cell?: vscode.NotebookCell): Promise<void> {
// No-op
this.verifyStateForOutput();
return this.updateOutputs(out, cell, false);
}
public async appendOutput(out: vscode.NotebookCellOutput | vscode.NotebookCellOutput[], cell?: vscode.NotebookCell): Promise<void> {
// No-op
this.verifyStateForOutput();
return this.updateOutputs(out, cell, true);
}
public async replaceOutputItems(items: vscode.NotebookCellOutputItem | vscode.NotebookCellOutputItem[], output: vscode.NotebookCellOutput): Promise<void> {
// No-op
this.verifyStateForOutput();
return this.updateOutputItems(items, output, false);
}
public async appendOutputItems(items: vscode.NotebookCellOutputItem | vscode.NotebookCellOutputItem[], output: vscode.NotebookCellOutput): Promise<void> {
// No-op
this.verifyStateForOutput();
return this.updateOutputItems(items, output, true);
}
private async updateOutputs(outputs: vscode.NotebookCellOutput | vscode.NotebookCellOutput[], cell: vscode.NotebookCell | number | undefined, append: boolean): Promise<void> {
this.verifyStateForOutput();
const targetCell = typeof cell === 'number' ? this._cell.notebook.cellAt(cell) : (cell ?? this._cell);
const editor = this._extHostNotebookDocumentsAndEditors.getEditor(URI.from(targetCell.notebook.uri).toString());
await editor.edit(builder => {
const adsOutputs = VSCodeContentManager.convertToADSCellOutput(outputs);
builder.updateCell(targetCell.index, { outputs: adsOutputs }, append);
});
}
private async updateOutputItems(items: vscode.NotebookCellOutputItem | vscode.NotebookCellOutputItem[], output: vscode.NotebookCellOutput, append: boolean): Promise<void> {
this.verifyStateForOutput();
const editor = this._extHostNotebookDocumentsAndEditors.getEditor(URI.from(this._cell.notebook.uri).toString());
await editor.edit(builder => {
const adsOutput = VSCodeContentManager.convertToADSCellOutput({ id: output.id, items: asArray(items) }, undefined);
builder.updateCellOutput(this._cell.index, { outputs: adsOutput }, append);
});
}
/**
* Verify that the execution is in a state where it's valid to modify the output (currently executing).
*/
private verifyStateForOutput() {
if (this._state === NotebookCellExecutionTaskState.Init) {
throw new Error('Must call start before modifying cell output');
}
if (this._state === NotebookCellExecutionTaskState.Resolved) {
throw new Error('Cannot modify cell output after calling end');
}
}
}

View File

@@ -17,6 +17,7 @@ import { VSCodeSerializationProvider } from 'sql/workbench/api/common/vscodeSeri
import { IExtensionDescription } from 'vs/platform/extensions/common/extensions';
import { ADSNotebookController } from 'sql/workbench/api/common/adsNotebookController';
import { VSCodeExecuteProvider } from 'sql/workbench/api/common/vscodeExecuteProvider';
import { ExtHostNotebookDocumentsAndEditors } from 'sql/workbench/api/common/extHostNotebookDocumentsAndEditors';
type Adapter = azdata.nb.NotebookSerializationProvider | azdata.nb.SerializationManager | azdata.nb.NotebookExecuteProvider | azdata.nb.ExecuteManager | azdata.nb.ISession | azdata.nb.IKernel | azdata.nb.IFuture;
@@ -27,7 +28,7 @@ export class ExtHostNotebook implements ExtHostNotebookShape {
private _adapters = new Map<number, Adapter>();
// Notebook URI to manager lookup.
constructor(_mainContext: IMainContext) {
constructor(_mainContext: IMainContext, private _extHostNotebookDocumentsAndEditors: ExtHostNotebookDocumentsAndEditors) {
this._proxy = _mainContext.getProxy(SqlMainContext.MainThreadNotebook);
}
@@ -264,7 +265,7 @@ export class ExtHostNotebook implements ExtHostNotebookShape {
createNotebookController(extension: IExtensionDescription, id: string, viewType: string, label: string, handler?: (cells: vscode.NotebookCell[], notebook: vscode.NotebookDocument, controller: vscode.NotebookController) => void | Thenable<void>, rendererScripts?: vscode.NotebookRendererScript[]): vscode.NotebookController {
let addLanguagesHandler = (id, languages) => this._proxy.$updateProviderDescriptionLanguages(id, languages);
let controller = new ADSNotebookController(extension, id, viewType, label, addLanguagesHandler, handler, extension.enableProposedApi ? rendererScripts : undefined);
let controller = new ADSNotebookController(extension, id, viewType, label, addLanguagesHandler, this._extHostNotebookDocumentsAndEditors, handler, extension.enableProposedApi ? rendererScripts : undefined);
let executeProvider = new VSCodeExecuteProvider(controller);
this.registerExecuteProvider(executeProvider);
return controller;

View File

@@ -12,15 +12,9 @@ import { readonly } from 'vs/base/common/errors';
import { MainThreadNotebookDocumentsAndEditorsShape } from 'sql/workbench/api/common/sqlExtHost.protocol';
import { ExtHostNotebookDocumentData } from 'sql/workbench/api/common/extHostNotebookDocumentData';
import { CellRange, ISingleNotebookEditOperation, ICellRange } from 'sql/workbench/api/common/sqlExtHostTypes';
import { CellRange, INotebookEditOperation, ICellRange, NotebookEditOperationType } from 'sql/workbench/api/common/sqlExtHostTypes';
import { HideInputTag } from 'sql/platform/notebooks/common/outputRegistry';
export interface INotebookEditOperation {
range: azdata.nb.CellRange;
cell: Partial<azdata.nb.ICellContents>;
forceMoveMarkers: boolean;
}
export interface INotebookEditData {
documentVersionId: number;
edits: INotebookEditOperation[];
@@ -64,7 +58,7 @@ export class NotebookEditorEdit {
replace(location: number | CellRange, value: Partial<azdata.nb.ICellContents>): void {
let range: CellRange = this.getAsRange(location);
this._pushEdit(range, value, false);
this._pushEdit(NotebookEditOperationType.ReplaceCells, range, value);
}
private getAsRange(location: number | CellRange): CellRange {
@@ -99,7 +93,7 @@ export class NotebookEditorEdit {
value.metadata.tags.push(HideInputTag);
}
}
this._pushEdit(new CellRange(index, index), value, true);
this._pushEdit(NotebookEditOperationType.InsertCell, new CellRange(index, index), value);
}
deleteCell(index: number): void {
@@ -114,15 +108,24 @@ export class NotebookEditorEdit {
throw new Error('Unrecognized index');
}
this._pushEdit(range, null, true);
this._pushEdit(NotebookEditOperationType.DeleteCell, range, null);
}
private _pushEdit(range: azdata.nb.CellRange, cell: Partial<azdata.nb.ICellContents>, forceMoveMarkers: boolean): void {
updateCell(index: number, updatedContent: Partial<azdata.nb.ICellContents>, append: boolean): void {
this._pushEdit(NotebookEditOperationType.UpdateCell, new CellRange(index, index + 1), updatedContent, append);
}
updateCellOutput(cellIndex: number, updatedContent: Partial<azdata.nb.ICellContents>, append: boolean): void {
this._pushEdit(NotebookEditOperationType.UpdateCellOutput, new CellRange(cellIndex, cellIndex + 1), updatedContent, append);
}
private _pushEdit(type: NotebookEditOperationType, range: azdata.nb.CellRange, cell: Partial<azdata.nb.ICellContents>, append?: boolean): void {
let validRange = this._document.validateCellRange(range);
this._collectedEdits.push({
type: type,
range: validRange,
cell: cell,
forceMoveMarkers: forceMoveMarkers
append: append
});
}
}
@@ -188,7 +191,7 @@ export class ExtHostNotebookEditor implements azdata.nb.NotebookEditor, IDisposa
return this._proxy.$changeKernel(this._id, kernel);
}
public edit(callback: (editBuilder: azdata.nb.NotebookEditorEdit) => void, options?: { undoStopBefore: boolean; undoStopAfter: boolean; }): Thenable<boolean> {
public edit(callback: (editBuilder: NotebookEditorEdit) => void, options?: { undoStopBefore: boolean; undoStopAfter: boolean; }): Thenable<boolean> {
if (this._disposed) {
return Promise.reject(new Error('NotebookEditor#edit not possible on closed editors'));
}
@@ -228,11 +231,12 @@ export class ExtHostNotebookEditor implements azdata.nb.NotebookEditor, IDisposa
}
// prepare data for serialization
let edits: ISingleNotebookEditOperation[] = editData.edits.map((edit) => {
let edits: INotebookEditOperation[] = editData.edits.map((edit) => {
return {
type: edit.type,
range: toICellRange(edit.range),
cell: edit.cell,
forceMoveMarkers: edit.forceMoveMarkers
append: edit.append
};
});

View File

@@ -89,8 +89,8 @@ export function createAdsApiFactory(accessor: ServicesAccessor): IAdsExtensionAp
const extHostDashboard = rpcProtocol.set(SqlExtHostContext.ExtHostDashboard, new ExtHostDashboard(rpcProtocol));
const extHostModelViewDialog = rpcProtocol.set(SqlExtHostContext.ExtHostModelViewDialog, new ExtHostModelViewDialog(rpcProtocol, extHostModelView, extHostBackgroundTaskManagement));
const extHostQueryEditor = rpcProtocol.set(SqlExtHostContext.ExtHostQueryEditor, new ExtHostQueryEditor(rpcProtocol));
const extHostNotebook = rpcProtocol.set(SqlExtHostContext.ExtHostNotebook, new ExtHostNotebook(rpcProtocol));
const extHostNotebookDocumentsAndEditors = rpcProtocol.set(SqlExtHostContext.ExtHostNotebookDocumentsAndEditors, new ExtHostNotebookDocumentsAndEditors(rpcProtocol));
const extHostNotebook = rpcProtocol.set(SqlExtHostContext.ExtHostNotebook, new ExtHostNotebook(rpcProtocol, extHostNotebookDocumentsAndEditors));
const extHostExtensionManagement = rpcProtocol.set(SqlExtHostContext.ExtHostExtensionManagement, new ExtHostExtensionManagement(rpcProtocol));
const extHostWorkspace = rpcProtocol.set(SqlExtHostContext.ExtHostWorkspace, new ExtHostWorkspace(rpcProtocol));
return {

View File

@@ -19,7 +19,7 @@ import { ITaskHandlerDescription } from 'sql/workbench/services/tasks/common/tas
import {
IItemConfig, IComponentShape, IModelViewDialogDetails, IModelViewTabDetails, IModelViewButtonDetails,
IModelViewWizardDetails, IModelViewWizardPageDetails, IExecuteManagerDetails, INotebookSessionDetails,
INotebookKernelDetails, INotebookFutureDetails, FutureMessageType, INotebookFutureDone, ISingleNotebookEditOperation,
INotebookKernelDetails, INotebookFutureDetails, FutureMessageType, INotebookFutureDone, INotebookEditOperation,
NotebookChangeKind,
ISerializationManagerDetails
} from 'sql/workbench/api/common/sqlExtHostTypes';
@@ -990,7 +990,7 @@ export interface MainThreadNotebookDocumentsAndEditorsShape extends IDisposable
$trySetTrusted(_uri: UriComponents, isTrusted: boolean): Thenable<boolean>;
$trySaveDocument(uri: UriComponents): Thenable<boolean>;
$tryShowNotebookDocument(resource: UriComponents, options: INotebookShowOptions): Promise<string>;
$tryApplyEdits(id: string, modelVersionId: number, edits: ISingleNotebookEditOperation[], opts: IUndoStopOptions): Promise<boolean>;
$tryApplyEdits(id: string, modelVersionId: number, edits: INotebookEditOperation[], opts: IUndoStopOptions): Promise<boolean>;
$runCell(id: string, cellUri: UriComponents): Promise<boolean>;
$runAllCells(id: string, startCellUri?: UriComponents, endCellUri?: UriComponents): Promise<boolean>;
$clearOutput(id: string, cellUri: UriComponents): Promise<boolean>;

View File

@@ -633,10 +633,47 @@ export class CellRange {
}
}
export interface ISingleNotebookEditOperation {
export const enum NotebookEditOperationType {
/**
* Inserts a new cell with the specified content at the specified position.
*/
InsertCell = 0,
/**
* Deletes a single cell.
*/
DeleteCell = 1,
/**
* Replace the specified cell range with a new cell made from the specified content.
*/
ReplaceCells = 2,
/**
* Update a cell with the specified new values. Currently only supports updating cell output.
*/
UpdateCell = 3,
/**
* Updates a cell outputs with the specified new values.
*/
UpdateCellOutput = 4
}
// TODO This should be split up into separate edit operation types
export interface INotebookEditOperation {
/**
* The type of edit operation this is
*/
type: NotebookEditOperationType;
/**
* The range of cells that this edit affects
*/
range: ICellRange;
/**
* The cell metadata to use for the edit operation (only for some edit operations)
*/
cell: Partial<nb.ICellContents>;
forceMoveMarkers: boolean;
/**
* Whether to append the content to the existing content or replace it.
*/
append?: boolean;
}
export class ConnectionProfile {

View File

@@ -130,6 +130,7 @@ class VSCodeKernel implements azdata.nb.IKernel {
let executePromise: Promise<void>;
if (this._controller.executeHandler) {
let cell = <vscode.NotebookCell>{
index: content.cellIndex,
document: <vscode.TextDocument>{
uri: content.notebookUri,
languageId: this._kernelSpec.language,

View File

@@ -9,27 +9,27 @@ import { VSBuffer } from 'vs/base/common/buffer';
import { NotebookCellKind } from 'vs/workbench/api/common/extHostTypes';
import { CancellationTokenSource } from 'vs/base/common/cancellation';
import { OutputTypes } from 'sql/workbench/services/notebook/common/contracts';
import { asArray } from 'vs/base/common/arrays';
import { NBFORMAT, NBFORMAT_MINOR } from 'sql/workbench/common/constants';
/**
* A Notebook Content Manager that is used as part of converting VS Code notebook extension APIs into ADS equivalents.
*/
export class VSCodeContentManager implements azdata.nb.ContentManager {
constructor(private readonly _serializer: vscode.NotebookSerializer) {
}
public static convertToADSCellOutput(output: vscode.NotebookCellOutput, executionOrder?: number): azdata.nb.IExecuteResult {
let outputData = {};
for (let item of output.items) {
outputData[item.mime] = VSBuffer.wrap(item.data).toString();
}
return {
output_type: 'execute_result',
data: outputData,
execution_count: executionOrder,
metadata: output.metadata,
id: output.id
};
public static convertToADSCellOutput(outputs: vscode.NotebookCellOutput | vscode.NotebookCellOutput[], executionOrder?: number): azdata.nb.IDisplayResult[] {
return asArray(outputs).map(output => {
let outputData = {};
for (let item of output.items) {
outputData[item.mime] = VSBuffer.wrap(item.data).toString();
}
return {
output_type: 'execute_result',
data: outputData,
execution_count: executionOrder,
metadata: output.metadata,
id: output.id
};
});
}
public async deserializeNotebook(contents: string): Promise<azdata.nb.INotebookContents> {
@@ -45,7 +45,7 @@ export class VSCodeContentManager implements azdata.nb.ContentManager {
language: cell.languageId
},
execution_count: executionOrder,
outputs: cell.outputs?.map<azdata.nb.IExecuteResult>(output => VSCodeContentManager.convertToADSCellOutput(output, executionOrder))
outputs: cell.outputs ? VSCodeContentManager.convertToADSCellOutput(cell.outputs, executionOrder) : undefined
};
}),
metadata: notebookData.metadata ?? {},

View File

@@ -30,7 +30,7 @@ import { Deferred } from 'sql/base/common/promise';
import { Taskbar } from 'sql/base/browser/ui/taskbar/taskbar';
import { AddCellAction, KernelsDropdown, AttachToDropdown, TrustedAction, RunAllCellsAction, ClearAllOutputsAction, CollapseCellsAction, RunParametersAction, NotebookViewsActionProvider } from 'sql/workbench/contrib/notebook/browser/notebookActions';
import { DropdownMenuActionViewItem } from 'sql/base/browser/ui/buttonMenu/buttonMenu';
import { ISingleNotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
import { INotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
import { IConnectionDialogService } from 'sql/workbench/services/connection/common/connectionDialogService';
import { ICapabilitiesService } from 'sql/platform/capabilities/common/capabilitiesService';
import { CellModel } from 'sql/workbench/services/notebook/browser/models/cell';
@@ -696,7 +696,7 @@ export class NotebookComponent extends AngularDisposable implements OnInit, OnDe
return this.notebookParams.input.isDirty();
}
executeEdits(edits: ISingleNotebookEditOperation[]): boolean {
executeEdits(edits: INotebookEditOperation[]): boolean {
if (!edits || edits.length === 0) {
return false;
}

View File

@@ -37,7 +37,7 @@ import * as DOM from 'vs/base/browser/dom';
import { AnchorAlignment } from 'vs/base/browser/ui/contextview/contextview';
import { LabeledMenuItemActionItem } from 'sql/platform/actions/browser/menuEntryActionViewItem';
import { IActionViewItem } from 'vs/base/browser/ui/actionbar/actionbar';
import { INotebookEditOperation } from 'sql/workbench/api/common/extHostNotebookEditor';
import { INotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
export const NOTEBOOKVIEWS_SELECTOR: string = 'notebook-view-component';

View File

@@ -23,7 +23,7 @@ import { NotebookViewsExtension } from 'sql/workbench/services/notebook/browser/
import { INotebookView, INotebookViewCell, INotebookViewMetadata, INotebookViews } from 'sql/workbench/services/notebook/browser/notebookViews/notebookViews';
import * as TelemetryKeys from 'sql/platform/telemetry/common/telemetryKeys';
import { ITelemetryEventProperties } from 'sql/platform/telemetry/common/telemetry';
import { INotebookEditOperation } from 'sql/workbench/api/common/extHostNotebookEditor';
import { INotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
export class NotebookModelStub implements INotebookModel {
constructor(private _languageInfo?: nb.ILanguageInfo, private _cells?: ICellModel[], private _testContents?: nb.INotebookContents) {

View File

@@ -12,7 +12,7 @@ import { localize } from 'vs/nls';
import * as notebookUtils from 'sql/workbench/services/notebook/browser/models/notebookUtils';
import { CellTypes, CellType, NotebookChangeType, TextCellEditModes } from 'sql/workbench/services/notebook/common/contracts';
import { NotebookModel } from 'sql/workbench/services/notebook/browser/models/notebookModel';
import { ICellModel, IOutputChangedEvent, CellExecutionState, ICellModelOptions, ITableUpdatedEvent, CellEditModes, ICaretPosition } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { ICellModel, IOutputChangedEvent, CellExecutionState, ICellModelOptions, ITableUpdatedEvent, CellEditModes, ICaretPosition, ICellEdit, CellEditType } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { IConnectionManagementService } from 'sql/platform/connection/common/connectionManagement';
import { IConnectionProfile } from 'sql/platform/connection/common/interfaces';
import { INotificationService, Severity } from 'vs/platform/notification/common/notification';
@@ -31,6 +31,8 @@ import { Disposable } from 'vs/base/common/lifecycle';
import * as TelemetryKeys from 'sql/platform/telemetry/common/telemetryKeys';
import { IInsightOptions } from 'sql/workbench/common/editor/query/chartState';
import { IPosition } from 'vs/editor/common/core/position';
import { CellOutputEdit, CellOutputDataEdit } from 'sql/workbench/services/notebook/browser/models/cellEdit';
import { ILogService } from 'vs/platform/log/common/log';
let modelId = 0;
const ads_execute_command = 'ads_execute_command';
@@ -93,6 +95,7 @@ export class CellModel extends Disposable implements ICellModel {
@optional(INotebookService) private _notebookService?: INotebookService,
@optional(ICommandService) private _commandService?: ICommandService,
@optional(IConfigurationService) private _configurationService?: IConfigurationService,
@optional(ILogService) private _logService?: ILogService
) {
super();
this.id = `${modelId++}`;
@@ -611,6 +614,7 @@ export class CellModel extends Disposable implements ICellModel {
if (tryMatchCellMagic(this.source[0]) !== ads_execute_command || !this._isCommandExecutionSettingEnabled) {
const future = kernel.requestExecute({
code: content,
cellIndex: this.notebookModel.findCellIndex(this),
stop_on_error: true,
notebookUri: this.notebookModel.notebookUri
}, false);
@@ -1014,6 +1018,39 @@ export class CellModel extends Disposable implements ICellModel {
return CellEditModes.WYSIWYG;
}
public processEdits(edits: ICellEdit[]): void {
for (const edit of edits) {
switch (edit.type) {
case CellEditType.Output:
const outputEdit = edit as CellOutputEdit;
if (outputEdit.append) {
this._outputs.push(...outputEdit.outputs);
} else {
this._outputs = outputEdit.outputs;
}
break;
case CellEditType.OutputData:
const outputDataEdit = edit as CellOutputDataEdit;
const outputIndex = this._outputs.findIndex(o => outputDataEdit.outputId === o.id);
if (outputIndex > -1) {
const output = this._outputs[outputIndex] as nb.IExecuteResult;
// TODO: Append overwrites existing mime types currently
const newData = (edit as CellOutputDataEdit).append ?
Object.assign(output.data, outputDataEdit.data) :
outputDataEdit.data;
output.data = newData;
// We create a new object so that angular detects that the content has changed
this._outputs[outputIndex] = Object.assign({}, output);
} else {
this._logService.warn(`Unable to find output with ID ${outputDataEdit.outputId} when processing ReplaceOutputData`);
}
break;
}
}
this.fireOutputsChanged(false);
}
private setLanguageFromContents(cell: nb.ICellContents): void {
if (cell.cell_type === CellTypes.Markdown) {
this._language = 'markdown';

View File

@@ -3,8 +3,9 @@
* Licensed under the Source EULA. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import type * as azdata from 'azdata';
import { IResourceUndoRedoElement, UndoRedoElementType } from 'vs/platform/undoRedo/common/undoRedo';
import { ICellModel, MoveDirection } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { CellEditType, ICellEdit, ICellModel, MoveDirection } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { NotebookModel, SplitCell } from 'sql/workbench/services/notebook/browser/models/notebookModel';
import * as TelemetryKeys from 'sql/platform/telemetry/common/telemetryKeys';
import { deepClone } from 'vs/base/common/objects';
@@ -112,3 +113,19 @@ export class ConvertCellTypeEdit implements IResourceUndoRedoElement {
this.model.sendNotebookTelemetryActionEvent(TelemetryKeys.NbTelemetryAction.RedoCell, this.cellOperation);
}
}
/**
* Edit for modifying the outputs of a cell.
*/
export class CellOutputEdit implements ICellEdit {
type = CellEditType.Output;
public constructor(public readonly outputs: azdata.nb.ICellOutput[], public readonly append: boolean) { }
}
/**
* Edit for modifying the data of a specific output of a cell.
*/
export class CellOutputDataEdit implements ICellEdit {
type = CellEditType.OutputData;
public constructor(public readonly outputId: string, public readonly data: azdata.nb.DisplayResultData, public readonly append: boolean) { }
}

View File

@@ -26,7 +26,7 @@ import { QueryResultId } from 'sql/workbench/services/notebook/browser/models/ce
import { IPosition } from 'vs/editor/common/core/position';
import * as TelemetryKeys from 'sql/platform/telemetry/common/telemetryKeys';
import { ITelemetryEventProperties } from 'sql/platform/telemetry/common/telemetry';
import { INotebookEditOperation } from 'sql/workbench/api/common/extHostNotebookEditor';
import { INotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
export enum ViewMode {
@@ -554,6 +554,20 @@ export interface ICellModel {
addAttachment(mimeType: string, base64Encoding: string, name: string): string;
richTextCursorPosition: ICaretPosition;
markdownCursorPosition: IPosition;
/**
* Processes a list of edits for the cell
* @param edits List of edits to apply to the cell
*/
processEdits(edits: ICellEdit[]): void;
}
export const enum CellEditType {
Output,
OutputData
}
export interface ICellEdit {
readonly type: CellEditType
}
export interface ICaretPosition {

View File

@@ -9,7 +9,7 @@ import { localize } from 'vs/nls';
import { Event, Emitter } from 'vs/base/common/event';
import { Disposable } from 'vs/base/common/lifecycle';
import { IClientSession, INotebookModel, INotebookModelOptions, ICellModel, NotebookContentChange, MoveDirection, ViewMode } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { IClientSession, INotebookModel, INotebookModelOptions, ICellModel, NotebookContentChange, MoveDirection, ViewMode, ICellEdit } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { NotebookChangeType, CellType, CellTypes } from 'sql/workbench/services/notebook/common/contracts';
import { KernelsLanguage, nbversion } from 'sql/workbench/services/notebook/common/notebookConstants';
import * as notebookUtils from 'sql/workbench/services/notebook/browser/models/notebookUtils';
@@ -19,7 +19,7 @@ import { NotebookContexts } from 'sql/workbench/services/notebook/browser/models
import { IConnectionProfile } from 'sql/platform/connection/common/interfaces';
import { INotification, Severity, INotificationService } from 'vs/platform/notification/common/notification';
import { URI } from 'vs/base/common/uri';
import { ISingleNotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
import { INotebookEditOperation, NotebookEditOperationType } from 'sql/workbench/api/common/sqlExtHostTypes';
import { ConnectionProfile } from 'sql/platform/connection/common/connectionProfile';
import { uriPrefixes } from 'sql/platform/connection/common/utils';
import { ILogService } from 'vs/platform/log/common/log';
@@ -35,7 +35,7 @@ import { isUUID } from 'vs/base/common/uuid';
import { TextModel } from 'vs/editor/common/model/textModel';
import { QueryTextEditor } from 'sql/workbench/browser/modelComponents/queryTextEditor';
import { CodeEditorWidget } from 'vs/editor/browser/widget/codeEditorWidget';
import { AddCellEdit, ConvertCellTypeEdit, DeleteCellEdit, MoveCellEdit, SplitCellEdit } from 'sql/workbench/services/notebook/browser/models/cellEdit';
import { AddCellEdit, CellOutputEdit, ConvertCellTypeEdit, DeleteCellEdit, MoveCellEdit, CellOutputDataEdit, SplitCellEdit } from 'sql/workbench/services/notebook/browser/models/cellEdit';
import { IUndoRedoService } from 'vs/platform/undoRedo/common/undoRedo';
import { deepClone } from 'vs/base/common/objects';
@@ -870,27 +870,59 @@ export class NotebookModel extends Disposable implements INotebookModel {
}
}
pushEditOperations(edits: ISingleNotebookEditOperation[]): void {
pushEditOperations(edits: INotebookEditOperation[]): void {
if (this.inErrorState || !this._cells) {
return;
}
for (let edit of edits) {
let newCells: ICellModel[] = [];
if (edit.cell) {
// TODO: should we validate and complete required missing parameters?
let contents: nb.ICellContents = edit.cell as nb.ICellContents;
newCells.push(this._notebookOptions.factory.createCell(contents, { notebook: this, isTrusted: this._trustedMode }));
this.undoService.pushElement(new AddCellEdit(this, newCells[0], edit.range.start));
for (const edit of edits) {
const startCell = this.cells[edit.range.start];
switch (edit.type) {
case NotebookEditOperationType.UpdateCell:
if (!startCell) {
this.logService.warn(`Did not receive a valid starting cell when processing edit type ${edit.type}`);
continue;
}
startCell.processEdits([
new CellOutputEdit(edit.cell.outputs ?? [], !!edit.append)
]);
break;
case NotebookEditOperationType.UpdateCellOutput:
if (!startCell) {
this.logService.warn(`Did not receive a valid starting cell when processing edit type ${edit.type}`);
continue;
}
const cellEdits: ICellEdit[] = [];
edit.cell.outputs?.forEach(o => {
const targetOutput = startCell.outputs.find(o2 => o.id === o2.id);
if (!targetOutput) {
this.logService.warn(`Could not find target output with ID ${o.id} when updating cell output`);
return;
}
cellEdits.push(new CellOutputDataEdit(targetOutput.id, (o as nb.IDisplayData).data, !!edit.append));
});
startCell.processEdits(cellEdits);
break;
case NotebookEditOperationType.InsertCell:
case NotebookEditOperationType.ReplaceCells:
case NotebookEditOperationType.DeleteCell:
let newCells: ICellModel[] = [];
if (edit.cell) {
// TODO: should we validate and complete required missing parameters?
let contents: nb.ICellContents = edit.cell as nb.ICellContents;
newCells.push(this._notebookOptions.factory.createCell(contents, { notebook: this, isTrusted: this._trustedMode }));
this.undoService.pushElement(new AddCellEdit(this, newCells[0], edit.range.start));
}
this._cells.splice(edit.range.start, edit.range.end - edit.range.start, ...newCells);
if (newCells.length > 0) {
this.updateActiveCell(newCells[0]);
}
this._contentChangedEmitter.fire({
changeType: NotebookChangeType.CellsModified,
isDirty: true
});
break;
}
this._cells.splice(edit.range.start, edit.range.end - edit.range.start, ...newCells);
if (newCells.length > 0) {
this.updateActiveCell(newCells[0]);
}
this._contentChangedEmitter.fire({
changeType: NotebookChangeType.CellsModified,
isDirty: true
});
}
}

View File

@@ -11,7 +11,7 @@ import { URI, UriComponents } from 'vs/base/common/uri';
import { RenderMimeRegistry } from 'sql/workbench/services/notebook/browser/outputs/registry';
import { ModelFactory } from 'sql/workbench/services/notebook/browser/models/modelFactory';
import { IConnectionProfile } from 'sql/platform/connection/common/interfaces';
import { ISingleNotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
import { INotebookEditOperation } from 'sql/workbench/api/common/sqlExtHostTypes';
import { ICellModel, INotebookModel } from 'sql/workbench/services/notebook/browser/models/modelInterfaces';
import { NotebookChangeType, CellType } from 'sql/workbench/services/notebook/common/contracts';
import { IBootstrapParams } from 'sql/workbench/services/bootstrap/common/bootstrapParams';
@@ -221,7 +221,7 @@ export interface INotebookEditor {
isDirty(): boolean;
isActive(): boolean;
isVisible(): boolean;
executeEdits(edits: ISingleNotebookEditOperation[]): boolean;
executeEdits(edits: INotebookEditOperation[]): boolean;
runCell(cell: ICellModel): Promise<boolean>;
runAllCells(startCell?: ICellModel, endCell?: ICellModel): Promise<boolean>;
clearOutput(cell: ICellModel): Promise<boolean>;

View File

@@ -33,7 +33,7 @@ suite('ExtHostNotebook Tests', () => {
let mainContext = <IMainContext>{
getProxy: proxyType => mockProxy.object
};
extHostNotebook = new ExtHostNotebook(mainContext);
extHostNotebook = new ExtHostNotebook(mainContext, undefined);
notebookUri = URI.parse('file:/user/default/my.ipynb');
serializationProviderMock = TypeMoq.Mock.ofType(SerializationProviderStub, TypeMoq.MockBehavior.Loose);
serializationProviderMock.callBase = true;

View File

@@ -222,16 +222,18 @@ suite('Notebook Serializer', () => {
metadata: {},
id: '1'
};
let expectedADSOutput: azdata.nb.IExecuteResult = {
id: '1',
output_type: 'execute_result',
data: {
'text/plain': '2',
'text/html': '<i>2</i>'
},
metadata: {},
execution_count: 1
};
let expectedADSOutput: azdata.nb.IExecuteResult[] = [
{
id: '1',
output_type: 'execute_result',
data: {
'text/plain': '2',
'text/html': '<i>2</i>'
},
metadata: {},
execution_count: 1
}
];
let actualOutput = VSCodeContentManager.convertToADSCellOutput(cellOutput, 1);
assert.deepStrictEqual(actualOutput, expectedADSOutput);

View File

@@ -317,7 +317,7 @@ export class ExtHostNotebookKernels implements ExtHostNotebookKernelsShape {
}
enum NotebookCellExecutionTaskState {
export enum NotebookCellExecutionTaskState { // {{SQL CARBON EDIT}} Use for our Notebook executions
Init,
Started,
Resolved