mirror of
https://github.com/ckaczor/azuredatastudio.git
synced 2026-01-26 17:23:15 -05:00
Merge VS Code 1.26.1 (#2394)
* Squash merge commits for 1.26 (#1) (#2323) * Polish tag search as per feedback (#55269) * Polish tag search as per feedback * Updated regex * Allow users to opt-out of features that send online requests in the background (#55097) * settings sweep #54690 * Minor css tweaks to enable eoverflow elipsis in more places (#55277) * fix an issue with titlebarheight when not scaling with zoom * Settings descriptions update #54690 * fixes #55209 * Settings editor - many padding fixes * More space above level 2 label * Fixing Cannot debug npm script using Yarn #55103 * Settings editor - show ellipsis when description overflows * Settings editor - ... fix measuring around links, relayout * Setting descriptions * Settings editor - fix ... for some short lines, fix select container width * Settings editor - overlay trees so scrollable shadow is full width * Fix #54133 - missing extension settings after reload * Settings color token description tweak * Settings editor - disable overflow indicator temporarily, needs to be faster * Added command to Run the selected npm script * fixes #54452 * fixes #54929 * fixes #55248 * prefix command with extension name * Contribute run selected to the context menu * node-debug@1.26.6 * Allow terminal rendererType to be swapped out at runtime Part of #53274 Fixes #55344 * Settings editor - fix not focusing search when restoring editor setInput must be actually async. Will be fixed naturally when we aren't using winJS promises... * Settings editor - TOC should only expand the section with a selected item * Bump node-debug2 * Settings editor - Tree focus outlines * Settings editor - don't blink the scrollbar when toc selection changes And hide TOC correctly when the editor is narrow * Settings editor - header rows should not be selectable * fixes #54877 * change debug assignee to isi * Settings sweep (#54690) * workaround for #55051 * Settings sweep (#54690) * settings sweep #54690 * Don't try closing tags when you type > after another > * Describe what implementation code lens does Fixes #55370 * fix javadoc formatter setting description * fixes #55325 * update to officical TS version * Settings editor - Even more padding, use semibold instead of bold * Fix #55357 - fix TOC twistie * fixes #55288 * explorer: refresh on di change file system provider registration fixes #53256 * Disable push to Linux repo to test standalone publisher * New env var to notify log level to extensions #54001 * Disable snippets in extension search (when not in suggest dropdown) (#55281) * Disable snippits in extension search (when not in suggest dropdown) * Add monaco input contributions * Fix bug preventing snippetSuggestions from taking effect in sub-editors * Latest emmet helper to fix #52366 * Fix comment updates for threads within same file * Allow extensions to log telemetry to log files #54001 * Pull latest css grammar * files.exclude control - use same style for "add" vs "edit" * files.exclude control - focus/keyboard behavior * don't show menubar too early * files.exclude - better styling * Place cursor at end of extensions search box on autofill (#55254) * Place cursor at end of extensions search box on autofill * Use position instead of selection * fix linux build issue (empty if block) * Settings editor - fix extension category prefixes * Settings editor - add simple ellipsis for first line that overflows, doesn't cover case when first line does not overflow but there is more text, TODO * File/Text search provider docs * Fixes #52655 * Include epoch (#55008) * Fixes #53385 * Fixes #49480 * VS Code Insiders (Users) not opening Fixes #55353 * Better handling of the case when the extension host fails to start * Fixes #53966 * Remove confusing Start from wordPartLeft commands ID * vscode-xterm@3.6.0-beta12 Fixes #55488 * Initial size is set to infinity!! Fixes #55461 * Polish embeddedEditorBackground * configuration service misses event * Fix #55224 - fix duplicate results in multiroot workspace from splitting the diskseach query * Select all not working in issue reporter on mac, fixes #55424 * Disable fuzzy matching for extensions autosuggest (#55498) * Fix clipping of extensions search border in some third party themes (#55504) * fixes #55538 * Fix bug causing an aria alert to not be shown the third time (and odd numbers thereafter) * Settings editor - work around rendering glitch with webkit-line-clamp * Settings editor - revert earlier '...' changes * Settings editor - move enumDescription to its own div, because it disturbs -webkit-line-clamp for some reason * Settings editor - better overflow indicator * Don't show existing filters in autocomplete (#55495) * Dont show existing filters in autocomplete * Simplify * Settings Editor: Add aria labels for input elements Fixes: #54836 (#55543) * fixes #55223 * Update vscode-css-languageservice to 3.0.10-next.1 * Fix #55509 - settings navigation * Fix #55519 * Fix #55520 * FIx #55524 * Fix #55556 - include wordSeparators in all search queries, so findTextInFiles can respect isWordMatch correctly * oss updates for endgame * Fix unit tests * fixes #55522 * Avoid missing manifest error from bubbling up #54757 * Settings format crawl * Search provider - Fix FileSearchProvider to return array, not progress * Fix #55598 * Settings editor - fix NPE rendering settings with no description * dont render inden guides in search box (#55600) * fixes #55454 * More settings crawl * Another change for #55598 - maxResults applies to FileSearch and TextSearch but not FileIndex * Fix FileSearchProvider unit tests for progress change * fixes #55561 * Settings description update for #54690 * Update setting descriptions for online services * Minor edits * fixes #55513 * fixes #55451 * Fix #55612 - fix findTextInFiles cancellation * fixes #55539 * More setting description tweaks * Setting to disable online experiments #54354 * fixes #55507 * fixes #55515 * Show online services action only in Insiders for now * Settings editor - change toc behavior default to 'filter' * Settings editor - nicer filter count style during search * Fix #55617 - search viewlet icons * Settings editor - better styling for element count indicator * SearchProvider - fix NPE when searching extraFileResources * Allow extends to work without json suffix Fixes #16905 * Remove accessability options logic entirely Follow up on #55451 * use latest version of DAP * fixes #55490 * fixes #55122 * fixes #52332 * Avoid assumptions about git: URIs (fixes #36236) * relative path for descriptions * resourece: get rid of isFile context key fixes #48275 * Register previous ids for compatibility (#53497) * more tuning for #48275 * no need to always re-read "files explorer" fixes #52003 * read out active composites properly fixes #51967 * Update link colors for hc theme to meet color contrast ratio, fixes #55651 Also updated link color for `textLinkActiveForeground` to be the same as `textLinkForeground` as it wasn't properly updated * detect 'winpty-agent.exe'; fixes #55672 * node-debug@1.26.7 * reset counter on new label * Settings editor - fix multiple setting links in one description * Settings editor - color code blocks in setting descriptions, fix #55532 * Settings editor - hover color in TOC * Settings editor - fix navigation NPE * Settings editor - fix text control width * Settings editor - maybe fix #55684 * Fix bug causing cursor to not move on paste * fixes #53582 * Use ctrlCmd instead of ctrl for go down from search box * fixes #55264 * fixes #55456 * filter for spcaes before triggering search (#55611) * Fix #55698 - don't lose filtered TOC counts when refreshing TOC * fixes #55421 * fixes #28979 * fixes #55576 * only add check for updates to windows/linux help * readonly files: append decoration to label fixes #53022 * debug: do not show toolbar while initialising fixes #55026 * Opening launch.json should not activate debug extensions fixes #55029 * fixes #55435 * fixes #55434 * fixes #55439 * trigger menu only on altkey up * Fix #50555 - fix settings editor memory leak * Fix #55712 - no need to focus 'a' anymore when restoring control focus after tree render * fixes #55335 * proper fix for readonly model fixes #53022 * improve FoldingRangeKind spec (for #55686) * Use class with static fields (fixes #55494) * Fixes #53671 * fixes #54630 * [html] should disable ionic suggestions by default. Currently forces deprecated Ionic v1 suggestions in .html files while typing. Fixes #53324 * cleanup deps * debug issues back to andre * update electron for smoketest * Fix #55757 - prevent settings tabs from overflowing * Fix #53897 - revert setting menu defaults to old editor * Add enum descriptions to `typescript.preferences.importModuleSpecifier` * Fix #55767 - leaking style elements from settings editor * Fix #55521 - prevent flashing when clicking in exclude control * Update Git modified color for contrast ratio, fixes #53140 * Revert "Merge branch 'master' of github.com:Microsoft/vscode" This reverts commit bf46b6bfbae0cab99c2863e1244a916181fa9fbc, reversing changes made to e275a424483dfb4ed33b428c97d5e2c441d6b917. * Revert "Revert "Merge branch 'master' of github.com:Microsoft/vscode"" This reverts commit 53949d963f39e40757557c6526332354a31d9154. * don't ask to install an incomplete menu * Fix NPE in terminal AccessibilityManager Fixes #55744 * don't display fallback menu unless we've closed the last window * fixes #55547 * Fix smoke tests for extension search box * Update OSSREADME.json for Electron 2.0.5 * Update distro Includes Chromium license changes * fix #55455 * fix #55865 * fixes #55893 * Fix bug causing workspace recommendations to go away upon ignoring a recommendation (#55805) * Fix bug causing workspace recommendations to go away upon ignoring a recommendation * ONly show on @recommended or @recommended:workspace * Make more consistant * Fix #55911 * Understand json activity (#55926) * Understand json file activity * Refactoring * adding composer.json * Distro update for experiments * use terminal.processId for auto-attach; fixes #55918 * Reject invalid URI with vscode.openFolder (for #55891) * improve win32 setup system vs user detection fixes #55840 fixes #55840 delay winreg import related to #55840 show notification earlier related to #55840 fix #55840 update inno setup message related to #55840 * Fix #55593 - this code only operates on local paths, so use fsPath and Uri.file instead * Bring back the old menu due to electron 2.0 issues (#55913) * add the old menu back for native menus * make menu labels match * `vscode.openFolder`: treat missing URI schema gracefully (for #55891) * delay EH reattach; fixes #55955 * Mark all json files under appSettingsHome as settings * Use localized strings for telemetry opt-out * Exception when saving file editor opened from remote file provider (fixes #55051) * Remove terminal menu from stable Fixes 56003 * VSCode Insiders crashes on open with TypeError: Cannot read property 'lastIndexOf' of undefined. Fixes #54933 * improve fix for #55891 * fix #55916 * Improve #55891 * increase EH debugging restart delay; fixes #55955 * Revert "Don't include non-resource entries in history quick pick" This reverts commit 37209a838e9f7e9abe6dc53ed73cdf1e03b72060. * Diff editor: horizontal scrollbar height is smaller (fixes #56062) * improve openFolder uri fix (correctly treat backslashes) * fixes #56116 repair ipc for native menubar keybindings * Fix #56240 - Open the JSON settings editor instead of the UI editor * Fix #55536 * uriDisplay: if no formatter is registered fall back to getPathlabel fixes #56104 * VSCode hangs when opening python file. Fixes #56377 * VS Code Hangs When Opening Specific PowerShell File. Fixes #56430 * Fix #56433 - search extraFileResources even when no folders open * Workaround #55649 * Fix in master #56371 * Fix tests #56371 * Fix in master #56317 * increase version to 1.26.1 * Fixes #56387: Handle SIGPIPE in extension host * fixes #56185 * Fix merge issues (part 1) * Fix build breaks (part 1) * Build breaks (part 2) * Build breaks (part 3) * More build breaks (part 4) * Fix build breaks (part 5) * WIP * Fix menus * Render query result and message panels (#2363) * Put back query editor hot exit changes * Fix grid changes that broke profiler (#2365) * Update APIs for saving query editor state * Fix restore view state for profiler and edit data * Updating custom default themes to support 4.5:1 contrast ratio * Test updates * Fix Extension Manager and Windows Setup * Update license headers * Add appveyor and travis files back * Fix hidden modal dropdown issue
This commit is contained in:
Karl Burtram
659
src/vs/workbench/api/node/extHostSearch.fileIndex.ts
Normal file
659
src/vs/workbench/api/node/extHostSearch.fileIndex.ts
Normal file
@@ -0,0 +1,659 @@
|
||||
/*---------------------------------------------------------------------------------------------
|
||||
* Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
* Licensed under the Source EULA. See License.txt in the project root for license information.
|
||||
*--------------------------------------------------------------------------------------------*/
|
||||
'use strict';
|
||||
|
||||
import * as path from 'path';
|
||||
import * as arrays from 'vs/base/common/arrays';
|
||||
import { CancellationTokenSource } from 'vs/base/common/cancellation';
|
||||
import { toErrorMessage } from 'vs/base/common/errorMessage';
|
||||
import * as glob from 'vs/base/common/glob';
|
||||
import * as resources from 'vs/base/common/resources';
|
||||
import * as strings from 'vs/base/common/strings';
|
||||
import URI from 'vs/base/common/uri';
|
||||
import { TPromise } from 'vs/base/common/winjs.base';
|
||||
import { compareItemsByScore, IItemAccessor, prepareQuery, ScorerCache } from 'vs/base/parts/quickopen/common/quickOpenScorer';
|
||||
import { IFileMatch, IFolderQuery, IRawSearchQuery, ISearchCompleteStats, ISearchQuery } from 'vs/platform/search/common/search';
|
||||
import * as vscode from 'vscode';
|
||||
|
||||
export interface IInternalFileMatch {
|
||||
base: URI;
|
||||
original?: URI;
|
||||
relativePath?: string; // Not present for extraFiles or absolute path matches
|
||||
basename: string;
|
||||
size?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Computes the patterns that the provider handles. Discards sibling clauses and 'false' patterns
|
||||
*/
|
||||
export function resolvePatternsForProvider(globalPattern: glob.IExpression, folderPattern: glob.IExpression): string[] {
|
||||
const merged = {
|
||||
...(globalPattern || {}),
|
||||
...(folderPattern || {})
|
||||
};
|
||||
|
||||
return Object.keys(merged)
|
||||
.filter(key => {
|
||||
const value = merged[key];
|
||||
return typeof value === 'boolean' && value;
|
||||
});
|
||||
}
|
||||
|
||||
export class QueryGlobTester {
|
||||
|
||||
private _excludeExpression: glob.IExpression;
|
||||
private _parsedExcludeExpression: glob.ParsedExpression;
|
||||
|
||||
private _parsedIncludeExpression: glob.ParsedExpression;
|
||||
|
||||
constructor(config: ISearchQuery, folderQuery: IFolderQuery) {
|
||||
this._excludeExpression = {
|
||||
...(config.excludePattern || {}),
|
||||
...(folderQuery.excludePattern || {})
|
||||
};
|
||||
this._parsedExcludeExpression = glob.parse(this._excludeExpression);
|
||||
|
||||
// Empty includeExpression means include nothing, so no {} shortcuts
|
||||
let includeExpression: glob.IExpression = config.includePattern;
|
||||
if (folderQuery.includePattern) {
|
||||
if (includeExpression) {
|
||||
includeExpression = {
|
||||
...includeExpression,
|
||||
...folderQuery.includePattern
|
||||
};
|
||||
} else {
|
||||
includeExpression = folderQuery.includePattern;
|
||||
}
|
||||
}
|
||||
|
||||
if (includeExpression) {
|
||||
this._parsedIncludeExpression = glob.parse(includeExpression);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Guaranteed sync - siblingsFn should not return a promise.
|
||||
*/
|
||||
public includedInQuerySync(testPath: string, basename?: string, hasSibling?: (name: string) => boolean): boolean {
|
||||
if (this._parsedExcludeExpression && this._parsedExcludeExpression(testPath, basename, hasSibling)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (this._parsedIncludeExpression && !this._parsedIncludeExpression(testPath, basename, hasSibling)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Guaranteed async.
|
||||
*/
|
||||
public includedInQuery(testPath: string, basename?: string, hasSibling?: (name: string) => boolean | TPromise<boolean>): TPromise<boolean> {
|
||||
const excludeP = this._parsedExcludeExpression ?
|
||||
TPromise.as(this._parsedExcludeExpression(testPath, basename, hasSibling)).then(result => !!result) :
|
||||
TPromise.wrap(false);
|
||||
|
||||
return excludeP.then(excluded => {
|
||||
if (excluded) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return this._parsedIncludeExpression ?
|
||||
TPromise.as(this._parsedIncludeExpression(testPath, basename, hasSibling)).then(result => !!result) :
|
||||
TPromise.wrap(true);
|
||||
}).then(included => {
|
||||
return included;
|
||||
});
|
||||
}
|
||||
|
||||
public hasSiblingExcludeClauses(): boolean {
|
||||
return hasSiblingClauses(this._excludeExpression);
|
||||
}
|
||||
}
|
||||
|
||||
function hasSiblingClauses(pattern: glob.IExpression): boolean {
|
||||
for (let key in pattern) {
|
||||
if (typeof pattern[key] !== 'boolean') {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
export interface IDirectoryEntry {
|
||||
base: URI;
|
||||
relativePath: string;
|
||||
basename: string;
|
||||
}
|
||||
|
||||
export interface IDirectoryTree {
|
||||
rootEntries: IDirectoryEntry[];
|
||||
pathToEntries: { [relativePath: string]: IDirectoryEntry[] };
|
||||
}
|
||||
|
||||
// ???
|
||||
interface IInternalSearchComplete {
|
||||
limitHit: boolean;
|
||||
results: IInternalFileMatch[];
|
||||
}
|
||||
|
||||
export class FileIndexSearchEngine {
|
||||
private filePattern: string;
|
||||
private normalizedFilePatternLowercase: string;
|
||||
private includePattern: glob.ParsedExpression;
|
||||
private maxResults: number;
|
||||
private exists: boolean;
|
||||
private isLimitHit: boolean;
|
||||
private resultCount: number;
|
||||
private isCanceled: boolean;
|
||||
|
||||
private activeCancellationTokens: Set<CancellationTokenSource>;
|
||||
|
||||
private globalExcludePattern: glob.ParsedExpression;
|
||||
|
||||
constructor(private config: ISearchQuery, private provider: vscode.FileIndexProvider) {
|
||||
this.filePattern = config.filePattern;
|
||||
this.includePattern = config.includePattern && glob.parse(config.includePattern);
|
||||
this.maxResults = config.maxResults || null;
|
||||
this.exists = config.exists;
|
||||
this.resultCount = 0;
|
||||
this.isLimitHit = false;
|
||||
this.activeCancellationTokens = new Set<CancellationTokenSource>();
|
||||
|
||||
if (this.filePattern) {
|
||||
this.normalizedFilePatternLowercase = strings.stripWildcards(this.filePattern).toLowerCase();
|
||||
}
|
||||
|
||||
this.globalExcludePattern = config.excludePattern && glob.parse(config.excludePattern);
|
||||
}
|
||||
|
||||
public cancel(): void {
|
||||
this.isCanceled = true;
|
||||
this.activeCancellationTokens.forEach(t => t.cancel());
|
||||
this.activeCancellationTokens = new Set();
|
||||
}
|
||||
|
||||
public search(_onResult: (match: IInternalFileMatch) => void): TPromise<{ isLimitHit: boolean }> {
|
||||
if (this.config.folderQueries.length !== 1) {
|
||||
throw new Error('Searches just one folder');
|
||||
}
|
||||
|
||||
const folderQuery = this.config.folderQueries[0];
|
||||
|
||||
return new TPromise<{ isLimitHit: boolean }>((resolve, reject) => {
|
||||
const onResult = (match: IInternalFileMatch) => {
|
||||
this.resultCount++;
|
||||
_onResult(match);
|
||||
};
|
||||
|
||||
if (this.isCanceled) {
|
||||
return resolve({ isLimitHit: this.isLimitHit });
|
||||
}
|
||||
|
||||
// For each extra file
|
||||
if (this.config.extraFileResources) {
|
||||
this.config.extraFileResources
|
||||
.forEach(extraFile => {
|
||||
const extraFileStr = extraFile.toString(); // ?
|
||||
const basename = path.basename(extraFileStr);
|
||||
if (this.globalExcludePattern && this.globalExcludePattern(extraFileStr, basename)) {
|
||||
return; // excluded
|
||||
}
|
||||
|
||||
// File: Check for match on file pattern and include pattern
|
||||
this.matchFile(onResult, { base: extraFile, basename });
|
||||
});
|
||||
}
|
||||
|
||||
return this.searchInFolder(folderQuery, _onResult)
|
||||
.then(() => {
|
||||
resolve({ isLimitHit: this.isLimitHit });
|
||||
}, (errs: Error[]) => {
|
||||
const errMsg = errs
|
||||
.map(err => toErrorMessage(err))
|
||||
.filter(msg => !!msg)[0];
|
||||
|
||||
reject(new Error(errMsg));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
private searchInFolder(fq: IFolderQuery<URI>, onResult: (match: IInternalFileMatch) => void): TPromise<void> {
|
||||
let cancellation = new CancellationTokenSource();
|
||||
return new TPromise((resolve, reject) => {
|
||||
const options = this.getSearchOptionsForFolder(fq);
|
||||
const tree = this.initDirectoryTree();
|
||||
|
||||
const queryTester = new QueryGlobTester(this.config, fq);
|
||||
const noSiblingsClauses = !queryTester.hasSiblingExcludeClauses();
|
||||
|
||||
const onProviderResult = (uri: URI) => {
|
||||
if (this.isCanceled) {
|
||||
return;
|
||||
}
|
||||
|
||||
// TODO@rob - ???
|
||||
const relativePath = path.relative(fq.folder.path, uri.path);
|
||||
if (noSiblingsClauses) {
|
||||
const basename = path.basename(uri.path);
|
||||
this.matchFile(onResult, { base: fq.folder, relativePath, basename, original: uri });
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
// TODO: Optimize siblings clauses with ripgrep here.
|
||||
this.addDirectoryEntries(tree, fq.folder, relativePath, onResult);
|
||||
};
|
||||
|
||||
new TPromise(resolve => process.nextTick(resolve))
|
||||
.then(() => {
|
||||
this.activeCancellationTokens.add(cancellation);
|
||||
return this.provider.provideFileIndex(options, cancellation.token);
|
||||
})
|
||||
.then(results => {
|
||||
this.activeCancellationTokens.delete(cancellation);
|
||||
if (this.isCanceled) {
|
||||
return null;
|
||||
}
|
||||
|
||||
results.forEach(onProviderResult);
|
||||
|
||||
this.matchDirectoryTree(tree, queryTester, onResult);
|
||||
return null;
|
||||
}).then(
|
||||
() => {
|
||||
cancellation.dispose();
|
||||
resolve(undefined);
|
||||
},
|
||||
err => {
|
||||
cancellation.dispose();
|
||||
reject(err);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
private getSearchOptionsForFolder(fq: IFolderQuery<URI>): vscode.FileIndexOptions {
|
||||
const includes = resolvePatternsForProvider(this.config.includePattern, fq.includePattern);
|
||||
const excludes = resolvePatternsForProvider(this.config.excludePattern, fq.excludePattern);
|
||||
|
||||
return {
|
||||
folder: fq.folder,
|
||||
excludes,
|
||||
includes,
|
||||
useIgnoreFiles: !this.config.disregardIgnoreFiles,
|
||||
followSymlinks: !this.config.ignoreSymlinks
|
||||
};
|
||||
}
|
||||
|
||||
private initDirectoryTree(): IDirectoryTree {
|
||||
const tree: IDirectoryTree = {
|
||||
rootEntries: [],
|
||||
pathToEntries: Object.create(null)
|
||||
};
|
||||
tree.pathToEntries['.'] = tree.rootEntries;
|
||||
return tree;
|
||||
}
|
||||
|
||||
private addDirectoryEntries({ pathToEntries }: IDirectoryTree, base: URI, relativeFile: string, onResult: (result: IInternalFileMatch) => void) {
|
||||
// Support relative paths to files from a root resource (ignores excludes)
|
||||
if (relativeFile === this.filePattern) {
|
||||
const basename = path.basename(this.filePattern);
|
||||
this.matchFile(onResult, { base: base, relativePath: this.filePattern, basename });
|
||||
}
|
||||
|
||||
function add(relativePath: string) {
|
||||
const basename = path.basename(relativePath);
|
||||
const dirname = path.dirname(relativePath);
|
||||
let entries = pathToEntries[dirname];
|
||||
if (!entries) {
|
||||
entries = pathToEntries[dirname] = [];
|
||||
add(dirname);
|
||||
}
|
||||
entries.push({
|
||||
base,
|
||||
relativePath,
|
||||
basename
|
||||
});
|
||||
}
|
||||
|
||||
add(relativeFile);
|
||||
}
|
||||
|
||||
private matchDirectoryTree({ rootEntries, pathToEntries }: IDirectoryTree, queryTester: QueryGlobTester, onResult: (result: IInternalFileMatch) => void) {
|
||||
const self = this;
|
||||
const filePattern = this.filePattern;
|
||||
function matchDirectory(entries: IDirectoryEntry[]) {
|
||||
// self.directoriesWalked++;
|
||||
for (let i = 0, n = entries.length; i < n; i++) {
|
||||
const entry = entries[i];
|
||||
const { relativePath, basename } = entry;
|
||||
|
||||
// Check exclude pattern
|
||||
// If the user searches for the exact file name, we adjust the glob matching
|
||||
// to ignore filtering by siblings because the user seems to know what she
|
||||
// is searching for and we want to include the result in that case anyway
|
||||
const hasSibling = glob.hasSiblingFn(() => entries.map(entry => entry.basename));
|
||||
if (!queryTester.includedInQuerySync(relativePath, basename, filePattern !== basename ? hasSibling : undefined)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const sub = pathToEntries[relativePath];
|
||||
if (sub) {
|
||||
matchDirectory(sub);
|
||||
} else {
|
||||
// self.filesWalked++;
|
||||
if (relativePath === filePattern) {
|
||||
continue; // ignore file if its path matches with the file pattern because that is already matched above
|
||||
}
|
||||
|
||||
self.matchFile(onResult, entry);
|
||||
}
|
||||
|
||||
if (self.isLimitHit) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
matchDirectory(rootEntries);
|
||||
}
|
||||
|
||||
private matchFile(onResult: (result: IInternalFileMatch) => void, candidate: IInternalFileMatch): void {
|
||||
if (this.isFilePatternMatch(candidate.relativePath) && (!this.includePattern || this.includePattern(candidate.relativePath, candidate.basename))) {
|
||||
if (this.exists || (this.maxResults && this.resultCount >= this.maxResults)) {
|
||||
this.isLimitHit = true;
|
||||
this.cancel();
|
||||
}
|
||||
|
||||
if (!this.isLimitHit) {
|
||||
onResult(candidate);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private isFilePatternMatch(path: string): boolean {
|
||||
// Check for search pattern
|
||||
if (this.filePattern) {
|
||||
if (this.filePattern === '*') {
|
||||
return true; // support the all-matching wildcard
|
||||
}
|
||||
|
||||
return strings.fuzzyContains(path, this.normalizedFilePatternLowercase);
|
||||
}
|
||||
|
||||
// No patterns means we match all
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
export class FileIndexSearchManager {
|
||||
|
||||
private static readonly BATCH_SIZE = 512;
|
||||
|
||||
private caches: { [cacheKey: string]: Cache; } = Object.create(null);
|
||||
|
||||
private readonly folderCacheKeys = new Map<string, Set<string>>();
|
||||
|
||||
public fileSearch(config: ISearchQuery, provider: vscode.FileIndexProvider, onBatch: (matches: IFileMatch[]) => void): TPromise<ISearchCompleteStats> {
|
||||
if (config.sortByScore) {
|
||||
let sortedSearch = this.trySortedSearchFromCache(config);
|
||||
if (!sortedSearch) {
|
||||
const engineConfig = config.maxResults ?
|
||||
{
|
||||
...config,
|
||||
...{ maxResults: null }
|
||||
} :
|
||||
config;
|
||||
|
||||
const engine = new FileIndexSearchEngine(engineConfig, provider);
|
||||
sortedSearch = this.doSortedSearch(engine, config);
|
||||
}
|
||||
|
||||
return new TPromise<ISearchCompleteStats>((c, e) => {
|
||||
sortedSearch.then(complete => {
|
||||
this.sendAsBatches(complete.results, onBatch, FileIndexSearchManager.BATCH_SIZE);
|
||||
c(complete);
|
||||
}, e, onBatch);
|
||||
}, () => {
|
||||
sortedSearch.cancel();
|
||||
});
|
||||
}
|
||||
|
||||
const engine = new FileIndexSearchEngine(config, provider);
|
||||
return this.doSearch(engine)
|
||||
.then(complete => {
|
||||
this.sendAsBatches(complete.results, onBatch, FileIndexSearchManager.BATCH_SIZE);
|
||||
return <ISearchCompleteStats>{
|
||||
limitHit: complete.limitHit
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
private getFolderCacheKey(config: ISearchQuery): string {
|
||||
const uri = config.folderQueries[0].folder.toString();
|
||||
const folderCacheKey = config.cacheKey && `${uri}_${config.cacheKey}`;
|
||||
if (!this.folderCacheKeys.get(config.cacheKey)) {
|
||||
this.folderCacheKeys.set(config.cacheKey, new Set());
|
||||
}
|
||||
|
||||
this.folderCacheKeys.get(config.cacheKey).add(folderCacheKey);
|
||||
|
||||
return folderCacheKey;
|
||||
}
|
||||
|
||||
private rawMatchToSearchItem(match: IInternalFileMatch): IFileMatch {
|
||||
return {
|
||||
resource: match.original || resources.joinPath(match.base, match.relativePath)
|
||||
};
|
||||
}
|
||||
|
||||
private doSortedSearch(engine: FileIndexSearchEngine, config: ISearchQuery): TPromise<IInternalSearchComplete> {
|
||||
let searchPromise: TPromise<void>;
|
||||
let allResultsPromise = new TPromise<IInternalSearchComplete>((c, e) => {
|
||||
searchPromise = this.doSearch(engine).then(c, e);
|
||||
}, () => {
|
||||
searchPromise.cancel();
|
||||
});
|
||||
|
||||
const folderCacheKey = this.getFolderCacheKey(config);
|
||||
let cache: Cache;
|
||||
if (folderCacheKey) {
|
||||
cache = this.getOrCreateCache(folderCacheKey);
|
||||
cache.resultsToSearchCache[config.filePattern] = allResultsPromise;
|
||||
allResultsPromise.then(null, err => {
|
||||
delete cache.resultsToSearchCache[config.filePattern];
|
||||
});
|
||||
allResultsPromise = this.preventCancellation(allResultsPromise);
|
||||
}
|
||||
|
||||
let chained: TPromise<void>;
|
||||
return new TPromise<IInternalSearchComplete>((c, e) => {
|
||||
chained = allResultsPromise.then(complete => {
|
||||
const scorerCache: ScorerCache = cache ? cache.scorerCache : Object.create(null);
|
||||
return this.sortResults(config, complete.results, scorerCache)
|
||||
.then(sortedResults => {
|
||||
|
||||
c({
|
||||
limitHit: complete.limitHit || typeof config.maxResults === 'number' && complete.results.length > config.maxResults, // ??
|
||||
results: sortedResults
|
||||
});
|
||||
});
|
||||
}, e);
|
||||
}, () => {
|
||||
chained.cancel();
|
||||
});
|
||||
}
|
||||
|
||||
private getOrCreateCache(cacheKey: string): Cache {
|
||||
const existing = this.caches[cacheKey];
|
||||
if (existing) {
|
||||
return existing;
|
||||
}
|
||||
return this.caches[cacheKey] = new Cache();
|
||||
}
|
||||
|
||||
private trySortedSearchFromCache(config: ISearchQuery): TPromise<IInternalSearchComplete> {
|
||||
const folderCacheKey = this.getFolderCacheKey(config);
|
||||
const cache = folderCacheKey && this.caches[folderCacheKey];
|
||||
if (!cache) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const cached = this.getResultsFromCache(cache, config.filePattern);
|
||||
if (cached) {
|
||||
let chained: TPromise<void>;
|
||||
return new TPromise<IInternalSearchComplete>((c, e) => {
|
||||
chained = cached.then(complete => {
|
||||
return this.sortResults(config, complete.results, cache.scorerCache)
|
||||
.then(sortedResults => {
|
||||
c({
|
||||
limitHit: complete.limitHit || typeof config.maxResults === 'number' && complete.results.length > config.maxResults,
|
||||
results: sortedResults
|
||||
});
|
||||
});
|
||||
}, e);
|
||||
}, () => {
|
||||
chained.cancel();
|
||||
});
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
private sortResults(config: IRawSearchQuery, results: IInternalFileMatch[], scorerCache: ScorerCache): TPromise<IInternalFileMatch[]> {
|
||||
// we use the same compare function that is used later when showing the results using fuzzy scoring
|
||||
// this is very important because we are also limiting the number of results by config.maxResults
|
||||
// and as such we want the top items to be included in this result set if the number of items
|
||||
// exceeds config.maxResults.
|
||||
const query = prepareQuery(config.filePattern);
|
||||
const compare = (matchA: IInternalFileMatch, matchB: IInternalFileMatch) => compareItemsByScore(matchA, matchB, query, true, FileMatchItemAccessor, scorerCache);
|
||||
|
||||
return arrays.topAsync(results, compare, config.maxResults, 10000);
|
||||
}
|
||||
|
||||
private sendAsBatches(rawMatches: IInternalFileMatch[], onBatch: (batch: IFileMatch[]) => void, batchSize: number) {
|
||||
const serializedMatches = rawMatches.map(rawMatch => this.rawMatchToSearchItem(rawMatch));
|
||||
if (batchSize && batchSize > 0) {
|
||||
for (let i = 0; i < serializedMatches.length; i += batchSize) {
|
||||
onBatch(serializedMatches.slice(i, i + batchSize));
|
||||
}
|
||||
} else {
|
||||
onBatch(serializedMatches);
|
||||
}
|
||||
}
|
||||
|
||||
private getResultsFromCache(cache: Cache, searchValue: string): TPromise<IInternalSearchComplete> {
|
||||
if (path.isAbsolute(searchValue)) {
|
||||
return null; // bypass cache if user looks up an absolute path where matching goes directly on disk
|
||||
}
|
||||
|
||||
// Find cache entries by prefix of search value
|
||||
const hasPathSep = searchValue.indexOf(path.sep) >= 0;
|
||||
let cached: TPromise<IInternalSearchComplete>;
|
||||
for (let previousSearch in cache.resultsToSearchCache) {
|
||||
|
||||
// If we narrow down, we might be able to reuse the cached results
|
||||
if (strings.startsWith(searchValue, previousSearch)) {
|
||||
if (hasPathSep && previousSearch.indexOf(path.sep) < 0) {
|
||||
continue; // since a path character widens the search for potential more matches, require it in previous search too
|
||||
}
|
||||
|
||||
const c = cache.resultsToSearchCache[previousSearch];
|
||||
cached = this.preventCancellation(c);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!cached) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return new TPromise<IInternalSearchComplete>((c, e) => {
|
||||
cached.then(complete => {
|
||||
// Pattern match on results
|
||||
let results: IInternalFileMatch[] = [];
|
||||
const normalizedSearchValueLowercase = strings.stripWildcards(searchValue).toLowerCase();
|
||||
for (let i = 0; i < complete.results.length; i++) {
|
||||
let entry = complete.results[i];
|
||||
|
||||
// Check if this entry is a match for the search value
|
||||
if (!strings.fuzzyContains(entry.relativePath, normalizedSearchValueLowercase)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
results.push(entry);
|
||||
}
|
||||
|
||||
c({
|
||||
limitHit: complete.limitHit,
|
||||
results
|
||||
});
|
||||
}, e);
|
||||
}, () => {
|
||||
cached.cancel();
|
||||
});
|
||||
}
|
||||
|
||||
private doSearch(engine: FileIndexSearchEngine): TPromise<IInternalSearchComplete> {
|
||||
const results: IInternalFileMatch[] = [];
|
||||
const onResult = match => results.push(match);
|
||||
return new TPromise<IInternalSearchComplete>((c, e) => {
|
||||
engine.search(onResult).then(result => {
|
||||
c({
|
||||
limitHit: result.isLimitHit,
|
||||
results
|
||||
});
|
||||
}, e);
|
||||
}, () => {
|
||||
engine.cancel();
|
||||
});
|
||||
}
|
||||
|
||||
public clearCache(cacheKey: string): TPromise<void> {
|
||||
if (!this.folderCacheKeys.has(cacheKey)) {
|
||||
return TPromise.wrap(undefined);
|
||||
}
|
||||
|
||||
const expandedKeys = this.folderCacheKeys.get(cacheKey);
|
||||
expandedKeys.forEach(key => delete this.caches[key]);
|
||||
|
||||
this.folderCacheKeys.delete(cacheKey);
|
||||
|
||||
return TPromise.as(undefined);
|
||||
}
|
||||
|
||||
private preventCancellation<C>(promise: TPromise<C>): TPromise<C> {
|
||||
return new TPromise<C>((c, e) => {
|
||||
// Allow for piled up cancellations to come through first.
|
||||
process.nextTick(() => {
|
||||
promise.then(c, e);
|
||||
});
|
||||
}, () => {
|
||||
// Do not propagate.
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
class Cache {
|
||||
|
||||
public resultsToSearchCache: { [searchValue: string]: TPromise<IInternalSearchComplete>; } = Object.create(null);
|
||||
|
||||
public scorerCache: ScorerCache = Object.create(null);
|
||||
}
|
||||
|
||||
const FileMatchItemAccessor = new class implements IItemAccessor<IInternalFileMatch> {
|
||||
|
||||
public getItemLabel(match: IInternalFileMatch): string {
|
||||
return match.basename; // e.g. myFile.txt
|
||||
}
|
||||
|
||||
public getItemDescription(match: IInternalFileMatch): string {
|
||||
return match.relativePath.substr(0, match.relativePath.length - match.basename.length - 1); // e.g. some/path/to/file
|
||||
}
|
||||
|
||||
public getItemPath(match: IInternalFileMatch): string {
|
||||
return match.relativePath; // e.g. some/path/to/file/myFile.txt
|
||||
}
|
||||
};
|
||||
Reference in New Issue
Block a user