Added new Kusto ServiceLayer (#1009)

* Copy smoModel some rename

* Copy entire service layer

* Building copy

* Fixing some references

* Launch profile

* Resolve namespace issues

* Compiling tests. Correct manifest.

* Fixing localization resources

* ReliableKustoClient

* Some trimming of extra code and Kusto code

* Kusto client creation in bindingContent

* Removing Smo and new Kusto classes

* More trimming

* Kusto schema hookup

* Solidying DataSource abstraction

* Solidifying further

* Latest refatoring

* More refactoring

* Building and launching Kusto service layer

* Working model which enumerates databases

* Refactoring to pass IDataSource to all tree nodes

* Removing some dependencies on the context

* Working with tables and schema

* Comment checkin

* Refactoring to give out select script

* Query created and sent back to ADS

* Fix query generation

* Fix listing of databases

* Tunneling the query through.

* Successful query execution

* Return only results table

* Deleting Cms

* Delete DacFx

* Delete SchemaCompare and TaskServices

* Change build definition to not stop at launch

* Fix error after merge

* Save Kusto results in different formats (#935)

* save results as csv etc

* some fixes

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* 2407 Added OrderBy clause in KustoDataSource > GetDatabaseMetaData and GetColumnMetadata (#959)

* 2405 Defaulted Options when setting ServerInfo in ConnectionService > GetConnectionCompleteParams (#965)

* 2747 Fixed IsUnknownType error for Kusto (#989)

* 2747 Removed unused directives in Kusto > DbColumnWrapper. Refactored IsUnknownType to handle null DataTypeName

* 2747 Reverted IsUnknownType change in DbColumnWrapper. Changed DataTypeName to get calue from ColumnType. Refactored SafeGetValue to type check before hard casting to reduce case exceptions.

* Added EmbeddedResourceUseDependentUponConvention to Microsoft.Kusto.ServiceLayer.csproj. Also renamed DACfx to match Microsoft.SqlTools.ServiceLayer. Added to compile Exclude="**/obj/**/*.cs"

* Srahman cleanup sql code (#992)

* Removed Management and Security Service Code.

* Remove FileBrowser service

* Comment why we are using SqlServer library

* Remove SQL specific type definitions

* clean up formatter service (#996)

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* Code clean up and Kusto intellisense (#994)

* Code clean up and Kusto intellisense

* Addressed few comments

* Addressed few comments

* addressed comments

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* Return multiple tables for Kusto

* Changes required for Kusto manage dashboard (#1039)

* Changes required for manage dashboard

* Addressed comments

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* 2728 Kusto function support (#1038)

* loc update (#914)

* loc update

* loc updates

* 2728 moved ColumnInfo and KustoResultsReader to separate files. Added Folder and Function to TreeNode.cs

* 2728 Added FunctionInfo. Added Folder to ColumnInfo. Removed partial class from KustoResultsReader. Set Function.IsAlwaysLeaf=true in TreeNode.cs. In KustoDataSource changed tableMetadata type to TableMetaData. Added folder and function dictionaries. Refactored GetSchema function. Renamed GenerateColumnMetadataKey to GenerateMetadataKey

* 2728 Added FunctionInfo. Added Folder to ColumnInfo. Removed partial class from KustoResultsReader. Set Function.IsAlwaysLeaf=true in TreeNode.cs. In KustoDataSource changed tableMetadata type to TableMetaData. Added folder and function dictionaries. Refactored GetSchema function. Renamed GenerateColumnMetadataKey to GenerateMetadataKey

* 2728 Created new SqlConnection within using block. Refactored KustoDataSource > columnmetadata to sort on get instead of insert.

* 2728 Added GetFunctionInfo function to KustoDataSource.

* 2728 Reverted change to Microsoft.Kusto.ServiceLayer.csproj from merge

* 2728 Reverted change to SqlTools.ServiceLayer\Localization\transXliff

* 2728 Reverted change to sr.de.xlf and sr.zh-hans.xlf

* 2728 Refactored KustoDataSource Function folders to support subfolders

* 2728 Refactored KustoDataSource to use urn for folders, functions, and tables instead of name.

* Merge remote-tracking branch 'origin/main' into feature-ADE

# Conflicts:
#	Packages.props

* 2728 Moved metadata files into Metadata subdirectory. Added GenerateAlterFunction to IDataSource and DataSourceBase.

* 2728 Added summary information to SafeAdd in SystemExtensions. Renamed local variable in SetTableMetadata

* 2728 Moved SafeAdd from SystemExtensions to KustoQueryUtils. Added check when getting database schema to return existing records before querying again. Added AddRange function to KustoQueryUtils. Created SetFolderMetadataForFunctions method.

* 2728 Added DatabaseKeyPrefix to only return tables to a database for the dashboard. Added logic to store all database tables within the tableMetadata dictionary for the dashboard.

* 2728 Created TableInfo and moved info objects into Models directory. Refactored KustoDataSource to lazy load columns for tables. Refactored logic to load tables using cslschema instead of schema.

* 2728 Renamed LoadColumnSchema to GetTableSchema to be consistent.

Co-authored-by: khoiph1 <khoiph@microsoft.com>

* Addressed comments

Co-authored-by: Shafiq Rahman <srahman@microsoft.com>
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
Co-authored-by: Justin M <63619224+JustinMDotNet@users.noreply.github.com>
Co-authored-by: rkselfhost <rkselfhost@outlook.com>
Co-authored-by: khoiph1 <khoiph@microsoft.com>

src/Microsoft.Kusto.ServiceLayer/DataSource/DataReaderWrapper.cs

@@ -0,0 +1,59 @@
+// <copyright file="KustoQueryUtils.cs" company="Microsoft">
+// Copyright (c) Microsoft. All Rights Reserved.
+// </copyright>
+using System;
+using System.Text.RegularExpressions;
+using System.Linq;
+using System.Data;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    class DataReaderWrapper:IDataReader
+    {
+        private readonly IDataReader _inner ;
+        public DataReaderWrapper(IDataReader inner)
+        {
+            _inner = inner;
+        }
+
+        public object this[int i] => _inner[i];
+
+        public object this[string name] => _inner[name];
+
+        public int Depth => _inner.Depth;
+
+        public bool IsClosed => _inner.IsClosed;
+
+        public int RecordsAffected => _inner.RecordsAffected;
+
+        public int FieldCount => _inner.FieldCount;
+
+        public void Close() => _inner.Close();
+        public void Dispose() => _inner.Dispose();
+        public bool GetBoolean(int i) => _inner.GetBoolean(i);
+        public byte GetByte(int i) => _inner.GetByte(i);
+        public long GetBytes(int i, long fieldOffset, byte[] buffer, int bufferoffset, int length) => _inner.GetBytes(i, fieldOffset, buffer, bufferoffset, length);
+        public char GetChar(int i) => _inner.GetChar(i);
+        public long GetChars(int i, long fieldoffset, char[] buffer, int bufferoffset, int length) => _inner.GetChars(i, fieldoffset, buffer, bufferoffset, length);
+        public IDataReader GetData(int i) => _inner.GetData(i);
+        public string GetDataTypeName(int i) => _inner.GetDataTypeName(i);
+        public DateTime GetDateTime(int i) => _inner.GetDateTime(i);
+        public decimal GetDecimal(int i) => _inner.GetDecimal(i);
+        public double GetDouble(int i) => _inner.GetDouble(i);
+        public Type GetFieldType(int i) => _inner.GetFieldType(i);
+        public float GetFloat(int i) => _inner.GetFloat(i);
+        public Guid GetGuid(int i) => _inner.GetGuid(i);
+        public short GetInt16(int i) => _inner.GetInt16(i);
+        public int GetInt32(int i) => _inner.GetInt32(i);
+        public long GetInt64(int i) => _inner.GetInt64(i);
+        public string GetName(int i) => _inner.GetName(i);
+        public int GetOrdinal(string name) => _inner.GetOrdinal(name);
+        public DataTable GetSchemaTable() => _inner.GetSchemaTable();
+        public string GetString(int i) => _inner.GetString(i);
+        public object GetValue(int i) => _inner.GetValue(i);
+        public int GetValues(object[] values) => _inner.GetValues(values);
+        public bool IsDBNull(int i) => _inner.IsDBNull(i);
+        public virtual bool NextResult() => _inner.NextResult();
+        public bool Read() => _inner.Read();
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/DataSourceBase.cs

@@ -0,0 +1,132 @@
+// <copyright file="DataSourceUtils.cs" company="Microsoft">
+// Copyright (c) Microsoft. All Rights Reserved.
+// </copyright>
+
+using System;
+using System.Collections.Generic;
+using System.Threading;
+using System.Data;
+using System.Threading.Tasks;
+using Microsoft.Kusto.ServiceLayer.Utility;
+using Microsoft.Kusto.ServiceLayer.DataSource.DataSourceIntellisense;
+using Microsoft.Kusto.ServiceLayer.DataSource.Metadata;
+using Microsoft.Kusto.ServiceLayer.LanguageServices;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Contracts;
+using Microsoft.Kusto.ServiceLayer.Workspace.Contracts;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Completion;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    /// <inheritdoc cref="IDataSource"/>
+    public abstract class DataSourceBase : IDataSource
+    {
+        protected Object dataSourceLock = new Object();
+
+        private string _database;
+
+        #region IDisposable
+
+        /// <summary>
+        /// Finalizes an instance of the <see cref="DataSourceBase"/> class.
+        /// </summary>
+        ~DataSourceBase()
+        {
+            Dispose(false);
+        }
+
+        /// <summary>
+        /// Disposes resources.
+        /// </summary>
+        public void Dispose()
+        {
+            Dispose(true);
+            GC.SuppressFinalize(this);
+        }
+
+        /// <summary>
+        /// Disposes resources.
+        /// </summary>
+        /// <param name="disposing">true if disposing; false if finalizing.</param>
+        protected virtual void Dispose(bool disposing)
+        {
+        }
+
+        #endregion
+
+        #region IDataSource
+
+        /// <inheritdoc/>
+        public abstract Task<IDataReader> ExecuteQueryAsync(string query, CancellationToken cancellationToken, string databaseName = null);
+
+        /// <inheritdoc/>
+        public async Task<T> ExecuteScalarQueryAsync<T>(string query, CancellationToken cancellationToken, string databaseName = null)
+        {
+            ValidationUtils.IsArgumentNotNullOrWhiteSpace(query, nameof(query));
+
+            using (var records = await ExecuteQueryAsync(query, cancellationToken, databaseName))
+            {
+                return records.ToScalar<T>();
+            }
+        }
+
+        public abstract Task<IEnumerable<T>> ExecuteControlCommandAsync<T>(string command, bool throwOnError, CancellationToken cancellationToken);
+
+        /// <inheritdoc/>
+        public abstract DiagnosticsInfo GetDiagnostics(DataSourceObjectMetadata parentMetadata);
+
+        /// <inheritdoc/>
+        public abstract IEnumerable<DataSourceObjectMetadata> GetChildObjects(DataSourceObjectMetadata parentMetadata, bool includeSizeDetails = false);
+
+        /// <inheritdoc/>
+        public abstract void Refresh();
+
+        /// <inheritdoc/>
+        public abstract void Refresh(DataSourceObjectMetadata objectMetadata);
+
+        /// <inheritdoc/>
+        public abstract void UpdateDatabase(string databaseName);
+
+        /// <inheritdoc/>
+        public abstract CompletionItem[] GetAutoCompleteSuggestions(ScriptDocumentInfo queryText, Position index, bool throwOnError = false);
+        /// <inheritdoc/>
+        public abstract Hover GetHoverHelp(ScriptDocumentInfo scriptDocumentInfo, Position textPosition, bool throwOnError = false);
+        
+        /// <inheritdoc/>
+        public abstract DefinitionResult GetDefinition(string queryText, int index, int startLine, int startColumn, bool throwOnError = false);
+
+        /// <inheritdoc/>
+        public abstract ScriptFileMarker[] GetSemanticMarkers(ScriptParseInfo parseInfo, ScriptFile scriptFile, string queryText);
+
+        /// <inheritdoc/>
+        public abstract Task<bool> Exists();
+
+        /// <inheritdoc/>
+        public abstract bool Exists(DataSourceObjectMetadata objectMetadata);
+
+        public abstract string GenerateAlterFunctionScript(string functionName);
+
+        /// <inheritdoc/>
+        public DataSourceType DataSourceType { get; protected set; }
+
+        /// <inheritdoc/>
+        public string ClusterName { get; protected set; }
+
+        /// <inheritdoc/>
+        public string DatabaseName { 
+            get
+            {
+                return _database;
+            }
+            
+            set
+            {
+                lock(dataSourceLock)
+                {
+                    _database = value;
+                }
+            }
+        }
+
+        #endregion
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/DataSourceFactory.cs

@@ -0,0 +1,159 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using Microsoft.Kusto.ServiceLayer.Utility;
+using Microsoft.Kusto.ServiceLayer.Admin.Contracts;
+using Microsoft.SqlTools.ServiceLayer.Connection.ReliableConnection;
+using Microsoft.Kusto.ServiceLayer.Metadata.Contracts;
+using Microsoft.Kusto.ServiceLayer.DataSource.DataSourceIntellisense;
+using Microsoft.Kusto.ServiceLayer.DataSource.Metadata;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Contracts;
+using Microsoft.Kusto.ServiceLayer.Workspace.Contracts;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Completion;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    /// <summary>
+    /// Data source factory.
+    /// </summary>
+    public static class DataSourceFactory
+    {
+        public static IDataSource Create(DataSourceType dataSourceType, string connectionString, string azureAccountToken)
+        {
+            ValidationUtils.IsArgumentNotNullOrWhiteSpace(connectionString, nameof(connectionString));
+            ValidationUtils.IsArgumentNotNullOrWhiteSpace(azureAccountToken, nameof(azureAccountToken));
+
+            switch (dataSourceType)
+            {
+                case DataSourceType.Kusto:
+                    {
+                        return new KustoDataSource(connectionString, azureAccountToken);
+                    }
+
+                default:
+                    throw new ArgumentException($"Unsupported data source type \"{dataSourceType}\"", nameof(dataSourceType));
+            }
+        }
+
+        public static DataSourceObjectMetadata CreateClusterMetadata(string clusterName)
+        {
+            ValidationUtils.IsArgumentNotNullOrWhiteSpace(clusterName, nameof(clusterName));
+
+            return new DataSourceObjectMetadata{
+                MetadataType = DataSourceMetadataType.Cluster,
+                MetadataTypeName = DataSourceMetadataType.Cluster.ToString(),
+                Name = clusterName,
+                PrettyName = clusterName,
+                Urn = $"{clusterName}"
+            };
+        }
+
+        public static DataSourceObjectMetadata CreateDatabaseMetadata(DataSourceObjectMetadata clusterMetadata, string databaseName)
+        {
+            ValidationUtils.IsTrue<ArgumentException>(clusterMetadata.MetadataType == DataSourceMetadataType.Cluster, nameof(clusterMetadata));
+            ValidationUtils.IsArgumentNotNullOrWhiteSpace(databaseName, nameof(databaseName));
+
+            return new DatabaseMetadata{
+                ClusterName = clusterMetadata.Name,
+                MetadataType = DataSourceMetadataType.Database,
+                MetadataTypeName = DataSourceMetadataType.Database.ToString(),
+                Name = databaseName,
+                PrettyName = databaseName,
+                Urn = $"{clusterMetadata.Urn}.{databaseName}"
+            };
+        }
+
+        public static FolderMetadata CreateFolderMetadata(DataSourceObjectMetadata parentMetadata, string path, string name)
+        {
+            ValidationUtils.IsNotNull(parentMetadata, nameof(parentMetadata));
+
+            return new FolderMetadata{
+                MetadataType = DataSourceMetadataType.Folder,
+                MetadataTypeName = DataSourceMetadataType.Folder.ToString(),
+                Name = name,
+                PrettyName = name,
+                ParentMetadata = parentMetadata,
+                Urn = $"{path}.{name}"
+            };
+        }
+
+        // Gets default keywords for intellisense when there is no connection.
+        public static CompletionItem[] GetDefaultAutoComplete(DataSourceType dataSourceType, ScriptDocumentInfo scriptDocumentInfo, Position textDocumentPosition){
+            switch (dataSourceType)
+            {
+                case DataSourceType.Kusto:
+                    {
+                        return KustoIntellisenseHelper.GetDefaultKeywords(scriptDocumentInfo, textDocumentPosition);
+                    }
+
+                default:
+                    throw new ArgumentException($"Unsupported data source type \"{dataSourceType}\"", nameof(dataSourceType));
+            }
+        }
+
+        // Gets default keywords errors related to intellisense when there is no connection.
+        public static ScriptFileMarker[] GetDefaultSemanticMarkers(DataSourceType dataSourceType, ScriptParseInfo parseInfo, ScriptFile scriptFile, string queryText){
+            switch (dataSourceType)
+            {
+                case DataSourceType.Kusto:
+                    {
+                        return KustoIntellisenseHelper.GetDefaultDiagnostics(parseInfo, scriptFile, queryText);
+                    }
+
+                default:
+                    throw new ArgumentException($"Unsupported data source type \"{dataSourceType}\"", nameof(dataSourceType));
+            }
+        }
+
+        // Converts database details shown on cluster manage dashboard to DatabaseInfo type. Add DataSourceType as param if required to show different properties
+        public static List<DatabaseInfo> ConvertToDatabaseInfo(IEnumerable<DataSourceObjectMetadata> clusterDBDetails)
+        {
+            var databaseDetails = new List<DatabaseInfo>();
+
+            if(typeof(DatabaseMetadata) == clusterDBDetails.FirstOrDefault().GetType()){
+                foreach(var dbDetail in clusterDBDetails)
+                {
+                    DatabaseInfo databaseInfo = new DatabaseInfo();
+                    Int64.TryParse(dbDetail.SizeInMB.ToString(), out long sum_OriginalSize);
+                    databaseInfo.Options["name"] = dbDetail.Name;
+                    databaseInfo.Options["sizeInMB"] = (sum_OriginalSize /(1024 * 1024)).ToString();
+                    databaseDetails.Add(databaseInfo);
+                }
+            }
+
+            return databaseDetails;
+        }
+
+        // Converts tables details shown on database manage dashboard to ObjectMetadata type. Add DataSourceType as param if required to show different properties
+        public static List<ObjectMetadata> ConvertToObjectMetadata(IEnumerable<DataSourceObjectMetadata> dbChildDetails)
+        {
+            var databaseChildDetails = new List<ObjectMetadata>();
+
+            foreach(var childDetail in dbChildDetails)
+            {
+                ObjectMetadata dbChildInfo = new ObjectMetadata();
+                dbChildInfo.Name = childDetail.PrettyName;
+                dbChildInfo.MetadataTypeName = childDetail.MetadataTypeName;
+                dbChildInfo.MetadataType = MetadataType.Table;         // Add mapping here.
+                databaseChildDetails.Add(dbChildInfo);
+            }
+            return databaseChildDetails;
+        }
+
+        public static ReliableConnectionHelper.ServerInfo ConvertToServerinfoFormat(DataSourceType dataSourceType, DiagnosticsInfo clusterDiagnostics)
+        {
+            switch (dataSourceType)
+            {
+                case DataSourceType.Kusto:
+                    {
+                        ReliableConnectionHelper.ServerInfo serverInfo = new ReliableConnectionHelper.ServerInfo();
+                        serverInfo.Options = new Dictionary<string, object>(clusterDiagnostics.Options);
+                        return serverInfo;
+                    }
+
+                default:
+                    throw new ArgumentException($"Unsupported data source type \"{dataSourceType}\"", nameof(dataSourceType));
+            }
+        }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/DataSourceIntellisense/KustoIntellisenseHelper.cs

@@ -0,0 +1,329 @@
+//
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+//
+
+using System;
+using System.Collections.Generic;
+using System.Threading;
+using System.Threading.Tasks;
+using System.Linq;
+using Kusto.Language;
+using Kusto.Language.Editor;
+using Kusto.Language.Syntax;
+using Kusto.Language.Symbols;
+using Microsoft.Kusto.ServiceLayer.LanguageServices;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Contracts;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Completion;
+using Microsoft.Kusto.ServiceLayer.Workspace.Contracts;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource.DataSourceIntellisense
+{
+    /// <summary>
+    /// Kusto specific class for intellisense helper functions.
+    /// </summary>
+    public static class KustoIntellisenseHelper
+    {
+
+        public class ShowDatabasesResult
+        {
+            public string DatabaseName;
+            public string PersistentStorage;
+            public string Version;
+            public bool IsCurrent;
+            public string DatabaseAccessMode;
+            public string PrettyName;
+            public bool CurrentUserIsUnrestrictedViewer;
+            public string DatabaseId;
+        }
+
+        public class ShowDatabaseSchemaResult
+        {
+            public string DatabaseName;
+            public string TableName;
+            public string ColumnName;
+            public string ColumnType;
+            public bool IsDefaultTable;
+            public bool IsDefaultColumn;
+            public string PrettyName;
+            public string Version;
+            public string Folder;
+            public string DocName;
+        }
+
+        public class ShowFunctionsResult
+        {
+            public string Name;
+            public string Parameters;
+            public string Body;
+            public string Folder;
+            public string DocString;
+        }
+
+        /// <summary>
+        /// Convert CLR type name into a Kusto scalar type.
+        /// </summary>
+        private static ScalarSymbol GetKustoType(string clrTypeName)
+        {
+            switch (clrTypeName)
+            {
+                case "System.Byte":
+                case "Byte":
+                case "byte":
+                case "System.SByte":
+                case "SByte":
+                case "sbyte":
+                case "System.Int16":
+                case "Int16":
+                case "short":
+                case "System.UInt16":
+                case "UInt16":
+                case "ushort":
+                case "System.Int32":
+                case "System.Single":
+                case "Int32":
+                case "int":
+                    return ScalarTypes.Int;
+                case "System.UInt32": // unsigned ints don't fit into int, use long
+                case "UInt32":
+                case "uint":
+                case "System.Int64":
+                case "Int64":
+                case "long":
+                    return ScalarTypes.Long;
+                case "System.Double":
+                case "Double":
+                case "double":
+                case "float":
+                    return ScalarTypes.Real;
+                case "System.UInt64": // unsigned longs do not fit into long, use decimal
+                case "UInt64":
+                case "ulong":
+                case "System.Decimal":
+                case "Decimal":
+                case "decimal":
+                case "System.Data.SqlTypes.SqlDecimal":
+                case "SqlDecimal":
+                    return ScalarTypes.Decimal;
+                case "System.Guid":
+                case "Guid":
+                    return ScalarTypes.Guid;
+                case "System.DateTime":
+                case "DateTime":
+                    return ScalarTypes.DateTime;
+                case "System.TimeSpan":
+                case "TimeSpan":
+                    return ScalarTypes.TimeSpan;
+                case "System.String":
+                case "String":
+                case "string":
+                    return ScalarTypes.String;
+                case "System.Boolean":
+                case "Boolean":
+                case "bool":
+                    return ScalarTypes.Bool;
+                case "System.Object":
+                case "Object":
+                case "object":
+                    return ScalarTypes.Dynamic;
+                case "System.Type":
+                case "Type":
+                    return ScalarTypes.Type;
+                default:
+                    throw new InvalidOperationException($"Unhandled clr type: {clrTypeName}");
+            }
+        }
+
+        private static IReadOnlyList<Parameter> NoParameters = new Parameter[0];
+
+        /// <summary>
+        /// Translate Kusto parameter list declaration into into list of <see cref="Parameter"/> instances.
+        /// </summary>
+        private static IReadOnlyList<Parameter> TranslateParameters(string parameters)
+        {
+            parameters = parameters.Trim();
+
+            if (string.IsNullOrEmpty(parameters) || parameters == "()")
+                return NoParameters;
+
+            if (parameters[0] != '(')
+                parameters = "(" + parameters;
+            if (parameters[parameters.Length - 1] != ')')
+                parameters = parameters + ")";
+
+            var query = "let fn = " + parameters + " { };";
+            var code = KustoCode.ParseAndAnalyze(query);
+            var let = code.Syntax.GetFirstDescendant<LetStatement>();
+            var variable = let.Name.ReferencedSymbol as VariableSymbol;
+            var function = variable.Type as FunctionSymbol;
+            return function.Signatures[0].Parameters;
+        }
+
+        /// <summary>
+        /// Loads the schema for the specified databasea into a a <see cref="DatabaseSymbol"/>.
+        /// </summary>
+        public static async Task<DatabaseSymbol> LoadDatabaseAsync(IDataSource dataSource, string databaseName, bool throwOnError = false)
+        {
+            var members = new List<Symbol>();
+            CancellationTokenSource source = new CancellationTokenSource();
+            CancellationToken cancellationToken = source.Token;
+
+            var tableSchemas = await dataSource.ExecuteControlCommandAsync<ShowDatabaseSchemaResult>($".show database {databaseName} schema", throwOnError, cancellationToken).ConfigureAwait(false);
+            if (tableSchemas == null)
+                return null;
+
+            tableSchemas = tableSchemas
+                .Where(r => !string.IsNullOrEmpty(r.TableName) && !string.IsNullOrEmpty(r.ColumnName))
+                .ToArray();
+
+            foreach (var table in tableSchemas.GroupBy(s => s.TableName))
+            {
+                var columns = table.Select(s => new ColumnSymbol(s.ColumnName, GetKustoType(s.ColumnType))).ToList();
+                var tableSymbol = new TableSymbol(table.Key, columns);
+                members.Add(tableSymbol);
+            }
+
+            var functionSchemas = await dataSource.ExecuteControlCommandAsync<ShowFunctionsResult>(".show functions", throwOnError, cancellationToken).ConfigureAwait(false);
+            if (functionSchemas == null)
+                return null;
+
+            foreach (var fun in functionSchemas)
+            {
+                var parameters = TranslateParameters(fun.Parameters);
+                var functionSymbol = new FunctionSymbol(fun.Name, fun.Body, parameters);
+                members.Add(functionSymbol);
+            }
+
+            var databaseSymbol = new DatabaseSymbol(databaseName, members);
+            return databaseSymbol;
+        }
+
+        public static CompletionItemKind CreateCompletionItemKind(CompletionKind kustoKind)
+        {
+            CompletionItemKind kind = CompletionItemKind.Variable;
+            switch (kustoKind)
+            {
+                case CompletionKind.Syntax:
+                    kind = CompletionItemKind.Module;
+                    break;
+                case CompletionKind.Column:
+                    kind = CompletionItemKind.Field;
+                    break;
+                case CompletionKind.Variable:
+                    kind = CompletionItemKind.Variable;
+                    break;
+                case CompletionKind.Table:
+                    kind = CompletionItemKind.File;
+                    break;
+                case CompletionKind.Database:
+                    kind = CompletionItemKind.Method;
+                    break;
+                case CompletionKind.LocalFunction:
+                case CompletionKind.DatabaseFunction:
+                case CompletionKind.BuiltInFunction:
+                case CompletionKind.AggregateFunction:
+                    kind = CompletionItemKind.Function;
+                    break;
+                default:
+                    kind = CompletionItemKind.Keyword;
+                    break;
+            }
+
+            return kind;
+        }
+
+        /// <summary>
+        /// Gets default keyword when user if not connected to any Kusto cluster.
+        /// </summary>
+        public static LanguageServices.Contracts.CompletionItem[] GetDefaultKeywords(ScriptDocumentInfo scriptDocumentInfo, Position textDocumentPosition){
+                var kustoCodeService = new KustoCodeService(scriptDocumentInfo.Contents, GlobalState.Default);
+                var script = CodeScript.From(scriptDocumentInfo.Contents, GlobalState.Default);
+                script.TryGetTextPosition(textDocumentPosition.Line + 1, textDocumentPosition.Character, out int position);     // Gets the actual offset based on line and local offset      
+                var completion = kustoCodeService.GetCompletionItems(position);
+
+                List<LanguageServices.Contracts.CompletionItem> completions = new List<LanguageServices.Contracts.CompletionItem>();
+                foreach (var autoCompleteItem in completion.Items)
+                {
+                    var label = autoCompleteItem.DisplayText;
+                    // convert the completion item candidates into vscode format CompletionItems
+                    completions.Add(AutoCompleteHelper.CreateCompletionItem(label, label + " keyword", label, CompletionItemKind.Keyword, scriptDocumentInfo.StartLine, scriptDocumentInfo.StartColumn, textDocumentPosition.Character));
+                }
+
+                return completions.ToArray();
+        }
+
+        /// <summary>
+        /// Gets default diagnostics when user if not connected to any Kusto cluster.
+        /// </summary>
+        public static ScriptFileMarker[] GetDefaultDiagnostics(ScriptParseInfo parseInfo, ScriptFile scriptFile, string queryText){
+            var kustoCodeService = new KustoCodeService(queryText, GlobalState.Default);
+            var script = CodeScript.From(queryText, GlobalState.Default);
+            var parseResult = kustoCodeService.GetDiagnostics();
+            
+            parseInfo.ParseResult = parseResult;
+            
+            // build a list of Kusto script file markers from the errors.
+            List<ScriptFileMarker> markers = new List<ScriptFileMarker>();
+            if (parseResult != null && parseResult.Count() > 0)
+            {
+                foreach (var error in parseResult)
+                {
+                    script.TryGetLineAndOffset(error.Start, out var startLine, out var startOffset);
+                    script.TryGetLineAndOffset(error.End, out var endLine, out var endOffset);
+
+                    // vscode specific format for error markers.
+                    markers.Add(new ScriptFileMarker()
+                    {
+                        Message = error.Message,
+                        Level = ScriptFileMarkerLevel.Error,
+                        ScriptRegion = new ScriptRegion()
+                        {
+                            File = scriptFile.FilePath,
+                            StartLineNumber = startLine,
+                            StartColumnNumber = startOffset,
+                            StartOffset = 0,
+                            EndLineNumber = endLine,
+                            EndColumnNumber = endOffset,
+                            EndOffset = 0
+                        }
+                    });
+                }
+            }
+
+            return markers.ToArray();
+        }
+
+        /// <summary>
+        /// Loads the schema for the specified database and returns a new <see cref="GlobalState"/> with the database added or updated.
+        /// </summary>
+        public static async Task<GlobalState> AddOrUpdateDatabaseAsync(IDataSource dataSource, GlobalState globals, string databaseName, string clusterName, bool throwOnError)
+        {   // try and show error from here.
+            DatabaseSymbol databaseSymbol = null;
+
+            if(databaseName != null){
+                databaseSymbol = await LoadDatabaseAsync(dataSource, databaseName, throwOnError).ConfigureAwait(false);
+            }
+
+            if(databaseSymbol == null){
+                return globals;
+            }
+
+            var cluster = globals.GetCluster(clusterName);
+            if (cluster == null)
+            {
+                cluster = new ClusterSymbol(clusterName, new[] { databaseSymbol }, isOpen: true);
+                globals = globals.AddOrUpdateCluster(cluster);
+            }
+            else
+            {
+                cluster = cluster.AddOrUpdateDatabase(databaseSymbol);
+                globals = globals.AddOrUpdateCluster(cluster);
+            }
+
+            globals = globals.WithCluster(cluster).WithDatabase(databaseSymbol);
+
+            return globals;
+        }
+
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/DataSourceIntellisense/ScriptParseInfo.cs

@@ -0,0 +1,48 @@
+//
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+//
+
+using System.Collections.Generic;
+using Kusto.Language;
+using Kusto.Language.Editor;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource.DataSourceIntellisense
+{
+    /// <summary>
+    /// Data Source specific class for storing cached metadata regarding a parsed KQL file.
+    /// </summary>
+    public class ScriptParseInfo
+    {
+        private object buildingMetadataLock = new object();
+
+        /// <summary>
+        /// Event which tells if MetadataProvider is built fully or not
+        /// </summary>
+        public object BuildingMetadataLock
+        { 
+            get { return this.buildingMetadataLock; }
+        }
+
+        /// <summary>
+        /// Gets or sets a flag determining is the LanguageService is connected
+        /// </summary>
+        public bool IsConnected { get; set; }
+
+        /// <summary>
+        /// Gets or sets the binding queue connection context key
+        /// </summary>
+        public string ConnectionKey { get; set; }
+
+        /// <summary>
+        /// Gets or sets the previous Kusto diagnostics result. TODOKusto: Check exact usage.
+        /// </summary>
+        public IReadOnlyList<Diagnostic> ParseResult { get; set; }
+        
+        /// <summary>
+        /// Gets or sets the current autocomplete suggestion list retrieved from the Kusto language library.
+        /// So that other details like documentation can be later retrieved in ResolveCompletionItem.
+        /// </summary>
+        public IEnumerable<CompletionItem > CurrentSuggestions { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/DataSourceType.cs

@@ -0,0 +1,28 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    /// <summary>
+    /// Represents the type of a data source.
+    /// </summary>
+    public enum DataSourceType
+    {
+        /// <summary>
+        /// Unknown.
+        /// </summary>
+        None,
+
+        /// <summary>
+        /// A Kusto cluster.
+        /// </summary>
+        Kusto,
+
+        /// <summary>
+        /// An Application Insights subscription.
+        /// </summary>
+        ApplicationInsights,
+
+        /// <summary>
+        /// An Operations Management Suite (OMS) Log Analytics workspace.
+        /// </summary>
+        OmsLogAnalytics
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/DiagnosticsInfo.cs

@@ -0,0 +1,17 @@
+using System.Collections.Generic;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    public class DiagnosticsInfo
+    {
+        /// <summary>
+        /// Gets or sets the options
+        /// </summary>
+        public Dictionary<string, object> Options { get; set; }
+
+        public DiagnosticsInfo()
+        {
+            Options = new Dictionary<string, object>();
+        }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/IDataSource.cs

@@ -0,0 +1,132 @@
+using System;
+using System.Collections.Generic;
+using System.Data;
+using System.Threading;
+using System.Threading.Tasks;
+using Microsoft.Kusto.ServiceLayer.DataSource.DataSourceIntellisense;
+using Microsoft.Kusto.ServiceLayer.DataSource.Metadata;
+using Microsoft.Kusto.ServiceLayer.LanguageServices;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Completion;
+using Microsoft.Kusto.ServiceLayer.LanguageServices.Contracts;
+using Microsoft.Kusto.ServiceLayer.Workspace.Contracts;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    /// <summary>
+    /// Represents data source utilities.
+    /// </summary>
+    public interface IDataSource : IDisposable
+    {
+        /// <summary>
+        /// The data source type.
+        /// </summary>
+        DataSourceType DataSourceType { get; }
+
+        /// <summary>
+        /// The cluster/server name.
+        /// </summary>
+        string ClusterName { get; }
+
+        /// <summary>
+        /// The current database name, if there is one.
+        /// </summary>
+        string DatabaseName { get; set; }
+
+        /// <summary>
+        /// Executes a query.
+        /// </summary>
+        /// <param name="query">The query.</param>
+        /// <returns>The results.</returns>
+        Task<IDataReader> ExecuteQueryAsync(string query, CancellationToken cancellationToken, string databaseName = null);
+
+        /// <summary>
+        /// Executes a Kusto query that returns a scalar value.
+        /// </summary>
+        /// <typeparam name="T">The type of the result.</typeparam>
+        /// <param name="query">The query.</param>
+        /// <returns>The result.</returns>
+        Task<T> ExecuteScalarQueryAsync<T>(string query, CancellationToken cancellationToken, string databaseName = null);
+
+        /// <summary>
+        /// Executes a Kusto query that returns a scalar value.
+        /// </summary>
+        /// <typeparam name="T">The type of the result.</typeparam>
+        /// <param name="query">The query.</param>
+        /// <returns>The result.</returns>
+        Task<IEnumerable<T>> ExecuteControlCommandAsync<T>(string command, bool throwOnError, CancellationToken cancellationToken);
+
+        /// <summary>
+        /// Get children of the  given parent
+        /// </summary>
+        /// <param name="parentMetadata">Parent object metadata.</param>
+        /// <returns>Metadata for all children.</returns>
+        DiagnosticsInfo GetDiagnostics(DataSourceObjectMetadata parentMetadata);
+
+        /// <summary>
+        /// Get children of the  given parent
+        /// </summary>
+        /// <param name="parentMetadata">Parent object metadata.</param>
+        /// <param name="includeSizeDetails"></param>
+        /// <returns>Metadata for all children.</returns>
+        IEnumerable<DataSourceObjectMetadata> GetChildObjects(DataSourceObjectMetadata parentMetadata, bool includeSizeDetails = false);
+
+        /// <summary>
+        /// Refresh object list for entire cluster.
+        /// </summary>
+        void Refresh();
+
+        /// <summary>
+        /// Refresh object list for given object.
+        /// </summary>
+        /// <param name="objectMetadata">Object metadata.</param>
+        void Refresh(DataSourceObjectMetadata objectMetadata);
+
+        /// <summary>
+        /// Updates database and affected variables like GlobalState for given object.
+        /// </summary>
+        /// <param name="updateDatabase">Object metadata.</param>
+        void UpdateDatabase(string databaseName);
+
+        /// <summary>
+        /// Gets autocomplete suggestions at given position.
+        /// </summary>
+        /// <param name="GetAutoCompleteSuggestions">Object metadata.</param>
+        CompletionItem[] GetAutoCompleteSuggestions(ScriptDocumentInfo queryText, Position index, bool throwOnError = false);
+        /// <summary>
+        /// Gets quick info hover tooltips for the current position.
+        /// </summary>
+        /// <param name="GetHoverHelp">Object metadata.</param>
+        Hover GetHoverHelp(ScriptDocumentInfo scriptDocumentInfo, Position textPosition, bool throwOnError = false);
+
+        /// <summary>
+        /// Gets definition for a selected query text.
+        /// </summary>
+        /// <param name="GetDefinition">Object metadata.</param>
+        DefinitionResult GetDefinition(string queryText, int index, int startLine, int startColumn, bool throwOnError = false);
+        
+        /// <summary>
+        /// Gets a list of semantic diagnostic marks for the provided script file
+        /// </summary>
+        /// <param name="GetSemanticMarkers">Object metadata.</param>
+        ScriptFileMarker[] GetSemanticMarkers(ScriptParseInfo parseInfo, ScriptFile scriptFile, string queryText);
+
+        /// <summary>
+        /// Tells whether the data source exists.
+        /// </summary>
+        /// <returns>true if it exists; false otherwise.</returns>
+        Task<bool> Exists();
+
+        /// <summary>
+        /// Tells whether the object exists.
+        /// </summary>
+        /// <returns>true if it exists; false otherwise.</returns>
+        bool Exists(DataSourceObjectMetadata objectMetadata);
+
+        /// <summary>
+        /// Gets FunctionInfo object for a function
+        /// </summary>
+        /// <param name="functionName"></param>
+        /// <returns></returns>
+        string GenerateAlterFunctionScript(string functionName);
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/KustoQueryUtils.cs

@@ -0,0 +1,115 @@
+// <copyright file="KustoQueryUtils.cs" company="Microsoft">
+// Copyright (c) Microsoft. All Rights Reserved.
+// </copyright>
+using System;
+using System.Collections.Concurrent;
+using System.Collections.Generic;
+using System.Text.RegularExpressions;
+using System.Linq;
+using Microsoft.Kusto.ServiceLayer.DataSource.Metadata;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    public static class KustoQueryUtils
+    {
+        public const string StatementSeparator = "\n | "; // Start each statement on a new line. Not required by Kusto, but doing this for readability of scripts generated from here.
+
+        /// <summary>
+        /// Escape table/column/database names for a Kusto query.
+        /// </summary>
+        /// <param name="name">The name to be escaped</param>
+        /// <param name="alwaysEscape">Always escape if this flag is set</param>
+        /// <returns>The escaped string</returns>
+        public static string EscapeName(string name, bool alwaysEscape = false)
+        {
+            if (name.StartsWith("[@") || name == "*") // Field already escaped. No escaping required for '*' operand
+            { 
+                return name;
+            } 
+
+            string result = name;
+            Regex  rx = new Regex("[^_a-zA-Z0-9]");
+            string [] kustoKeywordList = {"and", "anomalychart", "areachart", "asc", "barchart", "between", "bool", "boolean", "by",
+                "columnchart", "consume", "contains", "containscs", "count", "date", "datetime", "default", "desc", "distinct",
+                "double", "dynamic", "endswith", "evaluate", "extend", "false", "filter", "find", "first", "flags", "float",
+                "getschema", "has", "hasprefix", "hassuffix", "in", "int", "join", "journal", "kind", "ladderchart", "last",
+                "like", "limit", "linechart", "long", "materialize", "mvexpand", "notcontains", "notlike", "of", "or", "order",
+                "parse", "piechart", "pivotchart", "print", "project", "queries", "real", "regex", "sample", "scatterchart",
+                "search", "set", "sort", "stacked", "stacked100", "stackedareachart", "startswith", "string", "summarize",
+                "take", "time", "timechart", "timeline", "timepivot", "timespan", "to", "top", "toscalar", "true", "union", 
+                "unstacked", "viewers", "where", "withsource"}; // add more keywords here
+
+            var escapeName = rx.IsMatch(name) || kustoKeywordList.Any(name.Contains) || alwaysEscape;
+            if (escapeName) 
+            {
+                if (name.IndexOf('"') > -1) 
+                {
+                    result = "[@'" + name + "']";
+                }
+                else 
+                {
+                    result = "[@\"" + name + "\"]";
+                }
+            }
+
+            return result;
+        }
+
+        
+        public static bool  IsClusterLevelQuery(string query) 
+        {
+            string [] clusterLevelQueryPrefixes = {
+                ".show databases",
+                ".show schema"
+            };
+
+            return clusterLevelQueryPrefixes.Any(query.StartsWith);
+        }
+        
+        /// <summary>
+        /// Adds an object of type DataSourceObjectMetadata to a dictionary<string, Dictionary<string, T>>. If the key exists then the item is added
+        /// to the list. If not then the key is created and then added.
+        /// </summary>
+        /// <param name="dictionary">The dictionary of the dictionary that the list should be added to.</param>
+        /// <param name="key">The key to be added.</param>
+        /// <param name="metadata">The metadata to be added to the list.</param>
+        /// <typeparam name="T"></typeparam>
+        public static void SafeAdd<T>(this Dictionary<string, Dictionary<string, T>> dictionary, string key,
+            T metadata) where T : DataSourceObjectMetadata
+        {
+            if (dictionary.ContainsKey(key))
+            {
+                if (dictionary[key].ContainsKey(metadata.Name))
+                {
+                    return;
+                }
+                    
+                dictionary[key].Add(metadata.Name, metadata);
+            }
+            else
+            {
+                dictionary[key] = new Dictionary<string, T> {{metadata.Name, metadata}};
+            }
+        }
+        
+        /// <summary>
+        /// Add a range to a dictionary of ConcurrentDictionary. Adds range to existing IEnumerable within dictionary
+        /// at the same key.
+        /// </summary>
+        /// <param name="dictionary"></param>
+        /// <param name="key"></param>
+        /// <param name="metadatas"></param>
+        /// <typeparam name="T"></typeparam>
+        public static void AddRange<T>(this ConcurrentDictionary<string, IEnumerable<T>> dictionary, string key,
+            List<T> metadatas) where T : DataSourceObjectMetadata
+        {
+            if (dictionary.ContainsKey(key))
+            {
+                metadatas.AddRange(dictionary[key]);
+            }
+            
+            dictionary[key] = metadatas.OrderBy(x => x.PrettyName, StringComparer.OrdinalIgnoreCase).ToList();
+        }
+
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/KustoResultsReader.cs

@@ -0,0 +1,22 @@
+using System.Data;
+
+namespace Microsoft.Kusto.ServiceLayer.DataSource
+{
+    internal class KustoResultsReader : DataReaderWrapper
+    {
+        public KustoResultsReader(IDataReader reader)
+            : base(reader)
+        {
+        }
+
+        /// <summary>
+        /// Kusto returns 3 results tables - QueryResults, QueryProperties, QueryStatus. When returning query results
+        /// we want the caller to only read the first table. We override the NextResult function here to only return one table
+        /// from the IDataReader.
+        /// </summary>
+        /*public override bool NextResult()
+        {
+            return false;
+        }*/
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Metadata/ColumnMetadata.cs

@@ -0,0 +1,11 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Metadata
+{
+    /// <summary>
+    /// Column metadata information
+    /// </summary>
+    public class ColumnMetadata : TableMetadata
+    {
+        public string TableName { get; set; }
+        public string DataType { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Metadata/DataSourceMetadataType.cs

@@ -0,0 +1,15 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Metadata
+{
+    /// <summary>
+    /// Metadata type enumeration
+    /// </summary>
+    public enum DataSourceMetadataType
+    {
+        Cluster = 0,
+        Database = 1,
+        Table = 2,
+        Column = 3,
+        Function = 4,
+        Folder = 5
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Metadata/DataSourceObjectMetadata.cs

@@ -0,0 +1,20 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Metadata
+{
+    /// <summary>
+    /// Object metadata information
+    /// </summary>
+    public class DataSourceObjectMetadata 
+    {
+        public DataSourceMetadataType MetadataType { get; set; }
+    
+        public string MetadataTypeName { get; set; }
+
+        public string Name { get; set; }
+
+        public string PrettyName { get; set; }
+        
+        public string Urn { get; set; }
+        
+        public string SizeInMB { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Metadata/DatabaseMetadata.cs

@@ -0,0 +1,10 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Metadata
+{
+    /// <summary>
+    /// Database metadata information
+    /// </summary>
+    public class DatabaseMetadata : DataSourceObjectMetadata
+    {
+        public string ClusterName { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Metadata/FolderMetadata.cs

@@ -0,0 +1,10 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Metadata
+{
+    /// <summary>
+    /// Folder metadata information
+    /// </summary>
+    public class FolderMetadata : DataSourceObjectMetadata
+    {
+        public DataSourceObjectMetadata ParentMetadata { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Metadata/FunctionMetadata.cs

@@ -0,0 +1,11 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Metadata
+{
+    public class FunctionMetadata : DatabaseMetadata
+    {
+        public string DatabaseName { get; set; }
+        
+        public string Parameters { get; set; }
+        
+        public string Body { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Metadata/TableMetadata.cs

@@ -0,0 +1,11 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Metadata
+{
+    /// <summary>
+    /// Database metadata information
+    /// </summary>
+    public class TableMetadata : DatabaseMetadata
+    {
+        public string DatabaseName { get; set; }
+        public string Folder { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Models/ColumnInfo.cs

@@ -0,0 +1,26 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Models
+{
+    public class ColumnInfo
+    {
+        /// <summary>
+        /// The table name.
+        /// </summary>
+        public string Table { get; set; }
+
+        /// <summary>
+        /// The column name.
+        /// </summary>
+        public string Name { get; set; }
+
+        /// <summary>
+        /// The data type.
+        /// </summary>
+        public string DataType { get; set; }
+
+
+        /// <summary>
+        /// The folder name.
+        /// </summary>
+        public string Folder { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Models/FunctionInfo.cs

@@ -0,0 +1,11 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Models
+{
+    public class FunctionInfo
+    {
+        public string Name { get; set; }
+        public string Parameters { get; set; }
+        public string Body { get; set; }
+        public string Folder { get; set; }
+        public string DocString { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/Models/TableInfo.cs

@@ -0,0 +1,8 @@
+namespace Microsoft.Kusto.ServiceLayer.DataSource.Models
+{
+    public class TableInfo
+    {
+        public string TableName { get; set; }
+        public string Folder { get; set; }
+    }
+}

src/Microsoft.Kusto.ServiceLayer/DataSource/ReliableDataSourceConnection.cs

@@ -0,0 +1,258 @@
+//
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+//
+
+// This code is copied from the source described in the comment below.
+
+// =======================================================================================
+// Microsoft Windows Server AppFabric Customer Advisory Team (CAT) Best Practices Series
+//
+// This sample is supplemental to the technical guidance published on the community
+// blog at http://blogs.msdn.com/appfabriccat/ and  copied from
+// sqlmain ./sql/manageability/mfx/common/
+//
+// =======================================================================================
+// Copyright © 2012 Microsoft Corporation. All rights reserved.
+// 
+// THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER 
+// EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF 
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. YOU BEAR THE RISK OF USING IT.
+// =======================================================================================
+
+using System;
+using System.Collections.Generic;
+using System.Data;
+using System.Collections;
+using System.Data.Common;
+using System.Data.SqlClient;
+using System.Diagnostics;
+using System.Globalization;
+using System.Text;
+using System.Threading;
+using System.Text.RegularExpressions;
+using System.Threading.Tasks;
+using Microsoft.SqlTools.Utility;
+using Microsoft.SqlTools.ServiceLayer.Connection.ReliableConnection;
+using Microsoft.Kusto.ServiceLayer.DataSource;
+
+namespace Microsoft.Kusto.ServiceLayer.Connection
+{
+    /// <summary>
+    /// Provides a reliable way of opening connections to and executing commands
+    /// taking into account potential network unreliability and a requirement for connection retry.
+    /// </summary>
+    public sealed partial class ReliableDataSourceConnection : IDisposable
+    {
+        private IDataSource _dataSource;
+        private readonly RetryPolicy _connectionRetryPolicy;
+        private RetryPolicy _commandRetryPolicy;
+        private Guid _azureSessionId = Guid.NewGuid();
+
+        private string _connectionString;
+        private string _azureAccountToken;
+
+        /// <summary>
+        /// Initializes a new instance of the ReliableKustoClient class with a given connection string
+        /// and a policy defining whether to retry a request if the connection fails to be opened or a command
+        /// fails to be successfully executed.
+        /// </summary>
+        /// <param name="connectionString">The connection string used to open the SQL Azure database.</param>
+        /// <param name="connectionRetryPolicy">The retry policy defining whether to retry a request if a connection fails to be established.</param>
+        /// <param name="commandRetryPolicy">The retry policy defining whether to retry a request if a command fails to be executed.</param>
+        public ReliableDataSourceConnection(string connectionString, RetryPolicy connectionRetryPolicy, RetryPolicy commandRetryPolicy, string azureAccountToken)
+        {
+            _connectionString = connectionString;
+            _azureAccountToken = azureAccountToken;
+            _dataSource = DataSourceFactory.Create(DataSourceType.Kusto, connectionString, azureAccountToken);
+            
+            _connectionRetryPolicy = connectionRetryPolicy ?? RetryPolicyFactory.CreateNoRetryPolicy();
+            _commandRetryPolicy = commandRetryPolicy ?? RetryPolicyFactory.CreateNoRetryPolicy();
+
+            _connectionRetryPolicy.RetryOccurred += RetryConnectionCallback;
+            _commandRetryPolicy.RetryOccurred += RetryCommandCallback;
+        }
+
+        private void RetryCommandCallback(RetryState retryState)
+        {
+            RetryPolicyUtils.RaiseSchemaAmbientRetryMessage(retryState, SqlSchemaModelErrorCodes.ServiceActions.CommandRetry, _azureSessionId); 
+        }
+
+        private void RetryConnectionCallback(RetryState retryState)
+        {
+            RetryPolicyUtils.RaiseSchemaAmbientRetryMessage(retryState, SqlSchemaModelErrorCodes.ServiceActions.ConnectionRetry, _azureSessionId); 
+        }
+
+        /// <summary>
+        /// Disposes resources.
+        /// </summary>
+        public void Dispose()
+        {
+            Dispose(true);
+            GC.SuppressFinalize(this);
+        }
+
+        /// <summary>
+        /// Performs application-defined tasks associated with freeing, releasing, or
+        ///  resetting managed and unmanaged resources.
+        /// </summary>
+        /// <param name="disposing">A flag indicating that managed resources must be released.</param>
+        public void Dispose(bool disposing)
+        {
+            if (disposing)
+            {
+                if (_connectionRetryPolicy != null)
+                {
+                    _connectionRetryPolicy.RetryOccurred -= RetryConnectionCallback;
+                }
+
+                if (_commandRetryPolicy != null)
+                {
+                    _commandRetryPolicy.RetryOccurred -= RetryCommandCallback;
+                }
+
+                _dataSource.Dispose();
+            }
+        }
+
+        /// <summary>
+        /// Gets or sets the connection string for opening a connection to the SQL Azure database.
+        /// </summary>
+        public string ConnectionString { get; set; }
+        
+        /// <summary>
+        /// Gets the policy which decides whether to retry a connection request, based on how many
+        /// times the request has been made and the reason for the last failure. 
+        /// </summary>
+        public RetryPolicy ConnectionRetryPolicy
+        {
+            get { return _connectionRetryPolicy; }
+        }
+
+        /// <summary>
+        /// Gets the policy which decides whether to retry a command, based on how many
+        /// times the request has been made and the reason for the last failure. 
+        /// </summary>
+        public RetryPolicy CommandRetryPolicy
+        {
+            get { return _commandRetryPolicy; }
+            set
+            {
+                Validate.IsNotNull(nameof(value), value);
+
+                if (_commandRetryPolicy != null)
+                {
+                    _commandRetryPolicy.RetryOccurred -= RetryCommandCallback;
+                }
+
+                _commandRetryPolicy = value;
+                _commandRetryPolicy.RetryOccurred += RetryCommandCallback;
+            }
+        }
+
+        /// <summary>
+        /// Gets the server name from the underlying connection.
+        /// </summary>
+        public string ClusterName
+        {
+            get { return _dataSource.ClusterName; }
+        }
+
+        /// <summary>
+        /// If the underlying SqlConnection absolutely has to be accessed, for instance
+        /// to pass to external APIs that require this type of connection, then this
+        /// can be used.  
+        /// </summary>
+        /// <returns><see cref="SqlConnection"/></returns>
+        public IDataSource GetUnderlyingConnection()
+        {
+            return _dataSource;
+        }
+
+        /// <summary>
+        /// Changes the current database for an open Connection object.
+        /// </summary>
+        /// <param name="databaseName">The name of the database to use in place of the current database.</param>
+        public void ChangeDatabase(string databaseName)
+        {
+            _dataSource.UpdateDatabase(databaseName);
+        }
+
+        /// <summary>
+        /// Opens a database connection with the settings specified by the ConnectionString
+        /// property of the provider-specific Connection object.
+        /// </summary>
+        public void Open()
+        {
+            // TODOKusto: Should we initialize in the constructor or here. Set a breapoint and check.
+            // Check if retry policy was specified, if not, disable retries by executing the Open method using RetryPolicy.NoRetry.
+            if(_dataSource == null)
+            {
+                _connectionRetryPolicy.ExecuteAction(() =>
+                {
+                    _dataSource = DataSourceFactory.Create(DataSourceType.Kusto, _connectionString, _azureAccountToken);
+                });
+            }
+        }
+
+        /// <summary>
+        /// Opens a database connection with the settings specified by the ConnectionString
+        /// property of the provider-specific Connection object.
+        /// </summary>
+        public Task OpenAsync(CancellationToken token)
+        {
+            // Make sure that the token isn't cancelled before we try
+            if (token.IsCancellationRequested)
+            {
+                return Task.FromCanceled(token);
+            }
+
+            // Check if retry policy was specified, if not, disable retries by executing the Open method using RetryPolicy.NoRetry.
+            try
+            {
+                return _connectionRetryPolicy.ExecuteAction(async () =>
+                {
+                    await Task.Run(() => Open());
+                });
+            }
+            catch (Exception e)
+            {
+                return Task.FromException(e);
+            }
+        }
+
+        /// <summary>
+        /// Closes the connection to the database.
+        /// </summary>
+        public void Close()
+        {
+        }
+
+        /// <summary>
+        /// Gets the time to wait while trying to establish a connection before terminating
+        /// the attempt and generating an error.
+        /// </summary>
+        public int ConnectionTimeout
+        {
+            get { return 30; }
+        }
+
+        /// <summary>
+        /// Gets the name of the current database or the database to be used after a
+        /// connection is opened.
+        /// </summary>
+        public string Database
+        {
+            get { return _dataSource.DatabaseName; }
+        }
+
+        private void VerifyConnectionOpen(ReliableDataSourceConnection conn)
+        {
+            if(conn.GetUnderlyingConnection() == null)
+            {
+                conn.Open();
+            }
+        }
+    }
+}
+