This change introduces new boolean flag for `addToProject`, `addScriptItem` and `addFolderItem` APIs that allows to skip files/folders if they are already included in the project file. Original behavior was to throw an error if duplicate entry is encountered, so I kept that as a default (new flag is not set).
I started by fixing the original behavior, as it was not always working as expected. In our extension that relies on `addToProject` API we've hit an issue where duplicate items were added and no errors were thrown. There was a test for this scenario, but unfortunately the test had few bugs in it as well, so I addressed that first. First issue with the test code was missing `await` on the `testUtils.shouldThrowSpecificError` calls, so test was not actually testing anything. After adding missing keywords, I hit an issue where exception was not thrown, but this turned out to be different issue, compared to what we were hitting. In the test code, it was using the very first folder from the generate list of test entires. This folder wass actually the root of the project (where sqlproj file is located), and `addToProject` API had a special case to ignore the root. This means neither first call nor second call was actually adding anything to the project and no errors were produced. I fixed this problem by using next available folder in the generated files list.
After addressing the test code I could not reproduce the issue that we were seeing with duplicate entires being added, everything was working as expected and errors were thrown. I started adding more tests that better resemble our production scenario - add files in subfolders, add files to existing project, rather than a new one. Finally I was able to reproduce the problem in tests when adding a file in a subfolder to an existing project. After investigation this turned out to be an issue with mismatch in how `relativePath` is maintained within the `FileProjectEntry`. When loading an existing project, `relativePath` is populated based on the value of the `Include` attribute of the `Build` item. This attribute is normalized to Windows-style path, using `\`, so for nested file you will have `folder\file.sql`. When adding new item to the project, one could pass either Windows or Unix-style path (`folder/file.sql`), so the path comparison between loaded Windows-style path and newly added Unix-style path was failing, resulting in them being treated as different items. In fact, `addToProject` API that we were using was relying on `Uri` helpers to extract relative path, thus the path was forced to be Unix-style and path was never the same as the loaded one. After this discovery I added a dedicated test to validate the round-trip of the `relativePath` for serialized and desirialized project file.
In order to address this problem, I updated the factory method `createFileProjectEntry` to always run `utils.convertSlashesForSqlProj` on the relative path to ensure we have Windows-style path stored in there. I also optimized the helper code slightly to not do split/join, if there are no split points in the input string, which should eliminate unnecessary array instantiation. It is worth mentioning that I had to normalize the input relative paths in the `addScriptItem` and `addFolderItem` APIs, because there is no guarantee that they will be Windows-style when we try to compare them to `relativePath` of the existing project items.
Finally I was able to add a simply flag and update the condition to return existing record, if duplicates were allowed. I also updated typings file for the extension and added tests to cover this scenario.
* don't allow adding the same file or folder to the sqlproj if it has already been added
* add a couple more checks in test
* toLowerCase when comparing
* relative paths written to sqlproj, but can't delete yet
* only keep track of relative path
* remove leading slash
* add test
* use path.relative
* Add error message if dacpac reference is located on a different drive
* expose addToProject in dataworkspace.d.ts
* remove changes in data workspace extension
* add sqldbproj.d.ts
* change list to be Uris instead of strings
* don't add files/folders if any don't exist
* fix test on windows
* Added publish and schema compare telemetry
* Adding telemetry points for add/exclude/delete
* telemetry for validation
* Adding telemetry from project roundtrip updates, editing sqlproj, and db refs
* Changed method for obtaining extension ID during registration
* Fixing test failures, updating ads telemetry package dependency
* replacing action strings with enums
* change database project string actions to enums
* Changed action name to better match dialog
* PR feedback
* Fix project context menu actions (#12541)
* delete works again
* make fewer changes
* update all sql db project commands
* cleanup
* Remove old projects view (#12563)
* remove old projects view from file explorer view
* fix tests failing
* remove projects in open folder opening up in old view
* Update db reference dialog to show projects in the workspace (#12580)
* update database reference dialog to show projects in the workspace in the project dropdown
* remove workspace stuff from sql projects extension
* undo change
* add class that implements IExtension
* undo a change
* update DataWorkspaceExtension to take workspaceService as a parameter
* add type
* Update sql database project commands (#12595)
* remove sql proj's open and create new project from comman palette
* hook up create project from database to data workspace
* rename the remaining import databases to create project from database
* remove open, new, and close commands
* expose addProjectsToWorkspace() in IExtension instead of calling command
* Addressing comments
* fix failing sql project tests (#12651)
* update SSDT projects opened in projects viewlet (#12669)
* fix action not refreshing the tree issue (#12692)
* fix adding project references in new projects viewlet (#12688)
* Remove old projects tree provider (#12702)
* Remove old projects tree provider and fix tests
* formatting
* update refreshProjectsTree() to accept workspaceTreeItem()
* Cleanup ProjectsController (#12718)
* remove openProject from ProjectController and some cleanup
* rename
* add project and open project dialogs (#12729)
* empty dialogs
* wip
* new project dialog implementation
* revert gitattributes
* open project dialog
* implement add project
* remove icon helper
* refactor
* revert script change
* adjust views
* more updates
* make data-workspace a builtin extension
* show the view only when project provider is detected (#12819)
* only show the view when proj provider is available
* update
* fix sql project tests after merge (#12793)
* Update dialogs to be closer to mockups (#12879)
* small UI changes to dialogs
* center radio card group text
* Create workspace if needed when opening/new project (#12930)
* empty dialogs
* wip
* new project dialog implementation
* revert gitattributes
* open project dialog
* implement add project
* remove icon helper
* refactor
* revert script change
* create workspace
* initial changes
* create new workspace working
* fix tests
* cleanup
* remove showWorkspaceRequiredNotification()
* Add test for no workspace open
* update blue buttons
* move loading temp project to activate() instead of workspaceService constructor
* move workspace creation warning message to before project is created
* pass uri to createWorkspace
* add tests
Co-authored-by: Alan Ren <alanren@microsoft.com>
* Additional create workspace changes (#13004)
* Dialogs workspace updates (#13010)
* adding workspace text boxes
* match new project dialog to mockups
* Add validation error message for workspace file
* add enterWorkspace api
* add warning message for opening workspace
* cleanup
* update commands to remove project so they're more generic
* remove 'empty' from string
* Move default project location setting to data workspace extension (#13022)
* remove project location setting and notification from sql database projects extension
* add default project location setting to data workspace extension
* fix typo
* Add back project name incrementing
* other merge fixes
* fix strings from other PR
* default to last opened directory instead of home directory if no specified default location
* A few small updates (#13092)
* fix build error
* update title for inputboxes
* add missing file
* Add tests for data workspace dialogs (#13324)
* add tests for dialogs
* create helper functions
* New project dialog workspace inputbox fixes (#13407)
* workspace inputbox fixes
* fix folder icons
* Update package.jsons and readme (#13451)
* update package.jsons
* update readme
* add workspace information to open existing dialog (#13455)
Co-authored-by: Alan Ren <alanren@microsoft.com>
* Added Tools Service call for ValidateStreamingJob
* Partial addition of ESJ
* adding test mocks
* Validation working
* Modifying command visibility logic to submatch ESJs in addition to files
* Changed string literal to constant, corrected attribute order
* Added tests
* correcting casing that's causing test failures on linux
* Swapping Thenable for Promise
* excluded validate from command palette
* Add quick pick to select target platform for a project
* add test
* show current target platform and info message for new target platform
* fix test failing
* remove ItemGroup if node being removed is the only one
* fix for if ItemGroup has elements with different tag names
* fix for ItemGroups not at the end of the sqlproj
* add delete for db references
* fix failing tests
* add test
* cleanup
* Addressing comments and fixing a string
* remove ItemGroup if node being removed is the only one
* fix for if ItemGroup has elements with different tag names
* fix for ItemGroups not at the end of the sqlproj
* show default values in text boxes
* add sqlcmd formatting
* add tests
* Add some sqlcmd variable name validation
* Addressing comments
* fixes after merge
* fix test
* don't localize OtherServer
* fix for windows
* one more fix
* fix test
* add projects to add database reference dialog
* able to add project references
* check for circular dependency
* only allow adding reference to project in the same workspace
* fix location dropdown when project reference is enabled
* add tests
* more tests
* cleanup
* fix flakey test
* addressing comments
* add suppress missing dependencies checkbox
* add example usage
* set font-styles
* Fix typo
* allow example usage to be selectable
* don't localize example usage
* move example usage strings to a new section
* Initial changes for adding pre/post deploy script in project
* Right click > Add pre/post deploy script
* Print script files in tree
* Add new pre-post deploy items with their own tags and additional ones with None
* Add tests
* Fix error due to merge conflicts
* Addressed comments and fixed tests.
* Fix code scan error
* Addressed comments
* Update Open Project to display pre-post deployment scripts in the project tree
* Address comments
* Fixed tests
* Update nonDeployScripts to noneDeployScripts and throw a warning message instead of informational message for prePostDeployCount
* first changes for showing project references
* add tests
* fix interface and formatting
* add try so that project still gets loaded even if dependency project loading fails
* use instanceof
* add circular reference error
* Add images
* Splitting to light and dark mode icons
* Hooks up icons to treeItems
* updating package.json with new icon and vbump
* move icon loader before tree view created
* initial changes for reading connection from profile
* connection string can now be read from publish.xml
* fix build errors and update test
* move publish profile tests to their own file
* cleanup
* update message
* fix string
* remove apiWrapper
* Slight import refactor
* Test updates
* Adds test, sets script name for File extractType to projectName
* generating correct folder list
* Correctly setting serverId for Imports started from Object Explorer
* Adding regression test
* uncommented test check
* show error when trying to create file or folder that already exists
* add test for file that already exists
* add folder tests
* fix error messages
* hide Properties folder
* update comment
* don't allow adding duplicate database references
* add test
* addressing comments
* remove XML checking from databaseReferenceExists()
* change to get
* fix for windows path to show in project tree across platforms correctly
* allow retrying on project that failed to load
* test for project failed to load case
* Adding platform safe uri for database ref in tree
* load database name from profile.xml
* load sqlcmd variables from profile
* Add warning text
* add tests
* fix file filter for windows
* add comments
* show SQLCMD variables in a table
* reset dialog before testing readPublishProfile callback
* add system database reference for VS
* a few fixes
* update tests
* update tests
* fix tests after merge
* addressing comment to make logic a little more clear
* fix replacing SSDT reference name in tree
* Adding reading SqlCmdVars from project file
* organized apiWrapper, added calls
* Adding test to confirm deployment/script gen profiles with sqlcmd vars
* convert system database references from SSDT
* remove empty ItemGroup if no other database references
* fix baseline files
* also update sqlproj if system database references were added in SSDT since the sqlproj got updated with ADS imports
* undo change
* move updating system db references out of updateProjectForRoundTrip()
* update test to have an already updated system db ref
* add clean target after merge from master
* add await
* addressing comments
