[DO NOT REVIEW] Test pipeline for new TSP flow

[VeryEarly]

Description

Mandatory Checklist

• Please choose the target release of Azure PowerShell. (⚠️Target release is a different concept from API readiness. Please click below links for details.)

  • General release
  • Public preview
  • Private preview
  • Engineering build
  • No need for a release

• Check this box to confirm: I have read the Submitting Changes section of CONTRIBUTING.md and reviewed the following information:

• SHOULD update ChangeLog.md file(s) appropriately
  • Update src/{{SERVICE}}/{{SERVICE}}/ChangeLog.md.
    • A snippet outlining the change(s) made in the PR should be written under the ## Upcoming Release header in the past tense.
  • Should not change ChangeLog.md if no new release is required, such as fixing test case only.
• SHOULD regenerate markdown help files if there is cmdlet API change. Instruction
• SHOULD have proper test coverage for changes in pull request.
• SHOULD NOT adjust version of module manually in pull request

[azure-client-tools-bot-prd]

Thanks for your contribution! The pull request validation has started. Please revisit this comment for updated status.

[github-actions]

‼️ DO NOT MERGE THIS PR ‼️
This PR was labeled "Do Not Merge" because it contains code change that cannot be merged. Please contact the reviewer for more information.

[Copilot]

Pull request overview

This pull request introduces a new TypeSpec (TSP) flow for generating Azure PowerShell modules directly from TypeSpec definitions. The PR is marked as "[DO NOT REVIEW]" and labeled as a pipeline test, indicating this is experimental infrastructure work to support TypeSpec-based code generation alongside the existing AutoRest flow.

Key changes:

• Adds a new New-DevTSPModule cmdlet to the AzDev tooling that handles TypeSpec-based module generation, including fetching TSP configurations from remote or local sources, merging configurations, and invoking the TypeSpec compiler
• Integrates the new TSP flow into the existing build pipeline by modifying PrepareAutorestModule.ps1 and BuildScripts.psm1 to detect and use TSP-based generation when tsp-location.yaml is present
• Implements YAML serialization support in YamlHelper.cs to enable configuration file generation

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 27 comments.

Show a summary per file

File	Description
tools/AzDev/src/Typespec/NewTSPModuleCmdlet.cs	New 784-line cmdlet implementing TypeSpec module generation with remote/local TSP fetching, configuration merging, and compilation orchestration
tools/AzDev/src/Typespec/README.md	Comprehensive documentation for the new cmdlet including terminology, prerequisites, usage examples, and migration guidance
tools/AzDev/src/Services/YamlHelper.cs	Implements YAML serialization by adding Serialize method and ISerializer instance
tools/AzDev/AzDev/AzDev.psd1	Exports the new New-DevTSPModule cmdlet from the AzDev module
tools/AzDev/build.ps1	Refactors build script to use Join-Path for cross-platform compatibility and adds clean build logic
tools/BuildScripts/PrepareAutorestModule.ps1	Adds AzDev module build and import logic to make New-DevTSPModule available during module preparation
tools/BuildScripts/BuildScripts.psm1	Integrates TSP flow by calling New-DevTSPModule when tsp-location.yaml is detected instead of AutoRest
src/DependencyMap/DependencyMap/ChangeLog.md	Adds test changelog entry
src/DependencyMap/DependencyMap.Autorest/tspconfig.yaml	New TypeSpec configuration file with PowerShell emitter options and directives for the DependencyMap module
src/DependencyMap/DependencyMap.Autorest/tsp-location.yaml	Updates remote repository reference from Azure to VeryEarly fork with new commit hash
src/DependencyMap/DependencyMap.Autorest/generate-info.json	Updates generation ID to track this new generation

[Copilot]

The script uses Join-Path correctly for cross-platform compatibility, which is good. However, verify that the dotnet publish and Copy-Item commands work correctly on Unix-like systems (Linux, macOS), especially the wildcard pattern in Copy-Item on line 7. Consider testing this script on non-Windows platforms to ensure full compatibility.

Suggested change

	Copy-Item (Join-Path $PSScriptRoot $module "*") $moduleOut -Recurse -Force
	Get-ChildItem -Path (Join-Path $PSScriptRoot $module) | Copy-Item -Destination $moduleOut -Recurse -Force

[Copilot]

The using statement 'NuGet.Protocol.Plugins' is imported but doesn't appear to be used in this file. Consider removing unused imports to keep the code clean and reduce compilation dependencies.

Suggested change

using NuGet.Protocol.Plugins;

[Copilot]

The cmdlet uses Console.WriteLine and Console.Error.WriteLine directly instead of PowerShell's standard WriteVerbose, WriteWarning, or WriteError methods. This bypasses PowerShell's standard stream handling and preference variables. Replace Console.WriteLine with WriteVerbose or WriteInformation, and Console.Error.WriteLine with WriteError for consistency with PowerShell conventions.

Suggested change

	Console.WriteLine(e.Data);
	};
	process.ErrorDataReceived += (sender, e) =>
	{
	if (!string.IsNullOrEmpty(e.Data))
	Console.Error.WriteLine(e.Data);
	{
	// Route standard output through PowerShell Information stream
	WriteInformation(e.Data, new[] { "NewTSPModuleCmdlet" });
	}
	};
	process.ErrorDataReceived += (sender, e) =>
	{
	if (!string.IsNullOrEmpty(e.Data))
	{
	// Route standard error through PowerShell Error stream
	var errorRecord = new ErrorRecord(
	new InvalidOperationException(e.Data),
	"ExternalCommandError",
	ErrorCategory.NotSpecified,
	targetObject: null);
	WriteError(errorRecord);
	}

[Copilot]

The code catches a general Exception and wraps it, but the inner exception message is redundant since it's already included in the outer exception. Consider either removing the inner exception message from the outer message string, or simplifying the error handling to avoid duplicate information in the exception chain.

[Copilot]

The condition checks for string.IsNullOrEmpty(_npmPath) but _npmPath is assigned a value inside the if block. If the FirstOrDefault returns null, _npmPath will be null, and on the next call, the condition will still be true, causing the PATH to be searched again unnecessarily. The logic should check if _npmPath is null or empty after the assignment and only proceed if a valid path was found. Additionally, if no npm directory is found, this should throw an appropriate error rather than continuing with a null _npmPath.

Suggested change

	string commandSuffix = Environment.OSVersion.Platform == PlatformID.Win32NT ? ".cmd":"";
	if ( string.IsNullOrEmpty(_npmPath) || !File.Exists(_npmPath))
	{
	string pathEnv = Environment.GetEnvironmentVariable("PATH") ?? string.Empty;
	string npmPath = pathEnv.Split(Path.PathSeparator).FirstOrDefault(path => path.EndsWith("npm"));
	_npmPath = npmPath;
	}
	string commandPath = Path.Combine(_npmPath, command+commandSuffix);
	if (!File.Exists(commandPath))
	{
	throw new FileNotFoundException($"Command '{command}' not found in system PATH.");
	}
	string commandSuffix = Environment.OSVersion.Platform == PlatformID.Win32NT ? ".cmd" : "";
	if (string.IsNullOrEmpty(_npmPath) || !Directory.Exists(_npmPath))
	{
	string pathEnv = Environment.GetEnvironmentVariable("PATH") ?? string.Empty;
	string npmPath = pathEnv
	.Split(Path.PathSeparator)
	.FirstOrDefault(path => path.EndsWith("npm", StringComparison.OrdinalIgnoreCase) && Directory.Exists(path));
	if (string.IsNullOrEmpty(npmPath))
	{
	throw new DirectoryNotFoundException("Unable to locate an npm directory in the system PATH.");
	}
	_npmPath = npmPath;
	}
	string commandPath = Path.Combine(_npmPath, command + commandSuffix);
	if (!File.Exists(commandPath))
	{
	throw new FileNotFoundException($"Command '{command}' not found in system PATH.");
	}

[Copilot]

The sample URLs point to personal forks (VeryEarly) rather than official Azure repositories. For documentation intended for broader use, consider using examples from the official Azure repositories, or clearly label these as example/test URLs that users should replace with their own.

Suggested change

	- **Sample remote TSP Location without PowerShell emitter option**: https://github.com/VeryEarly/azure-rest-api-specs/blob/yabo/test-tspconfig/specification/azuredependencymap/DependencyMap.Management/tspconfig.yaml
	- **Sample AsPSConfig with PowerShell emitter option**: https://github.com/VeryEarly/azure-powershell/blob/yabo/tsp-client-bugbash/src/DependencyMap/DependencyMap.Autorest/tspconfig.yaml
	- **Sample remote TSP Location without PowerShell emitter option**: https://github.com/Azure/azure-rest-api-specs/blob/29e9e3ca1a1bccba66a6cf092dbc317c639989b1/specification/azuredependencymap/DependencyMap.Management/tspconfig.yaml
	- **Sample AzPSConfig with PowerShell emitter option**: https://github.com/Azure/azure-powershell/blob/main/path-to-your-module/tspconfig.yaml (replace with the path to your own tspconfig.yaml)

[Copilot]

The ChangeLog entry "Pipeline test" is not user-facing and doesn't follow the required format. According to the ChangeLog.md guidelines, entries should describe changes from the user's perspective and explain what changed and how it affects their usage. This appears to be a test entry that should be removed or replaced with a proper description of the changes being introduced.

[Copilot]

Similar to other Console.WriteLine usage in this file, use PowerShell's WriteVerbose or WriteInformation methods instead for consistency with PowerShell cmdlet conventions.

[Copilot]

The variable name 'commandSuffix' could be more descriptive. Consider renaming it to 'commandFileExtension' or 'platformCommandExtension' to make it clearer that this represents the file extension for commands on different platforms.

Suggested change

	string commandSuffix = Environment.OSVersion.Platform == PlatformID.Win32NT ? ".cmd":"";
	if ( string.IsNullOrEmpty(_npmPath) || !File.Exists(_npmPath))
	{
	string pathEnv = Environment.GetEnvironmentVariable("PATH") ?? string.Empty;
	string npmPath = pathEnv.Split(Path.PathSeparator).FirstOrDefault(path => path.EndsWith("npm"));
	_npmPath = npmPath;
	}
	string commandPath = Path.Combine(_npmPath, command+commandSuffix);
	string commandFileExtension = Environment.OSVersion.Platform == PlatformID.Win32NT ? ".cmd" : "";
	if (string.IsNullOrEmpty(_npmPath) || !File.Exists(_npmPath))
	{
	string pathEnv = Environment.GetEnvironmentVariable("PATH") ?? string.Empty;
	string npmPath = pathEnv.Split(Path.PathSeparator).FirstOrDefault(path => path.EndsWith("npm"));
	_npmPath = npmPath;
	}
	string commandPath = Path.Combine(_npmPath, command + commandFileExtension);

[Copilot]

The new call to New-DevTSPModule in the pipeline context processes tsp-location.yaml, whose directory field can point to any local file path and is fully attacker-controlled via the PR. Inside New-DevTSPModule, this path is read and, if the file is not a valid TSP YAML dictionary, its entire contents are concatenated into an exception message (Invalid parent TSP config: ...), which will be written into the build logs, allowing a malicious contributor to exfiltrate arbitrary readable files (including potential secrets) from the build agent. To mitigate this, strictly validate or restrict the allowed paths from tsp-location.yaml (e.g., only under the repo root), and avoid echoing raw file contents in error messages for invalid configs.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 21 comments.

[Copilot]

Typo in URL - "typespec compiler installed?" should be phrased as a statement rather than a question. Change to "TypeSpec compiler must be installed" for clarity.

[Copilot]

The section references "When last time generated from remote" which is grammatically incorrect. It should read "When last generation was from remote" to match standard English grammar and maintain consistency with line 15 which uses similar phrasing.

[Copilot]

The git commands use --filter=tree:0 for partial clones, which is a relatively new Git feature (Git 2.19+). Consider adding validation to check the Git version or document the minimum required Git version in the README.md to avoid confusing errors when users have older Git installations.

[Copilot]

Performance concern: The git sparse-checkout commands on lines 380-381 are redundant. The "git sparse-checkout set" command already configures sparse checkout, so "git sparse-checkout add" immediately after will just re-add the same path. Remove line 381 as it's unnecessary.

[Copilot]

The platform check is incorrect. PlatformID.Win32NT is obsolete and doesn't reliably detect Windows on modern .NET. Use RuntimeInformation.IsOSPlatform(OSPlatform.Windows) from System.Runtime.InteropServices for cross-platform detection.

[Copilot]

The comment indicates this is not good practice but provides no context on why or what the alternative should be. Comments like this should either explain the issue and planned resolution, or link to a tracking issue. Better yet, implement the proper solution instead of leaving a note about bad practice.

[Copilot]

The HttpClient is static and shared across all cmdlet invocations, but the timeout is set per request using CancellationTokenSource. This is good, but consider that HttpClient timeout should be set to Timeout.InfiniteTimeSpan when using per-request timeouts to avoid conflicts. Also, the static HttpClient should ideally be disposed when the module unloads, though this is a minor concern for long-running tools.

[Copilot]

Empty catch block silently suppresses all exceptions when loading context. At minimum, use WriteVerbose or WriteDebug to log that context loading failed and why. This makes debugging configuration issues much harder.

[Copilot]

The collection expression syntax with square brackets for array literals requires C# 12. Consider using traditional array initialization "new string[] { "tsp-output", "node_modules" }" for better compatibility, or ensure the project explicitly targets C# 12+.

[Copilot]

The method MergeTSPConfig returns type 'object' but always returns an IDictionary. This should return IDictionary<object, object> for type safety and to match the actual usage pattern where the result is immediately cast to Dictionary<object, object>.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 9 comments.

[Copilot]

The catch block that loads context catches all exceptions but doesn't log or handle them specifically. While the code continues with context = null, silently swallowing exceptions could hide configuration issues. Consider logging a warning when context loading fails so users are aware of the issue.

Suggested change

	catch
	{
	catch (Exception ex)
	{
	WriteWarning($"Failed to load development context. Continuing without context. Details: {ex.Message}");

[Copilot]

The string interpolation in UriBuilder uses null-coalescing operators but the fallback values may not make sense. For example, RemoteCommit defaults to "main" which is a branch name, not a commit hash. This could cause issues when the URI is used later. Consider validating that required values are present before constructing the URI, or document why "main" is an acceptable fallback for a commit parameter.

Suggested change

	Path = $"{RemoteRepositoryName ?? "Azure/azure-rest-api-specs"}/{RemoteCommit ?? "main"}/{RemoteDirectory ?? ""}/tspconfig.yaml"
	Path = $"{RemoteRepositoryName}/{RemoteCommit}/{RemoteDirectory}/tspconfig.yaml"

[Copilot]

Using Console.WriteLine and Console.Error.WriteLine in a PowerShell cmdlet is not best practice. These write directly to stdout/stderr and bypass PowerShell's output streams. Use WriteObject, WriteVerbose, WriteWarning, WriteError, or WriteDebug instead, which properly integrate with PowerShell's output system and allow users to redirect or suppress output as needed.

[Copilot]

The variable naming is inconsistent with PowerShell conventions. PowerShell variables typically use PascalCase for clarity, but some variables here use lowercase (like $module, $artifacts, $moduleOut). While not incorrect, using consistent PascalCase (like $Module, $Artifacts, $ModuleOut) would be more conventional and improve readability.

[Copilot]

The file check uses File.Exists on tspLocationPWDContent["directory"] without validating that the dictionary contains that key or that the value is a non-null string. This could throw a KeyNotFoundException or NullReferenceException. Add proper validation before accessing dictionary values.

Suggested change

	//if tspconfig emitted previously was from local, only record the absolute directory name
	if (File.Exists((string)tspLocationPWDContent["directory"]) && string.IsNullOrEmpty((string)tspLocationPWDContent["repo"]) && string.IsNullOrEmpty((string)tspLocationPWDContent["commit"]))
	{
	if (remoteInfo != (null, null, null, null))
	{
	throw new ArgumentException("Emitted by local TSP last time, cannot update by remote info. Please provide remote `-TSPLocation`.");
	}
	return (string)tspLocationPWDContent["directory"];
	}
	(string RemoteDirectory, string RemoteCommit, string RemoteRepositoryName, string RemoteForkName) = remoteInfo;
	//otherwise it was from remote, construct its url
	string repo = !string.IsNullOrEmpty(RemoteForkName) ? $"{RemoteForkName}/azure-rest-api-specs" : (!string.IsNullOrEmpty(RemoteRepositoryName) ? RemoteRepositoryName : (string)tspLocationPWDContent["repo"]);
	string commit = !string.IsNullOrEmpty(RemoteCommit) ? RemoteCommit : (string)tspLocationPWDContent["commit"];
	string directory = !string.IsNullOrEmpty(RemoteDirectory) ? RemoteDirectory : (string)tspLocationPWDContent["directory"];
	// Safely extract expected values from the deserialized content
	string directoryFromFile = null;
	if (tspLocationPWDContent != null &&
	tspLocationPWDContent.TryGetValue("directory", out object directoryObj) &&
	directoryObj is string directoryStr)
	{
	directoryFromFile = directoryStr;
	}
	string repoFromFile = null;
	if (tspLocationPWDContent != null &&
	tspLocationPWDContent.TryGetValue("repo", out object repoObj) &&
	repoObj is string repoStr)
	{
	repoFromFile = repoStr;
	}
	string commitFromFile = null;
	if (tspLocationPWDContent != null &&
	tspLocationPWDContent.TryGetValue("commit", out object commitObj) &&
	commitObj is string commitStr)
	{
	commitFromFile = commitStr;
	}
	// if tspconfig emitted previously was from local, only record the absolute directory name
	if (!string.IsNullOrEmpty(directoryFromFile) &&
	File.Exists(directoryFromFile) &&
	string.IsNullOrEmpty(repoFromFile) &&
	string.IsNullOrEmpty(commitFromFile))
	{
	if (remoteInfo != (null, null, null, null))
	{
	throw new ArgumentException("Emitted by local TSP last time, cannot update by remote info. Please provide remote `-TSPLocation`.");
	}
	return directoryFromFile;
	}
	(string RemoteDirectory, string RemoteCommit, string RemoteRepositoryName, string RemoteForkName) = remoteInfo;
	// otherwise it was from remote, construct its url
	string repo = !string.IsNullOrEmpty(RemoteForkName)
	? $"{RemoteForkName}/azure-rest-api-specs"
	: (!string.IsNullOrEmpty(RemoteRepositoryName)
	? RemoteRepositoryName
	: repoFromFile);
	string commit = !string.IsNullOrEmpty(RemoteCommit) ? RemoteCommit : commitFromFile;
	string directory = !string.IsNullOrEmpty(RemoteDirectory) ? RemoteDirectory : directoryFromFile;
	if (string.IsNullOrEmpty(repo) || string.IsNullOrEmpty(commit) || string.IsNullOrEmpty(directory))
	{
	throw new ArgumentException("The TSP location file does not contain valid 'repo', 'commit', or 'directory' information and no remote override was provided.");
	}

[Copilot]

The hardcoded command suffix logic only handles Windows. On Windows, the suffix should be ".cmd", but on Unix-like systems it should be empty. However, the condition uses Platform == Win32NT which won't properly distinguish when to use no suffix. The ternary operator should use an empty string as the true branch result for Windows and empty for non-Windows, but the logic appears inverted. On Windows, npm commands are typically "npm.cmd", "tsp.cmd", etc., while on Unix they have no extension.

[Copilot]

The tuple equality check using "!= (null, null, null, null)" is unreliable in C#. Tuple equality compares each element, but this pattern doesn't clearly express the intent. Consider using a more explicit check like checking if any of the tuple elements is not null: "remoteInfo.Item1 != null || remoteInfo.Item2 != null || remoteInfo.Item3 != null || remoteInfo.Item4 != null".

Suggested change

	if (remoteInfo != (null, null, null, null))
	if (remoteInfo.Item1 != null || remoteInfo.Item2 != null || remoteInfo.Item3 != null || remoteInfo.Item4 != null)

[Copilot]

The HTTP and HTTPS validation comment says "Ensure HTTPS for security" but the code actually allows both HTTP and HTTPS. If security is a concern, the code should reject HTTP URLs and only allow HTTPS. Either remove the misleading comment or enforce HTTPS-only.

Suggested change

	// Ensure HTTPS for security
	// Ensure the URI uses a supported scheme (HTTP or HTTPS)

[Copilot]

The PrepareAutorestModule.ps1 script doesn't include a shebang line for cross-platform compatibility. Add "#!/usr/bin/env pwsh" at the top to ensure the script can be executed directly on Unix-like systems.