Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The --msbuild-parameters flag is performing incorrectly when spawned by child_process #358

Open
1 task
Levelleor opened this issue Dec 24, 2024 · 3 comments
Assignees
Labels
bug This issue is a bug. module/cli-ext p2 This is a standard priority issue queued xs Effort estimation: tiny

Comments

@Levelleor
Copy link

Describe the bug

I am executing the following command via a JS Github Action:

dotnet lambda package --disable-interactive True --project-location /sources/dotnet/lambda/src --output-package publish/output.zip --function-runtime dotnet8 --function-architecture arm64 --msbuild-parameters "--no-restore --no-build"

I need the ability to pass multiple arguments via msbuild-parameters flag and thus I've encapsulated them into double quotes as requested by the help manual. Both --no-restore and --no-build here are appended as a single command-line argument and are not parsed by dotnet correctly, instead being set on MSBuild call and failing the execution.

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

I expect this to be called:

dotnet publish "/sources/dotnet/lambda/src" --output "/sources/dotnet/lambda/src/bin/Release/net8.0/publish" --configuration "Release" --framework "net8.0" --no-restore --no-build /p:GenerateRuntimeConfigurationFiles=true --runtime linux-arm64 --self-contained False 

Current Behavior

I get this result instead:

dotnet publish "/sources/dotnet/lambda/src" --output "/sources/dotnet/lambda/src/bin/Release/net8.0/publish" --configuration "Release" --framework "net8.0" "--no-restore --no-build" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-arm64 --self-contained False 

.NET isn't parsing --no-restore and --no-build because it didn't recognize them and thus it passed these down to msbuild it seems like.

... publish: MSBUILD : error MSB1001: Unknown switch.
... publish:     Full command line: 'MSBuild.dll -maxcpucount -verbosity:m -nologo -target:Restore --property:_IsPublishing=true --property:GenerateRuntimeConfigurationFiles=true -property:PublishDir=/sources/dotnet/lambda/src/bin/Release/net8.0/publish -property:_CommandLineDefinedOutputPath=true -property:SelfContained=False -property:_CommandLineDefinedSelfContained=true -property:RuntimeIdentifier=linux-arm64 -property:_CommandLineDefinedRuntimeIdentifier=true -property:Configuration=Release -property:DOTNET_CLI_DISABLE_PUBLISH_AND_PACK_RELEASE=true /sources/dotnet/lambda/src --no-restore --no-build -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/.dotnet/sdk/8.0.404/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,/.dotnet/sdk/8.0.404/dotnet.dll'
... publish:   Switches appended by response files:
... publish: Switch: --no-restore --no-build
... publish: For switch syntax, type "MSBuild -help"
ERROR: The dotnet publish command return unsuccessful error code

Reproduction Steps

I already tried the following:

  1. Splitting every publish argument I want applied into a separate argument prepended with msbuild-properties:
    args.push(...this.inputs.publishArgs.split(/\s+/).flatMap(a => [--msbuild-parameters, a]));
  2. Dropping all arguments without any quotation marks after msbuild-parameters:
    args.push(--msbuild-parameters, ...this.inputs.publishArgs.split(/\s+/));
  3. Encapsulating parameters into single and double quotes in order to group them:
    args.push(--msbuild-parameters, '${this.inputs.publishArgs}');
  4. Setting parameters via an equal sign:
    args.push(--msbuild-parameters="${this.inputs.publishArgs}");

None of these provide the desired results. Some of these result into missing arguments (effectively just ignoring them), others fail with the error of MSBuild not recognizing given options.

Later after these args are set they are fed into an actions/exec package which then spawns them in a child_process.
await exec(cmd, args, opt);

Possible Solution

No response

Additional Information/Context

No response

Targeted .NET platform

.NET 8

CLI extension version

Package Id Version Commands

amazon.lambda.tools 5.12.3 dotnet-lambda

Environment details (OS name and version, etc.)

RHEL9

@Levelleor Levelleor added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Dec 24, 2024
@ashishdhingra ashishdhingra self-assigned this Dec 24, 2024
@ashishdhingra ashishdhingra added needs-reproduction This issue needs reproduction. module/cli-ext p2 This is a standard priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Dec 24, 2024
@ashishdhingra
Copy link
Contributor

ashishdhingra commented Dec 26, 2024

Not reproducible.

Executed locally
Executed dotnet lambda package --disable-interactive true --project-location /Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/ --output-package publish/output.zip --function-runtime dotnet8 --function-architecture arm64 --msbuild-parameters "--no-restore --no-build" locally:

Amazon Lambda Tools for .NET Core applications (5.12.3)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet
	
Executing publish command
Deleted previous publish folder
... invoking 'dotnet publish', working folder '/Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/bin/Release/net8.0/publish'
... dotnet publish "/Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test" --output "/Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/bin/Release/net8.0/publish" --configuration "Release" --framework "net8.0" --no-restore --no-build /p:GenerateRuntimeConfigurationFiles=true --runtime linux-arm64 --self-contained False 
... publish:   LambdaNet8Test -> /Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/bin/Release/net8.0/publish/
Changed permissions on published file (chmod +rx LambdaNet8Test.pdb).
Changed permissions on published file (chmod +rx LambdaNet8Test.runtimeconfig.json).
Changed permissions on published file (chmod +rx LambdaNet8Test.deps.json).
Changed permissions on published file (chmod +rx LambdaNet8Test.dll).
Changed permissions on published file (chmod +rx Amazon.Lambda.Serialization.SystemTextJson.dll).
Changed permissions on published file (chmod +rx Amazon.Lambda.Core.dll).
Zipping publish folder /Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/bin/Release/net8.0/publish to /Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/publish/output.zip
... zipping:   adding: LambdaNet8Test.pdb (deflated 42%)
... zipping:   adding: LambdaNet8Test.runtimeconfig.json (deflated 38%)
... zipping:   adding: LambdaNet8Test.deps.json (deflated 68%)
... zipping:   adding: LambdaNet8Test.dll (deflated 61%)
... zipping:   adding: Amazon.Lambda.Serialization.SystemTextJson.dll (deflated 56%)
... zipping:   adding: Amazon.Lambda.Core.dll (deflated 60%)
Created publish archive (/Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/publish/output.zip).
Lambda project successfully packaged: /Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/publish/output.zip

It passes --no-restore --no-build parameters properly in the command dotnet publish "/Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test" --output "/Users/TestUser/dev/repros/LambdaNet8Test/src/LambdaNet8Test/bin/Release/net8.0/publish" --configuration "Release" --framework "net8.0" --no-restore --no-build /p:GenerateRuntimeConfigurationFiles=true --runtime linux-arm64 --self-contained False.

Executed as part of GitHub workflow action
Developed below workflow:

name: AWS .NET Extensions for CLI Lambda tools package test

on:
  push:
    branches: [ main ]
  workflow_dispatch:

concurrency: 
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build:
    name: Build
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: '8.0.x'
      
      - name: Setup AWS .NET Extensions for CLI Lambda tools
        run: dotnet tool install -g Amazon.Lambda.Tools
      
      - name: Build project
        run: dotnet build ./LambdaNet8Test -c Release

      - name: Package using dotnet lambda
        run: dotnet lambda package --disable-interactive true --project-location ./LambdaNet8Test --output-package publish/output.zip --function-runtime dotnet8 --function-architecture arm64

      - name: Package using dotnet lambda (with "--no-restore --no-build")
        run: dotnet lambda package --disable-interactive true --project-location ./LambdaNet8Test --output-package publish/output.zip --function-runtime dotnet8 --function-architecture arm64 --msbuild-parameters "--no-restore --no-build"

Works fine with below output for Package using dotnet lambda (with "--no-restore --no-build") step:

Run dotnet lambda package --disable-interactive true --project-location ./LambdaNet8Test --output-package publish/output.zip --function-runtime dotnet8 --function-architecture arm64 --msbuild-parameters "--no-restore --no-build"
  
Amazon Lambda Tools for .NET Core applications (5.12.3)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet
	
Executing publish command
Deleted previous publish folder
... invoking 'dotnet publish', working folder '/home/runner/work/testrepo/testrepo/./LambdaNet8Test/bin/Release/net8.0/publish'
... dotnet publish "/home/runner/work/testrepo/testrepo/./LambdaNet8Test" --output "/home/runner/work/testrepo/testrepo/./LambdaNet8Test/bin/Release/net8.0/publish" --configuration "Release" --framework "net8.0" --no-restore --no-build /p:GenerateRuntimeConfigurationFiles=true --runtime linux-arm[6](https://github.com/ashishdhingra/testrepo/actions/runs/12507961922/job/34895351932#step:7:6)4 --self-contained False 
... publish:   LambdaNet8Test -> /home/runner/work/testrepo/testrepo/LambdaNet8Test/bin/Release/net8.0/publish/
Changed permissions on published file (chmod +rx LambdaNet8Test.runtimeconfig.json).
Changed permissions on published file (chmod +rx Amazon.Lambda.Serialization.SystemTextJson.dll).
Changed permissions on published file (chmod +rx LambdaNet8Test.pdb).
Changed permissions on published file (chmod +rx LambdaNet8Test.dll).
Changed permissions on published file (chmod +rx LambdaNet8Test.deps.json).
Changed permissions on published file (chmod +rx Amazon.Lambda.Core.dll).
Zipping publish folder /home/runner/work/testrepo/testrepo/./LambdaNet8Test/bin/Release/net8.0/publish to /home/runner/work/testrepo/testrepo/publish/output.zip
... zipping:   adding: LambdaNet8Test.runtimeconfig.json (deflated 38%)
... zipping:   adding: Amazon.Lambda.Serialization.SystemTextJson.dll (deflated 5[6](https://github.com/ashishdhingra/testrepo/actions/runs/12507961922/job/34895351932#step:7:7)%)
... zipping:   adding: LambdaNet8Test.pdb (deflated 40%)
... zipping:   adding: LambdaNet[8](https://github.com/ashishdhingra/testrepo/actions/runs/12507961922/job/34895351932#step:7:9)Test.dll (deflated 60%)
... zipping:   adding: LambdaNet8Test.deps.json (deflated 68%)
... zipping:   adding: Amazon.Lambda.Core.dll (deflated 60%)
Created publish archive (/home/runner/work/testrepo/testrepo/publish/output.zip).
Lambda project successfully packaged: /home/runner/work/testrepo/testrepo/publish/output.zip

Notice that parameters --no-restore --no-build are passed correctly to dotnet publish command.

@Levelleor Please share the minimal repro setup for troubleshooting the issue.

Thanks,
Ashish

@ashishdhingra ashishdhingra added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-reproduction This issue needs reproduction. labels Dec 26, 2024
@Levelleor
Copy link
Author

Levelleor commented Dec 27, 2024

Sure, thank you, @ashishdhingra. The issue is not github actions themselves but how arguments are handled by the aws tool when spawning commands in a child process using javascript, I am assuming in Python I'll be getting the same issue, but I haven't tested that.

A minimal example looks like this:
https://github.com/Levelleor/test-net-lambda-tool/blob/main/.github/workflows/ci.yaml
Here is an unsuccessful execution:
https://github.com/Levelleor/test-net-lambda-tool/actions/runs/12511014146/job/34902532872#step:5:30
It tries to run this command:
/usr/share/dotnet/dotnet lambda package --output-package package.zip --msbuild-parameters "--no-restore --no-build"
But fails with this error:

 /usr/share/dotnet/dotnet lambda package --output-package package.zip --msbuild-parameters "--no-restore --no-build"
Amazon Lambda Tools for .NET Core applications (5.12.3)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet
	
Executing publish command
... invoking 'dotnet publish', working folder '/home/runner/work/test-net-lambda-tool/test-net-lambda-tool/bin/Release/net8.0/publish'
... dotnet publish "/home/runner/work/test-net-lambda-tool/test-net-lambda-tool" --output "/home/runner/work/test-net-lambda-tool/test-net-lambda-tool/bin/Release/net8.0/publish" --configuration "Release" --framework "net8.0" "--no-restore --no-build" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-x64 --self-contained False 
... publish: MSBUILD : error MSB1001: Unknown switch.
... publish:     Full command line: '/usr/share/dotnet/sdk/8.0.404/MSBuild.dll -maxcpucount -verbosity:m -nologo -target:Restore --property:_IsPublishing=true --property:GenerateRuntimeConfigurationFiles=true -property:PublishDir=/home/runner/work/test-net-lambda-tool/test-net-lambda-tool/bin/Release/net8.0/publish -property:_CommandLineDefinedOutputPath=true -property:SelfContained=False -property:_CommandLineDefinedSelfContained=true -property:RuntimeIdentifier=linux-x64 -property:_CommandLineDefinedRuntimeIdentifier=true -property:Configuration=Release -property:DOTNET_CLI_DISABLE_PUBLISH_AND_PACK_RELEASE=true /home/runner/work/test-net-lambda-tool/test-net-lambda-tool --no-restore --no-build -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/usr/share/dotnet/sdk/8.0.404/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,/usr/share/dotnet/sdk/8.0.404/dotnet.dll'
... publish:   Switches appended by response files:
... publish: Switch: --no-restore --no-build
... publish: For switch syntax, type "MSBuild -help"
ERROR: The dotnet publish command return unsuccessful error code
Error: The process '/usr/share/dotnet/dotnet' failed with exit code 255

I included absolute bare minimum: some basic code and absolutely no dependencies. I install .NET lambda tool and run it with msbuild parameters set.

Please let me know if there's anything else needed. I'd love to see a workaround to get this working meanwhile! Thanks!

@ashishdhingra
Copy link
Contributor

@Levelleor Thanks for the repro steps. Issue is reproducible. Looks like ToolRunner used by exec.exec() is somehow passing quotes along with MSBuild parameters when executing dotnet command. .NET process somehow doesn't remove the quotes or .NET Lambda tooling doesn't handle it properly. I would review this with team.
CC @normj

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. module/cli-ext p2 This is a standard priority issue queued xs Effort estimation: tiny
Projects
None yet
Development

No branches or pull requests

2 participants