Squashed commit of the following:

commit 14b5e2f
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 20 14:55:40 2024 -0400

    Fix Aspire.Hosting.Testing.Tests.TestingFactoryTests.HttpClientGetTest by using httpclient with resilience

commit 251d59a
Merge: ac52e88 bae6c46
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 20 14:14:26 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

commit ac52e88
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 20 14:11:27 2024 -0400

    address review feedback from @ eerhardt

commit b1b0682
Author: Mitch Denny <[email protected]>
Date:   Wed Jun 19 22:36:45 2024 -0700

    Update container logs available condition to allow streaming build logs. (dotnet#4601)

commit f9e2eba
Author: David Negstad <[email protected]>
Date:   Wed Jun 19 17:17:55 2024 -0700

    Update additional DCP resource schema (dotnet#4600)

commit 023fd5c
Author: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Date:   Wed Jun 19 21:57:14 2024 +0000

    Update dependencies from https://github.com/microsoft/usvc-apiserver build 0.5.3 (dotnet#4598)

    [main] Update dependencies from microsoft/usvc-apiserver

commit fd2474e
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 20 14:12:39 2024 -0400

    Update tests/TestingAppHost1/TestingAppHost1.AppHost/TestingAppHost1.AppHost.csproj

    Co-authored-by: Eric Erhardt <[email protected]>

commit fe32fd3
Author: Ankit Jain <[email protected]>
Date:   Wed Jun 19 20:43:42 2024 -0400

    Use `b.AddStandardResilienceHandler`

commit c5f7f3e
Author: Ankit Jain <[email protected]>
Date:   Wed Jun 19 18:26:28 2024 -0400

    Add resilience handlers in the test itself

commit fb788aa
Author: Ankit Jain <[email protected]>
Date:   Wed Jun 19 17:16:23 2024 -0400

    Disable failing test `ProxylessAndProxiedEndpointBothWorkOnSameResource`

    Issue: dotnet#4599

commit fa3bce0
Merge: c0743ee 4e06e32
Author: Ankit Jain <[email protected]>
Date:   Wed Jun 19 17:12:14 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

commit c0743ee
Merge: 5e28b57 faf26a8
Author: Ankit Jain <[email protected]>
Date:   Wed Jun 19 16:12:32 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

commit 5e28b57
Merge: abad241 01f7399
Author: Ankit Jain <[email protected]>
Date:   Tue Jun 18 15:40:52 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

commit abad241
Merge: 4038329 c3d1e74
Author: Ankit Jain <[email protected]>
Date:   Tue Jun 18 13:11:13 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

commit 4038329
Merge: 4335fa0 c294706
Author: Ankit Jain <[email protected]>
Date:   Mon Jun 17 18:34:51 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

commit 4335fa0
Author: Ankit Jain <[email protected]>
Date:   Mon Jun 17 18:16:48 2024 -0400

    DistributedApplicationTests.cs: Bump timeout for docker based tests from 10secs for 1min

commit b32414e
Merge: 0742f8a 8e311f3
Author: Ankit Jain <[email protected]>
Date:   Mon Jun 17 15:05:08 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

commit 0742f8a
Merge: aa72d39 d447bda
Author: Ankit Jain <[email protected]>
Date:   Sat Jun 15 04:23:59 2024 -0400

    Merge remote-tracking branch 'origin/main' into hosting-testing-build

    # Conflicts:
    #	tests/Aspire.Components.Common.Tests/RequiresDockerAttribute.cs
    #	tests/Aspire.Hosting.Tests/Schema/SchemaTests.cs

commit aa72d39
Author: Ankit Jain <[email protected]>
Date:   Fri Jun 14 23:19:13 2024 -0400

    Remove SkipOnHelixTheory, and SkipOnHelixFact attributes

commit d2d7a18
Author: Ankit Jain <[email protected]>
Date:   Fri Jun 14 19:41:00 2024 -0400

    cleanup

commit 36654c0
Author: Ankit Jain <[email protected]>
Date:   Fri Jun 14 19:03:42 2024 -0400

    cleanup

commit 6fb3c82
Author: Ankit Jain <[email protected]>
Date:   Fri Jun 14 19:02:42 2024 -0400

    TestingAppHost1.AppHost: add default resilience handler

commit 322b8e1
Author: Ankit Jain <[email protected]>
Date:   Fri Jun 14 18:38:59 2024 -0400

    Add resilience handler to httpclient in Hosting.Testing tests

commit 61fd312
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 22:43:50 2024 -0400

    fix typo

commit 08a8299
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 22:43:31 2024 -0400

    Add RequiresTools attributes, and use for the node/npm tests

commit f4c4a77
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 21:54:45 2024 -0400

    Disable node/npm tests on CI as the binaries are not available

commit daccd46
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 21:26:08 2024 -0400

    cleanup

commit 6913816
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 21:24:49 2024 -0400

    [tests] Remove RequiresDocker from Hosting.Tests that don't need it

commit 106ed06
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 21:05:44 2024 -0400

    [tests] Split npm/nodeapp test

commit 502e36f
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 19:48:47 2024 -0400

    FIXME: add delay in ProxylessAndProxiedEndpointBothWorkOnSameResource to wait for all the endpoints to get allocated

commit a85b721
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 17:44:18 2024 -0400

    [tests] Fix VerifyNodeAppWorks by fixing the constructed path to nodeapp

commit cbc7897
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 17:40:24 2024 -0400

    [tests] FIXME: add delay to wait for the webapp to be ready

commit ca5d62f
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 17:37:25 2024 -0400

    [tests] Throw if application hasn't been initialized, useful for debugging issues

commit 78dc7a2
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 17:36:45 2024 -0400

    [tests] Install devcerts on linux/CI

commit fc2bf4f
Author: Ankit Jain <[email protected]>
Date:   Thu Jun 13 17:36:01 2024 -0400

    [tests] Use RequiresDocker instead of LocalOnlyFact/Theory attributes

    - this allows the tests to run on the build machine

commit 3d8949c
Author: Ankit Jain <[email protected]>
Date:   Wed Jun 12 21:30:41 2024 -0400

    [tests] Use .runsettings when running non-helix tests, so tests skipping can work

    To be specific, `category!=failing` is required.

commit 9c964a6
Author: Ankit Jain <[email protected]>
Date:   Wed Jun 12 21:28:43 2024 -0400

    [tests] RequiresDocker: Update condition to disallow docker based tests

    .. on windows/build-machine on CI.

# Conflicts:
#	eng/pipelines/templates/BuildAndTest.yml
#	tests/Aspire.Hosting.Testing.Tests/TestingBuilderTests.cs
#	tests/Aspire.Hosting.Testing.Tests/TestingFactoryCrashTests.cs
#	tests/Aspire.Hosting.Testing.Tests/TestingFactoryTests.cs
#	tests/Aspire.Hosting.Tests/SlimTestProgramTests.cs

eng/pipelines/templates/BuildAndTest.yml

@@ -31,6 +31,15 @@ steps:
     displayName: Build
 
   - ${{ if ne(parameters.skipTests, 'true') }}:
+    - ${{ if ne(parameters.isWindows, 'true') }}:
+      - script: mkdir ${{ parameters.repoArtifactsPath }}/devcert-scripts &&
+                cd ${{ parameters.repoArtifactsPath }}/devcert-scripts &&
+                wget https://raw.githubusercontent.com/BorisWilhelms/create-dotnet-devcert/main/scripts/ubuntu-create-dotnet-devcert.sh &&
+                wget https://raw.githubusercontent.com/BorisWilhelms/create-dotnet-devcert/main/scripts/common.sh &&
+                chmod +x ubuntu-create-dotnet-devcert.sh &&
+                ./ubuntu-create-dotnet-devcert.sh
+        displayName: Install devcerts
+
     #- script: ${{ parameters.dotnetScript }} dotnet-coverage collect
               #--settings $(Build.SourcesDirectory)/eng/CodeCoverage.config
               #--output ${{ parameters.repoTestResultsPath }}/NonHelix.cobertura.xml

tests/Aspire.Hosting.Tests/Utils/FileUtil.cs → tests/Aspire.Components.Common.Tests/FileUtil.cs

@@ -3,13 +3,13 @@
 
 using System.Diagnostics;
 
-namespace Aspire.Hosting.Utils;
+namespace Aspire.Components.Common.Tests;
 
 internal static class FileUtil
 {
-    public static string FindFullPathFromPath(string command) => FindFullPathFromPath(command, Environment.GetEnvironmentVariable("PATH"), Path.PathSeparator, File.Exists);
+    public static string? FindFullPathFromPath(string command) => FindFullPathFromPath(command, Environment.GetEnvironmentVariable("PATH"), Path.PathSeparator, File.Exists);
 
-    internal static string FindFullPathFromPath(string command, string? pathVariable, char pathSeparator, Func<string, bool> fileExists)
+    internal static string? FindFullPathFromPath(string command, string? pathVariable, char pathSeparator, Func<string, bool> fileExists)
     {
         Debug.Assert(!string.IsNullOrWhiteSpace(command));
 
@@ -28,6 +28,6 @@ internal static string FindFullPathFromPath(string command, string? pathVariable
             }
         }
 
-        return command;
+        return null;
     }
 }

tests/Aspire.Components.Common.Tests/PlatformDetection.cs

@@ -0,0 +1,11 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+namespace Aspire.Components.Common.Tests;
+
+public static class PlatformDetection
+{
+    public static bool IsRunningOnBuildMachine => Environment.GetEnvironmentVariable("BUILD_BUILDID") is not null;
+    public static bool IsRunningOnHelix => Environment.GetEnvironmentVariable("HELIX_WORKITEM_ROOT") is not null;
+    public static bool IsRunningOnCI => IsRunningOnBuildMachine || IsRunningOnHelix;
+}

tests/Aspire.Components.Common.Tests/RequiresDockerAttribute.cs

@@ -22,8 +22,7 @@ public class RequiresDockerAttribute : Attribute, ITraitAttribute
     // - Linux - Local, or CI: always assume that docker is installed
     public static bool IsSupported =>
         !OperatingSystem.IsWindows() ||
-        (Environment.GetEnvironmentVariable("BUILD_BUILDID") is null && // NOT CI - build machine or helix
-            Environment.GetEnvironmentVariable("HELIX_WORKITEM_ROOT") is null);
+        (!PlatformDetection.IsRunningOnBuildMachine && !PlatformDetection.IsRunningOnHelix);
 
     public string? Reason { get; init; }
     public RequiresDockerAttribute(string? reason = null)

tests/Aspire.Components.Common.Tests/RequiresToolsAttribute.cs

@@ -0,0 +1,19 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using Xunit.Sdk;
+
+namespace Aspire.Components.Common.Tests;
+
+[TraitDiscoverer("Aspire.Components.Common.Tests.RequiresToolsDiscoverer", "Aspire.Components.Common.Tests")]
+[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = false)]
+public class RequiresToolsAttribute : Attribute, ITraitAttribute
+{
+    public RequiresToolsAttribute(string[] executablesOnPath)
+    {
+        if (executablesOnPath.Length == 0)
+        {
+            throw new ArgumentException("At least one executable must be provided", nameof(executablesOnPath));
+        }
+    }
+}

tests/Aspire.Components.Common.Tests/RequiresToolsDiscoverer.cs

@@ -0,0 +1,20 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using Microsoft.DotNet.XUnitExtensions;
+using Xunit.Abstractions;
+using Xunit.Sdk;
+
+namespace Aspire.Components.Common.Tests;
+
+public class RequiresToolsDiscoverer : ITraitDiscoverer
+{
+    public IEnumerable<KeyValuePair<string, string>> GetTraits(IAttributeInfo traitAttribute)
+    {
+        string[] executablesOnPath = (string[])traitAttribute.GetConstructorArguments().First();
+        if (!executablesOnPath.All(executable => FileUtil.FindFullPathFromPath(executable) is not null))
+        {
+            yield return new KeyValuePair<string, string>(XunitConstants.Category, "failing");
+        }
+    }
+}

tests/Aspire.Hosting.Testing.Tests/DistributedApplicationFixtureOfT.cs

@@ -3,7 +3,6 @@
 
 using Microsoft.Extensions.Hosting;
 using Xunit;
-using Xunit.Sdk;
 
 namespace Aspire.Hosting.Testing.Tests;
 
@@ -12,10 +11,6 @@ public class DistributedApplicationFixture<TEntryPoint> : DistributedApplication
     public DistributedApplicationFixture()
         : base(typeof(TEntryPoint), [])
     {
-        if (Environment.GetEnvironmentVariable("BUILD_BUILDID") != null)
-        {
-            throw new SkipException("These tests can only run in local environments.");
-        }
     }
 
     protected override void OnBuilderCreating(DistributedApplicationOptions applicationOptions, HostApplicationBuilderSettings hostOptions)

tests/Aspire.Hosting.Testing.Tests/DistributedApplicationHttpClientExtensionsForTests.cs

@@ -0,0 +1,30 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using Microsoft.Extensions.DependencyInjection;
+
+namespace Aspire.Hosting.Testing.Tests;
+
+internal static class DistributedApplicationHttpClientExtensionsForTests
+{
+    private static readonly Lazy<IHttpClientFactory> s_httpClientFactory = new(CreateHttpClientFactoryWithResilience);
+    public static HttpClient CreateHttpClientWithResilience(this DistributedApplication app, string resourceName, string? endpointName = default)
+    {
+        var baseUri = app.GetEndpoint(resourceName, endpointName);
+        var client = s_httpClientFactory.Value.CreateClient();
+        client.BaseAddress = baseUri;
+        return client;
+    }
+
+    private static IHttpClientFactory CreateHttpClientFactoryWithResilience()
+    {
+        var services = new ServiceCollection();
+        services.AddHttpClient()
+            .ConfigureHttpClientDefaults(b =>
+            {
+                b.AddStandardResilienceHandler();
+            });
+
+        return services.BuildServiceProvider().GetRequiredService<IHttpClientFactory>();
+    }
+}

tests/Aspire.Hosting.Testing.Tests/TestingBuilderTests.cs

@@ -2,7 +2,7 @@
 // The .NET Foundation licenses this file to you under the MIT license.
 
 using System.Net.Http.Json;
-// using Aspire.Hosting.Tests.Helpers;
+using Aspire.Components.Common.Tests;
 using Microsoft.Extensions.Configuration;
 using Microsoft.Extensions.DependencyInjection;
 using Microsoft.Extensions.Hosting;
@@ -13,6 +13,7 @@ namespace Aspire.Hosting.Testing.Tests;
 public class TestingBuilderTests
 {
     [Theory]
+    [RequiresDocker]
     [InlineData(false)]
     [InlineData(true)]
     public async Task HasEndPoints(bool genericEntryPoint)
@@ -35,6 +36,7 @@ public async Task HasEndPoints(bool genericEntryPoint)
     }
 
     [Theory]
+    [RequiresDocker]
     [InlineData(false)]
     [InlineData(true)]
     public async Task CanGetResources(bool genericEntryPoint)
@@ -52,6 +54,7 @@ public async Task CanGetResources(bool genericEntryPoint)
     }
 
     [Theory]
+    [RequiresDocker]
     [InlineData(false)]
     [InlineData(true)]
     public async Task HttpClientGetTest(bool genericEntryPoint)
@@ -62,13 +65,14 @@ public async Task HttpClientGetTest(bool genericEntryPoint)
         await using var app = await appHost.BuildAsync();
         await app.StartAsync();
 
-        var httpClient = app.CreateHttpClient("mywebapp1");
+        var httpClient = app.CreateHttpClientWithResilience("mywebapp1");
         var result1 = await httpClient.GetFromJsonAsync<WeatherForecast[]>("/weatherforecast");
         Assert.NotNull(result1);
         Assert.True(result1.Length > 0);
     }
 
     [Theory]
+    [RequiresDocker]
     [InlineData(false)]
     [InlineData(true)]
     public async Task GetHttpClientBeforeStart(bool genericEntryPoint)
@@ -81,6 +85,7 @@ public async Task GetHttpClientBeforeStart(bool genericEntryPoint)
     }
 
     [Theory]
+    [RequiresDocker]
     [InlineData(false)]
     [InlineData(true)]
     public async Task SetsCorrectContentRoot(bool genericEntryPoint)
@@ -95,6 +100,7 @@ public async Task SetsCorrectContentRoot(bool genericEntryPoint)
     }
 
     [Theory]
+    [RequiresDocker]
     [InlineData(false)]
     [InlineData(true)]
     public async Task SelectsFirstLaunchProfile(bool genericEntryPoint)
@@ -117,6 +123,7 @@ public async Task SelectsFirstLaunchProfile(bool genericEntryPoint)
 
     // Tests that DistributedApplicationTestingBuilder throws exceptions at the right times when the app crashes.
     [Theory]
+    [RequiresDocker]
     [InlineData(true, "before-build")]
     [InlineData(true, "after-build")]
     [InlineData(true, "after-start")]

tests/Aspire.Hosting.Testing.Tests/TestingFactoryCrashTests.cs

@@ -1,7 +1,7 @@
 // Licensed to the .NET Foundation under one or more agreements.
 // The .NET Foundation licenses this file to you under the MIT license.
 
-// using Aspire.Hosting.Tests.Helpers;
+using Aspire.Components.Common.Tests;
 using Xunit;
 
 namespace Aspire.Hosting.Testing.Tests;
@@ -10,6 +10,7 @@ namespace Aspire.Hosting.Testing.Tests;
 public class TestingFactoryCrashTests
 {
     [Theory]
+    [RequiresDocker]
     [InlineData("before-build")]
     [InlineData("after-build")]
     [InlineData("after-start")]

tests/Aspire.Hosting.Testing.Tests/TestingFactoryTests.cs

@@ -2,7 +2,7 @@
 // The .NET Foundation licenses this file to you under the MIT license.
 
 using System.Net.Http.Json;
-// using Aspire.Hosting.Tests.Helpers;
+using Aspire.Components.Common.Tests;
 using Microsoft.Extensions.Configuration;
 using Microsoft.Extensions.DependencyInjection;
 using Microsoft.Extensions.Hosting;
@@ -15,6 +15,7 @@ public class TestingFactoryTests(DistributedApplicationFixture<Projects.TestingA
     private readonly DistributedApplication _app = fixture.Application;
 
     [Fact]
+    [RequiresDocker]
     public async Task HasEndPoints()
     {
         // Get an endpoint from a resource
@@ -29,6 +30,7 @@ public async Task HasEndPoints()
     }
 
     [Fact]
+    [RequiresDocker]
     public void CanGetResources()
     {
         var appModel = _app.Services.GetRequiredService<DistributedApplicationModel>();
@@ -37,30 +39,33 @@ public void CanGetResources()
     }
 
     [Fact]
+    [RequiresDocker]
     public async Task HttpClientGetTest()
     {
-        var httpClient = _app.CreateHttpClient("mywebapp1");
+        var httpClient = _app.CreateHttpClientWithResilience("mywebapp1");
         var result1 = await httpClient.GetFromJsonAsync<WeatherForecast[]>("/weatherforecast");
         Assert.NotNull(result1);
         Assert.True(result1.Length > 0);
     }
 
     [Fact]
+    [RequiresDocker]
     public void SetsCorrectContentRoot()
     {
         var appModel = _app.Services.GetRequiredService<IHostEnvironment>();
         Assert.Contains("TestingAppHost1", appModel.ContentRootPath);
     }
 
     [Fact]
+    [RequiresDocker]
     public async Task SelectsFirstLaunchProfile()
     {
         var config = _app.Services.GetRequiredService<IConfiguration>();
         var profileName = config["AppHost:DefaultLaunchProfileName"];
         Assert.Equal("https", profileName);
 
         // Explicitly get the HTTPS endpoint - this is only available on the "https" launch profile.
-        var httpClient = _app.CreateHttpClient("mywebapp1", "https");
+        var httpClient = _app.CreateHttpClientWithResilience("mywebapp1", "https");
         var result = await httpClient.GetFromJsonAsync<WeatherForecast[]>("/weatherforecast");
         Assert.NotNull(result);
         Assert.True(result.Length > 0);

tests/Aspire.Hosting.Tests/Aspire.Hosting.Tests.csproj

@@ -47,6 +47,8 @@
     <ProjectReference Include="..\..\src\Components\Aspire.Npgsql\Aspire.Npgsql.csproj" IsAspireProjectResource="false" />
     <ProjectReference Include="..\..\src\Components\Aspire.Oracle.EntityFrameworkCore\Aspire.Oracle.EntityFrameworkCore.csproj" IsAspireProjectResource="false" />
 
+    <ProjectReference Include="..\Aspire.Components.Common.Tests\Aspire.Components.Common.Tests.csproj" />
+
     <PackageReference Include="Microsoft.Extensions.Http.Resilience" />
     <PackageReference Include="Newtonsoft.Json.Schema" />
 
@@ -63,6 +65,8 @@
     <Compile Include="$(RepoRoot)src\Aspire.Hosting.Valkey\ValkeyContainerImageTags.cs" />
     <Compile Include="$(RepoRoot)src\Aspire.Hosting.Azure.EventHubs\EventHubsEmulatorContainerImageTags.cs" />
 
+    <Compile Include="$(RepoRoot)tests\Aspire.Hosting.Testing.Tests\DistributedApplicationHttpClientExtensionsForTests.cs" />
+
     <Content Include="$(RepoRoot)src\Schema\aspire-8.0.json" Link="Schema\aspire-8.0.json">
       <CopyToOutputDirectory>Always</CopyToOutputDirectory>
     </Content>