Skip to content

test(cli): quarantine ResourceCommand_FailsWhenInteractionServiceIsRequired (#17485)#17486

Open
mitchdenny wants to merge 1 commit into
mainfrom
mitchdenny/quarantine-resource-command-interaction
Open

test(cli): quarantine ResourceCommand_FailsWhenInteractionServiceIsRequired (#17485)#17486
mitchdenny wants to merge 1 commit into
mainfrom
mitchdenny/quarantine-resource-command-interaction

Conversation

@mitchdenny
Copy link
Copy Markdown
Member

Description

Quarantines Aspire.Cli.EndToEnd.Tests.ResourceCommandTests.ResourceCommand_FailsWhenInteractionServiceIsRequired while we investigate the underlying shutdown-timing issue tracked by #17485.

Why now

The test was observed flaking on PR #17452 (run 26425521402) — attempts 1 and 3 failed, attempt 2 passed — even though that PR only touches CLI init/channel/scaffolding code paths and has no plausible causal link to aspire stop, ProcessShutdownService, the auxiliary backchannel, or DCP container shutdown.

What's actually broken

The Hex1b automator times out on WaitUntilText(" stopped successfully."), but the cast file shows aspire stop actively prints:

⠳ Stopping apphost.cs...
❌ Failed to stop apphost.cs.
📄 See logs at /root/.aspire/logs/cli_<id>.log
🔍 See AppHost logs at /root/.aspire/logs/cli_<id>_detach-child_<id>.log

after spinning for ~60s. At the moment of failure, the post-step docker container ls in the same CI job shows the redis container still running:

CONTAINER ID   IMAGE          STATUS         NAMES
5aa9f3938387   redis:latest   Up 7 seconds   cache-xuhbbbzq

ProcessShutdownService.StopProcessesAsync (10s graceful → force-kill → monitor) is exhausting its budget while DCP is still tearing down the container under Docker-in-Docker contention on ubuntu-latest. Issue #17485 captures the suspected root cause and the speculative follow-up of extending the shutdown monitor window for DCP-managed-container scenarios.

Change

Adds [QuarantinedTest("https://github.com/microsoft/aspire/issues/17485")] to the single affected test method via tools/QuarantineTools. No production code changes.

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
    • No

Addresses #17485. Related PR where the failure was observed: #17452.

This E2E test is intermittently failing when 'aspire stop' is invoked
immediately after a deliberately-failing resource command. The CLI prints
'Failed to stop apphost.cs' and the DCP-managed redis container is still
running at the moment of failure, indicating that
ProcessShutdownService.StopProcessesAsync exhausts its graceful + force-kill
+ monitor budget before DCP finishes the container teardown under
Docker-in-Docker contention on the ubuntu-latest runner.

Quarantine while investigation of the underlying shutdown-timing behavior
continues.

- Quarantined: Aspire.Cli.EndToEnd.Tests.ResourceCommandTests.ResourceCommand_FailsWhenInteractionServiceIsRequired
- Issue: #17485

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 26, 2026 03:18
@mitchdenny mitchdenny added flaky-test area-cli quarantined-test Quarantined tests that run only in the Outerloop Tests workflow labels May 26, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 17486

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 17486"

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR quarantines a single flaky Aspire CLI E2E test (ResourceCommand_FailsWhenInteractionServiceIsRequired) by adding a [QuarantinedTest] trait with a link to the tracking issue (#17485). This aligns with the repository’s existing quarantine mechanism (Aspire.TestUtilities.QuarantinedTestAttribute) used to sequester unstable tests without changing production behavior.

Changes:

  • Added [QuarantinedTest("https://github.com/microsoft/aspire/issues/17485")] to ResourceCommand_FailsWhenInteractionServiceIsRequired.
  • Added using Aspire.TestUtilities; to support the quarantine attribute.
Show a summary per file
File Description
tests/Aspire.Cli.EndToEnd.Tests/ResourceCommandTests.cs Marks the flaky E2E test as quarantined via Aspire.TestUtilities.QuarantinedTestAttribute (issue-linked).

Copilot's findings

  • Files reviewed: 1/1 changed files
  • Comments generated: 0

@github-actions
Copy link
Copy Markdown
Contributor

CLI E2E Tests failed — 93 passed, 1 failed, 6 unknown (commit 986cbc0)

Failed Tests

View all recordings
Status Test Recording
AddPackageInteractiveWhileAppHostRunningDetached ▶️ View recording
AddPackageWhileAppHostRunningDetached ▶️ View recording
AgentCommands_AllHelpOutputs_AreCorrect ▶️ View recording
AgentInitCommand_DefaultSelection_InstallsDefaultSkills ▶️ View recording
AgentInitCommand_MigratesDeprecatedConfig ▶️ View recording
AgentMcpListStructuredLogsFromStarterAppCore ▶️ View recording
AllPublishMethodsBuildDockerImages ▶️ View recording
AspireAddPackageVersionToDirectoryPackagesProps ▶️ View recording
AspireInitSingleFileAppHostRunsViaDotnetRunAppHost ▶️ View recording
AspireInitWithExistingAppHostDirRecreatesMissingNuGetConfigAndPreservesFiles ▶️ View recording
AspireInitWithSolutionFileGeneratesAppHostThatBuildsAgainstChannelHive ▶️ View recording
AspireStartUpdatesStaleTypeScriptAppHostPath ▶️ View recording
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps ▶️ View recording
AspireUpdateRemovesOrphanAppHostPackageVersionWhenSdkAlreadyCurrent ▶️ View recording
Banner_DisplayedOnFirstRun ▶️ View recording
Banner_DisplayedWithExplicitFlag ▶️ View recording
Banner_NotDisplayedWithNoLogoFlag ▶️ View recording
CertificatesClean_RemovesCertificates ▶️ View recording
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate ▶️ View recording
CertificatesTrust_WithUntrustedCert_TrustsCertificate ▶️ View recording
ConfigSetGet_CreatesNestedJsonFormat ▶️ View recording
CreateAndRunAspireStarterProject ▶️ View recording
CreateAndRunAspireStarterProjectWithBundle ▶️ View recording
CreateAndRunEmptyAppHostProject ▶️ View recording
CreateAndRunJavaEmptyAppHostProject ▶️ View recording
CreateAndRunJsReactProject ▶️ View recording
CreateAndRunPythonReactProject ▶️ View recording
CreateAndRunTypeScriptEmptyAppHostProject ▶️ View recording
CreateAndRunTypeScriptStarterProject ▶️ View recording
CreateJavaAppHostWithViteApp ▶️ View recording
CreateTypeScriptAppHostWithViteApp_UsesConfiguredToolchain ▶️ View recording
DashboardRunWithAgentMcpCore ▶️ View recording
DashboardRunWithOtelTracesReturnsNoTracesCore ▶️ View recording
DeployK8sBasicApiService ▶️ View failure recording
DeployK8sWithExternalHelmChart ▶️ View recording
DeployK8sWithGarnet ▶️ View recording
DeployK8sWithMongoDB ▶️ View recording
DeployK8sWithMySql ▶️ View recording
DeployK8sWithPostgres ▶️ View recording
DeployK8sWithRabbitMQ ▶️ View recording
DeployK8sWithRedis ▶️ View recording
DeployK8sWithSqlServer ▶️ View recording
DeployK8sWithValkey ▶️ View recording
DeployTypeScriptAppToKubernetes ▶️ View recording
DescribeCommandResolvesReplicaNames ▶️ View recording
DescribeCommandShowsRunningResources ▶️ View recording
DetachFormatJsonProducesValidJson ▶️ View recording
DetachFormatJsonProducesValidJsonWhenRestartingExistingInstance ▶️ View recording
DoListStepsShowsPipelineSteps ▶️ View recording
DocsCommand_RendersInteractiveMarkdownFromLocalSource ▶️ View recording
DoctorCommand_DetectsDeprecatedAgentConfig ▶️ View recording
DoctorCommand_TypeScriptAppHostReportsMissingConfiguredToolchain ▶️ View recording
DoctorCommand_WithSslCertDir_ShowsTrusted ▶️ View recording
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted ▶️ View recording
GeneratedAspireDevScript_StartsWatchMode_WithConfiguredToolchain ▶️ View recording
GlobalMigration_HandlesCommentsAndTrailingCommas ▶️ View recording
GlobalMigration_HandlesMalformedLegacyJson ▶️ View recording
GlobalMigration_PreservesAllValueTypes ▶️ View recording
GlobalMigration_SkipsWhenNewConfigExists ▶️ View recording
GlobalSettings_MigratedFromLegacyFormat ▶️ View recording
InitTypeScriptAppHost_AugmentsExistingViteRepoAtRoot ▶️ View recording
InteractiveCSharpInitCreatesExpectedFiles ▶️ View recording
InvalidAppHostPathWithComments_IsHealedOnRun ▶️ View recording
JavaScriptHostingApisRunFromTypeScriptAppHost ▶️ View recording
LatestCliCanStartStableChannelAppHost ▶️ View recording
LatestCliCanStartStableChannelTypeScriptAppHost ▶️ View recording
LegacySettingsMigration_AdjustsRelativeAppHostPath ▶️ View recording
LogLevelTrace_ProducesTraceEntriesInCliLogFile ▶️ View recording
LogsCommandShowsResourceLogs ▶️ View recording
OtelLogsReturnsStructuredLogsFromStarterApp ▶️ View recording
OtelLogsReturnsStructuredLogsFromStarterAppIsolated ▶️ View recording
PsCommandListsRunningAppHost ▶️ View recording
PsFormatJsonOutputsOnlyJsonToStdout ▶️ View recording
PublishJavaScriptPatternsGeneratesExpectedDockerComposeArtifacts ▶️ View recording
PublishWithConfigureEnvFileUpdatesEnvOutput ▶️ View recording
PublishWithDockerComposeServiceCallbackSucceeds ▶️ View recording
PublishWithoutOutputPathUsesAppHostDirectoryDefault ▶️ View recording
ResourceCommand_FailedExecution_DisplaysAppHostLogPathAndLogContainsEntries ▶️ View recording
ResourceCommand_SetAndDeleteParameterUpdatesDescribeOutput ▶️ View recording
RestoreGeneratesSdkFiles ▶️ View recording
RestoreGeneratesSdkFiles_WithConfiguredToolchain ▶️ View recording
RestoreRefreshesGeneratedSdkAfterAddingIntegration ▶️ View recording
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes ▶️ View recording
RunFromParentDirectory_UsesExistingConfigNearAppHost ▶️ View recording
RunPublishFailureScenarioAsync ▶️ View recording
RunReportsSyntaxErrorsForDotNetAppHost ▶️ View recording
RunReportsSyntaxErrorsForTypeScriptAppHost ▶️ View recording
SecretCrudOnDotNetAppHost ▶️ View recording
SecretCrudOnTypeScriptAppHost ▶️ View recording
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels ▶️ View recording
StartAndWaitForTypeScriptSqlServerAppHostWithNativeAssets ▶️ View recording
StartReportsSyntaxErrorsForDotNetAppHost ▶️ View recording
StartReportsSyntaxErrorsForTypeScriptAppHost ▶️ View recording
StopAllAppHostsFromAppHostDirectory ▶️ View recording
StopJavaPolyglotAppHostUsingApphostDirectory ▶️ View recording
StopNonInteractiveSingleAppHost ▶️ View recording
StopTypeScriptPolyglotAppHostUsingApphostDirectory ▶️ View recording
StopWithNoRunningAppHostExitsSuccessfully ▶️ View recording
UnAwaitedChainsCompileWithAutoResolvePromises ▶️ View recording
UpdateProjectChannelToStable_TypeScript_PicksUpStablePackages ▶️ View recording

📹 Recordings uploaded automatically from CI run #26430255552

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-cli flaky-test quarantined-test Quarantined tests that run only in the Outerloop Tests workflow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants