Skip to content

Implement label-driven test workflow for bug fixes #3239

Description

@BenBtg

Summary

Develop a label-driven test workflow that, after a fix is created, runs the relevant tests in isolation, compiles the result, and posts the outcome back to the relevant issue. This is the third stage (assess → fix → test) of the semi-automated bug pipeline.

Context

Builds on the bug-assess workflow (.github/workflows/bug-assess.md) and the planned bug fix workflow. Each stage is triggered by a human applying a label, keeping a maintainer as gatekeeper — especially important for public repos.

Requirements

  • Label-driven trigger: runs when a maintainer applies a bug-test (or equivalent) label, after a fix exists.
  • Run tests in isolation: execute the relevant test suite against the generated fix in an isolated run.
  • Compile the result: summarize pass/fail outcomes into a readable report.
  • Post outcome back to the issue: add the compiled result as a comment on the originating issue.
  • Verification support: enable comparing a generated fix against the actual historical fix (e.g. testing against an old/closed bug) to surface discrepancies and improve reliability.
  • Safety & portability: authored as a markdown file compiled into a YAML (.lock.yml) workflow, scoped permissions, validation, untrusted-input handling, and decoupled from Spec Kit-specific logic.

Acceptance criteria

  • bug-test markdown workflow added under .github/workflows/ and compiled to its .lock.yml.
  • Triggered by applying the test label; runs tests in isolation against the fix.
  • Compiles and posts the test outcome back to the issue.
  • Supports validation against an old/closed bug to compare generated vs. historical fix.
  • Maintainer remains the gatekeeper; consistent safety model with the other stages.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions