test-reliability
Before starting: Check for .agents/qa-project-context.md in the project root. It contains known flaky areas, selector strategy, and CI environment details.
Discovery Questions
-
What is your current flaky test rate? Check CI failure stats over the last 30 days. Below 2% is healthy. 2-5% needs attention. Above 5% is eroding team trust.
-
Where is the pain concentrated? Is it locator breakage? Timing issues? Test data? Environment instability? If unknown, instrument first (see Flake Classification).
-
What is your current selector strategy? data-testid everywhere? Mixed CSS and role-based? No strategy (whatever works)?
-
How do you handle flaky tests today? Retry and hope? Skip and forget? Something structured?
-
What CI environment runs the tests? Same machine every time or different runners? Consistent resources or variable? How does the CI runner compare to local dev machines?