Test Flakiness
Detect flaky tests by analyzing CI logs and pass rate history
What You Can Do
Analyze CI run logs and test result history to detect tests that fail non-deterministically, calculate per-test pass rates across multiple runs, and distinguish between genuinely broken tests and intermittent failures. You get actionable recommendations on whether to quarantine or fix each flaky test, plus an updated registry to prevent masking real failures.
Features
automatically detects your CI platform
Example Output
Input: CI logs from 15 recent runs
Output:
## Flaky Tests Detected (Pass Rate < 95%)
### High Priority (40-80% pass rate)
- `test_player_spawn_collision` — 65% pass rate — Likely: race condition in physics engine initialization
- **Recommendation:** Quarantine immediately, add synchronization to test setup
- **Severity:** Critical (masks real failures)
### Medium Priority (80-95% pass rate)
- `test_ui_animation_complete` — 92% pass rate — Likely: timing-dependent, platform-specific
- **Recommendation:** Increase wait timeout or add retry logic
- **Severity:** Moderate
What's Included
- `test-flakiness` SKILL.md instruction file with multi-platform CI log parsing:
- Flakiness detection algorithm with statistical thresholds:
- Quarantine registry template for regression-suite.md:
- Pass-rate calculation framework and failure pattern analyzer:
- Remediation recommendation matrix (quarantine vs. fix guidance):
Who It's For
- QA/Test Engineers — diagnose why CI is unreliable and decide quarantine vs. fix strategies
- Release Engineers — reduce false positives during release validation runs
- Software Engineers — identify which flaky tests to prioritize fixing during Polish phase
- Platform Teams — maintain test reliability and prevent test rot across CI/CD pipelines
- DevOps Engineers — monitor test suite health and flag systemic flakiness trends
Best For
- Detecting intermittent test failures across multiple CI runs without manual log review
- Quarantining known flaky tests to prevent masking real failures in CI
- Diagnosing root causes of test non-determinism (timing, race conditions, environment sensitivity)
- Building a historical registry of flaky tests and remediation status
- Polish phase regression testing when sufficient CI data has accumulated
You might also like
Query your local Screenpipe instance to retrieve screen recordings, audio transcriptions, UI element accessibility trees, keyboard/mouse input logs, and productivity analytics. You can search by keywords, filter by content type (audio, OCR, accessibility, input), set time ranges, and extract structured data about your applications, meetings, and work sessions without sending data to external servers.
by az9713
You can submit architectural design documents, floor plans, and project parameters to receive a comprehensive compliance analysis that cross-references IBC, local amendments, zoning ordinances, and ADA accessibility requirements. Claude identifies specific gaps, cites applicable code sections, explains why each gap matters, and provides concrete remediation strategies—all before formal design review submission when corrections are costly and schedule-impacting.
by az9713
You can leverage Claude to synthesize regional seismic hazard data from authoritative sources (USGS maps, published studies), interpret probabilistic versus deterministic earthquake scenarios, and analyze how site conditions modify ground motions. Claude helps you justify seismic design parameters to clients and regulators, evaluate liquefaction and earthquake-induced landslide risks, and develop defensible recommendations aligned with ASCE 7 and current guidelines—accelerating report preparation without replacing specialized geotechnical software.
by az9713
You systematically evaluate your internal developer platform across dimensions like onboarding, deployment, tooling, and documentation to identify quantifiable friction points. Claude analyzes your platform's usability, creates a friction scoring matrix, and generates a prioritized improvement roadmap ranked by team impact and implementation effort. You get a data-driven strategy to reduce developer toil and accelerate delivery.
by az9713
This skill enables you to rapidly produce structural analyses for timber members, including bending, shear, compression, and combined stress calculations. You can determine lateral load capacity for shear walls, design connections, evaluate code compliance against NDS standards, assess serviceability limits, and generate calculation frameworks suitable for permit submission and engineer stamping.
by az9713
Design earth retention systems by inputting soil properties, site geometry, and regulatory requirements. Claude calculates lateral earth pressures using Coulomb, Rankine, and Mononobe-Okabe methods, verifies factors of safety for internal and external stability, compares competing retention concepts (gravity walls, cantilever, anchored, sheet-pile, soil-nail), and generates cost-benefit analysis and regulatory documentation packages ready for permit submission.
by az9713
Quickly evaluate pavement design scenarios using AASHTO 1993 methodology to calculate required structural thickness for flexible (asphalt) and rigid (concrete) pavements. Analyze traffic loading by computing Equivalent Single Axle Loads (ESALs), assess subgrade conditions through CBR interpretation, and compare multiple design alternatives with different material combinations to optimize cost and performance. Generate preliminary design documentation with technical justification for client presentations and design reports.
by az9713
You can accelerate HVAC design workflows by inputting building parameters—envelope characteristics, occupancy data, climate conditions, and existing systems—and receiving calculated heating/cooling loads, equipment sizing recommendations, and zone-by-zone analysis. Claude organizes complex building data into actionable design parameters and specification summaries for client review, permit submission, and professional validation against ASHRAE and ACCA standards.
by az9713