Summarize Experiment
Extract and summarize experiment results into lightweight markdown reports
What You Can Do
This skill parses experiment metadata and outputs to extract critical metrics from completed machine learning experiments. It reads experiment_summary.yaml to identify runs, pulls final training loss from SLURM logs, extracts accuracy metrics from inspect-ai evaluation files, and generates a clean summary.md report—all while logging the extraction process for reproducibility.
Features
Example Output
Example summary.md output:
Experiment Summary: bert-classification-v2
Run Status
| Run Name | Type | Model | Status | Training Loss | Accuracy |
|---|---|---|---|---|---|
| run-001-lr0.001 | fine-tuned | bert-base | Completed | 0.245 | 0.924 |
| run-002-lr0.0001 | fine-tuned | bert-base | Completed | 0.198 | 0.931 |
| control-baseline | control | bert-base | Completed | — | 0.847 |
Key Findings
- Best accuracy: 0.931 (run-002-lr0.0001, learning rate 0.0001)
- Improvement over baseline: +8.4 percentage points
- Training converged in all runs
Log entry example:
[2024-01-15 14:32:01] Parsing experiment_summary.yaml...
[2024-01-15 14:32:02] Found 3 runs (2 fine-tuned, 1 control)
[2024-01-15 14:32:03] Extracted training loss from run-001: 0.245
[2024-01-15 14:32:04] Extracted accuracy from eval/logs/run-002.eval: 0.931
[2024-01-15 14:32:05] summary.md generated successfully
What's Included
- `summarize-experiment.md` instruction file with full workflow documentation:
- YAML parsing template for reading experiment_summary.yaml structures:
- Python extraction script (parse_eval_log.py) for inspect-ai eval file parsing:
- summary.md template with markdown table and findings sections:
- SLURM log extraction patterns and regular expressions:
Who It's For
- Machine learning researchers conducting hyperparameter tuning experiments
- ML engineers documenting model fine-tuning results for team review
- Research scientists tracking multiple experimental runs for publications
- AI labs automating experiment result documentation workflows
- Data scientists creating reproducible experiment reports
Best For
- Summarizing multi-run fine-tuning experiments with varied hyperparameters
- Extracting metrics from completed inspect-ai evaluation workflows
- Generating quick reference documents comparing control vs. fine-tuned models
- Automating post-experiment documentation after run-experiment completion
- Creating audit trails of experiment execution and metric extraction
You might also like
You can structure and run Kaizen events from start to finish, guiding your team through waste analysis, root-cause identification, and rapid improvement design. The skill generates detailed event agendas, countermeasure documentation, and performance metrics—enabling you to close improvement opportunities within days rather than weeks.
by kylezantos
Prepare comprehensive audit documentation and manage ISO 9001/14001 compliance faster. You can generate tailored audit checklists, analyze nonconformance findings systematically, and develop corrective action plans that meet ISO standards — all in hours instead of days. This skill ensures consistent documentation across your manufacturing operations and maintains audit readiness year-round.
by kylezantos
Transform raw manufacturing P&L data into actionable insights by automatically identifying cost variances, pinpointing root causes, and generating executive-ready narratives. You upload your plant's P&L statement, provide context about recent operational changes, and Claude analyzes the numbers to explain why costs moved and what actions drive profitability.
by kylezantos
This skill systematically analyzes out-of-spec chemical manufacturing batches to identify root causes and recommend parameter adjustments. You provide process data, operating conditions, and quality metrics; Claude performs hypothesis testing, statistical correlation analysis, and multi-variate optimization to pinpoint the failure mechanism. You get actionable recommendations ranked by impact and implementation risk.
by kylezantos
You can leverage Claude to streamline every aspect of film production coordination—from managing crew schedules and budgets to troubleshooting on-set logistics and creative challenges. This skill helps you create call sheets, coordinate locations, track production expenses, and quickly solve problems that arise during shooting without disrupting your workflow.
by kylezantos
You provide production data and process details, and this skill systematically analyzes the information to identify root causes of quality issues, yield losses, or equipment failures. It develops prioritized corrective actions with implementation timelines, compliance documentation, and performance metrics to track improvement effectiveness.
by kylezantos
Accelerate literature reviews by automatically synthesizing research papers, extracting key methodologies and findings, and identifying research gaps. This skill analyzes multiple papers simultaneously to spot unexplored areas, methodological blind spots, and emerging opportunities in your field. You'll get structured analysis, comparison matrices, and research landscape summaries that would normally take weeks to produce manually.
Prepare comprehensive audit documentation, track non-conformance issues from identification through resolution, and generate corrective action plans. You can document audit findings, prioritize risks by severity, assign accountability, and create formal reports that demonstrate compliance readiness to auditors and stakeholders.
by kylezantos