RAG Optimization Engineer
Optimize RAG pipelines for retrieval quality and accuracy
What You Can Do
Debug retrieval failures, optimize chunking strategies, and validate answer accuracy across different RAG configurations. You can systematically test embedding models, chunk sizes, query expansion techniques, and reranking parameters to maximize your pipeline's performance. This skill helps you measure the impact of changes and identify which retrieval bottlenecks are hurting your system most.
Features
Analyze which documents are retrieved for each query and why they rank in that order
Test different chunk sizes, overlap, and splitting strategies to maximize relevance
Evaluate embedding model performance and identify relevance scoring issues
Experiment with query rewriting and expansion to improve recall
Systematically compare multiple RAG parameter sets with accuracy scores
Generate hit rate, precision@k, relevance scores, and other key metrics
Trace why specific queries fail and recommend targeted fixes
Compare answer accuracy before and after optimization changes
Example Output
Retrieval Quality Report
Current Metrics:
- Hit Rate (top-5): 72%
- Precision@1: 45%
- Avg Relevance Score: 0.68
Failing Queries Analysis:
- 15% fail due to poor embedding similarity
- 8% fail due to suboptimal chunk boundaries
- 5% fail due to insufficient query expansion
Configuration Comparison
| Strategy | Hit Rate | Precision@1 | Impact |
|---|---|---|---|
| Current (512 chunks) | 72% | 45% | Baseline |
| Test A (1024 chunks) | 78% | 52% | +20% tokens |
| Test B (Query expansion) | 81% | 58% | +15% latency |
| Test A+B (Combined) | 84% | 63% | +32% tokens |
Recommendation: Implement combined approach for 18 percentage-point improvement in Precision@1.
What's Included
- SKILL.md: Complete RAG optimization workflow with diagnostic trees and validation checklists
- RAG Diagnostic Template: Structured format for capturing your pipeline architecture and current metrics
- Configuration Comparison Worksheet: Side-by-side testing framework for parameter combinations
- Query Expansion Checklist: Techniques for rewriting and expanding queries to improve recall
- Retrieval Validation Suite: Test framework with accuracy scoring and metric calculation
- Troubleshooting Guide: Common RAG failure patterns and their root causes
Who It's For
- ML Engineers building and tuning RAG applications for production
- Data Scientists optimizing search and retrieval systems for accuracy
- LLM Product Managers responsible for Q&A accuracy and user satisfaction
- AI/ML Architects designing enterprise information retrieval systems
- Search Engineers debugging why queries return irrelevant results
Best For
- Diagnosing retrieval failures in underperforming RAG pipelines
- Comparing chunking strategies (size, overlap, splitting methods)
- Validating embedding model upgrades before production deployment
- Root cause analysis of queries that return irrelevant documents
- Optimizing cost vs. accuracy tradeoffs by testing parameter combinations
You might also like
Create comprehensive KPI dashboards that track supply chain performance across cost, quality, delivery, and service metrics. Automate variance analysis to identify root causes of performance deviations and generate executive-level reports that translate operational data into actionable insights for leadership and stakeholders.
by skillslib
You can design end-to-end onboarding programs that reduce time-to-productivity, ensure regulatory compliance, and deliver consistent first-day experiences across all departments. The skill generates role-specific workflows, compliance checklists, stakeholder coordination plans, and success metrics—everything your HR team needs to scale onboarding with confidence.
by skillslib
You'll design robust SCADA system architectures, configure industrial protocols (Modbus, Profibus, OPC-UA, DNP3), and diagnose connectivity and performance issues across distributed control networks. Get step-by-step configuration guidance, integration workflows, and troubleshooting decision trees tailored to your specific hardware and protocol stack.
by skillslib
You can establish systematic color grading workflows that accelerate project delivery, ensure visual consistency across episodes and projects, and maintain broadcast-quality standards. Claude generates reusable templates, quality control checklists, and grading decision frameworks tailored to your project's color science and deliverable requirements.
by skillslib
You can orchestrate every aspect of your live event with a unified command center that responds to real-time incidents, coordinates teams across departments, and adjusts plans on the fly. The skill generates contingency workflows, communication protocols, and decision frameworks so you stay ahead of problems instead of reacting to them. You'll have structured guidance for everything from timeline adjustments to vendor coordination to emergency escalations.
by skillslib
You'll build complete activation funnels by analyzing user behavior, identifying conversion bottlenecks, and designing targeted onboarding campaigns. Claude creates customer journey maps, funnel stage definitions, engagement messaging frameworks, and A/B testing strategies tailored to your product and audience.
by skillslib
Use Claude to automate your entire collection workflow, from first touch to recovery, with legally sound escalation protocols. You'll recover more money in less time, reduce bad debt write-offs by 25-40%, and build defensible collection records that withstand legal scrutiny. Claude handles customized demand letters, compliance validation, debtor profiling, and escalation strategies tailored to account age, debtor behavior, and jurisdiction.
by skillslib
This skill helps you troubleshoot Wwise integration issues, diagnose audio performance bottlenecks, and generate comprehensive documentation for interactive audio specifications. You'll get step-by-step fixes for common implementation problems, performance optimization strategies, and best practices for audio systems across different game engines and platforms.
by skillslib