Rag Pipeline Builder
Design and optimize end-to-end RAG pipelines with intelligent document retrieval
What You Can Do
You'll design complete RAG architectures from ingestion through inference, making strategic decisions on chunking strategies, embedding models, vector database selection, and retrieval tuning. This skill guides you through all five core pipeline phases—document preprocessing, semantic embedding, vector storage, ranking optimization, and context window assembly—ensuring your system balances accuracy, latency, and cost for domain-specific knowledge retrieval at scale.
Features
Design semantic-aware splitting with overlap, handling different document types (PDFs, markdown, structured data)
Choose between dense embeddings (OpenAI, Cohere, open-source) based on cost, latency, and domain fit
Evaluate and configure PostgreSQL pgvector, Pinecone, Weaviate, Chroma, or Milvus for scalable retrieval
Implement hybrid search, BM25 + semantic fusion, re-ranking, and metadata filtering to maximize relevance
Intelligently pack retrieved chunks within token limits while preserving semantic coherence
Deploy via LangChain, LlamaIndex, or custom Python implementations with minimal scaffolding
Design metrics for retrieval quality (NDCG, MRR) and end-to-end answer accuracy
Example Output
Example 1: E-commerce Product Support RAG
- Chunking: 256-token sections from product manuals with 64-token overlap, preserving section headers
- Embedding: OpenAI text-embedding-3-small for cost efficiency
- Vector Store: Pinecone with product category metadata filtering
- Retrieval: Hybrid (BM25 + semantic) with MMR reranking for diversity
- Output: Customer service bot retrieving exact product specs with 94% accuracy on FAQ queries
Example 2: Legal Document Discovery System
- Chunking: Clause-aware segmentation respecting legal document structure
- Embedding: Self-hosted sentence-transformers for data sovereignty
- Vector Store: PostgreSQL pgvector for on-premise deployment
- Retrieval: Dense passage retrieval + legal metadata filters (document type, jurisdiction, date range)
- Output: Paralegal tool surfacing relevant precedents in 200ms with full citation trails
What's Included
- RAG Pipeline Builder SKILL.md: Complete instruction framework for designing RAG systems
- Chunking Decision Matrix: Comparison of splitting strategies (semantic, fixed-size, recursive) with trade-offs
- Vector Store Comparison Checklist: Evaluation criteria for 6+ vector databases (cost, latency, scalability)
- Retrieval Tuning Workflow: Step-by-step process for testing embeddings, ranking strategies, and hybrid search
- Context Packing Template: Code patterns for intelligent document assembly within token windows
- RAG Evaluation Framework: Metrics and benchmarking approach for retrieval quality
Who It's For
- AI/ML Engineers — Building production RAG systems for customer-facing applications
- Data Scientists — Designing knowledge retrieval architectures for proprietary datasets
- LLM Product Managers — Making technical decisions on embedding and vector storage stack
- Enterprise Architects — Planning document intelligence solutions across organizational knowledge bases
- Prompt Engineers — Optimizing retrieval performance to ground AI responses in domain expertise
Best For
- Designing document-grounded Q&A systems (customer support, FAQ automation)
- Building legal/compliance document discovery tools
- Creating research or scientific paper recommendation engines
- Architecting enterprise knowledge management systems
- Optimizing existing RAG pipelines for speed, cost, or accuracy
- Evaluating and selecting appropriate vector databases and embedding models
You might also like
This skill provides a systematic framework for preparing training data, benchmarking model performance, identifying deployment risks, and calculating true ROI before committing fine-tuning investments. You'll validate datasets, run structured evaluations against baseline Claude models, test edge cases, and iterate toward production-ready fine-tuned versions. The framework ensures your fine-tuned models deliver meaningful improvements while maintaining safety and cost-effectiveness.
by moonwell-fi
You can design statistically valid experiments with proper power analysis, choose the right statistical tests for your data type, analyze results while controlling for multiple comparisons, and generate publication-ready reports with accurate interpretation of findings. Claude helps you avoid common statistical pitfalls and ensures your experimental claims are well-supported by evidence.
by moonwell-fi
This skill automatically documents your entire analytics infrastructure by analyzing data sources, transformations, and outputs. You'll generate production-ready data dictionaries with field definitions, lineage maps showing data flow across systems, and transformation documentation that explains logic and dependencies. Save weeks of manual documentation work while keeping your analytics stack discoverable as it evolves.
by moonwell-fi
You'll build comprehensive data quality test suites that validate transformations, detect anomalies, and document standards across your dbt and SQL pipelines. This skill generates production-ready test configurations, anomaly detection protocols, and validation rules that catch data issues before they impact analytics.
by moonwell-fi
You can create comprehensive documentation for your data platforms, metrics, and analytics infrastructure that stakeholders actually understand. The skill generates clear data dictionaries, metric definitions, pipeline diagrams, and runbooks that bridge the gap between technical teams and business users. Your documentation stays consistent with your actual infrastructure while being instantly accessible to analysts, managers, and engineers.
by moonwell-fi
You can rapidly compile, organize, and format disparate health economic evidence—from clinical trials to cost-effectiveness analyses—into structured, regulatory-compliant dossiers. This skill maps your evidence to specific payer and HTA requirements, automatically generates evidence hierarchies, and produces submission-ready dossier outlines with formatting that meets regulatory standards for NICE, EUnetHTA, and other major bodies.
by moonwell-fi
You can structure comprehensive health technology assessments (HTAs) that organize clinical evidence, economic analyses, and regulatory considerations into evidence-based coverage recommendations. This skill helps you synthesize clinical trial data, health economic models, and real-world evidence into clear, defensible payer coverage determinations. You'll generate professional HTA reports that align with major frameworks like ICER, CADTH, and NICE standards.
by moonwell-fi
This skill helps you systematically diagnose failures in distributed ML training and serving infrastructure. You provide system logs, metrics, and error traces, and Claude performs structured root-cause analysis to identify the underlying issue—whether it's resource exhaustion, distributed system deadlock, data pipeline corruption, or model serving misconfiguration. You get a detailed diagnosis with remediation steps ranked by likelihood and implementation effort.
by moonwell-fi