
IDP API and CLI Design Validation
Validate IDP APIs and CLIs for consistency, safety, and developer experience
What You Can Do
You audit Internal Developer Platform APIs and CLIs against industry best practices, evaluating design patterns, error handling, documentation, and operational safety. Claude scores your API or CLI across multiple dimensions (consistency, discoverability, safety, completeness) and delivers prioritized recommendations for improvement—from critical gaps to quick wins you can implement immediately.
Features
Validates REST/GraphQL conventions, naming patterns, versioning strategy, and schema consistency across all endpoints
Evaluates command hierarchy, flag naming, help text quality, and tab-completion readiness for usability
Assesses error codes, messages, debugging information, and recovery suggestions for clarity and actionability
Reviews API docs, CLI help, code examples, and troubleshooting guides for completeness and accuracy
Rates APIs and CLIs on discoverability, learnability, consistency, and cognitive load (1-5 scale)
Identifies authentication gaps, rate-limiting issues, audit logging, and deprecation path planning
Validates webhook design, SDK/client patterns, OAuth flows, and third-party tool integration
Audit multiple APIs or CLIs side-by-side with consistent scoring for platform-wide consistency analysis
Example Output
API Audit: Deployment Service v1.2
Overall Score: 3.7/5 — Solid REST design with consistency gaps
Strengths
- Clear resource hierarchy (services → deployments → logs)
- Comprehensive error codes (25+ types) with descriptive messages
- Versioning strategy clear:
/v1/prefix, migration guide provided
Critical Gaps
- Pagination: Three different patterns used (limit/offset, cursor, keyset) across endpoints
- Rate limiting: Missing
X-RateLimit-*response headers; clients can't detect approaching limits - Datetime formats: Mixed ISO 8601 and Unix timestamps; timezone context unclear
Recommended Actions
- Standardize pagination → Cursor-based for all endpoints (supports large datasets, efficient scanning)
- Add rate-limit headers → Include Limit, Remaining, Reset to all responses
- Normalize timestamps → ISO 8601 everywhere; document UTC requirement
Effort: Medium | Impact: High → Implement in next 2 sprints
CLI Audit: deploy-cli v2.1
Overall Score: 4.3/5 — Excellent usability with minor polish
Strengths
- Consistent naming (verb-object):
deploy create,deploy list,deploy logs - Comprehensive
--helpwith examples and links to docs - Tab-completion for all commands and resource names
Quick Wins
- Add typo detection:
deploy creat→ suggestsdeploy create - Add
--waitflag with progress spinner for long operations - Improve error messages for auth failures: show token refresh command
What's Included
- SKILL.md: Complete evaluation skill with decision trees, scoring rubrics, and validation workflows
- API Design Checklist: 40+ REST and GraphQL patterns to evaluate (conventions, versioning, pagination, errors)
- CLI Audit Template: Structured framework for command hierarchy, flags, help text, and completion readiness
- Scoring Rubric: 5-point scale dimensions: consistency, safety, developer experience, documentation, completeness
- Error Handling Catalog: Common error patterns, HTTP status code mapping, debugging info best practices
- Documentation Review Guide: Checklist for API docs, CLI help, code examples, troubleshooting, and runbooks
- Integration Patterns Worksheet: Evaluation criteria for webhooks, SDKs, OAuth flows, rate-limiting, and deprecation paths
- Audit Report Template: Formatted markdown structure for presenting scores, findings, and prioritized recommendations
- Case Studies: 3 annotated IDP platform audits (Kubernetes-like API, GitHub CLI patterns, internal deployment tool)
Who It's For
- Platform Engineers — Design and maintain APIs/CLIs used by internal development teams
- Engineering Leads — Set quality standards and oversee developer experience across platform tools
- API Architects — Plan new Internal Developer Platform services before implementation
- DevOps and SRE Teams — Ensure CLIs are discoverable, safe, and operationally sound
- Engineering Managers — Evaluate developer friction points and platform investment priorities
Best For
- Designing and validating new Internal Developer Platform APIs before implementation
- Conducting design reviews for platform tools used by 50+ engineers
- Migrating between API versions or CLI frameworks while maintaining consistency
- Assessing third-party SDKs and integrations against your platform standards
- Planning usability and documentation improvements for existing APIs/CLIs
- Comparing two design approaches (REST vs RPC, positional vs flag-based CLIs) for a new feature







