Survey Psychometric Validation & Scale Development
Validate surveys with psychometric rigor and scientific precision
by omkamalSocial Sciences & PolicyPsychology (Applied)Industrial-Organizational PsychologistSurvey/AnalyticsWhat You Can Do
You design, validate, and optimize survey instruments using systematic psychometric analysis. The skill systematically evaluates item quality, tests reliability through Cronbach's alpha and test-retest correlations, assesses construct validity via factor analysis and convergent/discriminant testing, and generates data-driven recommendations for survey improvement. Whether building a new measurement scale or auditing an existing one, you gain scientific evidence that your survey measures what it claims.
Features
Calculate item difficulty, discrimination indices, item-total correlations, and inter-item relationships to identify high-performing and problematic questions
Compute Cronbach's alpha, McDonald's omega, test-retest reliability, and split-half consistency to quantify internal consistency and measurement stability
Perform exploratory and confirmatory factor analysis, evaluate model fit (CFI, RMSEA, TLI), and visualize factor loadings to validate theoretical structure
Test convergent validity with related measures, discriminant validity with unrelated constructs, and criterion validity against external outcomes
Analyze Likert scale performance, evaluate optimal response anchors, detect ceiling/floor effects, and assess whether 5-point vs 7-point scales work better
Identify items with low discrimination, high redundancy, or unclear wording that should be revised, removed, or reordered
Compare psychometric performance across survey formats, response scales, or item phrasings to determine which design performs best
Generate comprehensive psychometric reports with statistical summaries, interpretation, and publication-ready documentation of measurement quality
Example Output
Item Analysis Summary
| Item | Mean | SD | Discrimination | Item-Total r | Recommendation |
|---|---|---|---|---|---|
| Q1: I feel energized at work | 4.2 | 1.1 | 0.58 | 0.62 | Keep |
| Q3: My work feels meaningful | 3.8 | 1.3 | 0.71 | 0.69 | Strengthen—highest discriminator |
| Q7: Feedback is confusing | 2.1 | 1.4 | 0.12 | 0.28 | Revise—ambiguous wording |
Reliability Results
- Cronbach's alpha: 0.82 (95% CI: 0.78–0.86) — Acceptable internal consistency
- McDonald's omega: 0.84 — Reliable latent variable measurement
- Test-retest (4-week): r = 0.76, p < .001 — Stable over time
Factor Structure
Confirmatory Factor Analysis (Single Factor Model):
- CFI = 0.91 (Acceptable; >0.90 threshold)
- RMSEA = 0.068 (Good fit; <0.08)
- TLI = 0.88
Factor Loadings (all p < .001):
- Q1: 0.73 | Q3: 0.81 | Q5: 0.77 | Q9: 0.69
Key Recommendations
✓ Remove Q7 (low discrimination, confusing wording)
✓ Revise Q2 and Q6 (item-total r < 0.40)
✓ Reorder items: place highest-loading items (Q3, Q5) earlier to establish clarity
✓ Consider 7-point scale: pilot data suggests broader item spread would improve discrimination
What's Included
- Item Performance Dashboard: Complete statistics for every survey item including means, standard deviations, discrimination, and correlations with total score
- Reliability Analysis Suite: Multiple reliability methods (Cronbach's alpha, omega, test-retest, split-half) with confidence intervals and interpretation guidelines
- Factor Analysis Workflow: Exploratory and confirmatory factor analysis templates with guidance on determining optimal number of factors and evaluating model fit
- Validity Evidence Framework: Systematic approach to assess convergent, discriminant, and criterion validity against related and unrelated measures
- Optimization Recommendations: Data-driven suggestions for which items to remove, revise, reorder, or strengthen based on statistical performance
- Publication-Ready Report Template: Structured format for psychometric findings with APA-style tables, statistical interpretation, and discussion of measurement quality
Who It's For
- Research psychologists designing personality, ability, or clinical assessment scales
- Survey researchers validating measurement instruments before large-scale deployment
- UX/Product researchers developing usability scales and satisfaction surveys
- HR professionals creating employee engagement, culture, or competency assessments
- Educational researchers building learning outcome or teaching effectiveness measures
Best For
- Validating new survey instruments before publication or deployment
- Analyzing existing survey data for reliability and quality issues
- Comparing psychometric performance across alternative survey designs
- Optimizing items based on statistical performance and theoretical fit
- Generating evidence of construct validity for research papers or reports
You might also like
You can design and validate selection systems that maximize predictive accuracy while minimizing legal risk. The skill applies industrial-organizational psychology principles to create structured assessments, job analyses, and validity evidence that withstand regulatory scrutiny. You'll generate documentation that demonstrates how your selection criteria connect to job performance and comply with EEOC/OFCCP standards.
by omkamal
Conduct structured Failure Mode and Effects Analysis (FMEA) with risk quantification and mitigation tracking. Generate comprehensive safety documentation including hazard analysis, root cause analysis, and compliance artifacts that meet regulatory standards (ISO 26262, FDA, IEC 61508). Transform complex safety requirements into actionable verification and validation plans with full traceability.
by omkamal
You can systematically examine aerospace incidents, cockpit procedures, and interface designs using established human factors frameworks like SHEL, Norman's action theory, and Reason's accident causation model. The skill identifies human performance bottlenecks, error traps, and design vulnerabilities—then recommends evidence-based mitigations that reduce cognitive workload, enhance situational awareness, and prevent accidents.
by omkamal
You systematically architect feature requirements for connected car infotainment systems by synthesizing customer insights, technical constraints, and regulatory mandates. This skill structures the translation of market data into unambiguous specifications that engineering teams can execute against, accounting for legacy hardware integration, multi-generational vehicle platforms, and safety-critical certification requirements.
by omkamal
This skill transforms raw clinical findings and assessment data into structured, legally defensible competency evaluation reports. You provide clinical observations, test scores, and relevant history; Claude organizes them into coherent narratives with proper test cross-referencing, expert reasoning frameworks, and standards-compliant conclusions that withstand legal scrutiny.
by omkamal
Conduct systematic usability evaluations using Nielsen's 10 heuristics and other industry-standard frameworks to identify friction points and design improvements. Synthesize user research data into actionable insights, correlation patterns, and prioritized recommendations that connect quantitative findings to qualitative user behaviors. Generate detailed evaluation reports with severity ratings, evidence mapping, and implementation roadmaps for your design team.
by omkamal
This skill provides comprehensive human factors analysis of vehicle interface designs. You can assess cognitive load, identify driver distraction risks, evaluate NHTSA compliance, and uncover usability issues before design freeze. It combines HCI best practices, automotive safety standards, and workload assessment methodologies to deliver actionable feedback that keeps drivers safe.
by omkamal
This skill helps you develop comprehensive expert witness reports grounded in forensic psychology research, case law, and evidentiary standards. It guides you through crafting testimony strategies that anticipate cross-examination challenges, identify weaknesses in opposing arguments, and present psychological conclusions with maximum credibility before court. You'll produce documentation that withstands legal scrutiny and jury standards.
by omkamal