
Scientific Critical Thinking
Evaluate research rigor using GRADE and Cochrane frameworks for scientific validity
What You Can Do
You can rigorously evaluate scientific research by assessing methodology, experimental design, and statistical validity using established frameworks like GRADE and Cochrane Risk of Bias. This skill helps you identify confounding variables, detect biases, and determine the overall quality of evidence in research papers, enabling you to critically challenge scientific claims and distinguish robust findings from flawed conclusions.
Features
Evaluate study design, sample size, control groups, and experimental protocols for soundness
Analyze p-values, confidence intervals, effect sizes, and statistical power to assess significance
Identify selection bias, publication bias, confirmation bias, and conflicts of interest in research
Spot unmeasured and measured confounding variables that may explain reported associations
Rate evidence quality from high to very low based on study design and limitations
Evaluate domain-specific bias risks including randomization, blinding, and reporting
Compare multiple studies to determine consistency, generalizability, and overall evidence strength
Produce structured reports highlighting limitations, implications, and recommendations
Example Output
Example 1: Study Evaluation Report
- Study Design: Randomized controlled trial (RCT) — appropriate
- Sample Size: n=150, adequate statistical power for primary outcome
- Risk of Bias: Low risk across selection, performance, and attrition domains
- Confounding: Age and baseline severity controlled via randomization
- GRADE Rating: High quality evidence — Consistent with meta-analysis findings
Example 2: Critical Analysis of Claim
- Claim: "Coffee reduces cancer risk by 50%"
- Evidence Found: Observational study with 10-year follow-up
- Key Limitation: Unmeasured confounding (smoking, alcohol use)
- Bias Risk: Publication bias toward positive findings in coffee industry-funded research
- Conclusion: Moderate quality evidence; association may not be causal
Example 3: Systematic Review Synthesis
- 25 RCTs reviewed; 18 showed treatment benefit, 7 null results
- Heterogeneity: I² = 64% (substantial); possible effect modification by age
- GRADE Downgrade: One level for inconsistency across studies
- Final Rating: Moderate quality evidence — Recommend treatment for most patients with caution in elderly
What's Included
- SKILL.md instruction file: Complete framework for scientific critical thinking analysis
- Research Evaluation Checklist: Systematic template covering methodology, statistics, bias, and confounding assessment
- GRADE Evidence Quality Framework: Structured approach to rating evidence from high to very low quality
- Cochrane Risk of Bias Template: Domain-by-domain bias assessment for randomized and observational studies
- Critical Analysis Workflow: Step-by-step process for evaluating claims, identifying limitations, and synthesizing findings
Who It's For
- Researchers and academics — Evaluating peer research and literature reviews
- Medical professionals — Assessing clinical trial evidence for treatment decisions
- Science communicators and journalists — Fact-checking scientific claims in public discourse
- Policy makers — Determining evidence quality for evidence-based decision-making
- Graduate students — Conducting systematic reviews and meta-analyses for theses
Best For
- Peer review and manuscript evaluation for journals
- Systematic reviews and evidence synthesis across multiple studies
- Clinical guideline development requiring rigorous evidence grading
- Fact-checking and debunking pseudoscientific or sensationalized claims
- Research proposal assessment for methodology soundness and feasibility







