SkillsLib.ai

just-scrape CLI

AI-powered CLI for web scraping, data extraction, and page monitoring

3.7(6 reviews)
10,000+ downloads
Updated Oct 2026

What You Can Do

Search the web, scrape URLs into markdown/HTML/screenshots/links/images, extract structured JSON with AI prompts, crawl multi-page sites, and monitor pages for changes on a schedule. Supports browser automation features like JavaScript rendering, stealth mode, cookies, and custom headers. Perfect for data engineers, researchers, and developers who need API-free web data collection without writing scrapers.

Features

Web Search

Find pages and sources with optional AI-powered data extraction and filtering by time range or country

Multi-format Scraping

Extract a single URL as markdown, HTML, screenshot, links, images, summary, or branding in one command

Structured Data Extraction

Use AI prompts and JSON schemas to pull structured data from any webpage with guaranteed output shape

Multi-page Crawling

Bulk extract multiple pages from a site section with depth limits, include/exclude patterns, and format control

Page Monitoring

Schedule recurring checks with cron intervals or simple shorthands (30m, 1h, 1d) and optional webhook notifications

Browser Automation

JavaScript rendering, stealth mode, custom cookies, authentication headers, and configurable scroll depths

Request History

Track all scrape, extract, search, and crawl operations with status, timestamps, and full request/response payloads

Credit & Health Checks

Monitor API credits in real-time and validate ScrapeGraph API configuration before large operations

Example Output

Search for Python tutorials and extract titles

code
just-scrape search "python best practices" --num-results 5 -p "Extract page title and description" --json

Returns JSON with matched pages, their titles, descriptions, and URLs from the search results.

Scrape a homepage and extract multiple formats

code
just-scrape scrape "https://example.com" -f markdown,links,screenshot --json

Downloads the page as markdown, extracts all links, captures a screenshot, and saves all formats to JSON in a single request.

Monitor a pricing page hourly with webhook alerts

code
just-scrape monitor create --url "https://example.com/pricing" --interval 1h --webhook-url "https://your-webhook.com/alerts" --name "Pricing tracker"

Schedules hourly checks of the page and POSTs change notifications to your webhook endpoint automatically.

What's Included

  • 8 CLI commands: search, scrape, extract, crawl, monitor, history, credits, validate
  • Claude agent skill: SKILL.md for seamless integration with Claude Code workflows
  • Configuration management: Supports environment variables, .env files, config.json, and interactive prompts
  • NPM package: Globally installable via npm, pnpm, yarn, or bun with zero additional dependencies

Who It's For

  • Data engineers
  • Web developers and full-stack engineers
  • Researchers and data scientists
  • DevOps and SRE teams
  • Business intelligence and competitive intelligence analysts

Best For

  • Web scraping and data extraction
  • Competitive intelligence and market research
  • Page change monitoring and alerting
  • API-free data collection from websites
  • Structured data extraction with AI validation

You might also like

FREE
Developer Workflow and Architecture Skills

This collection of skills improves code quality, architecture documentation, and team workflow coordination. Review Architecture Decision Records with fresh perspective and rewrite them for clarity, implement type-safe AST visitor patterns for robust code organization, automate Biome linting upgrades across monorepos, and generate well-structured GitHub pull requests integrated with Linear project tracking. Perfect for teams that care about maintainable code, clear decision documentation, and smooth development workflows.

P
by prisma
FREE
Azure Skills for Claude Agents

This comprehensive collection includes 79 dedicated agent skills for all major Azure services and operations. You can automate Azure resource deployment and management, query infrastructure and logs with natural language using Kusto Graph and IRQL, analyze security events and detect threats, configure identity and access management through Entra ID, monitor application performance with Application Insights, manage storage and messaging services, execute cloud migration workflows, validate compliance policies, and query Azure resources by metadata or properties.

FREE
Remotion Video Creation Agent Skills

This comprehensive skill collection guides you through creating professional videos programmatically using Remotion and React. Create new video projects, structure complex multi-scene compositions, and animate content with full interactivity. Add captions by transcribing audio, importing SRT files, or generating them programmatically. Build animated maps with Mapbox, MapLibre, or GeoJSON, including 3D geographic flyovers. Process multimedia by trimming, cropping, and extracting metadata from video and audio files.

FREE
HyperFrames

Render dynamic videos directly from HTML/code using Puppeteer and ffmpeg, with built-in support for AI voiceover, caption overlays, and motion animation. Convert markdown changelogs into polished 45-60 second branded videos with animated UI mockups, captions, and AI narration. Create explainer videos, product demos, and animated visualizations using code-first workflows and motion doctrines that enforce consistency and quality.

FREE
RigorPilot Research Skills

RigorPilot is a comprehensive skill suite for deep learning research, providing auditable repository analysis, README-first reproducibility workflows, bounded exploration with evidence-based ranking, and publication-ready scientific documentation. Each skill enforces rigor through documented assumptions, comparability tracking, and explicit decision checkpoints. Use these skills to understand ML repository internals, faithfully reproduce published results, systematically explore novel ideas with proper baselines, and generate standardized research artifacts.

FREE
Sleek Design Mobile Apps

Create production-ready mobile app designs by describing what you want in plain language. Use chat-based requests to design screens, iterate with follow-up messages, and export code implementations for HTML, React Native, and SwiftUI. Features include live design preview in the Sleek editor, component screenshot generation, and built-in design references to seed your visual style.

FREE
SoulTrace Personality Assessment

Present users with an adaptive 24-question personality assessment that uses Bayesian active learning to efficiently map their psychological profile. The skill automatically maps user responses (1-7 scales or natural language) to archetype classifications based on a 5-color psychological model, tracks probability distributions in real-time, and delivers detailed personality insights including core strengths, weaknesses, and top archetype matches.

FREE
inference.sh Agent Skills Library

Access a comprehensive library of Claude AI skills that integrate with the inference.sh API, giving your agents capabilities to automate content creation, orchestrate multi-step workflows, generate images and videos, and connect to hundreds of external apps and services. Build batch processing pipelines, schedule recurring tasks, and chain multiple AI models together for complex automation.

Free