Video Understand
Extract frames and transcribe video locally with ffmpeg and Whisper
What You Can Do
You can intelligently extract representative frames from videos using scene detection, keyframe analysis, or regular intervals, while simultaneously transcribing audio to text using OpenAI's Whisper model. This skill processes videos entirely offline, giving you frame-by-frame visual context and complete transcripts without relying on external APIs or cloud services.
Features
automatically identifies shot changes and extracts frames at transitions
isolates the most important frames based on visual content analysis
captures frames at regular time intervals for consistent sampling
converts video audio to text using offline speech recognition
control maximum frames extracted to balance detail vs. file size
choose from tiny to large models based on accuracy needs
extract visual content without transcription for faster processing
structured metadata with timestamps, frame paths, and transcription data
Example Output
Scene Detection with Transcription
{
"video": "interview.mp4",
"duration_seconds": 180,
"frames_extracted": 12,
"frames": [
{"timestamp": 0.0, "mode": "scene", "path": "frame_0000.jpg"},
{"timestamp": 15.3, "mode": "scene", "path": "frame_0001.jpg"}
],
"transcription": {
"full_text": "Welcome to the interview. Today we're discussing video production techniques...",
"segments": [
{"start": 0.0, "end": 5.2, "text": "Welcome to the interview."},
{"start": 5.2, "end": 12.1, "text": "Today we're discussing video production techniques..."}
]
}
}
Keyframe-Only Output
✓ Extracted 8 keyframes from video.mp4
✓ Identified critical visual moments
✓ Saved frames: frame_0000.jpg, frame_0001.jpg, ...
What's Included
- SKILL.md: complete setup and CLI reference
- understand_video.py: main script with scene detection, keyframe, and interval extraction modes
- Whisper integration: optional local transcription with configurable model sizes
- CLI flags & options: control frame limits, models, output formats, and processing modes
- JSON schema templates: structured output format for frame metadata and transcription segments
Who It's For
- Video editors — quickly understand video structure and content before detailed editing
- Content creators — extract key moments and auto-generate captions locally
- Researchers — analyze video datasets offline without API costs or privacy concerns
- Accessibility specialists — transcribe video audio to create subtitles and transcripts
- Post-production teams — preview shot sequences and identify editing points visually
Best For
- Transcribing video interviews and podcasts to searchable text
- Extracting thumbnail candidates and key frames for previews
- Analyzing shot composition and scene transitions
- Creating subtitle files from video audio tracks
- Cataloging video library content without external dependencies
You might also like
You can transform raw video footage into structured, professional content by leveraging AI to identify key moments, organize clips, transcribe audio, and augment with voiceovers, overlays, and subtitles. This skill guides you through a proven pipeline that emphasizes compression and smart structuring rather than AI generation—turning hours of footage into minutes of compelling content optimized for any platform.
by dliangthinks
You'll develop systematic viral video strategies grounded in platform algorithms and audience psychology rather than guesswork. Analyze trending patterns, audience demand signals, and engagement metrics to create content calendars designed for maximum views, shares, and monetization. Move from reactive content creation to proactive strategic planning that justifies every creative decision with data.
by dliangthinks
Systematically troubleshoot failing simulations by identifying root causes in solver instability, constraint errors, cache corruption, and performance bottlenecks. You'll receive targeted diagnostic workflows, parameter tuning strategies, and rebuild recommendations that save render farm resources and maintain deadline adherence. This skill acts as your technical partner when simulations produce artifacts, crash mid-cache, or consume excessive memory.
by dliangthinks
Upload your asset specifications, render bottlenecks, or performance metrics, and receive a comprehensive optimization roadmap. Claude analyzes polygon topology, texture memory footprint, shader complexity, and LOD strategies specific to your target engine—whether game engines, live VFX systems, or AR/VR pipelines. You'll get concrete recommendations for polycount reduction, UV layout optimization, texture atlasing, and bake-versus-real-time lighting decisions, complete with documentation for handing off to animation and rigging teams.
by dliangthinks
Plan comprehensive grade structures for multi-scene sequences, extract and analyze color palettes from reference imagery, generate technical LUT specifications for custom creation, and validate your grades against broadcast and cinema standards (ITU-R BT.709, DCI-P3, Rec.2020). This skill handles the analytical and planning work that precedes timeline color correction, helping you make faster aesthetic decisions and ensure technical compliance across projects.
by dliangthinks
You'll design dynamic audio systems that shift music and sound effects in real-time based on gameplay intensity, player decisions, environmental conditions, and narrative branching. This skill guides you through building adaptive mixing strategies, stem-based music systems, and procedural audio routing that keeps audio responsive without becoming chaotic—allowing you to maximize asset reusability while maintaining sonic coherence across unpredictable gameplay states.
by dliangthinks
Transform raw video analytics into platform-specific optimization plans that boost reach and engagement. You'll identify why certain videos outperform others, uncover trending patterns in your niche, optimize metadata and technical specs for algorithmic distribution, and systematically test creative approaches. This skill bridges creative instinct with data-driven decision-making to help you scale social video performance.
by dliangthinks
This skill helps you architect complete color strategies before grading sessions begin. You'll generate detailed shot-by-shot grading maps that synchronize visual mood to dynamic audio, develop LUT strategies for specific music genres with documented creative reasoning, establish technical parameter ranges per scene or musical section, and create mood board documentation that aligns stakeholder expectations early. By treating color as a primary creative element rather than post-production cleanup, you reduce revision cycles and deliver grades that amplify both narrative and musical intent.
by dliangthinks