Huy's Wiki

claude-code-video-editing

8 min read

Editing Videos with Claude Code — Complete Guide

A synthesis of everything in this wiki about using Claude Code for video editing and production.

Why Claude Code for Video?

Claude Code transforms video editing from a manual timeline operation into a programmable pipeline. Instead of clicking and dragging, you describe what you want in natural language — Claude Code writes the code that builds, edits, or assembles the video.

The key insight: Claude Code is not a video editor. It’s an automation layer that orchestrates the real tools (FFmpeg, Remotion, HyperFrames, Whisper, CapCut project files) into a repeatable, version-controlled workflow.

The Landscape

Three Eras of AI Video

EraWhat It MeansExample
AI-assistedClaude helps with scripts, captions, cutsClaude + CapCut manual workflow
AI-automatedClaude Code programs the pipelineSandy Lee’s fully automated pipeline
AI-generatedOne prompt → finished video, no editingHyperFrames + HeyGen agents

All three are active today. Which one you use depends on how much control you need.

The Major Approaches

1. HyperFrames (HeyGen) — Agent-Native Video Creation

Best for: Creating videos from scratch, text-to-video, programmatic motion graphics

HyperFrames is HeyGen’s open-source HTML-based video framework built for AI agents. Claude Code writes HTML with GSAP animations, and HyperFrames renders it to video.

How it works:

Claude Code → writes HTML → HyperFrames renders → finished video

Key features:

  • Frame-level control via HTML + JavaScript
  • GSAP timelines, CSS animations, Web Animations API, Anime.js, Three.js, Lottie
  • Scene transitions, captions, audio-reactive visuals, text-to-speech
  • Deterministic seek-driven rendering (each frame is reproducible)
  • Tailwind CSS v4 browser runtime support

The pipeline:

  • npx hyperframes init — scaffold a project
  • npx hyperframes preview — preview in browser
  • npx hyperframes render — render to MP4
  • npx hyperframes lint/inspect — validate compositions

Installed skills (16 total): hyperframes, hyperframes-cli, hyperframes-media, hyperframes-registry, website-to-hyperframes, remotion-to-hyperframes, plus engine adapters (gsap, animejs, waapi, css-animations, lottie, three, tailwind, typegpu).

Sources:

2. Remotion — Programmatic Motion Graphics

Best for: Complex motion graphics, data-driven videos, branded templates

Remotion is a React-based video framework. Claude Code generates React components that render to video frames. Each frame is a React component at a point in time.

The pipeline:

Claude Code → writes React components → Remotion renders frame-by-frame → MP4

Common use cases with Claude Code:

  • Animated title cards and lower thirds
  • Data visualization videos (charts, graphs change over time)
  • Social media ads with brand templates
  • Automated video series (same format, different content each time)

The Claude + Remotion + CapCut stack (Matt Loui’s workflow):

  1. Claude Code generates Remotion motion graphics (with transparent/alphas)
  2. Export as ProRes with alpha channel
  3. Layer into CapCut timeline over screen recording or footage

Sources:

3. Claude + CapCut — Free Tier Power

Best for: Social media content, quick edits, no-code motion graphics

CapCut (ByteDance’s free editor) is the most accessible video editor on the market. Claude enhances it in several ways:

Workflow 1 — Claude plans, CapCut executes:

  • Claude writes scripts, storyboards, effect sequences
  • You import into CapCut and assemble

Workflow 2 — Claude edits CapCut project files directly:

  • CapCut project files are accessible (XML/JSON)
  • Claude Code manipulates them programmatically
  • Opens CapCut to find the timeline already edited

Workflow 3 — Claude generates CapCut assets:

  • Premium motion graphics parameters (no coding needed)
  • SRT caption files with precise timestamps
  • LUTs and color grading parameters

Workflow 4 — Claude + Remotion graphics into CapCut:

  • Claude generates Remotion motion graphics with alpha channels
  • Import as overlay into CapCut timeline

Sources:

4. Claude + DaVinci Resolve — Professional Tier

Best for: Professional post-production, color grading, long-form content

DaVinci Resolve’s AI Neural Engine handles the grunt work. Claude adds orchestration and automation.

DaVinci Resolve AI features:

  • Facial Recognition — auto-tracks faces for color grading
  • Object Detection — track objects without manual keyframes
  • Smart Reframing — auto-crop for vertical/landscape repurposing
  • Speed Warp — AI-optimized slow-motion
  • Super Scale — up-res SD/HD to 4K/8K
  • Auto Color / Color Match
  • Voice Isolation — separate dialog from noise

Claude integration points:

  • DaVinci Resolve plugins that bring Claude into the editing timeline
  • Transcript-based editing — Claude analyzes transcript, generates edit decision list
  • Claude scripts grading parameters, generates Fusion compositions

Sources:

5. Claude + Video Generation Models

Best for: UGC ads, faceless channels, high-volume content

Claude Code orchestrates video generation models (Seedance, Kling, Sora, etc.) into a production pipeline.

Claude + Seedance 2.0 pipeline:

Claude Code → writes script → Seedance generates video → Claude assembles final cut

Seedance 2.0 (ByteDance) excels at UGC-style video — authentic-looking user-generated content ads. Native 9:16 vertical, synchronized audio, lip-sync support.

Claude Code as orchestrator:

  • Generates the script and shot list
  • Calls Seedance API for each shot
  • Assembles shots into final video
  • Optionally adds captions, transitions, music

Sources:

6. Tella — Screen Recording + Claude Editing

Best for: Tutorials, screencasts, educational content

Tella (screen recorder) + Claude Code = automated post-production for recordings.

Tella’s workflow:

  1. Record in Tella (screen + camera + mic)
  2. Claude Code takes over: removes silences, filler words, mistakes
  3. Claude discovers and places relevant memes/images from your files
  4. Claude generates Remotion motion graphics (titles, lower thirds)
  5. Claude sources and places sound effects
  6. Finished video — no manual timeline work

The key differentiator: Claude understands semantic context for cuts, not just silence trimming.

Source: Tella — I Use Claude to Edit Videos

7. Claude Design for Video Assets

Best for: Generating visual assets (lower thirds, transitions, motion graphics)

Claude Design (claude.ai/design) can generate motion graphics as HTML video with green screen backgrounds, which you then:

  1. Export from Claude Design
  2. Pass to HyperFrames for rendering
  3. Drop into Premiere/CapCut with Ultra Key to remove green screen

Prompt pattern:

“Use motion graphics to create assets in a single playable video with a green screen background. Animate everything in and out. Include a lower third title, a ‘what we’re covering’ card, and a wipe transition.”

Sources:

8. Brad Bonanno’s claude-video — Make Claude “Watch” Videos

Best for: Video analysis, Q&A, content repurposing, fact-checking

Not editing per se, but a critical companion skill. The open-source claude-video skill gives Claude the ability to “watch” any video by combining frames + timestamped transcript.

/watch https://youtube.com/watch?v=... → Download → Extract frames → Transcribe → Claude analyzes

Use cases:

  • Ask questions about any video
  • Tutorial review — extract every command from a coding tutorial
  • Meeting analysis — get minutes + decisions from a recording
  • Content repurposing — video → blog post, Twitter thread, LinkedIn post

Repo: github.com/bradautomates/claude-video

Source: Brad Bonanno — Claude Code Video Watcher

The Minimal Tool Stack

Across all approaches, 3 tools dominate:

ToolRole
Claude CodeAutomation layer — scripts, orchestrates, generates
FFmpegCore video processing — cutting, merging, transcoding, frame extraction
WhisperTranscription — captions, timestamped transcripts, analysis

Everything else (Remotion, HyperFrames, CapCut, DaVinci) is a specialized renderer on top of this foundation.

Source: The ONLY 3 Tools You Need for AI Video

Key Patterns

Tool Coupling

Two free/cheap tools combined > one expensive pro tool. Examples:

  • Claude + CapCut > Adobe Premiere
  • Claude + Remotion > After Effects
  • Claude + Seedance > hiring a UGC creator

The Green Screen Pipeline

  1. Claude Design generates motion graphics with green background
  2. HyperFrames renders to video
  3. Premiere/CapCut removes green with Ultra Key/chroma key
  4. Result: professional overlays without manual animation

Programmatic Editing

If the editor’s project format is accessible (XML, JSON, DRP), Claude Code can manipulate it directly:

  • CapCut project files → Claude edits timeline
  • DaVinci Resolve project files → Claude adjusts cuts, grades
  • Remotion components → Claude generates React code → rendered to video

Sources in This Wiki

Full Transcripts & Detailed Workflows

Specific Techniques

Frameworks & Libraries

Analysis & Commentary

Video Generation

Professional Tools

Claude Video “Watch” Skill

Tags

#ClaudeCode #VideoEditing #HyperFrames #Remotion #CapCut #DaVinciResolve #Seedance #AI #ContentCreation #Automation

Graph View