OpenClaw and Remotion Pipeline for Automated Video Editing

A developer has detailed a practical automated video editing pipeline using OpenClaw for agent orchestration and Remotion as a React-based video renderer. The setup processes hundreds of raw video clips into finished social media Reels without manual editing.

Workflow Components

The stack consists of:

• OpenClaw – for agent orchestration and automation
• Remotion – React-based video editor/renderer
• Python + JSON glue code for montage generation

Processing Pipeline

The workflow follows these concrete steps:

1. Raw Ingestion: Videos from Telegram are dumped into a raw folder without manual renaming: remotion-lab/source-videos/<brand>/raw/

2. Filtering & Cleaning: A script filters clips to keep only vertical smartphone resolutions (720×1280, 1080×1920) and excludes junk like old campaign footage, AI/stock visuals, and unrelated brands. This produces a filtered/ folder and a catalog_filtered.json file listing usable videos.

3. JSON-defined Montages: Another script generates montage structures with categories like "Prep," "Baking," and "Final vibe" clips. It creates a JSON storyboard like:

[{"type": "video", "src": "filtered/file_001.mp4", "duration": 80},{"type": "video", "src": "filtered/file_037.mp4", "duration": 80}]

4. Remotion Composition: A single Remotion React composition reads montage_filtered.json and applies:

• TransitionSeries with fade transitions
• Ken Burns effect (subtle scale + pan) on all clips
• Original clip audio preserved
• Background music added underneath
• White screen + logo outro

Creative rules are encoded here: remove on-screen text, avoid AI/stock visuals, keep original audio, use crossfade transitions, and finish with a white fade and logo.

5. Render + Compress: OpenClaw triggers rendering via:

npx remotion render src/index.tsx StoryFinal out.mp4

Then compresses with ffmpeg:

ffmpeg -i out.mp4 -vcodec libx264 -crf 20 -preset slow -acodec aac -b:a 192k out_hq.mp4

This produces 9:16 vertical videos around 45 seconds long.

Batch Autopilot Mode

The agent handles batch generation with instructions like: "Generate 20 unique Reels, don't reuse clips between them." It:

• Tracks usage in a state.json file
• Loops through generating new montage_filtered.json using only unused clips
• Runs Remotion render and ffmpeg compression
• Creates send_ready_#.json files describing outputs
• OpenClaw sends finished videos to Telegram

The result: 20 different Reels (18-30s each) with consistent transitions, logo outros, and no off-brand footage, delivered overnight.

This approach works because agents handle the boring parts (cataloging, filtering, picking clips, batch rendering) while creative control remains centralized in one Remotion composition and JSON generator scripts.