Introduction to Renku

Renku is a workflow orchestration system for generating long-form video content using AI. Think of it as a build system for video production—like Make or Bazel, but for AI-generated multimedia content.

What is Renku?

Renku coordinates multiple AI providers (OpenAI, Replicate, fal.ai, ElevenLabs, and more) to produce narrated documentaries, educational videos, and other multimedia content from simple text prompts.

From a single inquiry like “Explain the water cycle for middle school students,” Renku can:

1. Generate a structured script with narration segments
2. Create image or video prompts for each segment
3. Generate visual content (images, videos) for each segment
4. Synthesize voiceover audio for the narration
5. Compose background music
6. Assemble everything into a playable timeline
7. Export the final video

Problems Renku Solves

Manual Provider Juggling

Without Renku, creating AI-generated video means:

• Going to ChatGPT to generate a script
• Copying prompts to Midjourney or DALL-E for images
• Using ElevenLabs or another service for voiceover
• Downloading everything manually
• Stitching it together in video editing software

Renku automates this entire pipeline with a single command.

Specialized Prompting

Each AI model works best with specific prompting techniques. Renku uses specialized prompt templates per model, ensuring you get optimal results without becoming an expert in every provider’s quirks.

No State Tracking

Hand-built scripts regenerate everything from scratch. Changed the voiceover style? You’d rebuild all audio, even segments that didn’t change.

Renku tracks the state of every artifact. It knows which inputs changed and only regenerates what’s needed—saving time and API costs.

Serial Execution

Generating 10 video segments one after another is slow. Renku analyzes dependencies and runs independent tasks in parallel across execution layers.

Hard to Extend

Adding a new AI model to a custom script means rewriting code. Renku’s producer architecture lets you add new models through YAML configuration.

Architecture Overview

┌─────────────────────────────────────────────────────────────────────┐
│                           Blueprint YAML                             │
│  (Defines workflow: inputs, producers, connections, collectors)      │
└─────────────────────────────────────────────────────────────────────┘
                                   │
                                   ▼
┌─────────────────────────────────────────────────────────────────────┐
│                              Planner                                 │
│  • Loads blueprint tree (blueprints + producers)                    │
│  • Resolves dimensions and expands loops                            │
│  • Builds execution graph with canonical IDs                        │
│  • Creates layered execution plan                                   │
└─────────────────────────────────────────────────────────────────────┘
                                   │
                                   ▼
┌─────────────────────────────────────────────────────────────────────┐
│                              Runner                                  │
│  • Executes jobs layer by layer                                     │
│  • Resolves inputs from upstream artifacts                          │
│  • Materializes fan-in collections                                  │
│  • Invokes AI providers via producer implementations                │
│  • Stores artifacts and logs events                                 │
└─────────────────────────────────────────────────────────────────────┘
                                   │
                                   ▼
┌─────────────────────────────────────────────────────────────────────┐
│                            Providers                                 │
│  • OpenAI (text generation, structured output)                      │
│  • Replicate (video, audio, image models)                          │
│  • fal.ai (video, audio, image models)                             │
│  • Renku (timeline composition, video export)                       │
└─────────────────────────────────────────────────────────────────────┘

Typical Video Generation Workflow

1. Script Generation: An LLM generates a narration script divided into segments
2. Prompt Generation: The LLM creates visual prompts for each segment
3. Media Generation: AI models generate video clips, images, or audio per segment
4. Timeline Composition: Renku assembles segments into a playable timeline
5. Video Export: Remotion renders the final video file

Key Concepts

Understanding these concepts will help you work effectively with Renku:

Blueprint

A YAML file that defines your complete video generation workflow. Blueprints specify:

• What inputs the user provides (topic, duration, style, etc.)
• What artifacts are produced (script, images, audio, video)
• Which producers to use and how to connect them
• How to iterate over segments (loops)

Blueprints are the “recipe” for your video.

Producer

A reusable module that invokes AI models. Producers:

• Accept inputs (prompts, configuration)
• Call one or more AI provider APIs
• Produce artifacts (text, images, audio, video)

Examples: ScriptProducer, VideoProducer, AudioProducer, TimelineComposer

Artifact

Any output produced by a producer:

• Text (scripts, prompts)
• Images (PNG, JPEG)
• Audio (MP3, WAV)
• Video (MP4)
• JSON (timelines, metadata)

Artifacts can be scalar (single value) or arrays (one per segment).

Loop

A dimension that enables parallel execution. If you have 5 narration segments, a loop with countInput: NumOfSegments creates 5 instances of any producer assigned to that loop.

Connection

Data flow edges that wire outputs to inputs. Connections define how artifacts flow from one producer to another:

• Blueprint input → Producer input
• Producer artifact → Another producer’s input
• Producer artifact → Blueprint artifact (final output)

Layer

An execution layer contains jobs that can run in parallel. The planner uses topological sorting to determine which jobs depend on others and groups independent jobs into the same layer.

Workspace

The directory structure where Renku stores:

• Blueprint catalog
• Build outputs (artifacts, manifests, logs)
• Configuration

What’s Next?

Ready to try Renku? Continue to the Quick Start guide to install Renku and generate your first video.