Everyframe · engine
Video pipelines that compile.
Everything on this site runs on one engine: a typed pipeline compiler with 222 operations, 17 AI models and 4 runtimes. Each graph is validated before a single frame is decoded.
redact.ts
import { graph } from '@jtdigital/renderbox-sdk';
// Read a clip, find every face, blur them, write the result.
const g = graph();
const [video, audio] = g.open('uploads/crowd.mp4');
const faces = video.detect({ model: 'retinaface_mv2' });
g.write('renders/redacted.mp4', video.redact(faces, { mode: 'blur', radius: 25 }), audio);
// g.json() is the job description. Submit it and the engine runs it.Two SDKs, one typed graph
Describe pipelines in TypeScript or Rust. Both build the same dataflow graph, and both catch invalid wiring while you type: a video filter never accepts an audio stream.
pipeline.rs
use renderbox_sdk::{input, Stream, Video, Audio};
use renderbox_sdk::ops::video::scale;
use renderbox_sdk::ops::audio::volume;
let (v, a) = input("clip.mp4"); // (Stream<Video>, Stream<Audio>)
let v = v.pipe(scale(1280, 720)); // Video -> Video
let a = a.pipe(volume(0.8)); // Audio -> Audio
// v.pipe(volume(0.5)); // compile error: sort mismatchTypeScript SDK
A fluent graph builder on npm. Open an input, chain 222 typed operations, write the output. Your code produces a job description; the engine renders it.
Rust SDK
Phantom-typed Stream<S> pipelines. Codec, container and media sort live in the type system, so invalid pipelines are compile errors, not runtime crashes.
The pipeline you write is not the pipeline that runs
The compiler analyzes the dataflow graph, fuses operations, eliminates dead nodes and emits an optimized dispatch plan for each runtime. Eleven optimization passes in total; the five that matter most:
Filter fusion
Adjacent compatible filters merge into a single filtergraph node, eliminating intermediate buffer copies.
Identity elimination
No-op nodes (scale to the same size, volume at 1.0) are removed before any frame is decoded.
Scale distribution
Resolution changes propagate through the graph so downstream nodes operate on smaller frames.
Dead node removal
Unreachable subgraphs are pruned. If a branch produces no output, it never executes.
Input deduplication
Multiple references to the same source file share a single decode pass.
One pipeline, four runtimes
A single pipeline spans FFmpeg, ONNX Runtime, OpenCV and native Rust. The executor topologically sorts segments, wires streaming pipes between them and dispatches each to the right runtime, with zero temp files.
FFmpeg
Decode, encode and filter. The compiler emits optimized filtergraphs with fused operations and minimal temp files.
ONNX Runtime
GPU-accelerated inference for all 17 AI models. Frames stream directly from the decoder, with no disk I/O in between.
OpenCV
Vision effects that need per-pixel control: blur, pixelate, annotate and depth-based compositing over a multiplexed stdin protocol.
Native Rust
In-process functions for metadata, hashing, format conversion and orchestration logic that needs no external process.
17 models, first-class operations
Detection, segmentation, depth, pose, OCR, scene classification, transcription and density estimation. Every model is a typed pipeline operation, not a bolted-on integration.
Detection
Face, object, license plate and text detection with configurable confidence thresholds and class filtering.
Segmentation
Instance and semantic segmentation for pixel-accurate masks: background removal, subject isolation, scene parsing.
Depth estimation
Monocular depth maps from single frames. Powers depth-based compositing, parallax effects and 3D-aware blur.
Pose estimation
Skeleton keypoint detection for body and hand tracking. Enables motion analysis, gesture recognition and pose-driven effects.
OCR & scene text
Text recognition in video frames. Extracts on-screen text, subtitles, signage and document content.
Scene & audio
Scene classification, shot boundary detection and speech transcription: structural analysis for the full pipeline.
Compilation ends in a seal
Every pipeline compiles to a sealed execution plan, content-addressed by its SHA-256. That plan id is what an export receipt cites, so what ran is provable, not asserted. The receipt also names every AI model that touched the footage, identified by the hash of its weights.
Three steps to a running pipeline
The SDKs and the showcases on this site drive the engine through the same door. Nothing on this page is a second system.
Describe
Write the pipeline in the TypeScript SDK or the Rust SDK. Both produce the same typed graph.
Compile
The compiler type-checks every connection, runs its optimization passes and seals the plan.
Execute
The executor dispatches segments across FFmpeg, ONNX Runtime, OpenCV and native Rust.
Stop wiring. Start shipping.
Pick an SDK, describe your first pipeline and let the engine do the rendering.