ECCV 2026

Large-Scale Light Field Synthesis from Videos

Enables Geometrically Consistent Bokeh Editing

Haoming Cai1,†   Zhoutong Zhang2,†   Christopher Metzler1   Shumian Xin2

1University of Maryland  ·  2Adobe  ·  †Equal contribution

TL;DR Bokeh training data is either hard to scale (hardware capture) or corrupted by estimated depth (synthetic rendering). Our principle: render bokeh supervision from light fields reconstructed out of casual video — faithful and scalable at once.

Easy bokeh data collection. Bokeh supervision is rendered from a light field synthesized by a feed-forward 3D model. How it works
Geometrically faithful bokeh supervision, at 100K-scene scale. Every training sample is integrated from a dense 625-view grid, making it robust to reconstruction errors. Is it faithful?
Better data alone makes a better editor. A plain diffusion-model fine-tune with no depth branch, trained on Vid2Bokeh, stays geometrically consistent where depth-based editors fail. See Results
Paper soon Video soon Resources
Wide-aperture focus sweep through a transparent lid — rendered from a light field reconstructed out of one casual video.
1 · Motivation2 · Pipeline3 · Dataset4 · Data Validation5 · Editing Result

Where depth-based bokeh breaks

Bokeh editors consistently fail on complex geometry — reflections, thin structures — exactly where depth estimation struggles. The cause is the training data: captured datasets are accurate but non-scalable; synthetic ones scale but render labels through estimated depth, inheriting its errors.

Depth estimation fails on reflective and see-through surfaces, and the errors pass straight into the rendered bokeh.

Claim The bottleneck is data, not architecture: retraining BokehDiff on Vid2Bokeh, architecture unchanged, improves it across the board.

1 · Motivation2 · Pipeline3 · Dataset4 · Data Validation5 · Editing Result

The Vid2Bokeh Data Pipeline

Casual video → 25×25 light field → bokeh at any focus and aperture. All three clips below come from the same handheld video.

01Casual video

A handheld walk-around on an ordinary camera. No rig, no light-field camera, no per-focus DSLR capture.

0225×25 light field

Feed-forward 3D synthesizes 625 sub-aperture views in one pass — under 30 s on a single A100.

03Bokeh on the fly

Integrating over the light field gives any focus and aperture — occlusion correct, no depth map involved.

A feed-forward 3D model (LaCT) lifts each clip into a 25×25 light field around selected key frames; bokeh pairs render directly from it — no depth map anywhere in label generation.

The data generation pipeline. A single video becomes a dense 25×25 light field; bokeh renders on the fly at arbitrary focal plane κ and aperture radius a, giving supervision pairs (κ1, a1) → (κ1+Δκ, a1+Δa).

Try it: render bokeh at any (κ, a)

Rendered by the dataset pipeline (not model outputs); previews are downscaled.
Rendered from the light field of a single handheld clip
Loading…
Focal plane κ
Far Near
Aperture radius a
Small Large
1 · Motivation2 · Pipeline3 · Dataset4 · Data Validation5 · Editing Result

The Vid2Bokeh Dataset

Built on DL3DV: several key frames per video each yield a 25×25 light field — about 10K videos become 100K+ light fields, with continuous (κ, a) sampling.

Method Editing Space Training Dataset Data Generation
FocusDoFNameScenes Core PhilosophyDepth-Est-FreeScalable
DC²✓✓–100Physical Capture (Multi-Cam)
Tedla et al.✓Focal Stack1,637Physical Capture (Multi-Cam)✓
DPDNet↑DPDD500Physical Capture (Multi-Cam)✓
AiFNet↑LFDOF100Physical Capture (Light Field)✓
Bokehlicious↓RealBokeh4.4kPhysical Capture✓
RVR-LAAF✓↓Aperture2,942Physical Capture✓
PyNET↓EBB!5kPhysical Capture✓
BokehDiff↓––Compositional Synthesis✓
BokehMe↓–150Compositional Synthesis✓
DiffCamera✓✓–20kDepth-based Hybrid Synthesis✓
GenRefocus✓✓––Depth-based Hybrid Synthesis✓
Vid2Bokeh (Ours)✓✓Vid2Bokeh100kSynthetic Light Field from Video✓✓

Vid2Bokeh is the only pipeline that is simultaneously depth-estimation-free and scalable. ↑/↓ in DoF: one-directional blur change; ✓: full bidirectional control supporting blurry inputs.

Rows shift κ, columns grow a; any two cells form a supervision pair.
1 · Motivation2 · Pipeline3 · Dataset4 · Data Validation5 · Editing Result

Data Validation

The light field is synthesized, so we validate the rendered bokeh directly — against Blender path-traced ground truth, and side by side on real captures.

A toy behind a transparent lid; cables in front of far text. Depth collapses each to a single surface; light-field integration keeps the see-through structure.
Against Blender path-traced ground truth, bokeh from our synthesized light field closely matches the reference (quantitative table in the paper).

Side by side on a real capture

Different parameterizations on the two sides — match them by eye, not by number.
Near all-in-focus view from the captured video
Near AiF input
Estimated depth map (DepthAnything V3)
Depth map
Ours — light-field integration
Our bokeh rendering
Aperture radius aa = 06.00
SmallLarge
Focal plane κκ = +0.000
FarNear
BokehDiff — depth-based rendering
Depth-based bokeh
Blur radiusK = 18.0
SmallLarge
Focal distancevar = +0.000
FarNear
1 · Motivation2 · Pipeline3 · Dataset4 · Data Validation5 · Editing Result

Editing Results

A plain diffusion model fine-tuned on Vid2Bokeh. Input: one photograph plus offsets (Δκ, Δa); output: the re-rendered photograph at the new focus and aperture — in either direction.

Against a depth-based baseline

Near all-in-focus input
Near AiF input
BokehDiff — depth-based
BokehDiff rendering
Blur radiusK = 8.0
SmallLarge
Focal distancevar = 0.00
FarNear
Ours — trained on Vid2Bokeh
Our bokeh rendering
Δ Aperture radiusΔa = +0.00
SmallLarge
Δ Focal planeΔκ = +0.00
FarNear

Note the two sides are parameterized differently: ours takes relative offsets from the input's own optical setting, the baseline takes absolute blur radius and focal distance.

Bidirectional joint focus–DoF editing

Δκ is sensitive: when the input has a sparse depth distribution, most positions push the whole scene out of focus. The dot marks the input's own setting.
Input
Our editing result
Δκ — focal plane shift
Toward background
Input's focal plane
Toward foreground
Δa — aperture radius change
Less blur
Input's aperture
More blur

After the argument · Limitations

What this does not do yet

After the argument · Citation

BibTeX

@inproceedings{cai2026vid2bokeh, title = {Large-Scale Light Field Synthesis from Videos Enables Geometrically Consistent Bokeh Editing}, author = {Cai, Haoming and Zhang, Zhoutong and Metzler, Christopher and Xin, Shumian}, booktitle = {European Conference on Computer Vision (ECCV)}, year = {2026} }