RehearsalNeRF: Decoupling Intrinsic Neural Fields of Dynamic Illuminations for Scene Editing

Changyeon Won1*, Hyunjun Jung1*, Jungu Cho1,3, Seonmi Park1, Chi-Hoon Lee3, Hae-Gon Jeon2†
1Gwangju Institute of Science and Technology, 2Yonsei University, 3CJ Corporation
*Equal contribution, †Corresponding author
IJCV 2026
arXiv Paper Code

RehearsalNeRF decomposes scenes into static objects, dynamic objects, and dynamic illumination, enabling independent control of each component.

Abstract

Although there has been significant progress in neural radiance fields, an issue on dynamic illumination changes still remains unsolved. Different from relevant works that parameterize time-variant/-invariant components in scenes, subjects' radiance is highly entangled with their own emitted radiance and lighting colors in spatio-temporal domain. In this paper, we present a new effective method to learn disentangled neural fields under the severe illumination changes, named RehearsalNeRF. Our key idea is to leverage scenes captured under stable lighting like rehearsal stages, easily taken before dynamic illumination occurs, to enforce geometric consistency between the different lighting conditions. In particular, RehearsalNeRF employs a learnable vector for lighting effects which represents illumination colors in a temporal dimension and is used to disentangle projected light colors from scene radiance. Furthermore, our RehearsalNeRF is also able to reconstruct the neural fields of dynamic objects by simply adopting off-the-shelf interactive masks. To decouple the dynamic objects, we propose a new regularization leveraging optical flow, which provides coarse supervision for the color disentanglement. We demonstrate the effectiveness of RehearsalNeRF by showing robust performances on novel view synthesis and scene editing under dynamic illumination conditions.

Method

RehearsalNeRF decomposes scenes into three independent neural fields: Static Field (S) for time-invariant geometry, Dynamic Field (D) for space-time motion, and Lightning Field (L) for illumination and hue conditioning. These fields are composed via BRDF interaction and tone mapping to produce the final rendering.

Method Overview

Learning Illumination Vector

(a) The illumination vector vh is initialized by selecting top-k hue candidates from the difference between main stage and rehearsal stage frames via k-means clustering in HSV space. (b) During training, the lighting field ψL predicts a probability vector PH at each timestep τ, which is multiplied with vh to produce the illumination color. The illumination vector is jointly optimized through back-propagation by matching the color of decoupled dynamic objects with the rehearsal prior.

Learning Illumination Vector

Dataset: Main & Rehearsal

Each scene consists of a main stage video (dynamic lighting, left) and a rehearsal video (stable lighting, right).

Applications

Novel View Synthesis

Scene Decomposition

RehearsalNeRF decomposes each scene into its constituent components: ground truth (top-left), dynamic illumination (top-right), dynamic objects (bottom-left), and static objects (bottom-right).

Color Change

By modifying the illumination vector, RehearsalNeRF can change the lighting color of the scene.

Intensity Change

RehearsalNeRF supports spatially varying intensity control, adjusting the brightness of the rendered image.

Time Control (Frozen Motion)

RehearsalNeRF allows independent time control over lighting and motion.

Comparisons

Quantitative Results

Novel view synthesis performance on our real-world dataset. We compare results from the original dataset and the temporally subsampled version (every 5th frame) to evaluate robustness against rapid scene dynamics.

Method Original Dataset Subsampled Dataset (5x Speed)
Full PSNR↑ Dynamic PSNR↑ Objects SSIM↑ Full PSNR↑ Dynamic PSNR↑ Objects SSIM↑
D2NeRF 29.776 22.962 0.978 28.274 21.899 0.976
K-Planes 28.113 25.122 0.981 27.849 25.784 0.986
TensoIR PBR 18.681 16.934 0.972 16.955 15.347 0.971
TensoIR VR 28.663 21.918 0.978 28.389 23.238 0.981
4DGaussian 28.199 24.523 0.984 28.189 24.245 0.983
Ours 28.250 26.318 0.986 28.666 27.090 0.988

Comparison with Previous Works: Scene Editing & Decomposition

The goal of scene editing is to render the red-colored illumination. Only our method produces the high-quality edited scenes through the successful dynamic illumination decomposition.

Qualitative comparison of scene editing and decomposition

Qualitative Comparison: Novel View Synthesis

Comparison of novel view synthesis performance for dynamic objects across methods.

Dynamic objects comparison across 5 frames

Dynamic Region Quality

Error map comparison for dynamic regions. Our model consistently achieves superior quality with the lowest error in dynamic regions.

Dynamic region quality comparison with error maps

Without Rehearsal Prior

Without rehearsal stage videos, our pipeline enables scene editing by generating the rehearsal prior. (Nano banana is used)

Generated rehearsal prior pipeline

BibTeX

@article{won2026rehearsalnerf,
  title={RehearsalNeRF: Decoupling Intrinsic Neural Fields of Dynamic Illuminations for Scene Editing},
  author={Won, Changyeon and Jung, Hyunjun and Cho, Jungu and Park, Seonmi and Lee, Chi-Hoon and Jeon, Hae-Gon},
  journal={International Journal of Computer Vision},
  volume={134},
  number={6},
  pages={257},
  year={2026},
  publisher={Springer}
}