Nerfstudio Contributions

Since Jan 2023 I have made contributions to the Nerfstudio API system including some features and small improvements to other parts. I am also acknowledged in the Nerfstudio SIGGRAPH paper and my Blender add-on is mentioned during presentations of Nerfstudio.

Nerfstudio Blender VFX Add-on

I created a Blender add-on that allows NeRFs to be used in visual effects. This enables a pipeline for integrating NeRFs into traditional compositing VFX pipelines using Nerfstudio. This approach leverages Blender, a widely used open-source 3D creation software, to align camera paths and composite NeRF renders with meshes and other NeRFs, allowing for seamless integration of NeRFs into traditional VFX pipelines. It allows for more controlled camera trajectories of photorealistic scenes, compositing meshes and other environmental effects with NeRFs, and compositing multiple NeRFs in a single scene. This approach of generating NeRF aligned camera paths can be adapted to other 3D tool sets and workflows, enabling a more seamless integration of NeRFs into visual effects and film production. This also supports Nerfstudio gaussian splatting as well.

The exported mesh or point cloud representation is imported into Blender and a render camera path is generated by transforming the coordinate space of the NeRF scene to the Blender virtual camera, allowing aligned camera paths.

I created documentation for it here and a tutorial video demonstrating basic examples using the add-on as well as a breakdown of other effects that can be done with it.


I have also written a technical write-up on Arxiv here.

🥽 VR Video Rendering

I implemented VR180 and VR360 (Omnidirectional stereo) render cameras to support VR video rendering. This allows users to render stereo equirectangular videos to view on VR headsets or post on YouTube. Documentation is here. The Blender add-on is used to create the final render path and to correctly scale the NeRF to real-world size since the scanned NeRF coordinate system is arbitrary.

Additional contributions

I have also made smaller contributions towards Nerfstudio including towards Viser, the new 3D viewer that Nerfstudio uses, as well as adding Nerfstudio support for Instruct-GS2GS. I was part of a group that implemented equirectangular image/video input support for Nerfstudio datasets.