Cyrus Vachha

We present a Unity toolkit for authoring multi-layered XR experiences with 3D Gaussian Splatting It provides building blocks for a renderer for standalone AR/VR devices, splat-to-mesh reconstruction, splat physics, splat blending, and offloaded on-headset splat object reconstruction. We validate it through demonstrative applications and a developer study.

A VR-based world building tool for editing radiance field scenes using generative AI in an immersive, real-time interface through a modular architecture that combines direct manipulation with different levels of abstraction and controllability interfacing with generative image and 3D models, offering workflows for creating and modifying 3D Gaussian Splatting scenes and staging scenes for video generative models in real-time using proxy representations.

We propose a method for editing 3DGS scenes with text-instructions in a method similar to Instruct-NeRF2NeRF. Given a 3DGS scene of a scene and the collection of images used to reconstruct it, our method uses an image-conditioned diffusion model (InstructPix2Pix) to iteratively edit the input images while optimizing the underlying scene, resulting in an optimized 3D scene that respects the edit instruction. We demonstrate that our proposed method is able to accomplish more realistic, targeted edits than prior work.

We present a pipeline for integrating NeRFs into traditional compositing VFX pipelines using Nerfstudio, an open-source framework for training and rendering NeRFs. Our approach involves using Blender to align camera paths and composite NeRF renders with meshes and other NeRFs, allowing for seamless integration of NeRFs into traditional VFX pipelines. Shown in the CVPR 2023 Art Gallery. View more NeRF VFX renders (here)

In this work, we identify problems associated with interaction carried out with large groups of users. To address this, we introduce an additional user role: the co-host. They mediate communications between the VR user and many spectators. To facilitate this mediation, we present StreamFunnel, which allows the co-host to be part of the VR application's space and interact with it.

We propose WebTransceiVR, an asymmetric collaboration toolkit which when integrated into a VR application, allows multiple non-VR users to share the virtual space of the VR user. It allows external users to enter and be part of the VR application's space through standard web browsers on mobile and computers. WebTransceiVR also includes a cloud-based streaming solution that enables many passive spectators to view the scene through any of the active cameras. We conduct informal user testing to gain additional insights for future work.

Datasets for radiance field training (3DGS and NeRFs) of dozens of captures primarily of buildings of UC Berkeley and Princeton University, including exteriors and interiors. These datasets are composed of captures of videos (iPhone), images (captured via Sony a7iii), and fisheye images (Insta360 X4 and Samsung Gear360). Online VR splat gallery coming soon (preview).