Physically-based simulation for interactive audio-visual applications, particularly sound simulation for games and Virtual/Augmented Reality.
Particularly interested in projects with both research and practical impact, such as project Triton for modeling sound propagation that shipped in the Microsoft game title Gears of War 4, and past work on impact sound synthesis techniques employed in Crackdown 2. An important current research focus is to improve sound simulation in interactive augmented and virtual reality experiences.
Graduated with a B.Tech. from IIT Kanpur, India, in 2003 followed by a PhD in 2009 under the guidance of Prof. Ming Lin at UNC Chapel Hill. My thesis focussed on fast computational acoustic techniques combined with harnessing human auditory perception to enable sound synthesis and propagation in real interactive applications with limited computational budget.
My PhD work initiated sound research as a new direction in the UNC Gamma research group. The source code from my research served as a starting point for many students at UNC, and was also licensed by Microsoft, aiding further research within MSR.
In collaboration with The Coalition studio. Shipped Oct 11, 2016
Presented at GDC 2017. [Presentation Slides w/ videos](600MB)
Triton is a first-of-a-kind wave acoustics engine shipped in Gears of War 4. It automatically models sound wave propagation on full 3D game maps for moving sources and listener, while also providing the sound designer artistic control on the acoustics. This results in believable environmental effects that transition smoothly as sounds and player move through the world, such as scene-dependent reverberation, and smooth occlusion & obstruction effects.
I led research and development for the project, creating the precomputation tool chain and runtime acoustic engine. Research priorities and system architecture were set in close collaboration with the audio team at The Coalition. This partnership resulted in a practical system that integrates with industry-standard game technology. The underlying technology is described in our SIGGRAPH paper.
Triton is the first demonstration that full wave acoustics is now feasible for AAA games, opening up new opportunities for sound design and spatial audio in games and Augmented/Virtual Reality.
Andrew Allen and Nikunj Raghuvanshi
ACM Transactions on Graphics (SIGGRAPH), 34(4), July 2015
This paper describes the first real-time technique to synthesize full-audible-bandwidth sounds for 2D virtual wind instruments. The user is presented with a sandbox interface where they can draw any bore shape and create tone holes, valves or mutes. The system is always online, synthesizing sounds from the drawn geometry as governed by wave physics. Our main contribution is an interactive wave solver that executes entirely on modern graphics cards with a novel numerical formulation supporting glitch-free online geometry modification.
Nikunj Raghuvanshi and John Snyder
ACM Transactions on Graphics (SIGGRAPH), 33(4), July 2014
This paper presents a precomputed wave propagation technique that is immediately practical for games and VR. We demonstrate convincing spatially-varying effects in complex scenes including occlusion/obstruction and reverberation. The technique simultaneously reduces the memory and signal processing computation by orders of magnitude compared to prior wave-based approaches. The key observation is that while raw acoustic fields are quite chaotic in complex scenes and depend sensitively on source and listener location, perceptual parameters derived from these fields, such as loudness or reverberation time, are far smoother, and thus amenable to efficient representation.
This technology has shipped in Gears of War 4.
Ravish Mehra, Nikunj Raghuvanshi, Lakulish Antani, Anish Chandak, Sean Curtis, Dinesh Manocha
ACM Transactions on Graphics, 32(2), 2013
This paper explores using the Equivalent Source Method to compactly encode the sound field in outdoor scenes. We showed that in many outdoor scenes this could be a far more efficient encoding than storing the volumetric acoustic field directly.
Brandon Lloyd, Nikunj Raghuvanshi, Naga K. Govindaraju
ACM Symposium on Interactive 3D Graphics and Games (I3D), 2011
Video games typically store recordings of many variations of a sound event to avoid repetitiveness, such as multiple footstep sounds for a walking animation. We present a technique that can produce unlimited variations on an impact sound while usually costing about the same memory as a single clip. The main idea is to use an analysis-synthesis approach: a single audio clip is used to extract the resonant mode frequencies of the object and their time-decay, along with a fixed residual signal in time domain. The modal model is then amenable to on-the-fly variation and re-synthesis.
This technology shipped in the Xbox title Crackdown 2. In collaboration with Guy Whitmore and Kristofor Mellroth at Microsoft Game Studios.
Nikunj Raghuvanshi, John Snyder, Ravish Mehra, Ming C. Lin, and Naga K. Govindaraju
ACM Transactions on Graphics (SIGGRAPH), 29(3), July 2010
This paper presents the first technique for precomputed wave propagation on complex, 3D game scenes. It utilizes the ARD wave solver to compute acoustic responses for a large set of potential source and listener locations in the scene. Each response is represented with the time and amplitude of arrival of multiple wavefronts, along with a residual frequency trend. Our system demonstrates realistic, wave-based acoustic effects in real time, including diffraction low-passing behind obstructions, sound focusing, hollow reverberation in empty rooms, sound diffusion in fully-furnished rooms, and realistic late reverberation.
Nikunj Raghuvanshi, Rahul Narain and Ming C. Lin
IEEE Transactions on Visualization and Computer Graphics(TVCG), 15(5), 2009
We present a technique which relies on an adaptive rectangular decomposition of 3D scenes to enable efﬁcient and accurate simulation of sound propagation in complex virtual environments. It exploits the known analytical solution of the Wave Equation in rectangular domains, allowing the field within each rectangular spatial partition to be time-stepped without incurring numerical errors. The spatial partitions communicate using finite-difference-like linear operators, that do incur numerical error as weak artificial reflections. The use of analytic solutions allows this technique to provide reasonable accuracy at low spatial resolutions close to the Nyquist limit.
Nikunj Raghuvanshi and Ming C. Lin
ACM Symposium of Interactive 3D Graphics and Games (I3D), 2006
We present various perceptually-based optimizations for modal sound synthesis that allow scalable synthesis for virtual scenes with hundreds of sounding objects.
Naga K. Govindaraju, Nikunj Raghuvanshi and Dinesh Manocha
ACM SIGMOD 2005
We present algorithms for fast quantile and frequency estimation in large data streams using graphics processor units (GPUs). We exploit the high computational power and memory bandwidth of graphics processors and present a novel sorting algorithm that performs rasterization operations on the GPUs.