SIGGRAPH Researchers Debate Gaussian Splatting as VR Rendering Path
The artificial intelligence keynotes at SIGGRAPH this week attract large crowds. Neural rendering demonstrations draw audiences standing three deep. Headset manufacturers hint at products shipping later this year. All of these developments matter. Yet people seeking insight into the actual appearance of virtual reality in the coming years may wish to skip the marquee stage and locate Room 518 on Tuesday morning. Researchers and engineers will spend ninety minutes in that room debating a rendering technique with an unglamorous name: Gaussian splatting.
For decades, everything visible in a game or virtual reality application has been constructed from polygons. Millions of tiny triangles are stitched into meshes, wrapped in textures, and lit by hand. This approach works, yet it is slow to produce and always appears more like a model of the world than the world itself. Gaussian splatting discards that pipeline entirely. Rather than triangles, a scene is stored as a cloud of soft, coloured blobs. Each blob is a fuzzy three-dimensional dot, termed a Gaussian after the statistical distribution, with a position, a colour, a size, and a transparency. Stack several million of them together, and the result looks less like a video game and more like a photograph one can walk around inside.
The technique has become prominent because those blobs are not modelled by hand. They are captured. Point a mobile phone at an object, take a few dozen photographs or a short video, and software solves for the cloud of Gaussians that reproduces what the camera observed. A weekend project can transform a living room into a scene viewable in a headset. This represents a very different economy from paying an artist to build the same room one triangle at a time.
The session in Room 518 is not primarily concerned with that aspect, however. It is built around the jump from three-dimensional to four-dimensional representation, and the fourth dimension is time. A static splat of a living room is a noteworthy accomplishment. A splat that moves, a person talking, a dancer, a chef cooking, captured as a volume one can walk around rather than a flat video, constitutes a genuinely new medium. The agenda resembles a task list for making that practical: compressing these enormous point clouds, animating them, streaming volumetric media from capture all the way to a viewer’s headset, and even treating splats as a long-term archival format for real places. There is also a thread on using splats as a building block for world models, the artificial intelligence (AI) systems that attempt to understand and predict physical space. That element quietly connects this side room to the prominent AI keynotes elsewhere in the venue.
Flat screens have failed to convey the full effect of Gaussian splatting. On a monitor, a splat scene looks like a very good photograph. In a headset, it creates a sense of presence. Because the representation is genuinely three-dimensional, real parallax and real depth emerge as the user leans and moves. These are the exact cues that make virtual reality feel like a location rather than a picture of a location. Volumetric video is the clearest example. A concert or a family moment captured as moving splats is something one attends, not something one watches. There is no polygon budget, no hand modelling, merely the recorded light of a real moment played back around the user’s head.
The difficulty is performance. Drawing several million soft, overlapping blobs at the frame rate virtual reality demands is computationally severe. A standalone headset must achieve approximately 72 frames per second (fps), twice, once per eye, across a wide field of view, on a mobile chip with no fan and a battery to protect. On a desktop with a strong graphics processing unit (GPU), one can process six million splats and remain smooth.
On a Quest running standalone, the practical ceiling is closer to a few hundred thousand before the frame rate collapses and the experience becomes uncomfortable for the user. Filling that gap is precisely what much of this year’s research is pursuing. Foveated approaches render splats in full detail only where the user’s eyes are actually pointed and allow the periphery to go coarse, which is a natural fit for headsets with eye tracking. Others focus on smarter compression and on streaming a scene a slice at a time rather than loading all of it at once.
None of this is theoretical. Moving volumetric captures can already be loaded on a Quest 3 or a Pico 4 today, standalone, with no PC in the loop, through tools such as Gracia. Research groups have published datasets for walking through real splat scenes in virtual reality, and companies working on world models are shipping splat-based environments one explores in a headset. The components exist. SIGGRAPH is where the people building them compare notes and determine which approaches succeed.
The significance of the development goes beyond photorealism. The entire field is open in a way most graphics breakthroughs are not. The core method was published without restriction. The tooling is spreading through open-source projects and hobbyist communities as rapidly as through large studios. The barrier to capturing a scene is a mobile phone and some patience rather than a capture stage and a budget. That shift determines who can produce virtual reality content, not simply what that content looks like. It parallels the OpenXR standards push happening elsewhere at the same event: the people who build this technology continue to choose open foundations even while they compete on the layers above.
The headset wars will receive headlines later this year. New chips, new displays, new prices. Yet a headset is only ever as good as the content it displays. The debate happening in a side room in Los Angeles this week concerns whether the next generation of virtual reality gets built by hand, one triangle at a time, or captured from the real world one photograph at a time.
