Sony XYN Turns Real Locations Into Detailed 3D Sets
Sony’s XYN Spatial Capture Solution is designed to turn photographs of real-world locations into production-quality 3D computer graphics for virtual production. The system combines a smartphone-based shooting assistant, cloud-based 3D generation, and a rendering plugin for virtual production environments, including large LED walls used on film, television, and commercial sets.
The technology is designed around the idea that a physical location can be captured once, converted into a detailed 3D asset and later brought onto an LED stage when production requires the environment.
The first part of the system is XYN Spatial Scan Navi. It works with compatible Sony Alpha cameras and a smartphone, using augmented reality to guide photographers to the positions and angles needed to capture a location.
A bird’s-eye preview tracks the areas that have already been captured. This allows crews to identify gaps in the scan before leaving the site.
The application also adjusts camera settings and capture parameters according to the subject being recorded. It records lens information, shooting data and camera metadata for the 3D generation process.
Sony says the approach can make spatial capture accessible to crews without specialist expertise in creating 3D environments.
Once the photographs have been captured, XYN Spatial Scan converts them into 3D assets. Sony’s cloud application uses proprietary algorithms to generate high-resolution 3D assets from the captured material.
The resulting assets can include details such as reflections and gloss. Sony says the environments are intended to remain convincing when scaled up on LED walls used as virtual-production backgrounds.
The system also supports HDR-oriented colour workflows. Because processing occurs in the cloud, crews do not need a dedicated local system to generate and review assets.
The third component is the XYN Spatial Renderer Plugin. It brings the generated environments onto the virtual set and is designed to maintain image quality across different viewing distances and angles.
The renderer also provides controls for depth of field, lighting matches between performers and backgrounds, localised colour correction and CG overlays. These functions are among the tools available for integrating captured environments into virtual production.
XYN is designed to work with software already used by production teams. It works with Unreal Engine, and the renderer supports Disguise and integrates with Sony’s Virtual Production Tool Set.
The system is intended to make it easier to bring a captured real-world environment into a professional 3D workflow. In virtual production, a physical location can become digital production material that can be used for another scene, another camera angle or another day of shooting.
The location does not have to be captured again for each of those uses. Instead, the captured environment can be used as part of the virtual production process.
Sony has also identified applications for XYN beyond film and television. The company sees potential uses in gaming, animation, architecture, manufacturing, digital twins and cultural heritage preservation.
Sony also plans additional plugins and services for established 3D tools and game engines.
The technology could be used to create a detailed digital representation of a physical place for cultural heritage preservation. Such a representation could remain available even after the physical environment changes.
The XYN system therefore uses camera-based capture as the starting point for creating reusable digital environments. Its workflow combines location scanning, cloud-based 3D generation and rendering tools to convert real-world places into digital assets for virtual production and other identified applications.
