NeRF and Gaussian Splatting: What the Breakthrough in Neural Scene Reconstruction Means for Visualization

The premise of Neural Radiance Fields — representing a three-dimensional scene as a learned function that maps spatial coordinates and viewing directions to colour and opacity — felt, when the original NeRF paper was published in 2020, like a research curiosity with limited practical application. The renders were impressive. The training times were prohibitive. The editability of the resulting representation was essentially zero.

Three years of refinement, followed by the introduction of 3D Gaussian Splatting in 2023, transformed the trajectory entirely.

Gaussian Splatting trades the implicit neural representation of NeRF for an explicit point-cloud-like structure: millions of small, semi-transparent Gaussian ellipsoids, each with a learned position, size, orientation, colour, and opacity. The visual result is indistinguishable from NeRF in quality. The rendering speed is orders of magnitude faster — real-time on a consumer GPU. And the representation, while still not straightforwardly editable in the way a polygonal mesh is, is at least explicit enough to permit targeted manipulation.

For architectural visualization, the most immediate practical application is site documentation. A photographer walks a building site with a DSLR or a drone, captures 200–400 overlapping images from controlled positions, and ingests those images into a Gaussian Splatting pipeline. The output is a photorealistic, navigable 3D reconstruction of the site as it currently exists — an as-built model that captures material weathering, site context, planting, and environmental conditions with a fidelity that no CAD model can match.

That reconstruction can then be used as a foundation for proposed-design visualization. Compositing a photorealistic CG building into a Gaussian Splat of its real site context — aligning light direction, matching atmospheric haze, calibrating the camera parameters — produces imagery with a credibility that traditional matte painting or HDRI-based context rarely achieves.

The limitations are real. Gaussian Splat reconstructions do not produce clean polygonal geometry that can be imported into a DCC application and textured from scratch. They are representations of appearances, not of structure. Dynamic objects — people, vehicles, wind-moved vegetation — are handled badly by current pipelines. And the capture requirements, while modest by survey standards, still require methodical execution that most clients and many studios are not currently equipped to provide.

But the trajectory is clear. Within three to five years, every visualization project brief that involves an existing site will include a photogrammetric capture phase as a standard deliverable, producing a Gaussian Splat or its successor as the context environment. The studios that are building this capability now are acquiring a structural advantage that will compound.