Spatial Artifacts

My wife cut a peony from the garden and brought it in the house and I can't stop looking at it. I've been taking photos, sitting and staring at it. I'm also taking a sculpture class¹ at a local atelier where I've wanted to capture my instructor's model so I can reference it at home. Both of these sent me to the app store on a whim, revisiting the idea of 3D scenes from 2D images. I found Scaniverse, pointed it at the peony, and it kicked me down a rabbit hole.

I train at King's Boxing in Fruitvale three to five times a week. It's old and historic. Everything you want in a gym. I take photos of the posters on the walls all the time. Before wrapping up a session I took a quick capture of the speed bag. It looked so good. Used the whole thing as a test case to see how far I could take it. Turns out, pretty far.

This is a 3D Gaussian splat. It's not a mesh, not a point cloud, not a photogrammetry reconstruction. It's a few hundred thousand translucent ellipsoids, each with a position, shape, color, and opacity, learned from a set of photos and rendered in real-time in your browser.

The lighting shifts as you move, the same math that describes how light scatters off a surface depending on your viewing angle. The format preserves that. A photo can't.

Think of them like image formats.

• PLY — your RAW. Archival, never serve on the web.
• SPZ — your JPEG. Niantic's format, 90% compression, keeps the spherical harmonics. Scaniverse exports this natively.
• SPLAT — your GIF. Broadest compatibility, strips the good stuff.

Choosing a renderer

There are a handful of WebGL splat viewers out there — GaussianSplats3D, gsplat.js, antimatter15's zero-dependency OG viewer. I went with SparkJS because it's Three.js-based, which played nicely with my existing stack, and it handles the widest range of formats including SPZ.

Returning to auto-orbit

I wanted auto-orbit — the splat slowly rotating to invite interaction. But when you grab the controls and let go, the view cuts to the orbit path with no transition, and it feels like a glitch. The solution was a 1.5-second blend: capture the user's camera angles when they release, then ease back into the orbit with a cubic curve. Both azimuth and polar angle interpolate, so if you tilted up, it settles back gently. Grab mid-blend and it cancels instantly. Small thing, but it's the difference between feeling like a tech demo and feeling like an object you're holding.

Authoring controls for tuning

I built a dev-only panel — camera position, rendering parameters, depth of field, presentation — with sliders that write directly to the Three.js refs for 60fps feedback. When I land on settings I like, I copy the config JSON to clipboard and paste it into the config file. The viewer in production just reads that config silently.

Pausing offscreen viewers

A friend pointed out that scrolling this page on mobile iOS was laggy. Which makes sense — there are four splats running requestAnimationFrame loops offscreen. I added an Intersection Observer, per his recommendation, that pauses the render loop when a viewer leaves the viewport and restarts it on the way back in.

Now I want to push it further in two directions.

Capture over time

I want to try scanning something over time — export each session as SPZ and build a viewer with a time slider. A 3D timelapse. There's a research direction called 4D Gaussian Splatting that adds time as a dimension, smoothly warping the splats between frames instead of cutting between discrete snapshots.²

Keep scanning

I like the idea that something becomes visible when you can orbit around the things in your life — the gym, a flower, a workspace, a garden over time. Will continue capturing objects and see where it leads. If you have any ideas send them my way.

---

Reading list:

• NYT R&D: "A Field Guide To Gaussian Splatting"
• NYT R&D: "Pushing the Limits of Gaussian Splatting for Spatial Storytelling"
• NYT R&D: Spatial Journalism — The namesake of this article — NYT R&D's term for this whole area of work.

Footnotes

1. Specifically an anatomy and écorché class — where you build a full anatomical figure starting at the bones and layering on each muscle over six months. ↩

2. A few researchers on 4D Gaussian splat: Guanjun Wu, Taoran Yi, Jamin Fong. ↩