On the road to VR’s holy grail, there are several orbiting technologies including positional tracking, graphical processing, field of view, haptics, etc. But one of the more technically complex areas is lightfields.

We already have graphical VR experiences that let us move throughout volumetric spaces, such as video games. And we have photorealistic media that lets us look around a 360 plane from a fixed position (a.k.a. head tracking).

But what about the best of both worlds? We’re talking volumetric spaces in which you can move around, but are also photorealistic. Central to this vision are lightfields. They define how photons hit our eyes and render what we see.

But because it’s a challenge to capture photorealistic imagery from every possible angle in a given space — as our eyes do in real reality — the art of lightfields in VR involves extrapolating many vantage points, once a fixed point is captured.

And that requires clever algorithms, processing, and whole lot of data. Our friends at Visby are working on making the latter more tenable. And the always-innovative (and heavily funded) Lytro continues to lead the way with optics.

For a glimpse into Lytro’s latest, see the video below about the making of Hallelujah. Debuting last week at Tribeca Film Festival, this is a volumetric photorealistic VR short created by Within, using Lytro’s cameras. Holy Grail indeed.