Last time I mentioned we were working on PCC / VCT Hybrid (PCC = Parallax Corrected Cubemaps, VCT = Voxel Cone Tracing).

I just implemented storing depth into the alpha channel, which is used to improve the quality of the hybrid.

Because we often use RGBA8 cubemaps, 8-bits is not enough to store the depth. Therefore we only store the deviation from the ideal probe shape.

The original PCC algorithm boils down to:

    posInProbeShape = cameraCenter + reflDir * fDistance;

Where cameraCenter is artist-defined (where the PCC camera was placed to build the probe) and posInProbeShape is also artist-defined (by controlling the size of the probe)

PCC is about finding reflDir mathematically:

    reflDir = (posInProbeShape - cameraCenter) / fDistance;

However we already know reflDir after executing PCC’s code.

Now the depth compression comes to effect by slightly altering the formula:

    realReconstructedPosition = cameraCenter + reflDir * fDistance * depthDeviation;

The variable depthDeviation is in range [0; 2] (which we store in the alpha channel) and thus 8-bit should be enough.

Technically this could introduce a few artifacts because we only store the depth in range [0; maximum_distance * 2]

Storing the depth deviation dramatically improves the hybrid’s quality!

But let’s not rush.

HW texture filtering issues

VCT reconstructs the reflected position by cone tracing along the reflection dir.
This means we have to perform a lot of texture samples, and trilinear filtering helps a lot in improving quality. Except, HW texture filtering is good enough for reconstructing colour, but not so much for reconstructing 3D positions.

You see, GPUs are cheap. They interpolate in 8-bit. That means between pixel 0 and pixel 1, you only get 256 different steps of interpolation, even if the texture is stored in RGBA32_FLOAT.

GPUs do something like this (1D for simplicity):

    float fWeight = fract( uv.x * 255.0f );
    fWeight = (uint8_t)(fWeight * 255.0f) / 255.0f; //Quantization
    return lerp( pixel0, pixel1, fWeight );

And this is very noticeable in the reconstruction:

This complicates the hybrid because the reconstructed position is somewhat quantized.

We could do trilinear interpolation by hand, but I fear this would be too expensive. We’re not performing a few taps per fragment. We’re running a cone trace. That’s a lot of taps.

Depth Variation as a mask

Just by accident, I noticed the depth variation happens to be a mask of how much error there is in PCC!

Of course!!! Depth Variation == Error variation!

And this mask is smooth, unlike the reconstructed error. When the mask is 0.5 (grey), error is minimum.
When the mask is 0.0 (black) or 1.0 (white) error is maximum. And we even have mipmaps!

By transforming the mask into something we can use:

    pccErrorMask = ( depthDeviation - 0.5f ) * ( depthDeviation - 0.5f ) * 4.0f;

By combining the error mask with our existing hybrid (quantized) code, we can mask many of the artifacts we were having.

We must combine the mask with the existing code, rather than replacing it because this error mask isn’t perfect as it has false negatives.

If the probe is built like this with the probe camera at the specified location:

…then during rendering reflections would go through the separating wall straight into the other wall.
The depthDeviation mask would not notice anything wrong because the reflected wall matches exactly the probe’s shape, thinking the error variation is minimum. But it’s blatantly wrong:

Thus this mask isn’t perfect and needs to be combined with extra informaiton. Of course, a simple solution in this example would be use two probes, but more probes means more RAM consumption, lower performance, and I fear not every example could be fixed this way.

Furthermore, we can’t rely on the artist knowing all this so he can make the perfect probe placement.
The algorithm needs to be robust enough to handle this case.

Results

After using the compressed depth to improve accuracy, and also using it as a mask to hide errors, these are the results:

Living Room Scene, PCC Only
(btw it seems a bug creeped into the sky as it is black in the reflection)
Living Room Scene, VCT Only
Living Room Scene, PCC/VCT Hybrid

But there’s still a problem. Sometimes pictures need to be manually tweaked in brightness to match.

PCC and VCT need to match

Although quality was an important goal, our VCT implementation wasn’t rigorously tested against reference raytracers. It’s brand new and besides, Global Illumination is already a huge improvemnt right? No one’s gonna notice.

Except that now that VCT and PCC are being combined, this little detail becomes to obvious. When they don’t match, transitioning when between the two the contrast change is quite stark (exaggerated for this picture):

Differences between VCT & PCC when each other take over, exaggerated

Some contrast differences are always going to be expected because VCT tends to have blurrier reflections (due to its limited resolution) than PCC.

But right now we have differences in brightness we can fix.

I suspect part of this problem is that we’re building PCC probe data with specular reflections, and VCT has no such info. So probably if we want them to match, we would need to make PCC generation diffuse only. This is a theory though.

Another big reason is that clearly VCT needs more rigorous benchmarking against reference material, such as raytracers.

Next work

So that’s what we will probably have to focus next: Working towards making our PCC and/or VCT implementations more exact so that they both match reality, and more importantly‚Ķ with each other.

Another issue is that right now I force-disabled the use of VCT while generating the PCC probes, and that takes away GI from the probes! That can cause huge differences in lighting. I need to reevaluate this decision. At the very least, diffuse VCT needs to be present during PCC generation

Last but not least, if the probe is blocked by an object, and the reflection comes from behind, the hybrid will think the reflection is OK when it’s not.

For example this causes reflections in the ground to reflect what’s on the other side of a table:

The only solution I see to this is to detect & discard reflections that come from behind, or to use more probes (i.e. one below the table).

Unrelated to all this, it seems a bug creeped into per-pixel PCC where influence areas are not always being respected. It seems either ForwardClustered from C++ or shader side is using the wrong area (i.e. using the probe’s shape instead of the area of influence). That’s a bug in need of fixing.

Well that’s all for last weekend’s work

Cheers

Further discussion in forum post.