Cloud flickering

[Njen]

I am having some trouble removing cloud flickering from the following specific example. In the following scene I disabled everything except for the cloud layer I am having issues with. The most amount of flickering is at the end of the frame range in the file. The example movie attached to this post is from f5300 to f5320 (20 frames), though please note, the compression in the movie reduces the perceived flickering, so I am providing a link to the exr files in a zip file to view the actual results (111mb):
https://www.ryangrobins.com/misc/sr_100_tgdEnvFull_beauty_v018.zip

I am using sparse GI files (number to blend: 3, 32 frames apart; which I have not uploaded in this example, but I can provide them for testing). I've tried increasing 'Cloud GI quality' and tried different combination of settings of 'GI cache detail', 'GI sample quality' and GI blur radius' without success. I tried 2D shadows and voxels. I tried a number of other tips and tricks researched on the forums, but none seem to work either. Though most likely I am missing something.

Thank you in advance!

[Hetzen]

Looks like GI flicker. I've always rendered every 5th frame for GI.

[Njen]

I have tried lower amounts of sparse GI files like 16 and 8, but it still flickers. Unfortunately I don't want to go lower than 12 if I have to, as I am rendering a 4000 frame animation, and every 5 would be a lot of GI to render if I can help it.

[Hetzen]

I think the issue is that the clouds are moving forwards and your GI cache is not keeping up with the movement, when you're sampling the illumination every second+. The cache shouldn't take too long to render, maybe 3hrs on a good workstation at every 5th frame.

[Kevin Kipper]

This portion of your animation, frames 5300 - 5320, does not appear to have a lot of visual changes, i.e. large camera field of view changes due to changes in its position or rotation.  I would think sampling more than 3 frames would begin to smooth out an GI flickering.   For example, on frame 5300 you're currently blending the GI for frames 5288, 5300 and 5352 (assuming those gi frames exist).  If you set the "Number of frames to blend" to 5, for example, then frames 5256 and 5364 would also be blended.  The interpolation of those 5 frames should result in smoother GI.  

[Matt]

A couple of other things I noticed, but I don't know if they are the cause. 1) The frames appear to be missing large areas of GI on the ground, which makes me wonder if you have adequate GI cache coverage for these frames. 2) In the file you posted, 'ren_anim_high' is loading v001 of your cache files while 'ren_giCache' is writing to v002.

[Matt]

If you set the "Number of frames to blend" to 5, for example, then frames 5256 and 5364 would also be blended.  The interpolation of those 5 frames should result in smoother GI. 

If the cached files are once every 32 frames, and there are no old files left in the folder, then there should not be any frame-to-frame flickering because the results will be interpolated over 32 frames (like an animation curve with key frames 32 frames apart). If there aren't enough files being blended then this could result in large changes over the 32 frames but the changes would at least be interpolated. Frame-to-frame flickering comes from something that isn't being cached, either because the cache files don't cover this part of the scene or because some the flicker is from some component of the lighting that isn't cacheable. I'm not sure which is happening here.

On the other hand, if we were to cache the GI on every single frame then the number of frames to blend would affect the amount of flicker, but we don't recommend caching every frame. This also makes it more difficult to tell whether the flicker is in the cache or something else that isn't cached.

[Njen]

A couple of other things I noticed, but I don't know if they are the cause. 1) The frames appear to be missing large areas of GI on the ground, which makes me wonder if you have adequate GI cache coverage for these frames. 2) In the file you posted, 'ren_anim_high' is loading v001 of your cache files while 'ren_giCache' is writing to v002.

The mismatch on the version of GI being used was me messing around with different settings. I can 100% confirm that it is set appropriately in the render.

For this test I did remove all of the terrain and instance nodes to speed up the render times for this test. In the actual output I see the exact same type of flickering. I have attached an image of the final render for reference.

I can also confirm that there are no old GI files in the sequence (see attached image)

I could try blending every 5 frames and see what happens, I haven't tried that option yet, thanks!