Path Tracer Denoiser Image Effect idea

[WAS]

I think it would be cool to incorporate a image effect for the path tracer to smooth out pixelation noise. It seems there are benefits to be had with certain scenes (especially landscapes) and could be balanced between MPD/AA settings and Denoiser intensity spread/sharpening settings to obtainer quicker PT renders.

This could be done in your sequencing, but from searching about it seems to becoming normal to handle this on the rendering side for optimal results (no clue what is all going on behind the scenes).

Apparently this Moana scene utilized it in Intel OSPray:

http://www.ospray.org/
https://openimagedenoise.github.io/

And the results in the example, imho are a little strong, but I can immediately see the benefits.

Perhaps Intel OID or another open source library could be incorporated into Terragen for quicker smooth stills or frames.

[KyL]

I tried the Intel denoiser with a Terragen render a few month ago, using the implementation in Blender. You can feed it the beauty, normals and albedo pass. I was not convinced with the results, the noise was nicely removed in some areas, but too much or not enough in others.

Maybe having a native implementation withing Terragen would make things better, as you could probably input more data (sample rate pass maybe?)
Anyway, having an additional post-process for this sounds like a sweet idea indeed!

[WAS]

I was curious what the library alone might produce. Do you still have the tests around? I'd be curious to see. I'm pretty sensitive to effects (like the Moana demonstration seems too strong, very Topaz Labs like where colours become solid)

But I do think subtly using it on the right settings with the PT could benefit renders in the time area, and I was also curious if a native implementation could sample it better, such as darker areas more than light (where noise will be more apparent).

I wanted to try the Nvidia AI too, but I don't have a RTX card (and still waiting on approval into the program), and it being proprietary there (i believe) doesn't make it very practical imo.

[WAS]

Speaking of Topaz Lab, I have their current denoiser trial, which is supposedly AI....

That being said, it has less control than past versions of Topaz Labs Denoise from CS 5.1 days, and imo worse results. For the price, and the bloat of the stnad alone software and the plugin for Photoshop, I don't think it's worth it at all. I wish I could still download the old versions.

[KyL]

Here is an example. I couldn't upload the exr files as they were too big. But you can clearly see how the denoiser is performing well when the noise is flat and uniform (like in the car reflection), but doesn't handle really well the shadow areas.

Maybe this is because of the noise pattern generated from using the robust adaptive sampler. I didn't try denoising a frame renderer with the Legacy sampler, it may be more uniform and easier to denoise...

[WAS]

Here is an example. I couldn't upload the exr files as they were too big. But you can clearly see how the denoiser is performing well when the noise is flat and uniform (like in the car reflection), but doesn't handle really well the shadow areas.

Maybe this is because of the noise pattern generated from using the robust adaptive sampler. I didn't try denoising a frame renderer with the Legacy sampler, it may be more uniform and easier to denoise...

Huh, that is interesting. It seems to have given some areas stronger effects which didn't need it. Like the TV, light neon light glows, seem more denoised than other areas like lamp light and stuff. And than some areas it looks pretty good, like you said, the car.

[WAS]

Since I have the trial still installed I tried Topaz AI on your file. Man I really do not like the results. Some areas look ok, but the deconstruction of edges and colours is atrocious (look at construction signs reds). It also seems to ignore bright areas with noise... It also seems to destroy muddled areas with JPEG artifacting despite being low compression. Take a look at the cup before export. Definitely not buying this as means to help PT renders.

Huh, the cup issue seems to just be a bug with their settings. Even in PNG it exports as a blur, and not what the program is showing you... Lol

[KyL]

Nice attempt. But indeed, it seems to destroy some areas and even introduce artifacts.

I will try rendering the same frame with the legacy sampler, see if it helps.

[Kadri]

Optix worked good with the box pixel reconstruction filter so far i remember.
How are they with your tests? Did you tested with all of them?

[Tangled-Universe]

Ideally we would train a network ourselves by feeding it undersampled and noise-free renders. Pity it would probably need an impractical number of renders (from our perspective) to make this work.

[Kadri]

Optix worked good with the box pixel reconstruction filter ...

"Worked good" is only relative here of course to the other filters.

[WAS]

Isn't trained AI for fake reconstruction of images based on other images, IE Deep Network Learning? I thought these denoisers from topaz and such are doing pixel reconstruction. Google and Nvidia have something similar with deep learning, but the results are bizarre. Using other people's eyes and skin textures and nose elements on people, etc. It's becoming popular in old 19th century and early 20th century images and they make people freaky.