Tiling large renders

Started by sboerner, May 14, 2020, 11:10:11 AM

Previous topic - Next topic

sboerner

Just lost a 40-hour rendering because of an overnight Windows 10 update. Between updates and power interruptions I've lost more big renderings lately than I've finished.

I've paused the updates (I think) but there's nothing I can do about the power interruptions, which seem to be happening more frequently these days. Sometimes it's just for a second or two, but that's all it takes to wipe out everything.

Not only do you lose a rendering but you've given up the use of the workstation for 2-3 days and nothing to show.

OK, end of rant and I feel somewhat better now.

Is it OK to use tiling for big renderings? I'm thinking about slicing it into strips that can be done in a few hours. That way the most I'll lose is one night's work. I already use a GI cache so there shouldn't be much (if any) difference between the sections. Does anyone else do this? Anything to watch for?

Dune

If using a GI cache it should be fine. I do it too. Using cache to blend one image, not animation.

Can't you unplug while rendering, so it's offline? That's my main complaint about Win10, the unannounced updates.

sboerner

OK, I'll give tiling a try. Cache is set for one image.

You're supposed to be able to pause Windows updates for up to 35 days. I'm going to see how it works if I regularly set it to the first of each month. At least that way maybe I can plan around it. Unplugging might be an option, too.

The most frustrating part is not having the use of the machine during a long render. Maybe it's time to start saving my pennies for a second rendering node.

jaf

On the Windows Update settings, select "Pause updates for 7 days" and then  select "Advanced options".  Then use the drop-down box under "Pause until".
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

cyphyr

Take a look in your Docs folder of the Terragen install and check out the "win_command_line.txt".

Towards the bottom of the document there are entries about tilex and tiley and also cropoutput. These will let you output just the area of the crop (without the padding) via the command line executable tgdcli.exe.

Just change the crop region each render (you could animate it).

I have rendered images over 60k with this method with no issues.

Takes a bit of fiddling so try with a very much smaller render first.

Quote-tilex <minx> <maxx>

Limit the rendered region to a fraction of the image or crop region to be rendered.
minx and maxx should be numbers between 0 and 1, where minx < maxx.
For example, "-tilex 0 0.5" renders only the left half of the image or the left half of whatever crop region was already set.
You can use both -tilex and -tiley simultaneously to define a rectangular region.


-tiley <miny> <maxy>

Limit the rendered region to a fraction of the image or crop region to be rendered.
miny and maxy should be numbers between 0 and 1, where miny < maxy.
For example, "-tiley 0 0.5" renders only the bottom half of the image or the bottom half of whatever crop region was already set.
You can use both -tilex and -tiley simultaneously to define a rectangular region.


-cropoutput

Crop the output image(s) to the dimensions of the rendered region (defined by "crop region" and/or -tilex, -tiley), i.e. do not pad to the full image dimensions.
www.richardfraservfx.com
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 5950X OC@4Ghz, 64Gb (TG4 benchmark 4:13)

sboerner

QuoteOn the Windows Update settings, select "Pause updates for 7 days" and then  select "Advanced options".  Then use the drop-down box under "Pause until".

Thanks, jaf. Pausing it to a specific date seems to be the way to go.


QuoteJust change the crop region each render (you could animate it).

Now, that's a great idea. This way you get the best of both worlds – a continuous rendering but saving to disk periodically. Thanks.

I wonder if tiling large images is more efficient to begin with. The rendering that I lost was 3840 x 2160. The last time I checked it had run for more than 25 hours, and at the rate it was going had at least another 6-8 hours to go. Let's just say it would have been finished in 32 hours. I rendered the first tile of the same image last night, same settings. Size was 960x2160, a vertical strip that included atmosphere and ground detail. It rendered in 3 hours. If the rest take the same amount of time that's a total of 12 hours . . . hmm.

Dune

If you would animate the crops (blur off!), wouldn't it be easier to not use cropoutput, but normal output (with padding). In Photoshop it's then a piece of cake to layer, no need to fit each crop in place. Even setting each layer to lighten would do the trick.

cyphyr

When I was doing very large renders I found that the render wasn't the the issue. The problem came with saving it. Cropping the padding out makes a much smaller image to save which was the point of the exercise in my case.
Probably worth trying both to see what works best in this case.
Doing a very MPD and AA image will complete faster but the resultant file will be the same size so you could test both options that way.
www.richardfraservfx.com
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 5950X OC@4Ghz, 64Gb (TG4 benchmark 4:13)

Dune

Makes sense, but it's actually strange that it makes such a size difference. After all, the rest is just black. Must be in the way the pixels are saved.

sboerner

I'm still kicking myself for not thinking of this. :P It's so obvious and it would have prevented a lot of frustration.

So thanks again, Richard.

sboerner

Besides disabling blur, you also have to set the GI cache blend mode to "Nearest file in sequence."

Worked like a charm. The total rendering time was 15h14m, less than half the time it would have taken to do it in one frame. Not sure I understand this, but I'll take it.

cyphyr

Glad it worked. I don't use it that much simply because of a lack of need that often and I always forget the optimal settings :)
www.richardfraservfx.com
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 5950X OC@4Ghz, 64Gb (TG4 benchmark 4:13)