CPU under-utilization Last few buckets - Render Bucket Controls - Auto Reduction

Started by D.A. Bentley, August 21, 2019, 01:46:33 pm

Previous topic - Next topic

D.A. Bentley

Buckets.PNG

I have had this rendering issue for as long as I can remember where the last few buckets take forever to render.  Usually this is in areas near the horizon where there are light beams shooting down through the clouds.  I thought Allow auto reduction was supposed to help break up these last buckets into smaller buckets so the workload could be spread across the CPU more efficiently, but it doesn't seem to be doing that.

The only work around I can think of is to render the horizon area in a cropped strip with a smaller bucket size, but when I tried reducing the bucket size to 16 (default is 256) it seemed to go even slower.  

Would it be possible for the Terragen tenderer to monitor the number of cores being used, and if under-utilization is happening it can reset the progress of those slow buckets and re-render them chopped up into smaller buckets so more of the CPU can be used?  It's just frustrating sitting there watch a 16 core CPU only use one or two cores at the end of a render which sometimes can be an hour or more.

Thanks for any suggestions,

-Derek

Oshyan

"Allow auto-reduction" doesn't do that, unfortunately. It just splits tiles into smaller pieces if the specified tile size will result in underutilization of available cores/threads, I believe. E.g. with default 256x256 tiles, if you are rendering a 512x512 image and have 8 threads, you want 8 tiles, not 4.

Auto-subdividing tiles would certainly be nice but seems to be more complicated than one would like as far as implementation. It's definitely a known feature request and either that, or some other way to render those "stubborn tiles", will hopefully be implemented in the future.

- Oshyan

D.A. Bentley

Thanks Oshyan!  So is there an ideal tile size to use for various resolutions?  I typically render out 8k Spherical Panoramas so 8000 x 4000 on a 16-core Threadripper.

-Derek

Oshyan

The tile size is generally best when set to auto. A size that is too small can reduce efficiency and actually increase render time, while a size that's too large can simply exacerbate the "last bucket problem" you're referring to. In most cases it's difficult or impossible to determine a better bucket size than the default/auto. However *if* you are doing multiple successive renders of a scene you *know* to have some really stubborn final buckets somewhere, then you can experiment in lowering the bucket size. However I would not go below 64 as I've never seen benefit at e.g. 32. 128 can sometimes be a "sweet spot" in those stubborn bucket situations, but the default is still generally better for the average situation.

- Oshyan

Dune

Maybe not handy in all circumstances, but I sometimes do crop renders of 'really stubborn or slow' areas, which then go a lot faster. Problem with standard render is the GISD and post stuff (if you want to stop earlier), so you'd have to know these areas in advance and crop them out of the first render. And find a decent way to merge the crops.