Tile rendering HUGE images.

Started by cyphyr, July 26, 2015, 05:39:51 AM

Previous topic - Next topic

cyphyr

Hi guys, long time no post but I'm back, hopefully for more than just a while at least.

Has anyone had any experience rendering huge scenes.  I'm talking images over 15k (more than 15,000 pixels x 15,000 pixels).  I can render out very large images but I cant save the resulting image.  So tile rendering is the obvious alternative.  I've looked for a tutorial but can't find one.  Is it simply a matter of creating a crop region and animating it's location over a number of frames equal to the number of tiles?  If so I don't see how this would get over my problem of saving such a large image. Even if only a tiny part of the image is rendered, the entire 15k image still has to be saved so the issue still remains. If I change the camera to pan and scan across the same field of view (as the original 15k image) then there will be distortion at the edges of each image that won't match up top the next image.

Any links to a tutorial very much appreciated. Thanks in advance

cheers

Richard
www.richardfraservfx.com
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 5950X OC@4Ghz, 64Gb (TG4 benchmark 4:13)

archonforest

Chk out the Museum Wall project by Dune. That might give u some ideas as that render was big.
Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

bobbystahr

something borrowed,
something Blue.
Ring out the Old.
Bring in the New
Bobby Stahr, Paracosmologist

TheBadger

#3
Your problem is interesting to me.
Memory issue of some kind?  very strange indeed if you can make the image but just not save it.

I also want to render huge at some point, but not if I could have the same problem. I have future plans that I make efforts at little by little, but now you have me worried. Please make sure to post the verified cause/solution when you learn it. Sucks to do lots of work and just get stopped right at the end, as you know.
It has been eaten.

Kadri

Quote from: cyphyr on July 26, 2015, 05:39:51 AM
... I can render out very large images but I cant save the resulting image...

Richard do you mean manual saving or automatic or both? Just curious if there is any difference.

Would be good if Planetside could add a real(!) cropping rendering option for this kind of things where the outside of the cropping area isn't included in the final saved file .
There would be no distortion at all in this way.

cyphyr

Yeah, the saving fail is with both manual and auto (render to disk). Yes I agree about the crop rendering.
www.richardfraservfx.com
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 5950X OC@4Ghz, 64Gb (TG4 benchmark 4:13)

TheBadger

Richard, just out of curiosity, can you save it to a external drive? Suppose that is unlikely, but if you can you will know it is a hardware problem. Be good to just rule that out right off.
It has been eaten.

kaedorg

IMO, it is a question of RAM. And you try to test a render using a too important percentage of your RAM,
then you won't have enough to save.
Saving takes some more memory and you won't get back the memory after rendering and before saving (at leats not enough).

With 32 RAM, i helped Ulco for his Museum Wall, the frames were 9640 x 5800

My biggest render without any problem is 18000 x 4688

David

cyphyr

I'm saving to my primary SSD and I have 16mb of ram. It should be sufficient but maybe more ram is the answer.
I'm rendering a square image at 15+k each dimension which would be considerably larger than your 18000 x 4688.
Maybe it's time to buy yet more ram !!! :)
Thanks
www.richardfraservfx.com
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 5950X OC@4Ghz, 64Gb (TG4 benchmark 4:13)

Dune

Ram is the point indeed; I had to divide my next wall in 4 sections or I would go over 16Gb with just the bare terrain and 2 houses. Now (with 1/4) I am already at 12+, but I'm putting more stuff in all the time.
So perhaps ram and divide in sections. What are you working on, Richard; big secret or sharable?

kaedorg

Another point before big renders, good thing is to close everything and even restart your PC.
I noticed, and if I remembered well there were some topics about, that it avoids some crashs.

David

Kadri

#11

Even the default scene with 15 000x 15 000 dimensions goes directly to 25 GB RAM usage when i tried Richard.
And that is with low detail and antialiasing. It goes up with higher settings of course.
Crop rendering doesn't do much too as it looks.
Dividing the scene looks like a good solution as Ulco said.

I have an SSD and it is nice. But for 3D work my choice would be first more RAM then SSD.

As David said there are some ways that could help too.
If you can get until the render stage, closing the preview window and maybe other any window in Terragen you don't need
and other things you have in Windows 8 or whatever you use might help.
I saw 1 GB difference with and without the 3D preview window even with the default scene and some basic tests.

I haven't used it but wasn't there a command line Terragen executable? That might be helpful too probably.

cyphyr

Thanks for the replies guys, looks like ram is definitely the issue. I think I have a couple of spare slots, hope so otherwise this is going to get pricey!

Once the render is done, the memory is freed up, but even so the image is still unsavable.
www.richardfraservfx.com
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 5950X OC@4Ghz, 64Gb (TG4 benchmark 4:13)

Kadri


Any error message,crash? Maybe it does take too long only?

Oshyan

Indeed it is probably a memory issue. *However*, there is a newer-ish feature in the commandline options that *may* help you. Assuming you are familiar and comfortable with stitching tiles together (with possible overlap, to minimize tile border issues, even *with* GI caching), then you might try the -cropout CLI option. Details here:
http://planetside.co.uk/wiki/index.php?title=Windows_Command_Line_Reference
You can try it initially with just a full-image GI cache and non-overlapping crop tiles, however I suspect you may need to do some overlap (say 10% of the size of each tile, 5% on each side, at least) to avoid issues like missed shadows casting into the crop from out of the crop area.

Now, what can you tell us about what you're working on that requires such huge resolutions? :D

- Oshyan