Importance of the amount of DDR4 in TG4.

Started by Jgone, March 07, 2017, 04:22:14 PM

Previous topic - Next topic

jaf

I purposely added massive populations to see how far I could "push" TG4 and my new build.  I built a scene with the camera at x=0 and z=0.  When I added the pops, I just but them at 0,0,0 so a majority would be out of camera view.  And yes, it used vm with the ssd.

I normally wouldn't be that sloppy, but this was just a combination of testing and me making the mistake of not clipping the pops from the camera.

I've been building my scenes lately around 0,y,0 and it seems to be quite efficient with my workflow.  First, I use World Machine to get the basic terrain.  Then, instead of moving the camera, I move the heightfield shader. I try to not rotate the camera in x and z, but sometimes I must.  I use macros in World Machine to rotate the terrain build so I can get close to the view I want before it gets to TG4.

Now I  put in the foreground pops at o,y, o with camera clipping.  If I want a dense grass pop out to 20m, I make it 40x40.  Background pops obviously need to be positioned either by x,z or altitude/slope, or some combination of those parameters.  Now instead of having to move the camera and all the populations, I can move the terrain and (usually) adjust the camera height.  Or swap in a new or rotated heightfield.

Okay, as far as 64 GB of memory.... I can set TG4 to 3/4 of my cores, start a render, and then run Lightwave/3D-Coat, or other apps.  So far TG4 has not crashed except when I pause a render (Matt mentioned this was a known problem.)

(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

zaxxon

Jaf, I'm also curious as to how you "peaked" out at 83 gigs of ram with 64 gigs installed? I currently have 64 gigs in my main TG render machine (HP Z800 dual zeon 5690 @3.47 mhz) and use some pretty 'dense' scenes that sometimes run up to the ram limit (63.7 out of 64 gigs utilized). All that happens is that the machine starts paging the HD and render times slow dramatically. I'm upgrading to 96 gigs due to the asset load in current projects becoming unwieldy and unstable. Some optimization strategies may be called for ya think... ;). Still 32 gigs should be fine, unless as Oshyan points out that the scene requirements put you at the limit, and yes V3 clouds with hundreds of millions of voxels are a beastly load on your system!

Dune

If an ssd can be used for vm, great. It sure is fast, so swapping wouldn't perhaps really slow the machine down. Can you tell about that? Might be an interesting venue if you're at max memory possible (due to slots, or so).

I always plant my pop centers at half the distance of the pop size needed in front of the camera, and rotate like the camera rotation, so a maximum efficiency is secured.

jaf

Zaxxon, easy.  Just load your default scene and add a population like "Gras_by_Klas" (gras.tgo) and set the object spacing to a crazy number like .01 and hit populate.

Here's what it looked like at 2% (I use Sysinternals Process Explorer since it will graph an individual process -- the scale will increase when it "hits" the top of the graph.)


(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

jaf

Quote from: Dune on June 15, 2017, 11:23:17 AM
If an ssd can be used for vm, great. It sure is fast, so swapping wouldn't perhaps really slow the machine down. Can you tell about that? Might be an interesting venue if you're at max memory possible (due to slots, or so).

I always plant my pop centers at half the distance of the pop size needed in front of the camera, and rotate like the camera rotation, so a maximum efficiency is secured.

I did a quick test and didn't see a difference in a centered at the camera 40x40 pop (with camera clipping) and a 20x20 positioned right in front of the camera.  I like my method because if I rotate the camera, my foreground population is still centered on the camera.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

Dune

Good to know, jaf, thanks for checking. I think the populating mechanism must have changed over the years to be faster and more efficient. Still, I'd like to know if using an ssd for VM would be really fast and workable. Then you could even use a say 64 gig SSD and just 16 gig of memory banks for peaking to about 80 gig. Cool concept.

jaf

#21
I ran Performance Test 9 on my M.2 SSD and got a score of 15284.  Then on my Western Digital a 726, so the SSD was 21 time faster.  What this would mean time-wise with a TG4 render, I'm not sure.  I guess I can rig up a scene that would use more than 64GB and render it.  Then switch my virtual memory drive and re-render.

I'll give that some thought and see what I can do.

[edit]  Just a thought ..... I wonder if there's a way to use GPU memory?  I've got 11 GB of DDR5 memory "sitting" there basically doing nothing.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

Oshyan

SSDs are getting very fast but are still much slower than RAM. So if you can avoid swapping to disk (SSD included) it is best to do so.

I think I've heard of weird hacks for using GPU memory as RAM, but remember that the connection from CPU to GPU is not as wide as that from CPU to main memory (especially in quad channel setups), so it is not going to be as fast as main RAM still. With memory being relatively cheap, the only limiting factor these days is the number of slots on your motherboard. For new machine builds/purchases there's not much good reason not to get a board capable of handling at least 128GB of RAM, even if you only start out with 32 or 64GB.

- Oshyan

jaf

Yes, I agree.  I've only gone over 32 GB when I've made a mistake anyway, but I thought it would be interesting to see if I could at least get an approximate render time savings between a M.2 SSD and a HD. But..........

Now I'm stumped.  I tried the default scene with a single population, again using 1000x1000 area "Gras by Klas" tgo, with a small (.05) object separation. But after it populated and did the set-up and inserting,  I watched the memory TG4 used (64 GB+) continue to climb and the render not starting.  It just kept going -- no error message.  It finally hit 100 GB and I killed the process.   So I'm wondering if there's a limit on instances of a single object?  Or the area the object could be too small to contain the number of instances?

So I tried again except I used changed to .1 separation.  This one rendered.  I added a second population, identical to the first but in a different area of the scene.  Both populated, did the set-up, but the render never started and the memory went from 32 GB+ to 100 GB -- kind of ratcheting up .3 or .4 and down .1 or .2, but always climbing higher.  Clicking the "X" on the render window didn't abort and TG4 gave a "not responding", so again I had to kill the process.

I'm at loss on what to try next.  I wish I would have kept that scene that used 83 GB and successfully rendered.  I'm pretty sure I had a bunch of populations -- probably 15 with large areas but not as condensed as my test today.   

Maybe trying the command line will help narrow this down?

(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

Dune

Thanks for these comprehensive replies. At least it's good to know SSD VM can be helpful if memory requirements surpass the banks a bit, despite rendertime.