Anim issue

Started by archonforest, April 30, 2018, 05:09:52 AM

Previous topic - Next topic

Hannes

Probably no other way than restarting the animation where it stopped. :(

archonforest

#16
Yeah sound like. As I said before I restarted the render from 61 where the memory usage was around 60%. Now at frame 68 the memory usage is 75% which is makes no sense for me.(Fully offline PC). When I first rendered my scene as a still picture the whole train were on the picture and 4g was enough for it. Now the resolution is lower, the AA is lower and TG uses around 7g for a frame. Its only me who do not understand the logic behind?

Just did another test:
frame 75: mem usage 60%
frame 76: mem usage 64%
frame 77: mem usage 68%
frame 78: mem usage 71%

So something is staying in the memory and growing. So even though TG renders your first frames without a problem you might run out of ram later. Now I would like to know what is this staff that cannot be purged when a frame is completed.
Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

DocCharly65

I have the same effect with all animation renders but with only a minimal increase in memory usage.

I can also watch this effect "in high speed" in GI rendering. Sometimes when reaching 99-100% memory usage a part memory seems to be set free (down to 60-70% usage) and then it grows again.

Sometimes it doesn't matter at all and I can finish animation renderjobs without any problems. Sometimes (I think it's my older "only" 8GB memory-PC rendering in fact stops after 40-60 frames. Often in that case I additionally get the message: "display driver is stopped and couldn't be restarted - windows standard VGA driver is started" (something like this in german ;) )

So far I could not notice any regularities - only that 99% of all problems appear with windows 10. With Windows 7 I can even run renderjobs on a 8 GB PC which have problems under Windows 10 with 32 GB memory...

archonforest

Thx Doc for the data. After some careful inspection it looks like the train object caused this problem. When the train started to move out from the screen then the memory usage also dropped. Hmmm..
Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

Oshyan

Interesting. Also are you using GI Cache blending? The blending will use more memory than a still render.

- Oshyan

archonforest

Quote from: Oshyan on May 02, 2018, 09:22:20 PM
Interesting. Also are you using GI Cache blending? The blending will use more memory than a still render.

- Oshyan

Most probably not since I do not know what is that :D
Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

Oshyan

It's useful for reducing flicker in GI/lighting in animations. But if you're not aware of it I guess you aren't using it. ;)

- Oshyan

zepeu

Hello World !

I have the same problem, it also depends of the image itself, my computer will be able to generate more images if they are simple (with nothing on it) than full images (with a lot of objects etc)

I have a i7 3.6Ghz processor with 16Gb DDR3 RAM, so I can renderer 100 up to 400 images before crashing.
I would like to let it run for 24H without caring ;-)
zepeu

;-)

zepeu

Quote from: zepeu on June 16, 2018, 07:46:34 AM
Hello World !

I have the same problem, it also depends of the image itself, my computer will be able to generate more images if they are simple (with nothing on it) than full images (with a lot of objects etc)
Before hitting the "renderer" buton, I systematically shut down everything I can except the anti virus and disconnect every external drive to free as much memory as possible

I have a i7 3.6Ghz processor with 16Gb DDR3 RAM, so I can renderer 100 up to 400 images before crashing.
I would like to let it run for 24H without caring ;-)
zepeu

;-)

Oshyan

Have you monitored memory usage while rendering? Does it increase with each frame rendered?

- Oshyan

zepeu

Actually no, I didn't monitor it, what software could be nice for that?
The thing is that my computer runs at 99% of what it is capable of, see by yourself :

[attach=1]

The loss of memory is between 2 frames

maybe having an inside feature permitting to limit the memory/CPU usage at 95% or something like that could be nice...
The OS still needs a bit of power to run.
By the way, I think it's more windows that shut down terragen cause it takes too much memory rather than terragen that bugs...
zepeu

;-)

archonforest

Just use windows for monitoring. I used that when I had this problem before.
Otherwise if u cannot put more ram in there then render only half or less of the frames you got. Or lessen the quality.
Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

Oshyan

It seems clear that you are using too much memory for the render and any small change in needed memory between frames can then cause it to run out and crash. Look for ways to reduce memory use, including slightly reducing scene complexity (for example if you have high density populations, consider slightly increasing their spacing) reducing detail of resources that may not be in the foreground (for instance resize large textures or high polygon objects), and reduction of detail settings that may be higher than needed. Use of render elements (the more you enable, the higher the memory requirement) is another possible culprit, so try to minimize those. And whatever you do don't change the size of the subdiv cache. :D

- Oshyan

WAS

#28
Try giving your machine a healthy sum of virtual memory (HDD cache). VRAM is crucial for things like Oshyan mentioned, and other processes kicking in while your RAM is allocated. These programs will see they do not have enough physical RAM to run, and will run of VRAM. Albeit much slower. By default this is usually pretty small, I think like 512mb, and some programs can't even allocate enough VRAM with other things using it. Often with a Windows install much more may happen than just a dropbox sync.

Also, a USB3.0 flashdrive activated as a ReadyBoost for windows will function as VRAM, and work much faster than your HDD which will see read/write from other sources such as applications, and TG.

Helps me by leaps and bounds rendering complex scenes with only 8GB of memory that would otherwise crash. Even worked on my intel stick with only 4GB of memory and large populations. I dedicated a whole 32GB flash drive to ReadyBoost. Though this is a windows feature.


Additionally the program "SmartClose" will close all programs and non-essential windows services freeing up both RAM and CPU utilization. Could even disable WDM (which is a good chunk of Windows RAM utilization) by killing "explorer.exe" and than when you want WDM and shell back, just Ctrl+Alt+Del and File->Run explorer.exe (Task Manager has it's own mini-wdm and shell instance specifically for it; which is why it can function with partial feezes).

Oshyan

"VRAM" is Video RAM, not Virtual RAM/memory. Definitely an important acronym not to mix up because they do very different things.

That being said I would absolutely not recommend trying to rely on Virtual Memory during rendering. It will slow things down *massively* once the RAM usage switches to virtual, on-disk "memory".

Readyboost mostly helps for bootup and program startup, and other frequently accessed small files. It doesn't work like virtual memory. It is intended to cache very specific, frequently used (and relatively small) things, in general. It's interesting that your experience has been it has helped with your rendering, but you are working in an extremely constrained environment (by modern rendering standards), so fair enough.

I'd sooner look at optimizing the render settings or scene elements to reduce memory usage for this scene. Minimize population areas (which are often larger than they need to be, especially if no masking is being used), use simpler population object geometry, or downsample textures to lower resolution, minimize the number of render elements in use, reduce Micropoly Detail and/or AA if possible, etc.

- Oshyan