I am rendering a very short train scene. The rendering stops at frame 33 with a "not enough memory" message. But why? What TG keeps in the memory that grows by each frame? I was thinking each frame is a new cycle and there is no reason why TG would not purge the memory every time a frame is finished. Can someone explain how this works? It sounds interesting.
The two things I would try would be to up the subdivision cache. (probably wrong)
You can also, since you should have a folder with 33 frames already....stop the rendering and then start it again beginning with frame 34....wouldn't hurt to try. I notice that time continues frame to frame. Curious event I agree.
There may be several reasons for this behavior, but I'd try to save your scene "render ready", restart your computer, open TG with the scene (nothing else!) and hit render. I had similar issues before with some scenes, and this helped sometimes. But I had to use luvsmuzik's solution as well on some occasions.
Okay thx guys. I will try both.
When you get that figured out, perhaps you can tell me why my perfect wheel is losing altitude upon rotation,,,,hehehe :) Pretty sure I have origin perfect center...is that wrong?
How much memory do you have? Are you watching Task Manager to see how much is being used?
Increasing the subdiv cache size would be more likely to *cause* such an error.
- Oshyan
Quote from: Oshyan on April 30, 2018, 07:34:01 PM
How much memory do you have? Are you watching Task Manager to see how much is being used?
Increasing the subdiv cache size would be more likely to *cause* such an error.
- Oshyan
On this pc I got only 4 gig of ram. I know it is very little but my workstation is busy with something else. Thus I wanted to render on this small machine since it is doing nothing anyways. TG is using the 4 gig pretty much. When it renders the 1st frame the ram is pretty full already. But I do not understand why it cannot render all the frames one by one and why it stops at 33. I am just curious about what is going on under the hood and what is filling up the leftover ram.
I too have a 4GIG on hand, but now have a 16GIG also.
I thought of a couple of things that might help.
Just before you are ready to push Render Sequence, in the preview window, hide your objects. Then put your viewport to Terrain. Make sure RTPreview is off also. All of these features really slowed down my unit on a complex scene. If you have a lot of populations, see if using pop cache helps.
I am just giving some hints that may or may not have a thing to do with your problem, but my unit handled TG better doing those things.
Thx luvsmusik. I just put another 4g in the pc so it runs better. It will do until my workstation frees up. That got 32g.
Does it *always* stop at 33? It's possible that your machine was so close to its memory limit that some other random process just happened to kick on (a Dropbox sync as one example) and pushed it over the total amount of available RAM at a random time.
If you still want to try to render the whole thing in one go (rather than resuming at 33), I'd suggest first restarting your machine, then load Terragen and your scene file and close the 3D Preview entirely. Ideally before any populating happens, if any. Then go ahead and render. Alternatively (and even better), use the commandline to render and you'll save even more memory.
But I see you now have 8GB so hopefully that will solve the problem anyway. :D
- Oshyan
Thx Oshyan. Yes it always stopped at 33. That was the only reason I was asking as it sounded interesting. Otherwise I was just doing a very low res render to see what is going on before it hits my workstation and I thought I can get away with the 4g but looks like I was wrong.
Is it possible there is anything different enough in frame 33 that could increase memory use a bit? You might be surprised what kind of things might use slightly more or less memory to render. In normal circumstances such small shifts aren't worth worrying about, but in a highly memory constrained situation like yours it could make a difference.
- Oshyan
it is a possibility since a long train is passing through the scene and frame by frame you see it more and more...hmm... this could be the reason.
That is what I was figuring, knowing how many parts those trains have...... :)
Last night I let again to render with the 8g inside. Now it stopped at 60. I doubled the memory and got double amount of frames. Now I restarted from frame 61 and the memory usage was 62 percent. Lets see what happens.
Probably no other way than restarting the animation where it stopped. :(
Yeah sound like. As I said before I restarted the render from 61 where the memory usage was around 60%. Now at frame 68 the memory usage is 75% which is makes no sense for me.(Fully offline PC). When I first rendered my scene as a still picture the whole train were on the picture and 4g was enough for it. Now the resolution is lower, the AA is lower and TG uses around 7g for a frame. Its only me who do not understand the logic behind?
Just did another test:
frame 75: mem usage 60%
frame 76: mem usage 64%
frame 77: mem usage 68%
frame 78: mem usage 71%
So something is staying in the memory and growing. So even though TG renders your first frames without a problem you might run out of ram later. Now I would like to know what is this staff that cannot be purged when a frame is completed.
I have the same effect with all animation renders but with only a minimal increase in memory usage.
I can also watch this effect "in high speed" in GI rendering. Sometimes when reaching 99-100% memory usage a part memory seems to be set free (down to 60-70% usage) and then it grows again.
Sometimes it doesn't matter at all and I can finish animation renderjobs without any problems. Sometimes (I think it's my older "only" 8GB memory-PC rendering in fact stops after 40-60 frames. Often in that case I additionally get the message: "display driver is stopped and couldn't be restarted - windows standard VGA driver is started" (something like this in german ;) )
So far I could not notice any regularities - only that 99% of all problems appear with windows 10. With Windows 7 I can even run renderjobs on a 8 GB PC which have problems under Windows 10 with 32 GB memory...
Thx Doc for the data. After some careful inspection it looks like the train object caused this problem. When the train started to move out from the screen then the memory usage also dropped. Hmmm..
Interesting. Also are you using GI Cache blending? The blending will use more memory than a still render.
- Oshyan
Quote from: Oshyan on May 02, 2018, 09:22:20 PM
Interesting. Also are you using GI Cache blending? The blending will use more memory than a still render.
- Oshyan
Most probably not since I do not know what is that :D
It's useful for reducing flicker in GI/lighting in animations. But if you're not aware of it I guess you aren't using it. ;)
- Oshyan
Hello World !
I have the same problem, it also depends of the image itself, my computer will be able to generate more images if they are simple (with nothing on it) than full images (with a lot of objects etc)
I have a i7 3.6Ghz processor with 16Gb DDR3 RAM, so I can renderer 100 up to 400 images before crashing.
I would like to let it run for 24H without caring ;-)
Quote from: zepeu on June 16, 2018, 07:46:34 AM
Hello World !
I have the same problem, it also depends of the image itself, my computer will be able to generate more images if they are simple (with nothing on it) than full images (with a lot of objects etc)
Before hitting the "renderer" buton, I systematically shut down everything I can except the anti virus and disconnect every external drive to free as much memory as possible
I have a i7 3.6Ghz processor with 16Gb DDR3 RAM, so I can renderer 100 up to 400 images before crashing.
I would like to let it run for 24H without caring ;-)
Have you monitored memory usage while rendering? Does it increase with each frame rendered?
- Oshyan
Actually no, I didn't monitor it, what software could be nice for that?
The thing is that my computer runs at 99% of what it is capable of, see by yourself :
[attach=1]
The loss of memory is between 2 frames
maybe having an inside feature permitting to limit the memory/CPU usage at 95% or something like that could be nice...
The OS still needs a bit of power to run.
By the way, I think it's more windows that shut down terragen cause it takes too much memory rather than terragen that bugs...
Just use windows for monitoring. I used that when I had this problem before.
Otherwise if u cannot put more ram in there then render only half or less of the frames you got. Or lessen the quality.
It seems clear that you are using too much memory for the render and any small change in needed memory between frames can then cause it to run out and crash. Look for ways to reduce memory use, including slightly reducing scene complexity (for example if you have high density populations, consider slightly increasing their spacing) reducing detail of resources that may not be in the foreground (for instance resize large textures or high polygon objects), and reduction of detail settings that may be higher than needed. Use of render elements (the more you enable, the higher the memory requirement) is another possible culprit, so try to minimize those. And whatever you do don't change the size of the subdiv cache. :D
- Oshyan
Try giving your machine a healthy sum of virtual memory (HDD cache). VRAM is crucial for things like Oshyan mentioned, and other processes kicking in while your RAM is allocated. These programs will see they do not have enough physical RAM to run, and will run of VRAM. Albeit much slower. By default this is usually pretty small, I think like 512mb, and some programs can't even allocate enough VRAM with other things using it. Often with a Windows install much more may happen than just a dropbox sync.
Also, a USB3.0 flashdrive activated as a ReadyBoost for windows will function as VRAM, and work much faster than your HDD which will see read/write from other sources such as applications, and TG.
Helps me by leaps and bounds rendering complex scenes with only 8GB of memory that would otherwise crash. Even worked on my intel stick with only 4GB of memory and large populations. I dedicated a whole 32GB flash drive to ReadyBoost. Though this is a windows feature.
Additionally the program "SmartClose" will close all programs and non-essential windows services freeing up both RAM and CPU utilization. Could even disable WDM (which is a good chunk of Windows RAM utilization) by killing "explorer.exe" and than when you want WDM and shell back, just Ctrl+Alt+Del and File->Run explorer.exe (Task Manager has it's own mini-wdm and shell instance specifically for it; which is why it can function with partial feezes).
"VRAM" is Video RAM, not Virtual RAM/memory. Definitely an important acronym not to mix up because they do very different things.
That being said I would absolutely not recommend trying to rely on Virtual Memory during rendering. It will slow things down *massively* once the RAM usage switches to virtual, on-disk "memory".
Readyboost mostly helps for bootup and program startup, and other frequently accessed small files. It doesn't work like virtual memory. It is intended to cache very specific, frequently used (and relatively small) things, in general. It's interesting that your experience has been it has helped with your rendering, but you are working in an extremely constrained environment (by modern rendering standards), so fair enough.
I'd sooner look at optimizing the render settings or scene elements to reduce memory usage for this scene. Minimize population areas (which are often larger than they need to be, especially if no masking is being used), use simpler population object geometry, or downsample textures to lower resolution, minimize the number of render elements in use, reduce Micropoly Detail and/or AA if possible, etc.
- Oshyan
Quote from: Oshyan on August 05, 2018, 07:38:32 PM
"VRAM" is Video RAM, not Virtual RAM/memory. Definitely an important acronym not to mix up because they do very different things.
That being said I would absolutely not recommend trying to rely on Virtual Memory during rendering. It will slow things down *massively* once the RAM usage switches to virtual, on-disk "memory".
Readyboost mostly helps for bootup and program startup, and other frequently accessed small files. It doesn't work like virtual memory. It is intended to cache very specific, frequently used (and relatively small) things, in general. It's interesting that your experience has been it has helped with your rendering, but you are working in an extremely constrained environment (by modern rendering standards), so fair enough.
I'd sooner look at optimizing the render settings or scene elements to reduce memory usage for this scene. Minimize population areas (which are often larger than they need to be, especially if no masking is being used), use simpler population object geometry, or downsample textures to lower resolution, minimize the number of render elements in use, reduce Micropoly Detail and/or AA if possible, etc.
- Oshyan
Lol, That's dyslexia for you. You don't seem that familiar with windows virtual memory, though. Terragen shouldn't use paging unless it's forced to. In fact. Terragen should be allocating memory before rendering to prevent confliction, so those things that would conflict go to virtual memory, as RAM is allocated, leaving Terragen with physical RAM. Things running ontop of active programs will start using virtual memory as they will see they do not have enough memory to allocate... Not having sufficient space means programs like browsers opening up with near capped RAM that use a lot of memory, will cause applications to crash. Or like you mentioned drop box syncing files, or the forbidden Windows Update which will want to start caching compressed content for installers.
ReadyBoost doesn't even activate until Windows is booted, so not sure where you are getting that. In fact, the USB only recognizes with POST and sleeps until booted and windows recognizes it. ReadyBoost stores application files and data in cache. When I'm running my 7,200 with windows going, Steam, browsers, watching youtube, etc, this off-loads reading and writing. I don't know exactly what Windows reads/writes from it's cache file, but when TG is running with other stuff, you can see the counters ticking away and it's doing a lot, and the system runs more stable.
Virtual memory doesn't work that way, WAS. Please let Oshyan respond to the OP without confusing things. What Oshyan said about virtual memory is basically true unless you need to get into the nitty gritty details.
You should run you own software under subpar conditions with advised window settings for large RAM applications. You are wrong. It's exactly why virtual memory exists. Out of Physical RAM. That's why you never see message about physical ram but out of virtual memory because that's the last fail safe of memory to use in any event while RAM is allocated to something else. For example allocate almost all usable memory to a RAMDisk and than start running programs.off disk. It's crucial to system stability and default settings ARE NOT appropriate for rendering, let alone Photoshop or something less intensive. Virtual memory is a layer of the windows memory complex, part of RAM management. If a program tries to allocate more RAM than available, it is given virtual memory, while much slower. You can see this in real time with your memory diagnostics. However if it can allocate more memory, such as a program not holding on to it's memory, it will cache things not in use from RAM to free up RAM. This is why RAM allocation is crucial for things like RAMDisks, rendering, gaming, etc. As part of paging, this is also crucial for windows processes and services, as well as inactive programs you have running.
There's a reason editing size restrictions for performance on low RAM systems is advised across Winodws, Mac and Linux.. I can play games that require 8-16 GB ram such as KSP modded which with real scale solar system and planets, because of edited virtual ram, or I will run out of RAM and get a out of virtual memory and WDM start crashing along with the game.
Additionally on default settings like installing Windows on this new machine I could not render multiple 10k pops. Fine with virtual memory expanded to 12gb. This has always been a RAM limit until I discovered this with TG. Like in the past where I could only manage one population like my forest mountain seen back from like 2015.
Does Terragen even allocate memory? That should be a feature for actual renders.
Quote from: WASasquatch on August 06, 2018, 01:38:37 PM
You should run you own software under subpar conditions with advised window settings for large RAM applications. You are wrong.
What advised settings are you talking about? No-one ever said you shouldn't use virtual memory. Oshyan only said not to
rely on it. My interpretation of what he said is to try to have enough physical RAM so that you don't
rely on virtual memory, because if you start paging to disk it really slows things down. He never said to turn off the page file or anything like that. But you had argue to with him anyway.
Quote
It's exactly why virtual memory exists. Out of Physical RAM.
We know that. Oshyan never said otherwise.
Quote
There's a reason editing size restrictions for performance on low RAM systems is advised across Winodws, Mac and Linux.. I can play games that require 8-16 GB ram such as KSP modded which with real scale solar system and planets, because of edited virtual ram, or I will run out of RAM and get a out of virtual memory and WDM start crashing along with the game.
Additionally on default settings like installing Windows on this new machine I could not render multiple 10k pops. Fine with virtual memory expanded to 12gb. This has always been a RAM limit until I discovered this with TG. Like in the past where I could only manage one population like my forest mountain seen back from like 2015.
Of course. If you don't have enough physical RAM, virtual memory is essential.
Quote
Does Terragen even allocate memory? That should be a feature for actual renders.
Pretty much all software allocates memory.
Matt
Quote from: WASasquatch on August 05, 2018, 11:59:28 PM
Terragen shouldn't use paging unless it's forced to.
Right. And it doesn't. (It has no control over it though, that's up to the OS.)
Quote
In fact. Terragen should be allocating memory before rendering to prevent confliction, so those things that would conflict go to virtual memory, as RAM is allocated, leaving Terragen with physical RAM. Things running ontop of active programs will start using virtual memory as they will see they do not have enough memory to allocate...
Virtual memory does not work this way. When a program allocates a block of memory (I won't call it a page to avoid confusion), technically speaking it's
always allocating through the virtual memory layer, and the OS manages whether or not that block is currently residing in physical RAM, or paged to disk, or both. (I believe that in the exact moment when a program is accessing a block of memory it must be in physical RAM or at least one of the hardware caches.)
Even if physical RAM is full, when a new program comes along and needs to allocate memory it's
not forced to use the disk. But of course disk activity will occur because the OS needs to move something that was in physical RAM out of the way, onto disk, to free up RAM for the new allocation.
The OS does
not hold on to physical RAM for whichever program got there first. So the idea that you can force one program to use physical RAM by allocating more RAM ahead of time is wrong. What really happens when a new program comes along to allocate and use some memory is that blocks of memory that haven't been used recently (and/or predicted not to be needed for a while) are paged to disk so that physical RAM can be freed up for the new allocation. The app that got there first isn't given priority, as far as I'm aware. The OS gives priority to whichever app needs the memory block
now.
Quote
Not having sufficient space means programs like browsers opening up with near capped RAM that use a lot of memory, will cause applications to crash. Or like you mentioned drop box syncing files, or the forbidden Windows Update which will want to start caching compressed content for installers.
True.
There's actually a differences in committed virtual memory, and cached virtual memory. Committed memory is actually acting as slow RAM via the HDD. This is how I can do a render like right now that would be using well over my 8GB, but windows manages it nicely, and much of that object memory is held in committed RAM. In fact around 10GB of it as committed since around 500mb-1gb when not running anything. All that's going is this browser with this page which is only using 9.47mb ontop of firefox's 628mb for itself. And of course Windows chunk of 0.5-1gb RAM.
Some programs can lock memory, at least I've seen it before where entering a desired amount of RAM will take up that much and while the application is running, anything trying to pull over that runs like crap due to virtual memory, and what RAM is available. But doesn't crash.
In any case, without the virtual ram (I don't use ReadyBoost on this computer due to 7200rpm HDD) TG crashes when doing full resolution renders with lots of objects. I'll wake up and there will just be no TG open. Since I followed tutorials on windows performance for artists and people rendering movies (wasn't focused on rendering) I have none, zero crashes. None. The only crashes come in the GUI's bugs. Random crashes swapping node inputs, or what not.
Maybe windows 10's memory complex handles things better than you're familiar with. As I've never done this on other OSs except back on like Windows 98. lol
Like I mentioned too, unless virtual memory is disabled, windows will give you an "Out of Virtual Memory" error when RAM and Virtual Memory is full, not a "Out of Physical Memory" error, which you'd get with no virtual memory, or is RAM is locked up to ramdisk or application that does so.
Yes I would hope Windows 10 is better than previous versions. Virtual memory is essential on low RAM systems, there is no argument here. Any further discussion on virtual memory should be moved to Open Discussion (but I won't take part in it anymore).
@archonforest,
Jo recently identified a bug that we think might be causing a memory leak. I don't know if that's what caused your issue, but we will release an update soon and maybe we'll see!
Matt
Quote from: Matt on August 06, 2018, 04:29:13 PM
Yes I would hope Windows 10 is better than previous versions. Virtual memory is essential on low RAM systems, there is no argument here. Any further discussion on virtual memory should be moved to Open Discussion (but I won't take part in it anymore).
Look, I'm just trying to make a suggestion on what actually helps me run Terragen without crashing ever since I was complaining of crashes and Oshyan, or you made it clear my 4GB (now 8GB) wasn't enough for Terragen and my objects. I since found a way to bypass this limitation. Probably at a cost of speed no doubt. I'm sorry I have dyslexia and mixup terminology, but I wasn't the ones arguing over what they think they know in technicality compared to actually running programs with require more RAM than you have with virtual memory able to compensate for large files in memory. I'm not sure what ReadyBoost does on a micro system, but it definitely also gave improvements. Probably in I/O with TG's tiff images and such.
If he's hitting a peak and needed something done that he couldn't, this could very well allow him to finish it. All I was suggesting. I'm able to use max 3 pop's with large objects upwards 200-300mb in 10k+ populations for mountains because of this. I could manage 1 single population on my 4GB without extended Virtual Memory, and when I forgot to do it on this rebuilt desktop, same deal with my recent mountain scene. Though I didn't need to have 1k populations on a ground level scene and changed those to around 200m. But that was after remembering to increase memory limit. So I could do all 3 populations plus multiple single objects.
Also here is that SmartClose which is useful for cleaning up a workstation environment while being able to suspend previous setups in a snapshot.
http://bmproductions.fixnum.org/index.htm?http://bmproductions.fixnum.org/smartclose/index.htm
Quote from: Matt on August 06, 2018, 04:30:42 PM
@archonforest,
Jo recently identified a bug that we think might be causing a memory leak. I don't know if that's what caused your issue, but we will release an update soon and maybe we'll see!
Matt
Great! :)