Anim issue

[WAS]

"VRAM" is Video RAM, not Virtual RAM/memory. Definitely an important acronym not to mix up because they do very different things.

That being said I would absolutely not recommend trying to rely on Virtual Memory during rendering. It will slow things down *massively* once the RAM usage switches to virtual, on-disk "memory".

Readyboost mostly helps for bootup and program startup, and other frequently accessed small files. It doesn't work like virtual memory. It is intended to cache very specific, frequently used (and relatively small) things, in general. It's interesting that your experience has been it has helped with your rendering, but you are working in an extremely constrained environment (by modern rendering standards), so fair enough.

I'd sooner look at optimizing the render settings or scene elements to reduce memory usage for this scene. Minimize population areas (which are often larger than they need to be, especially if no masking is being used), use simpler population object geometry, or downsample textures to lower resolution, minimize the number of render elements in use, reduce Micropoly Detail and/or AA if possible, etc.

- Oshyan

Lol, That's dyslexia for you. You don't seem that familiar with windows virtual memory, though. Terragen shouldn't use paging unless it's forced to. In fact. Terragen should be allocating memory before rendering to prevent confliction, so those things that would conflict go to virtual memory, as RAM is allocated, leaving Terragen with physical RAM. Things running ontop of active programs will start using virtual memory as they will see they do not have enough memory to allocate... Not having sufficient space means programs like browsers opening up with near capped RAM that use a lot of memory, will cause applications to crash. Or like you mentioned drop box syncing files, or the forbidden Windows Update which will want to start caching compressed content for installers.

ReadyBoost doesn't even activate until Windows is booted, so not sure where you are getting that. In fact, the USB only recognizes with POST and sleeps until booted and windows recognizes it. ReadyBoost stores application files and data in cache. When I'm running my 7,200 with windows going, Steam, browsers, watching youtube, etc, this off-loads reading and writing. I don't know exactly what Windows reads/writes from it's cache file, but when TG is running with other stuff, you can see the counters ticking away and it's doing a lot, and the system runs more stable.

[Matt]

Virtual memory doesn't work that way, WAS. Please let Oshyan respond to the OP without confusing things. What Oshyan said about virtual memory is basically true unless you need to get into the nitty gritty details.

[WAS]

You should run you own software under subpar conditions with advised window settings for large RAM applications. You are wrong. It's exactly why virtual memory exists. Out of Physical RAM. That's why you never see message about physical ram but out of virtual memory because that's the last fail safe of memory to use in any event while RAM is allocated to something else. For example allocate almost all usable memory to a RAMDisk and than start running programs.off disk. It's crucial to system stability and default settings ARE NOT appropriate for rendering, let alone Photoshop or something less intensive. Virtual memory is a layer of the windows memory complex, part of RAM management. If a program tries to allocate more RAM than available, it is given virtual memory, while much slower. You can see this in real time with your memory diagnostics. However if it can allocate more memory, such as a program not holding on to it's memory, it will cache things not in use from RAM to free up RAM. This is why RAM allocation is crucial for things like RAMDisks, rendering, gaming, etc. As part of paging, this is also crucial for windows processes and services, as well as inactive programs you have running.

There's a reason editing size restrictions for performance on low RAM systems is advised across Winodws, Mac and Linux.. I can play games that require 8-16 GB ram such as KSP modded which with real scale solar system and planets, because of edited virtual ram, or I will run out of RAM and get a out of virtual memory and WDM start crashing along with the game.

Additionally on default settings like installing Windows on this new machine I could not render multiple 10k pops. Fine with virtual memory expanded to 12gb. This has always been a RAM limit until I discovered this with TG. Like in the past where I could only manage one population like my forest mountain seen back from like 2015.

Does Terragen even allocate memory? That should be a feature for actual renders.

[Matt]

You should run you own software under subpar conditions with advised window settings for large RAM applications. You are wrong.

What advised settings are you talking about? No-one ever said you shouldn't use virtual memory. Oshyan only said not to rely on it. My interpretation of what he said is to try to have enough physical RAM so that you don't rely on virtual memory, because if you start paging to disk it really slows things down. He never said to turn off the page file or anything like that. But you had argue to with him anyway.

It's exactly why virtual memory exists. Out of Physical RAM.

We know that. Oshyan never said otherwise.

There's a reason editing size restrictions for performance on low RAM systems is advised across Winodws, Mac and Linux.. I can play games that require 8-16 GB ram such as KSP modded which with real scale solar system and planets, because of edited virtual ram, or I will run out of RAM and get a out of virtual memory and WDM start crashing along with the game.

Additionally on default settings like installing Windows on this new machine I could not render multiple 10k pops. Fine with virtual memory expanded to 12gb. This has always been a RAM limit until I discovered this with TG. Like in the past where I could only manage one population like my forest mountain seen back from like 2015.

Of course. If you don't have enough physical RAM, virtual memory is essential.

Does Terragen even allocate memory? That should be a feature for actual renders.

Pretty much all software allocates memory.

Matt

[Matt]

Terragen shouldn't use paging unless it's forced to.

Right. And it doesn't. (It has no control over it though, that's up to the OS.)

In fact. Terragen should be allocating memory before rendering to prevent confliction, so those things that would conflict go to virtual memory, as RAM is allocated, leaving Terragen with physical RAM. Things running ontop of active programs will start using virtual memory as they will see they do not have enough memory to allocate...

Virtual memory does not work this way. When a program allocates a block of memory (I won't call it a page to avoid confusion), technically speaking it's always allocating through the virtual memory layer, and the OS manages whether or not that block is currently residing in physical RAM, or paged to disk, or both. (I believe that in the exact moment when a program is accessing a block of memory it must be in physical RAM or at least one of the hardware caches.)

Even if physical RAM is full, when a new program comes along and needs to allocate memory it's not forced to use the disk. But of course disk activity will occur because the OS needs to move something that was in physical RAM out of the way, onto disk, to free up RAM for the new allocation.

The OS does not hold on to physical RAM for whichever program got there first. So the idea that you can force one program to use physical RAM by allocating more RAM ahead of time is wrong. What really happens when a new program comes along to allocate and use some memory is that blocks of memory that haven't been used recently (and/or predicted not to be needed for a while) are paged to disk so that physical RAM can be freed up for the new allocation. The app that got there first isn't given priority, as far as I'm aware. The OS gives priority to whichever app needs the memory block now.

Not having sufficient space means programs like browsers opening up with near capped RAM that use a lot of memory, will cause applications to crash. Or like you mentioned drop box syncing files, or the forbidden Windows Update which will want to start caching compressed content for installers.

True.

[WAS]

There's actually a differences in committed virtual memory, and cached virtual memory. Committed memory is actually acting as slow RAM via the HDD. This is how I can do a render like right now that would be using well over my 8GB, but windows manages it nicely, and much of that object memory is held in committed RAM. In fact around 10GB of it as committed since around 500mb-1gb when not running anything. All that's going is this browser with this page which is only using 9.47mb ontop of firefox's 628mb for itself.  And of course Windows chunk of 0.5-1gb RAM.

Some programs can lock memory, at least I've seen it before where entering a desired amount of RAM will take up that much and while the application is running, anything trying to pull over that runs like crap due to virtual memory, and what RAM is available. But doesn't crash.

In any case, without the virtual ram (I don't use ReadyBoost on this computer due to 7200rpm HDD) TG crashes when doing full resolution renders with lots of objects. I'll wake up and there will just be no TG open. Since I followed tutorials on windows performance for artists and people rendering movies (wasn't focused on rendering) I have none, zero crashes. None. The only crashes come in the GUI's bugs. Random crashes swapping node inputs, or what not.

Maybe windows 10's memory complex handles things better than you're familiar with. As I've never done this on other OSs except back on like Windows 98. lol

Like I mentioned too, unless virtual memory is disabled, windows will give you an "Out of Virtual Memory" error when RAM and Virtual Memory is full, not a "Out of Physical Memory" error, which you'd get with no virtual memory, or is RAM is locked up to ramdisk or application that does so.

[Matt]

Yes I would hope Windows 10 is better than previous versions. Virtual memory is essential on low RAM systems, there is no argument here. Any further discussion on virtual memory should be moved to Open Discussion (but I won't take part in it anymore).

[Matt]

@archonforest,

Jo recently identified a bug that we think might be causing a memory leak. I don't know if that's what caused your issue, but we will release an update soon and maybe we'll see!

Matt

[WAS]

Yes I would hope Windows 10 is better than previous versions. Virtual memory is essential on low RAM systems, there is no argument here. Any further discussion on virtual memory should be moved to Open Discussion (but I won't take part in it anymore).

Look, I'm just trying to make a suggestion on what actually helps me run Terragen without crashing ever since I was complaining of crashes and Oshyan, or you made it clear my 4GB (now 8GB) wasn't enough for Terragen and my objects. I since found a way to bypass this limitation. Probably at a cost of speed no doubt. I'm sorry I have dyslexia and mixup terminology, but I wasn't the ones arguing over what they think they know in technicality compared to actually running programs with require more RAM than you have with virtual memory able to compensate for large files in memory. I'm not sure what ReadyBoost does on a micro system, but it definitely also gave improvements. Probably in I/O with TG's tiff images and such.

If he's hitting a peak and needed something done that he couldn't, this could very well allow him to finish it. All I was suggesting. I'm able to use max 3 pop's with large objects upwards 200-300mb in 10k+ populations for mountains because of this. I could manage 1 single population on my 4GB without extended Virtual Memory, and when I forgot to do it on this rebuilt desktop, same deal with my recent mountain scene. Though I didn't need to have 1k populations on a ground level scene and changed those to around 200m. But that was after remembering to increase memory limit. So I could do all 3 populations plus multiple single objects.

Also here is that SmartClose which is useful for cleaning up a workstation environment while being able to suspend previous setups in a snapshot.
http://bmproductions.fixnum.org/index.htm?http://bmproductions.fixnum.org/smartclose/index.htm

[archonforest]

@archonforest,

Jo recently identified a bug that we think might be causing a memory leak. I don't know if that's what caused your issue, but we will release an update soon and maybe we'll see!

Matt

Great!