4.5.38 Memory Management

Started by WAS, August 25, 2020, 02:14:02 pm

Previous topic - Next topic

WAS

Was anything changed in frontier handling memory? Terragen is now causing Windows to throw system interrupts left and right  (hangs TG, freezes PC, UI gets slow), and keep getting constant "Low Virtual Memory" warnings, for simple things like a blank project with a starfield shader on the background sphere. Seems to be consuming waaaay more memory than usual, and hitting 32gb limit on every project.

WAS

Anyone else using frontier 4.5.38, can they check their event viewer and see if they are get low memory errors and system interrupts? Never had these before and I fear windows throwing system interrupts could crash TG during render steps or cause issues in the renders themselves.

This is happening is bare bones projects, like just making stars or testing things in default scene, not even with populations and such. It's almost like TG is calculating how it can use all the resources available at once. Weird bit is how it's hitting virtual memory harder than actual physical memory is weird too.

Dune

I never have these problems, but I can have a look at an events viewer. You only have to tell me what and where that is on win 7.

WAS

August 31, 2020, 12:24:59 pm #3 Last Edit: August 31, 2020, 01:32:21 pm by WAS
Under "Summary of Administrator Events" on the main page of Windows Event Viewer you can expand the "Warnings" menu, and then expand "Resource-Exhaustion-Detector"

If you have anything it should read like

Quote from: Event ViewerWindows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: tgd.exe (9572) consumed 16136241152 bytes

For example that latest prompt is 15388.71875mb used in a matter of second rendering my stars project (I cleared the past warnings) I just started up.

Why TG isn't using physical memory first, before virtual memory, is really puzzling. In past versions TG uses physical memory, and overflows into virtual memory. But there was only 2gb being used during that render, while virtual memory was nuked.

WAS

September 01, 2020, 03:14:56 pm #4 Last Edit: September 01, 2020, 04:09:34 pm by WAS
Yeah I can't even render animations sequences of my stars. The computer restarts form resource exhaustion, and this scene shouldn't be tapping resources really at all.

It seems also with a sequence memory builds up until nothing is left for even the OS. Got a Out of Memory error I couldn't even clear before I got a Kernal-Power error and power cycled on the machine.

Before the critical out of memory error though PerfOS completely filled with this error: "Unable to collect NUMA physical memory utilization data. The first four bytes (DWORD) of the Data section contains the status code." for the process 6684 which I believe was Terragen.

Last frame completed (61 of 1200) attached... not much going on. Each TIFF is 5.6~ mb

Doing another sequence watching the process, it consumes a max 3,777mb of memory, and drops to 702mb before starting another frame. All this consumption is happening in virtual memory on C drive for some reason, not in actual physical memory where 18gb is cached, and 17gb committed, climbing with each frame. As of finishing writing that last sentence now 19.9gb committed and 19.1gb cached. Only 35.6gb of virtual memory available based on [32gb] physical memory (no alterations to extend virtual memory like on my old machines with low RAM)

Matt

Can you send me a TGD that produces this weird memory behaviour?
Just because milk is white doesn't mean that clouds are made of milk.

WAS

I get the Resource Exhaustion warnings even with the default TGD scene but I haven't tried animating it for a full meltdown. I'll send ya a TGD.

WAS

To note I searched about on the NUMA error but really can't find what it actually means. Every response is to just rebuild the services list.

WAS

Going to run a memtest over night, maybe RAM is faulty past a certain load and overflowing into cache because of some safety in Windows, though I'd assume there would be some IO exceptions with memory but you never know.