Terragen 2 crash on negative displacement again...

[jritchie777]

Didn't take me long to break T2 again.  Don't know what happened, preview renders on low detail were fine.  Then tried a final render - half way through got a whole string of ray trace errors and a heightfield error - I was not even using a heightfield.  Anyone know what this all means?
Thanks,
JR

[jo]

Hi,

It looks to me like you've run out of memory. Looking at the file you've set the subdiv cache size to 2600 MB and to have it preallocated. If you're using 32 bit Windows then this is very close to the limit of how much memory you can use if you're using the /3GB switch (approx. 3000 MB). It's far too much. You can see that the second error message is "Unable to allocate memory for subdiv cache". That's a good hint that your subdiv cache is too large. It may still work if you uncheck "Preallocate subdiv cache", but I would suggest taking it down to a smaller value because otherwise TG2 will try to use up to the amount you've set and that may be too much. Have you tried it using the default 400 MB?

You might want to try reading the error messages :-). The "Unable to allocate memory for subdiv cache" error actually goes on to explain how to fix the problem.

All the "Unknown error" errors following the "Unable to allocate memory for subdiv cache" are a very good indication that you've run out of memory.

The heightfield error comes about from a Heightfield load node in node inside your Plane 01 node. I found it by looking through the nodes in the Node Network project view list. The actual Heightfield load node was hidden behind the Default shader node in the network view, I didn't spot it there immediately.

BTW, TG2 hasn't actually crashed here. It's done the right thing - you've tried to make it do something it isn't able to and it's stopped doing it before it actually crashes i.e. "the application terminates unexpectedly" or whatever it is Windows says when an app actually crashes.

Just a note regarding the "Preallocate subdiv cache" setting - if you set it to a large amount you may find it fails even though you think you've set it smaller than the amount of memory available. This is because it may not be able to allocate such a large block of memory in one piece. As the application gets used there is the potential for the whole memory space to be broken up into chunks and sometimes that means there isn't a big enough chunk for one big allocation like that. This may be more of a factor if you've been using TG2 a while.

Regards,

Jo

[jritchie777]

I did read the error messages.  Why would it stop half way through and all of a sudden not have enough memory???  And what about the Heightfield error message - I wasn't using a heightfield.

And the default 400MB - way too slow!

[jritchie777]

Let's break this down then:
1) I deleted the heightfield shader from the tree and later found a lone node in the node list that I deleted as well.  So why are there orphaned nodes in the node list when you delete the heightfield from the treeview pane?

2) Pre-Allocate:  I think we differ on the definition of pre-allocation of memory.  As a programmer if I pre-allocate a chunk of memory (and get a successful return value) then I have the memory until I release it.  I don't have to keep "pre-allocating" memory I already have it.  So my question is what does the "pre-allocate" switch really do and how is T2 handling memory?  Does it keep releasing and requesting memory, which defeats the purpose of pre-allocating the memory in the first place, but I haven't seen the code so I'll let you tell me how T2 is using the "pre-allocation" of memory.

Thanks.
JR

[Henry Blewer]

I have run into this on my computer. It is not so much a RAM problem, but is related. The problem is with the SWAP file. I had to adjust mine to 8 GB. There is a way to fix the RAM by changing the large address aware flag in the registry.

I lost the link to Microsoft's knowledge base when I went to Windows 7. But I want to adjust it on Window 7 also. Stay tuned. I'll see if I can find it again.

[Henry Blewer]

http://www.tomshardware.com/forum/238910-45-large-address-aware

Found some info at a site I find reliable.

[jritchie777]

Interesting.  I have mine set at 2.5 times my total memory.  I believe the optimum is twice the installed memory, unless MS has changed it for Win7.

I still don't see the difference though between pre-allocating memory versus allocating on the fly as far as how T2 seems to be operating.

Also conducted an experiment.  I tend to like to use a powerfractal on my terrain instead of the heightfield shader.  So I load the powerfractal and move it to first position then deleted the heightfield shader.  Looked in the node tree and low and behold, there was the orphaned node.  Why is it not deleted from here as well, like every other node?  And it still takes up memory- aaarrrgggghhhh!

[jritchie777]

Here is the info:
The virtual address space of processes and applications is still limited to 2 GB unless the /3GB switch is used in the Boot.ini file. When the physical RAM in the system exceeds 16 GB and the /3GB switch is used, the operating system will ignore the additional RAM until the /3GB switch is removed. This is because of the increased size of the kernel required to support more Page Table Entries. The assumption is made that the administrator would rather not lose the /3GB functionality silently and automatically; therefore, this requires the administrator to explicitly change this setting.

The /3GB switch allocates 3 GB of virtual address space to an application that uses IMAGE_FILE_LARGE_ADDRESS_AWARE in the process header. This switch allows applications to address 1 GB of additional virtual address space above 2 GB.

The virtual address space of processes and applications is still limited to 2 GB, unless the /3GB switch is used in the Boot.ini file. The following example shows how to add the /3GB parameter in the Boot.ini file to enable application memory tuning:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(2)\WINNT
[operating systems]
multi(0)disk(0)rdisk(0)partition(2)\WINNT="?" /3GB

More on this can be found at:  http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx

[neuspadrin]

Whats your system specs?

And are you saying it runs at 400mb just fine just "too slow"?  You could try somewhere between 400 and 2600.... more towards the 400 side probably. 

[jritchie777]

I think the point is getting missed here, I know I can select less memory for rendering.  The question I have is if 2600 is successfully pre-allocated and works for half of a render - where did the memory go unless T2 released it and had to reallocate it and then the operation failed.  Thus my question on what does the "pre-allocate subdiv cache" really do?

[jo]

Hi,

AFAIK if you have "Preallocate subdiv cache" turned on then the subdiv cache is allocated when rendering begins. Before that setting was available it was always allocated on the fly up to the maximum you specified, which could lead to problems when the subdiv cache limit you'd set was reached. By preallocating the subdiv cache you can know ahead of time that the subdiv cache at least was safely allocated. Matt would need to clarify this, but I believe if the subdiv cache allocation fails then it goes back to allocating the cache on the fly, which is why the render proceeds. However I think it will continue allocating subdiv cache up to the very high limit you've set, and eventually that causes memory allocations to begin failing, which I'm 99% sure are causing the other errors you're seeing.

The cache is disposed of when rendering finishes. It doesn't make sense to have a big chunk of memory only used for rendering hanging around not doing anything, the shortage it potentially causes could prevent you doing other things with the app.

I think the "Unable to allocate memory for subdiv cache" error is actually shown at the beginning of rendering, not half way through as you think. The other errors would come later in the rendering process.

Using a 2600 MB subdiv cache, preallocated or not, is too large for 32 bit TG2. You need to try something smaller.

Regards,

Jo

[jritchie777]

I think the "Unable to allocate memory for subdiv cache" error is actually shown at the beginning of rendering, not half way through as you think. The other errors would come later in the rendering process.

Using a 2600 MB subdiv cache, preallocated or not, is too large for 32 bit TG2. You need to try something smaller.

Thanks that was the info I was looking for.  Max would be 2048 then.  As for the errors, the only error I get before I start rendering is unable to find heighfield 01 file.  I can't find the missing node, and still confused as to why it is still in the node tree when I delete it from the Terrain tree view.

[Oshyan]

Why even use a 2048 subdiv cache though? It's extremely high. I know in another thread people recommended that for improving render time. Did you prove through actual testing that it made a significant difference? If not, I'd suggest testing that to be sure. If the difference is not more than e.g. 50%, I don't think it's worth the risk, especially if you're pushing memory limits in other ways (e.g. high render resolution, high AA, etc.).

Perhaps you're not aware, but there are many other things besides the subdiv cache that take memory, some of which vary over the rendering process. So even if your memory for the subdiv cache was successfully allocated at the beginning, memory use through the render could vary e.g. with the image buffer or AA buffer, and thus cause a crash due to out of memory issues. Other things that can take up memory are images (either on objects or used as masks or textures on the terrain), heightfields, objects and populations, AA buffers, image buffer, etc. so it's really not a good idea to make assumptions about max buffer size allocation unless you have a very good idea of what else is using memory in the scene.

Btw njeneb, although increasing swap file size may help allocate large contiguous blocks of memory, swap file ("virtual") memory space is so much slower than real memory that it will dramatically slow down your renders. It's not at all recommended to use this as a solution.

- Oshyan

[Henry Blewer]

Sometimes there is not any other way to get the render done. I am hoping the 64 bit will help with RAM address space. Just upgraded to Windows 7 64 bit to take advantage of this when the new build is released.


                              _{The Media Center is so much better!}

[Oshyan]

njeneb, is that true though? I mean do we have example scenes that verifiably render correctly with higher cache values and not with lower values?

- Oshyan