Problem of renders
Impossible to do one render even by modifying the parameters by using the function crop. I have several error messages which appear as of the beginning or in the course of render.
NO displacement, power fractal of redirect shader or others. Just a water shader and it is true 11 populations!
Il has to you it there one among you who would have a solution or must I await a version of more stable TG2 patiently and satisfy to me to make is to make “traditional” images or many returned in 600x450 which does not function either !
Most of these errors are simply due to the images not being where Terragen thinks they are. You'll have to go into each seed object and re-assign each texture. I have found that several of the models I have DL'd from various sites (here and ashundar) still have the original file locations in them. You may also want to check that the images are using the correct mapping, ie plan y or UV. If your using camera projection mapping make sure you assign a camera (auto crash if you dont :) )
I'm not sure this will sort out all your problems but it will go a long way
Best of luck
Richard
ps try (after re-assigning the textures) removing the water texture. Heavy reflectivity can be an issue.
The problem of re-allocation of textures is evident (unable to load image) what is incomprehensible is "unable to allocate pixelmap: size 12960000 bytes for example!
If I need to remove water, then what interest?
We see images of beta testers with a shader transparency which seems very interesting, when will we be able to make an image without incurring perpetual crashes!
In french "inflates me"! (ça me gonfle !)
I' have even errors of this type, that returns nutcase to me !
the reason to remove the water is to eliminate it from the cause of the render crashes. If the scene renders ok without the water then it is likely that the water is the cause. If the scene still throws up errors then the water is less likely to be the cause.
I believe that the "unable to allocate pixelmap: size 12960000 bytes" is from an internal calculation where terragen is effectively trying to creating an image of the reflection. As you can imagine this could become very intensive when you have multiple reflections on top of eachother.
Best of Luck
Richard
I' stripped the water the problem is always same the " unable to allocate pixelmap: size 12960000 bytes" ?
At the point when these errors happen, can you take a look at how much memory your system is using in the Windows Task Manager?
EDIT: when the error happens the numbers will probably decrease, so it may be sufficient to look at the numbers after a render has started and all the populations have finished "inserting instances". But numbers will increase during rendering, so it's useful to keep an eye on these numbers during the render.
[attachimg=1]
If this gets above 2 Gb then you may need to check how much Virtual Memory tgd.exe is using in the Processes tab. The Virtual Memory column can be added to the Processes list by going to "Select Columns..." in the View menu (when on the Processes tab). If the VM size for tgd.exe gets close to 2Gb then it will not be able to allocate memory during render operations. This is a limitation of 32-bit applications on Windows, no matter how much RAM or virtual memory you have. It applies to 32-bit Windows and 64-bit Windows, although there are ways to increase this to 3Gb and I am not sure exactly how the rules change in 64-bit Windows.
If TG is using too much memory then you may need to save some memory by using simpler objects or less instances in your populations.
Matt
OK Matt,
I have as you watched the manager asked stains see screenshots attached. The tgd.exe is less, I like you as indicated to me decrease my people I did more than 9 (sic!) That the air to function, I rule this out and I start rendering (I touch wood you never know!)
Thank you !
Joël
You may already have done this by now but under Edit in the toolbar, go to preferences, in the window goto library and content. For the library folder find the folder that you normally keep all your Terragen stuff in, I keep everything in a folder called My TG2 inside My Documents. Also in User content folders you might want to add the folders of the resources you use. I don't think it reads sub-folders so you'll have to add all the folders that have resources in them.
Good idea ! Thank you !
You know, I have encountered this problem a few times when doing my Dogs of War scene, with 5 popualtions with image heavy objects, and I have a fairly simple solution that allmost certainly will work!
The problem is, the 11 popualtions use a lot of RAM, like Matt told us.
"Unable to allocate pixelmap: size 12960000 bytes" as far as I know means there are large texture maps that don't fit in the memory anymore. When ram is full you get the unable to initialise rendbuffer because the bucket is full.
What kind of populations do you use? Trees? Do some of them use the same textures? If so, copy the default shaders in question to the main node view, and copy their names into the shader space in the multi shaders, like in the attached image.
If you have illiminated enough images you might be able to render, if not, you have to resize the textures, and perhaps save them at lower compression quality.
This has deffinately worked for me, I hope it works for you as well.
I use trees for this image with different textures although people work in pairs as they are on either side of a river. I do not see how to make this assembly as in the example that you put me?
A study all the same!
Thank it for your answer which is interesting!
Joël
Joël,
I think you need to enable the "Taille VM" column in Affichage -> "Select Columns" (sorry, I don't know what that is in French).
I think the important number may the sum of both Util. mémoire and Taille VM.
It would be useful to render the same scene that crashed for you. These numbers at the start of the render (after TG has finished inserting instances) are important to know, but I understand that you don't want to start another render that you know may crash.
Matt
Quote from: Phylloxera on April 23, 2008, 07:44:57 AM
I use trees for this image with different textures although people work in pairs as they are on either side of a river. I do not see how to make this assembly as in the example that you put me?
Then I think one part of the solution could be to open all the texture maps in PhotoShop or a similar app and reduce pixel size and compression quality (perhaps save as jpg and in stead of quality 10 choose quality 8...)
Quote from: Matt on April 23, 2008, 10:05:26 AM
Joël,
I think you need to enable the "Taille VM" column in Affichage -> "Select Columns" (sorry, I don't know what that is in French).
I think the important number may the sum of both Util. mémoire and Taille VM.
It would be useful to render the same scene that crashed for you. These numbers at the start of the render (after TG has finished inserting instances) are important to know, but I understand that you don't want to start another render that you know may crash.
Matt
This is Matt's done OK if I understood the sum of the two VM and Memory Useful must not exceed use file!
No not always trying for over a week to leave this image, I reduced the population at least their surfaces I went to an area of 20 000 x 200 to 2 000 x 200 and the problem is the toujors ?
I even reduces the reflection of water by 50% and up the camera, no way, no change!
"ça me gave" (french expression)"
Quote from: Mohawk20 on April 23, 2008, 10:54:49 AM
Quote from: Phylloxera on April 23, 2008, 07:44:57 AM
I use trees for this image with different textures although people work in pairs as they are on either side of a river. I do not see how to make this assembly as in the example that you put me?
Then I think one part of the solution could be to open all the texture maps in PhotoShop or a similar app and reduce pixel size and compression quality (perhaps save as jpg and in stead of quality 10 choose quality 8...)
Thanks it's a possibility of course ! But it's not a good solution for me !
Strange this render in a small size 260 x 195 parameters: details: 0.8 - AA: 3 and GI : 1 enabled
The time is very long 20 minutes and this is not finished and we did not see the successive passes as usual ?
The models you use, are they free?
If so, can you point me to their download location, and post the tgd here? I'd like to try some things before giving more advice.
I am sorry that the scene you are working on is causing so many problems. Without knowing how much memory Terragen is using when it renders your scene (memory use + VM size), I do not know whether the problem is simply that Terragen has run out of address space (2Gb). There is little I can do to reduce the amount of memory used by populations, imported objects and textures. (OK, Terragen could handle textures more efficiently, but I think vegetation textures usually are not the main problem.)
With populations, the main thing that affects memory use is the number of instances. You should be able render 2 or 3 million instances in total (perhaps more if your scene uses few resources), but if you have multiple populations then each population would need to be reduced.
If you have heavy objects in the scene (no matter whether they are single objects or populated), they will use memory, leaving less available to the populators, so for a scene with multiple complex objects you're unlikely to be able to render 2 million instances. When thinking about choosing simple objects for your populations remember that memory use works like this:
Memory needed to load object + memory needed to store positions/sizes/rotations of all the instances in the population.
Therefore, when deciding what objects should be made simpler, you don't need to choose the one that has the most instances. Choose the object that uses the most memory when it is loaded.
Matt
Quote from: Phylloxera on April 23, 2008, 12:08:39 PM
Strange this render in a small size 260 x 195 parameters: details: 0.8 - AA: 3 and GI : 1 enabled
The time is very long 20 minutes and this is not finished and we did not see the successive passes as usual ?
What about "GI surface details"? That affects render time more than most other GI settings. I would not use it for very complex scenes.
The GI pre pass probably finished very quickly. This is rendering the final pass, where GI surface details makes a very big difference to render time.
EDIT: In the next update this scene may render more quickly. GI lookup in the final pass was quite slow with complex vegetation and that has been improved, but I would still recommend NOT using GI surface details. GI works and gives reasonable results without it.
Quote from: Phylloxera on April 23, 2008, 11:35:31 AM
I even reduces the reflection of water by 50% and up the camera, no way, no change!
The reflectivity of the water has no effect on render time, memory usage, or render stability. If reflectivity is at 50% it still makes the same calculations as at 100%. The only thing that changes is the final appearance of the water. It takes the reflection it has calculated and adds it to the water with only 50% weighting.
Only if reflectivity is at 0% would it disable reflection calculations.
Matt
Quote from: Mohawk20 on April 23, 2008, 01:26:12 PM
The models you use, are they free?
If so, can you point me to their download location, and post the tgd here? I'd like to try some things before giving more advice.
Sorry I am using XFrog populations and a grass Lightning, so I can not save link. You post it. Tgd here would therefore nothing!
I'm doing one handed down by crop, lower throughout the height of the image width of 0.20. It is not easy!
Data Task Manager, rendering underway ...
Thanks for those screenshots. Those numbers are unfortunately high. Nearly 1 800 000 Kb in total (about 1.7 Gb). This only leaves a few hundred Mb for rendering before it reaches Windows' 2 Gb limit per application. Additional RAM won't help - this is a limitation imposed by Windows.
When rendering a decent sized image, the ray-tracing engine used for shadows and reflections (and GI surface details if enabled) can easily use another 250 Mb. With the crop render you may be OK, but any larger renders would probably tip the renderer over the edge with the larger bucket/tile sizes which need memory for their anti-aliasing buffer.
Matt
Matt, excuse my ignorance if so, but isn't it possible for virtual memory usage to go way beyond this Window's 2G limit?
Virtual memory doesn't allow you to break the 2Gb barrier. Virtual memory is basically a way of allowing the OS and applications to use more memory than the amount of physical RAM installed in the machine, but Windows still only allows up to 2Gb of virtual memory per 32-bit application. Virtual memory can allow your combined applications and OS to use much more memory, but Windows still only allows each application to see up to 2Gb.
Matt
In the past it has been mentioned that TG2 uses a proprietary GI method but there has been no mention of what it is based on, to hazed a guess form the render times reported (Even of late) I would, like some others here in the past have theorized that its some derivative of the Monte-Carlo Method which would enplane when combined with render time displacement would go some way toward the slow render times.
I thought that in recent years in the research and academic areas that there had been a large body of pure research done into refining the Monte-Carlo method to make it faster at render time?
All of the above is just my take on this.
Regards to you.
Cyber-Angel ;D
Thanks, Matt. I'm looking for a magic bullet. ;D
Quote from: Matt on April 24, 2008, 12:02:45 PM
Virtual memory doesn't allow you to break the 2Gb barrier. Virtual memory is basically a way of allowing the OS and applications to use more memory than the amount of physical RAM installed in the machine, but Windows still only allows up to 2Gb of virtual memory per 32-bit application. Virtual memory can allow your combined applications and OS to use much more memory, but Windows still only allows each application to see up to 2Gb.
Matt
Quote from: Cyber-Angel on April 24, 2008, 12:17:35 PMI thought that in recent years in the research and academic areas that there had been a large body of pure research done into refining the Monte-Carlo method to make it faster at render time?
When computer scientists optimize the time complexity of an algorithm.. don't expect that this kind of optimization will help you in the real world. ;D
I recently saw a paper from some guy from Norway who had tested two algorithms.. one was optimal for the problem to be solved - O(log n), the other one was comparably worse O(log
²n).
Turned out that the "optimal" algorithm wasn't faster then the non-optimal one except for huge problem sizes (1.4e21). ;D
@calico Some operating systems use techniques like PAE to address more memory than it would normally. See http://en.wikipedia.org/wiki/Physical_Address_Extension
Thanks, Nikita. Interesting. I tried using the "fix" to take advantage of the extra RAM by manipulating the boot.ini, but it hasn't worked. Not sure why, but I'd like to try again.
The 3G-switch? Remember it only works with XP Professional.
Okay. It's working now. ;D The key is to use /3GB and not /3G. Ooops.
I'm using Professional and here is what is probably the safest way to do it -
Read this - http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx and then figure out how to edit the boot.ini. There you add the switch, but I'd recommend having two entries under the [Operating System] section. Something like this, depending on your OS that takes advantage of the PAE -
[boot loader]
timeout=15
default=multi(0)disk(0)rdisk(0)partition(2)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP Professional" /fastdetect
multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP Professional" /fastdetect /3GB
It's good to have the timeout so you can switch to the 3G memory OS. This will prevent possible screwups.
Hi C-A,
Terragen does some things that few other renderers can - particularly extreme displacement of large surfaces - and this means that many algorithms have to be adapted or completely redesigned to work in fairly uncharted domains.
A Global Illumination system can rarely be labelled with just one term. They are a combination of multiple algorithms which each aim to solve a different aspect of the overall global illumination problem. Terragen does apply Monte Carlo principles, but I have taken a few liberties. Many improvements could be made to make it statistically more correct and give much higher quality results in a shorter time, but most of the development time has gone into developing a proprietary GI lighting cache (loosely based on the idea behind an Irradiance Cache, but with an emphasis on improving the kind of appearance of very rough surfaces) and interfacing it all with very large procedural landscapes. Actually that second part is the most difficult because of the huge complexity of the procedural model. Developing a renderer is as much an art as it is a science (or so I like to tell myself ;) ), and you can't always lift existing techniques and apply them directly if the renderer has a lot of unusual technology. But there is certainly an endless supply of research which could directly benefit Terragen, given enough time and resources.
You should also consider the possibility that maybe Terragen isn't really as slow as you think it is, for what it is doing in some scenes. For what that's worth :) I know it would be a much better product if render times could be significantly reduced, and with the new update we've made many changes to improve its ability to work at higher resolutions where Terragen has tended to suffer the most.
Matt
Thanks for the reply Matt,
I had been wondering about it since I like to try and keep up with whats going on, one question if I may, is the Engine in TG2 based on the Paper "Robust Rendering of High Resolution Terrain" that you co wrote with Magnus Wrenninge and Mårten Larsson for Siggraph 2004?
;D
Regards to you.
Cyber-Angel
For the most part, Terragen 2 is the continuation of the robust terrain renderer that Magnus Wrenninge and I described in that 2004 sketch, but some of the specifics of the shader engine are different. In a separate sketch Mårten Larsson presented an alternative rendering method which he'd developed to improve rendering speed in animations (inter-frame caching), but that hasn't been applied to Terragen 2. Both Magnus and Mårten have made further contributions to Digital Domain's renderer independent from TG2, and likewise myself and the others at Planetside Software have continually developed TG2 beyond what was presented in the 2004 sketch by Magnus and myself.
Matt
Speaking of Mårten Larsson. Check this site out and download the PDFs. There is one that describes Procedural Oceans and even has a small render image with procedural crashing waves. Looks impressive judging from the image. :o
http://www.martenlarsson.com/martenlarsson2/page.php?page=1&subPage=1&officialmode=
and yes, there is lots more there at his site.
Hmm, that MOAM technique looks pretty nice...
Matt, can you use a technique like that? Or are you allready implementing it?
Quote from: Christopher on April 28, 2008, 12:58:37 AM
Speaking of Mårten Larsson. Check this site out and download the PDFs. There is one that describes Procedural Oceans and even has a small render image with procedural crashing waves. Looks impressive judging from the image. :o
http://www.martenlarsson.com/martenlarsson2/page.php?page=1&subPage=1&officialmode=
and yes, there is lots more there at his site.
Thanks for that link, it was a good read. :)
Quote from: Mohawk20 on April 28, 2008, 04:00:42 AM
Hmm, that MOAM technique looks pretty nice...
Matt, can you use a technique like that? Or are you allready implementing it?
Just as a random poll on one user's opinion concerning TG's development... I'd sooner see efforts put into MOAM-like techniques to bring the render times down along with solid data interchange features, before broader work on things like wave systems (as cool as they look, and they do :)) get a look-in. I'm guessing / hoping PS have this well in hand though. :)
Highly displaced terrain? http://www.martenlarsson.com/martenlarsson2/videos/TG_movie.mov Planets that look good close and far away? Faces in geometry? What are you holding out on us, Mr. Fairclough? ;D
Quote from: calico on April 28, 2008, 09:49:17 AM
Highly displaced terrain? http://www.martenlarsson.com/martenlarsson2/videos/TG_movie.mov Planets that look good close and far away? Faces in geometry? What are you holding out on us, Mr. Fairclough? ;D
Nothing really, we can do that with some workarounds. However, these things are made by Digital Domain, and are probably copyrighted by them.
After reading all those articles, it seems like they have their own customized version of Terragen 2... which seems to be faster as well.
So Matt, maybe you might want to get those guys on the planetside team, go get some of the workload off you shoulders?
IIRC Matt worked at Digital Domain. When he left, he continued to develop TG2. Digital Domain continued to develop what they now call Engen.
So...I seem to remember that planet backward zoom shot on the Planetside site. Is this something we could actually do as well as what we see there or is this movie an example of what can be done with the SDK?
Quote from: nikita on April 28, 2008, 12:13:46 PM
IIRC Matt worked at Digital Domain. When he left, he continued to develop TG2. Digital Domain continued to develop what they now call Engen.
Quote from: calico on April 28, 2008, 09:49:17 AM
Highly displaced terrain? http://www.martenlarsson.com/martenlarsson2/videos/TG_movie.mov Planets that look good close and far away? Faces in geometry? What are you holding out on us, Mr. Fairclough? ;D
Nothing at all. It's all down to how you use the technology :)
Matt
Quote from: calico on April 28, 2008, 01:26:15 PM
So...I seem to remember that planet backward zoom shot on the Planetside site. Is this something we could actually do as well as what we see there or is this movie an example of what can be done with the SDK?
The Mars movie was rendered entirely in alpha versions of Terragen 2, using the MOLA Map Shader and some other fractal shaders.
Matt
Cool, Matt. Perhaps I'm being arrogant to assume I can, but this gives me hope. Thanks for letting us know even more...awesome works going on with all you've been involved with.
Quote from: Moose on April 28, 2008, 07:20:07 AM
Thanks for that link, it was a good read. :)
No problem. :)
With the new version of TG2, previously identified problems are solved! (http://smileys.sur-la-toile.com/repository/Respect/0014.gif)
Quote from: Matt on April 24, 2008, 12:02:45 PM
Virtual memory doesn't allow you to break the 2Gb barrier. Virtual memory is basically a way of allowing the OS and applications to use more memory than the amount of physical RAM installed in the machine, but Windows still only allows up to 2Gb of virtual memory per 32-bit application. Virtual memory can allow your combined applications and OS to use much more memory, but Windows still only allows each application to see up to 2Gb.
Matt
Is it possible to have T2 to Calculate "blocks" of the image and store it to the Hard Drive rather than keep the image in momory so it can free up the additional memory?
For very large images this is probably a good idea, but the image itself doesn't use anywhere near as much memory as what's needed to render it so that's where I'm spending the most effort to improve efficiency for the time being.
Always the same problem, parts of the image are calculated evil!
Are there a solution for this?
Even using the crops the problem persists?
I had the same problem so I quit using the most recent Alpha and went back to the previous version.
I haven't seen these myself but I'm now working on a later revision so it may be fixed. It will be important to re-test once the next update is available.
- Oshyan
@Oshyan, I'm already in the queue for the next version. ;D
Again I'll say that every time I've seen similar things in the Alpha (Though Technically Tech Preview release 3) once the render is further along they always go away; it seems to be the way TG2 now handles the way geometry is passed to the drawing engine at render time, where some times part of the render is not fully completed at that time by the drawing engine which could be that is not enough room is memory to complete that small section, once a more complected section is completed and that memory segment is available then those small square sections are done. Just a theory.
Regards to you.
Cyber-Angel ;D
i have this problem to!
im currently rendering a picture and part of the mountain is boxed out
by the way that picture is amazing hope to see it fully completed soon!!!
dito @Ca... Be patient. If parts are missing and it's still rendering there's no reason to assume it won't be rendered at all.
I have yet to see a finished render with missing parts. All I saw till now were aborted renders that are missing parts of the image and well.. if you prevent TG2 from finishing all the tiles and then wonder about unfinished tiles.. that doesn't make sense.
Quote from: nikita on May 10, 2008, 11:51:09 PM
dito @Ca... Be patient. If parts are missing and it's still rendering there's no reason to assume it won't be rendered at all.
I have yet to see a finished render with missing parts. All I saw till now were aborted renders that are missing parts of the image and well.. if you prevent TG2 from finishing all the tiles and then wonder about unfinished tiles.. that doesn't make sense.
I was more referring to the areas in the image that Phylloxera indicated rendering errors on (the small squares) not the black portion of the image, which as you say is uncompleted rendering. I was trying to say that my theory about why some people are seeing those small squares at render time is that those parts of the render have not yet been yet completed by the drawing engine, I think (and I could be way off) is that TG2 uses a back to front rendering scheme.
With that in mind and maybe due to the way Terragen handles memory allocation at the moment there are ocasions when small sections of rendering are not completed all at the same time and are passed of in stages I think thats whats going on here.
;D
Regards to you.
Cyber-Angel
That's what I'm talking about too. :)
Quote from: calico on May 10, 2008, 05:48:46 PM
@Oshyan, I'm already in the queue for the next version. ;D
Me too ! ;D
QuoteWe do not recommend using this build for final production rendering
on the download page of the last version... damn ! we've been warned ;D