Planetside Software Forums

General => Terragen Discussion => Topic started by: Upon Infinity on February 11, 2011, 10:36:13 PM

Title: New Errors on large file sizes.
Post by: Upon Infinity on February 11, 2011, 10:36:13 PM
Hey,

I'm getting a whole new set of errors on images I have previously rendered in the past without problems on large render sizes.  Render size was 3300 x 2475.  The render was fine until about half way through and then stopped.  Any ideas.
Title: Re: New Errors on large file sizes.
Post by: choronr on February 11, 2011, 10:45:58 PM
Try reducing the cache size from the 400 default to 200.
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 11, 2011, 11:52:28 PM
Will do.  Results tomorrow.  But why all of a sudden would that NOT render properly?
Title: Re: New Errors on large file sizes.
Post by: Cyber-Angel on February 12, 2011, 08:51:39 PM
I've had these kinds of errors my self, you could also try reducing the Bucket size and see if that helps?

Regards to you.

Cyber-Angel 
Title: Re: New Errors on large file sizes.
Post by: Kadri on February 12, 2011, 09:41:03 PM
Quote from: UponInfinity on February 11, 2011, 11:52:28 PM
...But why all of a sudden would that NOT render properly?

I don't know if there is another explanation (maybe version differences and so)  , but the other programs in windows that you use at render time do use RAM too obviously .
So it could be that the free memory circumstances are not the same .
Try to shut down programs that you not need at render time to free up more memory , UponInfinity ( no pun intended :)).

Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 12, 2011, 09:47:49 PM
Ok, here's the results.  Not at all encouraging.  Render failed at the same place, and have these weird splotchy rectangles.  Please note, that this scene was originally created with the Technology Preview, so I'm wondering if the new update is not backwards compatible with these scenes, yet why start the render and not finish?  I rendered this same scene about a week and half ago (but not quite as large) with no problems.  It can't be the RAM, I just did an upgrade and there's always plenty left to go around.  This is the first time I've ever had these kind of problems.
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 12, 2011, 09:55:11 PM
Also, will reducing the bucket size affect image quality?  The box for allowing auto-reduction is ticked, but I guess that's the next try.  Also, if not the defaults, what values are ideal?
Title: Re: New Errors on large file sizes.
Post by: choronr on February 12, 2011, 10:09:37 PM
What are your quality settings, including clouds and atmo? Is water in the scene?
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 12, 2011, 10:53:46 PM
Quality is default

0.5 detail, 3 AA,
No water
1 cirrus 2D layer of clouds
1 cumulous 3D 0.422171 quality , 69 samples
and 1 more cumulous 'fog' layer 0.501113 quality, 7 samples.
Title: Re: New Errors on large file sizes.
Post by: Oshyan on February 12, 2011, 11:02:53 PM
Are you using any new rendering options since your last successful render, e.g. Raytrace Atmosphere? Or have you changed any rendering settings, like render buffer size, number of threads, etc?

- Oshyan
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 12, 2011, 11:08:54 PM
No, nothing fancy like that.  I have added a bit to the scene, however.  The 'fog' layer of clouds, couple objects.  Changed a texture on some rocks to 'power fractal shader'.  Again, nothing too crazy.
Title: Re: New Errors on large file sizes.
Post by: Oshyan on February 12, 2011, 11:33:28 PM
What are your specific render settings and system details? Were you monitoring Task Manager at the time and if so, how much memory was TG using?

- Oshyan
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 12, 2011, 11:39:39 PM
Specific render setting above - mostly default.

System details:

2.8 Ghz dual core machine.
3 gb RAM
WinXP

I did have a litte trick I was using where I would put the tgd.exe process into low priority mode, so I could work on other things while renders went through, however for the past render I had switched it back up to normal, although not restarted the PC or program.  Today I have restarted the PC and I will attempt the suggestions for settings above tonight, unless I can get something better.  Failing that, I'm looking at switching off everything I've added until I can track down the problem.
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 12, 2011, 11:41:48 PM
Oh, and no I wasn't monitoring.  I'll watch for that tonight too.
Title: Re: New Errors on large file sizes.
Post by: Oshyan on February 12, 2011, 11:46:35 PM
Very strange. I forgot you had posted the detail settings, and they're very reasonable, should not be causing any problems even at that resolution. 3GB of RAM is a bit low and XP means you only get 2GB of RAM for TG2 unless you are using the /3GB switch. A reboot might indeed help. Is your current render size the same or smaller than previous renders, or is it even slightly larger?

- Oshyan
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 12, 2011, 11:54:36 PM
Well, I was running on 1 GB of RAM and I had no problems in those days (two weeks ago), and my system limit is only 4 :(.  I guess I'll have to read up on the 3GB switch... 

The render size is exactly the same as when I had rendered it before (as in technology preview before), it renders great at smaller sizes as well.  Also, of note, is I had the 'Override automatic number of cores detection' in the preferences checked and had been setting it down to 1, then for this render switched it back to 2 but left the box checked, possibly confusing the memory even more, if it is in fact a memory issue.

As with yesterday, I'll try what I've got, results tomorrow.
Title: Re: New Errors on large file sizes.
Post by: Oshyan on February 13, 2011, 12:05:14 AM
Since you were rendering previously with the Tech Preview, it's quite possible some changes to the renderer since then have resulted in somewhat higher memory requirements. There have been a *lot* of changes since that original public version...

- Oshyan
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 13, 2011, 12:10:37 AM
Okay, what are the official system requirements now?  I've been rendering more or less with all the updates, and I've been good with the last version up to at least 1500 or 1800 across.
Title: Re: New Errors on large file sizes.
Post by: Oshyan on February 13, 2011, 12:19:12 AM
The minimum requirements have not changed as far as I'm aware, and we can't specify any particular requirements for rendering at a given size or scene complexity because there are too many variables. It's probably best just to focus on fixing the problem rendering with the latest version. The /3GB switch might be a good place to start, but you should also verify that it renders ok at lower resolution (if you have not already done so).

- Oshyan
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 13, 2011, 12:25:58 AM
Ok, render is trotting along using between 150000 and 250000 K of memory.  Render subdiv cache is 200, max bucket size is 128, render settings the same and I unclicked the automatic override number of cores.  Also, I did not mess with the process priority since the restart.  Let's see where this takes us.
Title: Re: New Errors on large file sizes.
Post by: jo on February 13, 2011, 03:38:10 AM
Hi,

Just to be sure, are you using version 2.2.22.1?

Regards,

Jo
Title: Re: New Errors on large file sizes.
Post by: Kadri on February 13, 2011, 06:23:06 AM
Quote from: jo on February 13, 2011, 03:38:10 AM
Hi,
Just to be sure, are you using version 2.2.22.1?
Regards,
Jo

Jo , just curious ,mine says 2.2.23.1 (Free v.) at startup ?
Title: Re: New Errors on large file sizes.
Post by: jo on February 13, 2011, 06:45:05 PM
Hi Kadri,
Quote from: Kadri on February 13, 2011, 06:23:06 AM
Jo , just curious ,mine says 2.2.23.1 (Free v.) at startup ?
You're right. I was looking at the change log to see what the version of the last public release was and it doesn't seem to have the right version number. v2.2.23.1 is the one to have.

Regards,

Jo
Title: Re: New Errors on large file sizes.
Post by: Kadri on February 13, 2011, 06:47:28 PM

Thanks , Jo.
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 14, 2011, 12:10:54 AM
Yes, I am using 2.2.23.1.  And we have have some new results.  Looks very similar to the old results.  Only this time, with more errors.  It was completely error free when I had started it.  When I got up this morning it had a few errors, but it was continuing to render the scene normally.  Getting back from work today, and I get a 'finished render' that looks much like this.  Frustrating to be sure.  Seems the errors build up as the render progresses.  Maybe memory, maybe I need a new machine.  I'll read up on the 3gb switch, and try that.  I can buy another gig of Ram if I have to (if anyone thinks that'll make a difference), I'd really rather not go out and buy a whole new system quite yet.
Title: Re: New Errors on large file sizes.
Post by: choronr on February 14, 2011, 12:13:57 AM
Perhaps posting your .tgd file for the Planetside staff to see and try would jelp you.
Title: Re: New Errors on large file sizes.
Post by: Oshyan on February 14, 2011, 12:19:51 AM
I would suspect it is memory-related. If you can setup system resource monitoring to record the memory use while it renders that would help. The /3GB switch would most likely help if memory is indeed the culprit. You can also try reducing Subdivision Cache size to the minimum 50MB/thread.

- Oshyan
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 14, 2011, 12:28:06 AM
Quote from: choronr on February 14, 2011, 12:13:57 AM
Perhaps posting your .tgd file for the Planetside staff to see and try would jelp you.

If they really want to pull the file apart because they think it is program related (instead of system related), I'd be happy to email them the .tdg.  Due to the nature of my work, I can't post it here for public consumption.
Title: Re: New Errors on large file sizes.
Post by: Oshyan on February 14, 2011, 12:29:14 AM
For now I don't think we need the file. I don't even have a machine that would likely reproduce the specific problem. But do run some additional tests and let us know how that goes.

- Oshyan
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 14, 2011, 12:51:07 AM
Hey,

I don't have Win XP Pro, just the Media Center Edition (Service Pack 3).  Does this mean I can't use the /3gb switch?  I've just been doing some reading up on it, and it sounds like it isn't compatable with all machines.  Has anyone else used it with the same windows version?
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 14, 2011, 01:11:56 AM
Okay, installed the switch.  System seems to be working.  Another test tonight.  And tomorrow...who knows?
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 14, 2011, 08:46:55 PM
Okay, finally some good news.  The render is now past the point where it had failed before.  I at least have the top of that mountain on the left there and still going strong.  Also, I have left the bucket sizes and other settings at their defaults too (instead of lowering them), so this definately seems to be a memory issue.  Filling in that last slot of RAM'll probably do me.  I didn't know how to set up to record the way memory was being used, but I have kept an eye on it.  At the start of the render I was at 800,000 to 900,000.  When the render was approaching the point where it had failed before, it dropped down to 300,000.  At this point, I thought there might be a memory leak in the program, but now, as we can see, it has jumped back up to 600,000 or so.
Title: Re: New Errors on large file sizes.
Post by: dandelO on February 14, 2011, 09:23:11 PM
Good news. :)

Quick ?uestion. The Eilean Donan picture on your main page to your site, is this a painting or a photo with some filters on? It's a beautiful place, and not too very far from me here, there's a place nearby to it that dates back to biblical times, with an old fort(who's name escapes me right now) that I have a small piece of the structure in my home(I probably shouldn't have taken it home but... ;)) I love the landscapes in this little part of the globe and I'm happy to see it here too, beautiful!
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 14, 2011, 09:49:24 PM
Damn buddy, sharp eye!  You might be the only one in the world to catch that, considering you can only see a slim part of it in the screenshot.  Actually I took that picture when I went to the UK last year in March.  I had walked there from Kyle of Lochalsh because the buses weren't running that early in the season and I couldn't afford to rent a car.  I stayed the night, but I shot the hell out of that castle for the better part of 24 hours or so (minus sleeping at the Loch Duich Hotel).  Most of the images I made of it are in HDR (just between you and me).  However, this particular image is just straight jpeg, but I had breathed ever so slightly on the lens to give it a more pronounced fog effect (although that morning it was already somewhat foggy).

And yes, I love the landscape in britain, not to mention the culture, castles, and girls with cute accents.  I'm toying with the idea of moving there, but it feels like it would be such an uphill battle.

You can see the whole image here:  http://www.imagekind.com/Fog-on-Loch-Duich_art?IMID=89302ea0-9659-4ddd-827e-548ddb068c13

I also have a gallery here with more images of Eilean Donan and everything else I've shot, if you're interested:   http://www.zazzle.com/uponinfinity
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 14, 2011, 09:56:35 PM
Oh, I thought you were talking about my windows background.  I hadn't realized you went to my website.  Yes, that picture is in HDR, so it has been heavily processed.  You must have just typed in my nick into a web address.  I hadn't realized I had any links back to my site on this forum.  lol!
Title: Re: New Errors on large file sizes.
Post by: dandelO on February 15, 2011, 09:06:38 AM
Hee hee! No, I didn't recognize the screenshot from your desktop, that would be a very sharp eye, indeed!

I actually thought I'd recognized the name in your task manager(CK) from somewhere and couldn't place where so I popped that into Uncle Google alongside Upon Infinity, only to realize that there's a singer in a pop-rock band with a very similar name, must be where I got mixed up. ;)

Anyhow, great shots of the homeland and moving here is a fine idea, in my opinion, if a little cold! :D

Martin.
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on February 15, 2011, 12:21:53 PM
Okay, definately memory related, for future reference, if anyone has a similar issue pop up in the future.  Render eventually completed sometime last night, using the /3gb switch.  Thanks for the help, everyone!
Title: Re: New Errors on large file sizes.
Post by: MoraFermi on February 15, 2011, 12:53:16 PM
This problem is definitely memory related. I have a scene with a few heavy (40+MB geometry, 40+MB textures) .obj props that rendered fine in previous version of Terragen Free (was that 2.1? I don't remember) but fails in the latest build.
Title: Re: New Errors on large file sizes.
Post by: Upon Infinity on March 02, 2011, 01:47:52 AM
I think I discovered what the 'real' problem was with this render.  It is memory, but it's because of the landscape I used - which is essentially three procedurals blended together.  Been doing a lot of rendering the last couple of weeks with different .tgd's and different size renders.  I'm doing one render right now with only 1 procedural terrain and the computer is just tearing through it.