Planetside Software Forums

Hello,

I am rendering the environment from the project I have been getting help on in this forum, from all of my last 5 threads. It looks really good, and its almost done!.. Thanks to everyone who helped.

But now Im back to the same render issues I was at when I started using TG2, 189hrs 51 min and still rendering :'(

2500x1046
detail:0.9
AA6
GIrd:3
GIsq:3
GIbr:8
Ray trace objects: Yes, Ray trace atmo: Yes, GI surface details: Yes
Do soft shadows: yes

I'm very sure the problem is my "do soft shadows" settings. I have them at: SSD: 1.05059, SSS: 11, SJ: 1
I set them arbitrarily, so I know this is whats causing the long render time.

So, what is the best practices for Do soft shadows? There are dos and don'ts for rendering in general, but I never saw a sticky for soft shadows.
The thing about it is using soft shadows dramatically improves every image (just about), so not using it is not an option

Any thoughts you want to share are welcome. Thanks guys.

Actually I would guess it's your primary Detail at 0.9 that is causing most of the high render time and seems unnecessary. If I read your soft shadow settings correctly, you're only using 11 samples, which is just slightly higher than the default and should not cause *dramatically* longer render times. Without knowing your system specs and the specifics of the scene it's hard to make any other recommendations. The details render settings you've mentioned also don't tell the entire story, if you are using raytraced atmosphere with AA6 and normal or increased atmosphere or cloud samples, you will definitely be wasting a lot of render time.

I suggest posting an image of previous versions of the scene, and your system specs.

- Oshyan

Re: 190h05m and still rendering
Hi Oshyan,

This is an early version of what Im rendering now. Add to it a ton of plants and FrankB's "over cast sky" nodes, and you will have a good idea of what I'm rendering now. I did not change any of Franks's cloud settings as they rendered very well in a early test render.

Hardware Overview:

  Model Name:   Mac Pro
  Model Identifier:   MacPro4,1
  Processor Name:   Quad-Core Intel Xeon
  Processor Speed:   2.26 GHz
  Number Of Processors:   2
  Total Number Of Cores:   8
  L2 Cache (per core):   256 KB
  L3 Cache (per processor):   8 MB
  Memory:   32 GB

^^I set my 0.9 detail based on the sticky. But I did get good result at .7 detail. I would not have guessed that the small jump would have made such an impact...Being that I stayed under a detail of 1. But that was without do soft on. Also I did my tests at a much reduced pix size


*do soft off

"You have thirteen hours in which to solve the labyrinth, before your baby brother becomes one of us... forever."

LOL, :D

I just bought a copy of the DVD to watch with my Son. He's way to young now but I am really looking forward to it.

This image is a tribute to Henson, but the story this project is for has nothing in common.

Quote from: TheBadger on February 28, 2012, 10:28:18 PM
LOL, :D

I just bought a copy of the DVD to watch with my Son. He's way to young now but I am really looking forward to it.

This image is a tribute to Henson, but the story this project is for has nothing in common.

In spite of it being unrelated, i like were it is going thus far. The vines are rather nifty.

Doesn't having GI surface details turned on add a lot to the render time?

Love the image BTW.

Reck is right, GI SD takes 3 times longer to render most of the times.
In the least worse case scenario it takes twice as long.

Also, as Oshyan pointed out, it's likely due to the ray trace atmosphere in combination with high atmosphere samples.
How many atmosphere samples do you use now?
I don't see clouds, but if you still use them somehow, how many samples do they have and which quality setting does that achieve?

Besides that the seemingly small increase from 0.7 render detail to 0.9 makes a huge difference, actually.
Not only will the geometry be subdivided much more (likely beyond the sub-pixel level), also the GI calculations are linked to them, exacerbating the rendertime indirectly.
Also, you probably use a very low gradient patch size in your compute terrain to get those walls look good, so that also may contribute to the longer rendertime, especially if you render at such high detail levels.

Looking at the image and your machine specs this should render under 12 hours easily I'd guess.
However, I wouldn't be surprised this also takes so long because of the huge amount of geometry you've imported with all those ivy's. Damn! :D


I'd restart the render, sorry, then use the following for a crop of the area with the most contrast, so darkest shadow vs direct lighting:

Detail 0.75
AA6 (full)
GI 2/4/6
No GI SD
Atmosphere samples = 32 (NO ray trace atmosphere)
Keep soft shadows as is (default is 0.5 degrees which is the soft shadow radius like in real life, so 1 is a bit exaggerated effect, but can often be desired)

This should render muuuuch faster.
If you're not happy with the detail of the ivy's in the shadow you can increase the GI from 2/4/6 to 3/6/6.

Good luck!

Cheers,
Martin

Crikey moses, I thought the scene I'm working on was complicated enough.  ;D
Would love to see this when it's done. Even that early version is beautiful!

Can't wait to see the final.  What's above looks very fine.

Hi,

You have the same Mac Pro as me, except way more RAM - I only have have 6 GB, quite jealous ;-). It's not a slow machine by any means so I definitely think it's to do with your render settings as the others have suggested.

Regards,

Jo

Great image. Look forward to the final version.

@reck & Tangled-Universe
Hi Guys, can you tell me when you do use and never use GI surface details turned on?

@Tangled-Universe
Hi again T-U,
I used FrankB's overcast sky in my final render. I did not make any changes to the nodes, the settings are as follows

Atmo
samples: 48
jitter:1

High Altitude cumulus
Quality: 1.12
Samples: 31
Low Cumulus
quality: 1.00968
Samples: 57
Cirrus
quality: 1.19738
Samples: 9

Does knowing that change any of the recommendations you gave above?
I do use a low gradient patch size 0.1, thanks for pointing that out, it makes me feel like I did something smart. But if you remember, it was you and Dune who told me how to do that ;)
Number of individual Ivy in final render: 255+, the file holding the ivy objects is 2.71GB. This is why I rendered at 2500pix wide, I got sick of making ivy which took between 20 min and 1hr per ivy to make :'(. So I plan on painting the rest in in photoshop as per Dylan Cole's methods: http://www.dylancolestudio.com/ Mostly just back ground work to do.
The render is at 203hr now. But there is only a little spot at the bottom left and right of the image remaining to render. Its so close to finishing I cant bring my self to stop it :-\ However, I will pay close attention to your settings for the rest of the shots for this world.! :)
ty again martin, for all of your help and time :D

@mhaze, calico, ajcgi, ChrisC
Thank you for the kind words

@Jo
Hi Jo,
I am glad to hear that you use the same system. It makes me feel like I made a good choice since I added the cost of my comp to my school loans, which I will be repaying for the rest of my life. We call that The golden chains ::) You probably already know this, but you can get a ton of memory for a little money. I replaced my 8GB mac memory for 32GB at a cost of only $300.00 US. The modules are basically the exact same thing, you can even get the exact same mac memory for a tiny bit more, but I was being frugal. If anyone reading this wants a link to reputable 3rd party dealers I can post what I have.

Thanks all!
I will post the finished work for your review.

Quote from: TheBadger on February 29, 2012, 11:16:25 AM
@reck & Tangled-Universe
Hi Guys, can you tell me when you do use and never use GI surface details turned on?

@Tangled-Universe
Hi again T-U,
I used FrankB's overcast sky in my final render. I did not make any changes to the nodes, the settings are as follows

Atmo
samples: 48
jitter:1

High Altitude cumulus
Quality: 1.12
Samples: 31
Low Cumulus
quality: 1.00968
Samples: 57
Cirrus
quality: 1.19738
Samples: 9

Does knowing that change any of the recommendations you gave above?
I do use a low gradient patch size 0.1, thanks for pointing that out, it makes me feel like I did something smart. But if you remember, it was you and Dune who told me how to do that ;)
Number of individual Ivy in final render: 255+, the file holding the ivy objects is 2.71GB. This is why I rendered at 2500pix wide, I got sick of making ivy which took between 20 min and 1hr per ivy to make :'(. So I plan on painting the rest in in photoshop as per Dylan Cole's methods:http://www.dylancolestudio.com/ Mostly just back ground work to do.
The render is at 203hr now. But there is only a little spot at the bottom left and right of the image remaining to render. Its so close to finishing I cant bring my self to stop it :-\ However, I will pay close attention to your settings for the rest of the shots for this world.! :)
ty again martin, for all of your help and time :D

I use GI SD only when I feel I need GI in tiny areas, like vegetation and small crevices.
Since that happens very rarely I can easily say I almost never use it.

The patch size is something I remember being suggested to you to make your walls work.
I didn't say it's not good. I only included it in the list of possible reasons why this takes so long.

About the atmo settings:

Atmo
samples: 48
jitter:1

High Altitude cumulus
Quality: 1.12
Samples: 31
Low Cumulus
quality: 1.00968
Samples: 57
Cirrus
quality: 1.19738
Samples: 9

You're using ray traced atmosphere at AA6, which is quite a high setting when it comes to raytracing atmosphere elements, but NOT vegetation.
So if you ever render a landscape without vegetation you also won't need high AA settings and thus should be more comfortable with using ray traced atmosphere then.
However, if you render objects/vegetation and need AA6 or higher then it is really not favorable to use ray trace atmosphere.
I recently made quite an extensive comparison showing this and this especially accounts for clouds.

So to get back to the settings: Disable ray trace atmosphere ;)
If you do so then you can keep the settings as is. Atmo samples @ 48 is high'ish, but definitely not too much.
Since your cloud layers do not need many samples to achieve a quality level of 1 I wouldn't bother too much changing those settings as it will not save that much rendertime.
Unless you use ray trace atmosphere ;)
So if you still feel you must use ray trace atmosphere @ AA6 then use 8 atmo samples and set the quality levels of the atmosphere to something like 0.2-0.3 and that should be enough.

Did I forget something?

Oh yes, the render settings recommendation thread should be re-written, completely! ;)

QuoteThe patch size is something I remember being suggested to you to make your walls work.
I didn't say it's not good. I only included it in the list of possible reasons why this takes so long.

I know. I was thanking you for helping me solve the problems with my walls because you said they looked good. It means a lot to me that you spend so much time helping people, It really is a good thing.
This is the only forum on the internet I post on. I don't find other forums on this subject are as knowledgeable and friendly. Its one of the biggest selling points of this software, and goes a long way to making up for the lack of in-depth tutorials. But I know you guys are working on that too 8)

The walls with the ivy's looks nice TheBadger. Curious about the final render :)

Hi Kadri ;D
lol, I haven't forgot about you and your help, Im really looking forward to you seeing the finished work. I'll be glad if you like it, being that your work helped me to understand how to to it.

Everybody does help each other here  :)
I feel like a beginner in my on going *#&½ still not finished animation.
It will barely useful i hope . Assuming naively that it will be OK and hoping the best is not the best way to do this kind of work .
What i learned ones more is that i should had payed more attention to posts here about animations and posted some questions right at the beginning.
I like this place TheBadger and because of your last post i had to write all these unnecessary things  ;D

Its all about the work, and the people who make it.

Incredible piece, Badger. So you probably saved your walled terrain and imported that in IvyGen? Your walls turn out very tight (nice, I mean). And the wall base is probably a repetitive mask? I wondered if it wouldn't have been easier (a bit late to mention this if I see your render time  ;)) to make an wall object and repeat that as a population. With 2-3 sets of ivy grown on that one wall set, it's easy to give the impression of different growth by mixing them with some blending PF. Just wondering, though.

agree with Ulco...
looking excellent so far!

Quote from: Dune on March 01, 2012, 03:18:52 AM
...So you probably saved your walled terrain and imported that in IvyGen?... And the wall base is probably a repetitive mask? I wondered if it wouldn't have been easier... to make an wall object and repeat that as a population. With 2-3 sets of ivy grown on that one wall set, it's easy to give the impression of different growth by mixing them with some blending PF. Just wondering, though.

Hi Dune,
Yes, you are right. But, I wanted a global environment from which I could make a near un-limited amount of shots from. I also wanted to stay entirely with-in Terragen2 for the terrain for a lot of reasons. I will have to do a lot of work outside of TG2 to finish but at least the main body of the project is in one program. This is just one shot that will be on screen for 5-6 seconds, but its a very important establishing shot.
If you remember, you helped me get started on this project, and helped me trouble shoot more than a few times, so I'm happy that you like it. I don't want you guys to think your waisting your time on me.
As for me, I have been looking at this project for three months and I still like it, that tells me something good.


@Inky, Thank you.

everything has already been said, but I would like to say this is going to be a great scene!

Is the maze procedural?

Quote from: FrankB on March 01, 2012, 01:59:53 PM
everything has already been said, but I would like to say this is going to be a great scene!

Is the maze procedural?

Hi Frank,
I am not where I should be with my vocabulary, where the 3D world is concerned. A shame, that makes it hard for me to ask my questions intelligently and efficiently. Also to answer back :-[

I believe that I can correctly say that the terrain hills are "procedural", but the terrain details ie., the walls are imaged mapped for shape and size, as well as detail. Additional colors were added via power fractals.
I do not know if image mapping can be regarded as procedural?

I hope you will like how your clouds work in the final image, Frank. They added a lot of personality to the image, and did wonders with the light.

Thanks for the clarification. So in your case the maze isn't procedural, which means it isn't based on any mathematic formula. Doesn't matter, I was just curiuos.
Looking forward to the final!

Cheers,
Frank

I'd love to see this if it's finished, it looks quite interesting.

A lot of good advice and feedback has been given here, but I want to weigh in on a few things.

As Martin said, the detail setting makes a big difference for relatively small (numerical) changes. When your detail scale is from 0-1, a change of even .1 (e.g. from 0.5 to 0.6) is large relative to the total range of the setting.

I hadn't noticed GI surface details being enabled in your original settings post or I definitely would have mentioned that. As again Martin said, it is seldom worth the render time hit and should generally be avoided. Basically I see it as a last resort when I am otherwise unable to get the detail I want with other combinations of GI settings. In other words, investigate all other options before using it, and definitely try renders - or at least crops - without it first, using settings that are otherwise "final". Only enable it if you feel something is "missing" in the GI (small-scale shadow detail), and even then you should do several small crops to make sure it makes a difference and a positive one.

Seeing the size of your object, I suspect that actually is also contributing to render time. If I understand correctly, you have a *several GB* OBJ file? That's pretty crazy, hehe. Do-able, but definitely taxing to the system.

The patch size no doubt also has an effect. I'd be curious just how much, and how much increasing it would affect your scene results visually. But with the high render time you're probably not interested in experimenting. ;)

Finally, regarding raytrace atmosphere, I think it's worth saying that Martin is right in that it is not always beneficial, however I must disagree with his blanket statement that when using high(er) AA for vegetation, it automatically makes raytraced atmosphere undesirable. As I am just now finishing rendering of the Garden of Eternity animation, I had the need to do a lot of testing and render optimizations and my tests were unambiguous and detailed: raytrace atmosphere was faster for equivalent or better quality in all areas of this scene. This is using multiple volumetric cloud layers, one in particular that creates low-level mist, and is a source of a lot of noise.

Where I think Martin's approach may be making his own tests skew the other direction is the use of "Max Samples". I did try this with the Garden animation but, while it may be suitable for stills, I don't think it works as well for animation. Ultimately I used AA8 with 1/4 first samples and a 0.02 noise threshold. With this, using raytraced atmosphere was faster *and* less noisy, by far. I suspect if I was using Max Samples the results would be different however.

I also want to mention something I discovered while tuning the Garden scene render settings. *Increasing* the number of samples for a cloud layer actually *decreased* render time (significantly) when using raytraced atmosphere! This seemed counterintuitive to me at first, but I realized it's probably because of the adaptive AA settings. There is a relatively low noise threshold (0.02) and relatively high max samples. The AA sampling algorithm is therefore throwing more samples at the problem, up to its max level, in an attempt to reduce the noise coming from the cloud layer. Attacking the problem directly by increasing sampling of the cloud layer alone focuses the sampling where it matters, on the scene element that is creating the most noise. Thus you get reduced noise *and* reduced render time. In fact, at 32 samples, the quality is 2.8 for that layer! Yet it's faster than 0.5 quality. Something to keep in mind.

All that being said Martin is definitely right that RT atmosphere is not right for all situations. Hopefully we can come up with more concise guidelines on all this as it can be rather confusing and time-consuming to test lots of settings to get it right.

I do hope you'll post the results of this lengthy render!

- Oshyan

Interesting observations, Oshyan, especially the ones concerning AA and cloud quality. Something to keep in mind indeed. And....

QuoteAs I am just now finishing rendering of the Garden of Eternity animation

:D :D :D

Ghehe Oshyan, you keep on trying it he

http://forums.planetside.co.uk/index.php?topic=13679.15

I think for everyone here it's interesting to read this topic.

My tests aren't skewed, they are real life examples of what one will use.
Regardless of whether I use full sampling or default 1/4th, since I stopped use RTA everything renders faster.
The arguments and proof are in the link above ;)

Now I'm going to be an ass, but if you still feel it's not true what I'm stating and what my tests have shown then I think the ways you use it is either in a too advanced way for many many users, but even more you should feel the urge to properly and finally document this all ;)

Oshyan,

Hello,
Thank you for such a in depth post on this matter. I think between you a T-U, you guys may just be able to get my brain to understand this stuff. If you can teach me you can teach anyone.

Ok, so the render in question was quit at 320+ hours. There is just a small area in the bottom left corner that did not finish. I am currently rendering a version with all of T-U's settings + I lowered soft shadows to default settings .
The curent render is almost finished, at this moment it is at 25h22m28s. I expect it will be done when I wake up.

I had intended to not post anything of this project until it was completely finished and as close to perfect as I could ever get it, including creating, animating, compositing my characters. But clearly, there is a great potential for me to really learn something from you guys if I post the WIPs. So I will. A little sad, I was looking forward to "the big reveal" ;D

I also wanted to render the image with RTO off. And now based on your last post TRA on, with the others off. SHould I do all that first? Or do you think you can tell me what the results would be just from the 1st and 2nd renders? Also, I have a lot of questions and statements to pose based on your last post, but I'll wait until I post the renders so it will be easier to answer.

One clarification though. I do not have a single object that comes in at 2 GB, I have 250+ individual IVYs, that =more than 2 GB + several populations of thousands of other plants. I'm not sure if thats what you meant or if it makes a difference, but yes, The file on my hard drive that the ivy are kept in is 2GB+.

Please let me know if you want me to do the other test renders I mentioned, or if I should post the first two only.

Thanks a bunch for yours, and everyones interest in this project, it really helps to motivate!

I really want to see your finished scene too, Badger, it looks great already.
On the rta thing, most of you'll know I'm a real render time miser due to a pretty low end computer so I always look extensively for the fastest render method. I find that rta is always faster than not using it BUT(big but) it really depends on lowest sampling levels of AA. This means making use of the adaptive sampling at render time.

Where TU(I believe) is absolutely right that his experience has proved quicker by not using rta, I think you must be using either default or max samples at render time for AA. This will certainly get into longer render times the higher AA you use because the minimum samples per pixel do begin to increase more quickly/drastically after AA=4, indeed. I like to drop the 'minimum first samples' and keep a reasonably high cloud detail and usually never below detail='1'. That should ensure that your cloud is nice and smooth at most settings and you will be able to use, for example, AA=8@1/64th sampling, to give a minimum 1 sample per pixel(a lot of samples isn't needed because the detail is already there from the atmo/cloud quality) and render time is managed well and fast.

The other way around, that I think Martin likes to use more, is to keep AA sampling either default or maxed out and use a lower cloud quality, because not so many samples are required at higher AA levels. The more minimum samples you throw at it, the longer a rta rendered atmosphere will take.

Popular advice and recommendation says that using higher AA and lowering cloud/atmo quality to balance is the best way to use rta. I think, contrary to that, that I can usually always make an atmosphere/cloud render faster with using rta than without it.

In a sentence, keeping a decent cloud quality and lowering AA minimum samples is faster to render than higher AA samples and lower cloud quality. And usually always of higher output quality than a non rta rendered scene.
That said, it's all scene dependent, sometimes you can't get away with lowering the AA samples.
I definitely prefer to use rta when I can because it is generally much faster, for me, in a typical scene.

I need a small clarification on a definition.

Atmosphere: as it relates to Terragen2.
We are or are not, simply talking about the *sky*?!
We are or are not, talking about all the space reveled by light in a scene, that is not terrain?

A simple but important distinction I think, and I'm really not sure of the answer. Please remember that TG2 is still the only 3D software that I am familiar with, so I need to ask.

Ok, so render two is at (right now) 37h6m and nearly finished, much faster than before.

Again the render is having a hard time finishing at the bottom left corner.

Most of you who are talking on this subject are in or near Europe (I thought). I am in the midwest US. So I will wait to stop the render until tonight, if it has not finished. Since you are all probably sleeping right now anyway ;D.
I can tell you that there is a clearly visible quality difference in the two renders; Long vs. short render, way more than I thought was being suggested.

both images are at 2500pix wide, so it should be easy for everyone to see the differences. I'm sure it will help you guys to pin this matter down ;)

See you soon.

Image 1

320+hours quit before finish
2500pix Wide
RTO-YES
RTA-YES
GI Surface Detail-YES
detail:0.9
AA6
GI 3:3:8
DO Soft-Yes 1:11:1

Image 2

Finished at 45:24:15
2500pix wide
RTO-YES
RTA-NO
GI Surface Detail-NO
DO Soft-Yes - Default
Detail 0.75
AA6
GI 2:4:6

There are, I feel, very noticeable differences in the two renders.
In both Renders there was a long hang in the render finishing, when the render got to the bottom left of the images. It work on a tiny part, for many hours.

These images are WIP(s)

Very nice indeed. I love the first one, such a shame that GI surface details takes up so much time because that's what's making image 1 really sing, the GI in number 2 seems flat in comparison. Great looking scene, though!

Great image and I agree with dandelO about the quality between the two.
So now for the fly over?  ;D

A little post in Photoshop will enhance the second render a lot. I think you can make it almost look like nr 1 with a little effort. Because 320 hours for only 2500px wide image is quite something.

Badger, great to finally see the scene. I think it looks really nice, but yes there is definitely a big difference with the GI Surface Details. This *may* be one of those rare instances where it's actually "needed". The many small ivy leaves are the cause of it, essentially. That being said I'm quite curious to determine if we can at least get closer to the GI SD result without quite so much render time. The high render time even without GI SD leads me to believe there may be further room for optimization, though with the large object(s) size, small compute terrain patch size, and very small leaf size, perhaps I am being optimistic. It may be a moot point anyway as it takes so long to render on your system, I wouldn't expect you to do more renders just for testing, and even if you were willing to share the scene and assets, they are so large it might be impractical to transfer them (depending on your connection speed). What are your thoughts on where to go from here? You said you had some more questions?

Martin (T-U, not dandelO ;) ), I think we're both right here. We're both speaking from our admittedly subjective experiences. dandelO is right too, from the same perspective. It is clear at this point that RTA can be both good and bad, depending on the scene specifics and the other settings used. Now, can we produce absolute, scene-independent guidelines on this? That's a much tougher question, and I'll agree with what you might say next: it probably shouldn't be (or at least it would be nice if we could). If it's that difficult to provide good, consistent rendering setting advice, then ideally either the settings would be made more automatic, or easier to use at least. It's something we do work on all the time, but it's a tough problem to crack. I think you'll find most renderers have lots of obscure settings that you need to understand both in function and in *practice*, across a variety of scenes, in order to fully utilize. Just a glance around at render setting discussions on Vray, Vue, Maya, and many more shows this to be true. So perhaps the most realistic goal, then, is to have a renderer whose base settings have the biggest impact on render time vs. quality in the *majority* of scenes, and the more advanced tweaks are much more seldom necessary. It's a work in progress (including the docs)...

- Oshyan

Thanks Badger.  Great information to think about and excellent renders. 

really great results!... some optimization tests sound encouraging also!
:)

Quote from: TheBadger on March 08, 2012, 03:38:59 PM
Atmosphere: as it relates to Terragen2.
We are or are not, simply talking about the *sky*?!
We are or are not, talking about all the space reveled by light in a scene, that is not terrain?

Sorry, missed this reply, Badger. When I say atmosphere I mean everything from cloud in the sky to the volume of the haze, which envelops the planet(I would dearly love to see an atmosphere node volume control that would accept shader inputs, wouldn't that be cool? But that's another story) For the best definition of atmospherics and all it relates to is to visit the default render and look at the difference between two renders, one with 'atmosphere visible', one without.

what a great render and scene, I totally love it! I've made a crop to fit this on my desktop (and adjusted color etc too a bit). Here it is attached.

Cheers,
Frank

Stunning!  Excellent job :)

I need to read the entire thread, but was this a heightfield with added population objects for the plants?

wow, I'm just stunned!

@ JO
If you are reading this, would you mind sharing what you would have your *advanced* settings at in the render tabs. Perhaps I can get some speed gain there? What would you have all those settings at if this was your project?

@Oshyan
I would be willing to send the project and all related files to you if you think you can make it work better. I do want the quality of the first render with a render time closer or better than render 2. Lets think of a way to transmit the file incase thats what ends up having to be done; my connection speed is fast, but for more than 2gb it going to take a bit no mater what.

As for right now I am resigned to doing crop renders of any changes I make to the shot and using photoshop to put them together.

@elipsis1, Inky, calico - thanks
@DandelO- Thank you for helping

@FrankB-
Hi, thanks for the complement. If you really like the concept here, I will make sure you get a high quality jpg when its finished.

I'm not ready to give up on this project just yet.

TheBadger  i really like the first image :)

But before i posted here i did some tests. I can not believe that the GI Surface Detail-NO  makes the difference (so much).
It looks (after some tests) to me that you probably have some different objects settings.
I think the second image do not have any ivy leaves shadows .
Are you sure you did not accidentally unchecked the objects "Cast shadows" settings or anything like that ?

Hi,

Quote from: TheBadger on March 09, 2012, 04:38:46 PM
@ JO
If you are reading this, would you mind sharing what you would have your *advanced* settings at in the render tabs. Perhaps I can get some speed gain there? What would you have all those settings at if this was your project?

If you mean me, I'm afraid I'm not much help when it comes to the finer details of rendering settings. Oshyan knows more than me about that, and Matt is the leading authority of course.

The only things I can venture an opinion on is the the threads and subdiv cache settings and I have to say I'd leave those at defaults. I would probably drop the bucket size down to 128 x 128 though. That quite often is faster and it would mean the final part of the render which you say takes some time would be divided up into smaller chunks allowing more threads to work on it.

It's a great looking scene though. The amount of vegetation and the fine detail is very impressive.

Regards,

Jo

Quote from: Kadri on March 09, 2012, 04:45:05 PM

TheBadger  i really like the first image :)

But before i posted here i did some tests. I can not believe that the GI Surface Detail-NO  makes the difference (so much).
It looks (after some tests) to me that you probably have some different objects settings.
I think the second image do not have any ivy leaves shadows .
Are you sure you did not accidentally unchecked the objects "Cast shadows" settings or anything like that ?

I don't know. I'll take a look.

Thanks Jo

I will check my bucket size and also wait for the others to share some advice. Truth is I never felt I had a good understanding of when to make changes to the settings there. I figured because you use the same system I could just put in what you use  ;)

Quote from: TheBadger on March 09, 2012, 05:04:13 PM

Quote from: Kadri on March 09, 2012, 04:45:05 PM

TheBadger  i really like the first image :)

But before i posted here i did some tests. I can not believe that the GI Surface Detail-NO  makes the difference (so much).
It looks (after some tests) to me that you probably have some different objects settings.
I think the second image do not have any ivy leaves shadows .
Are you sure you did not accidentally unchecked the objects "Cast shadows" settings or anything like that ?

I don't know. I'll take a look.

Object cast shadow *is checked* quality of all other populations is medium.

Strange! The shadows on the second image does still look problematic to me TheBadger...

Quote from: Kadri on March 09, 2012, 05:27:28 PM

Strange! The shadows on the second image does still look problematic to me TheBadger...

I don't like render 2  at all! I am doing some crop renders now. I lowered the bucket size as jo said. Im waiting for matt and oshyan to show up with some great news that all I have to do is change some setting in my render preferences and everything will work perfect ;)

Breakthrough ?

Following jo's bucket size change I am doing a crop render (about 9% of the image) I think I am seeing a massive jump in render speed! At this rate, the whole image should render (with original high settings) very close to what my 2nd render did.

Question:
Is there something about cropping that makes a render go faster? That is, if a crop renders fast so will a complete image, yes?

Some areas go faster than others, like you noticed yourself in the lower left corner. But it's a promising sign.

I don't know what problem you guys are having with the second render. I think it's great as it is. Shadows look ok to me...

TheBadger can you share the TGD file? No need for the maps, objects etc. I am curious about the settings only.

Quote from: FrankB on March 10, 2012, 04:35:23 AM
I don't know what problem you guys are having with the second render. I think it's great as it is. Shadows look ok to me...

No problem if you like the second image Frank ,but it looks to me that there are no shadows-or it has some kind of strange shadow settings (the ivy leaves) .

Quote from: Kadri on March 10, 2012, 05:48:43 AM

TheBadger can you share the TGD file? No need for the maps, objects etc. I am curious about the settings only.

Sure... After some sleep if its ok?

Quote from: TheBadger on March 10, 2012, 05:58:21 AM

Quote from: Kadri on March 10, 2012, 05:48:43 AM

TheBadger can you share the TGD file? No need for the maps, objects etc. I am curious about the settings only.

Sure... After some sleep if its ok?

:)

Quote from: Kadri on March 09, 2012, 04:45:05 PM

TheBadger  i really like the first image :)

But before i posted here i did some tests. I can not believe that the GI Surface Detail-NO  makes the difference (so much).
It looks (after some tests) to me that you probably have some different objects settings.
I think the second image do not have any ivy leaves shadows .
Are you sure you did not accidentally unchecked the objects "Cast shadows" settings or anything like that ?

The main difference in lighting is definitely coming from the GI SD being unchecked, Kadri. It affects the accuracy of shadows and draws them much more correctly, by the looks of things in my tests it calculates shadow distances and applies a beautiful falloff from the shadow's source, shadow-casting objects and elements that are really close to a receiving surface have a shadow darkest and most dense at the points of closest contact, and falloff to a less dense shadow nearer the edges.
I think you've seen this before but if you go to the bottom of the page you will see a few clear examples of the difference between SD=on/off. Notice the floor where the glowing sphere casts light onto it, the corners of the room, the shadow beneath the white sphere as it contacts the floor. https://sites.google.com/site/d4nd310/tg2gi

In Badger's 2nd image, there is nothing wrong with the plant shadowing that I notice, I can clearly see the ivy shadows, best visible on the right-hand wall but they are consistently flat in density from one shadow edge to the other. It's only that we have the really accurate GI surface detailed version to compare it to that I think we're noticing it isn't as 'correctly' lit.
I've been playing around with a great Sponza Atrium model that was offered free online a good while ago, got some lighting tests done but I think I'd like to try and play with the Ivy-Gen' again after seeing Badger's scene, I forgot how good the IG is and the Sponza model is just begging for it.
Can I fill the Atrium with Ivy, though? That's another matter entirely! :D Maybe have to do a fair bit of poly reduction but, we'll see...

You may be right Martin.
There are some different ivys near the middle, that made me a little suspicious too about the settings.
Anyway  :)

Edit: I do not remember the link you gave. But that does only say something about my memory. I probably have seen it  :D
       Nice test Martin.

Badger, I can give you access to an FTP account where you could upload >2GB of data, if need be. Would that work? Or do you think the connection might get interrupted before the full 2GB would transfer?

Actually, would I would suggest is this: compress *all* assets of the project with 7-zip or a 7-zip-capable compressor (ZipGenius, IzArc, etc.), and if possible set it to *split* the archive into say 100MB parts. Then you can upload each part individually and if the connection does get interrupted, much less upload time would be lost. Hopefully the 7-zip compression would also being down the file size a fair amount. I've had good luck compressing OBJ files in the past, sometimes up to 40% or better compression.

Let me know when you're ready to transfer and I'll get you the upload access info. Or if you prefer something other than FTP we can work that out too.

- Oshyan

Quote from: Oshyan on March 10, 2012, 04:18:45 PM
Badger, I can give you access to an FTP account where you could upload >2GB of data, if need be. Would that work? Or do you think the connection might get interrupted before the full 2GB would transfer?

Actually, would I would suggest is this: compress *all* assets of the project with 7-zip or a 7-zip-capable compressor (ZipGenius, IzArc, etc.), and if possible set it to *split* the archive into say 100MB parts. Then you can upload each part individually and if the connection does get interrupted, much less upload time would be lost. Hopefully the 7-zip compression would also being down the file size a fair amount. I've had good luck compressing OBJ files in the past, sometimes up to 40% or better compression.

Let me know when you're ready to transfer and I'll get you the upload access info. Or if you prefer something other than FTP we can work that out too.

- Oshyan

Hi Oshyan,
I'm familiar with FTP in general terms. So unless you can warn me of any security concerns with my network, I should be fine with using one. It will as I understand it, be much faster to upload than a file sharing site.
I am rendering a crop of the bottom left corner now, it looks like it will take several more hours to finish. I will prepare the zip after that. You should be able to take a look tonight or tomorrow. You can send to my inbox if the nature of the FTP is privet, or let me know the details here, whatever is best in your view.
And thank you again for the personalized support, very cool of you.

@Kadri,
I will post the terrain file with my settings intact as you asked when this render is finished. I thought it would be done when I woke up. Nope :(

@DandelO
That sounds like a great idea to me, Martin. I would love to see any project you make with IvyGen! I did read some resent post in the general conversation section of this forum on poloy reduction. If you can figure that stuff out as it relates to IvyGen and TG2 that would be great. I think my project would give people a lot of ideas for ivygen and TG2, but as everyone now knows, there are some serious concerns with large scale ivy work.  ;)

I'll have to try and navigate the atrium in the actual IG first, never imported it there before, it's a kind of closed object so I'll have a go later at trying to navigate inside it without having to break up the model. Otherwise, I'll probably just take the lower floors in and use the ivy on the whole model. Probably give it a go tonight/tomorrow to begin with.
I used the Ivy Generator before about 2-3 years ago with TG2 to cover a small bridge crossing a stream and it worked just fine, except as you know, the minor niggle of everything being mirrored and some texture map issues with the default .png files, which was easily fixed. I'll post later if I get anywhere. :)

Yeah, FTP should let you upload at your max outgoing speed, and I can download at my max download speed, no limitations. I have PM'd you the FTP access info. Looking forward to checking out this file and seeing if there are any possible revelations in settings to make it render faster. Quite possibly not. ;)

- Oshyan

Hi Oshyan
I think I still have an a free FTP app I had to use a few times back at school, so no problem. I am still waiting on the crop of the bottom left corner to finish. Its at 16 hours now. As soon as its finished I will 7z the files. and send them!

QuoteLooking forward to checking out this file and seeing if there are any possible revelations in settings to make it render faster. Quite possibly not.

I accept this. If we can get it going faster GREAT. If not, I am resolved to do all of the shots from this project the hard way. It will throw me off schedule a bit though.

If you need a free FTP app, I recommend Filezilla (http://filezilla-project.org/download.php?type=client).

- Oshyan

Oshyan,

I sent it. Please let me no if there are any problems with the transfer.
Thank you.

It compressed down beautifully! From 2.6GB down to 375MB. Files received successfully and I'll take a look a them shortly. Got some other things already rendering at the moment though, so not right away. ;)

- Oshyan

Thanks Oshyan
Thats great news for me. I would have been embarrassed if I screwed up the transfer.
I look forward to hearing your Ideas for improvement.

Wow, Badger, how much memory are you working with?

- Oshyan

32GB, but my renders never use more than, Id say 10-15gb. I don't know why. I wish I could set it up to use nearly all my memory, so that everything would go faster. I have also started looking into over clocking my mac.

It's never going to use more memory than it needs to and forcing it to use more isn't generally going to help. Memory is by far not the greatest determinant of speed, it's CPU resources that really affect render speed. More memory just helps the CPU do what it needs to do.

Anyway, I only ask because I've got 16GB and this scene file is causing me some memory issues, much to my surprise! I will see if I'll be able to load it properly at all, but if I do I doubt I'll be able to render it at the resolution you were doing. We'll see...

- Oshyan

Quotethis scene file is causing me some memory issues, much to my surprise!

lol, If I can surprise you guys for any reason, than Im happy even if its because I do something strange or wrong. At least I'm original! ;)

Did you create and load each one of these ivy objects by hand/manually?

- Oshyan

Quote from: Oshyan on March 12, 2012, 01:25:25 AM
Did you create and load each one of these ivy objects by hand/manually?

- Oshyan

Yes.

Wow. That's dedication. ;D

- Oshyan

Hi again Oshyan,
Please tell me what you think of this.

I recently lowered my bucket size from default to the number Jo gave me. When doing a crop of the middle portion of the scene, about 9% of the total image, I saw a speed boost in the render. I have been rendering a crop of the lower left corner, again about 9% of the total image, its at 32hours now. What is it about the lower left corner that it gets so slow there?

I am rendering these crops at my original settings from page 1.

What are you thinking now that you have seen the files?

I have still been running other tests so I haven't been able to load the file on my newer workstation with more RAM. Trying to load on my older machine (also an i7 though) resulted in a hang every time, so I'm waiting until I can try it on the new machine. I think this scene may just require more than 16GB of RAM though! Which would be a shame as I couldn't troubleshoot then.

As for the differences you're seeing, while bucket sizes can definitely help, I don't think you should really see a *dramatic* difference. I do also think that testing with crops is not really going to tell you the whole story.

I don't know why that particular corner takes so long, but I'm curious to find out if I can. Tomorrow I should be able to test loading of the file on my newer system. Hopefully it will work.

- Oshyan

Thanks.

@Oshyan; you could load one set of ivy image maps and rewire the (so many identical) object parts to the one (or two) defaults. That's what I did in the medieval town. It would of course save a lot of memory, TG not having to load the same texture hundreds of times. You're probably smart enough to do that even in a text editor, which I didn't.

Yeah, I may do that, but I'm discovering now that, though it uses a lot of memory, the objects actually load quickly. What it appears to be hanging on are some very large Painted Shader data blobs.

Badger, does this TGD take long to open for you? Maybe it's just slowing down right at the Painted Shader coincidentally, after it's loaded the objects and maxed out RAM, and now going to the page file or something. Not sure. But if it takes you a long time to load too, then it's probably just an efficiency issue with the Painted Shader. I expect there are quite a lot of strokes in the ones in your TGD. This may also be contributing to the render time, though I can't say for sure yet.

I'll continue attempts to load and investigate...

- Oshyan

Well, I was right, removing the Painted Shader data got it to load and it took *way* less memory. I suspect the way you're using the Painted Shader is probably pretty inefficient. There has to be better ways to do what you're doing. Without being able to visualize where the Painted Shader strokes are, I can't be more specific, but maybe you can illustrate that. Or maybe if I remove the objects I can look at the Painted Shader data on its own, but both loading together seems to be too much for my machine. Congratulations, you've killed my beastly (or so I thought) workstation! ;D

On another note, once it did finally load I discovered a bunch of image files I'm missing, so it looks like there's more to send me if you can. These are the ones I identified as missing (there may be more):
maze3.tif
stonewall2Bump.tif
stonewall2.tif
efeubranch.png (seems vital for the ivy objects - I suppose I could go download it with Ivygen, but easier if you send to me)

- Oshyan

Oshyan,

Yes, it takes a very long time to open, relative to my other projects. *But*, it took that long before I added the painted shaders. After they were added, I did not note any increase in the time it took to load.
When I stared to paint I realized that I had to deselect the populations of moss. Otherwise it was near impossible to move the camera to paint. When I deselected the Moss objects (everything other than the Ivy(s)) I was able to move the camera freely.
I was not happy with the control I gained over the moss populations in those few areas that I used the painted shaders, but it was close to what I wanted so I went with it.

Sorry I forgot those files! I am uploading them now. You should have them in less than 7min.

The painted shader strokes are *mostly* right in the center left wall, the only wall that stands by its self from my camera perspective. It has almost no ivy on it.

transfer complete, Please verify.

*I also forgot to tell you that the painted shaders a new. They were not used in the longer render posted on page3*

Confirmed, but since I still can't load with the Painted Shaders intact, I'm not sure how much more investigation I can do. I would really like to know how much the Painted Shaders might be affecting render time, but can't test that myself. Perhaps you can find a way (maybe try much lower resolution and detail tests, with Painted Shaders enabled and disabled). If they don't make a huge difference, then I can remove them and work on other possible optimizations, which I'd very much like to do.

Edit: I see you mentioned the Painted Shaders are new! Ok, interesting. So I'm going to remove them then and work on seeing if I can speed up the base render.

- Oshyan

Yes, please disconnect anything, make any change do what ever seems right. I only want to know if the render time is natural to the project, or if I am going about everything in the wrong way. In the end, if you can teach me how to get my render time down by some %, than that is great! Because as I said I will proceed with this project even with a render time horror show.
Also, I am just wondering, Is it now or will it be possible in the future to have two projects open on the same system. I would love to be working while Im rendering at the same station. Is this even possible in the technology sense?

QuoteEdit: I see you mentioned the Painted Shaders are new! Ok, interesting. So I'm going to remove them then and work on seeing if I can speed up the base render.

Yes, only added just before you provided a way to send the files. Get rid of them and don't think twice. I will just spend time playing with seed values until i find a similar effect.

Cool, I'll keep looking at it. Note however that I am heading to Alaska for a little over a week on Thursday and won't be able to do anything on it during that time. So it may have to wait until I return for any definitive feedback.

- Oshyan

Hi again,

Some more information.

Ok, so as I said I have been doing crop renders with the reduced bucket size. The first render with the new buckets finished in about 6 hours with mu setting from page one. This new crop, which you can see below, is at 106:13:50.

I just don't get it? that little section is whats slowing the whole render down as far as I can tell.

I have also posted a shot of whats going on with my memory (image 2). I have never seen TG2 do that to my system until this project, Its really crazy!

@Kadri, hey man, don't think I forgot to post the file you asked for, I just have to wait until this crop finishes. Because if there is no way to make this go faster, I really need that crop.

Hi,

It does seem odd that part of the image is so slow when from what I've seen it's not that much different to the rest.

What is about the memory use that's striking you as different? Is it the amount that's being used? It doesn't seem to be causing VM paging so that's not going to be slowing things down.

It is down to 5 threads and I presume you've had as many as 16 running? I'm assuming you haven't change the render node thread settings.

Regards,

Jo

Quote from: jo on March 14, 2012, 09:01:54 PM

What is about the memory use that's striking you as different? Is it the amount that's being used? It doesn't seem to be causing VM paging so that's not going to be slowing things down.

It is down to 5 threads and I presume you've had as many as 16 running? I'm assuming you haven't change the render node thread settings.

Regards,

Jo

Yes, it is just that it's using so much memory. It does not bother me, but that I have never seen it use so much before. I was just really surprised! I don't know what VM paging is.

You are correct Jo, I did not change thread settings.

I think the memory use could be cut down a fair amount by linking all the ivy leaf shaders to the same Default Shader, but it would take a lot of work. Hopefully in the future we can make TG automatically recognize true duplicate resources.

- Oshyan

 :'(

This crop by its self has now taken more time than the rest of the image combined. Same settings as page 1 of this thread. I am at a total loss to understand it.

Can you render the painted shader as a greyscale mask, then reproject from the camera the greyscale render onto the scene to use as an alternative to the painted shader mask?

I occasionally find that when I render crops they take longer than a larger crop, especially when there's lots of tiny details in there. I can only presume it's something to do with memory allocations.

Quote from: JimB on March 21, 2012, 07:26:42 AM
Can you render the painted shader as a greyscale mask, then reproject from the camera the greyscale render onto the scene to use as an alternative to the painted shader mask?

Hi JimB. Its very cool of you to try and help me despite our differences.

To answer your question, yes, maybe. But I think that I would just not try it at all. The painted shader is not really even doing much. And what I was trying to do with it, I can just do in post. So at this point... :-\


@ajcgi
Thanks. But I just don't know. Hopefully when Oshyan returns, he will be able to understand this better.



Update
The crop just finished. All of the black space you see in this image below rendered in like 30-40 hours. The crop took 265hrs. Its that bottom left corner. I did crops of the same size in other places of the image, the average time was 6 hours.

The problem as I can see it, is that the renderer gets exponentially slower the closer it gets to finishing. My computer is using a consistent amount of memory but not a consistent amount of threads.

I still think this TG2 renderer is one of the best quality in CG.  

I'd definitely use Jim's advice, TB. Using painted shaders really slows things down, I always look for another option before deciding to use the painted shader, for this very reason.
The more strokes you use, the worse it becomes. Using an image of your painted mask would really speed this up I'd think. Painted shaders are good in theory but there are some really bad issues with render time.

Quote from: dandelO on March 22, 2012, 02:58:46 PM
I'd definitely use Jim's advice, TB. Using painted shaders really slows things down, I always look for another option before deciding to use the painted shader, for this very reason.
The more strokes you use, the worse it becomes. Using an image of your painted mask would really speed this up I'd think. Painted shaders are good in theory but there are some really bad issues with render time.

totally agree. Just make sure the rendered mask image is really high resolution.

Hey guys, thanks. I always listen to your advise, believe me!

Oshyan,

If you have time, when you look at this project file. Would you please try rendering a view from ground level/6ft, from somewhere facing from a different direction then my current camera?! When I render with all objects turned off-no problem. When I render with objects turned on- everything in the image view warps. Walls end up right in front of the camera (they move) really strange stuff. this is even a bigger problem than render time, believe it or not.

Ready to quit on this. One problem after another. Please help.

Badger, can you just send me a TGD file (without objects, of course) that reproduces the issue? Actually, even better, just copy and paste a camera node with the right position into the forums here. You can do this easily by highlighting (clicking on) the camera node, using Copy (Ctrl-C), and then go to the forum post window and Ctrl-V/Paste. You'll see your camera node clip file show up as text and I can easily drop that into my version of the scene to test this.

As for the overall testing, figuring out what's going on, I'm not really sure how much further I can go with it. RAM seems to be a limitation for me at this point. Due to the amount of objects with duplicate textures you have, memory use is really much higher than it needs to be (largely a TG limitation). It would take ages to fix this in the node network though.

Essentially I think your rendering problems are down to a combination of wanting detailed lighting (thus high GI settings), and having lots of objects/memory use. Now that I'm back from vacation I can look into it more if you like, it's possible I will come across some eureka realization, but I kind of doubt it as I already skimmed through the most common problem settings. It's just a beast of a scene for TG to render. Frankly you may have better luck doing the maze in another program and compositing in the sky/background/lighting.

- Oshyan

I've made a version of the file which consolidates all the ivy textures (text editor and regex :-). There are still some wrinkles to sort out but I'll send it to you when I'm done Oshyan. I also took out the painted shaders because I think Badger said they weren't really needed.

Badger, would it be a problem if the bulk of the maze was only made up of the edge maze unit? That way you could just use an image 1/9th the size. You could still stick the centre part in the middle of the scene. I'm not really sure how much memory that would save but it would be a few hundred MB at least. It's a bit of a drop in the bucket compared to the whole scene though!

Regards,

Jo

What is regex, Jo?

Hi Ulco,

Regex is short for "regular expression". Basically it's kind of an advanced type of text searching. My text editor lets you use regular expressions for find and replace. I opened the project in the text editor. Basically what I did was copy and paste the default shaders from one object and then change all the other objects which used those shaders so they all linked to the ones I copied.

I haven't done much with regular expressions before so it took a bit of figuring out.

Regards,

Jo

@Oshyan,

Regarding that other issue- The problem seems to have been resolved by a restart. Thanks for wanting to help, Im just glad it turned out to be nothing.

On the issue of repeating textures... Is there a thread that talks about how I may set this up correctly in the first place? You have said that fixing it now would be a ton of work, implying that I may have been able to do it from the beginning. If so, how?

You asked if I want you to keep trying to figure this project out. No, you have done enough for me. Thank you. I think its clear that I just went about everything in the wrong way. That is, trying to do it all in TG alone. I guess the only thing left I would ask about is the stuff Jo is mentioning in his last 2 posts in this thread, in relation to repeating textures.

@jo
Hi jo
I dont think there would be any problem making changes to the maze, jo. Although I'm not entirely sure of what you mean, I will live with whatever you guys conclude on this. Can you elaborate on your text editing? Is this something I should know? Does it apply to what I asked Oshyan about repeating textures. And how would you answer that same question?
Thank you jo, very much.

OK, so... I finished the renders and crops of the scene. I worked on the image in photoshop, and just have some painting to do. Basically the image is done.
When I first started this project in other threads I was told I should build the maze in a modeling program and import. Actually several people have said so since then. But I did not have the ability to do it. Now I do, and I will take that advice.
In the end, the way I went about this can't work for more than one or two shots. Its just not practical.
Hell of a try though!
I will continue on this after I have built the maze in houdini.

Thank you everyone.

Hi Badger,

Quote from: TheBadger on March 26, 2012, 03:17:18 PM
On the issue of repeating textures... Is there a thread that talks about how I may set this up correctly in the first place? You have said that fixing it now would be a ton of work, implying that I may have been able to do it from the beginning. If so, how?

I don't think there is any way to set this up in the best way easily. Basically you have a bunch of models that all use the same images for textures. When TG2 loads each model it creates a set of default shaders which each load their own copy of the image. For example with 270 ivy models with 3 texture and 2 alpha images each you end up with 1350 images loaded. The ideal situation would be that only 5 images were loaded.

Unfortunately there isn't a way for you to set this up in TG2 so that only the 5 images are used, unless you did a lot of work editing nodes.

Quote
I dont think there would be any problem making changes to the maze, jo. Although I'm not entirely sure of what you mean, I will live with whatever you guys conclude on this.

Your maze is created by displacing the maze image map. The image map is basically 9 separate sections - 3 rows and 3 columns. All of the outer sections look like they're the same. The centre section is different. What I was thinking is that you could build up the maze using two images. One would be just one of the "edge" sections, the top left one for example. You would use this to create a base map that tiles across the whole scene as the maze image does now. The other image would be the centre section. You could place that in the middle of your scene so you still get the difference there.

I wonder if it would be worth using a distance shader to limit the image map to the visible part of the scene. Just far enough so you still get maze along the horizons.

Quote
Can you elaborate on your text editing?

I'm not sure what more I can say than I did already. The project file is XML and can therefore be opened in a text editor and edited. I opened the file, copied the default shaders from the first ivy models and pasted them so they were outside the objects, kind of at the top level of the file. I then used regular expressions to do a find and replace on all the shader params of the Object Part nodes so they pointed to the default shaders I copied. I used regular expressions because the shader lines weren't all the same as they had names with various numbers of "_1" appended (we need to make that better!).

I then used regular expressions to do a find and replace on all the unused default shaders. I just replaced them with blank space to delete them.

Quote
Is this something I should know?

Hard to say :-). Project files are XML and can be edited in a text editor or processed by other tools which can read XML. For example if I were more of a scripting guy I could have made these changes with a Perl or Python script etc. Project files aren't especially complex so if you're used to working with this sort of stuff they're not too difficult to modify. For example I'm comfortable editing them partly because I've done a lot of HTML editing in the past.

I don't think we'd officially support editing the files as in provide any sort of guidance how to do it but we'd answer questions if they came up I guess.

Quote
Does it apply to what I asked Oshyan about repeating textures.

Yes. I edited the file so it reduced the number of images loaded from 1300 odd to 5. I also took out the painted shaders as it seemed from what you said that it was ok to do. The original file took 9.52 GB when loaded, the edited one took "only" 6.43 GB. I will send you a copy of the edited file.

Anyway, it's an impressive scene. Do you mind if we hang on to it for testing purposes? There's a bunch of things I can think of that we could try out with it. Not so much to make it render faster for you anytime soon, but in terms of loading lots of models at once, optimising texture use, memory use during rendering, that sort of thing. Test cases like this are difficult to come up with ourselves, it's great to be able to work with "real life" projects.

Regards,

Jo

jo,

You may keep and use the scene as you see fit. But please let me know first if you intend to publish any images from it, for any reason. I still have plans to develop this project, I even have a script (Rough Draft)
I'm just glad to have finally contributed something here! Yay me! ;D

QuoteI wonder if it would be worth using a distance shader to limit the image map to the visible part of the scene. Just far enough so you still get maze along the horizons.

This is strange, because I thought that anything not in view of the camera had no effect on anything. Your solution sounds like it will work.

QuoteI edited the file so it reduced the number of images loaded from 1300 odd to 5. I also took out the painted shaders as it seemed from what you said that it was ok to do. The original file took 9.52 GB when loaded, the edited one took "only" 6.43 GB. I will send you a copy of the edited file.

That really is a massive reduction, I take it that the file is much easier to deal with now?
How can you transfer the file back to me? The same way I sent it?

Thanks a lot jo, you and Oshyan have really gone out of your way for me on this. Thats very generous.

Hi Badger,

Quote from: TheBadger on March 26, 2012, 08:24:06 PM
You may keep and use the scene as you see fit. But please let me know first if you intend to publish any images from it, for any reason.

We'd use it solely for testing internally, we wouldn't publish any images from it.

Quote

QuoteI wonder if it would be worth using a distance shader to limit the image map to the visible part of the scene. Just far enough so you still get maze along the horizons.

This is strange, because I thought that anything not in view of the camera had no effect on anything. Your solution sounds like it will work.

I really don't know if it will make any difference. I was thinking that stuff in the far distance which isn't very visible might be having an effect on atmosphere calculations or something.

Quote
That really is a massive reduction, I take it that the file is much easier to deal with now?

It still takes a fair old while to load, but that is down to the time needed to load the objects.

Quote
How can you transfer the file back to me? The same way I sent it?

I'll just PM you a link to the file when it's ready to go. I'll only send the project file, you've got everything else already :-). It's only about 725 kb.

Regards,

Jo

Thanks jo!

I was curious about this and read the whole thread. I have a question about the painted shader. Is that the module that stores the height info for the walls?

I have made a maze before in Maya and what I did was paint the maze white on a black background (or black on a white background, it's been YEARS lol) using Photoshop and then I coverted the image to paths where the edges of black and white created curves. I saved this out as an adobe illustrator paths file, imported that into Maya where I had a set of curves on a flat plane. I duplicated the curves and then lofted them to fill in the walls, converted that to polygons. I suspect if it's the walls giving you trouble you could try something similar though I notice your walls have a slight angle to them and I am not sure what kind of trouble that would give you with your existing ivys.

And I'm not even sure if that's your hangup anyway! :D

Looking forward to seeing this resolved. :cheers:

Quote from: King Mango on March 30, 2012, 10:48:57 PM
I was curious about this and read the whole thread. I have a question about the painted shader. Is that the module that stores the height info for the walls?
Looking forward to seeing this resolved. :cheers:

Hi,

No. The height for the walls was set in the image map projection node. The maze is grown out of an image map I placed on the surface of the planet. Everything in the maze is image mapped, except the terrain and plants. So the walls and all other details are image maps.
The resolution of this project was that it would probably have been better to use a .obj for the maze, with the ivy already on the walls. The way I did it as you read, caused a lot of render problems in terms of time. Although, Jo, did manage to get the file size down, and Im very interested to learn what impact his work will have on render time.

Ah ok. Thanks for the clarification. Don't forget this technique in the future. It's quite helpful even for cities if you want to creat a city block just start with a back canvas and paint your building footprints in white then create the curves and get them into your modeling program of choice and you have an instant city rough-out.

Here's a functional maze-type noise I made a while ago in this thread; http://forums.planetside.co.uk/index.php?topic=10826.msg112850#msg112850

It is pretty basic and unscientific, you can't really specify a complete path through it from a 'point A' to 'point B' but, it will cover the entire ground if you need it to, it can also be masked and you can easily rescale it with a transform shader. Saves messing around with fidgety image maps and you could probably export a terrain out of it, if you softened the edges of the borders some, I'd try a colour adjust shader or a smooth step range to try and smooth the edges out a bit to create a terrain. As it is, it's very sharp and rough so wouldn't displace very nicely without some extra work.

Quote from: dandelO on April 02, 2012, 10:03:24 AM
Here's a functional maze-type noise I made a while ago in this thread; http://forums.planetside.co.uk/index.php?topic=10826.msg112850#msg112850

It is pretty basic and unscientific, you can't really specify a complete path through it from a 'point A' to 'point B' but, it will cover the entire ground if you need it to, it can also be masked and you can easily rescale it with a transform shader. Saves messing around with fidgety image maps and you could probably export a terrain out of it, if you softened the edges of the borders some, I'd try a colour adjust shader or a smooth step range to try and smooth the edges out a bit to create a terrain. As it is, it's very sharp and rough so wouldn't displace very nicely without some extra work.

Hey Martin,

I would like to get back to you on this in several days. What you posted looks like it can really help me do something I was planing for this project, or at least help understand how else what I want can be done. I am sure I will have questions for you about it. In the meantime I am concentrating on modeling fundamentals.
Here is an image that you may like with respect to what you posted.
http://2.bp.blogspot.com/-r_phhodxBUs/Tcf-CWyinHI/AAAAAAAAAAU/GBSSG4D4-38/s1600/d188c1b2f47_labyrinth1.jpg

Also, I was thinking about this: http://xfrog.com/product/LS14.html

You can displace these ornamental bushes procedurally (I did that in the Vlaardingen winter scene, a few years ago), but you can also put thousands of squares on it with a leaf texture, blended by the same procedural map.

Quote from: Dune on April 04, 2012, 02:08:51 AM
You can displace these ornamental bushes procedurally (I did that in the Vlaardingen winter scene, a few years ago), but you can also put thousands of squares on it with a leaf texture, blended by the same procedural map.

Hi Ulco

I had thought about doing that with a wall object, but the problems of getting the walls flush over an uneven terrain killed that idea. Though with plants that sounds perfect.
But what I was struggling to understand is, how do you get the plants to not have trouble at corners and intersections? Will they overlap? and how do I make sure that the plant is alined in the right direction with the path?

What does, "you can also put thousands of squares on it with a leaf texture, blended by the same procedural map." mean?

I really like your maze renders so far. Quite an epic maze! :D

I also wonder, purely by curiosity:

Perhaps a maze could be generated procedurally using math functions in TG?

I know the Hilbert sequence is procedural, and produce a labyrinthine patchwork of lines. Problem with the Hilbert sequences is that they are a perfect pattern, and not very random.

Or something like this:
http://en.wikipedia.org/wiki/Maze_generation_algorithm

QuoteI had thought about doing that with a wall object, but the problems of getting the walls flush over an uneven terrain killed that idea. Though with plants that sounds perfect.

I mean that the 'walls of plants' (the hedges) are no plants, but an upward displaced area (in my case by image map). But if you populate that same area with thousands (millions) of simple squares, rotated by the angle of the displaced wall (you probably need an extra compute normal) you can add a semi-transparent leaf texture on those squares and give the walls more depth and structure.
The fact that it's a displacement hedge ensures it following the terrain.

I can render at 2800x1368 at 0.9 detail and 9 AAA on my friends TG3, his computer is a beast mind you. But I noticed I can render a image out in less then 12 hours with ray tracing out. With it active, my friend often has to fail my renders because they'd had been rendering a week or more.

So I think, that is your kicker: "Ray trace objects: Yes, Ray trace atmo: Yes, GI surface details: Yes" causing the most slowdown. Even on my demo version with it off sample renders run in minutes compared to hours.

If you turn it off, crank resolution a bit higher, and render, you can add some good effects via Photoshop and downscale and ray tracing wouldn't be visible if it were on or off. Least I've noticed not much a difference with it on our off in highly detailed scenes that have been downscaled. May be wrong though.

Quote from: jo on March 26, 2012, 04:55:06 AM
Hi Ulco,

Regex is short for "regular expression". Basically it's kind of an advanced type of text searching. My text editor lets you use regular expressions for find and replace. I opened the project in the text editor. Basically what I did was copy and paste the default shaders from one object and then change all the other objects which used those shaders so they all linked to the ones I copied.

I haven't done much with regular expressions before so it took a bit of figuring out.

Regards,

jo

That's quite smart Jo...brilliant even. TG could use a feature that would do that internally as more and more projects seem to need/use image maps and that ivy can be a killer if you use it a lot in a scene. Been there, scrapped the project.

Quote from: bobbystahr on December 19, 2014, 11:17:11 AM

Quote from: jo on March 26, 2012, 04:55:06 AM
Hi Ulco,

Regex is short for "regular expression". Basically it's kind of an advanced type of text searching. My text editor lets you use regular expressions for find and replace. I opened the project in the text editor. Basically what I did was copy and paste the default shaders from one object and then change all the other objects which used those shaders so they all linked to the ones I copied.

I haven't done much with regular expressions before so it took a bit of figuring out.

Regards,

jo

That's quite smart Jo...brilliant even. TG could use a feature that would do that internally as more and more projects seem to need/use image maps and that ivy can be a killer if you use it a lot in a scene. Been there, scrapped the project.

Notepad++ has regular expression as a internal tool. May be worth looking into. Also Windows Grep (preferred) as you can do this to multiple Terragen Projects at once! :D

However, that is a brilliant idea, what if TGF not only had it's note network, but a plain text editor of the profile file which allows custom commenting (like in the network, however more easily vieable then the auto font-scaling which hides text. You can then also add the regular expression functions to allow editing the network on a grand scale, easily. Or even just a "Search..." and "Search and replace..." option in the network in a little toolbar below the network.

I'm sure a text editor of the file world be a lot easier to incorporate then hooking in searching and replacing the network tree with regex. I've had to use this for the crater generator which doesn't allow you to input shaders for your rim and such making them very hard to edit when you have 200+