Planetside Software Forums

General => Terragen Animation => Topic started by: Tangled-Universe on November 25, 2012, 10:27:04 AM

Title: Refraction of water on fake stones @ 720p
Post by: Tangled-Universe on November 25, 2012, 10:27:04 AM
I recently animated this old scene you can find here:
http://www.planetside.co.uk/forums/index.php?topic=10476.0

You can find the animation here:
https://vimeo.com/54226393 => download source video for much better quality!

I rendered the animation using the GI cache feature, of course.
Every 8th frame was cached out and for every frame 6 frames were blended during the render process.

Render details:
Detail 0.7
AA4 (default)
GI 1/6/8 + ss prepass
Ray detail region multiplier @ 1 (so that underwater geometry is also being rendered @ 0.7).
16 atmosphere samples

Cheers,
Martin
Title: Re: Refraction of water on fake stones @ 720p
Post by: Oshyan on November 25, 2012, 03:13:18 PM
Nice to see this in a high-quality animation. I think it looks quite realistic. Curious why you apparently did output to FLV for upload to Vimeo, but that's a minor technical issue. ;)

- Oshyan
Title: Re: Refraction of water on fake stones @ 720p
Post by: Tangled-Universe on November 25, 2012, 04:10:59 PM
Quote from: Oshyan on November 25, 2012, 03:13:18 PM
Nice to see this in a high-quality animation. I think it looks quite realistic. Curious why you apparently did output to FLV for upload to Vimeo, but that's a minor technical issue. ;)

- Oshyan

Thanks, glad you like it Oshyan. Besides that you think FLV is strange, do you think the FLV doesn't look good?

Now you mention it, you may be able to help me. If you like?

I have K-lite codec pack installed and this week I bought CS6 master collection via work with After Effects etc.
That's where I edited the sequence in.

The output was with FLV because, somehow, I only have a couple of codec at my hands. Despite the extensive codecs I have installed.

FLV gave the best result. By far. All others like h264 and mpeg4 were totally crap, even at high bitrates (10k).

Perhaps you know a good/solid workflow for this?
Title: Re: Refraction of water on fake stones @ 720p
Post by: Kadri on November 25, 2012, 06:07:09 PM

Just for another opinion ;)
The way i work nowadays with those expensive programs is ,
that i output the video and audio without compression from the main program
and then use Virtualdub for the compression.
I had sometimes problems at the export part in the past and some codecs do not show up etc.
You might see more codecs in Virtualdub probably.
When you have a problem at the encoding part you have the Raw data still ready .
So this feels safer to me...kinda!

Sometimes it does so much depends on the video how this or that will look.
Your video is so much dynamic that with low settings it might look like a soup or blocky.
But i think you have to dig in the codecs settings deeper.
Actually with high bitrates and the codecs you mentioned you should get good results .
Are you sure that the main program can output with that codecs without problem?
This was one of the reasons i used other programs for encoding and only output in RAW...

We made a 30 minute video for the school this week and used Premiere and Virtualdub as i said.
The raw HD file ( 1920 x 1080 , 25 FPS) was nearly 180 GB big.
So you have to consider this too of course.

I like the animation .
Probably it looks real too.
I would try it with 2 different settings too just to see how it might look Martin.
One with less refraction and one with just a bit bigger and slower waves settings .
There is a kinda too much prickling(?) effect .

How long did it take to render Martin?
Title: Re: Refraction of water on fake stones @ 720p
Post by: Oshyan on November 25, 2012, 06:11:57 PM
Well, FLV is really just a container format. The newer (newest?) Flash-specific container format is F4V, and if you're aiming for Flash-oriented output, that's what you'd use. The funny thing though is it's just a container, usually for h.264 video (and AAC audio). FLV can also contain h.264 video, but as an older format it typically used VP6 or a Sorenson codec, both of which can be effective with certain types of content, but are generally inferior to h.264.

I don't know what the specific problems with your h.264 output were, but I grant it can be a bit tricky to tune it. It's possible the FLV defaults were just better set up to encode this content well by default, but if that's the case it's almost certain you could reach equivalent or better quality with appropriate h.264 settings. I suppose the only reason to try to achieve that is if there is some tangible benefit in doing so, and I can't say with certainty that Vimeo (or Youtube for that matter) would show higher quality in the streaming version even if you uploaded e.g. .MP4 with h.264. So perhaps it's not worth the effort. But with Adobe killing off Flash, and h.264 (and, to a lesser extend, .MP4/.M4V) being widely used de-facto standards at this point, it would be ideal to figure out how to best make use of them.

Were you outputting your video straight out of After Effects? It does have some settings you'll probably need to fiddle with to get best results. I don't have a lot of After Effects experience so I can't really give much specific advice unfortunately, but I imagine there would be some in tutorials online.

- Oshyan
Title: Re: Refraction of water on fake stones @ 720p
Post by: Andrew March on November 25, 2012, 06:30:48 PM
Oh, I thought the scene was available.
Title: Re: Refraction of water on fake stones @ 720p
Post by: Bjur on November 25, 2012, 07:21:36 PM
Hiho Martin, many thx here too for your answeres in the other animation topic!  :)

To your codec problem:

First thing i would do is to uninstall/delete the codec pack again.

That codec pack isn´t useful for anything (and also causes sometimes more problems then it helps i red).
Your CS6 incl. AE (maybe Premiere too) should come along with all you need to get clean results and renders in dozens of codecs and formats.
The most popular media players nowadays can handle tons of codecs and formats natively.
On my Windows 7 64 PC i am using the more or less famous VLC media player (freeware) to replay clips, movies and stuff, even self produced and rendered HQ ones.
That player eats all standarts until a file is rly broken/damaged.

But yeah, Flash Video, shouldnt be your 1st choice in all for high quallity clips for tubes like Vimeo.
Indeed, the best codec/format for HQ Vimeo clips should be your mentioned MP4/H.264 with a max. resolution of 1920/1080 pix.,
24,25 or 30 frames and with a max. quallity of 5000 kbps.

Dont know what went wrong with your settings or renders.

But 1st: Kill the normally useless codec pack and after this, let something run like CCleaner from Pirisoft (freeware and i love it).

http://www.ehow.com/how_6330416_remove-k_lite-codec.html (http://www.ehow.com/how_6330416_remove-k_lite-codec.html)
http://www.piriform.com/ccleaner (http://www.piriform.com/ccleaner)

Try again see if the problem appears again, GL!

Alternate you could also try to use Premiere instead of AE (of course IF you have Premiere in your CS).
Premiere is easy to manage if you just want to render a simple animation movie from rendered image files.
But i dont know if Premiere CS6 can handle EXR.´s in a proper way now..

At first you just have to choose your working folders, resolution and framerate you are aiming at.
Then import all of your rendered images as "clip" file. Throw that clipfile into the timeline, take a preview or not, be sure that your working area includes the whole scene,
go to export - media then.
Adobe Media Exporter will pop up and there you can choose any format and codec you want for output. There you can also cut out possible black bars if your scene doesnt fits perfect into a choosen standart resolution.
Dont forget to check all HQ options like "render with max depth" and stuff and use VBR, 2-Pass for moving, fuzzy contents like your water animation (Bitratecoding).

Thats it..

I hope you will find a solution soon as possible.  :)

To your picture:

It´s looking very natural and kinda "fresh, cool, and clear".. After i watched your clip a few times in a row, i became rly thirsty - no joke!  :o


Title: Re: Refraction of water on fake stones @ 720p
Post by: Oshyan on November 25, 2012, 07:28:44 PM
Oh and I should clarify, to the best of my knowledge AE and Premiere do not use the installed Windows codecs for encoding anyway, so the codec pack should have neither a positive nor negative effect on encoding from AE.

- Oshyan
Title: Re: Refraction of water on fake stones @ 720p
Post by: Tangled-Universe on November 27, 2012, 05:39:45 AM
Quote from: Kadri on November 25, 2012, 06:07:09 PM

...

...

...

One with less refraction and one with just a bit bigger and slower waves settings .
There is a kinda too much prickling(?) effect .

How long did it take to render Martin?

Thanks Kadri, I'm glad you like it :)

I can see the same codecs in Virtualdub as in AE/Premiere. Actually, AE and Premiere both use Adobe Media Encoder as video encoder.
So to be precise; I can see the same codecs in Virtualdub as in Adobe Media Encoder.

So I tested about a dozen encoders at their highest bitrates (2 passes, if available)and the FLV with VP6 (I believe it was) was the best by far.

You're right that the scene is extremely dynamic and therefore difficult to compress.
Slow big waves may give improvements for the dynamics and compressability of the scene, but generally slow TG2 waves are ugly.
I'll take your suggestion into cosideration.

I don't know how long it took to render. It was 9000 GHz-hour, so probably around 350 hours on a single i7-2600k as I have.

I'll delete the k-lite codec pack, seems it's no good anyway.
Title: Re: Refraction of water on fake stones @ 720p
Post by: Tangled-Universe on November 27, 2012, 05:45:06 AM
Quote from: Oshyan on November 25, 2012, 06:11:57 PM
...FLV can also contain h.264 video, but as an older format it typically used VP6 or a Sorenson codec, both of which can be effective with certain types of content, but are generally inferior to h.264.

I don't know what the specific problems with your h.264 output were, but I grant it can be a bit tricky to tune it. It's possible the FLV defaults were just better set up to encode this content well by default, but if that's the case it's almost certain you could reach equivalent or better quality with appropriate h.264 settings. I suppose the only reason to try to achieve that is if there is some tangible benefit in doing so, and I can't say with certainty that Vimeo (or Youtube for that matter) would show higher quality in the streaming version even if you uploaded e.g. .MP4 with h.264. So perhaps it's not worth the effort. But with Adobe killing off Flash, and h.264 (and, to a lesser extend, .MP4/.M4V) being widely used de-facto standards at this point, it would be ideal to figure out how to best make use of them.

Were you outputting your video straight out of After Effects? It does have some settings you'll probably need to fiddle with to get best results. I don't have a lot of After Effects experience so I can't really give much specific advice unfortunately, but I imagine there would be some in tutorials online.

- Oshyan

Thanks Oshyan. I didn't output straight out of After Effects.
If you do so, then it opens an instance of Adobe Media Encoder. Same happens if you output with Premiere.

Anyway, on all occassions I didn't use default settings. I knew in advance this is a hard one to encode so I cranked up the bitrates for all of them.
See rest of story in my previous post.

I need to get rid of the codec junk on my pc first and investigate on this further.

Cheers

Quote from: Andrew March on November 25, 2012, 06:30:48 PM
Oh, I thought the scene was available.

Ghehe no no sorry...it cleary says "source video" ;D
Drop me a line if you feel sad now ;)

Thanks Andrew!
Title: Re: Refraction of water on fake stones @ 720p
Post by: Tangled-Universe on November 27, 2012, 05:52:34 AM
Quote from: Bjur on November 25, 2012, 07:21:36 PM
Hiho Martin, many thx here too for your answeres in the other animation topic!  :)

To your codec problem:

First thing i would do is to uninstall/delete the codec pack again.

That codec pack isn´t useful for anything (and also causes sometimes more problems then it helps i red).
Your CS6 incl. AE (maybe Premiere too) should come along with all you need to get clean results and renders in dozens of codecs and formats.
The most popular media players nowadays can handle tons of codecs and formats natively.
On my Windows 7 64 PC i am using the more or less famous VLC media player (freeware) to replay clips, movies and stuff, even self produced and rendered HQ ones.
That player eats all standarts until a file is rly broken/damaged.

But yeah, Flash Video, shouldnt be your 1st choice in all for high quallity clips for tubes like Vimeo.
Indeed, the best codec/format for HQ Vimeo clips should be your mentioned MP4/H.264 with a max. resolution of 1920/1080 pix.,
24,25 or 30 frames and with a max. quallity of 5000 kbps.

Dont know what went wrong with your settings or renders.

But 1st: Kill the normally useless codec pack and after this, let something run like CCleaner from Pirisoft (freeware and i love it).

http://www.ehow.com/how_6330416_remove-k_lite-codec.html (http://www.ehow.com/how_6330416_remove-k_lite-codec.html)
http://www.piriform.com/ccleaner (http://www.piriform.com/ccleaner)

Try again see if the problem appears again, GL!

Alternate you could also try to use Premiere instead of AE (of course IF you have Premiere in your CS).
Premiere is easy to manage if you just want to render a simple animation movie from rendered image files.
But i dont know if Premiere CS6 can handle EXR.´s in a proper way now..

At first you just have to choose your working folders, resolution and framerate you are aiming at.
Then import all of your rendered images as "clip" file. Throw that clipfile into the timeline, take a preview or not, be sure that your working area includes the whole scene,
go to export - media then.
Adobe Media Exporter will pop up and there you can choose any format and codec you want for output. There you can also cut out possible black bars if your scene doesnt fits perfect into a choosen standart resolution.
Dont forget to check all HQ options like "render with max depth" and stuff and use VBR, 2-Pass for moving, fuzzy contents like your water animation (Bitratecoding).

Thats it..

I hope you will find a solution soon as possible.  :)

To your picture:

It´s looking very natural and kinda "fresh, cool, and clear".. After i watched your clip a few times in a row, i became rly thirsty - no joke!  :o

Thanks Bjur and you're welcome, am always glad to help.
Thanks for the details and the links to the uninstaller and ccleaner. I'll give it a go.

I'm surprised to read that a 5MBps bitrate would give satisfactory results.
For H264 I cranked it up to 16MBps and it was still inferior to the FLV with VP6 (if I'm correct).
I'll have a look at it again how I exactly did it, but something definitely isn't correctly configured. Somewhere.

You mention CS6 comes with a myriad of codecs...it seems it doesn't for me. Pity.

Cheers all,

Martin
Title: Re: Refraction of water on fake stones @ 720p
Post by: Kadri on November 27, 2012, 07:11:57 AM

Thanks Martin !
350 hours...Not bad .
Testing for a good scene with water takes a little too long for my taste.
You have patience !

When i searched i found many similar problems Martin.
One said that there was a bug in CS3(or was it cs4 ?) and that you have to open the custom settings place in Adobe Media Encoder
and change the encoding bitrate to the same as you choose before this step for example.
He said that it changed  to a much lower setting by itself despite choosing a high bitrate before.

Others said that the h.264 codec that comes in Premiere isn't good and that it is better to use other h.264 codecs etc.

Not to mention the many settings in After effects or Premiere...

In our last project for example another friend gave me files that were 60 FPS and even interlaced when he should have used only 25 FPS progressive ones.

One easy way is to output a small video with desired settings
from After Effects or Premiere and then look at the finished videos bitrate , field order, size , fps. etc.
If they are not consistent then there is a bug , some codec problem , or something you overlook .
Without the project and output settings to see it is hard to say something useful Martin.
Title: Re: Refraction of water on fake stones @ 720p
Post by: Tangled-Universe on November 27, 2012, 10:37:27 AM
Quote from: Kadri on November 27, 2012, 07:11:57 AM

Thanks Martin !
350 hours...Not bad .
Testing for a good scene with water takes a little too long for my taste.
You have patience !

...


Thanks Kadri, I'll have a look at the output configuration and share it here.

I didn't test this scene, to be honest.
All the settings are "educated guess-timates".
For example, I knew soft shadows with 0 jitter will not give flicker, so I used that.
If you sample GI more then you don't need to sample it densely. So I used GI 1/6/8 instead of GI 2/2/8.
Then if I combine enough GI caches for each frame I should get a nicely averaged result.
The animation check button is also useful as it automatically adjusts settings to be suitable for animation.

I used this same approach with a next animation I'm rendering and this will probably show that it will not hold up for a completely other type of scene. So let's see how solid these settings were.
Title: Re: Refraction of water on fake stones @ 720p
Post by: TheBadger on November 27, 2012, 06:02:53 PM
Good info here.

I'll just comment on the imagery though. Firstly I think its rather nice. My bet is that from at least a few more feet away (camera wise) it would look completely perfect. Up close like this though there are a few things that I am curious about.
It looks to me like there are elements of the water that are not in agreement with one another. What I mean is that the waters movement looks good. And the caustics look good. but the two don't seem to be in visual unity.

Actually I imagine that there is a perfect situation where this set up would be used exactly as is and be visually perfect. But just as a study of water movement and lighting I wonder how you did this.

obviously the waters movement and the caustics movements are separate branches of your node tree. So how do you go about setting them up to move in relation to one another? Moreover, how do the two things work in nature? And how to the two versions compare? This is not simply rhetorical I would really like to know.

The discussion of coding video aside for a moment. I was just thinking of some vague memory I have of seeing water in a cave. There was plenty of light to see by, and I remember the reflection of the caustics on the cave walls. I cant separate my emotions from the imagery in my memory so I am not sure if what I remember is real, or some kind of weird imagination based on a memory, hell, it may not really have been a cave but an indoor swimming pool.
The point is, should there be a greater level of transparency between the waters surface movement level and the caustics? Wouldn't that, if possible, make the overall movement smoother? Would this make the various elements of the water as a whole more sympatico with expectations?

How did you build this Martin? from your imagination, or from a singular video reference or many, or all of the above?

At any rate, I like it, it would be a good element to have. You may want to consider selling it as a pack. I can imagine plugging it in to your canyon pack and having a good fly over. Unless its for your work or portfolio that is. Anyway, there is nothing like this out there in the TG2 resource sites.

Nice work man.
Title: Re: Refraction of water on fake stones @ 720p
Post by: Kadri on November 27, 2012, 06:45:02 PM
Quote from: Tangled-Universe on November 27, 2012, 10:37:27 AM
...
I didn't test this scene, to be honest.
All the settings are "educated guess-timates".
...


I am not much surprised.
Your approach to TG 2 is more analytic and inclusive in general anyway.
You see numbers , i would pull more sliders ;)
Title: Re: Refraction of water on fake stones @ 720p
Post by: Tangled-Universe on November 28, 2012, 04:27:00 AM
It seemed that for the other animations these settings worked pretty well too.
Only a LOT of flicker in the reflections of the water. So I cancelled them, also because they took way too long.

Quote from: TheBadger on November 27, 2012, 06:02:53 PM
Good info here.

I'll just comment on the imagery though. Firstly I think its rather nice. My bet is that from at least a few more feet away (camera wise) it would look completely perfect. Up close like this though there are a few things that I am curious about.
It looks to me like there are elements of the water that are not in agreement with one another. What I mean is that the waters movement looks good. And the caustics look good. but the two don't seem to be in visual unity.

Actually I imagine that there is a perfect situation where this set up would be used exactly as is and be visually perfect. But just as a study of water movement and lighting I wonder how you did this.

obviously the waters movement and the caustics movements are separate branches of your node tree. So how do you go about setting them up to move in relation to one another? Moreover, how do the two things work in nature? And how to the two versions compare? This is not simply rhetorical I would really like to know.


Thanks Michael, I really appreciate your enthusiasm :)

Unfortunately I have to pinch your arm about caustics.
TG2 can't do caustics.

Matt once said, long long time ago, that it theoretically *could* do it, given that you use insane GI settings to "catch" the focussed light.
Being a TG2-nerd ;) I tried that of course, but gave up at GI 16-20 or so. I used a dark sphere above water and had bright sun directly lighting the water near the sphere. No luck.

So perhaps it's some other aspect you mention, but I'm not sure?

Cheers,
Martin
Title: Re: Refraction of water on fake stones @ 720p
Post by: TheBadger on November 30, 2012, 12:35:21 AM
Hi Marin.
I guess I knew that TG2 did not do caustics on its own because I remember seeing a caustics file by dandelO. I thought thats what you did here... Somehow create a similar effect with-in TG2's capabilities.
But I'll take your response to mean that you did not do that either. So I went back and tried to understand what it was I thought I was seeing.

perhaps what I said applies to the water surface clarity and the what lies beneath. That is, Is it possible to make the water at the surface, the waves (somehow) more transparent? So that the overall clarity is finer.
Because the water looks very clean, like a mountain lake. And the waves look right too. But at the speed the camera is moving, and the speed the water is moving, the waves should not obstruct the view into the water so much.

Or maybe it is motion blur, or just the position of the sun?

Like I said its a great animation. And the still was great too. Just some questions about the render as it compares to reality, and impressions of reality.
Title: Re: Refraction of water on fake stones @ 720p
Post by: Bjur on December 07, 2012, 12:19:14 PM
Hey Martin

Was bored a bit and tested some things with sequences in AE (because your problem was wondering me).

I dont used AE since ages for final compressed renderings, just for rendering out lossless footage, most of the time for Premiere projects (where i did all my final renders in the end). My results were indeed sudden ones, H.264 codec wise:

- AE (5.5), pic-sequence, HDTV 720 25, optimal settings, compressed via H.264 with 5 MBit/s, VBR 1-Pass (max. possible value in my version) by AE exporter (via renderlist and settings) = Look was fuzzy and muddy, not good in all.
- AE, same sequence and settings, output as lossless/.avi by AE exporter = Good result, as expected.

- PR (5.5), same sequence and settings, H.264 compressed via up-popping Adobe Media Encoder, 5 MBit/s, VBR-2 Pass, high quality/render depth boxes cheched = Much better result than in AE, hmnn..

Even the uncompressed .avi file from AE, H.264 compressed in Premiere with the Adobe Media Encoder, was looking as expected for 5 MBit/s.
AE cant handle the codec not that good as Premiere does it seems and i dont know why. Thats strange and disturbing - but my personal results now.

And why can do AE handle .exr files natively and Premiere cant (in one Creative Suite!)? But thats another story of many little annoying Adobe CS storys.. -.-

Conclusion: Blame Adobe for their bad H.264 integration - in AE at least!  :)

Greetings, Alex