Pixel Plow renders

[digitalguru]

Hi all,

I'm getting ready to submit a fairly big job to Pixel Plow and would be grateful if anyone that's used it has some tips on how to approach it.

It's about 800 frames long, split over 8 shots. Each shot currently has it's own camera and render node.

Since the gathered project is just over 3GB, and my upload speed is slow (1mb!) this is something I'd only like to upload once (or maybe at least twice to include a test).

I've just tried a test in the free version of Deadline (a render job management program) which had a function called "Submit All Render Nodes as Separate Jobs" but that didn't work. It looks like Terragen likes to render the render node checked as "Master" or the first node it finds.

I have a workaround which is to create a combined camera from the individual shots and render that from one render node. The downside to that is that some of the frame ranges overlap, and the transition frame from one shot to another would have a crazy motion blur where the camera suddenly jumps position. That would be fixable at home and I could pick up those few transition frames.

Additionally, each render node has a unique overscan from the project resolution, but that looks like I could set keyframes to change those values.

I think this might (reluctantly!) be the way to go.

Just reading through Pixel Plows terms I've found "Customer does not have adequate Internet bandwidth to upload their job or download their frames within their timeframe" - just wondering if my current ISP will cope  I might have to upgrade my Internet connection.

I'm firing off these questions to Pixel Plow as well, but if any of you guys have some advice or pointers using Pixel Plow, that would be a great help!

.p.s - just thought on how this would affect creating GI caches - whether I should generate these at home or include it in the Pixel Plow render, precalculating them would bump up the disk cost that I have to upload a fair bit though. (I'm not path tracing)

[SILENCER]

8 cameras: 8 different scene files rendering the frame ranges you want, with the appropriate camera slugged into the render node.

Will you have to gather that 8 times? I'd think you could gather the once scene, then point your other 7 to that same data, and save in the same directory. I actually don't recall if Pixelplow allows you to just upload a directory filled with pathed scene files for Terragen, then submits the batch. Similar to the way LightWave handles a content directory, you shouldn't need to re-up instance caches, trees, whatever for each shot.


General scene file preparation

With few exceptions, it is important to use relative pathing for any resources needed for the render that exist outside of your scene file.  These external resources should usually be located in the same directory as your scene file or within a subdirectory of that directory. You may submit a job pointing to the folder created during the Export Gathered Project process directly without modification.

[digitalguru]

Heard back from Pixel Plow and the gathered project gets cached and if a new scene is submitted it checks against the cache and if no assets change will use the first upload.

[Oshyan]

Yep, that was the first thing that came to my mind: they have a farm-side caching mechanism (totally independent of and unrelated to Terragen GI cache of course), so you should not have to resubmit the same files over and over again if all scenes share identical files, file names, etc. But... if you're doing a lot of remote work, man I would definitely recommend upgrading your home Internet connection if at all possible!

- Oshyan

[cyphyr]

Heard back from Pixel Plow and the gathered project gets cached and if a new scene is submitted it checks against the cache and if no assets change will use the first upload.

Ah this may explain why I am able to upload a 3gig project in a matter of seconds!

I would also recommend doing a full resolution final quality render on your home/studio computer and submit the time and system specs to their render calculator.

Also in no particular order ...

Do a crop sequence render of a couple of seconds of an area where there is fine detail, distant grasses or tree populations. Any area that may be prone to noise or "buzzing" where the detail level is higher than the pixel resolution will need higher AA.

Also make sure that you have a high enough atmosphere and cloud quality to avoid noise when its animated (a crop sequence of a couple of seconds will do again).

For your first render try at a very much lower resolution and maybe only every 10 (or even 100) frames. A sanity check that everything is working as expected will not cost a lot and will highlight any problems before you are committed.

[digitalguru]

if you're doing a lot of remote work, man I would definitely recommend upgrading your home Internet connection if at all possible!

I don't anticipate using Pixel Plow that regularly, but these assets will take about 7.5 hours to upload, so it might be time for an upgrade 

Ah this may explain why I am able to upload a 3gig project in a matter of seconds!

Good to hear it works in practice 

I would also recommend doing a full resolution final quality render on your home/studio computer and submit the time and system specs to their render calculator.

Thoroughly tested, I even have a spreadsheet going for the render times. The only thing left to test is GI caching on a short shot.

Do a crop sequence render of a couple of seconds of an area where there is fine detail, distant grasses or tree populations.

Also, just about to test along with the GI caching - what AA values do you end up using on motion blurred populations?

For your first render try at a very much lower resolution and maybe only every 10 (or even 100) frames.

Was going to submit a single 80 frame shot at full res to test, but that might be a good idea. Pixel Plow say the caches are kept if they've been accessed within a few weeks, so it leaves leeway to test.

[cyphyr]

Do a crop sequence render of a couple of seconds of an area where there is fine detail, distant grasses or tree populations.

Also, just about to test along with the GI caching - what AA values do you end up using on motion blurred populations?

This can vary a great deal and can depend on a great many factors.
I have gone as high as 12 on distant snow covered trees (did the snow create a high contrast that in turn caused buzz and noise?)

On the scene I am working on at the moment I am hoping to get away with 8 or 9 but there is a lot of camera movement and we are pointing almost directly at the sun.

A small cropped area should not cost much to render enough to make a useful judgement so try a couple of times.

Also at a certain point you can usefully start using a de-noise plug-in in your comping application.

[digitalguru]

Also at a certain point you can usefully start using a de-noise plug-in in your comping application.

I've just bought Neat Video for Davinci Resolve, but I don't really think that's a solution, sample noise from path tracing and video compression noise are good targets for those kinds of solutions, but not so much for inadequate anti-aliasing of fine detail. I'm mixing and matching population renders from Terragen and Arnold, so it will be a good opportunity to see how each hold up.

I did wonder if Terragen could output some render elements that Arnolds denoiser could use, but it uses variance Aovs and anyhow it probably would have required an extra render pass and more than use up the render time it would have saved  Matt did hint once that he might be thinking of a denoise solution.

[Oshyan]

Personally I'd do some tests with 2D motion blur. I know it's kind of looked down on, but I find it looks totally fine in a pretty large number of common scenarios. Obviously it's not appropriate for certain situations, especially down close to and/or moving through/within vegetation, but for the more typical bigger camera movements it can be largely indistinguishable from 3D, and a *ton* faster. Also don't forget to try/use Robust Adaptive Sampling, should help you hit lower render times with equivalent or higher quality.

- Oshyan

[digitalguru]

Personally I'd do some tests with 2D motion blur.

I did experiment with that and motion vectors, but I'm down close to the terrain and integration with the Maya/Arnold rendered elements is quite tight.

I'm actually quite pleased with how the 3d motion blurred elements are working together from both packages - I thought it would a lot more tricky to deal with.

p.s render times are not that bad actually - average 30 mins a frame, I just want to offload the Terragen renders while I render the Maya elements. In fact, it's my fake stones setup that bumps up my render times - in some cases doubling.

[Oshyan]

Ah ok, glad you have decent render times already then. Note that if you are using Adaptive Sampling already, but *not* using Robust Adaptive, then your results are going to be less accurate (biased toward dark/black samples) than with Robust. Just something to be aware of. Enabling Robust may require an adjustment to Pixel Noise Threshold to maintain render times, but end result should be equivalent or lower noise for same render time.

- Oshyan

[digitalguru]

I'm not noticing a big improvement with Robust sampling, though I will render a small test segment before I send it off to PP. Though I'm on v4.3.23 and I see those improvements might relate to the 4.4 builds?

Enabling Robust may require an adjustment to Pixel Noise Threshold to maintain render times

Does that mean I can increase that a little over thresholds if Robust was turned off?

[Oshyan]

Ah, yes it would primarily apply to 4.4. If you're still on 4.3 you can test Robust, but it may not provide as much benefit.

- Oshyan

[digitalguru]

I've just upgraded my internet connection from upload speed 1mbps to 18 mbps - so should get down from 7.5 hours to about 20 mins 

Thanks for the tips all, very useful

[Oshyan]

Sweet! That's a major upgrade.

- Oshyan