hey guys ;D
my IT teacher at school has let me use all the computers in my lab for rendering terragen
there are 33 computers so in total 33 intel dual core processors with 1gb of ram each.
now in vue i can use the render bull module so i can use all the processing power of the computers and it realy speeds up the render times.
is there a way i can do this in terragen 2?? is the some sort of render farm utility?
crop renders ? animations ?
I remember Oshyan or someone mentioning a render farm utility. Anyone remember? Let's search...
[edit - didn't see anything specific to this question; some things about render farms from this same level]
Using a render farm with 33 computers is really inefficient if all you're doing is rendering one frame.
To render one frame on a render farm, you'd have to crop it into 33 parts, and then combine ALL 33 parts in Photoshop. The length of time it would take to stitch together 33 layers of images is so huge that you might as well just use 2 or 3 machines, rather than all 33.
However, if you plan on rendering an animation, 33 computers might all be used, depending on how long the animation is. You could set it up like computer 1 renders frames 1-10, computer 2 renders frames 11-20, computer 3 renders frames 21-30, so on and so forth. This should work great.
Quote from: Blonderator on June 10, 2008, 03:46:05 PM
Using a render farm with 33 computers is really inefficient if all you're doing is rendering one frame.
To render one frame on a render farm, you'd have to crop it into 33 parts, and then combine ALL 33 parts in Photoshop. The length of time it would take to stitch together 33 layers of images is so huge that you might as well just use 2 or 3 machines, rather than all 33.
However, if you plan on rendering an animation, 33 computers might all be used, depending on how long the animation is. You could set it up like computer 1 renders frames 1-10, computer 2 renders frames 11-20, computer 3 renders frames 21-30, so on and so forth. This should work great.
Sorry this seems to be a little inaccurate. It is the job of the job manager (Software) to distribute the tiles (Terragen and other programs call them buckets) to nodes on the farm for rendering once all sent tiles have been rendered they are sent back to the client computer (That is the computer form which the job request was sent) and reassembled into the final image automatically or each pass can be saved as a separate file, and then assembled into the final image along with other elements in compositing; no Photoshop required.
Regards to you.
Cyber-Angel
Yes, but it was my understanding that Terragen currently lacks the ability to automatically assemble the parts into the final image.
Regardless, 33 computers seems overkill for a single image file, IMO. Unless you need a huge sized image, an extreme amount of AA, or you have lots of volumetrics in your scene, I think a smaller number of renderers would still get the image completed in a short enough time.
I think Planetside are considering supporting render farm software in the future or having a job manager built into TG2, at this time I don't recall which it is, in any case I was referring to the standard norm; rather than the current situation with TG2. Sorry for the miscommunication. End Of Line.
Regards to you.
Cyber-Angel
I may be completely off here but the alpha with multicore support must be using a rudimentary Job Manager to assign the render buckets to the available cores. With the addition of a network client-server manager is there any reason that the available cores need be on the same machine?
We don't explicitly support any particular render farm manager but I'm fairly sure people have been able to get Condor and/or other generalized systems to work.
You would of course need the corresponding number of legal licenses to utilize all those machines as well. ;) As it stands you can only buy additional render node licenses with a full license of TG2 Deep/Deep+Animation, but this will change in the future.
As for network rendering of individual frames, introducing that kind of latency into the bucket rendering process probably wouldn't be a good thing. There are proper ways to do it, but I'm not so sure it could just be bolted onto the existing multithreading system.
- Oshyan
Quote from: Oshyan on June 12, 2008, 12:14:25 AM
We don't explicitly support any particular render farm manager but I'm fairly sure people have been able to get Condor and/or other generalized systems to work.
You would of course need the corresponding number of legal licenses to utilize all those machines as well. ;) As it stands you can only buy additional render node licenses with a full license of TG2 Deep/Deep+Animation, but this will change in the future.
As for network rendering of individual frames, introducing that kind of latency into the bucket rendering process probably wouldn't be a good thing. There are proper ways to do it, but I'm not so sure it could just be bolted onto the existing multithreading system.
- Oshyan
I'd suggest you figure out and implement this soon, as it's starting to be taken seriously in industry and could be a shoe that loses the horse in terms of sales.
You can use my scripts here:
http://forums.planetside.co.uk/index.php?topic=2816.0 (http://forums.planetside.co.uk/index.php?topic=2816.0)
to tile a frame up into separate render files suitable for batching on any licensed machine.
The scripts to recombine all the images into one are also made - recombine is done using imagemagick.
There is even overlaps between tiles so there are no edge artifacts in AA calcs (which may be fixed now anyway ?)
All details are on this page.
It also makes panoramas if you add Hugin (nona specifically)
Full source is included.
The wedging version is almost done (pick a parameter and a range and get multiple renders collated into a single image
(example attached)
aw sweet!
thanks Mark fellow Kiwi!!!
You can create a batch file for a frame range and then run that on each computer, have a look at win_command_line.txt in the docs dir for flags.
We have got it working over Muster (a render farm manager) which basically sends out command line renders to each computer available, just needed to create a template that had the right flag options.
Batch example:
The example below will work if you have renderTest.tgd in the planetside dir, and it will save out the frames to where ever is defined in the terragen file (defaults to C:\temp.04%d.bmp)
tgdcli -p renderTest.tgd -hide -exit -r -f 1
tgdcli -p renderTest.tgd -hide -exit -r -f 2
tgdcli -p renderTest.tgd -hide -exit -r -f 3
tgdcli -p renderTest.tgd -hide -exit -r -f 4
tgdcli -p renderTest.tgd -hide -exit -r -f 5
tgdcli -p renderTest.tgd -hide -exit -r -f 6
tgdcli -p renderTest.tgd -hide -exit -r -f 7
tgdcli -p renderTest.tgd -hide -exit -r -f 8
tgdcli -p renderTest.tgd -hide -exit -r -f 9
tgdcli -p renderTest.tgd -hide -exit -r -f 10
tgdcli -p renderTest.tgd -hide -exit -r -f 11
tgdcli -p renderTest.tgd -hide -exit -r -f 12
tgdcli -p renderTest.tgd -hide -exit -r -f 13
tgdcli -p renderTest.tgd -hide -exit -r -f 14
tgdcli -p renderTest.tgd -hide -exit -r -f 15
tgdcli -p renderTest.tgd -hide -exit -r -f 16
tgdcli -p renderTest.tgd -hide -exit -r -f 17
tgdcli -p renderTest.tgd -hide -exit -r -f 18
tgdcli -p renderTest.tgd -hide -exit -r -f 19
tgdcli -p renderTest.tgd -hide -exit -r -f 20
tgdcli -p renderTest.tgd -hide -exit -r -f 21
Yes and don't forget that you can also set the destination directory for the rendered results (both image and matte). This helps to get them all in one place from several systems.
But I make a tgd for each frame and for each tile to be rendered. This was mainly to get around a bug where sequential frames would not render in the previous alphas - but I decided it was a good idea anyway as it made distribution very simple. The end result is a number of tgd/batch file pairs like this: (for an 800x600 image sequence with 2x2 tiles)
set TERRAGEN_PATH=C:/Program Files/Planetside Software/Terragen 2 Technology Preview (Deep)
set RENDIR=C:\testrenders
cd "C:\Documents and Settings\User\My Documents\CTW\Terragen etc\Scenes"
"%TERRAGEN_PATH%\tgdcli" -p "tmpCaveTest-16c_tile1_0_0_400_400_0_800.tgd" -hide -exit -r -f 2
-o "%RENDIR%\CaveTest-16c_tile1_0_0_400_400_0_800.0002.bmp"
-ox "%RENDIR%\CaveTest-16c_tile1_0_0_400_400_0_800.0002.alpha.bmp"
You can see how the o and ox option define the render destination. (This option overrides the internal render directory setting)
If I render them tiled then I use an auto generated ImageMagick script to recombine them like: (this is a 2x2 tiling of a 502sq image)
(Png files remember their virtual screen locations and so are an ideal intermediate format for tiled crops like this)
(convert is an ImageMagick command)
convert "C:\testrenders\CaveTest-16c_pano_0_0_1_251_251_0_251.0001.bmp"
-crop 255x255+0+-2 "C:\testrenders\CaveTest-16c_pano_0_0_1_251_251_0_251.0001_cropped.png"
convert "C:\testrenders\CaveTest-16c_pano_0_1_0_251_251_251_502.0001.bmp"
-crop 255x255+251+249 "C:\testrenders\CaveTest-16c_pano_0_1_0_251_251_251_502.0001_cropped.png"
convert "C:\testrenders\CaveTest-16c_pano_0_1_1_251_251_251_251.0001.bmp"
-crop 255x255+251+-2 "C:\testrenders\CaveTest-16c_pano_0_1_1_251_251_251_251.0001_cropped.png"
convert "C:\testrenders\CaveTest-16c_pano_0_0_0_251_251_0_502.0001.bmp" -page +
"C:\testrenders\CaveTest-16c_pano_0_0_1_251_251_0_251.0001_cropped.png" -page +
"C:\testrenders\CaveTest-16c_pano_0_1_0_251_251_251_502.0001_cropped.png" -page +
"C:\testrenders\CaveTest-16c_pano_0_1_1_251_251_251_251.0001_cropped.png"
-flatten "CaveTest-16c_pano_0.0001.bmp"
del "C:\testrenders\CaveTest-16c_pano_0_0_1_251_251_0_251.0001_cropped.png"
del "C:\testrenders\CaveTest-16c_pano_0_1_0_251_251_251_502.0001_cropped.png"
del "C:\testrenders\CaveTest-16c_pano_0_1_1_251_251_251_251.0001_cropped.png"
Likewise if combining 6 cube rendered views into a Panorama, I use an autogenerated Nona script like this:
# Nona script written by Batch_TG2
# panorama settings. See http://hugin.sourceforge.net/docs/nona/nona.txt
p f2 w1580 h790 v360 n"TIFF c:NONE" R0 T"UINT8"
m g1 i3
# output image lines
o w502 h502 f0 y0 p0 r0 v90 a0 b0 c0 d0 e0 g0 t0 n"C:\testrenders\CaveTest-16c_pano_0.0001.bmp"
o w502 h502 f0 y90 p0 r0 v90 a0 b0 c0 d0 e0 g0 t0 n"C:\testrenders\CaveTest-16c_pano_1.0001.bmp"
o w502 h502 f0 y-180 p0 r0 v90 a0 b0 c0 d0 e0 g0 t0 n"C:\testrenders\CaveTest-16c_pano_2.0001.bmp"
o w502 h502 f0 y-90 p0 r0 v90 a0 b0 c0 d0 e0 g0 t0 n"C:\testrenders\CaveTest-16c_pano_3.0001.bmp"
o w502 h502 f0 y0 p90 r0 v90 a0 b0 c0 d0 e0 g0 t0 n"C:\testrenders\CaveTest-16c_pano_4.0001.bmp"
o w502 h502 f0 y0 p-90 r0 v90 a0 b0 c0 d0 e0 g0 t0 n"C:\testrenders\CaveTest-16c_pano_5.0001.bmp"
which is called from a recombine script like this:
call "C:\testrenders\CaveTest-16c_pano_0.0001_recombine.bat"
call "C:\testrenders\CaveTest-16c_pano_1.0001_recombine.bat"
call "C:\testrenders\CaveTest-16c_pano_2.0001_recombine.bat"
call "C:\testrenders\CaveTest-16c_pano_3.0001_recombine.bat"
call "C:\testrenders\CaveTest-16c_pano_4.0001_recombine.bat"
call "C:\testrenders\CaveTest-16c_pano_5.0001_recombine.bat"
"C:\Documents and Settings\User\My Documents\My Software\Hugin Panotools\hugin-0.7_beta4_windows\nona.exe"
-o "C:\testrenders\CaveTest-16c_panorama.0001.tif" "C:\testrenders\CaveTest-16c.0001.Nona_script.txt"
All in all - unbelievably easy to do from a script and very difficult by hand. (not that it didn't take some time to work out the tiling overlaps script! )
Cheers...