So if I pre-purchase a license, it says it comes with 3 or 5 render node licenses. I assume this means I can run TG2 on 3 or 5 other machines for the purpose of rendering only. Can TG2 be setup to do a network render? If so, how?
Jason
aka
"He With Too Many Otherwise Useless Computers Sitting Around"
At this time you will need to use a 3rd party farm management tool with arbitrary application support for commandline renderers. TG2 includes a commandline rendering client which can be used by these tools.
- Oshyan
Thanks. As long as it works. In many ways I prefer using a 3rd party render manager that I may already be familiar with.
Is the command line tool documented anywhere yet?
Also, how does TG2 allow the work to be divided up? I mean, if "n" number of clients all open the same scene file to do a shared render, then how is the work spread across multiple machines? I know some tools will have each render node work on certain frames. This is fine for animation. But if they are all working on a large, single frame render, how is the work split up?
One last thing...
If anybody wants a good primer on setting up a render farm with spare PC's lying around, here's one I found.
http://www.extremetech.com/article2/0,1697,1847365,00.asp
The meat and potatoes is here....
http://common.ziffdavisinternet.com/util_get_image/10/0,1425,sz=1&i=104241,00.gif
Jason
There is a commandline documentation file in the Technology Preview install directory under /docs/command_line.txt
Generally speaking you would divide up the work by frames. This would be easier to do because you could just use the commandline to directly call a particular frame for rendering and this is most compatible with render managers. However if you took a little bit of time you could setup multiple crop renders manually, saving as separate project files and potentially have your render manager deal with starting them all up on the different machines. I'm not sure if what you work with is quite that flexible, but I think there are some tools that support doing this.
Note that the commandline on the Mac version is not functioning currently but we are working on a fix.
- Oshyan
Is there free software that you could recommend for farm rendering on the PC? unofficially. I have access to a large bank of computers and would like to use them to render out scenes in a shorter amount of time.
Thanks!
-s
Quote from: sashley on January 11, 2007, 05:19:29 PM
Is there free software that you could recommend for farm rendering on the PC? unofficially. I have access to a large bank of computers and would like to use them to render out scenes in a shorter amount of time.
Thanks!
-s
I wrote some free Terragen-specific render farm software - you can get it on my website (http://www.emptosoft.plus.com/). Unfortunately, it only works with the old Terragen at the moment. I am planning to update it so it works with TG2, but I am waiting until I have a good idea of how TG2 works and what people will want my software to be able to do. Thanks to the nice new 'TGD' file format, the new version will be able to do a lot more than the old one. Also, unfortunately, my software splits the work between computers by frames/pictures - I have not yet worked out an efficient way of splitting frames/pictures between computers. That means that it is not good if you only want to render one huge picture.
Thanks I'll check that out and look for your updates in the future. But I would be looking to have multiple computers help calculate one image at a time.
I was investigating other software when I realized our company has multiple copies of Backburner from our Max 9 licenses, anyone know if this would be compatible?
Thanks again.
-S
This link may be helpful (http://gpu.sourceforge.net/). I've never used GPU but from what it says on the introduction page ('...allows users to share CPU-resources...is able to get CPU-cycles from other clients on the network system...') it seems to share CPU cycles rather than assign the rendering of a frame to a computer.
Thanks, I'll give this a shot as well. But I was thinking that your tool could come in handy if its geared for TG2, have each machine take on a seperate frame, especially if I'm only rendering four frames to create a skybox.
Do you have a timelime when you might release your network renderer?
-S
A lot of the features need to be tweaked in some way or another so I think I will be looking at a release date between February and April (depending on how much time I have over the next few months). However, I can just skip straight to where Terragen is executed in the program and change the command line structure. I could then simply add disabled flags to all of the inputs for features no longer used by TG2 (or not yet rewritten for TG2), tweak a few bits here and there, and hopefully have an extreme beta version out tonight. I'll have a go now...
That would be very helpful, even to just a try a extreme beta version might be enough. Thanks! Let me know if you manage to post somehting.
-S
I have thrown together an extreme beta version - click here to get it (http://ccgi.emptosoft.plus.com/main_site/download.php?view.6). I'm not sure if it works yet - I am reluctant to try it because I have not yet had time to update the method it uses to store data which means that it will conflict with the version for the old Terragen. I will hopefully update the data storage method and test it in a while (Little Britain is on TV).
I have fixed all errors I know of with my software, but I seem to be encountering a problem with the command line. Terragen totally ignores my commands. Here is the way my program calls tgdcli.exe:
'"$TGLoc" "$TGDLoc" -hide -exit -r -f $R0'
$TGLoc = Location of tgdcli.exe
$TGDLoc = Location of TGD file
I could understand this problem if some negative feedback was thrown at me, but tgdcli.exe proceeds to bring up a command-prompt box, happily throws a load of 'loading' and 'creating' information at me and then starts the main GUI with a blank project. The old Terragen worked with a very similar system, and this system conforms to the command line instructions supplied with T2TP. What is going on?
I'll give it a shot and see. Thanks!
-Steve
Let me know if it works for you. If it does, it's probably because I'm using the free version, if it does not, hopefully Oshyan will drop by and explain why I can't control Terragen from the command line at all.
Took a quick look at it and I'm pretty excited about it. Spoke to the guys in IT and we'll try to set it up officially on the network on monday (its friday night here and they're all out the door). So we'll definately give it a workout. Thanks so much for putting the time into this, appreciate it!
-S
I will hopefully get some time during the weekend to look at that command line problem and to clean up the interface a bit (there are too many input boxes, output fields and variables, now that Terrain and Script files are not needed). I will also work on reading the total number of frames from the TGD file (for when no stop frame is specified). This will replace the much more complex frame counter for the old Terragen scripts.
Dark Fire,
Awesome! I was considering doing something like this myself, but haven't gotten around to it yet... in part because, as you found, command line rendering isn't working at the moment. Apparently it accidentally got broken, and should be fixed for the next update. So it's not anything you're doing wrong--we just have to wait for Planetside before this will work properly.
Quote from: Dark Fire on January 12, 2007, 05:17:41 PM
Terragen totally ignores my commands. What is going on?
I had to render 25 .tgd files and ran into the same problem.
From http://forums.planetside.co.uk/index.php?topic=56.msg310#msg310 :
"Ah, sorry! It's not working for me either - something has changed recently.
I will try to get this fixed in the next update."
That's a relief. I am now pretty sure that my software works OK except for the problems when it tries to run Terragen (which is basically the whole point). We shall have to wait until the next update arrives, which I hope is soon. Meanwhile, I shall continue cleaning up the program and give it a new version number - currently all of the programs appear to be version 1.3 because that's the version of the software for the old Terragen that I created the software for the new Terragen from. I shall also work on getting it put into the Emptosoft Installer, so it can be updated more easily.
I've been thinking about how to split up a single image between PCs, and I think it would be relatively easy (once the issue with TGD not working from the command line is fixed). Here's how I think it would go:
1) Server app loads the given tgd, queries the lan to see how many client apps are available for work.
2) Server app splits the given width (or height) of the render as given in the tgd, makes a new (in memory) version of the tgd for each client with the crop settings in each one set for it's piece.
3) Server app sends the modified tgd to each client.
4) Client saves the tgd to a location specified in the client setup, renders using tgdcli, then returns the finished image.
5) Server receives each finished image, stiching them together when all pieces have been returned.
This works best if all client PCs are roughly equal in power. If the client network was composed of different powered PCs, the server could break up the image into a set (specified by GUI) number of pieces (more than the number of available clients) and the faster PCs could get extra chunks while the slower PCs are rendering theirs.
Is this even close to how your software works? :)
Rich
That summary is virtually the same as how my software works, but my software currently takes the easy route of splitting frames between computers and letting those that are faster do more frames. This solution means that the TGD file does not need to be changed. However, I do like your idea of modifying the TGD file to split individual frames up into chunks - I hadn't thought of that before because you simply couldn't do it with the old Terragen. However, at the moment such a solution would cause problems if the global illumination is in use (http://forums.planetside.co.uk/index.php?topic=347.msg2269#msg2269). I will have no problem splitting the frames into chunks because that's just a load of maths - it should take me a few weeks to make that feature. However, the automated stitching together of crops is unlike anything I have ever attempted program before...
Bummer about the global illumination issue, but maybe they'll have that fixed in the same release that fixes command line rendering? :)
Rich
Alright, so from what I understand, this issue is basically on hold until the command line is fixed in with a new update. Well, I'll just have to wait then. Thanks and I'll keep my eyes open for any updates.
-S
Quote from: sashley on January 16, 2007, 11:12:41 AM
Alright, so from what I understand, this issue is basically on hold until the command line is fixed in with a new update. Well, I'll just have to wait then. Thanks and I'll keep my eyes open for any updates.
-S
That is correct. Render farms are stuck with TG 0.9 until the command line is fixed in T2TP. It is slightly annoying that T2TP is so amazing, and yet simple things like the command line are broken. Oh well...
Sooo, with the command line being fixed in this new update, is there a tool in the works to set up for network rendering? ;)
-S
I have no idea. I need somebody with a paid-for copy of T2TP test my software. I am particularly annoyed by the fact that I can no longer test my own software (http://forums.planetside.co.uk/index.php?topic=1087.msg10925#msg10925). I have requested a version with a working command line but broken render system for development purposes (see here (http://forums.planetside.co.uk/index.php?topic=1087.msg11105#msg11105)), but Oshyan has not replied yet.
For the moment all I require, before I continue development, is the confirmation that the command line is working - I can do virtually everything else (like the division of frames and more feedback statistics) without using the T2TP command line (the division of frames should just require some fiddling with the project file, and the statistics are calculated from internal measurements). There is only one thing I know of that I simply can't do without a copy of T2TP with a working command line: use the feedback from tgdcli.exe (I don't even know what sort of feedback it gives).
Anyway, here is the list of feature additions and removals, and bug fixes I will be working on first:
1. Clean up and optimise the interface (especially the job generator, which now has far too many interface features that are no longer needed because a single project file is used, as opposed to the world, terrain and script files used by TG 0.9).
2. Potential bug: warn about, and potentially offer to fix, problems due to local (rather than network) paths being used in the project file.
3. Tweak features that I simply ripped out before, so that they work for T2TP (such as the frame verification).
@DarkFire: I'll be more than happy to test your software. I utilise some of the PCs at work overnight to speed things up occasionally and the IT guy is quite happy for me to try it (oh wait.. that's me ;)) My previous "render farms" have been crude but effective using a database to generate a batch file to be started manually on each computer, outputting to a mapped network drive to store everything in the same folder. Each queue was then copied to it's relevant machine and started manually via VNC. If one machine finished before the others I just reallocated the remaining frames and restarted rendering.
As for splitting large renders into multiple frames... well I'm doing that at the moment with my hi-res QTVR test. This was split into 10 degree tiles because this was the largest tile that I could successfully render in a portion of the image with the most objects. Larger tiles crashed due to running out of RAM. I'm considering adapting my database to manage tiled output for large images as well, taking the camera rotation settings and fov and outputting new camera settings to import into the TGD to create a tiled output (including animated masks to reduce populations to just those objects influencing the current tile). Setting up the animation files is more painful than importing the data manually into the TGD and using a database takes all of the pain out of that process.
While I'm doing everything on the fly at the moment, using multiple computers would cause some potential synchronisation problems, so I'd simplify it by just rendering the tiles first and then stitching them together (stitching commands also output from my database using the same data).
I'll also add an animated crop for those renders where RAM usage wasn't an issue as this is a lot simpler for single images.
The CLI produces a lot of output (most probably for debugging given it's a TP). Attached a screengrab of the end of a render for you (above the echo commands which are creating my PTStitcher script)
Thanks for the screenshot. Now that I can see the data contained in the output and its structure I will also start working on capturing the output and extracting useful information from it (using the render times from T2TP and the running times of T2TP from my software, I could probably work out the load on the computer and/or network from the difference between the times - if the time T2TP is running for is much longer than the render time, obviously T2TP has been very slow to open and close for various reasons).
The software that needs testing can be found here (http://ccgi.emptosoft.plus.com/main_site/download.php?view.6). At the moment I only need confirmation that the command line is working so just stick to using one computer (there will be some path problems with the TGD files on a network because the paths are not relative), create a job and render a frame. If T2TP stays hidden but renders and outputs the frame, the software is running it correctly. If T2TP just runs like it does when you normally start it, or it closes immidiately, I will need to try alternative command line formats.
Quote from: Dark Fire on January 12, 2007, 05:17:41 PM
$TGLoc = Location of tgdcli.exe
This should be "%TERRAGEN_PATH%\tgdcli.exe" if we've followed the docs, so you shouldn't need to specify this. It should avoid some potential path issues. I usually install programs that use commandlines in simplified folders off root (e.g. c:\TG2 in this case)
Setting this environment variable throws up some errors for TG0.9 when it opens. Not sure if anything is broken as a result as I've only used it to open .TER files to determine sizing and find locations (3D view still works).
TGDCLI has been happily rendering non-stop for a couple of days now without any problems even though I've been using the computer for other stuff, with other application crashes.
Just reading the docs for now. Seems relatively straightforward. I can use network names OK, but you might also consider the case of files on a mapped network drive. I usually use mapped drives to simplify commandline stuff but it's not critical if this works.
I tried a quick job on one computer. It looks very promising...
The job automatically started but it only rendered a default scene (but the correct frame numbers). This may be due to calling tgdcli via the real path rather than using the environment variable. (this happened to me prior to setting the environment variable too) Given the impact setting the environment variable had on TG0.9 I'm guessing that some of the programming of the CLI requires the variable to locate program files.
Opened a command window and manually typed a command to render frame 100 of the same project using network paths and it's running at the moment. I'll let it finish to see if it correctly saves the file (I had to specify that in the command as the local path in the TGD didn't exist)... This worked
Tried a job using a mapped drive, same result (correct frame, default tgd)
Definitely suggest fixing TG command with environment variable for now rather than user input. You might also want to prompt for an output folder/image prefix to add to the command line to override the internally saved output path.
The command I used that worked on the remote machine (for frame 100) was:
%terragen_path%\tgdcli.exe -p \\host\RenderFarm\Project\project.tgd -r -f 100 -o \\host\RenderFarm\Renders\frame0100.bmp
I'd already considered path issues with the TGD for rendering on other computers and I placed all of my files into a single directory. I'm sure some scripting guru could whip up a script to prepare a TGD for output by parsing all of the filenames and making a copy of all of the files in a single directory, and stripping the path names out of the TGD.
... oh... and TG remained hidden but then that is the default as soon as you specify -r
Quote from: bigben on March 21, 2007, 09:56:11 PM
I'd already considered path issues with the TGD for rendering on other computers and I placed all of my files into a single directory. I'm sure some scripting guru could whip up a script to prepare a TGD for output by parsing all of the filenames and making a copy of all of the files in a single directory, and stripping the path names out of the TGD.
I considered adding the ability to move all the files needed into a single folder to software I wrote for TG 0.9, but I eventually decided to drop the idea. T2TP is forcing me to consider that option again. Stripping out the file names shouldn't be too hard, but it would take quite a while to write because I would have to be absolutely sure that no other data is stripped out.
Quote from: bigben on March 21, 2007, 09:56:11 PM
The job automatically started but it only rendered a default scene (but the correct frame numbers). This may be due to calling tgdcli via the real path rather than using the environment variable. (this happened to me prior to setting the environment variable too) Given the impact setting the environment variable had on TG0.9 I'm guessing that some of the programming of the CLI requires the variable to locate program files.
Opened a command window and manually typed a command to render frame 100 of the same project using network paths and it's running at the moment. I'll let it finish to see if it correctly saves the file (I had to specify that in the command as the local path in the TGD didn't exist)... This worked
Tried a job using a mapped drive, same result (correct frame, default tgd)
Definitely suggest fixing TG command with environment variable for now rather than user input. You might also want to prompt for an output folder/image prefix to add to the command line to override the internally saved output path.
The command I used that worked on the remote machine (for frame 100) was:
%terragen_path%\tgdcli.exe -p \\host\RenderFarm\Project\project.tgd -r -f 100 -o \\host\RenderFarm\Renders\frame0100.bmp
I'll start working on fixing that as soon as I can. The annoying thing is that, as far as I can remember, being able to set environment variables makes running Terragen with anything other than a normal priority more complicated.
The network path experiment could be made more interesting - I found with TG 0.9 that if the network connection was lost at any point during the render, Terragen would save the output file when it had finished rendering, but would not close and continue to use the CPU as much as it could.
Thanks for showing me exactly what works - when I release the next beta I will be a lot more confident that Terragen will be launched correctly than I would have been if you hadn't supplied me with that information.
Quote from: bigben on March 21, 2007, 09:56:11 PM
... oh... and TG remained hidden but then that is the default as soon as you specify -r
That's good. At least something is working.
ConclusionI am going to put your entire post in my notes on the next beta, since it contains so much useful information that I simply couldn't obtain with the free version. I am not yet sure when I am going to release the next beta - it will certainly take a while to fix all of these problems and add the requested features. I think I'm going to try to fix the environment variable problem first, so that I can release a beta that can at least launch T2TP properly. This first beta will take a few weeks (but the priority may be a problem and delay the release). Then I'll slowly implement the new features (possibly releasing a small beta featuring the 'output folder/image prefix' override thing), eventually releasing a near-final beta which should have most of the features suggested here as well as most of the features I removed when changing the program so it would work with T2TP rather than TG 0.9. After all of that, I will look at dividing frames.
Finally, I have a question for Oshyan: is the command line structure due to change significantly between now and the proper release of TG2?
Quote from: Dark Fire on March 22, 2007, 05:09:51 PM
I considered adding the ability to move all the files needed into a single folder to software I wrote for TG 0.9, but I eventually decided to drop the idea. T2TP is forcing me to consider that option again. Stripping out the file names shouldn't be too hard, but it would take quite a while to write because I would have to be absolutely sure that no other data is stripped out.
I've been thinking about what direction my database tools should take (generate frame specific TGDs or generate animation settings files to embed in the TGD) and I'm still undecided. One of the things I was looking at was parsing the TGD file to extract each of the variables with cross checking of output field values (text/numeric, number of values etc...) against an xml specification... from this.... the process of locating external file information could be simplified by only checking those tags that contain filenames and excluding node names which might also contain filenames (e.g. the XFrog tree models). This would reduce the chance of affecting other data.
I should be able to add this functionality to my database if nothing else is available... which would save me some time and disk space.
Using your software is a step further along the approach I have taken with my own crude render farms. I'd generate a .bat file for each computer and then manually start it (via VNC to save walking around the building). The main reason for this was that with long animations I ran progressive renders, rendering every 64 frames, then every 32,16,8,4,2,1 frames. I could still do this using you software with a simple modification of my database to generate a single joblist with a job for each frame. This would be vastly more efficient as it won't matter that there's a mix of PC specifications in the farm (and believe me, they go from race horses to cows ;)) or one or more get turned off along the way.
I'll look forward to the next testing round :) Drop me a line off list if there's anything else you want me to test along the way.
Dark Fire, I don't expect the commandline structure to change significantly, but Matt would know more than I.
- Oshyan
Quote from: Oshyan on March 22, 2007, 10:21:09 PM
Dark Fire, I don't expect the commandline structure to change significantly, but Matt would know more than I.
- Oshyan
I can deal with insignificant changes as long as somebody tells me about them. Thanks Oshyan.
Quote from: bigben on March 22, 2007, 07:36:13 PM
I've been thinking about what direction my database tools should take (generate frame specific TGDs or generate animation settings files to embed in the TGD) and I'm still undecided. One of the things I was looking at was parsing the TGD file to extract each of the variables with cross checking of output field values (text/numeric, number of values etc...) against an xml specification... from this.... the process of locating external file information could be simplified by only checking those tags that contain filenames and excluding node names which might also contain filenames (e.g. the XFrog tree models). This would reduce the chance of affecting other data.
I really need to have a good look at the TGD file format. One problem that I have realised since yesterday is that the format may still change, which may cause problems later.
Quote from: bigben on March 22, 2007, 07:36:13 PM
Using your software is a step further along the approach I have taken with my own crude render farms.
Hehe. My software is actually quite crude too.
Quote from: bigben on March 22, 2007, 07:36:13 PM
I'd generate a .bat file for each computer and then manually start it (via VNC to save walking around the building). The main reason for this was that with long animations I ran progressive renders, rendering every 64 frames, then every 32,16,8,4,2,1 frames. I could still do this using you software with a simple modification of my database to generate a single joblist with a job for each frame. This would be vastly more efficient as it won't matter that there's a mix of PC specifications in the farm (and believe me, they go from race horses to cows ;)) or one or more get turned off along the way.
You may have noticed that the job list file is actually an '.ini' file in disguise. It appears that you are trying to exploit two of the great features my software has - the ability to keep fast computers supplied with work to do, and the ability to work out when a computer has been turned off, crashed or been disconnected from the network.
Quote from: bigben on March 22, 2007, 07:36:13 PM
I'll look forward to the next testing round :) Drop me a line off list if there's anything else you want me to test along the way.
OK. Thanks.
ANNOUNCEMENT:This is an announcement that I've been trying to put off for a long time, but I'm afraid I can't put it off any longer. In order to actually get some work done on my software and also to get work done for my exams (http://en.wikipedia.org/wiki/Advanced_Level) this summer (and beyond), I will have to almost completely stop using for forums. I will keep an eye on general developments and topics like this, which I seem to have become an intrinsic part of, as well as the two topics I started (yes, only two - despite being the third highest poster). This post will put me on 567 posts - 67 posts ahead of old_blaggard (http://forums.planetside.co.uk/index.php?action=profile;u=26) (the closest person behind me), 364 posts behind Oshyan (http://forums.planetside.co.uk/index.php?action=profile;u=3) and 395 posts behind Will (http://forums.planetside.co.uk/index.php?action=profile;u=1152) (who both seem to live on these forums). I will undoubtedly drift down the rankings now and people will probably start to forget me as I will only post very occasionally, but I am glad that I have been a significant early force on these forums. So, in conclusion, goodbye and good luck everyone! (Especially the Planetside team).
While I still have a decent opportunity, I am going to pour a final list of suggestions onto these forums:
1. Terragen needs a proper logo.
2. Once Terragen has a proper logo, merchandise should be sold (I can't afford T2TP, but I will be able to buy merchandise)!
3. The eternal suggestion: make it render faster!
4. Make the icons in TG2 reflect the quality of the images that can be produced with TG2 - at the moment, the icons look very old...
5. More features in the free version. I think TG 0.9 had a good balance - reasonably large images, animation features and a working command line. It seems strange that as screens get bigger and better, the free version of T2TP won't let you render images bigger than 800x600.
I could put down more, but the five suggestions above are probably my top five.
We'll miss you, DarkFire. Hopefully you'll be able to pop back in after exams are done. I've got exams during May myself, but they're still far enough away that I can pretend they aren't there :P. Maybe by the time you get back, everything on your wish list will exist ;).
Quote from: old_blaggard on March 23, 2007, 04:38:32 PM
I've got exams during May myself, but they're still far enough away that I can pretend they aren't there.
I could still pretend my exams are not there, but I am currently trying to beat the rest of my maths class by getting the most maths exams in a row with a score of 100%. I currently have the joint record (four consecutive exams), but I am the only person currently still on a 100% streak. That means that if I get 100% in May, I alone will hold the record :D. I need to work hard to be sure of getting 100% in May, so that's a major reason for me leaving for now...
and there's the small fact that I still have a quarter of my Music Technology Coursework to do before May. :'(
Quote from: old_blaggard on March 23, 2007, 04:38:32 PM
Maybe by the time you get back, everything on your wish list will exist ;).
Suggestions 1 to 4 are possible, but I can't see suggestion 5 happening - TG2 seems to be a much more serious product than TG 0.9...
I use The Condor Project to handle my renderfarm needs.
It works with Windows,Os X,Linux all on the same pool.
I got it working with Terragen 2 OS X commmand line.
Indeed, there is already lots of software out there that can help to manage render farms, but software written specifically for Terragen will always be more useful because it will be able to carry out tasks that are specific to TG2. Software written specifically for Terragen will also be a lot quicker and easier to use. People who already use render farm management software should probably stick to what they're using, but people who are new to render farms would probably find it easier to learn how to use software written specifically for Terragen.
Progress report:
I think I've now got the client to launch tgdcli.exe correctly, with the priority specified by the user. Could people with copies of T2TP with working command line interfaces please test this software (http://ccgi.emptosoft.plus.com/main_site/download.php?view.6).
An extra program called 'Diagnostic Mode Client.exe', which shows lots of information that will be useful if things go wrong, is included.
It should be noted that even the normal 'Client.exe' is nowhere near finished - the annoying command prompt box, for example, will be hidden when I release version 1.0. Details shown in the box will hopefully be fed through onto the Client interface in version 1.0.
6 downloads of the new version already. Could somebody please tell me if T2TP is launching, opening projects and rendering images correctly?
I'm sorry, but I'm on a Mac, and I can't run .exe's. Good to see you back, by the way.
I'm only temporarily back - I should really be finishing coursework now...
I have added another program to the package: 'Client (Method 2).exe'. This new program tries to set the environment variable using a less messy method than 'Client.exe' and 'Diagnostic Mode Client.exe'.
Here is a summary of the feedback I need:
1. Does 'Client.exe' run T2TP properly?
2. If 'Client.exe' does not run T2TP properly, what does the command prompt window display when 'Diagnostic Mode Client.exe' tries to run T2TP?
3. Does 'Client (Method 2).exe' run T2TP properly?
A copy of T2TP with a working command line will be needed for these tests (which is why I am not performing them myself). T2TP should be successfully run by each program, but what I am looking for is whether or not T2TP opens the correct project and renders the desired frame.
Please leave feedback in a reply to this topic, or on my website (http://ccgi.emptosoft.plus.com/main_site/e107_plugins/forum/forum_viewtopic.php?534) (on which guests can post).
well, i dont completely understand, but i tried running it per instructions, and it made a bunch of bat files that seem to call up the render client without actually rendering frames.. while the monitor or the client was reporting actually rendering the frames.
note: getting error codes 3O and 4R
i dont have a lot of time to troubleshoot this, but if it turns out to be a quick solution, id be happy to spend a little time with feedback to save me the trouble of chasing individual frames.. my farm has different speeds of processors, and my sequence has frames that will take varying times to render.. so this could really make things easier for me.
it pm you on the forums with my email
Ooops... didn't have this topic on notify....
I've downloaded it but am on my wy home. I'll test it tomorrow morning.
Tomatomic: I'll post some stuff from my animation rendering later tonight. If you're in a real hurry there are a few ways to split things up and keep a track of them to fill in the gaps later.
nice. im all ears. im not ready to render final yet. im still only doing tests. matching my camera to the artists' scene, and testing for render times, which can be extreme. i expect to start rendering final sometime this week, earlier rather than later hopefully.
thanks :-)
I've tested an earlier version of this software over the network, and everything seemed to work as expected apart from launching TG correctly. This has been fixed in the current beta. I had a quick play on the train on the way home and got it to correctly launch TG (using Client (Method 2).exe). I'll try in the morning on a networked PC.
I've previously tested TGDCLI using UNC network names and mapped network drives successfully. For convenience/simplicity I'd probably use a mapped network drive for the joblist location and output directory.
I've added a frame "manager" to my database with some basic scripting. There's still a bit to do before I release an alpha (some extra scripts and custom menus). As mentioned in a previous post I was looking at some interim integration with this software. I previously used .bat files which were distributed to separate computers, with a log file created to keep a track of which frames were rendered. This worked OK, and if you're desperate you could do this, but this software is much more efficient even in it's beta form.
What I have done is set up a frame sorting to provide a progressive render, allowing you to check the whole animation as it renders. The progressive frame order is set to every 128 frames, then 64,32,16 etc... I've hopefully got the export of the joblist correct, even if it's a very crude one assuming you're only queuing up one joblist at a time.
Quote from: tomatomic on April 30, 2007, 02:10:14 AM
note: getting error codes 3O and 4R
The error codes produced are explained here (http://ccgi.emptosoft.plus.com/main_site/e107_plugins/forum/forum_viewtopic.php?1.0) (the list was made for the old version of this new software, which worked with Terragen 0.9 - as a consequence it might be a bit confusing). I'll have a look at the source code at some point in the next week (I have an exam on Friday, so my available time is unpredictable at the moment), to see if I can pinpoint the problem.
Quote from: bigben on April 30, 2007, 09:01:58 AM
I've tested an earlier version of this software over the network, and everything seemed to work as expected apart from launching TG correctly. This has been fixed in the current beta. I had a quick play on the train on the way home and got it to correctly launch TG (using Client (Method 2).exe). I'll try in the morning on a networked PC.
I am pleased that method 2 works - the original method generated ugly batch files, but was virtually guaranteed to work.
Keep on testing this week, and add feedback here. I will be busy because I have an exam to do, coursework to finish and a magazine (http://en.wikipedia.org/wiki/St_Ignatius_College#The_Ignatian) to write. I should have much more time from next week onwards to solve problems.
bigben,
i dont know what you mean by UNC network names. i do have a network directory with every windows client mapping that directory to the same letter.
i do see that dark fire's app is working, to some extent at least. all the clients got their jobs, and rendered frames.
however, i found no actual saved frame results anywhere, to the output location or in the clients directories.
i hacked the joblist.emp file -
it was
[OutputLoc]
1=
changed it to
[OutputLoc]
1=M:\render\farm\frames\
still no go :-/
off to bed
I am glad to see that you are using your initiative and fiddling with the settings. Unfortunately, the OutputLoc setting was originally used for frame verification (which is temporarily disabled, until I write the code for extracting the output directory from the project file), but I will be adding a custom output directory setting to the ERTfT2TP soon*. For the moment, you'll have to change the default output directory in the project file.
* = Could be tonight, could be in a week. I have no idea when time will randomly appear.
Quote from: tomatomic on May 01, 2007, 03:39:38 AM
bigben,
i dont know what you mean by UNC network names. i do have a network directory with every windows client mapping that directory to the same letter.
i do see that dark fire's app is working, to some extent at least. all the clients got their jobs, and rendered frames.
however, i found no actual saved frame results anywhere, to the output location or in the clients directories.
i hacked the joblist.emp file -
it was
[OutputLoc]
1=
changed it to
[OutputLoc]
1=M:\render\farm\frames\
You can use the hostname rather then the mapped drive in a .bat, but mapped drives are easier. Make sure the output path you specified above is included in your TGD in the sequence output tab. If you don't want the extra files, uncheck that as well. Your previously rendered frames will be on the client machines in the location specified in the sequence output so you can always recover these rather than re-rendering.
My testing got interrupted by a broken arm (not mine) but I'll be back at work tomorrow, but it sounds like you're close to up and running.
I forgot to announce that last night I released version 0.2 of the ERTfT2TP. The new version now lets you set a custom image file name (-o in the documentation). The data you enter will end up going after -o when launching Terragen, so you can enter 'C-style format strings' for the file name. If you want to be clever, you can enter this:
Output File Name" -ox "Extra Output Image FileName
(Notice how the start and end instances of " are missed out)
When T2TP is launched, that will come out as:
-o "Output File Name" -ox "Extra Output Image FileName"
Oi you, (Dark Fire) bog off, you're supposed to be swatting not wasting time on these boards. We'll still be here when your exams are finished and the magazine written. So get your priorities sorted out, we are last in the queue.
A very good point. I'll go now. I should be back at the end of June, or the start of July, when my exams are over.
OK, my first check was a bit premature as I closed everything after I saw TGDCLI startup. Ran the diagnostic mode to check the .bat.
<edit> The stuff I wrote before was crap ;) The only thing that was missing was the -p tag in front of the project name.
c:\tg2\tgdcli.exe -p r:\pathtotgd\filetgd -r -f 1
In the documentation that was supplied with 1.8.64.0, it says this:
-p [filename]
Open the project specified by [filename] before doing anything else.
Alternatively you can specify the project file in the first command line parameter before any other options.
Unless they have changed the command line structure, the batch file should still work. Does it work?
I've seen it in the docs but I've never had a batch file work without the -p flag on any of my systems. Adding it to the diagnostic bat fixes it, so it may be best to add it in just to be safe.
I'll add it in to be safe. The new version (0.3) should be available within the next 10 minutes.
I'll test it soon so you can disappear *again* comforted in the knowledge that it works ;)
Back at work and connected to my mapped drive again ;)
It's now running the correct TGD from my mapped drive and rendering images as per the Sequence/Output tab.
Even in this beta form this version is tremendously useful for me as I can only get access to extra machines on an adhoc basis and this beats the hell out of distributing .bat files and tracking the progress of each... (although I have just finished the job tracking component of my database). I'll save the rest of my feedback until after exams :P
Thanks. I'll be able to revise for my exams more efficiently now, as my mind won't have much of an excuse to wander...
The exams are finally over ;D. I am currently in the process of preparing to start major updates for all of my software. The Emptosoft Rendering Tool for Terragen 2 Technology Preview will become one of the flagship pieces of software, at the expense of the software I wrote for Terragen 0.9. Several more beta versions will be released between now and the release of the version 1.0 of the software, when it will be renamed 'The Emptosoft Render Farm Manager'. As with T2TP, complications will probably arise, and I am therefore not setting a definite release date. However, I am pretty sure that version 0.9 will not be released before October 2007 or after October 2008 - I have a lot of other things that I simply have to do in the next year, including more exams and applying for university. Version 1.0 will be released shortly after Terragen 2.0 is released (just in case there are any last-minute changes to Terragen 2.0).
On a more interesting note, I am looking into making the Emptosoft Rendering Tool for Terragen 2 Technology Preview automatically stitch together images that have been split for more efficient rendering. I have become interested in adding the image-stitching feature not only because it is a challenge (I've never done anything like it before) but because it doesn't require T2TP for me to test it.
Anyway, I will post news here when I release new beta versions. Use this thread for feedback.
Hope the exams went well. Just let us know if you need anything tested :)
I use The Condor Project to control my T2 renders on OS X. It is free, but you'll need to make yourself a simple program
to generate the render submit requests. You can use any language to make the render submit requests. Condor also works on Windows and Linux.
I've downloaded Condor but haven't gone to the trouble of setting it up yet. Both have their place. DarkFire's software is very quick to get running for the less technically minded and the development of it will include some TG specific features/ uses.
I started writing software for Terragen because I thought a could come up with software that is both easier to use than the commonly used rander farm management software, and also has handy Terragen-specific features. I actually wrote the software for my personal use originally, and I try to maintain that level of simplicity, speed and handy random features that I like.
I think that all of my software can be summarised by the following sentence:
Making technically complicated stuff fairly easy to do while maintaining a reasonable level of customisation. ;D
For various reasons, the main one being lack of time to do the huge amount of work that needs to be done, most of my software, including the Emptosoft Rendering Tool for Terragen 2 Technology Preview, will be going open source in the next few months (unless somebody wants to buy any of it, which I doubt).
Here it is (http://ccgi.emptosoft.plus.com/main_site/download.php?view.6)...Open-source at last! :D
(Released under a beerware (http://en.wikipedia.org/wiki/Beerware)-type licence).