I'll close this post off now since I started a new one in Image sharing (with a cylindrical version of the pano as well). Feel free to leave comments there
http://forums.planetside.co.uk/index.php?topic=1178.0 (http://forums.planetside.co.uk/index.php?topic=1178.0)
<Up-Update> Restarted rendering this now that a) I got the sequence rendering working and b) installed the latest TG update. The combination of these two has slightly affected the density shaders of the populations which has repositioned the trees. Also tweaked a few things that were bugging me in the first (and replaced GI with fill lights) Progress should be faster now :)
Now running an on the fly render and conversion just for the hell of it. Online version will now automatically update until completed.
http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons_v6.mov (http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons_v6.mov)
</Up-Update>
<Update> Render stopped at 12%... Very slow with manual frame changes. I've added a render without objects as a background onto which I'll drop the tiles with objects. The blue gaps in the clouds are due to the use of whole numbers to scale the cloud's density fractal (1,1,3 in this case). I'll be changing the clouds and dropping in the fixed version later (using 1.01, 1.02, 5.02) ((No, appears to be an acceleration cache issue?)).
Concentrated on adding some more grass at the bottom to get a better feel for the model. I like it, even if it is a bit green.
[attachthumb=#1]
QTVR (1Mb) (http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons001.mov)
</update>
I told you I like seeing how far I can push something until it breaks ;)
There are some other potential issues relating to rendering panoramas that I hadn't considered before. The first one that struck me was running out of RAM to render a single frame. I have a lot of objects in my project, and trying to render a large 90° tile with lots of objects usually crashed before the render finished.
I picked the camera angle that looked through the most trees and set about finding a tile size I could render. I ended up with a 10° tile of 100x100 pixels. Setting up the camera keyframes was pretty easy with excel, and a little more work converted the same numbers to a PTStitcher script to stitch the frames. A quick test to check it all worked: http://www.path.unimelb.edu.au/~bernardk/tgdemo/pano_10deg.mov (http://www.path.unimelb.edu.au/~bernardk/tgdemo/pano_10deg.mov) (1.9Mb... sorry max. jpeg quality, 3600 pixels wide)
The stitching was not entirely straightforward as 614 temp files were too much for my RAM (614 x 18.5Mb for my 3600x1800 panorama). I split it up into batches of 100 tiles and then merged the output panoramas in Photoshop. A little bit of mucking around but nearly all of it can be semi-automated so it's relatively painless. I'll probably change the PTStitcher script to just output a TIFF panorama for each tile (it includes an alpha) and then create a Photoshop action to merge them together, cutting it down to two steps.
Now all I have to do is render the other 580 frames of my project:
http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons001.mov (http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons001.mov)
The blue spruce tree in the foreground is slowing down the render at the moment. This population and my grass have reflectivity and translucency set to make them look pretty at the expense of render time.
looks like you have alot of manual camera changes to make in tg2 while rendering ;D ;D
the second pano will look great once its finshed.. BTW how long has it taken so far and how long will rendering another 500 frames take?
I'm manually rendering on one computer at the moment and trying to produce a workaround to automate the rendering on another (I have Deep + animation). Render times are around 6 minutes for sky, going up to 4 hours for a horizontal angle looking through the tree close to the camera. I was a bit shocked by this at first until I (intermittently) watched a frame render and saw all of the trees in the background. Once I get past this tree the times should halve or better (I hope) as the close up tree really does take a long time to render. There's also a lake visible in the distance to the left of the tree, and possibly another lake visible through a small gap in the trees to the right.
I should really do a low quality render of this as a preview. I haven't actually checked out the final distribution of the blue spruce population. I just wanted to look at a finished product with high quality render settings to see what else needs tweaking (other than the stuff I already know)
Great work! The second one had a halo feeling, you know how its a ring? :P
Well, anyway, nice work. The finished result for the second one should be VERY pleasing. I can see the grass and trees look very nice, however the grass is exceptional. Maybe you should have a grass fields for a QTVR next time :P
Heh, by the way, nice work for getting the Uni of Melbourne to host :P
Quote from: DiscoBall on March 06, 2007, 04:02:14 AM
Great work! The second one had a halo feeling, you know how its a ring? :P
Well, anyway, nice work. The finished result for the second one should be VERY pleasing. I can see the grass and trees look very nice, however the grass is exceptional. Maybe you should have a grass fields for a QTVR next time :P
Thanks. Yes I think a grassy field with rolling hills would be a good way to show off the grass texturing.
Quote from: DiscoBall on March 06, 2007, 04:02:14 AM
...
Heh, by the way, nice work for getting the Uni of Melbourne to host :P
It helps when your the server admin :P
Hey ben, nice work...
can you tell me/us how to make QTVR's??
Quote from: dhavalmistry on March 06, 2007, 10:33:06 AM
Hey ben, nice work...
can you tell me/us how to make QTVR's??
Agreed (even though I probably don't have the software I would need ::)) Great work so far bigben, it's looking fantastic. Can't wait for the finished version. Are you using xfrog grass?
I've already posted the software I use here:
http://forums.planetside.co.uk/index.php?topic=853.0 (http://forums.planetside.co.uk/index.php?topic=853.0)
Attached are my render and camera nodes. I have the animation version so I've set up the camera angles for all of the frames... even though I'm manually rendering each frame, it's very easy changing camera angles.
I'll add the clip file and the stitching script/photoshop action for this high res project to my next supplement to the file sharing post above. (I haven't made the Photoshop action yet)
Quote from: nvseal on March 06, 2007, 11:46:07 AM
... Are you using xfrog grass?
No. This is a free 3DS model I downloaded and converted. It's different to the model in this post: http://forums.planetside.co.uk/index.php?topic=933.0 (http://forums.planetside.co.uk/index.php?topic=933.0) but it uses the same surfacing which is definitely the key to its appearance. Unless you're getting *really* close to grass the model can be quite basic and still look very real.
I've posted the various scripts I used to create this panorama in File Sharing
http://forums.planetside.co.uk/index.php?topic=853.msg8090#msg8090 (http://forums.planetside.co.uk/index.php?topic=853.msg8090#msg8090)
It's looking really good, big B.
There's just enough rendered now to start getting a better feel for the objects in this QTVR.
View QTVR (1Mb) (http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons001.mov)
Nice, but whats with the tree to the left? Great terrien by the way, can wait to see what more you come up with!
Regards,
Will
Quote from: Will on March 14, 2007, 07:22:44 PM
Nice, but whats with the tree to the left? Great terrien by the way, can wait to see what more you come up with!
Regards,
Will
Hmmm the tree on the left in a forest of treees??? ;) I'll guess that you're referring to a wee grand fir that looks a little out of place just to the left of the closest spruce.... 'cos when I first saw it I thought "What's up with the tree on the left?"
I used 3 populations of the grand fir model (scale ranges 0.7 - 1) using each of the XFrog models. This li'l tree just got a bit separated from the rest. It does look a little out of place but this may not be as noticeable once there are more trees to the left.
The tree densities are a bit low for my liking which has contributed to this "problem" but then I'm pushing the limits of my RAM as it is. I've been toying around with ideas for working around this by animating an image map to only create trees that may be visible to the camera. The mask would consist of a circle around the camera to allow for trees located out of view but with branches extending into the frame, and a wedge extending into the distance with a fov slightly greater than the camera lens. Everything outside of this would then be subtracted from each population's density shader.
This should allow for populations covering greater areas, with greater densities.
Ben, I've thought of doing the same thing as you describe and have done some very basic tests successfully. Let us know if you do attempt that and how well it works, especially with the potential issues of branches extending into frame, as you describe.
- Oshyan
I'll let you know how I go. I tossed around a few ideas for the shape of a mask, primarily for use with animations. The mask will probably require two images. A circular mask for the area surrounding the camera and a wedge shaped image for the camera's fov. Separating the two allows the second to be scaled by a variable amount depending on how far you want to load objects away from the camera.
There are some projection issues that may crop up which may adversely affect the coverage of the mask. It may be necessary to define a projection camera for the wedge-shaped mask directly above the render camera, looking straight down in order to maintain the correct angle at different render camera altitudes.... we shall see.
<edit> Actually, I practically answered my own question there....
Mask 1: white image map, projected through a second camera with the same position and orientation as render camera, but with a slightly larger fov. This will catch all distant objects just out of frame.... much simpler ;)
Mask 2: white circle, plan Y projection, at render camera location. This will catch nearby objects out of frame. The exact size required remains to be seen, but keeping it as small as possible will reduce the number of unseen objects that need to be created.
</edit>
Hi Oshyan... Here are the results of my masking test... it's looking very promising :)
On reflection there are different situations that will break things in different ways. To deal with each of these, I ended up using all 3 options mentioned in my previous post.
- A circular image map for near camera objects
- A wedge-shaped mask projected through a camera pointing down directly above the render camera
- A "through the lens" mask through a duplicate of the render camera with a slightly increased fov.
[attachthumb=#1]
The first compensates for a low camera position, catching objects behind the camera that may extend into the render. The size of this is arbitrary here but you could easily calculate this.
radius of circle = width of object / tan (fov
mask - fov
render)
<edit> I should have calculated it. For a 10m wide object I could use 300m instead of the 500m I picked at random in this test
</edit>While the last two masks may appear to do the same thing, they do cover certain situations better. The last one may not populate the far side of ridges resulting in serious omissions in gently undulating terrain with a low camera angle, but it will correctly mask a wide angle camera tilted downwards from a high camera altitude.
Attached the TGD and image masks for you to try. Not sure how stable this will be in a render but it does have considerable potential to populate large areas efficiently, and the masks can be easily animated.
There are probably better ways to put these together but I find it easier to use little building blocks so that I can see what I'm doing, and modify things one bit at a time.
Crossing my fingers now... just started rendering a sequence from my QTVR. With a bit of luck this may work around my problem of the sequence crashing after frame 1. I'll find out in an hour or two....
:( Nope. Grass population was still high (133,000) and it crashed. Changed the scale and spacing of the grass and trying again. 65,000 this time.... still crossing fingers. Tree populations are only 2,500 though, which is a big improvement for a 20km square... and RAM usage is down to 500-600Mb, nearly half of previous renders!!
;D BINGO!!! ;D
... well it's started rendering frame 2 anyway. I'll have to work on a good fake grass surface for distances over say 100m. The distant grass objects were certainly increasing render times a lot anyway. Frame 3 now :) Thanks for giving me the extra nudge to extract the idea into a working model.
The other good thing is that it was relatively easy to incorporate it into an existing project. Edit the camera and image map locations and then slot the final shader into the existing node network.
[attachthumb=#3]
With the modular approach that I use, it's also easy to duplicate for use with multiple density shader networks. All you need to do is duplicate the last shader and input the last Add colour into its fractal breakup input.
Frame 4 now :D
bigben,
Just thought and a question :)
I was working on a similar situation using the through camera masking of a population and found that the projected image mask (location bottom left, size 1x1 position 0x0x0) does slightly go beyond the fov of the projection camera. Further experimenting showed that I was able to resize the image shader and "offset" it using a negative position (such as size 1.5x1.5, position -0.25x-0.25x0) giving me an even wider fov. Thus allowing me to eliminate a camera by using the render cam. In your case this would keep you from needing to add and animate you're 3rd camera. :P
Also I haven't gotten Deep+Animation yet :D but was wondering if you are needing to select "Repopulate every frame" in your populations?
~§ardine
Well I'm getting closer to getting this to work. There are still some stability issues I have to overcome (which were not expected to be fixed by the update) Rendering as a sequence I got the first 8 frames of sky OK but then it crashed. Restarted (at frame 9) and got the next two frames to render and then it crashed again....
Now that the update is out I'll have a serious play with the CLI over the weekend and try some on-the-fly stitching. It should overcome the remaining sequence render problems I'm having.
If the frames render OK I might even get to increase the fov of my tiles now that the RAM usage has dropped (or make the tiles bigger, and see if I can break it again ;))
http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons_pano_v6.mov (http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons_pano_v6.mov)
Quote from: §ardine on March 15, 2007, 11:58:06 PM
bigben,
Just thought and a question :)
...In your case this would keep you from needing to add and animate you're 3rd camera. :P
I'll have a play but I'm not sure. I haven't used a through the camera mask before. My main concern was does the mask cover the entire surface within the cameras view... specifically does it cover backfaces? This is critical for this particular use but I haven't tested this yet.
Anything that simplifies it further is worth a try.
Quote from: §ardine on March 15, 2007, 11:58:06 PM
Also I haven't gotten Deep+Animation yet :D but was wondering if you are needing to select "Repopulate every frame" in your populations?
~§ardine
I had tried repopulating every frame but it made no difference, but the issue was definitely related to the object populator.
http://forums.planetside.co.uk/index.php?topic=954.0 (http://forums.planetside.co.uk/index.php?topic=954.0)
Well i decided to do a little experimenting since I was on the subject...
It does appear that the through camera masking will allow populating on the backfaces of the terrain. I set up a camera looking at some very tall displacement. Added a population covering the whole aria with through camera masking. After populating, I brought the preview camera up and was able to confirm the coverage was on the backfaces of the terrain. A render using a different camera at the new location confirmed it as well.
Also regarding the question of repopulating every frame: I'm afraid I was asking a question totally unrelated to your crashes and caused some confusion by not explaining the reason for the question ;D...
I was wondering if you needed to repopulate in order that, as your camera rotates (and so also your masks), the population will follow your view. I'm not sure how populations work with an animation (as I said I haven't purchased anything yet), but am wondering if the population will stay the same in an animation unless you use the repopulate option. Thus causing it to recalculate the new mask position.
anyway... :P
-§ardine
Quote from: §ardine on March 16, 2007, 12:44:49 AM
It does appear that the through camera masking will allow populating on the backfaces of the terrain. I set up a camera looking at some very tall displacement. Added a population covering the whole aria with through camera masking. After populating, I brought the preview camera up and was able to confirm the coverage was on the backfaces of the terrain. A render using a different camera at the new location confirmed it as well.
Thanks, that's encouraging... the other potential issue I forgot to mention was terrain out of frame below the camera's line of sight. If this is included as well then I can probably remove the upper camera altogether... I'll check that later if you don't get a chance...
Quote from: §ardine on March 16, 2007, 12:44:49 AM
...but am wondering if the population will stay the same in an animation unless you use the repopulate option. Thus causing it to recalculate the new mask position.
Doing single renders, TG usually picks up when the distribution has changed (even if the effective result is an identical mask) and repopulates anyway. If this behaviour persists for animations then it may not be necessary, but it would probably be safest to repopulate in this instance... Rendering frame by frame via CLI will effectively be doing this anyway.
Quote from: bigben on March 16, 2007, 01:00:04 AM
the other potential issue I forgot to mention was terrain out of frame below the camera's line of sight. If this is included as well then I can probably remove the upper camera altogether...
Checked on this, and as far as I can tell, you will need to keep that upper camera. I tried doing a image size of Y=100 with a offset of -99 to get the angle down as far as possible. Even then objects directly below and in front of the camera at about a 5 degree angle don't populate.
I did find out that the image shader at size 1x1 is projected as a square out of the camera using the largest angle in your aspect ratio. Thus if you're rendering at 800x600 your image shader will be 800x800 and extend up beyond the top of the render. In this case it will be populating more than necessary outside the picture. :-\ This is different than I had expected as I would have thought that the image mask would have been stretched to fit the aspect ratio of the render. This doesn't change a whole lot but might be something to keep in mind when setting these up.
~§ardine
Thanks for checking these out, it's very helpful.
My sequence renders were still crashing anywhere between 2-20 frames. But the crashes aren't as bad as the previous version, as I'm now getting a notification that TG has stopped responding.
My database is half setup now, with rendering and stitching sorted out. I've started rendering with the CLI as a result, and no crashes so far ;D ;D ;D. Judging by the timestamps, there is no increase in render times with restarting TG and loading the TGD, image maps and objects for every frame, and it's a lot easier checking the progress on my work computer via VNC ;).
I'll setup the on the fly QTVR converson on Monday for whatever frames are remaining... not for any real practical reason other than to show it can be done.
Quote from: bigben on March 17, 2007, 12:00:32 PM
I'll setup the on the fly QTVR converson on Monday for whatever frames are remaining... not for any real practical reason other than to show it can be done.
This process is now running. I've re-ordered the rendering sequence to do row by row (top down) instead of columns, so there won't be too much change for the next 12 frames or so. I've also put the log file online so you can check the progress before donwloading it again.
Commandline processes:
- render frame
- generate script file for PTStitcher
- combine rendered frame with stitched panorama of completed frames
- convert to QTVR
- copy to local web directory (mapped network drive) [View QTVR] (http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons_v6.mov)
- log frame number to file to help track progress in the database [View Log file] (http://www.path.unimelb.edu.au/~bernardk/tgdemo/render.log)
Here's a sample of the code for frame 139. One of the reasons I use a database to generate this stuff ;)
%TERRAGEN_PATH%\tgdcli -p C:\TGD\Tetons\Project1\tetons_2049_v6.tgd -hide -exit -r -f 139
echo p w3600 h1800 f2 v360 u0 n"TIFF" > C:\TG2QTVR\hi-res_v6\stitch.txt
echo m g1 i0 >> C:\TG2QTVR\hi-res_v6\stitch.txt
echo i n"pano.tif" >> C:\TG2QTVR\hi-res_v6\stitch.txt
echo o f4 y0 r0 p0 v360 a0 b0 c0 d0 e0 g0 t0 >> C:\TG2QTVR\hi-res_v6\stitch.txt
echo i n"pano0139.bmp" >> C:\TG2QTVR\hi-res_v6\stitch.txt
echo o f0 y350 r0 p80 v10 a0 b0 c0 d0 e0 g0 t0 >> C:\TG2QTVR\hi-res_v6\stitch.txt
C:\TG2QTVR\PTStitcher.exe C:\TG2QTVR\hi-res_v6\stitch.txt -o C:\TG2QTVR\hi-res_v6\pano2.tif
REM precautionary clean up of temp files if left behind
del _ptstitcher_tmp*.*
move C:\TG2QTVR\hi-res_v6\pano2.tif C:\TG2QTVR\hi-res_v6\pano.tif
C:\TG2QTVR\PanoCUBE.exe pano.tif
REM local copy
move pano.mov tetons_v6.mov
REM online copy
copy tetons_v6.mov t:
echo 139,1 >> t:render.log
Speeding up the render today to get the sky out of the way. With GI turned off I really don't need the populations enabled. This will save TG from populating 6 billion triangles per frame :o The render log came in handy as it was easy to eliminate the frames that were rendered in the last batch. Once I hit the horizon I'll reverse the render order and render from the bottom up as those frames are quicker to render.
I'm using an explorer window to track the render progress. Viewed as a film strip with the stitched image selected I get a large preview that updates each frame :)
[attachimg=#1]
Looking great bigben, I'm looking forward to seeing the final QTVR.
Neither can I... although... I've finished the quick renders of the sky (saved heaps of time there (13min down to 3min per frame)) and I've started the bottom up rendering with objects.
The additional masking of objects has cut the render times for these tiles down to 13 - 45 minutes per tile (down from a previous maximum of 4hrs). Taking an average of say 30min per tile that leaves around 5 days left to go if I stick to just one machine... so I can stop blogging on this post for a while ;)
Heh bigbeg, is the grass that cut_lawn grass you uploaded last time? I am using that at the moment, looks good, however yours looks a bit more..'detailed'...
Same surface, different object. This one has long, thin blades.
http://forums.planetside.co.uk/index.php?topic=1032.msg11152#msg11152 (http://forums.planetside.co.uk/index.php?topic=1032.msg11152#msg11152)
A lawn mower in a forest wouldn't look right ;)
Oh thanks!
However..there doesn't seem to be a tgo I can use..neh, I'm fine with your cut lawn grass :p It's perfect...for now :P
Checked out the up-update, looking good!
Just image how long it would take to create a game with these kind of images (a la Myst 4 style...)
Can't wait to see the final result...
Quote from: DiscoBall on March 21, 2007, 09:25:02 AM
Oh thanks!
However..there doesn't seem to be a tgo I can use..neh, I'm fine with your cut lawn grass :p It's perfect...for now :P
I finally stopped long enough to get my head around the lambert and reflective shaders so I should have a finished grass model next week. My plant models will also contain two surfaces to allow for animation. Grass: live to dead, Trees: green - autumn (where applicable) I'll also release a template for people to drop their own OBJ files into.
Quote from: Mohawk20 on March 21, 2007, 06:36:38 PM
Checked out the up-update, looking good!
Just image how long it would take to create a game with these kind of images (a la Myst 4 style...)
Can't wait to see the final result...
It should be quicker next time around. The savings produced by the population masking method developed during this project mean I could render larger tiles, cutting down 10 minutes per tile for the populator to do its thing... that, and I'm rendering all 614 tiles 'cos I was too lazy to determine which tiles were redundant near the zenith and nadir where the camera angles produce substantial amounts of overlap. There are possibly two rows at the top and bottom where i could get away with only every second image.
<edit>And of course it's now Friday morning and I'm wishing I hadn't been lazy and removed those redundant frames from the batch. 62 frames to go... would have been finished by now, but I'll have to wait another day. Reversed the render direction back to top down to leave the longest render times until last (and see a nearly complete render by the time I go home)</edit>
I finally have 614 tiles rendered. My efforts to speed up the sky rendering by disabling objects cut the tops off a few trees which I am now re-rendering. There are also a few stitching glitches visible on the current qtvr due to various experiments at different stitching sequences. Most of them I expected, but left them in as a reminder to myself. Some of them were not expected and as far as I can tell are due to stitching rows anti-clockwise. Once I have the remaining 18 tiles I'll restitch the entire image from scratch and then have a close look at what worked well and what needs fixing. I already have a sizeable list of what needs fixing, but given how much surfacing I didn't do I'm also quite happy with the way it's shaping up... and I've also learnt a lot from this forum since starting this render.
I'm glad you're almost done ;). I can't wait to see the final result.
Well it's just about home time. I have all the frames (yay) but I won't be around to see the final stitch finish (boo) so I've bashed out a progressive stitch/qtvr conversion to a new file.
<edit>
It will upload itself after the last frame has been stitched. The first 2 columns are there now, give it a few more hours ;)
http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons_v6b.mov (http://www.path.unimelb.edu.au/~bernardk/tgdemo/tetons_v6b.mov)
3600x1800pixels, it will max out at around 800kb.
Great scene, Ben! The realism is great and I love the detail. I'm impressed by your tenacity and patience with this project.
Thanks O_B
Reading back through this thread you can see that I've learnt a lot from getting this through to a "finished product", with workarounds for problems that will be crucial to animations I'm planning (and applicable to large scale renders). I knew of a lot of things I could tweak on this scene prior to starting, but I find that looking at a completed QTVR gives you a whole new perspective of just what works and what doesn't work as well, both for QTVR and normal images.. so from that perspective I had no choice but to draw a line and say OK I'm going to finish a render of this as it is.
Hopefully by the time I get around to rendering the next version I'll have a wee render farm going ;)
I'll close this post off now since I started a new one in Image sharing (with a cylindrical version of the pano as well). Feel free to leave comments there
http://forums.planetside.co.uk/index.php?topic=1178.0 (http://forums.planetside.co.uk/index.php?topic=1178.0)