I have a 4200 x 3300 picture that I am crop rendering. I noticed in the activity monitor that the different crops used different amounts of the available cpu. The sky area and the bottom area used nearly 100%. However the middle area where the sunset is over the horizon only used a max of 60% which significantly slowed render down. Any ideas why that would be and any way to get around that? thx
That seems very odd. Are the crops all the same size and shape? Was CPU use consistently around 60% for the entirety of the render of the middle crop area?
- Oshyan
The top crop (40%) which consisted of a ray traced sky and the top half of trees was at 98% for the cpu. So was the bottom 20% which had sand and fake stones. I restarted the middle crop render (ocean horizon) started off great but then slowed down once it started rendering the horizon. My ram says there is still 2.26gigs free, 1.34inactive, and 3.47 active if it has something to do with ram.
To test the problem some more, I turned off the lake I had and the cpu overall was better. It had its moments where it was operating at 30% but then it would jump to 60 and then 95 for a while and then back down after some time. It is calculating the land far off in the horizon when all of this is happening.
Also... I noticed that it is every other cpu(multithreading) that basically cuts out on this part. I have a quad core but with multithreading, an eight core. Four are working fine by the looks of it. I think the cores aren't multithreading if I used the correct terminology. Once the ocean horizon part is done rendering though, all eight cores pick right back up. I attached picture of activity monitor and cropped render
Very interesting result. Would you be willing to share the scene so we can look into it a bit further? It may point the way to future optimizations, or we may at least find a workaround. You can email the files privately to support AT planetside.co.uk.
- Oshyan
Ok. I sent the file. I have been running some more tests and noticed that even a default project does this. For the pre pass it uses all eight but for the final render pass the multithreading is reduced. It is a slightly different scenario since the pre pass was not affected but thought to let you all know this as well. Maybe I have a problem with my mac... Thanks for looking into this.
Are you using v2.4?
Matt
So far in a test of the center crop region at the specified resolution, I am seeing 99% CPU usage average on an i7 across 4 cores/8 threads. It's been rendering for over 3hrs, and it has staid very steady around there (between 95% and 99%). The horizon does seem to be rendering very slowly, probably due to the very high detail setting, but it does at least appear to be using all CPU on my end...
I did notice however that you appeared to be attempting to use a GI cache file. I opted not to do so since I didn't want to have to generate the cache first, so it may be a bottleneck in the GI cache utilization somehow. I did see that your settings for GI cache use seemed odd, you've got it set to single file, but you're specifying a 5 file blend range. Did you render a GI solution for each crop, or 1 for the whole image? Either would work, depending on your settings.
- Oshyan
I am using the most recent update. 2.432. Yes, I ended up cropping the cache files. I had just started reading on them and didn't think to make one large file and crop the final renders.
I just started rendering the middle section with a cache file. As of right now the cpu is below 50%. Do you think it is an overheating issue on my mac's part or would it have anything to do with ram? (I have 8 gigs). It is not even a year old but I won't throw out the possibility.
Scene is currently using 5.6GB of memory on my machine *without* the cache file. Using a cache file it would almost certainly be more. So although you have 8GB of RAM, depending on what else you're running and how much the OS and system tasks take up, it's possible that the system could be paging out to virtual memory on disk, which would definitely slow things down. Of course it's also possible the cache use itself is slowing things down, so I'll restart my test with a cache file in a bit. Note however that in order to properly use multiple crop cache files you would need to use "equal blend within range" and set the number of blended files to the number of crops/cache solutions *and* you would need to name each gi cache file sequentially, e.g. "gicache-crop1.gic", "gicache-crop2.gic", etc. With the settings in the file you sent me, it's just using a single GI cache file, which corresponds only to one crop area.
Also, I assume you're using "Render 01" both to generate cache files and to do final render. If not, let me know and I'll change my tests.
- Oshyan
I guess I don't fully understand. How does using a cache file use more memory. I thought it was on the hard drive and it saves memory?
Also as a side question... it doesn't matter if ram is "free" or "inactive" right?
Oh and yes I am using Render 1
I just started the render without the cache file and the ram was nearly taken up so it may be that...
Also, I think I can scratch what I said earlier about rendering a blank scene. I was getting confused when a core would be done and stop rendering while the last few were. I remember that the cores stop once there is no sections left for it to take.
Quote from: Thejazzshadow on June 11, 2012, 02:39:45 PM
Ok. I sent the file. I have been running some more tests and noticed that even a default project does this. For the pre pass it uses all eight but for the final render pass the multithreading is reduced. It is a slightly different scenario since the pre pass was not affected but thought to let you all know this as well. Maybe I have a problem with my mac... Thanks for looking into this.
Hi,
Quote from: Thejazzshadow on June 11, 2012, 09:56:07 PM
As of right now the cpu is below 50%. Do you think it is an overheating issue on my mac's part or would it have anything to do with ram? (I have 8 gigs). It is not even a year old but I won't throw out the possibility.
I have never had problems with overheating with any of my Macs in the last 10 or so years so I don't think it's that. The only times I have shut them down due to heat issues was when the room they were in was also very warm and the fans were really going, and that was just a precaution.
I'll ask Oshyan to send me the files so I can look at them too.
CPU usage will not be at 100% the whole time. Certain parts of the render are able to use the CPUs more effectively. Typically I would expect that the CPU usage would be close to 100% during the GI pass but may drop off as the main render pass starts.
In Activity Monitor "Free" RAM is available to be used by applications. For example you may see this reduce as TG2 starts to use more memory during rendering. "Inactive" RAM is RAM that is being used but hasn't been touched for a while. I wouldn't worry about that at all.
Regards,
Jo
that makes a lot of sense. So it looks like my next step will be to try to get more ram to help with these big pictures.
I don't think use of GI cache will necessarily use *more* memory (depends on how you're using the GI cache feature, I would think), but I'm fairly certain it does not use *less* memory. The GI data still has to be loaded into memory to be used.
Still rendering the cache file on my end for the next test... I'll send your TGD on to Jo to take a look at.
I would also recommend decreasing the detail level, which may make it require less RAM to render. Detail 1 is seldom necessary, especially at such high resolution. 0.75 should be very good.
- Oshyan
thank you to both of you for your help and advice.
I hope this thread isn't closed, because I have a similar setup to Thejazzshadow and also have/had problems. I have an i7 chipped 27" iMac that I specifically bought to do renders. With the new TG2 upgrade, everything appeared to be working perfectly - until it comes time to finish the picture & save. No can do. Right now I have a beautiful, highly detailed, almost finished render at 2560 x 1440 that I can't save. The render is done & the clock stopped, but the program isn't responding in the activity monitor and yet still has a nearly 3 GB footprint in real memory. Would that be the uncompressed picture file I want to save? Kudos for all the improvements so far, but the program is useless if it won't save in a usable format.
We weren't able to reproduce the original issue mentioned in this thread, but it seemed to come down to possible memory issues. Your issue sounds different, although it may also be memory related. How much system memory do you have and how much is indicated as free/available?
The best way for us to diagnose further is to get a copy of the TGD file (and, ideally, any necessary resource files such as textures and objects).
- Oshyan
Hi ryst,
How long are you waiting for the render to finish? TG2 can take quite some time between when the rendering on the screen finishes and when TG2 is actually finished. Because TG2 is doing something computationally intensive during this time the rendering clock stops and TG2 will be described as not responding in Activity Monitor. It should still be working though, that just means it isn't responding to user events because it's busy.
How long is the render taking and how long are waiting for it to finish? Can you send me a project file for a scene you're having problems with? My email address is:
jomeder@planetside.co.uk
Regards,
Jo
Hi,
Further to my last message, look in the System Memory tab in Activity Viewer. How much Free RAM is showing at the end of the render while you're waiting?
Regards,
Jo
Thanks for the responses.
The render in question had about 8 hours time from the clock stopping to when I got fed up & force quit. Got a screen shot first, but that's not a solution.
I'll send you the file, it's a very simple scene.
QuoteNote however that in order to properly use multiple crop cache files you would need to use "equal blend within range" and set the number of blended files to the number of crops/cache solutions *and* you would need to name each gi cache file sequentially, e.g. "gicache-crop1.gic", "gicache-crop2.gic", etc. With the settings in the file you sent me, it's just using a single GI cache file, which corresponds only to one crop area.
Isn't it easier/better to run a GI cache prerun on the whole uncropped scene, even at lower detail and/or size, save that and then run the crops using that one GI cache file?
Dune, yes that's correct. I was just explaining how you *would* use crop cache files if you wanted or needed to for some reason.
- Oshyan
Right, thanks.
This 'unable to save a render' thing happens quite often to me since 2.4, usually if I have a small crop area enabled, for some reason.
If I raise the crop size and re-render, it usually finishes and saves fine(It has also happened when the full screen was rendered too so I couldn't say that it has anything to do with the cropping, since I always have crop area enabled so I can easily drag the crop handles around without editing render settings), I can only say that this hanging post-render issue mostly happens when I have a very small crop enabled, raising/changing the crop size usually stops it and it almost never happens when the full screen is rendering.
I've tested with some small crops that failed to complete and it repeatedly didn't finalize/save the render when it's done. Moving the crop handles has fixed it every time it has happened to me with a crop region.
Quote from: dandelO on June 15, 2012, 05:30:35 AM
This 'unable to save a render' thing happens quite often to me since 2.4, usually if I have a small crop area enabled, for some reason.
If I raise the crop size and re-render, it usually finishes and saves fine(It has also happened when the full screen was rendered too so I couldn't say that it has anything to do with the cropping, since I always have crop area enabled so I can easily drag the crop handles around without editing render settings), I can only say that this hanging post-render issue mostly happens when I have a very small crop enabled, raising/changing the crop size usually stops it and it almost never happens when the full screen is rendering.
I've tested with some small crops that failed to complete and it repeatedly didn't finalize/save the render when it's done. Moving the crop handles has fixed it every time it has happened to me with a crop region.
If you have something that's reproducible like that, it makes it much easier for us to fix the bug. Please send us one of these file :)
Matt
Hi,
Just thought I would mention that ryzst's problem seemed to primarily be related to running out of RAM. The render still continues but using virtual memory and it gets very very slow.
Regards,
Jo
I just wanted to mention in this discussion that I started that I upgraded my Macbook Pro to 16gigs of ram and that it fixed a lot of these issues. Although I notice that cpu usage drops during certain parts of the renders (for my pictures, it is correlated with water) at least the pictures are making progress now during these parts and not appearing to come to a halt.