cpu memory usage different at different parts of render.

[Thejazzshadow]

I have a 4200 x 3300 picture that I am crop rendering. I noticed in the activity monitor that the different crops used different amounts of the available cpu. The sky area and the bottom area used nearly 100%. However the middle area where the sunset is over the horizon only used a max of 60% which significantly slowed render down. Any ideas why that would be and any way to get around that? thx

[Oshyan]

That seems very odd. Are the crops all the same size and shape? Was CPU use consistently around 60% for the entirety of the render of the middle crop area?

- Oshyan

[Thejazzshadow]

The top crop (40%) which consisted of a ray traced sky and the top half of trees was at 98% for the cpu. So was the bottom 20% which had sand and fake stones. I restarted the middle crop render (ocean horizon) started off great but then slowed down once it started rendering the horizon. My ram says there is still 2.26gigs free, 1.34inactive, and 3.47 active if it has something to do with ram.

[Thejazzshadow]

To test the problem some more, I turned off the lake I had and the cpu overall was better. It had its moments where it was operating at 30% but then it would jump to 60 and then 95 for a while and then back down after some time. It is calculating the land far off in the horizon when all of this is happening.

Also... I noticed that it is every other cpu(multithreading) that basically cuts out on this part. I have a quad core but with multithreading, an eight core. Four are working fine by the looks of it. I think the cores aren't multithreading if I used the correct terminology. Once the ocean horizon part is done rendering though, all eight cores pick right back up. I attached picture of activity monitor and cropped render

[Oshyan]

Very interesting result. Would you be willing to share the scene so we can look into it a bit further? It may point the way to future optimizations, or we may at least find a workaround. You can email the files privately to support AT planetside.co.uk.

- Oshyan

[Thejazzshadow]

Ok. I sent the file. I have been running some more tests and noticed that even a default project does this. For the pre pass it uses all eight but for the final render pass the multithreading is reduced. It is a slightly different scenario since the pre pass was not affected but thought to let you all know this as well. Maybe I have a problem with my mac... Thanks for looking into this.

[Matt]

Are you using v2.4?

Matt

[Oshyan]

So far in a test of the center crop region at the specified resolution, I am seeing 99% CPU usage average on an i7 across 4 cores/8 threads. It's been rendering for over 3hrs, and it has staid very steady around there (between 95% and 99%). The horizon does seem to be rendering very slowly, probably due to the very high detail setting, but it does at least appear to be using all CPU on my end...

I did notice however that you appeared to be attempting to use a GI cache file. I opted not to do so since I didn't want to have to generate the cache first, so it may be a bottleneck in the GI cache utilization somehow. I did see that your settings for GI cache use seemed odd, you've got it set to single file, but you're specifying a 5 file blend range. Did you render a GI solution for each crop, or 1 for the whole image? Either would work, depending on your settings.

- Oshyan

[Thejazzshadow]

I am using the most recent update. 2.432. Yes, I ended up cropping the cache files. I had just started reading on them and didn't think to make one large file and crop the final renders. 

I just started rendering the middle section with a cache file. As of right now the cpu is below 50%. Do you think it is an overheating issue on my mac's part or would it have anything to do with ram? (I have 8 gigs). It is not even a year old but I won't throw out the possibility.

[Oshyan]

Scene is currently using 5.6GB of memory on my machine *without* the cache file. Using a cache file it would almost certainly be more. So although you have 8GB of RAM, depending on what else you're running and how much the OS and system tasks take up, it's possible that the system could be paging out to virtual memory on disk, which would definitely slow things down. Of course it's also possible the cache use itself is slowing things down, so I'll restart my test with a cache file in a bit. Note however that in order to properly use multiple crop cache files you would need to use "equal blend within range" and set the number of blended files to the number of crops/cache solutions *and* you would need to name each gi cache file sequentially, e.g. "gicache-crop1.gic", "gicache-crop2.gic", etc. With the settings in the file you sent me, it's just using a single GI cache file, which corresponds only to one crop area.

Also, I assume you're using "Render 01" both to generate cache files and to do final render. If not, let me know and I'll change my tests.

- Oshyan

[Thejazzshadow]

I guess I don't fully understand. How does using a cache file use more memory. I thought it was on the hard drive and it saves memory?

Also as a side question... it doesn't matter if ram is "free" or "inactive" right?

Oh and yes I am using Render 1

[Thejazzshadow]

I just started the render without the cache file and the ram was nearly taken up so it may be that...

Also, I think I can scratch what I said earlier about rendering a blank scene. I was getting confused when a core would be done and stop rendering while the last few were. I remember that the cores stop once there is no sections left for it to take.

Ok. I sent the file. I have been running some more tests and noticed that even a default project does this. For the pre pass it uses all eight but for the final render pass the multithreading is reduced. It is a slightly different scenario since the pre pass was not affected but thought to let you all know this as well. Maybe I have a problem with my mac... Thanks for looking into this.

[jo]

Hi,

As of right now the cpu is below 50%. Do you think it is an overheating issue on my mac's part or would it have anything to do with ram? (I have 8 gigs). It is not even a year old but I won't throw out the possibility.

I have never had problems with overheating with any of my Macs in the last 10 or so years so I don't think it's that. The only times I have shut them down due to heat issues was when the room they were in was also very warm and the fans were really going, and that was just a precaution.

I'll ask Oshyan to send me the files so I can look at them too.

CPU usage will not be at 100% the whole time. Certain parts of the render are able to use the CPUs more effectively. Typically I would expect that the CPU usage would be close to 100% during the GI pass but may drop off as the main render pass starts.

In Activity Monitor "Free" RAM is available to be used by applications. For example you may see this reduce as TG2 starts to use more memory during rendering. "Inactive" RAM is RAM that is being used but hasn't been touched for a while. I wouldn't worry about that at all.

Regards,

Jo

[Thejazzshadow]

that makes a lot of sense. So it looks like my next step will be to try to get more ram to help with these big pictures.

[Oshyan]

I don't think use of GI cache will necessarily use *more* memory (depends on how you're using the GI cache feature, I would think), but I'm fairly certain it does not use *less* memory. The GI data still has to be loaded into memory to be used.

Still rendering the cache file on my end for the next test... I'll send your TGD on to Jo to take a look at.

I would also recommend decreasing the detail level, which may make it require less RAM to render. Detail 1 is seldom necessary, especially at such high resolution. 0.75 should be very good.

- Oshyan