intermittent black areas in renders

Started by oysteroid, February 12, 2015, 01:28:42 PM

Previous topic - Next topic

oysteroid

Lately, while working on my Iceland scene, I have been plagued with problems with intermittent black areas in the shadowed regions lit by the enviro light. I am wondering if I might have too much in my scene for TG to handle. I have a number of high poly objects, two 4096x4096 vector displacements, several similarly sized masks and color maps, a complex rock bed, a waterfall made with a really dense cloud layer and a complex network of blue nodes that I use to shape the cloud into a parabolic, soft, long-exposure type waterfall.

For a while, my waterfall was intermittently disappearing or rendering black where it was in shadow. Sometimes, from test render (crop area on) to test render, with exactly the same settings, the problem would come and go. I rebooted. No help. Finally, I took my main nodes and made a clip file and created a new file and inserted them. Then I checked and made sure that all of my settings were perfectly identical between the two files. The problem disappeared! So I went on. That problem never returned.

Now, I am getting weird dark areas and little light areas that shouldn't be there under my water.

Bear in mind that my scene isn't finished. I hope the problems don't delay me and prevent me from finishing in time for the deadline! I have spent many hours fiddling, trying to fix some of these problems rather than working on the scene.

Here is a crop showing something that recently reared its ugly head:

[attachimg=1][attachimg=2]

The little dark areas shouldn't be there. I have a fake stones rock bed under the water. If I disable the lake to see if it is the rocks, the dark areas go away. Notice that the dark areas are obviously not something reflected from above the water. There is nothing above the water corresponding to it. And I don't know what the little white splotches are. Those aren't part of my waterfall and I haven't yet made the splashy stuff that should go at the bottom of the fall.

I have tried disabling the displacement on the water that makes the waves and the artefacts are still present.

Sometimes, while testing with crop renders, the whole area where the rocks are reflected in the water will be entirely black.

At one point, the whole area in shadow was rendering black, including the terrain, my objects, and so on. All looked fine in the 3D preview.

Note that I have "ray trace objects" turned off. That is because I am displacing the moss on the rocks. My rock outcrop is hand sculpted. The moss is an OBJ created using ZBrush's "extract" feature on masked areas of the rocks.

Here is another crop demonstrating a problem. I disabled and disconnected the waterfall. I turned "ray trace objects" on. This ruins my moss since it doesn't displace. Anyway, notice that there are no artefacts:
[attachimg=3]

So I turned "ray trace objects" back off and did a smaller crop near the bottom of the area in the previous image. The reason for the smaller crop is that with ray tracing of objects off, it becomes much, much slower, and I am impatient and in a hurry to finish before the deadline. Here's what I got:

[attachimg=4]

Notice how black the upper part is, which is totally unlike the previous render.

So, without changing any other settings at all, I turned "ray trace objects" back on and kept the same crop to see what would happen. Here's what I got:

[attachimg=5]

The only reason that the little black border around the last two is different is that I quickly cropped the output in PS, leaving some of the unrendered black area in.

Notice several differences between the last two. With "ray trace objects" turned off, everything is darker. And bear in mind that the green foreground part is just displaced terrain. It is not an object. And it has a constant color applied. But notice the dark splotches on it. And notice that even with "ray trace objects" turned back on, there is still a black area near the top that shouldn't be there and which wasn't there in the first render of these last three.

What is going on? Is there any hope that I can make this thing work before the deadline?



oysteroid

Here are another couple of test crop renders:

[attachimg=1]
[attachimg=2]

In the first one, "ray trace objects" is turned off. In the second one, it is turned on. Everything else is perfectly identical. I just changed that one setting. Both have artefacts, though one is clearly worse.

Could this have something to do with the render buckets? It seems to change depending on how I crop the image, though artefacts are present even with a full sized render. The very first image from the first post is a PS crop of a full render from TG.

If you want to help me try to understand this, I should probably describe how my scene is constructed. I have two major vector displacements. One is a distant mountain. The other is the basic shape of the whole ground area near the camera. It is raised up around the edges and depressed in the middle, where the water is. I have an image texture mapped on top of it. I have fake stones covering the area under the water, masked by an image. So the fake stones are on top of a vector displacement. The water is a vector displaced lake object (the problems remain when the vector displacement of the water is disabled and disconnected). I have two rock outcrops that are created in ZBrush and are fairly high-poly. The main one is about a million polygons. The other is much smaller. I have a grass object created with ZBrush's fibermesh tools. I have a single moss object for the rocks that I am displacing in TG. I have a waterfall that, as described, is created with a network of blue nodes plugged into the density input of a cloud layer. But I can disable and disconnect the waterfall and problems remain.

My machine has a Core i5 2500 and 16GB of RAM with Windows 7 Pro. Everything else is working fine. Well, ZBrush crashes some, but that seems typical. It is known to be fairly unstable.

oysteroid

I checked memory utilization for the same settings as the last render and it was 2.7GB. Some things in the scene are disabled.

While I rendered the last one again, which yielded the same results, I noticed that the heavy artefacts at the top, over the waterfall, rendered in last, on top of what was initially a clean waterfall. This wasn't true of the lower artefacts in the reflection.

oysteroid

I think I may have discovered the source of the problem. It seems to be the moss. Let me describe the moss object. I took my main rock outcrop in ZBrush and drew some masking on it. Then I applied the extract feature, which generates another mesh from wherever the mask was, and applies thickness to it. Then I dynameshed it to adjust poly density and round the edges a bit. I took that into Blender to rewrite the surface normals as smooth.

I created a procedural texture for the moss in TG to displace it to give the finer moss texture. But strangely, I found that to get any visible displacement, I had to crank up the displacement setting in the surface layer node to about 7. Consider that everything in the scene is roughly to scale. The waterfall is only about ten meters tall.

Well, remembering this and noticing that some of the artefacts seem to be where the moss is reflecting in the water and where the moss is behind the waterfall, I disabled my moss texture, replacing it with the pink color, and look at this:


[attachimg=1]

Here it is with the moss displacement set to 0.1. But notice the moss is black now!

[attachimg=2]

Here is my moss texture node setup. Does this look wrong somehow?

[attachimg=3]

I am going to go check the integrity of my moss mesh...

oysteroid

I found the problem! Finally! I lost a whole day though!

The way I set up my scene from the beginning in ZBrush was screwy. The imported plane I started with to do the vector displacement of the topography was too small and I didn't realize it. I never did correct the size, not realizing it would be a problem. And scaling by a precise value is problematic in ZBrush. So, in TG, I was scaling the objects up by a factor of about 40 to get them close to the proper size. Well, little did I realize, this apparently has an effect on the displacement values. I resized my moss object in Blender and using it in TG with no scaling, the problem is gone.

The thing that I don't understand is that the effect of scaling up such a small object on the displacement values in the surface shader node is the reverse of what I'd expect. If it were to have an effect, I'd expect that you'd need to input smaller values rather than larger ones to get displacements of a given size, since it seems that you'd be scaling the displacement up with the object.

Whatever the case, I am glad it's solved. Now, let's see if I can finish by the deadline! I still need to populate the scene with plants, arrange a nice atmosphere, fix my waterfall a bit, make some splash spray at the bottom, fix my mountain, and a few other things...

Oshyan

Wow, sorry for the late reply given the looming deadline, but I'm glad you figured it out! Indeed, extreme displacements and unusual scale mismatches can cause odd and intermittent artifacts. In fact, with what you were saying about your moss and displacement, it was one of my first thoughts when I read the initial posts above. That being said, hopefully you've done crop renders now with the fixed moss settings to ensure that you do need Raytraced Objects disabled, because not only is RTO faster, it's higher quality too. And when enabled, objects with displacement do still get a bump map version of that displacement, which works fine in many situations, especially at a distance.

Anyway, it looks like a very promising scene. Can't wait to see the whole thing. :)

- Oshyan

oysteroid

I am not sure whether to put this here or in the other thread (http://www.planetside.co.uk/forums/index.php/topic,19627.0.html) I made about intermittent darkness with the lake object which was fixed by increasing GI cache detail. Anyway...

Some similar problem came back. I turned GI cache detail up to 4 previously and that seemed to help. But after populating my scene with plants and adding a lot of complexity, it seems it decided to act up again. So I set GI cache detail to 6. That caused my shadowed areas to light properly, at least above the water. But now, there are weird large, sharp-edged areas of dark or light under my water that shouldn't be there. For now, since in my scene you shouldn't really be able to see under the water much, I just set the decay up to hide it.

Any ideas what might cause it? Nothing is being scaled up anywhere. GI cache detail is at 6. What else can I do?

Here are some examples:

[attachimg=1]
[attachimg=2]

In these images, you might notice the odd dark shapes under the water. Those aren't there if I disable the lake. Consider that these are both renders of the same area with the same exact settings and camera angle. The only difference is a different crop render size. See how the dark artefact changes? And in one of these, the rocks in the upper right are black again, even with the GI cache detail at 6.

Oshyan

Well, it may not be a GI issue, this might be more about direct shadow casting, especially now that you have vegetation and other things casting shadows (potentially) into this area. So you may be seeing the limitations of detail outside of the current crop or camera frustum. If you look in the Renderer settings under Advanced tab, you'll see Ray Detail Region padding settings. When doing crop regions, you could try changing to Detail in Camera (I think), or leave it at Detail in Crop and set Ray Detail Region Padding to 0.25 or 0.5 as a starting place to see if it helps. The basic answer, if that works, is that TG simplifies geometry outside of the current render area/camera view, and if you shift your crop, the geometry it sees may vary since the area of consideration is related to the crop region.

- Oshyan

oysteroid

Thanks, Oshyan. I haven't tried any of that yet. I've been too frantic trying to get the rest of my scene done before the deadline!   :D  So I just hid the stuff under the water. And now I am waiting on my final render. Hopefully it comes out alright with no surprises and finishes in time!