OK... my 2c worth on the problem (ie. I could be wrong). Whenever you mention image projections I think back on stitching panoramas, and a possible cause struck me while walking to work. It may be possible that recilinear distortion used in the render calculation is not being matched by remapping the imagemap to match the camera's fov. This would cause alignment errors which would get worse as you move away from the centre of the image (which appears to be the case).
My test that I mentioned in my previous post was performed without this in mind, but it does match the expected behaviour. The attached image shows the difference between a tiled grid image using plan Y and the render of this applied as a TTL image. The errors are radial in nature and get larger as you get closer to the edges. Lines passing through the centre of the image won't be affected much.
The zip file contains the two TGDs and source images.
Have to work now, but if anyone wants to check this further, the problem should be exacerbated by using a wider fov (try 120° - 140°) and should be negligible at narrower fovs (try 5°). Increase the foc on the first TGD, render it and apply the output to the second TGD. (I had to convert the output image to greyscale and resave in Photoshop because TG didn't like the raw output TIFF)
If I'm right, then this is a problem that would make through the lens projection unusable for very accurate masking, but it should also provide a clue for a fix. I'll investigate this later and see if I can confirm this is the case and whether it is possible to do a pre-render remapping of the image map to counter the problem as an interim measure.
[edit] A possible solution to try may be to treat the original image map to be used with through the lens projection as an equirectangular image with the same FOV as the render camera and remap it to a rectilinear image with the same dimensions as the source image. I'll try this later tonight. If it works it should be a relatively simple workaround [/edit]
