That's a memory-related error. It's likely that you have simply exceeded the amount of memory TG2 can access on your system. Although you may have a lot of memory, 4GB or more even, TG2 can only access up to 2GB of that at any time, even on a 64 bit system (since TG2 is not yet 64 bit), and it is rare to even get the full 2GB due to issues with allocation of large memory areas and other memory management problems.
The memory issues may be coming from any one of a number of sources - large objects as part of your populations (100,000 polygons or more), large population sizes (10,000 or more instances), large textures (higher than 1000x1000) or multiple textures, large heightfields (4097x4097 or higher), etc. Reducing memory use in any one area will help overall.
- Oshyan