I've rendered larger sphericals on a 12gb machine (using Terragen 3) with fairly large populations with no problems.
Does your machine start to thrash its virtual memory? It might do that before showing that error message if Terragen had really decided to chew up a lot of memory. Don't know if it means anything but your windows error box shows 0% progress from tgd.exe - has it actually started rendering?
Looking up the 0x40000015 error gives:
"According to the experts, 0x40000015 error code may be triggered by an incomplete installation of a particular software/update, incorrect uninstall, accidental removal or replacement of registries, and similar issues. Besides, it has been noticed that 0x40000015 error pop-up sometimes shows up after PC's recovery when a virus/malware/corrupted files are deleted."
Re-install Terragen?