Started by PabloMack, July 31, 2019, 03:45:01 pm
Quote from: WASasquatch on August 02, 2019, 11:48:25 amAlso I am not sure what you mean by images? Executables are not images. Images use sectors and headers cause originally they were snapshots of actual HDDs. IMG, IMA, etc have this info so they can be mountable as a medium.
Quote from: Matt on August 02, 2019, 01:39:10 amI think that's just the overall RAM use including data, right? The code portion of Terragen cannot be more than a few dozen Mb, but I don't know how much scattering occurs at run time.
Quote from: WASasquatch on August 02, 2019, 03:38:06 pmAlso what you are saying just confuses me as it would mean AMD64, a breakthrough is really no different than x86? Utilizing some sort of hack that even LAA 32bit processes don't even need to do. I'm just confused.
Quote from: PabloMack on August 02, 2019, 05:12:11 pmWhen the x86-32 became x86-64 (AMD64), the main thing that happened was that it now had a 64-bit address space. That is not a trivial improvement. That is a big improvement. But its addressing modes changed very little. Programs that are linked as one image, still only have a choice between 8- or 32-bit (signed) offsets. That's all. You can't build an image that is so large that references within the image can't be resolved. This is what caps the maximum size of your program to 2 G-Bytes. Also, keep in mind that program space pointed to by the code segment can be in a different 32-bit space from the data space pointed to by the data segment. Initialized data is part of the image. So Code Space plus Data Space together can be up to 4GBytes. Building an image is a pre-load (link time) process. This means that all references within itself must be resolved and there are no lingering references that know anything about anything outside of its 32-bit address space. I don't think this includes DLL's because they don't link until needed (That's what's meant by Dynamic which is the 'D' in DLL). But this does not say that you can't do post-load time (i.e. run time) address calculations to reach more than that. This is RAM and it is uninitialized and it is outside the image. It will still show up as memory usage and this part of memory can become very huge. When a program like TG loads such things as texture maps and geometries, these are not part of its image. But they are part of your process's memory. So when you see program usage in the Task Manager, it doesn't break down usage by image/segment and dynamically allocated memory. It just shows you the total.
Quote from: PabloMack on August 02, 2019, 07:49:01 pmI'm pretty much talking about hardware. There are software systems layered over the hardware for gaming and such that can manage additional memory beyond the 4GByte image. These software systems have their own switches, parameters and metrics to do their own work. A lot of this kind of thing is hidden in their software layers. But the bottom line is, ultimately, software can only do what the hardware can do. What is hidden in the software layers is out of sight. The IBM AS/400 got to be very good at hiding how it did things by this layering upon layers. There is nothing to prevent a program from loading additional executable code to many places in that huge 64-bit address space to make a program that goes far beyond a single image. But they do this at additional cost by making more calls to the OS and doing things like link-loading other files and using "manual" address calculations to get to those other places in memory. This kind of work-around is common in the Microsoft world. Remember memory extenders in DOS? It's very nasty stuff. It seems it's de-ha-veau all over again.
Quote from: WASasquatch on August 02, 2019, 10:35:08 pmHardware is what I am talking about. The hardware switches for 3gb and 4gb allow SINGLE 3gb/4gb address calls.
Quote from: WASasquatch on August 02, 2019, 10:35:08 pmNo where can I find that 64bit processes are limited to 2gb draws, for AMD64 or otherwise. The only limitation is in .NET. like you mentioned PE32+ but it's a extension for Windows CE, a different breed to WinXP or otherwise.
Quote from: WASasquatch on August 02, 2019, 10:35:08 pmAlso, you can explore sub processes of processes to see what is all linked. Expand the process in task manager or view it in process hacker.
Quote from: PabloMack on August 03, 2019, 11:37:57 amQuote from: WASasquatch on August 02, 2019, 10:35:08 pmHardware is what I am talking about. The hardware switches for 3gb and 4gb allow SINGLE 3gb/4gb address calls. The document you showed me was old and pre-AMD64. What you are talking about deals with how a 32-bit operating system can use a 32-bit address space (in a pre-AMD64 system) to map in system and user space. I believe the way both Windows and Linux work is that when the OS is in Supervisor Mode, it can see both user space and system space. This makes it easier to process requests because it can directly see what the user mode sees while simultaneously seeing its own resources. In a system call, the user passes an address to the supervisor in a request for some service to be provided. The user will always pass an address that has meaning in its own user space because it doesn't have direct access to supervisor space. If a user tries to access the space that is mapped for use by the supervisor, then it will cause a fault and the process will be aborted. This is because the MMU will not allow the processor to make the access that is protected for system use only. But when in supervisor mode, the supervisor can make accesses to both its own space and user's space. In the first implementation of NT, 2GBytes of address space were used for mapping in system resources while the other 2GBytes were used to map in user memory. Later on when many user mode programs were needing access to more memory, the partitioning was changed to give more to the user and less for use by the operating system. In order to provide a user mode process more than 2GB of address space, a special setup was used where the system will make-do with only 1GByte for itself, leaving 3GBytes for use for user programs. This is not so much a "hardware switch". It is the way the operating system configures the MMU (which is hardware) for simultaneous use by the system and the user. Quote from: WASasquatch on August 02, 2019, 10:35:08 pmNo where can I find that 64bit processes are limited to 2gb draws, for AMD64 or otherwise. The only limitation is in .NET. like you mentioned PE32+ but it's a extension for Windows CE, a different breed to WinXP or otherwise. Let me also say that "processes" are not hardware but are software entities created and managed by the operating system. What do you mean by a "draw"? Also, I've never used .NET but it has to run on the same hardware as everything else. Same applies to Windows-CE. That is just software. They use the same processors as regular Windows. But it uses the hardware differently by configuring memory management differently. Quote from: WASasquatch on August 02, 2019, 10:35:08 pmAlso, you can explore sub processes of processes to see what is all linked. Expand the process in task manager or view it in process hacker.This may be the case. But still, if different sub-processes are running within the same "process" and together they can grow to be very much larger than 4GB, then in effect, each sub-process has its own "image" so we are talking about multiple images now and multiple 4GB images can reside within the same 64-bit address space at the same time. This is something that Windows-32 or Linux-32 could not do because all they had was a 32-bit address space. But still, an instruction that is executing within one of the images can't directly reach a resource that is located within another image using one of the standard addressing modes that is defined in the Mod/RM byte of the instruction. The offsets are only 8 or 32-bits and there are no 64-bit offsets in any of the addressing modes. But you can load a 64-bit value into a register and then use it as an address but this is not a 64-bit addressing mode. It would be like one guy says "You can't kill anyone with an unloaded gun". Then to prove that he can, he takes the empty gun and hits someone over the head with it and kills him. So you can kill someone with an empty gun. You just can shoot someone with it to kill him. These conversations are all about how you say things and what you mean by saying them. I think in some ways we are defining our words differently. For example, the kind of "linking" you are talking about may be different than what I meant by the word. Sometimes in computer science, the same words are used for very different things. When they talk about "linked-lists" they are not talking about what a "linker" does. These are talking about completely different things. So it is no wonder that we sometimes disagree because we don't realize we are talking about different things in the first place or define our words to mean different things. To me a "hardware switch" is a flip-flop where you write a "1" into it and the way it controls other hardware is changed. To you, it might be like a command-line "switch" using terminology used in command line interpreters that somehow configure things behind the scenes so that the software works differently somehow.
Quote from: WASasquatch on August 03, 2019, 12:36:24 pmCan you provide any concrete evidence of this limitation outside of .NET software?