Date: Mon, 15 Mar 93 09:31:59 EST From: DJ Delorie To: djgpp AT sun DOT soe DOT clarkson DOT edu Subject: go32 v2.0 I've decided to start thinking about the next go32. There are a number of architectural deficiencies in the 1.0x series that I'd rather resolve with a rewrite (gdb and interrupts, for example). After a few hours of brainstorming, I came up with the attached "notes" file that has a lot of implementation ideas. It's a little rough, but it's only a start, and if you don't understand part of it, don't worry. The basic plan for 2.0 is to move a LOT of functionality from go32 into the application's libraries and runtime loadable modules, so that you rarely need to modify the extender. Comments and suggestions are welcome, as are pleas for explanations of parts of these notes. DJ -- ----- go32 2.00 notes ----- startup: if (proxy) program = proxy(argv[0]) args = proxy(args) else if (self-merged) program = argv[0] args = self else program = argv[1] args = shift(args) setup-pmode-handler setup-exceptions setup-paging start program mapping: support full mmap() functionality, plus add extension for switching map that sbrk() and brk() use. Allow for mapping physical memory to virtual space for devices. Memory map (application virtual offset 0x10000000 from physical): 00000000-00000fff reserved for zero address detection 00001000-7fffffff primary application (2G) 80000000-bfffffff secondary applications (emu, gdb, etc) (1G) c0000000-cfffffff system shared libraries (256M) d0000000-efffffff user shared libraries and mappings (512M) f0000000-f0ffffff rmode map, extender code, map for 0-16M (16M) f1000000-ffffffff physical bus mappings for 16M-4G (240M) Allow for custom page fault handlers, called as: (*pf)(address, reason, address_base, custom_data) ALL page faults are handled through these types of functions, but the primary ones are in go32 itself. For each nibble of address, there is a 16-way tree node. Each node has one bitmask word and 16 pointers. If the bit is set, it's a pointer to a handler structure. If not, it's a pointer to another table for finer control. Handler structures contain the starting and ending addresses, address of the handler function, and the custom data. Resolution of page tables restricts the tree to a maximum depth of 5 levels. Handlers are installed as high in the tree as possible without conflicting with other handlers, and are moved down as new handlers are added. Go32 contains the handlers for file mapping and physical to virtual bus mappings. shared libraries: manually (automatically?) built list of functions used to provide table linked at fixed, known, address at beginning of libc-XXX.so. Table is list of JMP instructions to actual routines. Slot entries cannot change once a symbol is placed there unless the version number of the library is changed also. link all objects into large a.out file, with table's .o FIRST so that it's always in the same place. Read out address of beginning of table and use to build shared archive. From list, generate .s/.o library libc-XXX.sa with only ABS symbols that refer to the locations of the jump table entries: bar = __need_libc_so_hook .globl bar bar = 0xd0000000 Any object module with a static symbol called "__noshare" will NOT have its symbols included in the shared library, but will instead be transferred to the shared archive intact. This allows for mapping the shared library through the hook functions that the stubs will reference: stubs reference __need_libc_so_hook __need_libc_so_hook has "__noshare", so it gets linked with the app directly. The hook has static initializers that map the shared library and call its __libc_so_main function The main function calls all the initializers and registers with atexit() to call uninitializers. The main module also has an identifier __libc_so_id so that if the mmap fails, the application can determine if the mapped file is the correct shared library. If the user does not link with the shared library, these support routines are never referenced and thus do not affect the use of the library. For the purposes of development, a copy of the latest libc-XXX.sa will also be stored as libc.sa, so that gcc can reference it without knowing the version number. 80387 Emulator: This will be handled as a special shared library. crt0.o will call a function called __emu387_install_hook. If the application links the emu387 shared library, this hook pulls it in, and it's libmain() attaches to the FPU not-present exception vector. If a real 80387 exists, the emulator will attach to a stub instead that allows the 80387 to operate. While testing, the emulator always attaches itself, and may use alternate methods of determining when to attach in production mode. There will be a stub in libc.a that reports a SIGFPE exception if any fpu opcodes are called, unless a real 80387 is present, in which case it is enabled. To allow 80387 emulators to be swapped out, all conforming emulators will list __emu387_install_hook as the first function in there table. No other functions are required or used. The non-shared emulator library may be linked into standalone applications to provide a simpler customer solution. signals/interrupts: * Hook pmode only, for exceptions * Chain from rmode, for timer and other chains * Hook both, for private interrupts. All of go32's exceptions are handled in protected mode to allow for more flexible paged memory usage. debugger: Each process in virtual space (primary and secondary applications) has a structure that defines it's files, signals, register state (TSS), etc. Ability to run another process until "something" happens (like wait) to support debugger. The debugger (gdb?) will take advantage of the run-until-something call in go32 and the mmap functions to control the debugged program. To prevent conflicts, the debugger should use NO shared libraries. Extender services: All current DOS services will be remapped through a universal interface. The application will pre-fill the transfer buffer in the rmode map and ask the extender to execute it. Registers will be passed and returned through a static buffer in rmode space. Addresses of buffers will be found in info table. A majority of the hooks provided in go32 1.09 will be delegated to the application itself through self-registering modules in the various libraries. Info table: This is a table of information that go32 provides to the application to assist it in communicating with the outside world. The table contents are listed below. Entries may be added, but never removed unless the lowest supported version number is reset. Version numbers are stored in decimal as MMmmUU: MM=major version, mm=minor version, uu=reserved for user-designated modifications. Example: go32 v2.0 would report 20000. The next official version would be 20100. If a user adds a personal feature, that would be 20001. All table entries are 4 bytes. All addresses are linear virtual addresses suitable for the application(s) to use directly. The first three entries MUST be present as specified in ALL versions of go32, present and future. size of table (bytes) version number of extender lowest compatible version number address of syscall function address of transfer buffer size of transfer buffer address of register structure (used with transfer buffer) address of per-process structure physical memory available to application (approximate) swap space available to application (approximate) pointer to argv[0] pointer to DOS command line buffer pointer to DOS initial environment The linear address of this table shall be stored in a fixed GDT entry so that any application (primary or secondary) may locate it. Each application knows what the lowest version number is that it will allow. Between this and the compatibility range provided by the structure above, incompatibilities can be detected. System calls: Execute rmode code, registers passed through register structure Add/remove file/memory mappings and custom page fault handlers. Hook/chain interrupts & signals. Forced partial or total relinquish of system resources (vectors, memory). This is used before running a child process. Handled through generic resource limits with adaptive followers. Can be set to -1 for unlimited in most cases. Execute another per-process structure until it stops for "something". The stub loader: The stub loader will attempt to locate a specific version of go32 (go32vXXX.exe)