Date: Tue, 9 Sep 1997 17:30:30 +0300 (IDT)
From: Eli Zaretskii <eliz AT is DOT elta DOT co DOT il>
To: DJ Delorie <dj AT delorie DOT com>
cc: djgpp-workers AT delorie DOT com
Subject: Open handles in nested programs
Message-ID: <Pine.SUN.3.91.970909171824.7828A-100000@is>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk

I still didn't have time to download the alpha and look inside, so I 
might be storming an open door here.  I just wanted to make sure this 
nasty problem gets corrected in v2.02.

The problem is that, for deeply nested DJGPP programs, if the parent(s) 
use up more than 20 handles, there's no handles left for the stub to open 
the .exe file, and spawn will fail.  (The libc functions call a DOS 
function to enlarge the maximum number of handles, but only the first 20 
are passed to the children, and when DOS creates the child, it gets the 
default handle table in the PSP which only has 20 places.)

One way to correct this is to add a call to the DOS function that 
enlarges the handle table to the stub, before it tries to open the .exe.  
A good place to do that would be after the stub resizes the PSP block 
(which will serve as the transfer buffer), because enlarging the table 
before that will cause DOS to allocate memory, which could interfere with 
the resizing.  Thus, every DJGPP program will have the maximum possible 
number of handles right from startup (and use about 1K more low memory), 
so some code from libc that handles this could be removed.  Assuming that 
we have enough free space in the stub to add this call, this would be the 
best solution, I think.

Another alternative would be to let libc allocate handles without the 
inherit bit, or to have it only set the inherit bit for the first 16 or 
so handles.  I doubt that any program relies on inheriting a large number 
of handles from its parent, so this should not break anything, although 
such a risk exists.

Comments?