To: djgpp AT delorie DOT com
From: "Stas Sergeev (stsp2 AT yandex DOT ru) [via djgpp AT delorie DOT com]" <djgpp AT delorie DOT com>
Subject: [PATCH] exec: fix inversions in leak detection logic
Message-ID: <964e3268-2f75-ee73-ab5a-b01bf1aadb98@yandex.ru>
Date: Sat, 17 Aug 2019 19:18:10 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.6.1
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="------------B15C3468A7EF12AF9F09EAA7"
Content-Language: en-MW
Reply-To: djgpp AT delorie DOT com

This is a multi-part message in MIME format.
--------------B15C3468A7EF12AF9F09EAA7
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Hello.

The attached patch fixes the inversions in a
leak-detection logic. That allows (with some
pre-conditions, see the full description below)
to spawn the prot-mode TSR apps like 32rtm,
which would be otherwise wiped out of memory
mistakenly treated as a leak.


--------------B15C3468A7EF12AF9F09EAA7
Content-Type: text/x-patch;
 name="0001-exec-fix-inversions-in-leak-detection-logic.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="0001-exec-fix-inversions-in-leak-detection-logic.patch"

From d4eb07adc0259622c434c31b40c6a07fd16f355e Mon Sep 17 00:00:00 2001
From: Stas Sergeev <stsp AT users DOT sourceforge DOT net>
Date: Sat, 17 Aug 2019 19:00:10 +0300
Subject: [PATCH] exec: fix inversions in leak detection logic

For some reason the conditions in the leak detection logic
were all inverted. They tested lar before and after descriptor
free, and assumed a leak if the lar byte changed. This is obviously
wrong, as "changed" means that __dpmi_free_ldt_descriptor() actually
worked and set the NP bit.
Then it proceeded to __dpmi_get_descriptor() and assumed a leak
if it returned an error. But an error means that there is no leak
and the descriptor is actually freed!

Without this patch it is not possible to spawn a prot-mode TSR
program like 32rtm. djgpp treats it as a leak and wipes out of
memory. With this patch things work properly if the DPMI server
is smart enough to direct the control to prev client after 32rtm
TSRed. The problem is that 32rtm just jumps to the realmode exit
addr, so the DPMI server doesn't see the exit and may get confused
unless the special logic is implemented for that case (i.e. not all
DPMI servers treat that correctly even after that patch).
---
 src/libc/dos/process/dosexec.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/libc/dos/process/dosexec.c b/src/libc/dos/process/dosexec.c
index 87d7fbd9..1dc9a347 100644
--- a/src/libc/dos/process/dosexec.c
+++ b/src/libc/dos/process/dosexec.c
@@ -514,16 +514,16 @@ static int direct_exec_tail (const char *program, const char *args,
       __dpmi_free_ldt_descriptor(sel1);
       flags = __dpmi_get_descriptor_access_rights(sel1);	/* freed */
 
-      if (larbyte != (flags & 0xf0))
-        workaround_descriptor_leaks = 1;	/* present+ring+sys changed */
+      if (larbyte == (flags & 0xf0))
+        workaround_descriptor_leaks = 1;	/* present+ring+sys not changed */
       else
       {				/* Win NT/2K/XP lie about lar */
         larbyte = desc_buf1[5] & 0xf0;
         ret = __dpmi_get_descriptor(sel1, &desc_buf2);
         if (ret == -1 || (larbyte != (desc_buf2[5] & 0xf0)))
-          workaround_descriptor_leaks = 2;
-        else
           workaround_descriptor_leaks = 0;	/* Don't do anything */
+        else
+          workaround_descriptor_leaks = 2;
       }
     }
     first_call = 0;
-- 
2.20.1


--------------B15C3468A7EF12AF9F09EAA7--