X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-6.3 required=5.0	tests=AWL,BAYES_00,KHOP_PGP_SIGNED,KHOP_THREADED,SPF_HELO_PASS,T_RP_MATCHES_RCVD
X-Spam-Check-By: sourceware.org
Message-ID: <50005EF2.1040001@dancol.org>
Date: Fri, 13 Jul 2012 10:46:26 -0700
From: Daniel Colascione <dancol AT dancol DOT org>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20120614 Thunderbird/13.0.1
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: clisp crashes on startup
References: <4FF8759A DOT 10400 AT gmail DOT com>	<4FF88907 DOT 5060100 AT dancol DOT org>	<CAHiT=DFvbMBSor+3cGQzpKE19YUHWH2uErkfGxcVOOAOt_J2sQ AT mail DOT gmail DOT com>	<4FFB4C82 DOT 7050702 AT dancol DOT org>	<4FFBB6B3 DOT 2090705 AT dancol DOT org>	<20120710081318 DOT GB3051 AT calimero DOT vinschen DOT de>	<4FFC4D3C DOT 2010004 AT dancol DOT org>	<4FFF9A81 DOT 90403 AT dancol DOT org>	<20120713074009 DOT GC26268 AT calimero DOT vinschen DOT de>	<CAHiT=DHCuOY0BuawAbZCgGw7qwY4hRdF4Ur_LoiZ1w02qMtBjQ AT mail DOT gmail DOT com>	<20120713142626 DOT GO3406 AT calimero DOT vinschen DOT de> <CAHiT=DFpN3RMcH7RkQckcNk0qN9e-iuuZyk9n8d9y4m4agBAgQ AT mail DOT gmail DOT com>
In-Reply-To: <CAHiT=DFpN3RMcH7RkQckcNk0qN9e-iuuZyk9n8d9y4m4agBAgQ@mail.gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig9EE2A530CB321FD28B3C8019"
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

--------------enig9EE2A530CB321FD28B3C8019
Content-Type: multipart/mixed;
 boundary="------------080709010803040207050301"

This is a multi-part message in MIME format.
--------------080709010803040207050301
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 7/13/12 9:30 AM, Reini Urban wrote:
> On Fri, Jul 13, 2012 at 9:26 AM, Corinna Vinschen wrote:
>> On Jul 13 07:52, Reini Urban wrote:
>>> On Fri, Jul 13, 2012 at 2:40 AM, Corinna Vinschen wrote:
>>>> On Jul 12 20:48, Daniel Colascione wrote:
>>>>> On 7/10/12 8:41 AM, Daniel Colascione wrote:
>>>>>> On 7/10/12 1:13 AM, Corinna Vinschen wrote:
>>>>>>> On Jul  9 21:59, Daniel Colascione wrote:
>>>>>>>> On 7/9/12 2:26 PM, Daniel Colascione wrote:
>>>>>>>>> [snip]
>>>>>>>>
>>>>>>>> It turns out that clisp crashes only when I've rebased DLLs into t=
he
>>>>>>>> high portion of the 4GB WOW64 address space.
>>>>>>>
>>>>>>> Where did you rebase them to?  You know that on WOW64 and with the
>>>>>>> bigaddr flag on, the application heap is located at 0x80000000 by
>>>>>>> default, right?  Perhaps some of your DLLs just collide with that?
>>>>>>
>>>>>> I'm using a starting base address of 0xC8000000; I haven't had
>>>>>> problems with any other program.
>>>>>
>>>>> It turns out that clisp uses bit 31 of each pointer for its gc mark
>>>>> bit. No wonder the thing blows in bigaddr-aware mode. clisp _does_
>>>>
>>>> Ouch.
>>>>
>>>>> work, however, when compiled with -DWIDE. In this mode, clisp uses two
>>>>> words for each lisp value --- one for the pointer and one for the
>>>>> metadata.
>>>
>>> Hmm, I do not really want to maintain lisp32.exe and lisp64.exe
>>> variants, but maybe
>>> upstream can be persuaded to make that distinction in the clisp.exe dri=
ver.
>>> It's only for huge data and bad rebase addresses.
>>> And for huge data a self-compiled clisp makes sense to me.
>>> Serious lisp users use better lisp compilers anyway which compile
>>> to native code. (sbcl, lispworks, allegro).
>>> clisp's main strength is fast startup and fast IO.
>>>
>>>>> Also, clisp has a LINUX_NOEXEC_HEAPCODES mode that also
>>>>> works, and without bloating memory use, but that requires that no real
>>>>> virtual address be in the range [0xC0000000, 0xDFFFFFFF].
>>>>
>>>> That can't be guaranteed.  WOW64 provides the full 32 bit VM address
>>>> space.
>>>
>>> Maybe also a documentation issue for rebaseing.
>>
>> That has nothing to do with rebasing.  If you build clisp with a recent
>> gcc, it will have the "bigaddr" flag set in the file header since
>> Cygwin's gcc sets that by default.  While DLLs should be rebased from
>> 0x70000000 downwards as usual, the applications heap, as well as
>> thread stacks, as well as shared memory regions created with mmap(2)
>> or shmat(2) will be in the high memory area starting at 0x80000000.
>>
>> THe bottom line is, if the distro clisp isn't 32 bit clean, which it
>> apparently isn't using default settings, it should either be rebuilt
>> with -DWIDE, or it should have the bigaddr flag removed from the
>> file header by default (peflags -l0).
>=20
> Thanks. This deserves an update now.

Indeed. One caveat is that WIDE seems to be incompatible with clisp
threading. It'd probably be a good idea to send an email upstream and
ask the clisp people whether there's anything they can do. You'll also
need the attached patch, which fixes some macro-name conflicts that
arise when compiling with WIDE.

I'd code up something like LINUX_NOEXEC_HEAPCODES myself, but I'm
nervous about making those changes: lots of places in the code (like
the bytecode interpreter?!) look at LINUX_NOEXEC_HEAPCODES and do
subtly different things if it'd set. In principle, I don't see why we
should have to reserve [0xC0000000, 0xDFFFFFFF]: the bits that force
us out of this range could come from the low bits of the pointer
instead, and the cost would just be a coarser alignment requirement.
I'm happier with requiring coarser alignment than I am with carving
out a portion of the address space.

--------------080709010803040207050301
Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0";
 name="clisp.diff"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="clisp.diff"

diff -r b689a1c1565e modules/syscalls/calls.c
--- a/modules/syscalls/calls.c	Wed Jun 27 11:08:08 2012 -0400
+++ b/modules/syscalls/calls.c	Fri Jul 13 09:42:45 2012 -0800
@@ -4550,7 +4550,7 @@
       }
     } else { /* Probably system just do not support UNICODE */
 #endif
-      gltext =3D GetClipboardData(CF_TEXT); /* ANSI TEXT */
+      HGLOBAL gltext =3D GetClipboardData(CF_TEXT); /* ANSI TEXT */
       if (gltext !=3D NULL) {
         const char * str =3D (const char *)GlobalLock(gltext);
         if (str !=3D NULL) {
diff -r b689a1c1565e src/lispbibl.d
--- a/src/lispbibl.d	Wed Jun 27 11:08:08 2012 -0400
+++ b/src/lispbibl.d	Fri Jul 13 09:42:45 2012 -0800
@@ -13704,7 +13704,7 @@
  < const chart* charptr: pointer to the characters
    (may be in string, may be on the stack) */
 #ifdef HAVE_SMALL_SSTRING
-  #define unpack_sstring_alloca_help_(string,len,offset,charptr_assignment=
,u) \
+  #define unpack_sstring_alloca_help_(string,len,offset,charptr_assignment=
,ign) \
     if (simple_nilarray_p(string)) {                                      =
     \
       if ((len) > 0) error_nilarray_retrieve();                           =
    \
       charptr_assignment NULL;                                            =
     \
@@ -13718,12 +13718,12 @@
         else if (sstring_eltype(TheSstring(string)) =3D=3D Sstringtype_8Bi=
t) \
           copy_8bit_32bit(&TheS8string(string)->data[offset],(cint32*)_unp=
acked_,len);\
         else                                                              =
     \
-          u;                                                            \
+          ign;                                                            \
       }                                                                   =
     \
       charptr_assignment (const chart*) _unpacked_;                       =
     \
     }
 #else
-  #define unpack_sstring_alloca_help_(string,len,offset,charptr_assignment=
,u) \
+  #define unpack_sstring_alloca_help_(string,len,offset,charptr_assignment=
,ign) \
     if (simple_nilarray_p(string)) {                                      =
     \
       if ((len) > 0) error_nilarray_retrieve();                           =
    \
       charptr_assignment NULL;                                            =
     \
@@ -13734,7 +13734,7 @@
 #define unpack_sstring_alloca(string,len,offset,charptr_assignment)     \
   unpack_sstring_alloca_help_(string,len,offset,charptr_assignment,NOTREAC=
HED)
 /* is used by */
-%% export_def(unpack_sstring_alloca_help_(string,len,offset,charptr_assign=
ment,u));
+%% export_def(unpack_sstring_alloca_help_(string,len,offset,charptr_assign=
ment,ign));
 %% puts("#define unpack_sstring_alloca(s,l,o,c) unpack_sstring_alloca_help=
_(s,l,o,c,NOTREACHED)");
=20
 /* UP: Fetches a character from a simple string.

--------------080709010803040207050301--

--------------enig9EE2A530CB321FD28B3C8019
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: GPGTools - http://gpgtools.org

iEYEARECAAYFAlAAXvMACgkQ17c2LVA10Vv1gwCgsNK+xPlSvrV7uZiicKZaizKA
2qwAoMwA3wiB9/p9x/T5OkNLSFV7Exw+
=pR6r
-----END PGP SIGNATURE-----

--------------enig9EE2A530CB321FD28B3C8019--