DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:message-id:date:from:mime-version:to:subject
	:references:in-reply-to:content-type:content-transfer-encoding;
	 q=dns; s=default; b=XHIe0p2RcCf5YrtoJWPhpZmWe7YLWh6IdCmJQISkYIq
	2lSLLjUqgJeW8/O5gR+o58Z2ITLpM4Ubq4YzH9eH+z0VNknacM9X9q6SV4KNtjMe
	pORLzoqvy9+0oAquwBhUKIG/ek96V1gaiu6tlEZSh4bUD1P3HuTsn3fDTLkWUIaI
	=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Message-ID: <5179735A.5080905@cs.utoronto.ca>
Date: Thu, 25 Apr 2013 14:18:02 -0400
From: Ryan Johnson <ryan DOT johnson AT cs DOT utoronto DOT ca>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: mmap(MAP_FIXED) vs mprotect
References: <5177CA05 DOT 3010500 AT cs DOT utoronto DOT ca> <20130424125018 DOT GA21193 AT calimero DOT vinschen DOT de>
In-Reply-To: <20130424125018.GA21193@calimero.vinschen.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 24/04/2013 8:50 AM, Corinna Vinschen wrote:
> On Apr 24 08:03, Ryan Johnson wrote:
>> Hi all,
>>
>> I'm trying to port a linux program that uses mmap to implement a
>> growable array; the ideas is to mmap(PROT_NONE, MAP_NORESERVE) a
>> chunk of address space (corresponding to the maximum array size) and
>> then call mmap(PROT_READ|PROT_WRITE, MAP_FIXED) to allocate actual
>> memory in the "blank" region. This works well in Linux but fails
>> with EINVAL in cygwin.
[snip]
>> Oddly, trying to map in blank pages in with mprotect succeeds on
>> cygwin but fails with ENOMEM on linux...
>>
>> Am I missing something here, or is this just a place where different
>> behavior between the two platforms is a fact of life? Which version
>> is the posixly "correct" way to reserve a chunk of address space and
>> later back it with actual memory?
> There is no POSIXly correct way to do that since MAP_NORESERVE is a
> non-POSIX extension in Linux as well as in Cygwin.
>
> The general idea of MAP_NORESERVE is to make sure that we get as much
> memory as requested, but to use only as much memory as is required.
> On Linux MAP_NORESERVE only performs bookkeeping but doesn't change
> the state of the memory, so a later mmap works.
Actually, MAP_NORESERVE is just to avoid getting OOM on machine with 
strict accounting if multiple 16GB spaces get allocated in this way; 
calling mmap with MAP_FIXED succeeds as long as the requested address 
range has already been allocated to the process in some way. In fact, 
the linux machine I tested on let me MAP_FIXED a 2MB region starting 
from inside a 16kB mapping, with the unfortunate side effect of 
clobbering the dynamic loader's state (which happened to be mapped 
nearby).  That's a ridiculous amount of rope to hang yourself with, and 
I'm not confident it's possible, let alone desirable, to implement it in 
Cygwin (Windows refuses to unmap statically linked .dlls, for example). 
Still, the ability to clobber anonymous memory regions with some other 
mapping would be useful at times (e.g. to "concatenate" two files in 
memory by mapping them side by side). To be safe, we could require that 
region to be clobbered be PROT_NONE (which erases any data that may have 
been there previously, thus guaranteeing that we don't lose anything 
important by changing the mapping).
> Having said that, we *could* also change mmap to handle this scenario
> gracefully as well.  http://cygwin.com/acronyms/#PTC.
I guess hacking it in would be much less work than trying to improve 
fork() was, but realistically I'm not going to have time to do it any 
time soon (rats).

Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple