X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-2.0 required=5.0 	tests=AWL,BAYES_00,SPF_SOFTFAIL
X-Spam-Check-By: sourceware.org
Message-ID: <4AF8E8B3.5070003@byu.net>
Date: Mon, 09 Nov 2009 21:14:43 -0700
From: Eric Blake <ebb9 AT byu DOT net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: Broken autoconf mmap test (was Re: 1.7] BUG - GREP slows to a  crawl with large number of matches on a single file)
References: <20091108141548 DOT GB26344 AT calimero DOT vinschen DOT de>	<4AF716DB DOT 8060904 AT cwilson DOT fastmail DOT fm>	<20091109115903 DOT GE26344 AT calimero DOT vinschen DOT de>	<4AF80FFB DOT 4040701 AT byu DOT net> <20091109140529 DOT GJ26344 AT calimero DOT vinschen DOT de>
In-Reply-To: <20091109140529.GJ26344@calimero.vinschen.de>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Corinna Vinschen on 11/9/2009 7:05 AM:
> This part of the testcase
> 
>   data2 = (char *) malloc (2 * pagesize);
>   if (!data2)
>     return 1;
>   data2 += (pagesize - ((long int) data2 & (pagesize - 1))) & (pagesize - 1);
>   if (data2 != mmap (data2, pagesize, PROT_READ | PROT_WRITE,
>                        MAP_PRIVATE | MAP_FIXED, fd, 0L))
>     return 1;
> 
> is bad.  The chance that the address of data2 is not usable for mmap on
> Windows/Cygwin is 100%.

But in testing this further, I discovered that you CAN do:

data2 = mmap(...);
munmap (data2,...);
mmap (data2, ... MAP_FIXED)

and get success on cygwin.  So I will be updating autoconf accordingly,
based on the STD below.  Unfortunately, it looks like I also found a hole
in cygwin.  Consider this (borrowing heavily from the autoconf test that I
am fixing):

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>

int
main (int argc, char **argv)
{
  char *data, *data2, *data3;
  int i, pagesize;
  int fd, fd2;

  pagesize = getpagesize ();
  /* First, make a file with some known garbage in it. */
  data = (char *) malloc (pagesize);
  if (!data)
    return 1;
  for (i = 0; i < pagesize; ++i)
    *(data + i) = rand ();
  umask (0);
  fd = creat ("conftest.mmap", 0600);
  if (fd < 0)
    return 2;
  if (write (fd, data, pagesize) != pagesize)
    return 3;
  close (fd);

  /* Next, check that a page is zero-filled if not backed by a file.  */
  fd2 = open ("conftest.txt", O_RDWR | O_CREAT | O_TRUNC, 0600);
  if (fd2 < 0)
    return 11;
  data2 = "";
  if (write (fd2, data2, 1) != 1)
    return 12;
  else
    /* We expect mmap to succeed, but reads to give SIGBUS, since mapped
       region is an entire page beyond bounds of mapped file.  */
    ;
  data2 = mmap (0, pagesize, PROT_READ | PROT_WRITE, MAP_SHARED, fd2, 0L);
  if (data2 == MAP_FAILED)
    return 14;
  printf ("mapped %p\n", data2);
  for (i = 0; i < pagesize; ++i)
    if (*(data2 + i))
      {
        printf ("%p, %x\n", data2 + i, *(data2 + i));
        return 15;
      }
  close (fd2);
  if (argc > 1)
    munmap (data2, pagesize);

  /* Next, try to mmap the file at a fixed address which already has
     something else allocated at it.  If we can, also make sure that
     we see the same garbage.  */
  fd = open ("conftest.mmap", O_RDWR);
  if (fd < 0)
    return 4;
  if (data2 != mmap (data2, pagesize, PROT_READ | PROT_WRITE,
                   MAP_PRIVATE | MAP_FIXED, fd, 0L))
    return 6;
  for (i = 0; i < pagesize; ++i)
    if (*(data + i) != *(data2 + i))
      {
        printf ("%p, exp %x, got %x\n", data2 + i, *(data + i), *(data2 + i));
        return 7;
      }

  /* Finally, make sure that changes to the mapped area do not
     percolate back to the file as seen by read().  (This is a bug on
     some variants of i386 svr4.0.)  */
  for (i = 0; i < pagesize; ++i)
    *(data2 + i) = *(data2 + i) + 1;
  data3 = (char *) malloc (pagesize);
  if (!data3)
    return 8;
  if (read (fd, data3, pagesize) != pagesize)
    return 9;
  for (i = 0; i < pagesize; ++i)
    if (*(data + i) != *(data3 + i))
      return 10;
  close (fd);
  return 0;
}

This test behaves differently on Linux than on cygwin; on Linux, both
'./foo' and './foo 1' give status 0, but on cygwin, './foo' gives status
6, and only './foo 1' succeeds.  In other words, the second mmap fails if
there is no intermediate munmap.

POSIX apparently allows cygwin's behavior:

"If MAP_FIXED is set, mmap() may return MAP_FAILED and set errno to
[EINVAL]. If a MAP_FIXED request is successful, the mapping established by
mmap() replaces any previous mappings for the pages in the range
[pa,pa+len) of the process."

However, since we already have to maintain a list of mappings in order to
implement fork(), it seems like it would be easy to fix cygwin to
implicitly munmap anything that would otherwise be in the way of a
subsequent MAP_FIXED request, rather than blindly calling
NtMapViewOfSection and failing because of the overlap, so that we could be
even more like Linux behavior.

> That's why I think we need at least two tests in autoconf, a generic
> mmap test and a mmap test for the "mmap private/shared fixed at
> somewhere already mapped" case, if an application actually insists on
> using that.

In the case of the autoconf test, I think a single test is still
sufficient, once it is fixed to be portable to what POSIX requires.

gnulib provides a more interesting test, for whether MMAP_ANON works.
http://git.savannah.gnu.org/cgit/gnulib.git/tree/m4/mmap-anon.m4

- --
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9 AT byu DOT net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkr46LMACgkQ84KuGfSFAYCBrwCgsu2/rWozZs/1R33RaAlUwHow
aLQAoNVjQ8P9it7nkDv8u2RRF4l0uDur
=D/jK
-----END PGP SIGNATURE-----

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple