Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Message-ID:	<018601c442a4$6c0197d0$4d1f1cac@THEODOLITE>
From:	"Bruce Dobrin" <dobrin AT imageworks DOT com>
To:	<cygwin AT cygwin DOT com>
References:	<HDEFLKBDJBPHMOHGIIPCKECICAAA DOT greno AT verizon DOT net>
Subject:	shell cmds crapping out with large numbers of files
Date:	Tue, 25 May 2004 15:05:41 -0700
MIME-Version:	1.0
X-BigFish:	v
X-IsSubscribed:	yes

{uname -a
CYGWIN_NT-5.1 THEODOLITE 1.5.9(0.112/4/2) 2004-03-18 23:05 i686 unknown
unknown Cygwin
}

I need to process very large numbers ( up to 100,000) of imagefiles.  I
noticed my foreach loops  start crapping out when the number of files grows
near 1500.  It feels like a 32bit memory addressing problem to me,  but I
don't know how to check for that.  I wrote a  foreach loop to generate files
(0 to xxxx) and then list them and it died at 1471

here is an example of the problem:

dobrin AT THEODOLITE:/home/dobrin/longtest> ls flern* | wc
   1471    1471   32726
dobrin AT THEODOLITE:/home/dobrin/longtest> touch flern0001471.plern.poo
dobrin AT THEODOLITE:/home/dobrin/longtest> ls flern* | wc
      2 [main] -tcsh 2396 cmalloc: cmalloc returned NULL
      0       0       0
Segmentation fault (core dumped)
dobrin AT THEODOLITE:/home/dobrin/longtest> rm flern0001471.plern.poo
dobrin AT THEODOLITE:/home/dobrin/longtest> ls flern* | wc
   1471    1471   32726


I Currently am processing the files in batches of 1000 to avoid the problem.
I tried the same thing on my linux box and it works fine.

Thankyou


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

- Raw text -