X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-0.6 required=5.0 	tests=AWL,BAYES_00,MSGID_FROM_MTA_HEADER
X-Spam-Check-By: sourceware.org
Message-Id: <814i7o$49eopi@dmzms99802.na.baesystems.com>
X-SENDER-IP: 10.37.225.66
X-SENDER-REPUTATION: None
X-SENDER-IP: 10.44.64.11
X-SENDER-REPUTATION: None
From: "Cooper, Karl (US SSA)" <karl DOT cooper AT baesystems DOT com>
To: "cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>
Date: Fri, 6 Nov 2009 08:48:10 -0500
Subject: RE: 1.7] BUG - GREP slows to a crawl with large number of matches    on a  single file
In-Reply-To: <4AF42027.80604@towo.net>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

Thomas Wolff wrote:
> Christopher Faylor wrote:
..snip..
>>> aputerguy wrote:
>>>=20

>>>> Running grep on a 20MB file with ~100,000 matches takes an
>>>> incredible almost 8 minutes under Cygwin 1.7 while taking just 0.2
>>>> seconds under Cygwin 1.5 (on a 2nd machine).
..snip..
>> The fact that it behaves differently between Cygwin 1.5 and 1.7 would
>> suggest that this isn't a grep problem.
>>=20
> This is likely to be triggered by the transition to UTF-8 as a
> default charset. The same problem is observed on Linux, with grep as
> well as with sed.=20=20
> That's why I have changed most of my shell scripts to use something
> like LC_ALL=3DC grep or LC_ALL=3DC sed where possible. Please try this.=
=20

I don't have Cygwin 1.5 for comparison, but the testcase provided does show=
 grep using a long time on *my* Cygwin 1.7.
And LC_ALL=3DC didn't seem to help:

$ time grep dog testfile | wc
 100000  900000 4500000

real    3m28.229s
user    3m26.951s
Sys     0m0.170s

$ LC_ALL=3DC time grep dog testfile | wc
207.26user 0.06system 3:28.32elapsed 99%CPU (0avgtext+0avtdata 278784maxres=
ident)k
0inputs+0outputs (1091major+0minor)pagefaults 0swaps
 100000  900000 4500000

$ time LC_ALL=3DC grep dog testfile | wc
 100000  900000 4500000

real    3m24.265s
user    3m24.124s
sys     0m0.202s

(Well . . . Doesn't help very *much*, anyway; a few seconds.)
I don't have the *latest* 1.7:

$ uname -a
CYGWIN_NT-5.1 gldlkcooper 1.7.0(0.214/5/3) 2009-10-03 14:33 i686 Cygwin

Karl Cooper

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple