X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:content-type
	:content-transfer-encoding:message-id:date:from:to:subject
	:in-reply-to:references; q=dns; s=default; b=VG7zhdd1QtlFjTaFPvc
	8snI+NEoaidFFHMTkkgN4ekrnMKCZiuctOy4wUanq67mynrSTkdP17e2OPr9rhot
	qIho1SN3nRWK0owjIXmSJj2fZCguDaOBGbSaY9X2fdtSCxR3yXqJby/UL6W0wSiY
	C9s4+g13fr2KTrBexb44AUfE=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:content-type
	:content-transfer-encoding:message-id:date:from:to:subject
	:in-reply-to:references; s=default; bh=oW26Ajw+GdgYv3pl7ymTvi2zv
	iw=; b=jq/l5xBGRTc45pf8PY5rupUqMdHiP+U9esTbPjTo8LrcKVhQhCgYe4EW0
	eSQfwTJr0xVn8QpnSofVDgefvYAQ2RwJF1bKD6vc4tjct6+dul6JfFR4OAkSXZt/
	+san8t8N/6d+G23eG97PvJ+r9mCVk0Or8zbSJ7QVl2MU3HGJD8=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=-0.3 required=5.0 tests=AWL,BAYES_00,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,RCVD_IN_DNSWL_LOW autolearn=no version=3.3.2
X-HELO:	sneak2.sneakemail.com
MIME-Version:	1.0
Message-ID:	<9116-1424712152-918733@sneakemail.com>
Date:	Mon, 23 Feb 2015 10:22:29 -0700
From:	"John Hein" <3fbmqnhaz4 AT snkmail DOT com>
To:	cygwin AT cygwin DOT com
Subject:	Re: TEST RELEASE: Cygwin 1.7.35-0.3
In-Reply-To:	<20150223111721.GH437@calimero.vinschen.de>
References:	<20150220102927 DOT GR26084 AT calimero DOT vinschen DOT de> <6695-1424465180-282140 AT sneakemail DOT com> <20150223111721 DOT GH437 AT calimero DOT vinschen DOT de>

Corinna Vinschen corinna-cygwin-at-cygwin.com |cygwin_ml_nodigest| wrote at 12:17 +0100 on Feb 23, 2015:
 > On Feb 20 13:46, John Hein wrote:
 > > Corinna Vinschen wrote at 11:29 +0100 on Feb 20, 2015:
 > >  > So I just changed the order of the objectClass and objectCategory test.
 > >  > If that's really the culprit, you can easily test it:
 > >  >
 > >  > Revert to the Cygwin 1.7.34-6   DLL, run `time mkpasswd -d >/dev/null'
 > >  > Revert to the Cygwin 1.7.35-0.3 DLL, run `time mkpasswd -d >/dev/null'
 > >  > Revert to the Cygwin 1.7.34-6   DLL, run `time mkpasswd -d >/dev/null'
 > >  > Revert to the Cygwin 1.7.35-0.3 DLL, run `time mkpasswd -d >/dev/null'
 > >  >
 > >  > Does that clearly show that 1.7.34-6 is faster performing the enumeration?
 > >  > If so, and despite my total puzzlement, the order in the expression seem
 > >  > to have an effect.
 > >
 > >
 > > mkpasswd w/1.7.35-0.3 finally finished - took about 5+ hours and only
 > > returned ~1800 results instead of ~8000.
 > >
 > > mkpasswd w/1.7.33 took about 50 minutes.  This was mostly after peak
 > > business hours, however (not clear how much that matters).
 > >
 > > Trying 1.7.34-6 now.  Looks like it's going to be slow.  It's been
 > > two hours and only about 200 entries have trickled in.
 > >
 > > Slow going here.  Note this last run is under strace (see below), so
 > > not exactly what you suggested.  I'll try without strace and redirecting
 > > to /dev/null to see if it matters after this test finishes (if ever).
 >
 > Come to think of it, it's probably really just slow.  The difference
 > between mkpasswd/mkgroup for domain accounts:
 >
 > 1.7.33:
 >
 >   Calls NetUserEnum/NetGroupEnum,NetLocalGroupEnum with maximum Buffer
 >   size.
 >
 > 1.7.34+:
 >
 >   Calls an LDAP enumerator fetching 100 SIDs per call.
 >   For each SID:
 >     Call LookupAccountSid.
 >     For each User:
 >       Depending on nsswitch.conf, call LDAP to fetch the extended passwd
 >       info (pw_shell, pw_home, pw_gecos).
 >
 > I guess there's some room for improvement.
 >
 > OTOH, keep in mind that you're not suppsoed to call mkpasswd/mkgroup
 > to enumerate your entire organization.  If you're using it at all, then
 > only to create the required entries in /etc/passwd and /etc/group for
 > your local acocunt to work, and then leave everything else to the "db"
 > setting.

Fair enough.  I'll stop stress testing mkpasswd and consider this
closed unless there's something we want to try.

But 1.7.33 seems much faster (if you can call 50 minutes fast) at it
than 1.7.34-6 or 1.7.35-0.3 in this large-ish AD.  Maybe a knob to
specify buffer size and/or some other knobs might help identifying the
slowest parts (and/or some stats).  Just a thought.

I'll add that the 1.7.34-6 'strace mkpasswd -d' that I had started
above finished in 20+ hours and spewed ~3500 of ~8000 entries.

Running 'time mkpasswd -d > /dev/null' under 1.7.35-0.3 is still
running 20 hours later.  No idea, of course, how much progress it has
made.

p.s. Also seems to be some differences running things under strace or
not - seems like strace issues (time stamp oddness for one, strace -p
<pid> followed by ctrl-c of the strace seems to kill the running
proc).  That's a separate thread(s) for another time.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -