From: Jim Meyering <jim@meyering.net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P@draigBrady.com>
Cc: Eric Blake <ebb9@byu.net>, cygwin@cygwin.com,
   bug-coreutils <bug-coreutils@gnu.org>
Subject: Re: "du -b --files0-from=-" running out of memory
In-Reply-To: <492C1512.9020706@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=  Brady"'s message of 	"Tue, 25 Nov 2008 15:09:06 +0000")
References: <nacii4p76633jbufvfoj4qjesrph05rjga@4ax.com> 	<49296551.4010801@byu.net> <87bpw5a5tp.fsf@rho.meyering.net> 	<874p1v52od.fsf@rho.meyering.net> <492C1512.9020706@draigBrady.com>
Date: Tue, 25 Nov 2008 17:56:22 +0100
Message-ID: <87myfn3ft5.fsf@rho.meyering.net>
Lines: 60
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm
Precedence: bulk
Sender: cygwin-owner@cygwin.com
Mail-Followup-To: cygwin@cygwin.com
Content-Transfer-Encoding: 8bit

Pádraig Brady <P@draigBrady.com> wrote:
> Jim Meyering wrote:
>> Subject: [PATCH 1/2] argv-iter: new module
>>
>> * gl/lib/argv-iter.h: New file.
>> * gl/lib/argv-iter.c: New file.
>> * gl/modules/argv-iter: New file.
>
> Very useful module!
>
> I see that --files0-from was added to `du` in Mar 2004,
> so it's a nice solution to this 4 year old issue.

Thanks.
I'm surprised it took so long to bite.

> I notice that argv_iter does a malloc() + memcpy() per entry.
> Since the sources are already NUL terminated strings
> perhaps it could just return a pointer to a getdelim
> realloc'd buffer which was referenced in the argv_iterator struct.

The only per-entry allocation I see is:
  - in argv-mode: strdup
  - in stream-reading mode: getdelim

Did I miss something?

char *
argv_iter (struct argv_iterator *ai, enum argv_iter_err *err)
{
  if (ai->fp)
    {
      char *name = NULL;
      size_t buf_len = 0;
      ssize_t len = getdelim (&name, &buf_len, '\0', ai->fp);
      if (len < 0)
        {
          free (name);
          *err = feof (ai->fp) ? AI_ERR_EOF : AI_ERR_READ;
          return NULL;
        }

      *err = AI_ERR_OK;
      ai->item_idx++;
      return name;
    }
  else
    {
      if (*(ai->p) == NULL)
        {
          *err = AI_ERR_EOF;
          return NULL;
        }
      else
        {
          *err = AI_ERR_OK;
          return strdup (*(ai->p++));
        }
    }
}

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/