DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 8BE8B3858002
Subject: Re: CRITICAL ls MEMORY LEAK
To: cygwin@cygwin.com
References: <003401d70864$cd3b3400$67b19c00$@gmail.com>
 <CAJ1FpuMbx1pyOqoRSwgieQbGPgcXOOiUusFtsmqKWoyJdzF0pg@mail.gmail.com>
From: Brian Inglis <Brian.Inglis@SystematicSw.ab.ca>
Organization: Systematic Software
Message-ID: <306dd40d-666d-4a27-0a2c-dc03053d2f8c@SystematicSw.ab.ca>
Date: Mon, 22 Feb 2021 13:30:01 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <CAJ1FpuMbx1pyOqoRSwgieQbGPgcXOOiUusFtsmqKWoyJdzF0pg@mail.gmail.com>
Content-Language: en-CA
Precedence: list
Reply-To: cygwin@cygwin.com
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces@cygwin.com
Sender: "Cygwin" <cygwin-bounces@cygwin.com>

On 2021-02-22 13:12, Doug Henderson wrote:
> On Sun, 21 Feb 2021 at 08:21, Satalink wrote:
>> I deal with a lot of very large files on a regular basis.  I've noticed that
>> when I delve into these directories using in mintty and issue the command ls
>> -l (or ls -color=auto),  a very large junk of memory is consumed.   The
>> memory leak seems to be proportionate to the number and size of files within
>> the containing folder.

> This is likely due to your virus scanner. If those files contain
> non-executable content, it is probably safe to disable virus scans for
> those files.
> 
> Something that ls does is triggering the scan. That scan causes the
> virus scanner to read the entire file. You should see extraordinary
> GPU and disk activity for some time after the ls completes. There
> might be processes or at least threads for each file being scanned.
> 
> Hopefully you will be able to identify a common folder in the path to
> those files where it is safe to disable scanning for that folder and
> all folders and files within that folder.

I've often wondered if the heavy activity is due to Windows' defaults to writing 
files with F+RX perms which triggers executable virus scans?

You can't have separate directory and file default perms and unlike Unix, 
Windows appears dumb about applying the X bit to files, probably because that 
would render downloaded executables non-X, and there is no easy way for users to 
change that, whereas Unix requires tools and users explicitly grant X perms.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple