DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 00C3C3858425
MIME-Version: 1.0
Date: Wed, 19 Apr 2023 13:56:54 +0200
To: L A Walsh <cygwin@tlinx.org>
Cc: cygwin@cygwin.com
Subject: Re: Can not stat file with utf char U+F020
In-Reply-To: <643F3F87.2050403@tlinx.org>
References: <992b3c28d7f1cfc17f7c9bb47b53f770@assyoma.it>
 <ZDmiyYS+m0x4QZmh@calimero.vinschen.de>
 <f4d0bd30-731a-fb5e-43d2-a86d1af761b6@Shaw.ca>
 <dfbdeec869acd6ed1ad7b4ec803336f2@assyoma.it>
 <ZDm20Rug7TswU4KI@calimero.vinschen.de>
 <1274a3199d9bedab4f15d209694c6e1f@assyoma.it>
 <de19dc1ecd891f748e5e4a317f347477@assyoma.it>
 <ZD0L5VoqmTHmHBT9@calimero.vinschen.de>
 <fc6d3aa5f3ae525a777e65d62ccbd6c2@assyoma.it>
 <1a7db5a68644e5b66634d5af9b402caf@assyoma.it> <643F3F87.2050403@tlinx.org>
Message-ID: <9f1593d259faf7f845b96947eaff8619@assyoma.it>
Precedence: list
From: Gionatan Danti via Cygwin <cygwin@cygwin.com>
Reply-To: Gionatan Danti <g.danti@assyoma.it>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie.com@cygwin.com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>

Il 2023-04-19 03:10 L A Walsh ha scritto:
> I'm a bit confused as to what char you are trying to access/use, as
> U+F020 is in the Private Use area (PUA)
> 
> Since it's in the PUA, it seems its meaning could differ by
> application/OS/User, no?
> I.e. have no set definition
> 
> I mean you can use it in Cygwin to represent some character not
> usually permitted in
> a DOS/Win filename (like :/\, etc.), but it wouldn't have the same 
> meaning then
> in Windows though.?  Isn't Private Use area application specific so an
> application can
> create and use its own symbol set -- even though it wouldn't be
> portable to another application.

The issue is with any clients/applications (even cygwin) creating a 
filename ending with a dot (or other chars) which is replaced with 
U+F020. If this file is later renamed adding some other character 
*after* the replaced dot, it become unreadable by cygwin.

Something similar to that:
- an user create a file name "project.", forgetting the extension, on an 
Windows share;
- the client replace the dot with U+F020;
- at this point all is good: the file can be read by the client, Windows 
and cygwin;
- the user notice the missing extension and rename the file in 
"project.txt";
- cygwin now does *not* traslate back U+F020 to dot and it is unable to 
read the file.

> I think characters in the PUA range are used to allow Cygwin filenames
> to contain colon, slashes
> and quotes -- so one wouldn't want Windows to understand the cygwin
> intent or it would defeat
> the purpose of using custom characters to represent filenames that are
> legal under POSIX but not
> under Windows.

True, but dot and spaces are somewhat different from the other reserved 
chars. While backslash, colons, etc. are rejected by NTFS itself (or by 
lower layer API), trailing dot and spaces are ignored/stripped by Win32. 
This means that Linux clients accessing an SMB share *can* successfully 
create such filenames without any issue and without replacing them with 
PUA chars.

For example, I created a file called "zzz." from a Linux+Mate client. 
Cygwin correctly see the filename as:
$ ls "zzz." | od -x --endian=big
0000000 7a7a 7a2e 0a00

True, Windows can not access this file, but this is fine because such a 
filename should never be understood by Windows. Not being able to open 
the file from Windows, its users themselves will find and correct the 
issue, renaming the file.

As things are now, we have the opposite issue: should (for whichever 
reason) a file exist with names as "zzz[U+F020]txt", cygwin will not be 
able to access this file. This means that anyone using cygwin+rsync to 
backup a Windows server will now have an inaccessible and impossible to 
backup file.

Thinking about that: how do you feel having an option to exclude 
trailing dots and spaces from PUA translations (effectively reverting 
them to the status of "normal" characters)?

Regards.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple