DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:subject:references:from:to:message-id:date
	:mime-version:in-reply-to:content-type
	:content-transfer-encoding; q=dns; s=default; b=Ds1oqRcN95DngTE7
	STO8nOF5OpG24qpyD41W+Ul15oBH56hR0svYTMQVQZcqXaEYMsdx+1velB7e/w3s
	bGISc51SL8BezuKhhxLLOLNJezypMDu/022mEpASA/+BtUGpPFrzDacBJ/+TMytf
	qeZ9/B0TAS1RNBEYmjKu0KcfQS8=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Subject: Re: textmode for stdout, what is "correct" now?
References: <739ed5ce-6902-d702-e152-65dc2c1da667 AT ssi-schaefer DOT com> <20190214162002 DOT GA4950 AT calimero DOT vinschen DOT de>
From: Michael Haubenwallner <michael DOT haubenwallner AT ssi-schaefer DOT com>
To: cygwin AT cygwin DOT com
Message-ID: <6aa280c2-4769-0772-91d9-c73a3a3d9680@ssi-schaefer.com>
Date: Fri, 15 Feb 2019 08:56:59 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0
MIME-Version: 1.0
In-Reply-To: <20190214162002.GA4950@calimero.vinschen.de>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

On 2/14/19 5:20 PM, Corinna Vinschen wrote:
> On Feb 14 16:23, Michael Haubenwallner wrote:
>> Hi,
>>
>> so I find myself struggling with textmode versus binmode for stdio again.
>>
>> Running the openssl command (from within the apps/ build directory here) does
>> yield different results regarding carriage return depending on the version:
>>
>> $ ./apps/openssl version
>> OpenSSL 1.0.2p  14 Aug 2018
>> $ ./apps/openssl x509 -hash -noout -in /etc/pki/tls/cert.pem | xxd
>> 00000000: 6139 3464 3039 6535 0a                   a94d09e5.
>>
>>
>> $ ./apps/openssl version
>> OpenSSL 1.1.0j  20 Nov 2018
>> $ ./apps/openssl x509 -hash -noout -in /etc/pki/tls/cert.pem | xxd
>> 00000000: 6139 3464 3039 6535 0d0a                 a94d09e5..
>>
>> Some subsequent shell script does create wrong symlink filenames
>> (with embedded CR) when used with openssl-1.1.x.
>>
>> The commit that changed this behaviour in openssl-1.1 is:
>> https://github.com/openssl/openssl/commit/bdd58d98467e9f0f6635c1628e1eae304383afb1
>>
>> >From an openssl developer's point of view, I can understand to set
>> textmode when the intent is to output some text, and to set
>> binmode when the intent is to output some binary data.
> 
> How do you create \r\n in this case?  The upstream patch never
> adds the explicit 't' flag.  It only adds 'b' or nothing.  So
> the output should be \n only unless you write to a file on a
> text mode mount.  What am I missing?

Down the line in their BIO module they do use setmode(fd, O_TEXT),
which is the one that does introduce the \r, as far as I know.

The backtrace in openssl-1.1.1a in this use case is:

https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/crypto/bio/bss_file.c#L257 in file_ctrl()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/crypto/bio/bio_lib.c#L528 in BIO_ctrl()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/crypto/bio/bss_file.c#L104 in BIO_new_fp()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/apps.c#L2471 in dup_bio_out()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/apps.c#L2624 in bio_open_default_()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/apps.c#L2652 in bio_open_default()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/x509.c#L595 in x509_main()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/openssl.c#L564 in do_cmd()
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/apps/openssl.c#L183 in main()

> 
>> Question now is: These days, what is the correct way to handle this?
>>
>> Is it up to openssl to not use textmode at all with Cygwin (what if
>> used from within some cmd.exe?), or is it up to the shell script to
>> explicitly drop any carriage return here?
> 
> They way openssl does it looks right.  Off the top of my head I don't 
> grok where you get the \r from in the example above.

So a "correct" and portable way in shell scripts really is something like this?
$ var=`openssl ... | tr -d "\\r"`
or even (with bash or similar):
$ var=$(openssl ...); var=${var%$'\r'}

Thanks!
/haubi/

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple