DMARC-Filter:	OpenDMARC Filter v1.4.2 delorie.com 51JN8ukZ4177824
Authentication-Results:	delorie.com; dmarc=pass (p=none dis=none) header.from=cygwin.com
Authentication-Results:	delorie.com; spf=pass smtp.mailfrom=cygwin.com
DKIM-Filter:	OpenDKIM Filter v2.11.0 delorie.com 51JN8ukZ4177824
Authentication-Results:	delorie.com;
	dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=AF90J/RR
X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org A7D223858405
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1740006534;
	bh=zfiUsAqvepUGGFtJRoJ1BMV1ikfy39iQ374yWuT4EWM=;
	h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	From;
	b=AF90J/RRJ8fqX+DFmvPgAsfRtyYls866Fkp3QvfH8KAiIJbLUir2peD12VjqMycPU
	SE/Ntq/MMgH0q7Au2lW2ktXYX2Lkz7ZOUyvQs7AOKpYCYqqA0HxY2YRLPyaF/8S6f1
	Zg6c2i1D8xMQEGbMX/oy+I0eDrzmgfo07gWT+dEc=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.2 sourceware.org C39C13858C42
ARC-Filter:	OpenARC Filter v1.0.0 sourceware.org C39C13858C42
ARC-Seal:	i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1740006498; cv=pass;
	b=L4lPaWzGYYdECtXe+XBQ/jr3lCEccAsf6wFdltm/fzCv7et3BdX6TuaHHAYxwLEh7VPEjIHGasKCd96iYViwAWLqRBbh1fsfizqr5SSk8NDahuEr/V9jbGodZTOG52yby7nbHo6YxL3K+YmGylbBuvORW/sTUhH/LRM165DKOEA=
ARC-Message-Signature:	i=2; a=rsa-sha256; d=sourceware.org; s=key;
	t=1740006498; c=relaxed/simple;
	bh=wfyhAUsljuBzQLxrd7T0Ms3OTvj7fXdd6R9rognG63w=;
	h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To;
	b=owXOdodBRggQpU1s4RITOXx3FEW/A+2tgK4duv6xwJ9EKBTjEmiIWa3+sQoJbMfboXQ/3fqoO3Q6DAWL2S2LXwGs579XomgINK6bG/cC+9CwJogFn31WGOgALJdJsiftoyLbXePkQXrmDuzRAj4ja3LIRNBkWvcHpDclz0gHhKw=
ARC-Authentication-Results:	i=2; server2.sourceware.org
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org C39C13858C42
X-Sender-Id:	dreamhost|x-authsender|gisburn AT nrubsig DOT org
ARC-Seal:	i=1; s=arc-2022; d=mailchannels.net; t=1740006496; a=rsa-sha256;
	cv=none;
	b=hwUJrOIkzfOjJBQ1wIF00hVGlvz5DTpS4U9LAMEuzy5RuIgIh4MFptTNXuS9/oQTI2HXjE
	qNaidp5HgjFY+EfYyqem6QfJPqlof96OR3k7dIPg8mmT3E6ZTu+fNOtubjMvW2FRGgJk2f
	b2gQfhNQZwZQ2fpNf5xTC/ckXpOHDXTxgNuhZgt+m0ir3ds7sfsg2/gfsr4PBXgLyeY8/a
	vq9kqVU4YH+8kACjTV6oDvqiyC5Is7ns0ewenyBT1XukQ9nuH+IhWeKPeCOxmHdkEv1phg
	hJ/sP9RHbOIOEe0BQg4pU6ZJtPsXt4KmrfqkEgYXxq3+YpsVAI4DU2XU9iL7lA==
ARC-Message-Signature:	i=1; a=rsa-sha256; c=relaxed/relaxed;
	d=mailchannels.net; s=arc-2022; t=1740006496;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	to:to:cc:mime-version:mime-version:content-type:content-type:
	content-transfer-encoding:content-transfer-encoding:
	in-reply-to:in-reply-to:references:references:dkim-signature;
	bh=wf1dMDP8etwlODFWSGWuL+lgl2vZ6PJUl49sMun8UVk=;
	b=1fGMggJxffls3rHPwRfjhJXMNq+P9EQXhZfZ8eZ/zIiPkvSP/kc/DBZCCS2TgNGfAj8fm6
	KS8ZQN4y+fDty2kYtZFWBoLatkce++3srANWPyhg6cp0ab1qXXOt9SSeWfGpiS3CQbbPS8
	zeMvSSGj1moRxLQytWU4D2Tm6Dww2LGtae5Ddji6AKeadA+tpSIRZVh4qWkPVmw8ucMuSw
	4c3o4tJEknIlTvr8Px4fwb4lwp4Krga7jglQ57af/ohdKpjbLhmyD3mprR7gqEBRm8Odhn
	2pj1C6hil4k2tGwWP6bR3OgDUKUVmc0vgT7bABXlIBYTZTUkvZwnnDNG8I2ikA==
ARC-Authentication-Results:	i=1; rspamd-78ddd997cc-qlmnw;
	auth=pass smtp.auth=dreamhost smtp.mailfrom=roland DOT mainz AT nrubsig DOT org
X-Sender-Id:	dreamhost|x-authsender|gisburn AT nrubsig DOT org
X-MC-Relay:	Neutral
X-MailChannels-SenderId:	dreamhost|x-authsender|gisburn AT nrubsig DOT org
X-MailChannels-Auth-Id:	dreamhost
X-Decisive-Relation:	638acdb723ab992d_1740006496479_3232344509
X-MC-Loop-Signature:	1740006496479:3462027436
X-MC-Ingress-Time:	1740006496478
X-Gm-Message-State:	AOJu0Yz1kyABZL4yKUzddr7hy2L0q9K4+gAJ9r7kqnIAzCS8TjyVpQaK
	09QQIQVJzkZoTyS9UtCqXxVybweOrG+OtKLlrby5J4xBN9Y8mnIBhlBVNXVvesjw510JdJxhjEy
	YzCG9sc6bnaVGCIL2yiiW8tOdQxo=
X-Google-Smtp-Source:	AGHT+IF0jjp1oNB7/dZIKYaiGuA1qkaHWISLQP0wbqV2LoVupaVnpB09NNAUEzOu92+uNKe37b/cqioPLa/T349NTKE=
X-Received:	by 2002:a05:600c:458f:b0:439:8a8c:d3e1 with SMTP id
	5b1f17b1804b1-4398a8cd6b3mr103813465e9.22.1740006494863; Wed, 19 Feb 2025
	15:08:14 -0800 (PST)
MIME-Version:	1.0
References:	<CALXu0Ue6z-QgfG3o7waBQ=n17xhAQAAH_Lx+0tyXGsKzhLMmtg AT mail DOT gmail DOT com>
	<20250219162341 DOT 6c907a00b9bd7b0ae6f78c93 AT nifty DOT ne DOT jp>
In-Reply-To:	<20250219162341.6c907a00b9bd7b0ae6f78c93@nifty.ne.jp>
Date:	Thu, 20 Feb 2025 00:07:48 +0100
X-Gmail-Original-Message-ID:	<CAKAoaQ=1QexEh_gzyRPpKsdJ4T4qz3BVM5YgXfnefC2YVcaUOA AT mail DOT gmail DOT com>
X-Gm-Features:	AWEUYZlC24yXVvXHF-ZoCuOe3GEjQBqi2oughPv76qfh0PRRZpwYkpJCB6POYOU
Message-ID:	<CAKAoaQ=1QexEh_gzyRPpKsdJ4T4qz3BVM5YgXfnefC2YVcaUOA@mail.gmail.com>
Subject:	Pipes/fifos/sockets&&|mmap()|'ed output files in POSIX shell x=$(cmd)
	and x=${ cmd ;
	} statements... / was: Re: [regression-3.6] Pipe between
	Cygwin and non Cygwin (CRT/URT) randomly loses characters
To:	cygwin AT cygwin DOT com
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.30
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Roland Mainz via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Roland Mainz <roland DOT mainz AT nrubsig DOT org>
Sender:	"Cygwin" <cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com>
X-MIME-Autoconverted:	from base64 to 8bit by delorie.com id 51JN8ukZ4177824

On Wed, Feb 19, 2025 at 8:23 AM Takashi Yano via Cygwin
<cygwin AT cygwin DOT com> wrote:
>
> On Wed, 19 Feb 2025 07:59:00 +0100
> Cedric Blancher wrote:
> > Good morning!
> >
> > Cygwin 3.6.0-0.374.g4dd859d01c22.x86_64 on Win10/AMD64/64bit:
> >
> > Pipe between Cygwin and non Cygwin (CRT/URT) randomly loses
> > characters. We do not have a reproducer yet, but it is happening at
> > least hourly on our test machines. Sometimes we only get a single
> > character, and sometimes words are missing.
> >
> > Typically it happens like this:
> > bash -c 'typeset bar="$( powershell -Command ...) ; echo "${bar}""',
> > where bar does not contain the full output
>
> Please provide full command line that can reproduce your problem.
> I can not see any pipe in above command.

I think Cedric is referring to the "hidden" pipes/fifos/sockets a
POSIX shell might use internally:
1. The x=$(cmd) and x={ cmd ; } statements have "hidden"
pipes/fifos/sockets to read from the child.
Sometimes it's just "redirect output to file, run child, once child
quits |mmap()| the output into parent shell address space and use that
mapped memory variable content", sometimes it's a traditional pipe or
fifo.
And in case of shells who use |I_PEEK| (e.g. ksh93) they might use
sockets if the systems's fifo implementation does not support
|I_PEEK|.

2. x=$(cmd) is intended to do a |fork()| and run "cmd" from that child
process (the "(" ")" in x=$(cmd) referring to a "subshell"), but many
shells optimise the |fork()| away if it just runs a shell function or
shell builtin command. ksh93's x=${ cmd ; } does not fork because it's
not running in a subshell, and only redirects stdout to a
file/pipe/fifo/socket or |mmap()|'ed output, and uses the resulting
data as value for variable "x"

And that can make testing in this case HARD if you do not know what
the shell implementation is doing. We had "fun" like that in the
OpenSolaris ksh93-integration project long ago, and found LOTS of bugs
in the Solaris kernel&&libc.

> > Same happens with dash and ksh93.

And THAT statement is BAD, because it means basic shell functionality
is broken, affecting dash/bash/ksh93 - reading that IMO means Cygwin
has a problem in the pipe/fifo code... ;-(

Best way forward is to grab the shell testsuite (e.g. both bash and
ast-ksh/ksh93 ship with test suites), and run the pipe/command
substitution tests.

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland DOT mainz AT nrubsig DOT org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -