Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Date: Fri, 30 Aug 2002 17:43:24 +1000 (EST)
From: luke DOT kendall AT cisra DOT canon DOT com DOT au
Subject: Re: Bug? Mixed CR/LF and LF line endings from different programs
To: cygwin AT cygwin DOT com
In-Reply-To: <20020830004815.GA25716@redhat.com>
MIME-Version: 1.0
Content-Type: TEXT/plain; CHARSET=US-ASCII
Message-Id: <20020830073942.094588B1F@bellmann.research.canon.com.au>

On 29 Aug, Christopher Faylor wrote:
>  On Fri, Aug 30, 2002 at 10:19:38AM +1000, luke DOT kendall AT cisra DOT canon DOT com DOT au wrote:
>  >On 29 Aug, Christopher Faylor wrote:
>  >>  On Thu, Aug 29, 2002 at 09:21:10AM -0700, Shankar Unni wrote:
>  >>  >Christopher Faylor wrote:
>  >>  >>awk and sed open their standard input in textmode.  This is by design.
>  >>  >
>  >>  >Don't they open their stdout in textmode, then?  Otherwise they should
>  >>  >have been "fixed up back" to \r\n when they wrote the lines out, no?
>  >>  
>  >>  I think you can draw your own conclusions on what is happening pretty
>  >>  easily.
>  >
>  >Yep.  I don't understand why, though.
>  
>  awk and sed both do input text and, on output, default to the mode
>  derived from the mount table.  So, \r\n is changed to \n internally and
>  is changed to whatever is appropriate on output.  That means that a '$'
>  will match eol in sed but may still output a \r\n.

Good.  Below, you say that a binmode  mount will force all \n outputs,
which is the opposite of what I want.  I'm using a textmode mount, and
still getting \n output for awk and sed, which seems be doing the
opposite of what would be appropriate.

Maybe I can just ask this: what combination of mount mode and CYGWIN
setting will ensure that all text processing will generate
platform-native (CR-LF) output?

>  >matching - depending on the mount type - DOS or Unix, seems correct to
>  >me.  Yet awk and sed don't do it.  This suggests that they're not
>  >opening the output file in text mode,
>  
>  They're not opening the output file in any mode.  The mount type of the
>  disk prevails.  Forcing them to write \r\n line endings is exactly the
>  wrong thing to do.  The default for cygwin should be binary out.
>  Actually they probably should both be using "automode" (input text,
>  output binary) but both packages probably predate that.

Does that mean there *is* a problem?

>  >which seems to contradict the Cygwin FAQ:
>  >
> >    It is rather easy for the porter to fix the source code by supplying
> >    the appropriate file processing mode switches to the open/fopen
> >    functions. Treat all text files as text and treat all binary files
> >    as binary. 
>  >
>  >Since awk and sed work on streams, I doubt that they'd be doing lseeks,
>  
>  That section of the FAQ is out-of-date.  lseeks are not a problem.

Excellent.

>  >so in fact I can't imagine why they're not opening *input* files in text
>  >mode, too.  They *are* text processing tools, after all.
>  
>  Huh?  I already said that they are opening input files in text mode.

Text mode means it does CR/LF -> LF translation on input, and LF ->
CR/LF translation on output (except CR/LF does *not* go to CR/CR/LF)?

>  >>  I just wanted to make sure that people understand that the behavior is
>  >>  not a random event.  It comes up from time to time here and I thought
>  >>  that it bears repeating that both are working the way they are designed
>  >>  to work.
>  >
>  >I understand that it's not random, but now I'm at a loss to understand
>  >why that behaviour was chosen.
>  >
>  >So, what's the recommended way for using Cygwin for any sort of text
>  >processing?
>  
>  Mount everything in binmode if you want to only see \n line endings.

That's the opposite of what I want.  I want consistent, CR/LF line
endings.  Surely that's possible?

>  >If you work in Unix mode this wouldn't happen, but then *all* the files
>  >you produce won't be acceptable to lots of native applications, since
>  >they won't have the native line ending.
>  
>  And, that is tough for native applications.  There is no magic bullet
>  here.


>  >Ah, hang on, I've just been poking through the user guide and found the
>  >CYGWIN=nobinmode option, that makes everything work as I would have
>  >expected.
>  >
>  >Whew!
>  
>  Actually, if it does that's a bug.  nobinmode only works on things for
>  which no mount info can be derived, like pipes.

Perhaps the CYGWIN binmode option could do with some extra explanation
in the user guide.  I confess I had to experiment to get the desired
result.

luke


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/