[Help-bash] behavior of -o pipefail

[Christof Warlich]

Hi,

I'm a bit surprised w.r.t. the implemented (and documented) behavior of
-o pipefail:

Why is the last instead of the first error code in a pipe returned? Example:

$ bash -o pipefail -c '(exit 42) | (exit 43) | (exit 0)'; echo $?
43

IMHO, returning the first error code (i.e. 42 in the previous example)
would be much more intuitive and useful, as errors in subsequent
pipeline steps are most likely just only due to errors in previous
steps. But maybe I'm missing an important point and there is (and / or
was) a good reason that matters are as they are?

Cheers,

Chris

P.S.: I know that the ${PIPESTATUS[@]}-array allows compehensive access
to all exit codes of a pipe, so admittedly, my question may only be
considered as a matter of aesthetics.

[Chet Ramey]

Hi,
I'm a bit surprised w.r.t. the implemented (and documented) behavior of -o
Why is the last instead of the first error code in a pipe returned?

Because that's how Bourne-style shells have always behaved, Posix
standardizes it, and there are millions of scripts that depend on it.

[Christof Warlich]

Post by Chet Ramey
Because that's how Bourne-style shells have always behaved,

But the standard (Bourne) shell doesn't even seem to support -o
pipefail:

$ sh -o pipefail -c '(exit 42) | (exit 43) | (exit 0)'; echo $?
sh: 0: Illegal option -o pipefail
2

And ksh seems to ignore it:

ksh -o pipefail -c '(exit 42) | (exit 43) | (exit 0)'; echo $?
0

And zsh seems to have implemented it recent versions only.

Could you give examples of other Borne-style shells that _do_ support
-o pipefail?

Post by Chet Ramey
Posix standardizes it

I could not find that. Instead,

http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.htm
l#tag_02_09_02

only says:

Exit Status

If the reserved word ! does not precede the pipeline, the exit status
shall be the exit status of the last command specified in the pipeline.
Otherwise, the exit status shall be the logical NOT of the exit status
of the last command. That is, if the last command returns zero, the
exit status shall be 1; if the last command returns greater than zero,
the exit status shall be zero.

Furthermore,
https://www.gnu.org/software/bash/manual/html_node/Bash-POSIX-Mode.html

gave me the impression that bash's standard (i.e. non-Posix) mode may
deviate from POSIX when it suggests something that (as in this case)
may certainly be regarded as bad practice.

Post by Chet Ramey
and there are millions of scripts that depend on it.

Scripts that use -o pipefail certainly do depend on getting an error
code when any of the pipeline steps fail, but I really doubt that any
script depends on getting the error code of the _last_ failing command
if a _previous_ pipeline step failed alredy. As I pointed out in my
initial post, returning the error of the last instead of the first
failing pipeline step is both counter-intuitive and useless.

Anyhow, with the PIPSTATUS array at hand, I'm fine anyway. I just
wanted to add my share to help make bash a tiny bit easier to use :-).

[Chet Ramey]

Post by Chet Ramey
Because that's how Bourne-style shells have always behaved,

 

You asked about returning the last exit status in the pipeline. That's
what the Bourne shell has always done.

[Chet Ramey]

Post by Chet Ramey

Post by Chet Ramey
Because that's how Bourne-style shells have always behaved,

 

You asked about returning the last exit status in the pipeline. That's
what the Bourne shell has always done.

My bad, I took another look at your message. The reason that pipefail
makes the rightmost non-zero exit status the status of the pipeline is
also historical. It's what ksh93 did, and therefore what the folks
involved in a discussion about multiple implementations of pipefail
converged on. This happened back in mid-2001, as part of a series of
messages about features to propose to Posix for standardization.

[Christof Warlich]

Post by Chet Ramey
My bad, I took another look at your message. The reason that pipefail
makes the rightmost non-zero exit status the status of the pipeline is
also historical. It's what ksh93 did, and therefore what the folks
involved in a discussion about multiple implementations of pipefail
converged on. This happened back in mid-2001, as part of a series of
messages about features to propose to Posix for standardization.

Ok. But can we agree that this converged decision (to return the last
instead of the first non-zero exit status) was rather unfortunate (for
the very reason that I pointed out before)?

[Chet Ramey]

Post by Christof Warlich

Post by Chet Ramey
My bad, I took another look at your message. The reason that pipefail
makes the rightmost non-zero exit status the status of the pipeline is
also historical. It's what ksh93 did, and therefore what the folks
involved in a discussion about multiple implementations of pipefail
converged on. This happened back in mid-2001, as part of a series of
messages about features to propose to Posix for standardization.

Ok. But can we agree that this converged decision (to return the last
instead of the first non-zero exit status) was rather unfortunate (for the
very reason that I pointed out before)?

I don't exactly buy that. It's just as likely that an early stage in a
pipeline exits due to a write error (or SIGPIPE) as the result of a
fatal error in a subsequent pipeline stage. We had to make a choice, and
we went with existing practice.

[Christof Warlich]

Post by Chet Ramey

Post by Christof Warlich

Post by Chet Ramey
My bad, I took another look at your message. The reason that pipefail
makes the rightmost non-zero exit status the status of the pipeline is
also historical. It's what ksh93 did, and therefore what the folks
involved in a discussion about multiple implementations of pipefail
converged on. This happened back in mid-2001, as part of a series of
messages about features to propose to Posix for standardization.

Ok. But can we agree that this converged decision (to return the last
instead of the first non-zero exit status) was rather unfortunate (for the
very reason that I pointed out before)?

I don't exactly buy that. It's just as likely that an early stage in a
pipeline exits due to a write error (or SIGPIPE) as the result of a
fatal error in a subsequent pipeline stage. We had to make a choice, and
we went with existing practice.

Good point, returning the first or the last error wouldn't matter for
cases where one of the stages breaks the pipe.

But what about filters like grep that are designed to return a status
depending of the data they saw? I would expect that (such) a
well-designed filter would usually not break the pipe, because it would
only exit (with the desired exit status) after seeing EOF.

With piplines being made up of such well-behaving filters, it would
still be beneficial to see the first error instead of the last one. Can
we agree with that?

[Tadeus Prastowo]

Post by Christof Warlich

Post by Chet Ramey

Post by Christof Warlich

Post by Chet Ramey
My bad, I took another look at your message. The reason that pipefail
makes the rightmost non-zero exit status the status of the pipeline is
also historical. It's what ksh93 did, and therefore what the folks
involved in a discussion about multiple implementations of pipefail
converged on. This happened back in mid-2001, as part of a series of
messages about features to propose to Posix for standardization.

Ok. But can we agree that this converged decision (to return the last
instead of the first non-zero exit status) was rather unfortunate (for the
very reason that I pointed out before)?

I don't exactly buy that. It's just as likely that an early stage in a
pipeline exits due to a write error (or SIGPIPE) as the result of a
fatal error in a subsequent pipeline stage. We had to make a choice, and
we went with existing practice.

Good point, returning the first or the last error wouldn't matter for
cases where one of the stages breaks the pipe.
But what about filters like grep that are designed to return a status
depending of the data they saw? I would expect that (such) a
well-designed filter would usually not break the pipe, because it would
only exit (with the desired exit status) after seeing EOF.

What about the case where grep as the last in the pipe is invoked with
switch `-m1', which may cause an error coming from some process
earlier in the pipeline?

Post by Christof Warlich
With piplines being made up of such well-behaving filters, it would
still be beneficial to see the first error instead of the last one. Can
we agree with that?

--
Best regards,
Tadeus

[Greg Wooledge]

Post by Christof Warlich
With piplines being made up of such well-behaving filters, it would
still be beneficial to see the first error instead of the last one. Can
we agree with that?

If you care about that, just use the PIPESTATUS array variable instead of
pipefail. Pipefail is only intended to report "something bad happened",
not to help you pinpoint exactly which bad thing happened to whom. It's
a sledgehammer, not a sewing needle.

[Chet Ramey]

Post by Greg Wooledge

Post by Christof Warlich
With piplines being made up of such well-behaving filters, it would
still be beneficial to see the first error instead of the last one. Can
we agree with that?

If you care about that, just use the PIPESTATUS array variable instead of
pipefail. Pipefail is only intended to report "something bad happened",
not to help you pinpoint exactly which bad thing happened to whom. It's
a sledgehammer, not a sewing needle.

You're not wrong. `set -o pipefail' is intended to fill the functionality
gap caused by $? only using the exit status of the last element of the
pipeline. If you want more, there are tools available.

[Christof Warlich]

Post by Chet Ramey

Post by Greg Wooledge

Post by Christof Warlich
With piplines being made up of such well-behaving filters, it would
still be beneficial to see the first error instead of the last one. Can
we agree with that?

If you care about that, just use the PIPESTATUS array variable instead of
pipefail. Pipefail is only intended to report "something bad happened",
not to help you pinpoint exactly which bad thing happened to whom. It's
a sledgehammer, not a sewing needle.

You're not wrong. `set -o pipefail' is intended to fill the functionality
gap caused by $? only using the exit status of the last element of the
pipeline. If you want more, there are tools available.

Sure, PIPESTATUS always comes to rescue. I just hoped for making the
sledgehammer to become a bit more versatile, but maybe you're right that
it's not worth it.