weird behaviour of quotes in bash variable assignments

[Gary Dale]

I have the same bash script on two different Debian/Buster AMD64
servers. However on one it refused to run. I tracked it down quickly to
a variable substitution problem.

The line causing the problem reads: report="/root/clamscan-report"

On one server  echo $report  prints  /root/clamscan-report  while on the
other it prints  "/root/clamscan-report".

Needless to say clamscan can't print to the latter. I fixed it by
removing the quotes on the one server but now the scripts are different
between the two servers, which isn't what I want. More importantly, I
don't understand why it refuses to remove the quotes.

Where does this behaviour (keeping the quotes) get set?

[The Wanderer]

Post by Gary Dale
I have the same bash script on two different Debian/Buster AMD64
servers. However on one it refused to run. I tracked it down quickly
to a variable substitution problem.
The line causing the problem reads: report="/root/clamscan-report"
On one server echo $report prints /root/clamscan-report while on
the other it prints "/root/clamscan-report".
Needless to say clamscan can't print to the latter. I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want.

Given the lack of spaces or other potentially-problematic characters in
the path, why not remove them on both servers? Will there potentially be
cases where the path is different, and such characters may be present?

Post by Gary Dale
More importantly, I don't understand why it refuses to remove the
quotes.
Where does this behaviour (keeping the quotes) get set?

First up: are you sure it's actually bash (rather than some other shell)
that's running the script in both cases?

Second, can you confirm that bash is the same version on both servers?
As reported by e.g 'bash --version'.

If both of those are confirmed, then it may be worth digging deeper. I'm
a bit reluctant to delve deep into the bash man page looking for
something like that without first ruling out other possibilities.

My first guess would be that one of the two might be using the shell
builtin command 'echo' and the other might be using /bin/echo, but
that's only a guess.

[David Christensen]

Post by The Wanderer

Post by Gary Dale
I have the same bash script on two different Debian/Buster AMD64
servers. However on one it refused to run. I tracked it down quickly
to a variable substitution problem.
The line causing the problem reads: report="/root/clamscan-report"
On one server echo $report prints /root/clamscan-report while on
the other it prints "/root/clamscan-report".
Needless to say clamscan can't print to the latter. I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want.

Given the lack of spaces or other potentially-problematic characters in
the path, why not remove them on both servers? Will there potentially be
cases where the path is different, and such characters may be present?

Post by Gary Dale
More importantly, I don't understand why it refuses to remove the
quotes.
Where does this behaviour (keeping the quotes) get set?

First up: are you sure it's actually bash (rather than some other shell)
that's running the script in both cases?
Second, can you confirm that bash is the same version on both servers?
As reported by e.g 'bash --version'.
If both of those are confirmed, then it may be worth digging deeper. I'm
a bit reluctant to delve deep into the bash man page looking for
something like that without first ruling out other possibilities.
My first guess would be that one of the two might be using the shell
builtin command 'echo' and the other might be using /bin/echo, but
that's only a guess.

And:


Is the environment identical on the two servers?

server1$ env > env1.out

server2$ env > env2.out

(move one file to the other host)

$ diff env1.out env2.out


Do you have a "shebang" line as the first line of the script? Please post.


I find that Bourne syntax shell scripts are more portable than Bash
syntax shell scripts. Have you tried '#!/bin/sh' ?


Have you tried single quotes?


David

[Gary Dale]

Post by David Christensen

Post by The Wanderer

Post by Gary Dale
I have the same bash script on two different Debian/Buster AMD64
servers. However on one it refused to run. I tracked it down quickly
to a variable substitution problem.
The line causing the problem reads: report="/root/clamscan-report"
On one server  echo $report  prints  /root/clamscan-report while on
the other it prints  "/root/clamscan-report".
Needless to say clamscan can't print to the latter. I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want.

Given the lack of spaces or other potentially-problematic characters in
the path, why not remove them on both servers? Will there potentially be
cases where the path is different, and such characters may be present?

Post by Gary Dale
More importantly, I don't understand why it refuses to remove the
quotes.
Where does this behaviour (keeping the quotes) get set?

First up: are you sure it's actually bash (rather than some other shell)
that's running the script in both cases?
Second, can you confirm that bash is the same version on both servers?
As reported by e.g 'bash --version'.
If both of those are confirmed, then it may be worth digging deeper. I'm
a bit reluctant to delve deep into the bash man page looking for
something like that without first ruling out other possibilities.
My first guess would be that one of the two might be using the shell
builtin command 'echo' and the other might be using /bin/echo, but
that's only a guess.

Is the environment identical on the two servers?
    server1$ env > env1.out
    server2$ env > env2.out
    (move one file to the other host)
    $ diff env1.out env2.out

The environments are identical. The server uses are similar - both run
Samba 4 and have Exim 4 set up for sending mail to me. Both run KVM
virtual machines. The server that works also runs bacula and borg while
the problem server runs NFS and Apache2.

Post by David Christensen
Do you have a  "shebang" line as the first line of the script? Please
post.
I find that Bourne syntax shell scripts are more portable than Bash
syntax shell scripts.  Have you tried '#!/bin/sh' ?

That's my shebang line. /bin/sh in both cases is a symlink to dash
(sorry, keep forgetting that Debian has been using dash instead of bash
for years now). The version of dash is identical between the two servers.

Post by David Christensen
Have you tried single quotes?

Wouldn't want to. Single quotes alter the behaviour.

[David Christensen]

Post by Gary Dale
The environments are identical.

Post by David Christensen
Have you tried '#!/bin/sh' ?

That's my shebang line.

Post by David Christensen
Have you tried single quotes?

Wouldn't want to. Single quotes alter the behaviour.

Double quotes, single quotes, and no quotes have the same behavior on my
machine:

2020-09-20 19:50:55 ***@tinkywinky ~/sandbox/sh
$ cat debian-user-20200920-1727-gary-dale.sh
#!/bin/sh

doublequote="/root/clamscan-report"
echo $doublequote

singlequote='/root/clamscan-report'
echo $singlequote

noquote='/root/clamscan-report'
echo $noquote

2020-09-20 19:53:02 ***@tinkywinky ~/sandbox/sh
$ /bin/sh -x debian-user-20200920-1727-gary-dale.sh
+ doublequote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report
+ singlequote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report
+ noquote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report


Please run the above script and the following commands, and post your
console session -- prompts, commands, output:

2020-09-20 19:52:29 ***@tinkywinky ~/sandbox/sh
$ cat /etc/debian_version
9.13

2020-09-20 19:52:38 ***@tinkywinky ~/sandbox/sh
$ uname -a
Linux tinkywinky 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05)
x86_64 GNU/Linux

2020-09-20 19:52:43 ***@tinkywinky ~/sandbox/sh
$ dpkg-query --show dash
dash 0.5.8-2.4

2020-09-20 19:52:51 ***@tinkywinky ~/sandbox/sh
$ dpkg --verify dash

2020-09-20 19:52:56 ***@tinkywinky ~/sandbox/sh
$ sha256sum /bin/sh
e803088e7938b328b0511957dcd0dd7b5600ec1940010c64dbd3814e3d75495f /bin/sh


David

[Gary Dale]

Post by David Christensen

Post by Gary Dale
The environments are identical.

Post by David Christensen
Have you tried '#!/bin/sh' ?

That's my shebang line.

Post by David Christensen
Have you tried single quotes?

Wouldn't want to. Single quotes alter the behaviour.

Double quotes, single quotes, and no quotes have the same behavior on
$ cat debian-user-20200920-1727-gary-dale.sh
#!/bin/sh
doublequote="/root/clamscan-report"
echo $doublequote
singlequote='/root/clamscan-report'
echo $singlequote
noquote='/root/clamscan-report'
echo $noquote
$ /bin/sh -x debian-user-20200920-1727-gary-dale.sh
+ doublequote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report
+ singlequote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report
+ noquote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report
Please run the above script and the following commands, and post your
$ cat /etc/debian_version
9.13
$ uname -a
Linux tinkywinky 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05)
x86_64 GNU/Linux
$ dpkg-query --show dash
dash    0.5.8-2.4
$ dpkg --verify dash
$ sha256sum /bin/sh
e803088e7938b328b0511957dcd0dd7b5600ec1940010c64dbd3814e3d75495f /bin/sh
David

In the simple case, the quotes are the same, but there are times when
they have different behaviours. I avoid single-quotes so that when I use
them, it's a clue that there is a reason.

Here's what I got with your script:

/root/clamscan-report
/root/clamscan-report
/root/clamscan-report

When I retried my script with the quotes, it started working. I have no
idea what changed from earlier today. I certainly didn't update anything
on either server...

[David Christensen]

Post by Gary Dale

Post by David Christensen
$ cat debian-user-20200920-1727-gary-dale.sh
#!/bin/sh
doublequote="/root/clamscan-report"
echo $doublequote
singlequote='/root/clamscan-report'
echo $singlequote
noquote='/root/clamscan-report'
echo $noquote
$ /bin/sh -x debian-user-20200920-1727-gary-dale.sh
+ doublequote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report
+ singlequote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report
+ noquote=/root/clamscan-report
+ echo /root/clamscan-report
/root/clamscan-report

In the simple case, the quotes are the same, but there are times when
they have different behaviours. I avoid single-quotes so that when I use
them, it's a clue that there is a reason.

AIUI the Bourne shell interpolates variables, and potentially other
constructs, within double-quotes. I avoid double quotes unless I want
interpolation.

Post by Gary Dale
/root/clamscan-report
/root/clamscan-report
/root/clamscan-report
When I retried my script with the quotes, it started working. I have no
idea what changed from earlier today. I certainly didn't update anything
on either server...

I find that a version control system can be very useful for system
administration, by allowing me to check in and verify all of my
configuration files, scripts, etc..


David

[David]

Post by Gary Dale
I have the same bash script on two different Debian/Buster AMD64
servers. However on one it refused to run. I tracked it down quickly
to a variable substitution problem.
The line causing the problem reads: report="/root/clamscan-report"
On one server echo $report prints /root/clamscan-report while on
the other it prints "/root/clamscan-report".

The shell should strip the double quotes before assigning the right hand
string to the left hand variable.

Another possibility to check that I haven't seen suggested already, even
if it is unlikely ...

From the perspective of not knowing how these bash scripts were deployed,

it's worth confirming that what we are assuming is the "same bash script" are
indeed exactly identical.

$ cmp script1 script2

If somehow the double quotes in the problem script have become something
other than plain ASCII 0x22 bytes, that could explain the observed behaviour.
Because extended character sets can contain a variety of characters that
visually appear identical to their plain ASCII counterparts. But the shell won't
parse and remove them as quote characters, even though they "look like" quote
characters.

[David Christensen]

From the perspective of not knowing how these bash scripts were deployed,
it's worth confirming that what we are assuming is the "same bash script" are
indeed exactly identical.
$ cmp script1 script2
If somehow the double quotes in the problem script have become something
other than plain ASCII 0x22 bytes, that could explain the observed behaviour.
Because extended character sets can contain a variety of characters that
visually appear identical to their plain ASCII counterparts. But the shell won't
parse and remove them as quote characters, even though they "look like" quote
characters.

A spurious space character at the end of the line is invisible in most
editors, but can cause problems for some programs. For other programs,
spaces at the end of a line have meaning (e.g. Markdown). Similarly,
leading or interior spaces vs. tab will drive you mad (e.g. Make).
Without the files and the servers they run on, we can only speculate.


David

[Cindy Sue Causey]

Post by David Christensen

From the perspective of not knowing how these bash scripts were deployed,
it's worth confirming that what we are assuming is the "same bash script" are
indeed exactly identical.
$ cmp script1 script2
If somehow the double quotes in the problem script have become something
other than plain ASCII 0x22 bytes, that could explain the observed behaviour.
Because extended character sets can contain a variety of characters that
visually appear identical to their plain ASCII counterparts. But the shell won't
parse and remove them as quote characters, even though they "look like" quote
characters.

A spurious space character at the end of the line is invisible in most
editors, but can cause problems for some programs. For other programs,
spaces at the end of a line have meaning (e.g. Markdown). Similarly,
leading or interior spaces vs. tab will drive you mad (e.g. Make).

Oh, yeah! I don't remember how long I fought this when I first started
playing with web design. They're invisible except under circumstances.
I accidentally FINALLY figured it out when I had opened a file up in a
super plain text editor that wasn't.. MS Word.

There was something like a one or two-character blob in the first line
that was keeping the file from displaying properly in about the only
browser that existed back then AND was also keeping it from validating
properly (via W3 dotORG).

Post by David Christensen
Without the files and the servers they run on, we can only speculate.

Thought about writing this yesterday then it looked like things might
have been solved or suchly. Since this is still being chatted up...

Another much more recent fix for me has been to delete the quotes, and
I think also apostrophes, and simply retype them (instead of
copy-and-paste).

If that has already been done since this was about that one line,
maybe it's about something similar in a line prior to that one. With a
head nod to what's typed below, try the same for ALL apostrophes, too.

It's that same deal about things being "invisible" until they're
displayed in the right text editor kind of package. There's quotes,
and there's fancy quotes. A quick search pumped out Wikipedia [0]:

'…' and "…" are known as neutral, vertical, straight, typewriter,
dumb, or ASCII quotation marks.

‘…’ and “…” are known as typographic, curly, curved, book, or smart
quotation marks.

Those lines from Wikipedia may print out the same in this method of
sharing. If they do, the second set are visually "curlier" than the
first. Wikipedia goes on to say the second set are for manuscripts,
printing, and typesetting.

My take: It's like the second "curly, curved" set envelopes, leans
into, visually gets cozier with the enclosed text a tiny bit more.

Both types of quotation marks frequently look ALMOST exactly the same
unless you have them blown up in an extra large font size on the
screen.

They function differently somehow, too. I don't remember if that
difference in functioning was by design or just bad placement in a
file. Maybe it's about the above reference to ASCII. Discovering the
difference between them was another one of those ah-ha moments tripped
over via a terminal window.

May or may not help, but you could try "editor <file>" to see if
anything odd shows up that way. I just did so with both mousepad and
editor (terminal) for the (.)Xauthority file.

Mousepad kicks out a "The file was not UTF-8 valid" error and aborts.
Editor in the terminal at least displays a bunch of characters that
include a number of question marks filling in for whatever part of
that is causing mousepad to fail. Not a great apples to apples
example, but it's the best I can think of just now.

Happy Monday morning..

Cindy :)

[0] https://en.wikipedia.org/wiki/Quotation_mark

[Greg Wooledge]

Post by Cindy Sue Causey
'…' and "…" are known as neutral, vertical, straight, typewriter,
dumb, or ASCII quotation marks.
‘…’ and “…” are known as typographic, curly, curved, book, or smart
quotation marks.

Yes. This is one of the possible causes for the behavior the OP was
reporting. But if this is true, then it reveals that they were lying
when they claimed that the scripts were the same on both servers.

Post by Cindy Sue Causey
They function differently somehow, too. I don't remember if that
difference in functioning was by design or just bad placement in a
file. Maybe it's about the above reference to ASCII. Discovering the
difference between them was another one of those ah-ha moments tripped
over via a terminal window.

Remember, the computer can't actually *see* the characters the way
that you do. To the computer, every character is just a number, or
a sequence of numbers.

To bash, the character " (byte value 0x22) has a special meaning, and
so does the character ' (byte value 0x27). However, the characters
“ (byte values 0xe2809c) and ” (byte values 0xe2809d) have no special
meaning. They're just some random data that the shell doesn't interpret.

unicorn:~$ x="foo"; echo "$x"
foo
unicorn:~$ x='foo'; echo "$x"
foo
unicorn:~$ x=“foo”; echo "$x"
“foo”

To beat a dead horse some more, if *this* was the OP's problem, then they
told multiple lies about it. They did not paste the actual failing line
from the failing script (probably retyped it instead), and they did not
ACTUALLY COMPARE the two scripts to see whether they were different,
instead simply ASSUMING that the two scripts were identical, even though
they very clearly weren't.

An actual troubleshooting would have done something like using md5sum
on the script on each machine, and pasting the md5sum commands (including
the full script pathname) and their output to the mailing list. Openness.

Or, hell, even "ls -l /full/pathname" would probably have revealed that
the scripts were not the same SIZE. That would also have shown immediately
that the scripts were not "the same".

[David Wright]

Post by Greg Wooledge

Post by Cindy Sue Causey
'…' and "…" are known as neutral, vertical, straight, typewriter,
dumb, or ASCII quotation marks.
‘…’ and “…” are known as typographic, curly, curved, book, or smart
quotation marks.

Yes. This is one of the possible causes for the behavior the OP was
reporting. But if this is true, then it reveals that they were lying
when they claimed that the scripts were the same on both servers.
[…]
To beat a dead horse some more, if *this* was the OP's problem, then they
told multiple lies about it. They did not paste the actual failing line
from the failing script (probably retyped it instead), and they did not
ACTUALLY COMPARE the two scripts to see whether they were different,
instead simply ASSUMING that the two scripts were identical, even though
they very clearly weren't.
An actual troubleshooting would have done something like using md5sum
on the script on each machine, and pasting the md5sum commands (including
the full script pathname) and their output to the mailing list. Openness.
Or, hell, even "ls -l /full/pathname" would probably have revealed that
the scripts were not the same SIZE. That would also have shown immediately
that the scripts were not "the same".

I think we should apply Hanlon's razor rather than saying the OP lied.
After all, "compare" means diff or cmp to us, whereas many might just
use their eyeballs. And we all know that authors are the worst people
to check their own work. Proof-reading is a special skill.

Even their fix is poorly described. Did they just type the quotes back
in with an editor, in which case there's no guarantee that the scripts
are identical between machines, or did they transfer a working script
to the failing machine? The best line is save until last: "I certainly
didn't update anything on either server...". Well, yes, that's
*precisely* what you did: you updated the script.

Cheers,
David.

[Gary Dale]

Post by David Wright

Post by Greg Wooledge

Post by Cindy Sue Causey
'…' and "…" are known as neutral, vertical, straight, typewriter,
dumb, or ASCII quotation marks.
‘…’ and “…” are known as typographic, curly, curved, book, or smart
quotation marks.

Yes. This is one of the possible causes for the behavior the OP was
reporting. But if this is true, then it reveals that they were lying
when they claimed that the scripts were the same on both servers.
[…]
To beat a dead horse some more, if *this* was the OP's problem, then they
told multiple lies about it. They did not paste the actual failing line
from the failing script (probably retyped it instead), and they did not
ACTUALLY COMPARE the two scripts to see whether they were different,
instead simply ASSUMING that the two scripts were identical, even though
they very clearly weren't.
An actual troubleshooting would have done something like using md5sum
on the script on each machine, and pasting the md5sum commands (including
the full script pathname) and their output to the mailing list. Openness.
Or, hell, even "ls -l /full/pathname" would probably have revealed that
the scripts were not the same SIZE. That would also have shown immediately
that the scripts were not "the same".

I think we should apply Hanlon's razor rather than saying the OP lied.
After all, "compare" means diff or cmp to us, whereas many might just
use their eyeballs. And we all know that authors are the worst people
to check their own work. Proof-reading is a special skill.
Even their fix is poorly described. Did they just type the quotes back
in with an editor, in which case there's no guarantee that the scripts
are identical between machines, or did they transfer a working script
to the failing machine? The best line is save until last: "I certainly
didn't update anything on either server...". Well, yes, that's
*precisely* what you did: you updated the script.
Cheers,
David.

You are taking my quote out of context. I didn't change anything on the
server to make the script start working. I updated the script to see if
it would work after trying Greg's test. There were no program or setting
updates on the server, and certainly nothing that updated dash. This is
Debian/Stable we're talking about, after all.

Since it is a file server, there probably were changes to the files on
its shares, but I'd hardly count that as an "update". Similarly, it was
running cron jobs for backups and virus scans (unsuccessfully) but again
I wouldn't call those "updates".

[David Wright]

Post by Gary Dale

Post by David Wright

Post by Greg Wooledge

Post by Cindy Sue Causey
'…' and "…" are known as neutral, vertical, straight, typewriter,
dumb, or ASCII quotation marks.
‘…’ and “…” are known as typographic, curly, curved, book, or smart
quotation marks.

Yes. This is one of the possible causes for the behavior the OP was
reporting. But if this is true, then it reveals that they were lying
when they claimed that the scripts were the same on both servers.
[…]
To beat a dead horse some more, if *this* was the OP's problem, then they
told multiple lies about it. They did not paste the actual failing line
from the failing script (probably retyped it instead), and they did not
ACTUALLY COMPARE the two scripts to see whether they were different,
instead simply ASSUMING that the two scripts were identical, even though
they very clearly weren't.
An actual troubleshooting would have done something like using md5sum
on the script on each machine, and pasting the md5sum commands (including
the full script pathname) and their output to the mailing list. Openness.
Or, hell, even "ls -l /full/pathname" would probably have revealed that
the scripts were not the same SIZE. That would also have shown immediately
that the scripts were not "the same".

I think we should apply Hanlon's razor rather than saying the OP lied.
After all, "compare" means diff or cmp to us, whereas many might just
use their eyeballs. And we all know that authors are the worst people
to check their own work. Proof-reading is a special skill.
Even their fix is poorly described. Did they just type the quotes back
in with an editor, in which case there's no guarantee that the scripts
are identical between machines, or did they transfer a working script
to the failing machine? The best line is save until last: "I certainly
didn't update anything on either server...". Well, yes, that's
*precisely* what you did: you updated the script.

↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Post by Gary Dale
You are taking my quote out of context. I didn't change anything on
the server to make the script start working. I updated the script to

↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Post by Gary Dale
see if it would work after trying Greg's test. There were no program
or setting updates on the server, and certainly nothing that updated
dash. This is Debian/Stable we're talking about, after all.

Sorry, I thought you wrote that on Sunday afternoon, "I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want."
Then on Sunday evening, you wrote "When I retried my script with the
quotes, it started working."

The general opinion is that the script was faulty, probably in the
quotes used. The narrative says that you removed the quotes, and
later put them back. It seems fair to suggest that the quotes you
put back were not the same ones that you removed. They were replaced
in the same location, but you didn't put the old (removed) quotes
into a little two-character file, so that you could put precisely
the same ones back into the script, did you?

Post by Gary Dale
Since it is a file server, there probably were changes to the files on
its shares, but I'd hardly count that as an "update". Similarly, it
was running cron jobs for backups and virus scans (unsuccessfully) but
again I wouldn't call those "updates".

Nor I. No, I'm only talking about your script. Does it bear any
relation to the one posted in your blog? The first line (after
the shebang) of the one in the blog is the same line that's under
discussion here, and has curly quotes. I can't parse the second
line's curly quotes, and the fourth line uses an n-dash for a
hyphen *though the other hyphens are ok). The fifth line uses
curly single-quotes. More curly quotes follow.

I don't see any cause for our wasting time pondering on dash
without your posting an MWE that unambiguously demonstrates a
problem.

Cheers,
David.

[Gary Dale]

Post by David Wright

Post by Gary Dale

Post by David Wright

Post by Greg Wooledge

Post by Cindy Sue Causey
'…' and "…" are known as neutral, vertical, straight, typewriter,
dumb, or ASCII quotation marks.
‘…’ and “…” are known as typographic, curly, curved, book, or smart
quotation marks.

Yes. This is one of the possible causes for the behavior the OP was
reporting. But if this is true, then it reveals that they were lying
when they claimed that the scripts were the same on both servers.
[…]
To beat a dead horse some more, if *this* was the OP's problem, then they
told multiple lies about it. They did not paste the actual failing line
from the failing script (probably retyped it instead), and they did not
ACTUALLY COMPARE the two scripts to see whether they were different,
instead simply ASSUMING that the two scripts were identical, even though
they very clearly weren't.
An actual troubleshooting would have done something like using md5sum
on the script on each machine, and pasting the md5sum commands (including
the full script pathname) and their output to the mailing list. Openness.
Or, hell, even "ls -l /full/pathname" would probably have revealed that
the scripts were not the same SIZE. That would also have shown immediately
that the scripts were not "the same".

I think we should apply Hanlon's razor rather than saying the OP lied.
After all, "compare" means diff or cmp to us, whereas many might just
use their eyeballs. And we all know that authors are the worst people
to check their own work. Proof-reading is a special skill.
Even their fix is poorly described. Did they just type the quotes back
in with an editor, in which case there's no guarantee that the scripts
are identical between machines, or did they transfer a working script
to the failing machine? The best line is save until last: "I certainly
didn't update anything on either server...". Well, yes, that's
*precisely* what you did: you updated the script.

↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Post by Gary Dale
You are taking my quote out of context. I didn't change anything on
the server to make the script start working. I updated the script to

↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Post by Gary Dale
see if it would work after trying Greg's test. There were no program
or setting updates on the server, and certainly nothing that updated
dash. This is Debian/Stable we're talking about, after all.

Sorry, I thought you wrote that on Sunday afternoon, "I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want."
Then on Sunday evening, you wrote "When I retried my script with the
quotes, it started working."
The general opinion is that the script was faulty, probably in the
quotes used. The narrative says that you removed the quotes, and
later put them back. It seems fair to suggest that the quotes you
put back were not the same ones that you removed. They were replaced
in the same location, but you didn't put the old (removed) quotes
into a little two-character file, so that you could put precisely
the same ones back into the script, did you?

I thought about that but then there would be no way I could demonstrate
it other than what I did - post the offending line via cut & paste, a
method people have been arguing can change the quotes.

Post by David Wright

Post by Gary Dale
Since it is a file server, there probably were changes to the files on
its shares, but I'd hardly count that as an "update". Similarly, it
was running cron jobs for backups and virus scans (unsuccessfully) but
again I wouldn't call those "updates".

Nor I. No, I'm only talking about your script. Does it bear any
relation to the one posted in your blog? The first line (after
the shebang) of the one in the blog is the same line that's under
discussion here, and has curly quotes. I can't parse the second
line's curly quotes, and the fourth line uses an n-dash for a
hyphen *though the other hyphens are ok). The fifth line uses
curly single-quotes. More curly quotes follow.
I don't see any cause for our wasting time pondering on dash
without your posting an MWE that unambiguously demonstrates a
problem.

Yes, that's the script - copy-pasted from the working server then with
the e-mail addresses changed (they are actually parameters to the
working script, but why complicate things when explaining a basic
script). As you noted, it has changed things - hopefully not to the
point that people won't be able to make it work. I haven't found a way
to stop Wordpress from doing the substitution but I note the raw text is
still correct (once you remove the html).

[David Wright]

Post by Gary Dale

Post by David Wright

Post by Gary Dale

Post by David Wright
I think we should apply Hanlon's razor rather than saying the OP lied.
After all, "compare" means diff or cmp to us, whereas many might just
use their eyeballs. And we all know that authors are the worst people
to check their own work. Proof-reading is a special skill.
Even their fix is poorly described. Did they just type the quotes back
in with an editor, in which case there's no guarantee that the scripts
are identical between machines, or did they transfer a working script
to the failing machine? The best line is save until last: "I certainly
didn't update anything on either server...". Well, yes, that's
*precisely* what you did: you updated the script.

↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Post by Gary Dale
You are taking my quote out of context. I didn't change anything on
the server to make the script start working. I updated the script to

↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Post by Gary Dale
see if it would work after trying Greg's test. There were no program
or setting updates on the server, and certainly nothing that updated
dash. This is Debian/Stable we're talking about, after all.

Sorry, I thought you wrote that on Sunday afternoon, "I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want."
Then on Sunday evening, you wrote "When I retried my script with the
quotes, it started working."
The general opinion is that the script was faulty, probably in the
quotes used. The narrative says that you removed the quotes, and
later put them back. It seems fair to suggest that the quotes you
put back were not the same ones that you removed. They were replaced
in the same location, but you didn't put the old (removed) quotes
into a little two-character file, so that you could put precisely
the same ones back into the script, did you?

I thought about that but then there would be no way I could
demonstrate it other than what I did - post the offending line via cut
& paste, a method people have been arguing can change the quotes.

Post by David Wright

Post by Gary Dale
Since it is a file server, there probably were changes to the files on
its shares, but I'd hardly count that as an "update". Similarly, it
was running cron jobs for backups and virus scans (unsuccessfully) but
again I wouldn't call those "updates".

Nor I. No, I'm only talking about your script. Does it bear any
relation to the one posted in your blog? The first line (after
the shebang) of the one in the blog is the same line that's under
discussion here, and has curly quotes. I can't parse the second
line's curly quotes, and the fourth line uses an n-dash for a
hyphen *though the other hyphens are ok). The fifth line uses
curly single-quotes. More curly quotes follow.
I don't see any cause for our wasting time pondering on dash
without your posting an MWE that unambiguously demonstrates a
problem.

Yes, that's the script - copy-pasted from the working server then with
the e-mail addresses changed (they are actually parameters to the
working script, but why complicate things when explaining a basic
script). As you noted, it has changed things - hopefully not to the
point that people won't be able to make it work. I haven't found a way
to stop Wordpress from doing the substitution but I note the raw text
is still correct (once you remove the html).

That I can believe. The substitutions appear to be systematic,
<non-space><quote> → 99-style, and <space><quote> → 66-style,
-- → n-dash, --- → m-dash; I think that covers it. And that might be
something you cannot change: I see it converts your apostrophes
in running text too.

But I don't see why you can't post emails with unadulterated
characters. You can obviously read and copy the working script
yourself, so you can at least paste it into a file on your
email computer, using cat >file if necessary¹, and then
InsertFile the file into your email composer. Your running text
has ASCII quotes, after all. That should avoid any problem like
https://lists.debian.org/debian-user/2020/09/msg00677.html
where your script fragment *still* has curly quotes but your
speech marks are ASCII.

I think it's reasonable, when you're reporting non-identical
behaviour from supposedly identical systems, to expect some
demonstration that the test cases and the reporting thereof
are, in fact, using identical scripts before eliminating
them from consideration.

¹ Run md5sum, as suggested earlier, on this file, the servers'
scripts, and any cut-down versions, to demonstrate they're
all the same. This avoids transferring files.

Cheers,
David.

[Gary Dale]

Post by Greg Wooledge

Post by Cindy Sue Causey
'…' and "…" are known as neutral, vertical, straight, typewriter,
dumb, or ASCII quotation marks.
‘…’ and “…” are known as typographic, curly, curved, book, or smart
quotation marks.

Yes. This is one of the possible causes for the behavior the OP was
reporting. But if this is true, then it reveals that they were lying
when they claimed that the scripts were the same on both servers.

Post by Cindy Sue Causey
They function differently somehow, too. I don't remember if that
difference in functioning was by design or just bad placement in a
file. Maybe it's about the above reference to ASCII. Discovering the
difference between them was another one of those ah-ha moments tripped
over via a terminal window.

Remember, the computer can't actually *see* the characters the way
that you do. To the computer, every character is just a number, or
a sequence of numbers.
To bash, the character " (byte value 0x22) has a special meaning, and
so does the character ' (byte value 0x27). However, the characters
“ (byte values 0xe2809c) and ” (byte values 0xe2809d) have no special
meaning. They're just some random data that the shell doesn't interpret.
unicorn:~$ x="foo"; echo "$x"
foo
unicorn:~$ x='foo'; echo "$x"
foo
unicorn:~$ x=“foo”; echo "$x"
“foo”
To beat a dead horse some more, if *this* was the OP's problem, then they
told multiple lies about it. They did not paste the actual failing line
from the failing script (probably retyped it instead), and they did not
ACTUALLY COMPARE the two scripts to see whether they were different,
instead simply ASSUMING that the two scripts were identical, even though
they very clearly weren't.
An actual troubleshooting would have done something like using md5sum
on the script on each machine, and pasting the md5sum commands (including
the full script pathname) and their output to the mailing list. Openness.
Or, hell, even "ls -l /full/pathname" would probably have revealed that
the scripts were not the same SIZE. That would also have shown immediately
that the scripts were not "the same".

As an FYI, the scripts were the same on both servers because I had ssh
sessions on both and copy/pasted the script from one to the other. (cat
<script> on server 1, then use Konsole's copy and paste to select and
paste it into nano on server 2).

[Greg Wooledge]

Post by Gary Dale
As an FYI, the scripts were the same on both servers because I had ssh
sessions on both and copy/pasted the script from one to the other. (cat
<script> on server 1, then use Konsole's copy and paste to select and paste
it into nano on server 2).

Copy and paste doesn't always preserve the actual contents of a file.
What you get depends on several factors, including the behavior of the
terminal emulator that you're copying from. Tabs may be converted to
spaces, and I have no idea what other changes may occur.

As a demonstration for myself, just now, I performed the following
experiment in both rxvt-unicode and xterm, on Debian 10:

1) Run the command: printf 'a\tb\n'
2) Run the command: hd
3) Highlight the line containing the output of the first command, and paste
it into the terminal, to be read by hd.
4) Press Ctrl-D.

On rxvt-unicode, I get this output:

00000000 61 09 62 0a |a.b.|
00000004

On xterm, I get this output:

00000000 61 20 20 20 20 20 20 20 62 0a |a b.|
0000000a

If your original script contained non-breaking spaces, or curly quotes,
or literal tabs, or who knows what else, then there's a nonzero chance
it will be altered by the copy/paste.

This is why you need to *verify* that the two scripts are the same, not
just assume that pasting did what you thought it did.

[Gary Dale]

Post by Greg Wooledge

Post by Gary Dale
As an FYI, the scripts were the same on both servers because I had ssh
sessions on both and copy/pasted the script from one to the other. (cat
<script> on server 1, then use Konsole's copy and paste to select and paste
it into nano on server 2).

Copy and paste doesn't always preserve the actual contents of a file.
What you get depends on several factors, including the behavior of the
terminal emulator that you're copying from. Tabs may be converted to
spaces, and I have no idea what other changes may occur.
As a demonstration for myself, just now, I performed the following
1) Run the command: printf 'a\tb\n'
2) Run the command: hd
3) Highlight the line containing the output of the first command, and paste
it into the terminal, to be read by hd.
4) Press Ctrl-D.
00000000 61 09 62 0a |a.b.|
00000004
00000000 61 20 20 20 20 20 20 20 62 0a |a b.|
0000000a
If your original script contained non-breaking spaces, or curly quotes,
or literal tabs, or who knows what else, then there's a nonzero chance
it will be altered by the copy/paste.
This is why you need to *verify* that the two scripts are the same, not
just assume that pasting did what you thought it did.

Did you try it with Konsole? I got

$ hd
a       b
00000000  61 20 20 20 20 20 20 20  62 0a |a       b.|
0000000a

Most shells (all that I am aware of) treat tabs and blanks as generic
white space - not relevant to the execution of the code.

[Greg Wooledge]

Post by Gary Dale
Did you try it with Konsole? I got
$ hd
a       b
00000000  61 20 20 20 20 20 20 20  62 0a |a       b.|
0000000a

In other words, konsole *did* alter the contents.

Post by Gary Dale
Most shells (all that I am aware of) treat tabs and blanks as generic white
space - not relevant to the execution of the code.

Between arguments, sure. Tabs and spaces are equivalent in those places.
But inside a quoted string argument? No, they're not equivalent.

Who knows what other conversions might have taken place?

You were trying to troubleshoot a script that wasn't doing what you
expected. And yet you just hand-waved everything, rather than getting
in there and doing the work to actually *check* things.

[Gary Dale]

Post by Greg Wooledge

Post by Gary Dale
Did you try it with Konsole? I got
$ hd
a       b
00000000  61 20 20 20 20 20 20 20  62 0a |a       b.|
0000000a

In other words, konsole *did* alter the contents.

Post by Gary Dale
Most shells (all that I am aware of) treat tabs and blanks as generic white
space - not relevant to the execution of the code.

Between arguments, sure. Tabs and spaces are equivalent in those places.
But inside a quoted string argument? No, they're not equivalent.
Who knows what other conversions might have taken place?
You were trying to troubleshoot a script that wasn't doing what you
expected. And yet you just hand-waved everything, rather than getting
in there and doing the work to actually *check* things.

That's where you are making unwarranted assumptions. I tried the line
outside of the script, hand entered rather than copied & pasted, and got
the same result. There was something weird going on in the dash shell at
the time. Just because I didn't report all the testing I'd done doesn't
mean you should assume that I didn't do it.

[David Wright]

Post by Greg Wooledge

Post by Gary Dale
As an FYI, the scripts were the same on both servers because I had ssh
sessions on both and copy/pasted the script from one to the other. (cat
<script> on server 1, then use Konsole's copy and paste to select and paste
it into nano on server 2).

Copy and paste doesn't always preserve the actual contents of a file.
What you get depends on several factors, including the behavior of the
terminal emulator that you're copying from. Tabs may be converted to
spaces, and I have no idea what other changes may occur.
[…]
If your original script contained non-breaking spaces, or curly quotes,
or literal tabs, or who knows what else, then there's a nonzero chance
it will be altered by the copy/paste.
This is why you need to *verify* that the two scripts are the same, not
just assume that pasting did what you thought it did.

Perhaps this paragraph illustrates the problems that can arise
with quotation marks, hyphens, etc:

#!/bin/sh
report=”/root/clamscan-report”
subject=$HOSTNAME” clamscan report: “

mv $report $report.prev
/usr/bin/clamscan –quiet -i -r -l $report /home
sed -i ‘/———– SCAN SUMMARY ———–/,$d’ $report
diff $report $report.prev > $report.diff

if [ -s $report.diff ] ; then
subject=$subject”ALERT – NEW VIRUSES FOUND”
else
subject=$subject”no new viruses”
fi
mail -r “<sender>” -s “$subject” <to e-mail> < $report.diff

Copied and pasted from firefox to emacs -nw, but checked
against the saved web page:

https://garydale.wordpress.com/

Cheers,
David.

[David Christensen]

Post by Gary Dale
As an FYI, the scripts were the same on both servers because I had ssh
sessions on both and copy/pasted the script from one to the other. (cat
<script> on server 1, then use Konsole's copy and paste to select and
paste it into nano on server 2).

That process may have changed the file in invisible ways. Use
hexdump(1) to check for differences. See below examples.


Use scp(1), rsync(1), sshfs(1), a networked version control system such
as cvs(1), etc., to copy files accurately.

Post by Gary Dale
Most shells (all that I am aware of) treat tabs and blanks as generic
white space - not relevant to the execution of the code.

"I guess so". (Obviously, I am not a dash(1) shell scripting expert.)


I have greater confidence in my ability to copy files accurately, and
thereby eliminate one set of potential problems. If the problem does
turn out to be spaces and tabs in code, hopefully the script will
misbehave consistently and I will figure it out.


David



2020-09-21 13:01:00 ***@tinkywinky ~
$ cat /etc/debian_version ; uname -a ; dpkg-query --show xfce4 ;
dpkg-query --show vim
9.13
Linux tinkywinky 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05)
x86_64 GNU/Linux
xfce4 4.12.3
vim 2:8.0.0197-4+deb9u3


Test #1 -- copy and paste from cat(1) to vim(1)
===============================================

Terminal #1
-----------

2020-09-21 12:13:03 ***@tinkywinky ~
$ vi foo

2020-09-21 12:13:54 ***@tinkywinky ~
$ cat foo
I am foo!
leading tab
Foo on you!
trailing tab

2020-09-21 12:13:57 ***@tinkywinky ~
$ hexdump foo
00000000 49 20 61 6d 20 66 6f 6f 21 0a 09 6c 65 61 64 69 |I am
foo!..leadi|
00000010 6e 67 20 74 61 62 0a 46 6f 6f 20 6f 6e 20 79 6f |ng tab.Foo
on yo|
00000020 75 21 0a 74 72 61 69 6c 69 6e 67 20 74 61 62 09 |u!.trailing
tab.|
00000030 0a |.|
00000031

<mouse drag and highlight cat(1) output>


Terminal #2
-----------

2020-09-21 12:01:15 ***@tinkywinky ~
$ vi bar

<mouse middle click and paste text into edit buffer, save, exit>

2020-09-21 12:15:07 ***@tinkywinky ~
$ cat bar
I am foo!
leading tab
Foo on you!
trailing tab


2020-09-21 12:16:06 ***@tinkywinky ~
$ hexdump bar
00000000 49 20 61 6d 20 66 6f 6f 21 0a 09 6c 65 61 64 69 |I am
foo!..leadi|
00000010 6e 67 20 74 61 62 0a 46 6f 6f 20 6f 6e 20 79 6f |ng tab.Foo
on yo|
00000020 75 21 0a 74 72 61 69 6c 69 6e 67 20 74 61 62 09 |u!.trailing
tab.|
00000030 0a 0a |..|
00000032


Result: there is an extra newline at the end of 'bar'.


Test #2 -- copy and paste from less(1) to vim(1)
================================================

Terminal #3
-----------

2020-09-21 12:25:49 ***@tinkywinky ~
$ less foo

<mouse drag and highlight text within less(1) display>


Terminal #4
-----------

2020-09-21 12:26:25 ***@tinkywinky ~
$ vi baz

<mouse middle click and paste text into edit buffer, save, exit>

2020-09-21 13:06:14 ***@tinkywinky ~
$ cat baz
I am foo!
leading tab
Foo on you!
trailing tab

2020-09-21 13:06:16 ***@tinkywinky ~
$ hexdump baz
00000000 49 20 61 6d 20 66 6f 6f 21 0a 20 20 20 20 20 20 |I am foo!.
|
00000010 20 20 6c 65 61 64 69 6e 67 20 74 61 62 0a 46 6f | leading
tab.Fo|
00000020 6f 20 6f 6e 20 79 6f 75 21 0a 74 72 61 69 6c 69 |o on
you!.traili|
00000030 6e 67 20 74 61 62 20 20 20 20 0a 0a |ng tab ..|
0000003c


Result: Both tabs have been replaced with spaces and there is an extra
newline at the end of 'baz'.

[Gary Dale]

Post by Gary Dale
As an FYI, the scripts were the same on both servers because I had
ssh sessions on both and copy/pasted the script from one to the
other. (cat <script> on server 1, then use Konsole's copy and paste
to select and paste it into nano on server 2).

That process may have changed the file in invisible ways.  Use
hexdump(1) to check for differences.  See below examples.
Use scp(1), rsync(1), sshfs(1), a networked version control system
such as cvs(1), etc., to copy files accurately.

Post by Gary Dale
Most shells (all that I am aware of) treat tabs and blanks as generic
white space - not relevant to the execution of the code.

"I guess so".  (Obviously, I am not a dash(1) shell scripting expert.)
I have greater confidence in my ability to copy files accurately, and
thereby eliminate one set of potential problems.  If the problem does
turn out to be spaces and tabs in code, hopefully the script will
misbehave consistently and I will figure it out.
David
$ cat /etc/debian_version ; uname -a ; dpkg-query --show xfce4 ;
dpkg-query --show vim
9.13
Linux tinkywinky 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05)
x86_64 GNU/Linux
xfce4    4.12.3
vim    2:8.0.0197-4+deb9u3
Test #1 -- copy and paste from cat(1) to vim(1)
===============================================
Terminal #1
-----------
$ vi foo
$ cat foo
I am foo!
    leading tab
Foo on you!
trailing tab
$ hexdump foo
00000000  49 20 61 6d 20 66 6f 6f  21 0a 09 6c 65 61 64 69  |I am
foo!..leadi|
00000010  6e 67 20 74 61 62 0a 46  6f 6f 20 6f 6e 20 79 6f  |ng
tab.Foo on yo|
00000020  75 21 0a 74 72 61 69 6c  69 6e 67 20 74 61 62 09
|u!.trailing tab.|
00000030  0a                                                |.|
00000031
<mouse drag and highlight cat(1) output>
Terminal #2
-----------
$ vi bar
<mouse middle click and paste text into edit buffer, save, exit>
$ cat bar
I am foo!
    leading tab
Foo on you!
trailing tab
$ hexdump bar
00000000  49 20 61 6d 20 66 6f 6f  21 0a 09 6c 65 61 64 69  |I am
foo!..leadi|
00000010  6e 67 20 74 61 62 0a 46  6f 6f 20 6f 6e 20 79 6f  |ng
tab.Foo on yo|
00000020  75 21 0a 74 72 61 69 6c  69 6e 67 20 74 61 62 09
|u!.trailing tab.|
00000030  0a 0a                                             |..|
00000032
Result: there is an extra newline at the end of 'bar'.
Test #2 -- copy and paste from less(1) to vim(1)
================================================
Terminal #3
-----------
$ less foo
<mouse drag and highlight text within less(1) display>
Terminal #4
-----------
$ vi baz
<mouse middle click and paste text into edit buffer, save, exit>
$ cat baz
I am foo!
        leading tab
Foo on you!
trailing tab
$ hexdump baz
00000000  49 20 61 6d 20 66 6f 6f  21 0a 20 20 20 20 20 20  |I am
foo!.     |
00000010  20 20 6c 65 61 64 69 6e  67 20 74 61 62 0a 46 6f  | leading
tab.Fo|
00000020  6f 20 6f 6e 20 79 6f 75  21 0a 74 72 61 69 6c 69  |o on
you!.traili|
00000030  6e 67 20 74 61 62 20 20  20 20 0a 0a              |ng tab   
..|
0000003c
Result: Both tabs have been replaced with spaces and there is an extra
newline at the end of 'baz'.

The two servers are for different customers. I would not want to create
a tunnel between them. Instead I have my normal ssh tunnels to each
server from my workstation. However the script is only readable by root
while my tunnels are for my non-root account. While I could copy the
file to my non-root account (while root), chown it, copy it to my
workstation then to the other server, where I'd move it to /root, that's
a lot more work than cat, copy, paste, save.

Again, the method I used should not have created any changes in the
script that would affect its operation. And to date I've seen no
indication that it did. I still don't know why the script was leaving
the quotes in nor why it started working.

[David Christensen]

Post by Gary Dale
The two servers are for different customers. I would not want to create
a tunnel between them. Instead I have my normal ssh tunnels to each
server from my workstation. However the script is only readable by root
while my tunnels are for my non-root account. While I could copy the
file to my non-root account (while root), chown it, copy it to my
workstation then to the other server, where I'd move it to /root, that's
a lot more work than cat, copy, paste, save.
Again, the method I used should not have created any changes in the
script that would affect its operation. And to date I've seen no
indication that it did. I still don't know why the script was leaving
the quotes in nor why it started working.

You might want to consider ssh-agent and SSH agent forwarding. These
allow you to access your version control server over SSH from remote
hosts by using your workstation credentials; no credentials required on
the remote host:


https://dev.to/levivm/how-to-use-ssh-and-ssh-agent-forwarding-more-secure-ssh-2c32


David

[Gary Dale]

Post by Gary Dale
The two servers are for different customers. I would not want to
create a tunnel between them. Instead I have my normal ssh tunnels to
each server from my workstation. However the script is only readable
by root while my tunnels are for my non-root account. While I could
copy the file to my non-root account (while root), chown it, copy it
to my workstation then to the other server, where I'd move it to
/root, that's a lot more work than cat, copy, paste, save.
Again, the method I used should not have created any changes in the
script that would affect its operation. And to date I've seen no
indication that it did. I still don't know why the script was leaving
the quotes in nor why it started working.

You might want to consider ssh-agent and SSH agent forwarding. These
allow you to access your version control server over SSH from remote
hosts by using your workstation credentials; no credentials required
https://dev.to/levivm/how-to-use-ssh-and-ssh-agent-forwarding-more-secure-ssh-2c32
David

I'm not sure that does anything for me. I would need to create a "root"
key to get access to the file, which is something I refuse to do.

Right now the ssh tunnel requires a key on the remote server and there
are no root keys so even if someone gains access, they still don't have
root access.

There are other tools that work better for pushing things to multiple
servers but all of these tools assume you are doing it often enough or
to enough machines to make it worthwhile. That's not my situation.

[David Christensen]

Post by Gary Dale

Post by Gary Dale
The two servers are for different customers. I would not want to
create a tunnel between them. Instead I have my normal ssh tunnels to
each server from my workstation. However the script is only readable
by root while my tunnels are for my non-root account. While I could
copy the file to my non-root account (while root), chown it, copy it
to my workstation then to the other server, where I'd move it to
/root, that's a lot more work than cat, copy, paste, save.
Again, the method I used should not have created any changes in the
script that would affect its operation. And to date I've seen no
indication that it did. I still don't know why the script was leaving
the quotes in nor why it started working.

You might want to consider ssh-agent and SSH agent forwarding. These
allow you to access your version control server over SSH from remote
hosts by using your workstation credentials; no credentials required
https://dev.to/levivm/how-to-use-ssh-and-ssh-agent-forwarding-more-secure-ssh-2c32
David

I'm not sure that does anything for me. I would need to create a "root"
key to get access to the file, which is something I refuse to do.

This book is relevant:

https://mwl.io/nonfiction/tools#ssh


Here is my workstation. I will demonstrate SSH agent forwarding using
it both as the workstation and as the server:

2020-09-22 14:57:40 ***@tinkywinky ~
# cat /etc/debian_version ; uname -a
9.13
Linux tinkywinky 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05)
x86_64 GNU/Linux


The workstation and the server must have at least one SSH host key pair.
The Debian installer creates three:

2020-09-22 14:49:56 ***@tinkywinky ~
$ ls -1 /etc/ssh/ssh_host*key*
/etc/ssh/ssh_host_ecdsa_key
/etc/ssh/ssh_host_ecdsa_key.pub
/etc/ssh/ssh_host_ed25519_key
/etc/ssh/ssh_host_ed25519_key.pub
/etc/ssh/ssh_host_rsa_key
/etc/ssh/ssh_host_rsa_key.pub


The workstation user account must have at least one SSH user key pair:

2020-09-22 14:51:21 ***@tinkywinky ~
$ ls -1 .ssh/id_rsa*
.ssh/id_rsa
.ssh/id_rsa.pub


No root account key pair is required on the server.


The server root account authorized_keys file must contain the
workstation user account public key(s). Append the contents of
~/.ssh/id_rsa.pub from the workstation to /root/.ssh/authorized_keys on
the server.


Agent forwarding must be enabled on the workstation:

2020-09-22 14:55:20 ***@tinkywinky ~
$ egrep ^ForwardAgent /etc/ssh/ssh_config
ForwardAgent yes


Root public key authentication must be enabled on the server (it is
enabled by default on my Debian machines):

2020-09-22 14:56:32 ***@tinkywinky ~
$ egrep PermitRootLogin /etc/ssh/sshd_config | head -n 1
PermitRootLogin prohibit-password


Restart sshd on the server, if required:

2020-09-22 14:57:35 ***@tinkywinky ~
# service sshd restart


Start a terminal on the workstation. Start ssh-agent(1) using your
shell of choice (with any required options or arguments):

2020-09-22 15:01:31 ***@tinkywinky ~
$ ssh-agent bash -l


Add your user account key pair to the SSH agent:

2020-09-22 15:01:34 ***@tinkywinky ~
$ ssh-add
Enter passphrase for /home/dpchrist/.ssh/id_rsa:
Identity added: /home/dpchrist/.ssh/id_rsa (/home/dpchrist/.ssh/id_rsa)


Log in as root on the server via public key authentication (no
passphrase required):

2020-09-22 15:02:09 ***@tinkywinky ~
$ ssh ***@localhost
Linux tinkywinky 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05) x86_64
Last login: Tue Sep 22 14:57:35 2020 from ::1

2020-09-22 15:02:35 ***@tinkywinky ~
#


Now I can use my workstation user account credentials on the remote
machine just like I would use them locally (via agent forwarding; no
passphrase required):

2020-09-22 15:02:35 ***@tinkywinky ~
# cvs diff


Optionally, you can disable password authentication on the server and
restart sshd. The only way to log in will be via the console or via SSH
public key authentication:

2020-09-22 15:04:40 ***@tinkywinky ~
# grep Password /etc/ssh/sshd_config
PasswordAuthentication no

2020-09-22 15:22:05 ***@tinkywinky ~
# service sshd restart

Post by Gary Dale
Right now the ssh tunnel requires a key on the remote server

I don't use SSH tunnels (yet). The SSH agent forwarding technique I
demonstrated above does not require any user or root keys on the server.

Post by Gary Dale
and there
are no root keys so even if someone gains access, they still don't have
root access.

If someone gains access to what?

Post by Gary Dale
There are other tools that work better for pushing things to multiple
servers but all of these tools assume you are doing it often enough or
to enough machines to make it worthwhile. That's not my situation.

Ansible, Puppet, etc., seem like overkill for my needs. I have not used
them.


I use ssh-agent(1), scp(1), and ssh(1) interactively, and I wrote shell
scripts, Perl scripts, and Makefiles to automate repetitive tasks.


CVS allows me to develop scripts and Makefiles on any machine, check
them in, and check out exact copies on every other machine. If I edit
one file on two machines, CVS knows how to solve the three-way merge
problem (for text files, not for binary files). CVS is very useful,
saves time, and eliminates the chaos of sneakernet, cut-and-paste, etc..
Any other good networked version control system should do the same.


David

[Greg Wooledge]

Post by Gary Dale

Post by David Christensen

Post by Gary Dale
The line causing the problem reads: report="/root/clamscan-report"

There is nothing wrong with that alleged line.

There is an incredible lack of openness and forthrightness in this
thread. What are you hiding? Why are you hiding it?

Post by Gary Dale

Post by David Christensen
Have you tried single quotes?

Wouldn't want to. Single quotes alter the behaviour.

False. Either type of quotes is perfectly fine in this case.

The fact that you make this claim tells me one of the following two
things is true:

* You have no idea what you're doing, and you are GUESSING that
changing the quotes would cause an issue, but you did not actually
try it.

* The whole discussion is founded on falsehoods. There's something
you're not telling us. The "line" in question is falsified, or it's
part of some larger context which is being hidden, and this context
is vitally important to understanding the problem.

I'm leaning toward the latter.

What you should do is produce the smallest possible script that still
exhibits the problem, and post that script, plus its output. Obviously
you'd add something like echo "$report" to the script. Even better
would be printf '%s\n' "$report" . Better still would be
printf %s "$report" | hd . Or you could just add set -x to the script.

[Gary Dale]

Post by Greg Wooledge

Post by Gary Dale

Post by David Christensen

Post by Gary Dale
The line causing the problem reads: report="/root/clamscan-report"

There is nothing wrong with that alleged line.
There is an incredible lack of openness and forthrightness in this
thread. What are you hiding? Why are you hiding it?

Nothing. Just not sharing extraneous details.

Post by Greg Wooledge

Post by Gary Dale

Post by David Christensen
Have you tried single quotes?

Wouldn't want to. Single quotes alter the behaviour.

False. Either type of quotes is perfectly fine in this case.

The key words being "in this case". As I explained elsewhere in this
thread, I avoid using single-quotes except when I need them. Otherwise
anyone reading the script wouldn't know if the single quotes were needed
or not. This is fairly standard practise, as far as I've seen, when
writing scripts.

Post by Greg Wooledge
The fact that you make this claim tells me one of the following two
* You have no idea what you're doing, and you are GUESSING that
changing the quotes would cause an issue, but you did not actually
try it.
* The whole discussion is founded on falsehoods. There's something
you're not telling us. The "line" in question is falsified, or it's
part of some larger context which is being hidden, and this context
is vitally important to understanding the problem.
I'm leaning toward the latter.
What you should do is produce the smallest possible script that still
exhibits the problem, and post that script, plus its output. Obviously
you'd add something like echo "$report" to the script. Even better
would be printf '%s\n' "$report" . Better still would be
printf %s "$report" | hd . Or you could just add set -x to the script.

The smallest possible script was what I posted - the single line copied
and pasted from the script it was running in. It was the first line
after the shebang.

[Gary Dale]

Post by The Wanderer

Post by Gary Dale
I have the same bash script on two different Debian/Buster AMD64
servers. However on one it refused to run. I tracked it down quickly
to a variable substitution problem.
The line causing the problem reads: report="/root/clamscan-report"
On one server echo $report prints /root/clamscan-report while on
the other it prints "/root/clamscan-report".
Needless to say clamscan can't print to the latter. I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want.

Given the lack of spaces or other potentially-problematic characters in
the path, why not remove them on both servers? Will there potentially be
cases where the path is different, and such characters may be present?

I'm in the habit of always quoting string constants in scripts. It's
clearer and avoids potential problems with future edits...

Post by The Wanderer

Post by Gary Dale
More importantly, I don't understand why it refuses to remove the
quotes.
Where does this behaviour (keeping the quotes) get set?

First up: are you sure it's actually bash (rather than some other shell)
that's running the script in both cases?
Second, can you confirm that bash is the same version on both servers?
As reported by e.g 'bash --version'.
If both of those are confirmed, then it may be worth digging deeper. I'm
a bit reluctant to delve deep into the bash man page looking for
something like that without first ruling out other possibilities.
My first guess would be that one of the two might be using the shell
builtin command 'echo' and the other might be using /bin/echo, but
that's only a guess.

It's the same version of bash, as you would expect since both servers
are running the same up-to-date version of Debian/Stable.

The echo command is reporting things accurately. I uncovered the problem
by running the script manually and seeing the messages coming up showing
that it was looking for a file enclosed in quotes. The other server was
running the script without errors.

[t***@tuxteam.de]

Post by Gary Dale
I have the same bash script on two different Debian/Buster AMD64
servers. However on one it refused to run. I tracked it down quickly
to a variable substitution problem.
The line causing the problem reads: report="/root/clamscan-report"
On one server  echo $report  prints  /root/clamscan-report  while on
the other it prints  "/root/clamscan-report".
Needless to say clamscan can't print to the latter. I fixed it by
removing the quotes on the one server but now the scripts are
different between the two servers, which isn't what I want. More
importantly, I don't understand why it refuses to remove the quotes.
Where does this behaviour (keeping the quotes) get set?

What does the builtin "shopt" say? Especially the value of `compat42'
is involved in quote removal.

Cheers
- t

[David Christensen]

Post by t***@tuxteam.de

Post by Gary Dale
Where does this behaviour (keeping the quotes) get set?

What does the builtin "shopt" say? Especially the value of `compat42'
is involved in quote removal.

Joy! Joy! Happy! Happy! Another whole dimension for potential problems!


2020-09-21 12:47:06 ***@tinkywinky ~
$ cat /etc/debian_version ; uname -a ; dpkg-query --show dash
9.13
Linux tinkywinky 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05)
x86_64 GNU/Linux
dash 0.5.8-2.4

2020-09-21 12:48:31 ***@tinkywinky ~
$ shopt
autocd off
cdable_vars off
cdspell off
checkhash off
checkjobs off
checkwinsize on
cmdhist on
compat31 off
compat32 off
compat40 off
compat41 off
compat42 off
compat43 off
complete_fullquote on
direxpand off
dirspell off
dotglob off
execfail off
expand_aliases on
extdebug off
extglob on
extquote on
failglob off
force_fignore on
globasciiranges off
globstar off
gnu_errfmt off
histappend off
histreedit off
histverify off
hostcomplete off
huponexit off
inherit_errexit off
interactive_comments on
lastpipe off
lithist off
login_shell on
mailwarn off
no_empty_cmd_completion off
nocaseglob off
nocasematch off
nullglob off
progcomp on
promptvars on
restricted_shell off
shift_verbose off
sourcepath on
xpg_echo off


RTFM dash(1) I don't see any 'compat' keywords... (?)


STFW https://html.duckduckgo.com/html?q=dash%20compat42 I don't see
'compat' anywhere on the first page of hits... (?)


Where are the above settings documented?


David

[Greg Wooledge]

Post by David Christensen

Post by t***@tuxteam.de
What does the builtin "shopt" say? Especially the value of `compat42'
is involved in quote removal.

Joy! Joy! Happy! Happy! Another whole dimension for potential problems!

There is great confusion here, because the OP was so vague and misleading
about *everything*. Including which shell was being used.

"compat42" and "shopt" are bashisms. They are not present nor relevant
in dash.

Post by David Christensen
RTFM dash(1) I don't see any 'compat' keywords... (?)

Correct.

Post by David Christensen
STFW https://html.duckduckgo.com/html?q=dash%20compat42 I don't see 'compat'
anywhere on the first page of hits... (?)
Where are the above settings documented?

In bash(1). They are bash settings.

To the best of my knowledge, there is no bash shopt or compat* setting
that will change the behavior of quotes in such a profound way as to
cause the problem that the OP imagined he had.

At this point, I'm still convinced that it was a curly-quote issue. Of
course, we'll never know for sure, because the OP is incapable of presenting
simple facts like "here is the script which fails" and "here is its output".

[Gary Dale]

Post by Greg Wooledge

Post by David Christensen

Post by t***@tuxteam.de
What does the builtin "shopt" say? Especially the value of `compat42'
is involved in quote removal.

Joy! Joy! Happy! Happy! Another whole dimension for potential problems!

There is great confusion here, because the OP was so vague and misleading
about *everything*. Including which shell was being used.
"compat42" and "shopt" are bashisms. They are not present nor relevant
in dash.

Post by David Christensen
RTFM dash(1) I don't see any 'compat' keywords... (?)

Correct.

Post by David Christensen
STFW https://html.duckduckgo.com/html?q=dash%20compat42 I don't see 'compat'
anywhere on the first page of hits... (?)
Where are the above settings documented?

In bash(1). They are bash settings.
To the best of my knowledge, there is no bash shopt or compat* setting
that will change the behavior of quotes in such a profound way as to
cause the problem that the OP imagined he had.
At this point, I'm still convinced that it was a curly-quote issue. Of
course, we'll never know for sure, because the OP is incapable of presenting
simple facts like "here is the script which fails" and "here is its output".

I presented the line that failed, copied and pasted from the Konsole
session. What more do you want, other than to complain?

[Andrei POPESCU]

Post by Gary Dale
I presented the line that failed, copied and pasted from the Konsole
session. What more do you want, other than to complain?

In such cases it is best to attach[1] the smallest complete[2] script
demonstrating the behaviour.

Based on the information provided so far it is highly likely the issue
was caused by some detail you omitted because you decided it is not
relevant.

[1] as demonstrated, copy-pasting can alter content
[2] as discussed, the shebang line can make a big difference

Kind regards,
Andrei

[Gary Dale]

Post by Andrei POPESCU

Post by Gary Dale
I presented the line that failed, copied and pasted from the Konsole
session. What more do you want, other than to complain?

In such cases it is best to attach[1] the smallest complete[2] script
demonstrating the behaviour.
Based on the information provided so far it is highly likely the issue
was caused by some detail you omitted because you decided it is not
relevant.
[1] as demonstrated, copy-pasting can alter content
[2] as discussed, the shebang line can make a big difference
Kind regards,
Andrei

Your first point makes it impossible for me to present anything because
this list doesn't (AFAIK) allow attachments. I can only present a
copy-pasted example.

Your second point would assume that I'm using a non-standard shebang. I
use #!/bin/sh unless I need something that is only available in bash
(can't recall a case of that). Moreover, the consensus seems to be that
if there was an error on the shebang line, the default shell would be used.

The smallest complete script then is

report=”/root/clamscan-report”
echo $report

but as I explained, that works on one server but not the other. Hence my
question wasn't "what's wrong with this line" but rather "how do I
change the behaviour". I'm not expecting anyone else to be able to
reproduce the problem because I can't even do it except on the one server.

And to do that, I didn't even need the shebang line. I could enter the
line at the command prompt then echo it and see the same result. Since
it wasn't the shebang line, and didn't even require being in a script,
why should I have included the shebang line?

[Greg Wooledge]

Post by Gary Dale
The smallest complete script then is
report=”/root/clamscan-report”
echo $report

There are three errors in this script.

1) There is no shebang line.
2) You've got curly quotes instead of ASCII double-quotes.
3) You used a variable expansion without enclosing it in double quotes,
in a context where that matters.

If you literally can't see the difference between ”/root/clamscan-report”
and "/root/clamscan-report" (the former is what you have, and the latter
is what it *should* be), then you need to start working with a different
font.

[Andrei POPESCU]

Your first point makes it impossible for me to present anything because this
list doesn't (AFAIK) allow attachments.

This is a myth, I just attached a small script to this message.

The list apparently does indeed filter some attachments most likely
based on size, possibly also type.

https://www.debian.org/MailingLists/#codeofconduct

Kind regards,
Andrei