switching from exim to postfix

[Marco d'Itri]

Is this the right time to do it?

[Russ Allbery]

Post by Marco d'Itri
Is this the right time to do it?

I'm not sure that I see the point, and I say that as someone who replaces
Exim with Postfix on all of my boxes.

There's nothing particularly wrong with Exim; it works just fine. It's
been the default in Debian for years, and it's actively maintained
upstream. And it's completely trivial to replace it with Postfix if one
desires. The disruption doesn't seem worth it even if we had consensus
that Postfix was marginally better in some way.

[Andrei POPESCU]

Post by Russ Allbery
There's nothing particularly wrong with Exim; it works just fine. It's
been the default in Debian for years, and it's actively maintained
upstream. And it's completely trivial to replace it with Postfix if one
desires. The disruption doesn't seem worth it even if we had consensus
that Postfix was marginally better in some way.

What about dma?

Kind regards,
Andrei

[George Danchev]

On Sun, 29 Apr 2012 12:05:06 +0300, Andrei POPESCU

Post by Andrei POPESCU

Post by Russ Allbery
There's nothing particularly wrong with Exim; it works just fine.
It's
been the default in Debian for years, and it's actively maintained
upstream. And it's completely trivial to replace it with Postfix if one
desires. The disruption doesn't seem worth it even if we had
consensus
that Postfix was marginally better in some way.

What about dma?

Well, dma does not listen on port 25 for incoming connections, it
accepts
mails from local MUAs and delivers them to either the local mailboxes
or remote
(full-fledged) SMTP servers. I'm sure there would be people claiming
that such a
feature reduction, read simplification, would be a decent default.

[Marco d'Itri]

Post by Russ Allbery
desires. The disruption doesn't seem worth it even if we had consensus

What kind of disruption are you thinking about?

[Russ Allbery]

Post by Marco d'Itri

Post by Russ Allbery
desires. The disruption doesn't seem worth it even if we had consensus

What kind of disruption are you thinking about?

Existing users who are familiar with Exim and who would get Postfix on a
new install and be surprised.

The giant endless flamewars on debian-devel required to make a decision to
change anything. :)

[Julien Cristau]

Post by Russ Allbery

Post by Marco d'Itri

Post by Russ Allbery
desires. The disruption doesn't seem worth it even if we had consensus

What kind of disruption are you thinking about?

Existing users who are familiar with Exim and who would get Postfix on a
new install and be surprised.
The giant endless flamewars on debian-devel required to make a decision to
change anything. :)

The 500 packages that would have to change their Depends from "exim4 |
mta" to something else.

Cheers,
Julien

[Roger Leigh]

Post by Julien Cristau
The 500 packages that would have to change their Depends from "exim4 |
mta" to something else.

The brokenness of having to have a default package hardcoded in
every virtual dependency rather than having a virtual defaults
package and/or central list is an insanity I wish we could have
fixed years ago. We should not have global policy WRT virtual
package defaults embedded (inconsistently!) into every single
dependency!


Roger

[Holger Levsen]

Hi,

Post by Roger Leigh

Post by Julien Cristau
The 500 packages that would have to change their Depends from "exim4 |
mta" to something else.

The brokenness of having to have a default package hardcoded in
every virtual dependency rather than having a virtual defaults
package and/or central list is an insanity I wish we could have
fixed years ago.

yeah, if only people used the bts more often ;) This problem is being tracked
since 4 years...

If the BTS is right, it's only three packages blocking #508644
"Sorting out mail-transport-agent mess":

Fix blocked by 645024: fwknop-server: dependencies spell default-mta wrong,
645022: nmh: please change Recommends to default-mta | mail-transport-agent,
645020: pyca: please change Depends to default-mta | mail-transport-agent

;-)


cheers,
Holger

P.S.: please only reply to the bug#, that will make sure it reaches debian-
devel@ - I just cc:ed the list to keep the thread views intact...

[Russ Allbery]

Post by Julien Cristau
The 500 packages that would have to change their Depends from "exim4 |
mta" to something else.

Well, it would be nice to change all of those to depend on default-mta |
mail-transport-agent anyway, but yeah. Making that low-priority change
urgent would be sort of annoying.

[Raphael Hertzog]

Post by Julien Cristau
The 500 packages that would have to change their Depends from "exim4 |
mta" to something else.

We're already on our way to update them with "default-mta |
mail-transport-agent".

That would provide an incentive to finish converting the dependencies :-)

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/

[Julien Cristau]

Post by Raphael Hertzog

Post by Julien Cristau
The 500 packages that would have to change their Depends from "exim4 |
mta" to something else.

We're already on our way to update them with "default-mta |
mail-transport-agent".

On a very slow way. I know. That's beside the point.

Post by Raphael Hertzog
That would provide an incentive to finish converting the dependencies :-)

If you need disruption to do it then probably changing the default isn't
all that important.

Cheers,
Julien

[Marco d'Itri]

Post by Russ Allbery

Post by Marco d'Itri
What kind of disruption are you thinking about?

Existing users who are familiar with Exim and who would get Postfix on a
new install and be surprised.

This does not really look like a big surprise.
If somebody is familiar enough with Exim to modify the default configuration
then I think it is safe to assume that he also knows how to install the
package.

Post by Russ Allbery
The giant endless flamewars on debian-devel required to make a decision to
change anything. :)

IIRC, last time we discussed this I think that even the exim maintainers
were in favour of the change...

[Andrey Rahmatullin]

Post by Marco d'Itri

Post by Russ Allbery
The giant endless flamewars on debian-devel required to make a decision to
change anything. :)

IIRC, last time we discussed this I think that even the exim maintainers
were in favour of the change...

What were the reasons?

[Marco d'Itri]

Post by Russ Allbery
The giant endless flamewars on debian-devel required to make a decision to
change anything. :)

Unrelated: you have just shown what poisons Debian and has been keeping
us behind innovation for the last years.
Not the flamewars themselves, most of us are grown ups and can handle
them, but the fear that proposing a change will cause endless
discussions and no results.

[Stefano Zacchiroli]

Post by Marco d'Itri

Post by Russ Allbery
The giant endless flamewars on debian-devel required to make a decision to
change anything. :)

Unrelated: you have just shown what poisons Debian and has been keeping
us behind innovation for the last years.
Not the flamewars themselves, most of us are grown ups and can handle
them, but the fear that proposing a change will cause endless
discussions and no results.

Still unrelated, but let me AOL the above. Russ was just joking, but I
completely agree with Marco's point here. We're often scared of
"starting a thread on -devel" to propose changes that are far reaching
enough not to be directly implementable. Every now and then I got asked
advice on whether proposing some change on -devel is a good idea or not.
I'm always happy to give advice, but the fact we sometime worry to
propose changes is worrisome in itself. And it can induce project-wide
inertia as much as worrying too much about performing NMUs to fix bugs.

But we also need to convince ourselves that -devel discussions are
useful and lead to progress. For that to happen, we need more people
that look back at past discussions, summarize their conclusions (if
there have been) or relaunch them (if not), and take concrete actions as
the natural next step of discussing. There are people doing that, but
not nearly enough.

Cheers.

[Russ Allbery]

Post by Stefano Zacchiroli

Post by Marco d'Itri
Unrelated: you have just shown what poisons Debian and has been keeping
us behind innovation for the last years. Not the flamewars themselves,
most of us are grown ups and can handle them, but the fear that
proposing a change will cause endless discussions and no results.

Still unrelated, but let me AOL the above. Russ was just joking, but I
completely agree with Marco's point here. We're often scared of
"starting a thread on -devel" to propose changes that are far reaching
enough not to be directly implementable. Every now and then I got asked
advice on whether proposing some change on -devel is a good idea or not.
I'm always happy to give advice, but the fact we sometime worry to
propose changes is worrisome in itself. And it can induce project-wide
inertia as much as worrying too much about performing NMUs to fix bugs.

Yes, I was just joking and I'm inclined to agree. (Although with respect
to Marco's specific proposal, there *is* a cost to debate and a request
for everyone's attention, and unless the gains are particularly clear, I'm
not sure it's worth incurring that cost.)

Post by Stefano Zacchiroli
But we also need to convince ourselves that -devel discussions are
useful and lead to progress. For that to happen, we need more people
that look back at past discussions, summarize their conclusions (if
there have been) or relaunch them (if not), and take concrete actions as
the natural next step of discussing. There are people doing that, but
not nearly enough.

Given recent experiences, I'm also coming around to Ian's position that
aggressive and confrontational contributions from people who don't
otherwise seem to be contributing to Debian are part of the problem and
are not useful, and possibly should be banned. I think that's been a
significant factor in the deterioration of the init system threads.

I want our technical discussions to be welcoming to anyone who has
information to share and who can bring additional clarity and insight to
the discussion. But once things start getting heated or people start
throwing around accusations or verge towards personal attacks, there's a
real psychological difference between people who are contributing to
Debian and people who aren't.

If I'm being attacked by a colleague, it's a lot easier for me to go
"well, that made me mad, but they've done a great job of maintaining this
package I use and I've seen them do lots of other work for Debian, so they
must just be having a bad day" and let it go. There's some built-up good
will because they're part of the community.

When that sort of attack comes from someone who I've never heard of
before, and when I then go to the PTS and db.debian.org and find no track
record of that person ever contributing to Debian other than via mailing
list discussions, it's a lot harder to give them that benefit of the
doubt. And it gets really frustrating to have to keep discussing and
defending decisions with people who don't appear to be doing anything for
Debian other than feeding discussions that are a net drain of energy.

I'm leery of this whole line of argument, since it's inherently a double
standard. That's why I've not raised it before. But, well, humans are
social animals and social dynamics involve some degree of double standard,
and it would be nice if people who were effectively guests in our
technical discussions would behave like house guests rather than diving in
with the degree of robust engagement that (while never exactly ideal) our
long-time contributors have to some degree made up for in advance.

[Bernhard R. Link]

Post by Russ Allbery
I want our technical discussions to be welcoming to anyone who has
information to share and who can bring additional clarity and insight to
the discussion. But once things start getting heated or people start
throwing around accusations or verge towards personal attacks, there's a
real psychological difference between people who are contributing to
Debian and people who aren't.

I'd rather argue that abusive behaviour from contributors is far worse
than from non-contributors. It's easier to ignore people not involved
and people not doing anything are usually not around for long.

There is also nothing keeping anyone with technical arguments out of a
discussion as someone insulting anyone with different opinions and if
running out of insults accusing people as not being contributors.

My suggestion to everyone feeling the need to tell anyone on a public
mailing list that they should shut up because they are no contributors
is thus: Please refrain from any more posts to this discussion. I do
not care if you rationalize it as "no need to feed the troll" or if
you understand you left the level of technical discussion and have
little chance to come back to it till the discussion will be over, but
once that point is reached there really is no sense it keeping it up.

Bernhard R. Link

[David Bremner]

Post by Bernhard R. Link
My suggestion to everyone feeling the need to tell anyone on a public
mailing list that they should shut up because they are no contributors
is thus: Please refrain from any more posts to this discussion.

I have nothing against this principle, and I do this. But I also stop
reading such threads. And this means I read less and less of this list.

d

[Russ Allbery]

Post by David Bremner

Post by Bernhard R. Link
My suggestion to everyone feeling the need to tell anyone on a public
mailing list that they should shut up because they are no contributors
is thus: Please refrain from any more posts to this discussion.

I have nothing against this principle, and I do this. But I also stop
reading such threads. And this means I read less and less of this list.

Right. As good as that idea sounds on the surface, what that actually
translates into in practice is making debian-devel useless.

[Bernhard R. Link]

Post by Russ Allbery

Post by David Bremner

Post by Bernhard R. Link
My suggestion to everyone feeling the need to tell anyone on a public
mailing list that they should shut up because they are no contributors
is thus: Please refrain from any more posts to this discussion.

I have nothing against this principle, and I do this. But I also stop
reading such threads. And this means I read less and less of this list.

Right. As good as that idea sounds on the surface, what that actually
translates into in practice is making debian-devel useless.

And how does enhancing the noise rate by adding mails not about technical
arguments make the mailing lists useful?

Bernhard R. Link

[Russ Allbery]

Post by Bernhard R. Link

Post by Russ Allbery

Post by David Bremner

Post by Bernhard R. Link
My suggestion to everyone feeling the need to tell anyone on a public
mailing list that they should shut up because they are no contributors
is thus: Please refrain from any more posts to this discussion.

I have nothing against this principle, and I do this. But I also stop
reading such threads. And this means I read less and less of this list.

Right. As good as that idea sounds on the surface, what that actually
translates into in practice is making debian-devel useless.

And how does enhancing the noise rate by adding mails not about
technical arguments make the mailing lists useful?

That's why I drew the distinction between "on the surface" and "in
practice." On the surface, it's a good idea because one doesn't add to
the noise. In practice, it leaves the problem unaddressed and technical
contributors just leave.

Telling people that there's nothing they can do about the noise and they
should just give up and ignore it means that people will stop reading the
mailing list and the only people left to have discussions are the people
who enjoy the noise. I don't want technical decisions in this project to
only be discussed by people who enjoy the noise.

So, while "don't add to the noise" is *part* of the solution, if one just
says that and puts a period at the end, it makes the problem worse. It
needs to be part of a solution that actually *reduces the noise*.

[Bernhard R. Link]

Post by Russ Allbery
I don't want technical decisions in this project to
only be discussed by people who enjoy the noise.

That's why it is cruical to get the noise reduced. If in any discussion
there is a DD escalating the flames then there won't be any people with
technical arguments left. Getting some non-contributers out of the
picture will not change it much. I wholeheartly believe that the only
reason you see those people so prominently is that everything else
already is in ignore mode because of contributors heating the flames.

Post by Russ Allbery
So, while "don't add to the noise" is *part* of the solution, if one just
says that and puts a period at the end, it makes the problem worse. It
needs to be part of a solution that actually *reduces the noise*.

Not adding to the noise is reducing the noise. And especially telling
people that you do not care about their arguments because you they are
not insiders, which this is from some point of view, is the noise that
makes an discussion in my eyes the most unwelcoming and thus a good
reason to only expect noise and no more signal.

Bernhard R. Link

[Raphael Hertzog]

Hi,

Post by Bernhard R. Link
Not adding to the noise is reducing the noise. And especially telling
people that you do not care about their arguments because you they are
not insiders, which this is from some point of view, is the noise that
makes an discussion in my eyes the most unwelcoming and thus a good
reason to only expect noise and no more signal.

Which is why I reply to those "outsiders" privately and gently point
them to a page that lists some of the mistakes they are doing
(FTR it's http://raphaelhertzog.com/go/ml/).

I certainly agree with Russ that several non-contributors had a
significant negative impact on recent discussions and that we ought
to be doing something about this.

When I see that the bad patterns tend to continue, I mail the listmasters
and ask them to send a warning to to the person. If enough persons
complain, they might even put a filter if that person doesn't stop.

But it's difficult to do it on a regular basis because:
- if I'm alone doing it, it won't have much impact
- given I have no way to know that others are doing the same, I tend to
assume that it doesn't help much, and thus loses some of the motivation
to draft gentle replies pointing out the problematic behaviour

Maybe we need a private DD-only list where people interested in
improving our lists can CC their private complaints. listmasters
could follow the list and take action when they notice that
the same person got multiple complaints.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/

[Chris Knadle]

On Thursday, May 03, 2012 02:50:41, Raphael Hertzog wrote:
...

Post by Raphael Hertzog
When I see that the bad patterns tend to continue, I mail the listmasters
and ask them to send a warning to to the person. If enough persons
complain, they might even put a filter if that person doesn't stop.
- if I'm alone doing it, it won't have much impact
- given I have no way to know that others are doing the same, I tend to
assume that it doesn't help much, and thus loses some of the motivation
to draft gentle replies pointing out the problematic behaviour
Maybe we need a private DD-only list where people interested in
improving our lists can CC their private complaints. listmasters
could follow the list and take action when they notice that
the same person got multiple complaints.

FWIW there's a [debian-private] mailing list:
debian-private: Private discussions among developers

and this list is not archived.

-- Chris

--
Chris Knadle
***@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74

[Raphael Hertzog]

Post by Chris Knadle

Post by Raphael Hertzog
Maybe we need a private DD-only list where people interested in
improving our lists can CC their private complaints. listmasters
could follow the list and take action when they notice that
the same person got multiple complaints.

debian-private: Private discussions among developers
and this list is not archived.

Heh, thank you for the notice. But I knew about this list (and any DD knows
about it since most of us are subscribed and it's one of the first things
that you do when you get the DD status) and it's really not its purpose.

I believe that such mails should not be imposed to debian-private readers
but only to people who specifically opted to help "moderate" our mailing
lists through "social influence".

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/

[Alexander Wirt]

Post by Raphael Hertzog

Post by Chris Knadle

Post by Raphael Hertzog
Maybe we need a private DD-only list where people interested in
improving our lists can CC their private complaints. listmasters
could follow the list and take action when they notice that
the same person got multiple complaints.

debian-private: Private discussions among developers
and this list is not archived.

Heh, thank you for the notice. But I knew about this list (and any DD knows
about it since most of us are subscribed and it's one of the first things
that you do when you get the DD status) and it's really not its purpose.
I believe that such mails should not be imposed to debian-private readers
but only to people who specifically opted to help "moderate" our mailing
lists through "social influence".

I am just speaking for myself as listmaster. But I don't think any DD has
more "right" to talk on a mailinglist than anybody else. I won't support such
a proposal nor want I participate in it. If you have a problem with someone
on a mailinglist, report it and listmasters decide if we should step in.

Alex - with his - personal - listmaster hat on

[Miles Bader]

Post by Alexander Wirt
I am just speaking for myself as listmaster. But I don't think any
DD has more "right" to talk on a mailinglist than anybody else. I
won't support such a proposal nor want I participate in it. If you
have a problem with someone on a mailinglist, report it and
listmasters decide if we should step in.

... and as a non-DD who's been using Debian for 15 years (and reading
this list for many of them), and understands at least some of the
technical issues, I find the suggestion that I be automatically
considered a negative influence and excluded kind of annoying.

The issues discussed here often do affect me, because I use Debian. I
don't actively participate most of the time but I do read, and every
once in a while, feel I have something to add.

The problem is not non-DDs, it's jerks and/or the clueless. Maybe on
this list there's _some_ correlation between "non-DDness" and those
things -- but it's far from perfect... (and not, IMHO, enough to
justify censorship).

-miles

[Milan P. Stanic]

Post by Miles Bader

Post by Alexander Wirt
I am just speaking for myself as listmaster. But I don't think any
DD has more "right" to talk on a mailinglist than anybody else. I
won't support such a proposal nor want I participate in it. If you
have a problem with someone on a mailinglist, report it and
listmasters decide if we should step in.

... and as a non-DD who's been using Debian for 15 years (and reading
this list for many of them), and understands at least some of the
technical issues, I find the suggestion that I be automatically
considered a negative influence and excluded kind of annoying.

I'm in the same bandwagon.
Sometimes I even package some packages which are not in Debian for some
users. Few years ago I backported selinux to Sarge to Woody (IIRC) and
some people from over the world downloaded it and used or played with
it. These days I maintain Kannel development release (packages are on
the Kannel site) for Debian Testing and people use it.

Do I help Debian? I really don't know but I'm sure that I did help some
Debian users.

This (and some other) Debian list are helpful for me and I sometimes
post some comment, question or even opinion about some subjects which
are interesting me.

If I have to pass some kind of meritocracy to post to this list I'll have
feeling of the 'second class' participant and probably will not post
anything.
That wouldn't be big loss for Debian anyway ;-)

Post by Miles Bader
The issues discussed here often do affect me, because I use Debian. I
don't actively participate most of the time but I do read, and every
once in a while, feel I have something to add.
The problem is not non-DDs, it's jerks and/or the clueless. Maybe on

Well said.

Post by Miles Bader
this list there's _some_ correlation between "non-DDness" and those
things -- but it's far from perfect... (and not, IMHO, enough to
justify censorship).
-miles

[Russ Allbery]

Well, obviously that wasn't a good idea at all, and I apologize for
bringing it up. Not only are most people rather opposed to having
different social expectations for people contributing to Debian or not,
they're so strongly opposed that the nuance of my original message was
lost. (And, seriously, that's largely my fault, since I knew the line
that I was drawing was probably too fine to be visible when I tried to
drew it and posted anyway.)

So we won't do that.

I do feel like I should make at least one comment in self-defense, because
a couple of people who are valuable contributors to debian-devel seem to
have felt personally attacked by the idea, or at least by the follow-up
discussion of how Ubuntu handled a similar situation, and that wasn't my
intention at all.

Post by Milan P. Stanic

Post by Miles Bader
... and as a non-DD who's been using Debian for 15 years (and reading
this list for many of them), and understands at least some of the
technical issues, I find the suggestion that I be automatically
considered a negative influence and excluded kind of annoying.

I'm in the same bandwagon.
Sometimes I even package some packages which are not in Debian for some
users. Few years ago I backported selinux to Sarge to Woody (IIRC) and
some people from over the world downloaded it and used or played with
it. These days I maintain Kannel development release (packages are on
the Kannel site) for Debian Testing and people use it.
Do I help Debian? I really don't know but I'm sure that I did help some
Debian users.
This (and some other) Debian list are helpful for me and I sometimes
post some comment, question or even opinion about some subjects which
are interesting me.
If I have to pass some kind of meritocracy to post to this list I'll have
feeling of the 'second class' participant and probably will not post
anything.
That wouldn't be big loss for Debian anyway ;-)

To be quite clear, I don't want to exclude people or even assume that
people are not valuable contributors to the mailing list just because they
aren't doing other things for Debian. (And both of your names are
familiar to me as valuable contributors to the list in the past.)

The phrasing I used in the original message was that I wished that people
who were in some sense house guests would behave as such; in other words,
it's more about social dynamics than a default assumption that people
*won't* behave as guests and need to be kept in a locked room until they
prove themselves. :)

But from the reaction this seems to be pretty clearly the entirely wrong
direction to go at this problem from, since the idea is just way too much
of a mismatch with expectations about how Debian mailing lists work.
Which, honestly, I should have realized in the first place.

What I'm getting from the rest of the thread is that there's no elegant
way to dodge around the core problem: some people (contributors or not)
sometimes start behaving in toxic ways on mailing lists, and someone with
authority to kick them off the list probably has to intervene when that
happens, and reporting that privately to the mailing list administrators
is the best method we have available to deal with that. (And probably the
best one that we're going to get, at least for the forseeable future.)

[Jon Dowland]

Post by Chris Knadle
debian-private: Private discussions among developers
and this list is not archived.

Actually it is: http://www.debian.org/doc/manuals/developers-reference/resources.html#mailing-lists-special

[Stefano Zacchiroli]

Post by Russ Allbery
Given recent experiences, I'm also coming around to Ian's position that
aggressive and confrontational contributions from people who don't
otherwise seem to be contributing to Debian are part of the problem and
are not useful, and possibly should be banned. I think that's been a
significant factor in the deterioration of the init system threads.

I agree that's a problem too and I share your feeling that it has been
particularly bad in recent discussions like the init system ones. To
look on the bright side, the problem seems concentrated in a few
specific topics rather than widespread to all discussions. But it is
still probably enough to keep some people from participating
constructively on -devel, which is a pretty serious problem.

Post by Russ Allbery
I want our technical discussions to be welcoming to anyone who has
information to share and who can bring additional clarity and insight to
the discussion. But once things start getting heated or people start
throwing around accusations or verge towards personal attacks, there's a
real psychological difference between people who are contributing to
Debian and people who aren't.

<snip>

Agreed also on your reasoning about the psychological effects of non
constructive participation by non contributors. Unfortunately, there
aren't many viable solutions to this kind of issues and all have
drawbacks.

1) our current solution: "don't feed the troll"

(even though the list participations we're talking about are not,
strictly speaking, trolling", that's basically our strategy)

It just doesn't work at this scale.

Sure, those who do respect the principle do reduce the noise (as
Bernhard pointed out), but you'll always have someone who will reply ---
maybe because they're new and accustomed to the list culture --- and it
is enough to have a few who do the feeding to make a discussion explode
and drive away people from it and, ultimately, decisions.

2) "don't feed the troll" + report abuses to listmasters and act
accordingly

I think we basically agree on the principle of this and IIRC we've even
discussed this about ~1 year ago without finding much opposition. But
either we're not doing this or it is not working.

Some of problems with this have been highlighted by Raphael. The
proposed fix, specifically for the "I don't know if I'm alone doing this
or not" part, sounds interesting. But even with that fix, you still
have the social awkwardness problem: the feeling is that of "censoring"
someone and it's a hard to ignore feeling, because the act of doing that
is much more concrete than the perception of the long term benefits of
doing so. I've the impression that the bar for silencing someone will
always remain high, higher than what would be needed to avoid the
behavior we're discussing.

Another problem you'll have with this "solution" is that it consumes a
lot of community energy (the people reporting bad behavior, who will do
that only after reaching some high frustration level; and the
listmasters who'll need to put time and emotions in judging the
behavior, implementing and probably explaining the "sanction").

3) public, but contributors-only list

This has been implemented by other FOSS projects. A notable example is
Ubuntu who have a split between ubuntu-devel (project members only +
whitelisting) and ubuntu-devel-discuss (free for all). I've never asked,
but I have always suspected that they've done so in an attempt to
improve over the experience of debian-devel participants.

The obvious drawback of this "solution" is that non-contributors will
need someone to vouch for them to be whitelisted.

----------

Each solution have advantages and disadvantages, but all in all I don't
think there aren't many other options. The question is blunt then: what
are we willing to give up of the current model in order to improve over
its defects?


Cheers.

[Riku Voipio]

Post by Stefano Zacchiroli
3) public, but contributors-only list
This has been implemented by other FOSS projects. A notable example is
Ubuntu who have a split between ubuntu-devel (project members only +
whitelisting) and ubuntu-devel-discuss (free for all). I've never asked,
but I have always suspected that they've done so in an attempt to
improve over the experience of debian-devel participants.
The obvious drawback of this "solution" is that non-contributors will
need someone to vouch for them to be whitelisted.

How about a "automated" contributors-only list.

To post to debian-devel, one would have to either submit a patch to a
bug, close a rt ticket, commit to one of the scm.debian.org or upload a
package to debian.

You include a url to that in the footer of the mail, and the mailing
list checks that it was really your change and it was done recently.
The check does not need to be perfect, if someone tries to cheat the
check we just ban the user for a while.

Non-contributors out of blue can still engage in the discussion, they
just need to contribute something first :) We enough "easy" jobs from
manpage writing to translations that the rule would not prevent
anyone who wants help debian to voicing their opinion. One just needs
to get out of their email client and actually do something useful for
a change.

It could also lessen the instant-flame-reply culture, as you would have
to do something else before replying - probably causing you to cool
down and articulate a better answer.

Before you scream "that's a technical solution to social problem", I
find it rather an economical solution to a social problem. Debian has
something people want (to voice the opinion) and Debian needs something
they can do (fix bugs). The solution brings them together.

Riku

For example, this mail would have been bought to you by the courtesy of:
http://incoming.debian.org/mtd-utils_1.4.9-1.dsc

[Gergely Nagy]

Post by Riku Voipio

Post by Stefano Zacchiroli
3) public, but contributors-only list
This has been implemented by other FOSS projects. A notable example is
Ubuntu who have a split between ubuntu-devel (project members only +
whitelisting) and ubuntu-devel-discuss (free for all). I've never asked,
but I have always suspected that they've done so in an attempt to
improve over the experience of debian-devel participants.
The obvious drawback of this "solution" is that non-contributors will
need someone to vouch for them to be whitelisted.

How about a "automated" contributors-only list.
To post to debian-devel, one would have to either submit a patch to a
bug, close a rt ticket, commit to one of the scm.debian.org or upload a
package to debian.

Require that for *every* post to debian-devel? That seems a tiiiny bit
excessive. As good as it might be to increase contributions, I'm afraid
this would have the opposite effect: decrease the posts to the list,
while not gaining anything.

It also rules out contributions done on this very list, which - despite
the tone of some recent threads - is not without precedent.

On the other hand, if I can participate in a thread by pre-seeding my
bucket of contributions, that might work.

And just for the heck of it, this mail, and any other mail in the
thread would have been made possible by the following things:
http://packages.qa.debian.org/d/dh-exec/news/20120503T114724Z.html
https://github.com/algernon/dh-exec/compare/b893f1eab5...bccacdb201

Nevertheless, if we adopt something like this - which I hope we don't
have to -, another problem arises: how recent the contributions must be?
Does opening bugs count? What if the contribution would be answering a
question on the list? What if upstream wants to chime in to a discussion
about his software on devel?

Truth be told, a moderated debian-devel@ would make me very, very sad,
no matter how the moderation would work. There must be a less forceful
way.

[Gergely Nagy]

Post by Stefano Zacchiroli
2) "don't feed the troll" + report abuses to listmasters and act
accordingly

Of the three, this is the least disruptive, in my opinion. Of course,
all the problems you mention (social awkwardity, effort from the
community and extra burden on listmasters) apply, BUT!

Perhaps a compromise could be to close threads forcibly, and temporarily
ban everyone from posting to the list, if they attempt to post to a
closed thread after its closing has been announced (a little window
of error should be given, of course, half an hour tops, or thereabouts).

This reduces the social awkwardness, as we'd be reporting bad threads
instead of bad people, and threads don't mind. It would reduce the load
on listmasters, as threads are fewer than people, and there's less
emotion involved, and justification is easier.

And if so need be, the temporary bans can gradually increase in length
if one keeps on posting to closed threads.

I've seen things like this work reasonably well on web-based forums, and
while it is considerably harder to implement it on a mailing list (and
probably impossible to make it entirely correct at that), something
reasonably similar that works in most cases shouldn't be terribly hard
to implement. People abusing the shortcomings of the solution can still
be banned on a case-by-case basis.

[Chris Knadle]

Post by Gergely Nagy

Post by Stefano Zacchiroli
2) "don't feed the troll" + report abuses to listmasters and act
accordingly

Of the three, this is the least disruptive, in my opinion. Of course,
all the problems you mention (social awkwardity, effort from the
community and extra burden on listmasters) apply, BUT!
Perhaps a compromise could be to close threads forcibly, and temporarily
ban everyone from posting to the list, if they attempt to post to a
closed thread after its closing has been announced (a little window
of error should be given, of course, half an hour tops, or thereabouts).

I've been helping moderate a LUG mailing list for a couple of years that uses
this strategy, and I think it works. The message of "this thread is closed,
anyone posting will be temporarily banned from posting if they reply" comes as
a relief when the thread has gone on long enough to have touched on seemingly
all the possibilities for solving an issue, but feels slightly heavy-handed
and "muzzle-ing" if done too quickly. Feedback on the list typically helps
the list moderators attain a reasonable equilibrium for the cuttoff point.

There are a couple of downsides to this strategy:

- one or more moderators need to be monitoring posts, and thus it's work.
The volume that this particular mailing list gets I think it's not a
one person task. [Come to think of it, how many DDs are currently
allowed to officially moderate the list?]

- There's a tendency to forget that the 'mod bit' is set for the user
that's been temporarily banned from posting

Post by Gergely Nagy
This reduces the social awkwardness, as we'd be reporting bad threads
instead of bad people, and threads don't mind. It would reduce the load
on listmasters, as threads are fewer than people, and there's less
emotion involved, and justification is easier.
And if so need be, the temporary bans can gradually increase in length
if one keeps on posting to closed threads.

Yes, this works. Thankfully it very rarely ever comes to this, but I've seen
a couple of instances where this became necessary.

Post by Gergely Nagy
I've seen things like this work reasonably well on web-based forums, and
while it is considerably harder to implement it on a mailing list (and
probably impossible to make it entirely correct at that), something
reasonably similar that works in most cases shouldn't be terribly hard
to implement. People abusing the shortcomings of the solution can still
be banned on a case-by-case basis.

It's always a judgement call. Not all judgements are going to be correct.

-- Chris

--
Chris Knadle
***@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74

[Lisandro Damián Nicanor Pérez Meyer]

On Jue 03 May 2012 08:23:29 Stefano Zacchiroli escribió:
[snip]

Post by Stefano Zacchiroli
3) public, but contributors-only list
This has been implemented by other FOSS projects. A notable example is
Ubuntu who have a split between ubuntu-devel (project members only +
whitelisting) and ubuntu-devel-discuss (free for all). I've never asked,
but I have always suspected that they've done so in an attempt to
improve over the experience of debian-devel participants.

If you happen to ask, it would also be nice to know how it worked for them.

Post by Stefano Zacchiroli
The obvious drawback of this "solution" is that non-contributors will
need someone to vouch for them to be whitelisted.

Well, I think if someone contributes something in the -discuss list with good
arguments/attitude/etc it will be much easier for him/her to get someone
whitelist her/him than to get a sponsor for a package ;-)

Kinds regards, Lisandro.

[Scott Kitterman]

Not enough information to check signature validity. Show Details
[snip]

Post by Stefano Zacchiroli
3) public, but contributors-only list
This has been implemented by other FOSS projects. A notable example is
Ubuntu who have a split between ubuntu-devel (project members only +
whitelisting) and ubuntu-devel-discuss (free for all). I've never asked,
but I have always suspected that they've done so in an attempt to
improve over the experience of debian-devel participants.

If you happen to ask, it would also be nice to know how it worked for them.

It was implemented because at the time ubuntu-devel had a very low signal to
noise ratio and developers were getting frustrated (sound familiar). My
opinion is that it worked pretty well.

Most of the noise immediately shifted to ubuntu-devel-discuss and a lot of
developers never subscribed to it, so they were immediately helped.

After some period of high noise, low value existence, the number of Ubuntu
developers that subscribed to ubuntu-devel-discuss declined further. It was
pointed out to some of the more problematic contributors that if they didn't
knock it off and be less abusive and more productive in their list messages,
they were going to have no developers left to talk to.

Eventually, the situation normalized and ubuntu-devel-discuss is a fairly low
volume list and most of the posts, if not particularly consistently well
informed, are from people that are trying to be constructive (not, of course,
right after controversial decisions get announced). The two lists separated
are, in my opinion much higher signal to noise than the old combined ones.

Scott K

[Steve Langasek]

Post by Scott Kitterman
It was implemented because at the time ubuntu-devel had a very low signal to
noise ratio and developers were getting frustrated (sound familiar). My
opinion is that it worked pretty well.
Most of the noise immediately shifted to ubuntu-devel-discuss and a lot of
developers never subscribed to it, so they were immediately helped.
After some period of high noise, low value existence, the number of Ubuntu
developers that subscribed to ubuntu-devel-discuss declined further. It was
pointed out to some of the more problematic contributors that if they didn't
knock it off and be less abusive and more productive in their list messages,
they were going to have no developers left to talk to.
Eventually, the situation normalized and ubuntu-devel-discuss is a fairly low
volume list and most of the posts, if not particularly consistently well
informed, are from people that are trying to be constructive (not, of course,
right after controversial decisions get announced). The two lists separated
are, in my opinion much higher signal to noise than the old combined ones.

I would also note that, in practice, ubuntu-devel is not so much moderated
as it is rate limited. The non-developer posts to the list are AFAICT
universally approved so long as they aren't spam; but the moderation delays
are substantial enough that non-contributors have a chance to say their
piece while having no opportunity to be disruptive.

[Riku Voipio]

Post by Russ Allbery
I'm not sure that I see the point, and I say that as someone who
replaces Exim with Postfix on all of my boxes.

Nobody's suggesting you need to change to anything. The worst you
have to do if debian changed default MTA, would be to
"apt-get install exim4" (gasp, what horrible pain) when doing new
installations.

Post by Russ Allbery
There's nothing particularly wrong with Exim; it works just fine.

Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
forcing quoted-printable conversions to make emails "7bit clean" is quite
horribly wrong.

Debian is the main source of Exim installs in internet, it is also our fault.
According to one old stat[1], 34% of mx records were exim, most probably almost
all simply because it came by default in debian and it was "good enough"
so people didnt' switch away from it.

So yes, switching to postfix by default would reduce the workload of email
servers around the globe (no need to burn cpu cycles and thus co2 to convert
emails to quoted-printable).

Yes that was a bit of a hyperbole, but this is my pet issue. I complained
last about it in 2009 [2] with no change from upstream since...

Riku

[1] http://www.securityspace.com/s_survey/data/man.201007/mxsurvey.html
[2] http://suihkulokki.blogspot.com/2009/07/cult-of-workarounds.html

[Russ Allbery]

Post by Russ Allbery
I'm not sure that I see the point, and I say that as someone who
replaces Exim with Postfix on all of my boxes.

Nobody's suggesting you need to change to anything. The worst you have
to do if debian changed default MTA, would be to "apt-get install exim4"
(gasp, what horrible pain) when doing new installations.

Did you miss the bit where I said that I replace Exim with Postfix on all
of my boxes, despite quoting it? Logically, you can draw the inference
that I know exactly how hard it is to replace the default MTA with a
different one.

Post by Russ Allbery
There's nothing particularly wrong with Exim; it works just fine.

Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
forcing quoted-printable conversions to make emails "7bit clean" is
quite horribly wrong.

I didn't realize that. I agree, that's an annoying missing feature. Has
someone talked with upstream about whether they have plans to implement
it?

Yes that was a bit of a hyperbole, but this is my pet issue. I
complained last about it in 2009 [2] with no change from upstream
since...
[2] http://suihkulokki.blogspot.com/2009/07/cult-of-workarounds.html

Okay, let me rephrase that: has anyone talked with upstream about this
without making sweeping, confrontational statements about a "cult of
workarounds" and otherwise insulting them?

[Andreas Barth]

Post by Russ Allbery

Post by Riku Voipio
Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
forcing quoted-printable conversions to make emails "7bit clean" is
quite horribly wrong.

I didn't realize that. I agree, that's an annoying missing feature. Has
someone talked with upstream about whether they have plans to implement
it?

Quoting the manual
| accept_8bitmime Use: main Type: boolean Default: false
|
| This option causes Exim to send 8BITMIME in its response to an SMTP EHLO
| command, and to accept the BODY= parameter on MAIL commands. However, though
| Exim is 8-bit clean, it is not a protocol converter, and it takes no steps to
| do anything special with messages received by this route. Consequently, this
| option is turned off by default.

So the *default* configuration doesn't advertise 8BITMIME for the
reason that exim won't do conversions later on when relaying to other
hosts (which might be a possible RFC violation, depending where the
mail is relayed to). One could certainly argue that isn't the best /
recommended state, but it's not that bad either.

(And this is one of the first hits while searching for 8BITMIME and
exim.)


Andi

[Chris Knadle]

...

Post by Riku Voipio

Post by Russ Allbery
There's nothing particularly wrong with Exim; it works just fine.

Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
forcing quoted-printable conversions to make emails "7bit clean" is quite
horribly wrong.

I think it would be useful to describe what issue(s) there are concerning
8BITMIME and why this is important. I've found some information [1] about
this, but it isn't clear what problems are actially *caused* by the lack of
8BITMIME support by default in Exim. Is it just slow sending of outbound
attachments?

Post by Riku Voipio
Debian is the main source of Exim installs in internet, it is also our
fault. According to one old stat, 34% of mx records were exim, most
probably almost all simply because it came by default in debian and it was
"good enough" so people didnt' switch away from it.

The quoted 2010 survey [2] showed Exim was the most popular MTA (which I found
surprising), deployment of Exim growing just slightly faster than Postfix, and
everything else falling in popularity. I don't know how one would verify (or
dispute) the claim that Debian was the main source of Exim installs, and I'm
not sure that's a "problem" that needs fixing. (Also if you look more closely
at the survey, ~55% of responding MTAs didn't identify themselves and are thus
not counted in the statistics, which is a potential wide margin of error.)

Post by Riku Voipio
So yes, switching to postfix by default would reduce the workload of email
servers around the globe (no need to burn cpu cycles and thus co2 to
convert emails to quoted-printable).

The statistics quoted showed that Exim was most popular, so wouldn't switching
to Postfix by default actually be more CPU costly than the reverse? :-/ [I'm
not saying you're wrong, just that I don't see the logic in the argument.]




I administer both Postfix and Exim and greatly prefer Exim (specifically
exim4-daemon-heavy and using a single config file), but I wouldn't mind if the
default were Postifx. Whatever the default MTA is should IMHO be whatever DDs
supporting Debian think is the most supportable and "the best default".

I've likewise often wondered if a low-resource MTA like DMA or ssmtp could be
the default MTA for Desktop installs (and I've occasionally tried them), but
as has been discussed there seem to be some issues with the idea. In my case
for Desktops I want the local MTA to be able to handle sending local outbound
mail to a server via port 587 over TLS with authentication, to retry sending
at increasing time intervals, using a "queue runner" but without a daemon
listening, and to notify the sender on a permanent failure. Thusfar I've only
been able to find all of that in a full-fledged MTA.



[1] http://cr.yp.to/smtp/8bitmime.html

[2] http://www.securityspace.com/s_survey/data/man.201007/mxsurvey.html

-- Chris

--
Chris Knadle
***@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74

[Philipp Kern]

Post by Chris Knadle
I think it would be useful to describe what issue(s) there are concerning
8BITMIME and why this is important. I've found some information [1] about
this, but it isn't clear what problems are actially *caused* by the lack of
8BITMIME support by default in Exim. Is it just slow sending of outbound
attachments?

The only issue I found so far is that Postfix will convert mails when sending
to an Exim that does not advertise 8BITMIME (like *.d.o). Which let me with
the need to handle qp although the mails are initially sent with 8bit (build
logs).

I also assume that Exim does send 8bit mails to non-8bit compliant MTAs (i.e.
not advertising 8BITMIME). I don't know if that's some sort of violation.

Kind regards
Philipp Kern

[Chris Knadle]

Post by Philipp Kern

Post by Chris Knadle
I think it would be useful to describe what issue(s) there are concerning
8BITMIME and why this is important. I've found some information [1]
about this, but it isn't clear what problems are actially *caused* by
the lack of 8BITMIME support by default in Exim. Is it just slow
sending of outbound attachments?

The only issue I found so far is that Postfix will convert mails when
sending to an Exim that does not advertise 8BITMIME (like *.d.o). Which
let me with the need to handle qp although the mails are initially sent
with 8bit (build logs).
I also assume that Exim does send 8bit mails to non-8bit compliant MTAs
(i.e. not advertising 8BITMIME). I don't know if that's some sort of
violation.

I think it is.

If 'accept_8bitmime' is enabled (default is disabled), then Exim will send
email containing 8bit MIME to non-8bit compliant MTAs. (For verification of
this, see [1] and [2].) This would violate RFC 1652 section 3 [3]:

"If a server SMTP does not support the 8-bit MIME transport extension
(either by not responding with code 250 to the EHLO command, or by
not including the EHLO keyword value 8BITMIME in its response), then
the client SMTP must not, under any circumstances, attempt to
transfer a content which contains characters outside the US-ASCII
octet range (hex 00-7F)."

and also RFC 6152 section 3 [4] (which contains the idential paragraph above
from RFC 1652).


I think the reason Exim does not do this protocol conversion is that from the
point of view of an MTA author, the point of an MTA is to transmit the body of
the message without any modification to it once received, and body
modification would be required to convert 8-bit MIME to 7-bit MIME. I seem to
remember reading something along these lines in the Exim documentation years
ago and I'm having another look through it, but haven't found a reference to
this yet.


[1] https://en.wikipedia.org/wiki/Extended_SMTP#8BITMIME

[2] http://www.exim.org/exim-html-3.20/doc/html/spec_11.html

[2] https://tools.ietf.org/html/rfc1652#section-3

[3] https://tools.ietf.org/html/rfc6152#section-3

-- Chris

--
Chris Knadle
***@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74

[Riku Voipio]

Post by Chris Knadle
I think it would be useful to describe what issue(s) there are concerning
8BITMIME and why this is important. I've found some information [1] about
this, but it isn't clear what problems are actially *caused* by the lack of
8BITMIME support by default in Exim. Is it just slow sending of outbound
attachments?

It's both extra traffic (not that much if western encodings) and extra
cpu work. In lesser annoyance, it means that you no longer can read
mailbox files with non-mime capable readers (for example less) with ease,
as there will be qp encodings here and there. People who only use english
only hit the issue when having lines longer than 76 characters.

Basicly that the qmail author is suggesting is violating the rfc by
announcing 8bitmime sopport but never doing any conversions if you bump
into a server that doesn't support 8bitmime. This is the same as the
exim's accept_8bitmime option.

In the real world, violating the rfc is a non-issue, as most likely the
only non-8bitmime compliant servers your server encouters are qmail and exim4,
both which have no problems with 8bit mails itself.

Honesstly. my grievance is really just having to convert things to 7bit.. still!

Post by Chris Knadle
The quoted 2010 survey [2] showed Exim was the most popular MTA (which I found
surprising), deployment of Exim growing just slightly faster than Postfix, and
everything else falling in popularity.

I think it is because other distro's have mostly stopped shipping
anything that listens on port 25 by default.

Post by Chris Knadle
I don't know how one would verify (or dispute) the claim that Debian was the
main source of Exim installs

Indeed that is not verifiable from the given report. However, they list
the most common exim version being 4.69 which is same as what was in
lenny at that time.

Post by Chris Knadle

Post by Riku Voipio
So yes, switching to postfix by default would reduce the workload of email
servers around the globe (no need to burn cpu cycles and thus co2 to
convert emails to quoted-printable).

The statistics quoted showed that Exim was most popular, so wouldn't switching
to Postfix by default actually be more CPU costly than the reverse? :-/ [I'm
not saying you're wrong, just that I don't see the logic in the argument.]

The less there are MTA's out there not announcing 8bitmime, the less
conversions there will be (eventually). While the exim->exim deliveries
are not generating conversions (unless the MUA does it), exim is still
only at 34% of mx, not 34% of the traffic.

Post by Chris Knadle
I've likewise often wondered if a low-resource MTA like DMA or ssmtp could be
the default MTA for Desktop installs (and I've occasionally tried them), but
as has been discussed there seem to be some issues with the idea. In my case
for Desktops I want the local MTA to be able to handle sending local outbound
mail to a server via port 587 over TLS with authentication, to retry sending
at increasing time intervals, using a "queue runner" but without a daemon
listening, and to notify the sender on a permanent failure. Thusfar I've only
been able to find all of that in a full-fledged MTA.

Indeed, switching to a lightweight mta like dma as default makes probably more
sense than switching by default postfix. I tried ssmpt and it didn't
work for me.. Would dma do TLS auth and queque fine?

Riku

[Scott Kitterman]

On Tuesday, May 01, 2012 06:55:20 PM Riku Voipio wrote:
...

Post by Riku Voipio
Honesstly. my grievance is really just having to convert things to 7bit.. s

...
In the future, you're likely to still be stuck doing this for other 'fun'
reasons. The one I ran into recently was that 8 bit -> 7 bit conversions will
break DKIM signatures, so it's only safe (from a reliability perspective) to
sign 8 bit mail if you know the entire path is 8 bit clean. Since there's
really no way to know that, now I flatten everything to 7 bit and then sign it.

Scott K

[Philipp Kern]

Post by Riku Voipio
It's both extra traffic (not that much if western encodings) and extra
cpu work. In lesser annoyance, it means that you no longer can read
mailbox files with non-mime capable readers (for example less) with ease,
as there will be qp encodings here and there. People who only use english
only hit the issue when having lines longer than 76 characters.
Basicly that the qmail author is suggesting is violating the rfc by
announcing 8bitmime sopport but never doing any conversions if you bump
into a server that doesn't support 8bitmime. This is the same as the
exim's accept_8bitmime option.
In the real world, violating the rfc is a non-issue, as most likely the
only non-8bitmime compliant servers your server encouters are qmail and exim4,
both which have no problems with 8bit mails itself.
Honesstly. my grievance is really just having to convert things to 7bit.. still!

So just stop Postfix doing the conversion? Or teach Exim to announce 8BITMIME
by default.

Kind regards
Philipp Kern

[Andrew Shadura]

Hello,

On Tue, 1 May 2012 20:18:07 +0200

Post by Philipp Kern
So just stop Postfix doing the conversion? Or teach Exim to announce
8BITMIME by default.

No, Exim should not announce 8BITMIME, or it will violate RFC, not
otherwise. Now it doesn't announce it, but accepts, so RFC-compliant
MUA or other MTA should do the conversion themselves and everything
just works. If it announces the support, it effectively lies as it
doesn't support conversion which is needed if it wants to send mail to
non-compliant MTA.

I wonder why many people in this thread still don't understand this.
And also I can't see why some find this annoying behaviour or something
wrong. There's absolutely nothing wrong with what it does now, as
re-encoding will happen somewhere anyway as there are still many really
non-8-bit-compliant MTAs, so why should Exim do this?

[Philipp Kern]

Post by Andrew Shadura
On Tue, 1 May 2012 20:18:07 +0200

Post by Philipp Kern
So just stop Postfix doing the conversion? Or teach Exim to announce
8BITMIME by default.

No, Exim should not announce 8BITMIME, or it will violate RFC, not
otherwise. Now it doesn't announce it, but accepts, so RFC-compliant
MUA or other MTA should do the conversion themselves and everything
just works. If it announces the support, it effectively lies as it
doesn't support conversion which is needed if it wants to send mail to
non-compliant MTA.

If I use sendmail it will happily accept 8bit, because there's no way to tell
if the MTA is 8BITMIME-compliant. And POSIX doesn't mandate 7bit, does it?

Post by Andrew Shadura
I wonder why many people in this thread still don't understand this.
And also I can't see why some find this annoying behaviour or something
wrong. There's absolutely nothing wrong with what it does now, as
re-encoding will happen somewhere anyway as there are still many really
non-8-bit-compliant MTAs, so why should Exim do this?

Exim transmits 8bit mails to non-8bit-compliant MTAs, so what exactly are you
arguing?

Kind regards
Philipp Kern

[Andrew Shadura]

Hello,

On Tue, 1 May 2012 23:03:38 +0200

Post by Philipp Kern

Post by Andrew Shadura
I wonder why many people in this thread still don't understand this.
And also I can't see why some find this annoying behaviour or
something wrong. There's absolutely nothing wrong with what it does
now, as re-encoding will happen somewhere anyway as there are still
many really non-8-bit-compliant MTAs, so why should Exim do this?

Exim transmits 8bit mails to non-8bit-compliant MTAs, so what exactly
are you arguing?

No it doesn't if 8BITMIME annouces are turned off!

[Jon Dowland]

Post by Andrew Shadura
No it doesn't if 8BITMIME annouces are turned off!

If exim receives an 8 bit mail, even if it hadn't announced 8BITMIME in
the EHLO response, it will relay that message verbatim to other hosts.

[Andrew Shadura]

Hello,

On Wed, 2 May 2012 10:06:31 +0100

Post by Jon Dowland

Post by Andrew Shadura
No it doesn't if 8BITMIME annouces are turned off!

If exim receives an 8 bit mail, even if it hadn't announced 8BITMIME
in the EHLO response, it will relay that message verbatim to other
hosts.

But it won't receive it at all if the sender is standards-compliant.

[Vincent Lefevre]

Post by Andrew Shadura
Hello,
On Wed, 2 May 2012 10:06:31 +0100

Post by Jon Dowland

Post by Andrew Shadura
No it doesn't if 8BITMIME annouces are turned off!

If exim receives an 8 bit mail, even if it hadn't announced 8BITMIME
in the EHLO response, it will relay that message verbatim to other
hosts.

But it won't receive it at all if the sender is standards-compliant.

True for SMTP. But what if exim received an 8-bit mail by another
mean (e.g. on the command line via the sendmail command)?
--
Vincent Lefèvre <***@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[Scott Kitterman]

Post by Riku Voipio

Post by Andrew Shadura
Hello,
On Wed, 2 May 2012 10:06:31 +0100

Post by Jon Dowland

Post by Andrew Shadura
No it doesn't if 8BITMIME annouces are turned off!

If exim receives an 8 bit mail, even if it hadn't announced

8BITMIME

Post by Andrew Shadura

Post by Jon Dowland
in the EHLO response, it will relay that message verbatim to other
hosts.

But it won't receive it at all if the sender is standards-compliant.

True for SMTP. But what if exim received an 8-bit mail by another
mean (e.g. on the command line via the sendmail command)?

Then it's irrelevant to the discussion. It's a local system issue and it's up to the local administrator. Internet standards have nothing to do with it.

Scott K

[Vincent Lefevre]

Post by Scott Kitterman

Post by Riku Voipio

Post by Andrew Shadura
Hello,
On Wed, 2 May 2012 10:06:31 +0100

Post by Jon Dowland

Post by Andrew Shadura
No it doesn't if 8BITMIME annouces are turned off!

If exim receives an 8 bit mail, even if it hadn't announced

8BITMIME

Post by Andrew Shadura

Post by Jon Dowland
in the EHLO response, it will relay that message verbatim to other
hosts.

But it won't receive it at all if the sender is standards-compliant.

True for SMTP. But what if exim received an 8-bit mail by another
mean (e.g. on the command line via the sendmail command)?

Then it's irrelevant to the discussion. It's a local system issue
and it's up to the local administrator. Internet standards have
nothing to do with it.

Wrong. It's relevant in the sense that Exim will transmit the mail
to a remote MTA. If it doesn't do the 8-bit to 7-bit conversion if
the remote MTA doesn't announce 8BITMIME, then Exim is broken.
--
Vincent Lefèvre <***@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[Scott Kitterman]

Post by Vincent Lefevre

Post by Scott Kitterman

Post by Riku Voipio

Post by Andrew Shadura
Hello,
On Wed, 2 May 2012 10:06:31 +0100

Post by Jon Dowland

Post by Andrew Shadura
No it doesn't if 8BITMIME annouces are turned off!

If exim receives an 8 bit mail, even if it hadn't announced

8BITMIME

Post by Andrew Shadura

Post by Jon Dowland
in the EHLO response, it will relay that message verbatim to other
hosts.

But it won't receive it at all if the sender is standards-compliant.

True for SMTP. But what if exim received an 8-bit mail by another
mean (e.g. on the command line via the sendmail command)?

Then it's irrelevant to the discussion. It's a local system issue
and it's up to the local administrator. Internet standards have
nothing to do with it.

Wrong. It's relevant in the sense that Exim will transmit the mail
to a remote MTA. If it doesn't do the 8-bit to 7-bit conversion if
the remote MTA doesn't announce 8BITMIME, then Exim is broken.

True, but that's a different bit of brokenness.

Scott K

[Jon Dowland]

Post by Andrew Shadura
Hello,
On Tue, 1 May 2012 20:18:07 +0200

Post by Philipp Kern
So just stop Postfix doing the conversion? Or teach Exim to announce
8BITMIME by default.

No, Exim should not announce 8BITMIME, or it will violate RFC, not
otherwise.

Only if it forwards the mail: if delivery stops on the local host, then
there isn't a problem. So it could be enabled if the debconf question
was answered accordingly without risking violating the RFC afaics

[Riku Voipio]

Post by Philipp Kern
So just stop Postfix doing the conversion?

It's not just postfix, it's at least courier and sendmail and various
propiertary MTA's do conversions when encountering default configured
exims.

It would be a RFC violation to just pass 8bit mails to servers not
advertizing 8bitmime. It would be rfc compatible to the sending server
to bounce instead of qp-converting 8bit mails, but that would arguably
be even worse.

Post by Philipp Kern
Or teach Exim to announce 8BITMIME by default.

That would be a rfc violation as well.

Riku

[Russell Coker]

Post by Riku Voipio
It would be a RFC violation to just pass 8bit mails to servers not
advertizing 8bitmime. It would be rfc compatible to the sending server
to bounce instead of qp-converting 8bit mails, but that would arguably
be even worse.

No, bouncing mail when it can't be properly delivered is much better than
violating RFCs.

Mail that is bounced with a human readable message describing the real cause
of the problem can then be re-sent once the problem is fixed.

Having mail be silently corrupted is not acceptable.

I've spent a lot of time debugging email problems over the years...

[Jon Dowland]

Post by Russell Coker
Having mail be silently corrupted is not acceptable.

Can you expand on "silently corrupted", here? Is that when you re-encode the
mail and send it on as 7-bit, or when you leave it alone and send it as 8 bit
to a host that doesn't advertise accepting 8-bit?

[Russell Coker]

Post by Jon Dowland

Post by Russell Coker
Having mail be silently corrupted is not acceptable.

Can you expand on "silently corrupted", here? Is that when you re-encode
the mail and send it on as 7-bit, or when you leave it alone and send it
as 8 bit to a host that doesn't advertise accepting 8-bit?

When you send 8 bit mail to a host that only supports 7 bit then it will be
corrupted, usually without any notification of what happened - definitely
silent corruption.

When you re-encode mail and send it on IFF the message is DKIM signed it could
be considered to be silent corruption as the change will usually count as
breakage.

It would be possible for a DKIM verification program to re-encode 7bit
messages to 8bit for a second attempt at verification. But if a DKIM milter
author was going to do tricky things then a better first option would be to
try removing anything between [] in the subject line which is the most common
cause of DKIM failures that I see on valid mail.

[Scott Kitterman]

Post by Russell Coker

Post by Jon Dowland

Post by Russell Coker
Having mail be silently corrupted is not acceptable.

Can you expand on "silently corrupted", here? Is that when you re-encode
the mail and send it on as 7-bit, or when you leave it alone and send it
as 8 bit to a host that doesn't advertise accepting 8-bit?

When you send 8 bit mail to a host that only supports 7 bit then it will be
corrupted, usually without any notification of what happened - definitely
silent corruption.
When you re-encode mail and send it on IFF the message is DKIM signed it
could be considered to be silent corruption as the change will usually
count as breakage.
It would be possible for a DKIM verification program to re-encode 7bit
messages to 8bit for a second attempt at verification. But if a DKIM milter
author was going to do tricky things then a better first option would be to
try removing anything between [] in the subject line which is the most
common cause of DKIM failures that I see on valid mail.

That and mailing list footers.

Receivers are, of course, free to manage inbound mail filtering however they
want, but if you take a message and try to recode it from 7 bit to 8 bit and
see if a DKIM signature passes verification, it's still not a valid DKIM
signature in any sense that RFC 4871 or its successors would recognize.

Scott K

[Russell Coker]

Post by Scott Kitterman

Post by Russell Coker
It would be possible for a DKIM verification program to re-encode 7bit
messages to 8bit for a second attempt at verification. But if a DKIM
milter author was going to do tricky things then a better first option
would be to try removing anything between [] in the subject line which
is the most common cause of DKIM failures that I see on valid mail.

That and mailing list footers.

Footers can be solved with the l= flag. The threat of a hostile party
appending data to a message probably isn't something you really worry about
when posting to a mailing list.

It would be possible for a DKIM signing program to use l= for every message
which has a recipient address containing the string "list".

Post by Scott Kitterman
Receivers are, of course, free to manage inbound mail filtering however
they want, but if you take a message and try to recode it from 7 bit to 8
bit and see if a DKIM signature passes verification, it's still not a
valid DKIM signature in any sense that RFC 4871 or its successors would
recognize.

If a milter replaced the message body such that it matched then it would be.

[Ben Hutchings]

Post by Russell Coker

Post by Jon Dowland

Post by Russell Coker
Having mail be silently corrupted is not acceptable.

Can you expand on "silently corrupted", here? Is that when you re-encode
the mail and send it on as 7-bit, or when you leave it alone and send it
as 8 bit to a host that doesn't advertise accepting 8-bit?

When you send 8 bit mail to a host that only supports 7 bit then it will be
corrupted, usually without any notification of what happened - definitely
silent corruption.
When you re-encode mail and send it on IFF the message is DKIM signed it could
be considered to be silent corruption as the change will usually count as
breakage.

[...]

So DKIM is broken by design. Not sure why anyone should care to work
around this.

Ben.

[Andreas Metzler]

Russell Coker <***@coker.com.au> wrote:
[...]

Post by Russell Coker
When you send 8 bit mail to a host that only supports 7 bit then it will be
corrupted, usually without any notification of what happened - definitely
silent corruption.

[...]


Have you really seen this happening in this century? Are there really
MTAs active in the wild nowadays which are not 8-bit clean?

I thought all of these were fixed when qmail was popular.

cu andreas

[Andreas Metzler]

Post by Andreas Metzler
[...]

Post by Russell Coker
When you send 8 bit mail to a host that only supports 7 bit then it will be
corrupted, usually without any notification of what happened - definitely
silent corruption.

[...]
Have you really seen this happening in this century? Are there really
MTAs active in the wild nowadays which are not 8-bit clean?
I thought all of these were fixed when qmail was popular.

Nevermind, I think I had asked this about a year ago. The answer then
was something like: "No, today all MTAs are 8bit clean, I just chose to
configure mine to drop the 8th bit, to make me suffer / reject spam."

cu andreas

[Christian PERRIER]

(slightly off-topic)

Post by Russell Coker
No, bouncing mail when it can't be properly delivered is much better than
violating RFCs.
Mail that is bounced with a human readable message describing the real cause
of the problem can then be re-sent once the problem is fixed.

You mean a message readable by a human even when it happens that this
human is a non English-speaking non-geek person? :-)

All MTA bounce messages are just plain unreadable crap for the average
human on Earth, I'm afraid. For some of them, it's even worse than
Vogon poetry.

(and, no, I have no solution to this, by the way...just reacting to
the assumption that bounce messages are readable: they are not)

[Henrique de Moraes Holschuh]

Post by Christian PERRIER
(slightly off-topic)

Post by Russell Coker
No, bouncing mail when it can't be properly delivered is much better than
violating RFCs.
Mail that is bounced with a human readable message describing the real cause
of the problem can then be re-sent once the problem is fixed.

You mean a message readable by a human even when it happens that this
human is a non English-speaking non-geek person? :-)

Well, FWIW postfix allows you to override all MTA notifications, not just
bounce messages, but the full set. We do that at work.

Post by Christian PERRIER
All MTA bounce messages are just plain unreadable crap for the average
human on Earth, I'm afraid. For some of them, it's even worse than
Vogon poetry.

IME this is true even after you translate it to the local language and dumb
it down a whole lot.

[Russell Coker]

Post by Henrique de Moraes Holschuh

Post by Christian PERRIER
All MTA bounce messages are just plain unreadable crap for the average
human on Earth, I'm afraid. For some of them, it's even worse than
Vogon poetry.

IME this is true even after you translate it to the local language and dumb
it down a whole lot.

For non-technical users the best thing to do is to have an error message
that's informative to a sysadmin which they can paste or forward to someone
who can fix it.

One of my pet hates with some MS email software is when it doesn't display the
real error message to the user and hides it in a way that's beyond the ability
of the non-technical user to discover and report.

[Chris Knadle]

...

Post by Riku Voipio

Post by Chris Knadle
The quoted 2010 survey [2] showed Exim was the most popular MTA (which I
found surprising), deployment of Exim growing just slightly faster than
Postfix, and everything else falling in popularity.

I think it is because other distro's have mostly stopped shipping
anything that listens on port 25 by default.

By default selecting "standard system" during the Debian Squeeze netinstall
installs Exim, but it only listens on port 25 on localhost, and not the public
interface. [IIRC I believe that was the default in Lenny also.] I'm pretty
sure this means that by default the local MTA won't show up on the internet.

...

Post by Riku Voipio

Post by Chris Knadle

Post by Riku Voipio
So yes, switching to postfix by default would reduce the workload of
email servers around the globe (no need to burn cpu cycles and thus
co2 to convert emails to quoted-printable).

The statistics quoted showed that Exim was most popular, so wouldn't
switching to Postfix by default actually be more CPU costly than the
reverse? :-/ [I'm not saying you're wrong, just that I don't see the
logic in the argument.]

The less there are MTA's out there not announcing 8bitmime, the less
conversions there will be (eventually). While the exim->exim deliveries
are not generating conversions (unless the MUA does it), exim is still
only at 34% of mx, not 34% of the traffic.

Yes that makes sense -- DNS MX records are probably easier to test for.
Agreed that the % of traffic is not captured in the statistics -- however the
number of MX records per MTA is more relevant than % of traffic per MTA to the
arugument you brought up anyway, because that had to do with the number of
installs and thus the number of boxes needing to switch the installed MTA.
;-) [BTW I found it odd that Qmail wasn't in the top 20 in terms of
deployments vs MX records, and perhaps it is in terms of % of net traffic.]

Post by Riku Voipio

Post by Chris Knadle
I've likewise often wondered if a low-resource MTA like DMA or ssmtp
could be the default MTA for Desktop installs (and I've occasionally
tried them), but as has been discussed there seem to be some issues with
the idea. In my case for Desktops I want the local MTA to be able to
handle sending local outbound mail to a server via port 587 over TLS
with authentication, to retry sending at increasing time intervals,
using a "queue runner" but without a daemon listening, and to notify the
sender on a permanent failure. Thusfar I've only been able to find all
of that in a full-fledged MTA.

Indeed, switching to a lightweight mta like dma as default makes probably
more sense than switching by default postfix. I tried ssmpt and it didn't
work for me.. Would dma do TLS auth and queque fine?

Yes, it looks like it can. Manpage [1] example setup [2].

The Readme.Debian document that comes with the dma package also states that
although a cron job is run by default every 5 minutes, that does *not* mean a
sending attempt is made every 5 minutes -- dma does an exponential back-off on
successive sending failures. It can also notify the sender of sending
failures [and has a "double bounce" setting in case the sender's address also
bounces].

I'm testing it now -- looks promising. Might be worth further consideration
for a default MTA.


[1] http://manpages.ubuntu.com/manpages/natty/man8/dma.8.html

[2] http://www.dragonflybsd.org/docs/howtos/HowTo_dma_gmail/

-- Chris

--
Chris Knadle
***@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74

[Vincent Lefevre]

Post by Riku Voipio

Post by Chris Knadle
I think it would be useful to describe what issue(s) there are concerning
8BITMIME and why this is important. I've found some information [1] about
this, but it isn't clear what problems are actially *caused* by the lack of
8BITMIME support by default in Exim. Is it just slow sending of outbound
attachments?

It's both extra traffic (not that much if western encodings) and extra
cpu work. In lesser annoyance, it means that you no longer can read
mailbox files with non-mime capable readers (for example less) with ease,
as there will be qp encodings here and there. People who only use english
only hit the issue when having lines longer than 76 characters.

It might also affect spam detection.

Now, the main reason I dislike Exim is

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=485751

Basically, when invoked as sendmail, Exim breaks the sendmail
compatibility by keeping the Bcc header, involving security
and/or privacy problems. And since this is by design, it won't
be fixed.

BTW, since it must accept 8-bit mail via this interface, it is
also broken as the mail may be sent to a MTA that doesn't announce
8BITMIME, since Exim doesn't know to do the conversion. :)
--
Vincent Lefèvre <***@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[Adam D. Barratt]

Post by Riku Voipio

Post by Russ Allbery
There's nothing particularly wrong with Exim; it works just fine.

Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
forcing quoted-printable conversions to make emails "7bit clean" is quite
horribly wrong.

fwiw, there's just (as in within the past couple of hours) been a change
committed upstream which defaults accept_8bitmime to true.

Regards,

Adam

[Joerg Jaspert]

Post by Marco d'Itri
Is this the right time to do it?

No.

It never will be.

It's insane to even think of switching one full featured MTA against
another full featured one. It feels like "gosh, i dislike $onepiece,
lets all move to $differentpiece", though both are bad as default.

If you really want to invest work, the time is much better spent in
getting one of those tiny replacements a default. And then anyone who
wants/needs a real one can change. Much more useful that way.

[Joey Hess]

Post by Joerg Jaspert
It's insane to even think of switching one full featured MTA against
another full featured one. It feels like "gosh, i dislike $onepiece,
lets all move to $differentpiece", though both are bad as default.

Yeah, Debian has certianly never done that before ..
(Remember smail? It's still mentioned in Policy even!)

Post by Joerg Jaspert
If you really want to invest work, the time is much better spent in
getting one of those tiny replacements a default. And then anyone who
wants/needs a real one can change. Much more useful that way.

That would be a fairly viable route. Although it slightly begs the
question of what task-mail-server would then pull in instead.

[Carsten Hey]

Post by Joey Hess

Post by Joerg Jaspert
It's insane to even think of switching one full featured MTA against
another full featured one. It feels like "gosh, i dislike $onepiece,
lets all move to $differentpiece", though both are bad as default.

Looks like the DragonFly Mail Agent (dma), which already has been
mentioned in this thread, could become a decent default for Wheezy+1
after some small changes.

In a nutshell: it's able to deliver locally and remotely, has a queue,
supports TLS/SSL, does not listen on port 25 and instead of running as
daemon, it if run every 5 minutes via cron to flush the queue. Maybe it
could be changed to use incron and not cron on Linux. It's
README.Debian contains more verbose information.

It currently lacks support for .forward files, ignores -bs, misses -F,
ignores -om and the newaliases command exits with 1 if run without
arguments. I'm nore sure if the latter two are problematic.

Post by Joey Hess
Yeah, Debian has certianly never done that before ..
(Remember smail? It's still mentioned in Policy even!)

JFTR: According to [1] and [2], the reason to switch away from smail was
that exim are easier to configure.

[1] http://lists.debian.org/debian-devel/1998/11/msg00415.html
[2] http://lists.debian.org/debian-devel/1998/11/msg00489.html

Post by Joey Hess

Post by Joerg Jaspert
If you really want to invest work, the time is much better spent in
getting one of those tiny replacements a default. And then anyone who
wants/needs a real one can change. Much more useful that way.

That would be a fairly viable route. Although it slightly begs the
question of what task-mail-server would then pull in instead.

If there will ever be a Debian user survey (again?), this question would
be a good fit. At least GRML did user surveys in the past:
http://grml.org/survey2011-results/


Carsten

[Adam Borowski]

Post by Carsten Hey
Looks like the DragonFly Mail Agent (dma), which already has been
mentioned in this thread, could become a decent default for Wheezy+1
after some small changes.
In a nutshell: it's able to deliver locally and remotely, has a queue,
supports TLS/SSL, does not listen on port 25 and instead of running as
daemon, it if run every 5 minutes via cron to flush the queue.

I hope you mean: to run retries (in which case every 5 minutes is an
overkill).

Delaying every outgoing mail by 5 minutes doesn't sound like a good idea to
me.

[Russ Allbery]

Post by Adam Borowski

Post by Carsten Hey
Looks like the DragonFly Mail Agent (dma), which already has been
mentioned in this thread, could become a decent default for Wheezy+1
after some small changes.
In a nutshell: it's able to deliver locally and remotely, has a queue,
supports TLS/SSL, does not listen on port 25 and instead of running as
daemon, it if run every 5 minutes via cron to flush the queue.

I hope you mean: to run retries (in which case every 5 minutes is an
overkill).
Delaying every outgoing mail by 5 minutes doesn't sound like a good idea
to me.

I think that was Carsten's point about switching to incron, which would
then do the right thing for new outgoing mail. You'd still need timed
handling of queued mail for retries.

(I was not previously familiar with incron; what a neat idea!)

[Carsten Hey]

As I'm not involved in developing dma at all, neither upstream nor in
Debian, I'm not the right one to discuss implementation details in depth
with.

Post by Adam Borowski

Post by Carsten Hey
Looks like the DragonFly Mail Agent (dma), which already has been
mentioned in this thread, could become a decent default for Wheezy+1
after some small changes.
In a nutshell: it's able to deliver locally and remotely, has a queue,
supports TLS/SSL, does not listen on port 25 and instead of running as
daemon, it if run every 5 minutes via cron to flush the queue.

I hope you mean: to run retries (in which case every 5 minutes is an
overkill).
Delaying every outgoing mail by 5 minutes doesn't sound like a good idea
to me.

... incron ... You'd still need timed handling of queued mail for retries.

There are two modes dma can run in, one is the deferred mode, which
seems to be basically how you think dma always works. The other is the
immediate mode that is default in Debian and upstream and as the name
suggests it delivers immediately if possible and it forks for mails that
can not be delivered immediately. The resulting processes then handle
besides obvious other things the timed handling of queued mail for
retries.


The rest of this mail is likely not interesting for most of you since it
only tries to answer the natural follow up question "Why does it need
a cronjob then?" and explains why I don't think anymore that a switch to
incron should be considered.


Two reasons for running dma -q via cronjob in my own words but stolen
from README.Debian are:
* If the queue is not empty after reboot, dma -q needs to be run at
least once to start delivering these mails. A @reboot cronjob or an
init script would also to this job.
* Delivery processes might die for various reasons, but the mails still
need to be delivered in a timely manner.

If dma would be the default MTA, then it should IMHO be as reliable as
possible and even try to prevent user errors. If a user would
unintentionally enables deferred mode (which is useful if you are behind
a dial-up line) but would not set up dma -q to run periodically, then
the mails would not be delivered without such a default cronjob.
A comment that reminds users to adapt the cronjob if needed should be
added to the config file. If dma -q is run every 5 minutes be default
anyway, the option -bq does not make that much sense anymore; this can
possibly be solved by implementing different ways of processing queued
mails. All in all, enabling the cronjob by default, as it is already
done in Debian, seems to be sane.

I think that was Carsten's point about switching to incron, which
would then do the right thing for new outgoing mail.

This is a reasonable and logical assumption, but it is wrong ;)

Actually the reason to mention incron was that I thought that running
dma -q if triggered by inotify could be a bit cheaper than running it
every five minutes. For a default MTA, the amount of systems that run
it could make considering even minimal differences in efficiency
worthwhile.

The idea was to use incron to restart failed delivery processes, if this
would be possible at all depends on details of dma and incron/inotify
I'm not aware off. An additional reason to the explanation above not to
use incron is that in rare cases dma might fail for example with ENOMEM
whilst reading its configuration file before it is able to open any file
in the spool dir, which would render running it by incron to be not 100%
bullet proof anyway.


Regards
Carsten

[Adam Borowski]

Post by Carsten Hey

Post by Carsten Hey
Looks like the DragonFly Mail Agent (dma), which already has been
mentioned in this thread, could become a decent default for Wheezy+1
after some small changes.
In a nutshell: it's able to deliver locally and remotely, has a queue,
supports TLS/SSL, does not listen on port 25 and instead of running as
daemon, it if run every 5 minutes via cron to flush the queue.

If dma would be the default MTA, then it should IMHO be as reliable as
possible and even try to prevent user errors. If a user would
unintentionally enables deferred mode (which is useful if you are behind
a dial-up line) but would not set up dma -q to run periodically, then
the mails would not be delivered without such a default cronjob.
A comment that reminds users to adapt the cronjob if needed should be
added to the config file. If dma -q is run every 5 minutes be default
anyway, the option -bq does not make that much sense anymore; this can
possibly be solved by implementing different ways of processing queued
mails. All in all, enabling the cronjob by default, as it is already
done in Debian, seems to be sane.

Not on a laptop or any machine that has to conserve power and avoid
unnecessary wakeups / disk spin-ups.

A cronjob every 5 minutes means you need to start up the process, which adds
quite a bit of churn. Worse, it will spam the logs, and since at least
auth.log is fsync()ed after every write, it needs to spin up the disk.

That's too big a price for a MTA on a system that typically goes months or
years without a single mail.

[Michael Tokarev]

[]

Post by Adam Borowski

Post by Carsten Hey
If dma would be the default MTA, then it should IMHO be as reliable as
possible and even try to prevent user errors. If a user would
unintentionally enables deferred mode (which is useful if you are behind
a dial-up line) but would not set up dma -q to run periodically, then
the mails would not be delivered without such a default cronjob.
A comment that reminds users to adapt the cronjob if needed should be
added to the config file. If dma -q is run every 5 minutes be default
anyway, the option -bq does not make that much sense anymore; this can
possibly be solved by implementing different ways of processing queued
mails. All in all, enabling the cronjob by default, as it is already
done in Debian, seems to be sane.

Not on a laptop or any machine that has to conserve power and avoid
unnecessary wakeups / disk spin-ups.
A cronjob every 5 minutes means you need to start up the process, which adds
quite a bit of churn. Worse, it will spam the logs, and since at least
auth.log is fsync()ed after every write, it needs to spin up the disk.
That's too big a price for a MTA on a system that typically goes months or
years without a single mail.

Hmm.. Now when you mentioned it...

On all our postfix servers (yes we use postfix), I mount a tmpfs over
/var/spool/postfix/run, create subdirs "pid", "private" and "public"
in there and change corresponding dirs in /var/spool/postfix/ into
symlinks to run/$subdir.

Exactly in order to avoid extra disk wakeups every so often, by default
every 5min -- when qmgr gets woken up to re-scan its queue. This is
done by master process according to master.cf, master writes a byte
into corresponding /var/spool/postfix/public/qmgr FIFO, which results
in mtime of that inode being updated every 5 minutes. Oh well.

(And when I create these symlinks, `postfix check' starts reporting
wrong permissions on /var/spool/postfix/p* - since they becomes
"world writable").

FWIW, but Postfix also has this issue, unless it is set up very
carefully and in a non-standard way :)

(This is something I use since about 2002)

Thanks,

/mjt

[Raf Czlonka]

Post by Adam Borowski
Not on a laptop or any machine that has to conserve power and avoid
unnecessary wakeups / disk spin-ups.

Or any device with an SSD or SD card (more and more popular net-tops
nowadays).

Post by Adam Borowski
A cronjob every 5 minutes means you need to start up the process, which adds
quite a bit of churn. Worse, it will spam the logs, and since at least
auth.log is fsync()ed after every write, it needs to spin up the disk.

This is the reason why I got rid of DMA on my systems, the defaults -
cron job and unnecessary entries in the logs.
Other than that I don't really have anything else against it.
If those two get fixed it could be a sane default MTA (IMHO).

Cheers,

[George Danchev]

On Monday 30 April 2012 12:58:18 Carsten Hey wrote:

Hi,

Post by Carsten Hey
The rest of this mail is likely not interesting for most of you since it
only tries to answer the natural follow up question "Why does it need
a cronjob then?" and explains why I don't think anymore that a switch to
incron should be considered.
Two reasons for running dma -q via cronjob in my own words but stolen
* If the queue is not empty after reboot, dma -q needs to be run at
init script would also to this job.

This still holds.

Post by Carsten Hey
* Delivery processes might die for various reasons, but the mails still
need to be delivered in a timely manner.

Then 'dma -q' will exit with non-zero status, and could be retried
chronologically (every 5 min.) until it succeeds.

Post by Carsten Hey
If dma would be the default MTA, then it should IMHO be as reliable as
possible and even try to prevent user errors. If a user would
unintentionally enables deferred mode (which is useful if you are behind
a dial-up line) but would not set up dma -q to run periodically, then
the mails would not be delivered without such a default cronjob.
A comment that reminds users to adapt the cronjob if needed should be
added to the config file. If dma -q is run every 5 minutes be default
anyway, the option -bq does not make that much sense anymore; this can
possibly be solved by implementing different ways of processing queued
mails. All in all, enabling the cronjob by default, as it is already
done in Debian, seems to be sane.

Post by Russ Allbery
I think that was Carsten's point about switching to incron, which
would then do the right thing for new outgoing mail.

This is a reasonable and logical assumption, but it is wrong ;)
Actually the reason to mention incron was that I thought that running
dma -q if triggered by inotify could be a bit cheaper than running it
every five minutes. For a default MTA, the amount of systems that run
it could make considering even minimal differences in efficiency
worthwhile.
The idea was to use incron to restart failed delivery processes, if this
would be possible at all depends on details of dma and incron/inotify
I'm not aware off. An additional reason to the explanation above not to
use incron is that in rare cases dma might fail for example with ENOMEM
whilst reading its configuration file before it is able to open any file
in the spool dir, which would render running it by incron to be not 100%
bullet proof anyway.

Actually, I still think that your idea of utilizing incron to trigger mail
queue flushing actions (inotify event-based), makes sense. We just need to
complete the chronological part (for the failed attempts) of the whole task.

While it is true that incron will not restart a failed dma run (AFAICT looking
at its current code; the failure will only be registered in the syslog and not
re-tried), it is also true that this task could be offloaded to a wrapper which
is to be started by incron when a file creation event happens for the mail
spool directory. The fact is that 'dma -q' will only exit with zero status if
it successfully and fully has flushed the mail queue. Having this in mind, I
think that some idempotent wrapper script around dma -q, which is to be
started by incron, should take a decent care of the whole job of flushing new
and failed mail found in spool.

It is of course not at the stage of 'verified design' and probably racy.

#!/bin/sh

SELF=`basename ${0}`
DELAY=300
RUNNING=1

is_running() {
# check for any dma instances
pidof dma >/dev/null
case $? in
0)
echo -n "dma is already running..."
RUNNING=1
return
;;
esac

# check for any instances of $SELF, excluding ourselves
pidof -o %PPID -x ${SELF} >/dev/null
case $? in
0)
echo -n "$SELF is already running..."
RUNNING=1
return
;;
esac
RUNNING=0
}

while true; do
echo -n "Attempting to flush..."
is_running
if test $RUNNING = 1; then
echo "Retrying in $DELAY seconds."
sleep $DELAY
continue
fi
/usr/sbin/dma -D -q 2>/dev/null
if test $? = 0; then
echo "ok."
break;
fi
echo "FAIL. Retrying in $DELAY seconds."
sleep $DELAY
done

[Christoph Anton Mitterer]

Post by Marco d'Itri
Is this the right time to do it?

I'd vote for it :)

Or better said for depending on a default-mta which is going to be
postifx, as already outlined.


Cheers,
Chris.

[Bernd Zeimetz]

Post by Marco d'Itri
Is this the right time to do it?

First lets fix all RC bugs and get other more important things done than
discussing - yet again - the replacement of a well working MTA by a
different well working MTA. Both are equally easy to setup and configure
with debconf for the "normal" user - and that is what counts. Those who
know what they want/need also know how to install exactly that.

[Roger Lynn]

Post by Chris Knadle
I think the reason Exim does not do this protocol conversion is that from the
point of view of an MTA author, the point of an MTA is to transmit the body of
the message without any modification to it once received, and body
modification would be required to convert 8-bit MIME to 7-bit MIME. I seem to
remember reading something along these lines in the Exim documentation years
ago and I'm having another look through it, but haven't found a reference to
this yet.

http://www.exim.org/exim-html-current/doc/html/spec_html/ch14.html#SECTalomo
says:
"This option causes Exim to send 8BITMIME in its response to an SMTP
EHLO command, and to accept the BODY= parameter on MAIL commands.
However, though Exim is 8-bit clean, it is not a protocol converter, and
it takes no steps to do anything special with messages received by this
route. Consequently, this option is turned off by default."

I have enabled accept_8bitmime in every exim I've installed for the last
10 years and no one has reported any problems. I think the risk of
encountering a truly 7 bit MTA in this decade is low enough to be
ignored for most purposes. Anyone still using one is likely to find that
a substantial fraction of their incoming mail is corrupted.

Roger

[brian m. carlson]

Post by Roger Lynn
I have enabled accept_8bitmime in every exim I've installed for the last
10 years and no one has reported any problems. I think the risk of
encountering a truly 7 bit MTA in this decade is low enough to be
ignored for most purposes. Anyone still using one is likely to find that
a substantial fraction of their incoming mail is corrupted.

I actually use Sendmail's strict 8BITMIME support to help catch spam. I
agree that 7-bit MTAs are essentially gone, but with the volume of spam
I receive, I set my mail software to be extremely strict with regard to
protocols. Legitimate software (of any sort) generally generates
protocol-compliant messages. Malicious and illicit software (and that
created by Microsoft) generally does not. Legitimate software also
generally has a userbase that will complain about rejected data if the
software is not protocol-compliant, which often leads to fixes.

I've complained to the listmasters that they send 8-bit data that is not
MIME (virtually all of which is spam) under the auspices of the 8BITMIME
extension; they refuse to fix this, and as a consequence they have to
deal with the occasional piece of undeliverable mail. This is not a
knock against the listmasters, just an observation that if you violate
the protocols, some places will reject your data.

[Roger Lynn]

Post by brian m. carlson

Post by Roger Lynn
I have enabled accept_8bitmime in every exim I've installed for the last
10 years and no one has reported any problems. I think the risk of
encountering a truly 7 bit MTA in this decade is low enough to be
ignored for most purposes. Anyone still using one is likely to find that
a substantial fraction of their incoming mail is corrupted.

I actually use Sendmail's strict 8BITMIME support to help catch spam. I
agree that 7-bit MTAs are essentially gone, but with the volume of spam
I receive, I set my mail software to be extremely strict with regard to
protocols. Legitimate software (of any sort) generally generates
protocol-compliant messages. Malicious and illicit software (and that
created by Microsoft) generally does not. Legitimate software also
generally has a userbase that will complain about rejected data if the
software is not protocol-compliant, which often leads to fixes.
I've complained to the listmasters that they send 8-bit data that is not
MIME (virtually all of which is spam) under the auspices of the 8BITMIME
extension; they refuse to fix this, and as a consequence they have to
deal with the occasional piece of undeliverable mail. This is not a
knock against the listmasters, just an observation that if you violate
the protocols, some places will reject your data.

Many MUAs have options for sending 8 bit mail[0]. Do they take notice of
whether the MTA they're talking to is 8 bit capable? It will be a while
until I have a chance to experiment.

Roger

[0] For example, in Iceape: "For messages that contain 8-bit characters,
use 'quoted printable' MIME encoding. Leave unchecked to send the
messages as is.

[Håkon Alstadheim]

Post by Roger Lynn

Post by brian m. carlson

Post by Roger Lynn
I have enabled accept_8bitmime in every exim I've installed for the last
10 years and no one has reported any problems. I think the risk of
encountering a truly 7 bit MTA in this decade is low enough to be
ignored for most purposes. Anyone still using one is likely to find that
a substantial fraction of their incoming mail is corrupted.

I actually use Sendmail's strict 8BITMIME support to help catch spam. I
agree that 7-bit MTAs are essentially gone, but with the volume of spam
I receive, I set my mail software to be extremely strict with regard to
protocols. Legitimate software (of any sort) generally generates
protocol-compliant messages. Malicious and illicit software (and that
created by Microsoft) generally does not. Legitimate software also
generally has a userbase that will complain about rejected data if the
software is not protocol-compliant, which often leads to fixes.
I've complained to the listmasters that they send 8-bit data that is not
MIME (virtually all of which is spam) under the auspices of the 8BITMIME
extension; they refuse to fix this, and as a consequence they have to
deal with the occasional piece of undeliverable mail. This is not a
knock against the listmasters, just an observation that if you violate
the protocols, some places will reject your data.

Many MUAs have options for sending 8 bit mail[0]. Do they take notice of
whether the MTA they're talking to is 8 bit capable? It will be a while
until I have a chance to experiment.
Roger
[0] For example, in Iceape: "For messages that contain 8-bit characters,
use 'quoted printable' MIME encoding. Leave unchecked to send the
messages as is.

I had a hell of a time getting gmail to accept my dkim signed messages.
SOME, not all, were being rejected. Google thought they had been
tampered with. Turns out my mail-client sends pure text mails with
Content-Transfer-Encoding: 7bit if I don't use any "8bit characters",
but it still puts in charset iso8859-1. Whenever my smarthost provider
(using qmail I believe) found a message with this impossible combination
it "helpfully" changed the headers. Don't remember which way they
shifted them.

I had to put the code below (perl) in the dkimproxy.out script at an
appropriate point:
----
if(defined($content_type) &&
defined($content_transfer_encoding) &&
($content_type =~ m(text\/plain)i) &&
($content_type =~ m(charset=iso-8859-1)i) &&
($content_transfer_encoding =~ m(7bit)i)){
...
$content_transfer_encoding =~ s(7bit)(7BIT)i;
$content_type =~ s(charset=iso-8859-1)(CHARSET=US-ASCII)i;
...
}

[Josselin Mouette]

Is this the right time to do it?

[Aron Xu]

Post by Marco d'Itri
Is this the right time to do it?

Not sure whether it's the right time, but I'm sure it's something I've
been waiting for quite some time. :-)

[Adam Borowski]

Post by Marco d'Itri
Is this the right time to do it?

No. Cron needs some way to report about its jobs, mdadm has to notify about
failures, etc, etc.

On the other hand, going from a full blown MTA like exim to something like
ssmtp or dma¹ would be a great idea.



[¹]. dma would be far better as it can handle transient failures when
configured to send to a remote host, however, that cronjob every 5 minutes
disqualifies it for the job of a laptop MTA. That's fixable, though.

[Jon Dowland]

Post by Adam Borowski

Post by Marco d'Itri
Is this the right time to do it?

No. Cron needs some way to report about its jobs, mdadm has to notify about
failures, etc, etc.

Indeed, some form of tighter guarantees that the user will get these messages
would be welcome IMHO. Perhaps all MUAs ship with a local account pre-defined
(for the "blessed" user who gets root's mail, at least).

[Aron Xu]

Post by Marco d'Itri
Is this the right time to do it?

No.  Cron needs some way to report about its jobs, mdadm has to notify about
failures, etc, etc.
On the other hand, going from a full blown MTA like exim to something like
ssmtp or dma¹ would be a great idea.

Ah, but then we should remove the "Mail server" option from d-i, which
is almost useless.

[Henrique de Moraes Holschuh]

Post by Aron Xu

Post by Marco d'Itri
Is this the right time to do it?

No.  Cron needs some way to report about its jobs, mdadm has to notify about
failures, etc, etc.
On the other hand, going from a full blown MTA like exim to something like
ssmtp or dma¹ would be a great idea.

Ah, but then we should remove the "Mail server" option from d-i, which
is almost useless.

No. We can keep it, it just doesn't need to be selected by default.

However, we really should switch the mail server option to something that
would be suitable for mail servers (i.e. something that doesn't play stupid
games with standards), such as postfix. I've advocated keeping the
status-quo in the past, but I was not aware of the exim4 brokenness.

I agree that installing "mda" by default on desktop installs would be
useful. However IME, server installs are much better served by installing
postfix, even when they're not MTAs, let alone when they are MTAs...
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh

[Aron Xu]

On Wed, May 2, 2012 at 7:24 PM, Henrique de Moraes Holschuh

Post by Aron Xu

Post by Marco d'Itri
Is this the right time to do it?

No.  Cron needs some way to report about its jobs, mdadm has to notify about
failures, etc, etc.
On the other hand, going from a full blown MTA like exim to something like
ssmtp or dma¹ would be a great idea.

Ah, but then we should remove the "Mail server" option from d-i, which
is almost useless.

No.  We can keep it, it just doesn't need to be selected by default.

Currently if I select only OpenSSH server and basic system tools in
expert mode, Exim gets installed, too.

However, we really should switch the mail server option to something that
would be suitable for mail servers (i.e. something that doesn't play stupid
games with standards), such as postfix.  I've advocated keeping the
status-quo in the past, but I was not aware of the exim4 brokenness.
I agree that installing "mda" by default on desktop installs would be
useful.  However IME, server installs are much better served by installing
postfix, even when they're not MTAs, let alone when they are MTAs...

I don't think installing mail servers for desktop installation can be
_that_ useful, not all Debian users are power users who read system
email and mutt (or likewise).

[Chris Knadle]

Post by Adam Borowski

Post by Marco d'Itri
Is this the right time to do it?

FWIW, de-selecting "standard system" tasksel option (at least when using the
netinstall .iso) results in an installation with no MTA.

Post by Adam Borowski
No. Cron needs some way to report about its jobs, mdadm has to notify
about failures, etc, etc.

There's been a lot of discussion in the past concerning several MUAs which by
default don't show local mail. [more on this below]

Post by Adam Borowski
On the other hand, going from a full blown MTA like exim to something like
ssmtp or dma¹ would be a great idea.
[¹]. dma would be far better as it can handle transient failures when
configured to send to a remote host, however, that cronjob every 5 minutes
disqualifies it for the job of a laptop MTA. That's fixable, though.

During testing DMA yesterday I realized that I hadn't set up my laptop to
forward mail externally. It turned out that the system was trying to notify
me about files existing in /lost+found. [This is not a surprise, because I'm
using XFS on top of LUKS encryption, and I recently had to hard-power-off the
box.]

I was successful in getting DMA to send mail using SMTP AUTH over TLS to port
587. [The only snag was that I had to reconfigure Mutt to set
"envelope_from=yes", otherwise the sending email address was invalid, but this
isn't DMA's fault. :-P]

-- Chris

--
Chris Knadle
***@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74

[Josh Triplett]

Post by Adam Borowski

Post by Marco d'Itri
Is this the right time to do it?

No. Cron needs some way to report about its jobs,

Cron came up in the previous discussion about this on -devel (which I'd
started), so I fixed that. See the patches in
http://bugs.debian.org/670118 for cron and http://bugs.debian.org/670137
for anacron. With those patches, cron and anacron can log job output to
syslog.

Post by Adam Borowski
mdadm has to notify about
failures, etc, etc.

mdadm already recommends an MTA, and mdadm has priority optional, making
it not part of the default install. So, an MTA could become priority
optional as well, and only get pulled in when needed. In any case,
mdadm already logs to syslog by default, and can run an arbitrary
user-specified command on alerts as well. Based on that, I'd suggest
changing the Recommends to a Suggests, so that people installing on RAID
don't end up with an MTA unless they otherwise want one.

I also fixed apt-listchanges (http://bugs.debian.org/666086) so that the
default configuration works with or without an MTA installed.

As far as I can tell, that seems sufficient to move an MTA from standard
to optional. Installing a package which Depends (or Recommends) on an
MTA will still pull one in as needed. Sysadmins who want a mail server
can easily select the "mail server" task in tasksel, or otherwise
install their mail server of choice.

- Josh Triplett

[Bjørn Mork]

Post by Marco d'Itri
Is this the right time to do it?

Wasn't this just recently discussed? Just replay the thread:
http://lists.debian.org/debian-devel/2011/10/msg00227.html


Bjørn

[Neil McGovern]

Post by Marco d'Itri
Is this the right time to do it?

No, we're about to freeze. I would try and dig out the discussion from
last time, when we were about to freeze, but I'm not sure it's worth it.
If you want to do this, then please look at it during the *start* of a
development cycle, not the end.

Neil (Release Manager du jour)