where the message originated (was: DKIM role?) (SM)

[Gordon Peterson]

It is important to remember that many individuals and many small
companies use personal or "vanity" domain names.

These domain names appear in their "From:" and "Reply-to:" addresses on
their outgoing e-mail messages, but SHOULD NOT be presumed to indicate
where the e-mail in question is arriving from.

For example, I might be travelling somewhere and sending e-mail messages
from an inhabitual location: a cruise ship Internet cafe, an Internet
access kiosk above a post office location in Beijing, a coffee bar, or
other such public-access location. In such a situation, I will
typically have NO control over what outgoing mail server is sending my
e-mail message, and since my being there is temporary I obviously don't
want to put a return address on my mail that would be connected with the
location where I physically am at the time.

Another example is where a small company also uses a company domain
name, although they might use very different mail handling for their
outgoing staff e-mail messages (for example, which might go through a
nearby ISP, often for historical reasons) as compared to their
automatically generated e-mails (example: invoices, price quotations,
EFT transfer notices, and the like) generated by their back-office
accounting systems... which might be sent directly by outgoing mail
servers inside their NAT router.

We've several times been hurt by stupid antispam approaches which refuse
perfectly legitimate e-mails because they don't like the IP address they
are coming from.

Another problem we have had is when we were sending company E-mails
through the outgoing mail server provided by our domain hosting company,
and one of the (tens of thousands) of companies they were hosting
domains for apparently became infected and began sending messages
containing a virus or something. The result was that ALL the companies
forwarding through that outgoing mail server became blocked, even though
our company (and, I'm sure, most of the companies using that same shared
outgoing mail server) was never infected or compromised.


While we're talking about the issue of antispam nuisances, another is
the situation where a spam blocker inadvertently serves as an "IP
address laundering agent" when they bounce back a message detected as
containing spam, or a virus. It is particularly stupid to bounce back a
message because it is detected to contain a virus WHICH IS KNOWN TO
FALSIFY OR COUNTERFEIT THE RETURN ADDRESS! In that case, the spam
blocker sending the 'bounce'/refused message does not know where the
original message was ACTUALLY sent from; commonly they will send the
bounce message to the COUNTERFEIT "from" address (n.b. the address the
original spammer ACTUALLY wanted to send the message to!) only now, it
arrives from a "clean" IP address (i.e. the IP address of the antispam
blocker which "bounced" it) rather than from the IP address actually
originating the message (which the actually intended recipient might
have refused, based on the sender IP address). So now the unwanted
message arrives at its intended destination, having "reflected" off a
spam reflector and thus now with a "reputable" IP address. (True, it is
now shown as an "undeliverable/spam/virus" bounce, but a lot of people
DO open such messages). ...ANYHOW, PARTICULARLY when a message is sent
by a virus KNOWN to falsify origin addresses, it is NOT helpful to send
a bounce message back to the "bogus" "origin" address indicated in the
message.

I have a folder here with probably something approaching a hundred
thousand such bounces, caused by spam messages which were sent using
bogus/falsified addresses but counterfeiting one of my domains, and
where these messages did not originate from any of my systems. The
point is that there is NO value in sending back a bounceback/refused
message unless you are SURE you know where it should be sent back to
(i.e. who actually sent it). It's NOT enough to just believe the
"From:" address.

Subject: Re: [Asrg] where the message originated (was: DKIM role?)
Content-Type: text/plain; charset="us-ascii"; format=flowed

http://www.uceprotect.net/en/rblcheck.php?ipr=202.170.42.206
but it tends to block by AS number and in the above case, AS9241 is
the whole country of Fiji.

That shouldn't be a problem if you don't communicate with people from Fiji. :-)

Also as a note, I think dealing with high volume mail sender is not
the issue, they are known, we know their technics, etc... it is more
to deal with the long tail of little companies, organisations, small
ISPs, etc..

Some receivers may view these small organizations as statistically
insignificant. These small organizations generally adopt antispam
policies without analyzing whether such policies can have a negative
impact on them in future.

White-listing based upon a domain would be dangerous without also
including the IP address of the SMTP client and message tracking.
There are companies currently providing this service, particularly
needed where spam remains largely unmanaged.

The question was whether it is important to note where the message
came from. I take it that your answer is yes.

The algorithm can remain oblivious to who owns the SMTP client. It
determines whether a conversation was observed, while also allowing
also users to submit corrections.

That only works as long as the two end-points for the conversation
are static. Such a constraint is only acceptable to users until they
move around and experience a communication failure.
Regards,
-sm

[Franck Martin]

In my experience, people are not able to modify their software to change the default smtp server. It seems more efficient to have them connect back via SMTPS+AUTH to the company mailserver and send their email that way.

Now, some webmail providers allows you to haev multiple reply-to address, that you use depending on your location. This is nice but then we fall in the categories listed below, and I'm not sure that DKIM provide a meaningful authentication of the reply-to address.

----- Original Message -----
From: "Gordon Peterson" <***@terabites.com>
To: ***@irtf.org
Sent: Sunday, 11 January, 2009 3:18:34 PM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated (was: DKIM role?) (SM)

It is important to remember that many individuals and many small
companies use personal or "vanity" domain names.

These domain names appear in their "From:" and "Reply-to:" addresses on
their outgoing e-mail messages, but SHOULD NOT be presumed to indicate
where the e-mail in question is arriving from.

For example, I might be travelling somewhere and sending e-mail messages
from an inhabitual location: a cruise ship Internet cafe, an Internet
access kiosk above a post office location in Beijing, a coffee bar, or
other such public-access location. In such a situation, I will
typically have NO control over what outgoing mail server is sending my
e-mail message, and since my being there is temporary I obviously don't
want to put a return address on my mail that would be connected with the
location where I physically am at the time.

Another example is where a small company also uses a company domain
name, although they might use very different mail handling for their
outgoing staff e-mail messages (for example, which might go through a
nearby ISP, often for historical reasons) as compared to their
automatically generated e-mails (example: invoices, price quotations,
EFT transfer notices, and the like) generated by their back-office
accounting systems... which might be sent directly by outgoing mail
servers inside their NAT router.

We've several times been hurt by stupid antispam approaches which refuse
perfectly legitimate e-mails because they don't like the IP address they
are coming from.

Another problem we have had is when we were sending company E-mails
through the outgoing mail server provided by our domain hosting company,
and one of the (tens of thousands) of companies they were hosting
domains for apparently became infected and began sending messages
containing a virus or something. The result was that ALL the companies
forwarding through that outgoing mail server became blocked, even though
our company (and, I'm sure, most of the companies using that same shared
outgoing mail server) was never infected or compromised.


While we're talking about the issue of antispam nuisances, another is
the situation where a spam blocker inadvertently serves as an "IP
address laundering agent" when they bounce back a message detected as
containing spam, or a virus. It is particularly stupid to bounce back a
message because it is detected to contain a virus WHICH IS KNOWN TO
FALSIFY OR COUNTERFEIT THE RETURN ADDRESS! In that case, the spam
blocker sending the 'bounce'/refused message does not know where the
original message was ACTUALLY sent from; commonly they will send the
bounce message to the COUNTERFEIT "from" address (n.b. the address the
original spammer ACTUALLY wanted to send the message to!) only now, it
arrives from a "clean" IP address (i.e. the IP address of the antispam
blocker which "bounced" it) rather than from the IP address actually
originating the message (which the actually intended recipient might
have refused, based on the sender IP address). So now the unwanted
message arrives at its intended destination, having "reflected" off a
spam reflector and thus now with a "reputable" IP address. (True, it is
now shown as an "undeliverable/spam/virus" bounce, but a lot of people
DO open such messages). ...ANYHOW, PARTICULARLY when a message is sent
by a virus KNOWN to falsify origin addresses, it is NOT helpful to send
a bounce message back to the "bogus" "origin" address indicated in the
message.

I have a folder here with probably something approaching a hundred
thousand such bounces, caused by spam messages which were sent using
bogus/falsified addresses but counterfeiting one of my domains, and
where these messages did not originate from any of my systems. The
point is that there is NO value in sending back a bounceback/refused
message unless you are SURE you know where it should be sent back to
(i.e. who actually sent it). It's NOT enough to just believe the
"From:" address.

[Alessandro Vesely]

Post by Gordon Peterson
For example, I might be travelling somewhere and sending e-mail messages
from an inhabitual location: a cruise ship Internet cafe, an Internet
access kiosk above a post office location in Beijing, a coffee bar, or
other such public-access location. In such a situation, I will
typically have NO control over what outgoing mail server is sending my
e-mail message, and since my being there is temporary I obviously don't
want to put a return address on my mail that would be connected with the
location where I physically am at the time.

That's a hopeless setting. If you carry a laptop, SMTP+AUTH to the
company server is the way to go, as Franck said already. If not,
assuming you won't configure an email client, you should use company's
webmail server. You are not going to store your userid/password at an
unknown box, are you? Thus, you should connect to your outgoing server
in the same way that you connect to your incoming one.

Post by Gordon Peterson
Another example is where a small company also uses a company domain
name, although they might use very different mail handling for their
outgoing staff e-mail messages (for example, which might go through a
nearby ISP, often for historical reasons) as compared to their
automatically generated e-mails (example: invoices, price quotations,
EFT transfer notices, and the like) generated by their back-office
accounting systems... which might be sent directly by outgoing mail
servers inside their NAT router.

Same comments as above apply. However, for vanity addresses the
company's MX just forwards to each employee's actual address. In that
case, outgoing mail should also originate from each employee's ESP.
The company's SPF record, if any, should take that into account.

The historical ISP habit to allow relaying from their own IP addresses
since clients already authenticated on connecting, has had its day.

Post by Gordon Peterson
We've several times been hurt by stupid antispam approaches which refuse
perfectly legitimate e-mails because they don't like the IP address they
are coming from.
Another problem we have had is when we were sending company E-mails
through the outgoing mail server provided by our domain hosting company,
and one of the (tens of thousands) of companies they were hosting
domains for apparently became infected and began sending messages
containing a virus or something. The result was that ALL the companies
forwarding through that outgoing mail server became blocked, even though
our company (and, I'm sure, most of the companies using that same shared
outgoing mail server) was never infected or compromised.

Obviously there are the same problems when sharing cars, buildings,
and sex mates: whoever catches accidents or infections affects
everybody else...

Post by Gordon Peterson
While we're talking about the issue of antispam nuisances, another is
the situation where a spam blocker inadvertently serves as an "IP
address laundering agent" when they bounce back a message detected as
containing spam, or a virus. It is particularly stupid to bounce back a
message because it is detected to contain a virus WHICH IS KNOWN TO
FALSIFY OR COUNTERFEIT THE RETURN ADDRESS!

Correct. The anti-virus filter should just swallow the message with no
complains nor notices. I don't know what RFC blesses that behavior,
though.

[Gordon Peterson]

Post by Alessandro Vesely

Post by Gordon Peterson
For example, I might be travelling somewhere and sending e-mail

messages

Post by Alessandro Vesely

Post by Gordon Peterson
from an inhabitual location: a cruise ship Internet cafe, an Internet
access kiosk above a post office location in Beijing, a coffee bar, or
other such public-access location. In such a situation, I will
typically have NO control over what outgoing mail server is sending my
e-mail message, and since my being there is temporary I obviously

don't

Post by Alessandro Vesely

Post by Gordon Peterson
want to put a return address on my mail that would be connected

with the

Post by Alessandro Vesely

Post by Gordon Peterson
location where I physically am at the time.

That's a hopeless setting. If you carry a laptop, SMTP+AUTH to the

company server is the way to go, as Franck said already.

That's precisely the point. I am not using my habitual mail client, nor
am I using my own familiar webmail service. I am using a mail
kiosk-type service which allows me to enter a subject, my return
address, a to address, and the body of my mail.

Post by Alessandro Vesely
If not, assuming you won't configure an email client, you should use

company's webmail server.

That's not an option, and nor can I configure what e-mail server is
being used.

Post by Alessandro Vesely
You are not going to store your userid/password at an

unknown box, are you?

No, of course not. And they don't ask for that. The mail message
doesn't go out through my normal e-mail channels, and that's just
exactly the point.

Post by Alessandro Vesely
Thus, you should connect to your outgoing server

in the same way that you connect to your incoming one.

Again, that's not one of the options available.

When you're travelling on a ship, understandably they want to have you
send your mail through the ship's own outgoing mail server, since that
minimizes the time they have to keep the (very expen$ive) satellite
channel open.

Post by Alessandro Vesely

Post by Gordon Peterson
Another example is where a small company also uses a company domain
name, although they might use very different mail handling for their
outgoing staff e-mail messages (for example, which might go through a
nearby ISP, often for historical reasons) as compared to their
automatically generated e-mails (example: invoices, price quotations,
EFT transfer notices, and the like) generated by their back-office
accounting systems... which might be sent directly by outgoing mail
servers inside their NAT router.

Same comments as above apply. However, for vanity addresses the

company's MX just forwards to each employee's actual address. In that
case, outgoing mail should also originate from each employee's ESP.
The company's SPF record, if any, should take that into account.

And again, that's okay, WHEN THEY ARE MAILING FROM IN-HOUSE. When they
are not, is when the problem turns up.

Post by Alessandro Vesely
The historical ISP habit to allow relaying from their own IP addresses

since clients already authenticated on connecting, has had its day.

That's not really the issue. The issue is that people occasionally have
very legitimate need to send mail other than through their normal
channels, PARTICULARLY when away from the office (at home, perhaps,
although sending a mail that's work-related; from a library or airpport
public access terminal; at a customer location; and particularly when
traveling out of the country.)

Post by Alessandro Vesely

Post by Gordon Peterson
We've several times been hurt by stupid antispam approaches which

refuse

Post by Alessandro Vesely

Post by Gordon Peterson
perfectly legitimate e-mails because they don't like the IP address

they

Post by Alessandro Vesely

Post by Gordon Peterson
are coming from.
Another problem we have had is when we were sending company E-mails
through the outgoing mail server provided by our domain hosting

company,

Post by Alessandro Vesely

Post by Gordon Peterson
and one of the (tens of thousands) of companies they were hosting
domains for apparently became infected and began sending messages
containing a virus or something. The result was that ALL the

companies

Post by Alessandro Vesely

Post by Gordon Peterson
forwarding through that outgoing mail server became blocked, even

though

Post by Alessandro Vesely

Post by Gordon Peterson
our company (and, I'm sure, most of the companies using that same

shared

Post by Alessandro Vesely

Post by Gordon Peterson
outgoing mail server) was never infected or compromised.

Obviously there are the same problems when sharing cars, buildings,

and sex mates: whoever catches accidents or infections affects
everybody else...

In this case, however, the overly-broad-brush of poisoning ALL traffic
transiting an IP address just because it has sent "some" unwanted or
malicious mail can create a GREAT deal of serious collateral damage.

Post by Alessandro Vesely

Post by Gordon Peterson
While we're talking about the issue of antispam nuisances, another is
the situation where a spam blocker inadvertently serves as an "IP
address laundering agent" when they bounce back a message detected as
containing spam, or a virus. It is particularly stupid to bounce

back a

Post by Alessandro Vesely

Post by Gordon Peterson
message because it is detected to contain a virus WHICH IS KNOWN TO
FALSIFY OR COUNTERFEIT THE RETURN ADDRESS!

Correct. The anti-virus filter should just swallow the message with no

complains nor notices. I don't know what RFC blesses that behavior,
though.

It is very frustrating to get (sometimes hundreds a day) bounce messages
that come back to my domain (catch-all address mailbox) for e-mail
messages which *I* NEVER SENT, just because the spammers' From: address
used a counterfeit/bogus e-mail address forging one of my domains.

[John Levine]

Post by Gordon Peterson
That's precisely the point. I am not using my habitual mail client, nor
am I using my own familiar webmail service. I am using a mail
kiosk-type service which allows me to enter a subject, my return
address, a to address, and the body of my mail.

Yeah, it's just one more way that the bad guys have screwed up mail
for everyone else. The trickle of mail with random to and from
addresses coming out of a mail kiosk is pretty much indistinguishable
from a zombie, so it's not surprising that they have trouble getting
mail through. Poorly conceived path authentication systems like SPF
don't help, since they encourage people to claim that their mail can
only come from their usual server.

Assuming DKIM gets traction, I can see that kiosk vendors will sign
all their mail with the kiosk's domain which will, with luck, get
a good enough reputation that receivers will say, oh, that's KioskCo,
their mail is OK. (This is an example, by the way, of the reason that
finer grain reputation is not always better.)

In the meantime, if you want to send mail while you're on the road,
you better either get a laptop with a MUA configured to relay through
home, or web mail.

Post by Gordon Peterson

Post by Alessandro Vesely
If not, assuming you won't configure an email client, you should use

company's webmail server.
That's not an option, and nor can I configure what e-mail server is
being used.

Too bad. I think it would be clearer to say that whoever runs your
mail system doesn't consider the problem of sending mail on the road
to be serious enough to fix.

Post by Gordon Peterson
In this case, however, the overly-broad-brush of poisoning ALL traffic
transiting an IP address just because it has sent "some" unwanted or
malicious mail can create a GREAT deal of serious collateral damage.

Quite true. See comments above.

R's,
John

[Alessandro Vesely]

Post by John Levine

Post by Gordon Peterson
That's precisely the point. I am not using my habitual mail
client, nor am I using my own familiar webmail service. I am
using a mail kiosk-type service which allows me to enter a
subject, my return address, a to address, and the body of my
mail.

It must be noted that being able to write but not to read one's mail
_is_ an abnormal situation, subject to severe restrictions. For one,
it breaks any address checking scheme.

Post by John Levine
Assuming DKIM gets traction, I can see that kiosk vendors will sign
all their mail with the kiosk's domain which will, with luck, get
a good enough reputation that receivers will say, oh, that's
KioskCo, their mail is OK.

However, anyone can write "Gordon Peterson <***@terabites.com>" on
that box's return address field. Do we really want that to be signed?

Post by John Levine

Post by Gordon Peterson
When you're traveling on a ship, understandably they want to
have you send your mail through the ship's own outgoing mail
server, since that minimizes the time they have to keep the (very
expen$ive) satellite channel open.

Nowadays there are satellite links whose costs are comparable to ADSL.
I'd reckon broadly available MSAs is the way to go.

A somewhat similar deceptive saving occasion used to be office MTAs.
Since most of the traffic is embodied by office-to-office messages,
one reasoned, it is a waste to route it through an external MTA that
requires a (possibly encrypted) leased connection. That's changed too.

Actually, most of the traffic is spam, and keeping ill-conditioned
mail transfer habits is not going to reduce it. Using the right MSA
for each return address allows SPF and DKIM (or other signing scheme)
to reliably authenticate messages at their respective protocol entry
points. That way, use of false identities will eventually be
eliminated, and zombies with it, for the sake of real savings.

--
"The best way of dealing with change is to help create it."

[John Levine]

Post by Alessandro Vesely

Post by John Levine
Assuming DKIM gets traction, I can see that kiosk vendors will sign
all their mail with the kiosk's domain which will, with luck, get
a good enough reputation that receivers will say, oh, that's
KioskCo, their mail is OK.

that box's return address field. Do we really want that to be signed?

Signed by KioskCo? Of course. It really did come from their kiosk.
The whole point of DKIM is that that people can reliably take
responsibility for the mail that passes through their systems,
independent of the addresses in the visible headers or the SMTP
envelope. I have no idea what kind of anti-abuse techniques KioskCo
might use, but if they're effective, why shouldn't people accept their
mail?

My point was that if all of KisokCo's kiosks apply the same signature,
that will be a large enough mailstream that recipients can form an
opinion of how good it is, even though the stream from each individual
kiosk would be too small.

R's,
John

[Alessandro Vesely]

Post by John Levine

Post by Alessandro Vesely
that box's return address field. Do we really want that to be signed?

Signed by KioskCo? Of course.

Hm.. I'm not much into DKIM. It technically allows to sign false
identities, but doesn't (or shouldn't) it semantically imply that the
signers must have some (possibly small but still positive) degree of
trust that what they sign is correct? In that case the question is
whether KioskCo would really want to sign that, and publish their
slyness in their policy.

Post by John Levine
My point was that if all of KisokCo's kiosks apply the same signature,
that will be a large enough mailstream that recipients can form an
opinion of how good it is, even though the stream from each individual
kiosk would be too small.

Although a critical mass is a common requirement of most anti-spam
measures, requiring some kind of threshold for each single sender is
more of a weakness.

[Steve Atkins]

Post by Alessandro Vesely

Post by John Levine

Post by Alessandro Vesely
that box's return address field. Do we really want that to be signed?

Signed by KioskCo? Of course.

Hm.. I'm not much into DKIM. It technically allows to sign false
identities, but doesn't (or shouldn't) it semantically imply that
the signers must have some (possibly small but still positive)
degree of trust that what they sign is correct?

No. The signature only means that the message you received was the one
signed by the signing identity.

Post by Alessandro Vesely
In that case the question is whether KioskCo would really want to
sign that, and publish their slyness in their policy.

Post by John Levine
My point was that if all of KisokCo's kiosks apply the same
signature,
that will be a large enough mailstream that recipients can form an
opinion of how good it is, even though the stream from each
individual
kiosk would be too small.

Although a critical mass is a common requirement of most anti-spam
measures, requiring some kind of threshold for each single sender is
more of a weakness.

Any mail system that only allows mail to be sent one at a time, and
requires that the mail be hand-typed (rather than stored in a
signature or pasted in) and which charges for the service via a credit
card is going to be a negligible source of abusive email.

KioskCo is definitely going to want to sign the outbound mail with
their identity, as that identity is unlikely to get a bad reputation
and will likely get a good reputation over time.

Worst case, DKIM signing the mail will have no effect. More likely it
will have some positive effect at some recipients. It's a nice example
of why DKIM signing even low volume sources of mail can be a good
idea, if they have the resources to actually do the signing.

Cheers,
Steve

[Dave CROCKER]

Post by Steve Atkins

Post by Alessandro Vesely
Hm.. I'm not much into DKIM. It technically allows to sign false
identities, but doesn't (or shouldn't) it semantically imply that the
signers must have some (possibly small but still positive) degree of
trust that what they sign is correct?

No. The signature only means that the message you received was the one
signed by the signing identity.

Not quite right. Or rather, not quite complete. And I'm compelled to pick this
nit, since it is fundamental to discussion about DKIM's purpose.

What you've described is a data integrity function. Yes, DKIM performs that on
the portions of the message it lists in the DKIM-Signature: header field.
However, data integrity is a side-effect of DKIM and not it's actual purpose.(*)

It's purpose is: "DKIM allows an organization to take responsibility for
transmitting a message, in a way that can be validated by a recipient. "

So the requirement on the signer is to choose naming granularity and use that
will provide the recipient with a stable label of a message stream.

The receiver is supposed to take the identifier being proffered by the signer
and run it through an assessment process.

Presumably, a fake or transient or new identifier is likely to get far less
trust than one with a track-record. As noted, the intended benefit of DKIM is
across a message stream, with the identifier being used to label that stream
consistently.

d/

(*) In fact, one can argue that DKIM doesn't perform data integrity all that
well, to which the response is that that's ok, it does it well enough for
validating the use of the identifier.

[Franck Martin]

I have run a series of tests, where I sign a message (sent by me) but with only the Return-path containing my domain (DKIM does not sign the return-path as recommended in the spec).

I used the DKIM reflectors on www.dkim.org

and the assessment I got was: neutral (none of the signed field contain the domain of the signer).

like if it is wrong.

I think it should be a pass. I fear that many people that verify DKIM make the same mistake.

I'm thinking of adding an X-header that will contain my domain and sign it via DKIM and see if the reflectors are happier.

----- Original Message -----
From: "Dave CROCKER" <***@dcrocker.net>
To: "Anti-Spam Research Group - IRTF" <***@irtf.org>
Sent: Tuesday, 13 January, 2009 4:27:20 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated

Post by Steve Atkins

Post by Alessandro Vesely
Hm.. I'm not much into DKIM. It technically allows to sign false
identities, but doesn't (or shouldn't) it semantically imply that the
signers must have some (possibly small but still positive) degree of
trust that what they sign is correct?

No. The signature only means that the message you received was the one
signed by the signing identity.

Not quite right. Or rather, not quite complete. And I'm compelled to pick this
nit, since it is fundamental to discussion about DKIM's purpose.

What you've described is a data integrity function. Yes, DKIM performs that on
the portions of the message it lists in the DKIM-Signature: header field.
However, data integrity is a side-effect of DKIM and not it's actual purpose.(*)

It's purpose is: "DKIM allows an organization to take responsibility for
transmitting a message, in a way that can be validated by a recipient. "

So the requirement on the signer is to choose naming granularity and use that
will provide the recipient with a stable label of a message stream.

The receiver is supposed to take the identifier being proffered by the signer
and run it through an assessment process.

Presumably, a fake or transient or new identifier is likely to get far less
trust than one with a track-record. As noted, the intended benefit of DKIM is
across a message stream, with the identifier being used to label that stream
consistently.

d/

(*) In fact, one can argue that DKIM doesn't perform data integrity all that
well, to which the response is that that's ok, it does it well enough for
validating the use of the identifier.

[Michael Thomas]

Post by Franck Martin
I have run a series of tests, where I sign a message (sent by me) but with only the Return-path containing my domain (DKIM does not sign the return-path as recommended in the spec).
I used the DKIM reflectors on www.dkim.org
and the assessment I got was: neutral (none of the signed field contain the domain of the signer).
like if it is wrong.
I think it should be a pass. I fear that many people that verify DKIM make the same mistake.

Note that this not about DKIM but about SSP/ADSP and Authentication-Results.
I believe that the SSP/ADSP result should be neutral, but that the DKIM
result is "pass". A lot of the reflectors haven't been updated for quite a
while, and the earlier drafts of Auth-Res didn't make a distinction between
DKIM and SSP/ADSP. So, true to form, differing implementations did differing
things in the face of the ambiguity.

Post by Franck Martin
I'm thinking of adding an X-header that will contain my domain and sign it via DKIM and see if the reflectors are happier.

I _think_ that my reflector does the right thing in that it separates out the
dkim results from the ssp results, but I'm pretty sure that it's out of date
wrt both the new auth-res draft and the adsp draft.

In either case, an X-header isn't going to change anything. The ADSP part is
always keyed of of the real live From address.

Mike

[Franck Martin]

I'm curious when you say ADSP is always keyed of the real live From address? You talk about the From: and not the Mail From: (Return-path)?

as a side note, all this SSP/ADSP processing looks like a blackbox (or black magic) to me. There is no recommended practices and no one explain what they do to filter mail. like in the statement "AOL will use DKIM to do build reputation based on domain", what does it mean?

We know well about spamassassin, DNSBL, DCC but this is about it. I thought security by obscurity was a bad idea? ;)

----- Original Message -----
From: "Michael Thomas" <***@mtcc.com>
To: "Anti-Spam Research Group - IRTF" <***@irtf.org>
Cc: ***@bbiw.net
Sent: Tuesday, 13 January, 2009 8:24:24 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated

Post by Franck Martin
I have run a series of tests, where I sign a message (sent by me) but with only the Return-path containing my domain (DKIM does not sign the return-path as recommended in the spec).
I used the DKIM reflectors on www.dkim.org
and the assessment I got was: neutral (none of the signed field contain the domain of the signer).
like if it is wrong.
I think it should be a pass. I fear that many people that verify DKIM make the same mistake.

Note that this not about DKIM but about SSP/ADSP and Authentication-Results.
I believe that the SSP/ADSP result should be neutral, but that the DKIM
result is "pass". A lot of the reflectors haven't been updated for quite a
while, and the earlier drafts of Auth-Res didn't make a distinction between
DKIM and SSP/ADSP. So, true to form, differing implementations did differing
things in the face of the ambiguity.

Post by Franck Martin
I'm thinking of adding an X-header that will contain my domain and sign it via DKIM and see if the reflectors are happier.

I _think_ that my reflector does the right thing in that it separates out the
dkim results from the ssp results, but I'm pretty sure that it's out of date
wrt both the new auth-res draft and the adsp draft.

In either case, an X-header isn't going to change anything. The ADSP part is
always keyed of of the real live From address.

Mike

[Robert Barclay]

Post by Franck Martin
I'm curious when you say ADSP is always keyed of the real live From
address? You talk about the From: and not the Mail From: (Return-path)?

Yes, the A is for Author. ADSP is built on top of DKIM and allows domain
owners to specify that they sign some or all mail using a specific domain in
the From: address

Post by Franck Martin
as a side note, all this SSP/ADSP processing looks like a blackbox (or
black magic) to me. There is no recommended practices and no one explain
what they do to filter mail. like in the statement "AOL will use DKIM to do
build reputation based on domain", what does it mean?

It means they are going to start establishing reputation for DKIM domains
based on the DKIM signed mail passing through their systems. This isn't any
more of a black box than their existing reputation systems.
As for recommended practices I'm not sure there's enough operational
experience is most situations to have anything useful to recommend yet. I
would be pleased to be wrong here but suspect that we may be starting to get
there for some uses of DKIM and are a long way away from that with ADSP.

Post by Franck Martin
We know well about spamassassin, DNSBL, DCC but this is about it. I thought
security by obscurity was a bad idea? ;)

Not sure this qualifies for security by obscurity. It's pretty
straightforward how all these technologies work. How people make use of the
data these technologies provide, that only seems obscure because people are
still figuring it out themselves. Compare this to how people use IP
addresses there's a pretty wide variety there and a lot of those uses would
qualify as "obscured" from outside users too.

Post by Franck Martin
----- Original Message -----
Sent: Tuesday, 13 January, 2009 8:24:24 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated

Post by Franck Martin
I have run a series of tests, where I sign a message (sent by me) but

with only the Return-path containing my domain (DKIM does not sign the
return-path as recommended in the spec).

Post by Franck Martin
I used the DKIM reflectors on www.dkim.org
and the assessment I got was: neutral (none of the signed field contain

the domain of the signer).

Post by Franck Martin
like if it is wrong.
I think it should be a pass. I fear that many people that verify DKIM

make the same mistake.
Note that this not about DKIM but about SSP/ADSP and
Authentication-Results.
I believe that the SSP/ADSP result should be neutral, but that the DKIM
result is "pass". A lot of the reflectors haven't been updated for quite a
while, and the earlier drafts of Auth-Res didn't make a distinction between
DKIM and SSP/ADSP. So, true to form, differing implementations did differing
things in the face of the ambiguity.

Post by Franck Martin
I'm thinking of adding an X-header that will contain my domain and sign

it via DKIM and see if the reflectors are happier.
I _think_ that my reflector does the right thing in that it separates out the
dkim results from the ssp results, but I'm pretty sure that it's out of date
wrt both the new auth-res draft and the adsp draft.
In either case, an X-header isn't going to change anything. The ADSP part is
always keyed of of the real live From address.
Mike
_______________________________________________
Asrg mailing list
http://www.irtf.org/mailman/listinfo/asrg
_______________________________________________
Asrg mailing list
http://www.irtf.org/mailman/listinfo/asrg

[Dave CROCKER]

Post by Franck Martin
I have run a series of tests, where I sign a message (sent by me) but with
only the Return-path containing my domain (DKIM does not sign the return-path
as recommended in the spec).

DKIM has nothing to do with the rfc5321.MailFrom address or anything else in
SMTP. It is a message-level mechanism, not transfer-level.

The dkimbase signing specification's reference to return-path cautions *against*
including it as part of the signature data.

What are you referring to about "as recommended in the spec"?

d/

[Franck Martin]

yes, that part, dkimbase reference to not include return-path as part of the signature data, because it is likely to be modified during transit.


----- Original Message -----
From: "Dave CROCKER" <***@dcrocker.net>
To: "Franck Martin" <***@avonsys.com>
Cc: ***@bbiw.net, "Anti-Spam Research Group - IRTF" <***@irtf.org>
Sent: Tuesday, 13 January, 2009 9:15:25 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated

Post by Franck Martin
I have run a series of tests, where I sign a message (sent by me) but with
only the Return-path containing my domain (DKIM does not sign the return-path
as recommended in the spec).

DKIM has nothing to do with the rfc5321.MailFrom address or anything else in
SMTP. It is a message-level mechanism, not transfer-level.

The dkimbase signing specification's reference to return-path cautions *against*
including it as part of the signature data.

What are you referring to about "as recommended in the spec"?

d/

[Alessandro Vesely]

Post by Steve Atkins

Post by Alessandro Vesely
Hm.. I'm not much into DKIM. It technically allows to sign false
identities, but doesn't (or shouldn't) it semantically imply that the
signers must have some (possibly small but still positive) degree of
trust that what they sign is correct?

No. The signature only means that the message you received was the one
signed by the signing identity.

Thanks for the clarification.

Post by Steve Atkins
Any mail system that only allows mail to be sent one at a time, and
requires that the mail be hand-typed (rather than stored in a signature
or pasted in) and which charges for the service via a credit card is
going to be a negligible source of abusive email.
KioskCo is definitely going to want to sign the outbound mail with their
identity, as that identity is unlikely to get a bad reputation and will
likely get a good reputation over time.

Wouldn't then make more sense to just sign, say, the date and the
message-ID?

Besides malicious abuses, typos are also a possible source of
confusion for end users. Considering that perhaps one day it will be
possible to read the correct email address from the payment card, if I
were KioskCo, I would avoid to sign From headers I don't trust, unless
specifically required by DKIM or related BCPs.

[N.B. "KioskCo" in this thread is understood as an example name, not
related to possibly existing companies bearing the same name.]

[Steve Atkins]

Post by Alessandro Vesely

Post by Steve Atkins

Post by Alessandro Vesely
Hm.. I'm not much into DKIM. It technically allows to sign false
identities, but doesn't (or shouldn't) it semantically imply that
the signers must have some (possibly small but still positive)
degree of trust that what they sign is correct?

No. The signature only means that the message you received was the
one signed by the signing identity.

Thanks for the clarification.

Post by Steve Atkins
Any mail system that only allows mail to be sent one at a time, and
requires that the mail be hand-typed (rather than stored in a
signature or pasted in) and which charges for the service via a
credit card is going to be a negligible source of abusive email.
KioskCo is definitely going to want to sign the outbound mail with
their identity, as that identity is unlikely to get a bad
reputation and will likely get a good reputation over time.

Wouldn't then make more sense to just sign, say, the date and the
message-ID?

Yes, absolutely it would...

... except for replay attacks.

DKIM is intended to allow the recipient to reliably identify the
signer of the message they receive. (It intentionally differentiates
between "signer" and "sender".)

Defining "the message" is important. Anyone can take a signed message,
and modify any part of the content that wasn't signed, and resend the
modified message and it will still be validly signed by the original
signer. For example, if my bank only signed the date and message-id
then I could take legitimate mail they sent me, replace the body of
the message with a phish and send it out, and it would still appear
validly signed by the original signer (the bank).

That's why DKIM (when used correctly, anyway) is also used to
demonstrate that important recipient-visible parts of the message
(such as the body and the subject line) haven't been modified since
the message was signed. (There are other possible ways to avoid replay
attacks, but DKIM chose - for good reasons, I think - to use this one).

Post by Alessandro Vesely
Besides malicious abuses, typos are also a possible source of
confusion for end users. Considering that perhaps one day it will be
possible to read the correct email address from the payment card, if
I were KioskCo, I would avoid to sign From headers I don't trust,
unless specifically required by DKIM or related BCPs.

I suspect that KioskCo would just sign the entire message - but the
reasons for that are long, vague and not terribly interesting.

Post by Alessandro Vesely
[N.B. "KioskCo" in this thread is understood as an example name, not
related to possibly existing companies bearing the same name.]

Yup.

Cheers,
Steve

[Dotzero]

Any mail system that only allows mail to be sent one at a time, and requires
that the mail be hand-typed (rather than stored in a signature or pasted in)
and which charges for the service via a credit card is going to be a
negligible source of abusive email.
KioskCo is definitely going to want to sign the outbound mail with their
identity, as that identity is unlikely to get a bad reputation and will
likely get a good reputation over time.
Worst case, DKIM signing the mail will have no effect. More likely it will
have some positive effect at some recipients. It's a nice example of why
DKIM signing even low volume sources of mail can be a good idea, if they
have the resources to actually do the signing.

Steve,

You assume the integrity of the KioskCo environment. That is not
necessarily a safe assumption to make. I will grant you that KioskCo
MIGHT be aggressive in responding to it's hosts being subverted. On
the other hand it may not. If blocking or discard of mail originating
from KioskCo is in effect, even for a short time, the individual using
that kiosk to send mail well may suffer the consequences of using a
mechanism with unknown security standards.

Just a thought.

[Ian Eiloart]

Post by Gordon Peterson

Post by Alessandro Vesely

Post by Gordon Peterson
For example, I might be travelling somewhere and sending e-mail

messages

Post by Alessandro Vesely

Post by Gordon Peterson
from an inhabitual location: a cruise ship Internet cafe, an

Internet

Post by Alessandro Vesely

Post by Gordon Peterson
access kiosk above a post office location in Beijing, a coffee bar,

or

Post by Alessandro Vesely

Post by Gordon Peterson
other such public-access location. In such a situation, I will
typically have NO control over what outgoing mail server is sending

my

Post by Alessandro Vesely

Post by Gordon Peterson
e-mail message, and since my being there is temporary I obviously

don't

Post by Alessandro Vesely

Post by Gordon Peterson
want to put a return address on my mail that would be connected with

the

Post by Alessandro Vesely

Post by Gordon Peterson
location where I physically am at the time.

That's a hopeless setting. If you carry a laptop, SMTP+AUTH to the

company server is the way to go, as Franck said already.
That's precisely the point. I am not using my habitual mail client, nor
am I using my own familiar webmail service. I am using a mail kiosk-type
service which allows me to enter a subject, my return address, a to
address, and the body of my mail.

So, what's to stop me using your email address as the return address in
that kiosk? That sort of service should not be used, except with very low
expectation of delivery. The recipient should not trust the content of the
email.

[John Levine]

Post by Ian Eiloart
So, what's to stop me using your email address as the return address
in that kiosk?

Technically, nothing.

In practice, it's a pretty poor way to defame people or send spam,
particularly if it's the kind of kiosk where you pay with a credit
card so there is an extremely reliable audit trail should someone take
offense.

Keep in mind that any kind of security is a tradeoff, and in this case
the tradeoff sure looks like a kiosk will send send mail that people
people want, not spam.

R's,
John

[Dotzero]

Post by John Levine

Post by Ian Eiloart
So, what's to stop me using your email address as the return address
in that kiosk?

Technically, nothing.
In practice, it's a pretty poor way to defame people or send spam,
particularly if it's the kind of kiosk where you pay with a credit
card so there is an extremely reliable audit trail should someone take
offense.

Given the amount of CC information floating around at relatively low
cost to people with ill intent, I would not rely on "an extremely
reliable audit trail". It will take you to a p0wn3d person in an
extremely reliable way.

Post by John Levine
Keep in mind that any kind of security is a tradeoff, and in this case
the tradeoff sure looks like a kiosk will send send mail that people
people want, not spam.

I don't have enough experience with mail kiosks (I've never used one
in my life and haven't noticed receiving from one - and yes, I do tend
to look at headers) to make an assertion one way or the other
regarding people wanting mail from kiosks. What do you base your
assertion on John? Not meaning to be combative, actually looking for a
data point.

[Rich Kulawiec]

Post by Gordon Peterson
That's not really the issue. The issue is that people occasionally have
very legitimate need to send mail other than through their normal
channels, PARTICULARLY when away from the office (at home, perhaps,
although sending a mail that's work-related; from a library or airpport
public access terminal; at a customer location; and particularly when
traveling out of the country.)

All the more reason why operations that have such a need should plan
and build to meet it: webmail clients are accessible from almost
anywhere, and decent open-source ones are free and not particularly
difficult to set up. Yes, this leaves out the tiny number of
people in odd situations like cruise ships, but that's an edge case.
(And surely someone who can afford a cruise can also afford a satellite
link or similar if email is REALLY that important to them.)

The days when we could freely forge our own addresses into the envelope
for convenience are coming to an end. I don't like it much, but
it's just another aspect of the damage caused by spammers.

Post by Gordon Peterson
In this case, however, the overly-broad-brush of poisoning ALL traffic
transiting an IP address just because it has sent "some" unwanted or
malicious mail can create a GREAT deal of serious collateral damage.

1. There's no such thing as "collateral damage" because there's no such
thing as "damage" in this context. Please review the archives of this
list from spring 2008.

2. If an address is emitting abuse, then why should anyone accept
*anything* from it, until that situation has been addressed? (Which
includes not just fixing the immediate problem, but taking steps to
avoid a repeat occurence, and, I think, issuing a public apology
and explanation. It's the least that people responsible for such
huge amounts of damage [1] can do.)

Post by Gordon Peterson
Correct. The anti-virus filter should just swallow the message with no
complains nor notices. I don't know what RFC blesses that behavior,
though.

Incorrect. The receiving MTA should reject with an appropriate error
message. While it's much more likely that a malware filter will generate
a FN than a FP, it's still possible, and an error message will be helpful
in tracking down such situations. It might -- and I may be over-optimistic
here -- also help notify the originating site that it has a problem,
if they're paying attention to their own logs, and if they're not
doing it on purpose.

The more general principle here is that mail should never disappear into
a black hole. (This is one reason why I've argued strongly against
quarantines, which I consider a fundamentally broken concept.) We all
make mistakes, the systems we deploy make mistakes, routers break,
DNS fails, etc., and the best way to give ourselves a fighting chance
of diagnosing such problems is to generate appropriate error messages.
The costs of doing so are tiny, and the potential savings in wasted
human effort are large. (Yes, I know that some horribly-broken systems
mangle carefully-generated error messages. <sigh>)

Post by Gordon Peterson
It is very frustrating to get (sometimes hundreds a day) bounce messages
that come back to my domain (catch-all address mailbox) for e-mail
messages which *I* NEVER SENT, just because the spammers' From: address
used a counterfeit/bogus e-mail address forging one of my domains.

This is backscatter spam, which results when improperly-configured mail
systems run by incompetent mail system administrators fail to reject and
instead bounce them to the putative sender. Some DNSBLs will list hosts
for backscatter -- as they should, as it's just another variety of spam.
You might consider using one of those (if you're not already), and you
should definitely consider disabling all catch-all addresses, as they're
likely targets for DDoS attacks. [2]

---Rsk

[1] I think one reasonable metric for the monetary value of damage
can be constructed by considering junk fax statutes. Here in the US,
a single junk fax is valued at $500, amount tripled under some circumstances
(and doubled again in California). If we consider a mail server
compromised for a brief period of time (one hour) and emitting messages
at a very low rate (1 per second to 1 user each), that's a minimum
of $1.8M in damages. More realistic estimates easily reach into 9 figures.

[2] I don't know if there's broad community consensus on this, but I've
been routinely disabling all catch-all addresses anywhere I've been for
over a decade. Given the huge amounts of spam being sent to never-existed
and no-longer-existing addresses, it's not only a good way to deflect a
fair amount of junk mail, it can also be useful in detecting spammers.
(The failure of some purportedly-legitimate senders to display rudimentary
mailing list management skills -- that is, to remove recipients after
a sufficient number of rejects/sufficient period of time -- is quite
annoying. I've observed that most, but not all, "amateur" mailing lists
perform far better in this regard than supposedly-"professional" senders.)

[Alessandro Vesely]

Post by Rich Kulawiec

Post by Alessandro Vesely
Correct. The anti-virus filter should just swallow the message with no
complains nor notices. I don't know what RFC blesses that behavior,
though.

Incorrect. The receiving MTA should reject with an appropriate error
message. While it's much more likely that a malware filter will generate
a FN than a FP, it's still possible, and an error message will be helpful
in tracking down such situations.

Rejecting has been my first approach when I first installed an
avfilter. When I realized the number of infections I was causing that
way, I started to drop rather than reject. Since 2001, I've had a
couple of FNs and no FP that I'm aware of. (I'm talking specific virus
detection, not heuristic anti-malware filters.)

Post by Rich Kulawiec
It might -- and I may be over-optimistic
here -- also help notify the originating site that it has a problem,
if they're paying attention to their own logs, and if they're not
doing it on purpose.

I have in front of me the latest instance of W32/Netsky-D I received.
(I still dump viral messages in a system folder not readable by
users.) It is a bounce, so in this case I could have rejected it with
less worries. What reaction would the remote postmaster have had? I
have difficulties in estimating the worthiness of amending my software
so as to reject rather than drop in case the envelope sender is void.

For regular messages, rejection causes viruses to be delivered to end
users, which is wrong because it spreads them. I don't think virus
writers spread their original copies via their own MTAs; that is to
say, IMHO MTAs are not doing it on purpose. However, I assume the
remote MTA has no avfilter, otherwise it wouldn't relay viruses. Most
probably, on rejection it will attempt to deliver a bounce. Failure to
do so would result in the same net effect as dropping; so let's
consider only infected bounces that are successfully delivered to end
users: what are the chances that one of them results in an alert
rather than yet another infection? Let's over-optimistically estimate
we can get 1 alert for every 10 new infections. I don't deem that is
anywhere near a desirable outcome. If I were a judge I would condemn
postmasters wittingly doing so.

Post by Rich Kulawiec
The more general principle here is that mail should never disappear into
a black hole.

Agreed. A virus should be caught right after the client submits it.
Reject vs. drop has a different meaning in this case: diagnosing an
infected client should sound all available whistles and bells in
either case.

For relayed infected stuff, feedback loops may be a valid alternative
to the black hole. However, zombie detection is not a topic restricted
to email management.

[Rich Kulawiec]

Post by Rich Kulawiec
Incorrect. The receiving MTA should reject with an appropriate error
message. While it's much more likely that a malware filter will generate
a FN than a FP, it's still possible, and an error message will be helpful
in tracking down such situations.

Rejecting has been my first approach when I first installed an avfilter.
When I realized the number of infections I was causing that way, I
started to drop rather than reject.

I'll probably have more to comment on in your message, but I stopped
here because I'm confused and puzzled, and could use some clarification
on what you mean by "infections...[you were] causing". I don't understand
how issuing an SMTP reject could cause an infection; wouldn't doing so
be the equivalent of telling an already-infected host "you're infected"?

Could you expand on what you mean by this?

---Rsk

[der Mouse]

Post by Alessandro Vesely
Rejecting has been my first approach when I first installed an
avfilter. When I realized the number of infections I was causing
that way, I started to drop rather than reject.

[I] could use some clarification on what you mean by
"infections...[you were] causing". I don't understand how issuing an
SMTP reject could cause an infection; wouldn't doing so be the
equivalent of telling an already-infected host "you're infected"?

Well, I didn't write it. But I interpreted it as, basically, this
scenario:

- Malware goes out, addressed to A, (forged) envelope-from B. Sending
channel ends up emitting it from a normal MTA, M.

- A's MX host rejects it at SMTP time.

- M generates and sends a bounce to B.

- B receives bounce with embedded malware. Somehow - perhaps B's MUA
aggressively looks for and executes live content; perhaps B clicks
on the wrong thing; perhaps something else - this ends up with a
malware infestation on B's machine. (Cue xkcd #350.)

If A's MX host had silently swallowed the mail, nothing would have
happened to B - or, at least, not on account of this message. (This is
not to say that _I_ think it's fair to say that A's rejection caused B
to get infected. Just that this is what I think Alessandro meant.)

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

[Rich Kulawiec]

Post by der Mouse
Well, I didn't write it. But I interpreted it as, basically, this
- Malware goes out, addressed to A, (forged) envelope-from B. Sending
channel ends up emitting it from a normal MTA, M.
- A's MX host rejects it at SMTP time.
- M generates and sends a bounce to B.
- B receives bounce with embedded malware. Somehow - perhaps B's MUA
aggressively looks for and executes live content; perhaps B clicks
on the wrong thing; perhaps something else - this ends up with a
malware infestation on B's machine. (Cue xkcd #350.)
If A's MX host had silently swallowed the mail, nothing would have
happened to B - or, at least, not on account of this message. (This is
not to say that _I_ think it's fair to say that A's rejection caused B
to get infected. Just that this is what I think Alessandro meant.)

Ah, gotcha. I agree that silently swallowing the message might have
spared B a possible infection, but I'm reluctant to blame A's MX for
this: it didn't originate, accept or transfer the malware-laden message.
I think responsibility lies with M, and/or with whatever system passed
the message to M (presuming the message didn't originated on M).

While this is a less-than-desirable outcome, I think it may be the best
we can do -- we can't block something before we see it, we can only do
our best to reject outright spam/malware/junk traffic at the earliest
available opportunity. (One of the ways I've expressed this elsewhere
is that the best place to stop spam is as near its source as possible.
The farther it gets from the origin, the trickier detecting it gets,
and the greater the possible consequences.)

---Rsk

[Alessandro Vesely]

Post by Rich Kulawiec

Post by der Mouse
- Malware goes out, addressed to A, (forged) envelope-from B. Sending
channel ends up emitting it from a normal MTA, M.
- A's MX host rejects it at SMTP time.
- M generates and sends a bounce to B.
- B receives bounce with embedded malware. Somehow - perhaps B's MUA
aggressively looks for and executes live content; perhaps B clicks
on the wrong thing; perhaps something else - this ends up with a
malware infestation on B's machine. (Cue xkcd #350.)
If A's MX host had silently swallowed the mail, nothing would have
happened to B - or, at least, not on account of this message.

Ah, gotcha. I agree that silently swallowing the message might have
spared B a possible infection, but I'm reluctant to blame A's MX for
this: it didn't originate, accept or transfer the malware-laden message.

A's MX knows that M lacks effective anti-virus filtering. Hence,
through inaction, it allowed a human being to come to harm. That
obviously breaks the first law.

[Rich Kulawiec]

Post by Rich Kulawiec
Ah, gotcha. I agree that silently swallowing the message might have
spared B a possible infection, but I'm reluctant to blame A's MX for
this: it didn't originate, accept or transfer the malware-laden message.

A's MX knows that M lacks effective anti-virus filtering. Hence, through
inaction, it allowed a human being to come to harm. That obviously breaks
the first law.

But none of this is necessarily true.

- While I'm not overly conversant in the nuances of anti-virus
systems (I prefer to use operating systems relatively unaffected
by them) I'm under the impression that there's wide variability
in the detection of various strains by different AV products.
So it might not be the case that M lacks effective anti-virus
filtering; it may well be the case that A's MX was simply fortuitous
enough to be presented with a specimen that it detected while
M did not. It may even be the case that this is the exception
to the rule -- that is, that M's virus detection is in fact
markedly better than that at A's MX, but the latter simply got lucky.

- Add to the discussion above the dependency of many (most?)
anti-virus products on signature databases which must be updated
rather frequently in order to deal with fast-propagating malware.
It may be the case that M is in fact running the same anti-virus
product as A's MX, but is mere minutes behind in updating.

- A human being won't necessarily be harmed by this. For starters,
there's no way to know that the message will be delivered. If the
putative sender is elsewhere, then the mail system there might
detect and reject the message. The putative sender's mail client
may be using an anti-virus product that detects it. Or the putative
sender may never "open" the message (which seems to effectively
defuse most, but not all, of these problems).

- Or the putative sender may not be running a mail client that's
easily hijacked by malware or may not be running an operating system
that's so brittle. (e.g.: mutt on OpenBSD.)

There are still more variables in play here, but I think this is enough to
illustrate why attempting to guess at them is futile. It's best to just
reject the message and content yourself that (a) you've done all you can do
(b) you've emitted a reasonable diagnostic message in case this is a goof
(c) you've minimized the amount of SMTP traffic you're emitting -- a key
factor and (d) you've done something that passes the "what if everyone
did this?" test, another key factor.

---Rsk

[David Wilson]

Post by Rich Kulawiec
It's best to just
reject the message and content yourself that (a) you've done all you can do
(b) you've emitted a reasonable diagnostic message in case this is a goof
(c) you've minimized the amount of SMTP traffic you're emitting -- a key
factor and (d) you've done something that passes the "what if everyone
did this?" test, another key factor.

If by "reject" you mean "give a 5xx response to the data from the SMTP
client, the problem with that is that, as previously discussed, the SMTP
client server may generate a DSN or similar which returns the infected
message. It does not know that the reason for non-delivery was the
suspect message content. You do. So the only safe responses are:
- discard the message (i.e. accept it over SMTP and then throw it away
with no further processing, apart from possible archiving).
- send a DSN or similar which suppresses return of content.

In the old days you could get messages which contain inadvertently
infected attachments (e.g. in the days of Word macro viruses). I would
think that the fraction of infected messages which are like this is
negligibly small. So, no-one needs to know that the message is being
non-delivered, so discarding the message is the best policy.

David

[Rich Kulawiec]

Post by David Wilson
If by "reject" you mean "give a 5xx response to the data from the SMTP
client, the problem with that is that, as previously discussed, the SMTP
client server may generate a DSN or similar which returns the infected
message. It does not know that the reason for non-delivery was the
- discard the message (i.e. accept it over SMTP and then throw it away
with no further processing, apart from possible archiving).
- send a DSN or similar which suppresses return of content.

I've got serious problems with both of those.

In the first case, you're telling the sending server one thing and
doing another. I don't think that's a good idea on two levels:
at a philosophical level, this seems like a bad way to run an Internet.
At a practical level, doing so fails to alert the sending site that
they may have a problem. (Yes, I'm presuming that they're paying
attention to their own logs...or that they will be more inclined to
do so when they notice 86,000 5xx error messages in them.)

In the second case, generating DSNs is a bad idea in general [1],
but it's a REALLY bad idea as a response to spam/virus messages.

Look, I understand the motivation, and it's very noble, but it's
also pointless. A hypothetical virus relying on this method of
propagation will also have many other -- many other SIMPLER --
ways of delivering itself to the very same targets. The best that
anyone external can do is (a) don't propagate the problem further
(b) communicate via appropriate use of SMTP that there's a problem
and (c) don't make the problem worse or more complicated or more obscure.

---Rsk

[1] It's not that hard to engineer mail architectures -- including
distributed and multi-tier architectures -- that always reject and
never bounce. Yes, there are always edge cases to be dealt with,
but making them exceptions and not the rule makes it easier to
then minimize the occurence/duration of those exceptions.

[David Wilson]

Post by Rich Kulawiec

Post by David Wilson
If by "reject" you mean "give a 5xx response to the data from the SMTP
client, the problem with that is that, as previously discussed, the SMTP
client server may generate a DSN or similar which returns the infected
message. It does not know that the reason for non-delivery was the
- discard the message (i.e. accept it over SMTP and then throw it away
with no further processing, apart from possible archiving).
- send a DSN or similar which suppresses return of content.

I've got serious problems with both of those.
In the first case, you're telling the sending server one thing and
at a philosophical level, this seems like a bad way to run an Internet.
At a practical level, doing so fails to alert the sending site that
they may have a problem. (Yes, I'm presuming that they're paying
attention to their own logs...or that they will be more inclined to
do so when they notice 86,000 5xx error messages in them.)
In the second case, generating DSNs is a bad idea in general [1],
but it's a REALLY bad idea as a response to spam/virus messages.
Look, I understand the motivation, and it's very noble, but it's
also pointless. A hypothetical virus relying on this method of
propagation will also have many other -- many other SIMPLER --
ways of delivering itself to the very same targets. The best that
anyone external can do is (a) don't propagate the problem further
(b) communicate via appropriate use of SMTP that there's a problem
and (c) don't make the problem worse or more complicated or more obscure.
---Rsk
[1] It's not that hard to engineer mail architectures -- including
distributed and multi-tier architectures -- that always reject and
never bounce. Yes, there are always edge cases to be dealt with,
but making them exceptions and not the rule makes it easier to
then minimize the occurence/duration of those exceptions.

I'm not sure what you are saying should be done. If you reject from an
SMTP MTA (not a submitting UA) using a 5xx response, what is that MTA
going to do: generate some kind of non-delivery message. Normally, that
includes the message which was non-delivered, which the rejecting MTA
knows to be malicious. In order to prevent that, we need a status code
which means "I am not receiving this message because it has a content
which is dangerous". SMTP does not have such a status code.

(We are not talking about spam here, which is a different case, we are
talking about infected messages.)

[Rich Kulawiec]

Post by David Wilson
I'm not sure what you are saying should be done. If you reject from an
SMTP MTA (not a submitting UA) using a 5xx response, what is that MTA
going to do: generate some kind of non-delivery message. Normally, that
includes the message which was non-delivered, which the rejecting MTA
knows to be malicious. In order to prevent that, we need a status code
which means "I am not receiving this message because it has a content
which is dangerous". SMTP does not have such a status code.

Yes, I know. (And while I recognize that having such a status code
might help solve some problems, it would also open others. Among other
things, "malicious" isn't universal. And anti-virus software does not
have a 0% FP rate.)

What I'm saying is that the recipient MTA should just 5XX the message
and hang up. [1] That's all it can do without either (a) making the problem
worse and/or (b) redirecting the problem and/or (c) doing something
very broken like claiming to accept a message and then /dev/null'ing it.

There's no way to "fix" the presenting MTA, if we agree for the sake
of argument that it's broken. There's not even any way, with the limited
information available during the SMTP conversation, to figure out *how*
it's broken. Maybe it's an open relay; maybe it's not but is relaying
a virus from an already-infected user; maybe it itself is infected;
maybe a kazillion other things.

So the best course of action is that which contains the damage, doesn't
break anything else, minimizes SMTP traffic: 5XX it and move on. There's
no way to know what the consequences will be on the presenting side --
maybe none, maybe some, maybe a lot -- so there's no point in guessing.
At least the 5XX gives the presenting side a "heads up", if they're
paying attention, that something's amiss.

---Rsk

[1] If the problem repeats incessantly or rapidly, maybe 4XX'ing
everything for a while would be in order. In some cases, refusing
the incoming connection might be appropriate.

[David Wilson]

And anti-virus software does not have a 0% FP rate.)

I would be interested to know what actual numbers for, or perhaps some
specific instances of, FP from anti-virus.

I would be surprised if a non-malicious message would fall foul of AV
software unless it contained some kind of executable content. It should
not be surprising that a message with executable content runs into
problems.

FN rates for AV are non-zero, particularly in the few hours after the
appearance of a new variant, when the different AV folk are updating
their signatures, and the users are downloading them. So it is sometimes
not surprising that one MTA finds a virus in a message from another MTA
which is doing AV, but the message just hit it in a unfortunate window.

[John Levine]

Post by David Wilson
I would be surprised if a non-malicious message would fall foul of AV
software unless it contained some kind of executable content. It should
not be surprising that a message with executable content runs into
problems.

When I catch a virus and the sending IP is one for whom I have a known
contact address, I send off an autoreport with the first 50 lines of
the virus (in case they're wondering what virus it is), and the first
line of the base64 of the virus denatured with xxxx so that even if an
overenthusiastic Windows MUA were to try to run it, it wouldn't start.

Nonetheless, I have problems all the time with my reports being
rejected by poorly written virus filters. In one case they've been
adding me to a virus sending blacklist, telling me that even though
they know I'm not sending viruses, their AV detects it so it must be
my fault. Sheesh.

R's,
John

[SM]

Post by John Levine
When I catch a virus and the sending IP is one for whom I have a known
contact address, I send off an autoreport with the first 50 lines of
the virus (in case they're wondering what virus it is), and the first

[snip]

Post by John Levine
Nonetheless, I have problems all the time with my reports being
rejected by poorly written virus filters. In one case they've been
adding me to a virus sending blacklist, telling me that even though
they know I'm not sending viruses, their AV detects it so it must be
my fault. Sheesh.

You also get that kind of reply (if detectors says so, then it must
be true) about spam.

Post by John Levine
I would be surprised if a non-malicious message would fall foul of AV
software unless it contained some kind of executable content. It should
not be surprising that a message with executable content runs into
problems.

The usual document formats (PDFs, .doc, etc) are also scanned for
viruses nowadays. They can fall foul of Anti-virus software. The
report (see first paragraph above) would have helped in identifying
problems if the postmaster actually cared about mail delivery.

Regards,
-sm

[Franck Martin]

I block a number of file extensions (all of which could contain a payload). I'd like to block doc attachments but they are a bt everywhere. I advice people to send as pdf (internally we usually don't do much than print/or read external documents) or to rename the extension. At least there is a not straight forward process, whihc gives an extra 5picosecond for people to think if it is a virus or not.

----- Original Message -----
From: "SM" <***@resistor.net>
To: "Anti-Spam Research Group - IRTF" <***@irtf.org>
Sent: Thursday, 15 January, 2009 3:56:34 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] virus detectors, was where the message originated

Post by John Levine
When I catch a virus and the sending IP is one for whom I have a known
contact address, I send off an autoreport with the first 50 lines of
the virus (in case they're wondering what virus it is), and the first

[snip]

Post by John Levine
Nonetheless, I have problems all the time with my reports being
rejected by poorly written virus filters. In one case they've been
adding me to a virus sending blacklist, telling me that even though
they know I'm not sending viruses, their AV detects it so it must be
my fault. Sheesh.

You also get that kind of reply (if detectors says so, then it must
be true) about spam.

Post by John Levine
I would be surprised if a non-malicious message would fall foul of AV
software unless it contained some kind of executable content. It should
not be surprising that a message with executable content runs into
problems.

The usual document formats (PDFs, .doc, etc) are also scanned for
viruses nowadays. They can fall foul of Anti-virus software. The
report (see first paragraph above) would have helped in identifying
problems if the postmaster actually cared about mail delivery.

Regards,
-sm

[Rich Kulawiec]

Post by David Wilson
I would be interested to know what actual numbers for, or perhaps some
specific instances of, FP from anti-virus.

As more and more AV packages are being (incorrectly, in my opinion)
taught how to recognize phishes, I'm seeing more and more FP rejects
of *discussions* of phishes.

Like John, I've seen rejects based on partial virus-related content -
in some cases, apparently based on string matches.

But it is axiomatic that all anti-virus (and anti-spam) systems have
nonzero FP and FN rates (with the exception of the edge cases "reject all"
and "accept all"). Given that we know we will make these mistakes,
we need to consider how we can make them visibly, so that we give
ourselves and everyone else a reasonable chance of observing and
correcting them.

---Rsk

[Alessandro Vesely]

Can we draw any conclusions from that thread?

Post by Rich Kulawiec

Post by David Wilson
I'm not sure what you are saying should be done. If you reject from an
SMTP MTA (not a submitting UA) using a 5xx response, what is that MTA
going to do: generate some kind of non-delivery message. Normally, that
includes the message which was non-delivered, which the rejecting MTA
knows to be malicious. In order to prevent that, we need a status code
which means "I am not receiving this message because it has a content
which is dangerous". SMTP does not have such a status code.

Yes, I know. (And while I recognize that having such a status code
might help solve some problems, it would also open others.

599 Bounce to postmaster. What would be wrong if it existed? (I mean,
besides how hard it would be to reliably introduce it now.)

Post by Rich Kulawiec
Among other things, "malicious" isn't universal. And anti-virus software
does not have a 0% FP rate.)

I agree it cannot be 0%, but better than 0.000001% is expected. See
e.g. http://www.sophos.com/support/knowledgebase/article/35504.html.
It is also false that any executable content is likely to be reported
as infected: the same AV software is used to scan hard disks, looking
at executables only. (I'm not saying the same reliability cannot be
reached for phishing or spam, though.)

Post by Rich Kulawiec
Our experience, issuing 55x at levels of 250,000-1,300,000 per day since
1997 indicates exactly the reverse. [...] Silently dropping emails would
mean that those dozen or so FPs would go unnoticed by anyone.

A dozen at those levels is more than 0.001%. It seems very poor. What
AV/config do you use? Perhaps, at that percentage I would do the same.
The reliability I experience reliefs me when I violate the SMTP
protocol by dropping messages. Since I have to err, I try and minimize
damages.

Curiously, some viruses (e.g. Netsky) directly produce a bounce.
Possibly, their authors reasoned that since there are brain damaged
MUAs that conceal the body of DSNs but not the attachments, an
attachment in a DSN has better probabilities of being opened. As a
matter of fact, some virus variants have been surviving autonomously
for years, which suggests that that reasoning was correct.

Bounce addresses are extremely likely to be spoofed. Sending
disinfected bounces is hardly an option. At any rate, even when it is
not spoofed, the author of a message is not always the right recipient
for a technical report. Besides malware, SPF or DKIM rejections are
difficult to understand and impossible to fix for end users.

MTAs strive to only produce bounces for messages that they have
accepted for relay from their legitimate users. However, in some cases
that's not possible. Forwarding comes to mind. Also, rfc5321 mandates
no AV filter, thus the much exemplified virus-relaying MTA is
perfectly legal.

Why don't we have a meta channel for those cases? Some bounces should
be sent to postmasters, who can then send more meaningful DSNs,
possibly after seeking the relevant message-ID in their logs, and fix
the problem. I don't think sending to <***@example.com> would
make much sense, there ought to be a better mechanism...

[Rich Kulawiec]

Post by Alessandro Vesely
599 Bounce to postmaster. What would be wrong if it existed? (I mean,
besides how hard it would be to reliably introduce it now.)

I think -- even if there was widespread concurrence that it's a great
idea -- "years" would be the timeline.

And I'm not sure it's advisable or even worth it. Let me explain:

I tend to loosely group mail system operators into two ad hoc categories:
the first read "postmaster" mail, examine their inbound/outbound logs,
run statistics on them, note anomalies, investigate them, and are pretty
much invisible to me because their system are rarely, if ever, a problem.
The second don't have a working "postmaster" address, pay no attention to
their logs, complain vociferously when [correct] allegations of problems
are made, and as a consequence, are a major irritant.

Those in the first group are likely to know before you need to
tell them. They'll note an unusually high number of 5XX responses
to outbound traffic, investigate, and figure it out. Those in the
second group will ignore you (most likely) or deny the facts,
or accuse you of some malicious act, or something less pleasant.

So there's little need to tell the first and little point in telling
the second.

Post by Alessandro Vesely

Post by Rich Kulawiec
Among other things, "malicious" isn't universal. And anti-virus software
does not have a 0% FP rate.)

I agree it cannot be 0%, but better than 0.000001% is expected.

I think that's hopelessly optimistic in real-world settings. I routinely
see a handful of FP's every month -- then again, I tend to send out mail
talking about spam and phishes and so on, which most people don't.
Also see Chris's excellent explanation, which I think is roughly
typical of that at many large sites (it's certainly similar to the
large sites I've worked on).

Besides: AV vendors issue incorrect signatures, people misconfigure
their mailers (I've seen multiple instances of reversed tests), networks
fail, routers hiccup, DNS botches, and so on. There are so many things
that go wrong that our only chance at diagnosis and repair relies on
appropriate error messages.

Post by Alessandro Vesely
Why don't we have a meta channel for those cases? Some bounces should be
sent to postmasters, who can then send more meaningful DSNs, possibly
after seeking the relevant message-ID in their logs, and fix the problem.

Bounces are a bad idea because they add still more SMTP traffic, they
can easily be abused to conduct DDoS attacks, and because of the
situation outlined above.

---Rsk

[David Wilson]

Post by Rich Kulawiec
Bounces are a bad idea because they add still more SMTP traffic, they
can easily be abused to conduct DDoS attacks, and because of the
situation outlined above.

Something on which we are agreed!

However, what I have been saying is that rejecting using 5xx will
generate a bounce if the reject is sent as a response to an intermediate
MTA.

If the message is very likely to be dangerous, rather than just
inconvenient, any bounce containing the message is itself dangerous. No
doubt, most such dangerous messages come directly from some compromised
PC, and a 5xx response will not give a bounce. But there will be some
proportion which come via an intermediate MTA, which will.

[Chris Lewis]

Post by David Wilson
However, what I have been saying is that rejecting using 5xx will
generate a bounce if the reject is sent as a response to an intermediate
MTA.

And what I've been saying is that this is an insignificant issue in reality.

"Intermediate" MTAs, in the sense of MTAs _other_ than the sender's or
recipient's perimeter/internal servers are insignificant in the scheme
of things.

[David Wilson]

Post by Franck Martin

Post by Alessandro Vesely

Post by Rich Kulawiec
Among other things, "malicious" isn't universal. And anti-virus

software

Post by Alessandro Vesely

Post by Rich Kulawiec
does not have a 0% FP rate.)

I agree it cannot be 0%, but better than 0.000001% is expected.

I think that's hopelessly optimistic in real-world settings. I routinely
see a handful of FP's every month -- then again, I tend to send out mail
talking about spam and phishes and so on, which most people don't.
Also see Chris's excellent explanation, which I think is roughly
typical of that at many large sites (it's certainly similar to the
large sites I've worked on).

If I read Chris' message, then I believe that he is not giving evidence
for AV false positives. He does not trust AV software, so the rejects
quoted are for *all* filtering (presumably various kinds of anti-spam,
as well as AV scanning) and the false positives quoted are for reports
he gets back from rejects (many of which I suspect are seen by the user
in a bounce generated from the MTA which received the reject). So there
is actually no data here on specifically AV false positives, they are
data on false positives from the overall filtering system.

If someone discusses the nature of spam messages or phishing messages
via email, I would think that false positives are not surprising;
annoying, but not surprising. After all, one can be sending a message
containing precisely the kind of thing filters are attempting to detect.
So, I would say that there is a class of positives which should not
count against the detection mechanism.

[Chris Lewis]

Post by David Wilson

Post by Franck Martin

Post by Alessandro Vesely

Post by Rich Kulawiec
Among other things, "malicious" isn't universal. And anti-virus

software

Post by Alessandro Vesely

Post by Rich Kulawiec
does not have a 0% FP rate.)

I agree it cannot be 0%, but better than 0.000001% is expected.

I think that's hopelessly optimistic in real-world settings. I routinely
see a handful of FP's every month -- then again, I tend to send out mail
talking about spam and phishes and so on, which most people don't.
Also see Chris's excellent explanation, which I think is roughly
typical of that at many large sites (it's certainly similar to the
large sites I've worked on).

If I read Chris' message, then I believe that he is not giving evidence
for AV false positives.

That wasn't my point. My point was directly as to the "hazard" of
550-rejecting viruses. In that, despite having 550-rejected millions of
viruses (1.3M Mydooms/day at peak), we haven't, in 11 years, heard of
_one_ virus bounced by a MTA receiving one of our 550's landing in
anyone's lap, let alone infecting anyone.

That even if somehow blaming a virus on us for a 550 is extremely
unlikely, that out of 10's of millions of real viruses being rejected,
we would have heard of at least _one_. But we haven't.

Thus, the hazards of 550'ing viruses are vastly overblown.

Furhermore, since virus-intended rules aren't FP-free, the hazard of
losing the DSN on a FP is far higher than the largely non-existent
hazard of 550-ing a virus.

[Alessandro Vesely]

Post by Rich Kulawiec

Post by Alessandro Vesely
599 Bounce to postmaster. What would be wrong if it existed? (I mean,
besides how hard it would be to reliably introduce it now.)

I think -- even if there was widespread concurrence that it's a great
idea -- "years" would be the timeline.

Yup, that's why I said "if it already existed".

Post by Rich Kulawiec
[attentive vs. lame postmasters dissection dissertation]
So there's little need to tell the first and little point in telling
the second.

Yet, attentive postmasters are not omniscient. They need a data feed.

Post by Rich Kulawiec

Post by Alessandro Vesely
I agree it cannot be 0%, but better than 0.000001% is expected.

I think that's hopelessly optimistic in real-world settings.

Not for an AV filter. People routinely scan their hard disks with AV;
perhaps they miss some viruses, but no legitimate software is
quarantined. Obviously bugs exist, and FPs are a particular kind of
bug for an AV package. I never saw one, but each AV vendor should have
a list of open issues, and possibly also of the closed historical records.

SMTP implementation also have bugs. However, when discussing the
protocol we take it for granted that they can be fixed.

Post by Rich Kulawiec
Also see Chris's excellent explanation, which I think is roughly
typical of that at many large sites (it's certainly similar to the
large sites I've worked on).

Chris said their filter is not able to distinguish viruses from
generic malware. Otherwise, for viruses they could issue a "599 Don't
bounce this dangerous content to the user" after data transfer, if
that code existed...

Post by Rich Kulawiec
Besides: AV vendors issue incorrect signatures, people misconfigure
their mailers (I've seen multiple instances of reversed tests), networks
fail, routers hiccup, DNS botches, and so on. There are so many things
that go wrong that our only chance at diagnosis and repair relies on
appropriate error messages.

Having an appropriate error message is not enough. It is also
necessary to deliver that message to the right operator. Delivering
error messages to end users may be counter productive. For a non-viral
example, what can users do if their mail is bounced because of bad
DKIM signatures?

Post by Rich Kulawiec

Post by Alessandro Vesely
Why don't we have a meta channel for those cases? Some bounces should be
sent to postmasters, who can then send more meaningful DSNs, possibly
after seeking the relevant message-ID in their logs, and fix the problem.

Bounces are a bad idea because they add still more SMTP traffic, they
can easily be abused to conduct DDoS attacks, and because of the
situation outlined above.

That's exactly why I proposed a meta channel: to direct error messages
to someone who can act appropriately.

Some large sites have established feedback loops whereby a message is
"bounced" to some postmaster. Apparently, they are mainly meant for
"this is spam" actions. However, the ARF format (quite similar to DSN)
provides fields for reporting bad DKIM signatures. I don't know at
what level such bounces could be generated. It is technically possible
to generate them right after the data transfer, just like for viral
content. If we recognize that viruses are a problem, don't they
deserve using that meta channel as well? This leaves us wondering how
can such a meta channel be established for small and medium sites as
well...

[Rich Kulawiec]

Post by Alessandro Vesely
Yet, attentive postmasters are not omniscient. They need a data feed.

It's called "their own log files". Those who do not pay attention to
what's in them are highly unlikely to pay attention to anything else.

Post by Alessandro Vesely
Having an appropriate error message is not enough. It is also necessary
to deliver that message to the right operator. Delivering error messages
to end users may be counter productive. For a non-viral example, what can
users do if their mail is bounced because of bad DKIM signatures?

Nothing. Again, it's up to mail system administrators to read their
own log files and act (if necessary) on what they find. I would
think a log file full of SMTP rejects with "Hey, your DKIM signature
is wrong" would get their attention. If not -- hey, if they don't care,
why should anyone else?

Post by Alessandro Vesely
That's exactly why I proposed a meta channel: to direct error messages
to someone who can act appropriately.

But there's no point in going through all the debate and design
and implementation and deployment effort associated with this.
The same people who are presently ignoring their own log files
will ignore this too.

A much easier way to reach that subset of postmasters who are actually
paying attention is to use appropriately verbose messages in rejection
messages. For example:

message rejected

isn't nearly as helpful as

message rejected; our antivirus scanner detected Blottosplang; contact ***@example.com if you believe this is an error and reference case 1234abcde

where of course ***@example.com is unfiltered and forwards to the
mailbox(es) of the right people. It also changes periodically because
not only will it be harvested by spammers and targeted, but because
every now and then someone's truly broken system will spew a backlog
of several hundred no-longer-relevant messages at it.

This is a better method because (a) it avoids generating more SMTP
traffic (b) it's much harder to game and (c) it caters the most to
the people who are paying the most attention: those who are monitoring
their mail server logs in real time and (d) it utilizes what we already
have without the need to invent or deploy new technology.

There *are* issues with it: for example, information leakage (do you
really want to tell the world *which* virus scanner you're running?) and
non-standardization of error messages, for starters. But the former
is addressed by some judicious local choices, and the latter with Perl.

---Rsk

[Chris Lewis]

Post by Alessandro Vesely
Chris said their filter is not able to distinguish viruses from
generic malware.

That should be read to mean "not in general", as opposed to "never".

Post by Alessandro Vesely
Having an appropriate error message is not enough. It is also
necessary to deliver that message to the right operator.

Thus becoming a DDOS vector.

Post by Alessandro Vesely
Some large sites have established feedback loops whereby a message is
"bounced" to some postmaster. Apparently, they are mainly meant for
"this is spam" actions. However, the ARF format (quite similar to DSN)
provides fields for reporting bad DKIM signatures. I don't know at
what level such bounces could be generated. It is technically possible
to generate them right after the data transfer, just like for viral
content. If we recognize that viruses are a problem, don't they
deserve using that meta channel as well? This leaves us wondering how
can such a meta channel be established for small and medium sites as
well...

Thus becoming a DDOS vector.

Went through this conversation on another list recently.

It is technically possible (in fact trivial in many cases) to instrument
a MTA to automatically generate and send ARF in real time. Even
assuming that the MTA can figure out the _right_ place to send the ARF,
it becomes a WMD.

Imagine, if you will, everybody did it. Some moderately sized site gets
a reasonably prolific (single) infection, and spews out a few million
viruses. You're expecting the site's MTAs to handle a few million ARFs,
when only one _should_ suffice.

If broadly implemented, it'd cause global meltdown.

God help us all if the site receiving the ARF somehow doesn't recognize
it as ARF, and replies with its own ARFs. Or, if the virus writer
figures out a way to get the ARF generators to send it to the wrong
place - believe me, they'd be trying...

ARF is good stuff. But only insofar as there is limitations on how it's
emitted/deployed.

[Franck Martin]

Well, any way to modify SMTP to still send ARF reports without taking down the whole Internet?

Speaking out loud, a new capability in ESMTP, that would advertise the reception of ARF report, or a special mailbox standard on all MTA to accept ARF reports. We have postmaster, abuse, why not ARF?

Or may be send this ARF report to a DNSBL service? After Authentication, it would place the original sender in a DNSBL stopping other from receiving further infected emails?

Cheers
----- Original Message -----
From: "Chris Lewis" <***@nortel.com>
To: "Anti-Spam Research Group - IRTF" <***@irtf.org>
Sent: Friday, 16 January, 2009 9:17:31 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] Meta channel, not bounces

Post by Alessandro Vesely
Chris said their filter is not able to distinguish viruses from
generic malware.

That should be read to mean "not in general", as opposed to "never".

Post by Alessandro Vesely
Having an appropriate error message is not enough. It is also
necessary to deliver that message to the right operator.

Thus becoming a DDOS vector.

Post by Alessandro Vesely
Some large sites have established feedback loops whereby a message is
"bounced" to some postmaster. Apparently, they are mainly meant for
"this is spam" actions. However, the ARF format (quite similar to DSN)
provides fields for reporting bad DKIM signatures. I don't know at
what level such bounces could be generated. It is technically possible
to generate them right after the data transfer, just like for viral
content. If we recognize that viruses are a problem, don't they
deserve using that meta channel as well? This leaves us wondering how
can such a meta channel be established for small and medium sites as
well...

Thus becoming a DDOS vector.

Went through this conversation on another list recently.

It is technically possible (in fact trivial in many cases) to instrument
a MTA to automatically generate and send ARF in real time. Even
assuming that the MTA can figure out the _right_ place to send the ARF,
it becomes a WMD.

Imagine, if you will, everybody did it. Some moderately sized site gets
a reasonably prolific (single) infection, and spews out a few million
viruses. You're expecting the site's MTAs to handle a few million ARFs,
when only one _should_ suffice.

If broadly implemented, it'd cause global meltdown.

God help us all if the site receiving the ARF somehow doesn't recognize
it as ARF, and replies with its own ARFs. Or, if the virus writer
figures out a way to get the ARF generators to send it to the wrong
place - believe me, they'd be trying...

ARF is good stuff. But only insofar as there is limitations on how it's
emitted/deployed.

[Seth]

Post by Chris Lewis
It is technically possible (in fact trivial in many cases) to
instrument a MTA to automatically generate and send ARF in real
time. Even assuming that the MTA can figure out the _right_ place
to send the ARF, it becomes a WMD.
Imagine, if you will, everybody did it. Some moderately sized site
gets a reasonably prolific (single) infection, and spews out a few
million viruses. You're expecting the site's MTAs to handle a few
million ARFs, when only one _should_ suffice.

If one suffices, then the site immediately closes or blocks the
infected machine, and it doesn't get all that many notices.

Post by Chris Lewis
If broadly implemented, it'd cause global meltdown.

For a value of "global" closer to "sites that don't react to their
infections in a sufficiently timely manner".

Post by Chris Lewis
God help us all if the site receiving the ARF somehow doesn't
recognize it as ARF, and replies with its own ARFs.

It would be better to create a new protocol for the ARF responses
(even if it's just email on a new port, though I'd include some
extensions to allow for "Report of virus emitter on <IP>" "I already
know" "QUIT")

Seth

[SM]

Post by Alessandro Vesely
599 Bounce to postmaster. What would be wrong if it existed? (I
mean, besides how hard it would be to reliably introduce it now.)

There was a discussion about enhanced error codes where having a code
conveying the reason for rejection was proposed. The idea was rejected.

Regards,
-sm

[Chris Lewis]

Post by Alessandro Vesely

Post by Rich Kulawiec

Post by der Mouse
- Malware goes out, addressed to A, (forged) envelope-from B. Sending
channel ends up emitting it from a normal MTA, M.
- A's MX host rejects it at SMTP time.
- M generates and sends a bounce to B.
- B receives bounce with embedded malware. Somehow - perhaps B's MUA
aggressively looks for and executes live content; perhaps B clicks
on the wrong thing; perhaps something else - this ends up with a
malware infestation on B's machine. (Cue xkcd #350.)
If A's MX host had silently swallowed the mail, nothing would have
happened to B - or, at least, not on account of this message.

Ah, gotcha. I agree that silently swallowing the message might have
spared B a possible infection, but I'm reluctant to blame A's MX for
this: it didn't originate, accept or transfer the malware-laden message.

A's MX knows that M lacks effective anti-virus filtering. Hence,
through inaction, it allowed a human being to come to harm. That
obviously breaks the first law.

A's MX didn't generate _any_ virus-laden email. It just 550'd. The
originator did, and M's mailer is complicit by constructing a new email
(the bounce) that contains the virus-laden email.

A knows its filtering isn't perfect and that every rejection is a
potential FP. So, the rejection is the best way to ensure that the
appropriate party (if any) is notified. Blackholing would violate the
first law.

M _should_ know that best practise is now to ensure that the recipient
of the bounce knows enough to know what email bounced, and should
truncate the email to that minimum. Eg: original recipient, sender (the
recipient of the bounce), date, subject, and perhaps a few other
snippets. A very large proportion of MTAs now do that by default.

[Ian Eiloart]

Post by Gordon Peterson
It is important to remember that many individuals and many small
companies use personal or "vanity" domain names.
These domain names appear in their "From:" and "Reply-to:" addresses on
their outgoing e-mail messages, but SHOULD NOT be presumed to indicate
where the e-mail in question is arriving from.
For example, I might be travelling somewhere and sending e-mail messages
from an inhabitual location: a cruise ship Internet cafe, an Internet
access kiosk above a post office location in Beijing, a coffee bar, or
other such public-access location. In such a situation, I will typically
have NO control over what outgoing mail server is sending my e-mail
message, and since my being there is temporary I obviously don't want to
put a return address on my mail that would be connected with the location
where I physically am at the time.

You need to use your email service provider's Message Submission Service.
That should be available to authenticating clients on port 587.
Alternatively, use their webmail. If your Internet cafe is willing to relay
your email, then they're willing to relay spam, too, and should therefore
be blacklisted.

[Ian Eiloart]

While we're talking about the issue of antispam nuisances, another is the
situation where a spam blocker inadvertently serves as an "IP address
laundering agent" when they bounce back a message detected as containing
spam, or a virus. It is particularly stupid to bounce back a message
because it is detected to contain a virus WHICH IS KNOWN TO FALSIFY OR
COUNTERFEIT THE RETURN ADDRESS!

Actually, it occurred to me that we have mechanisms whereby domain owners
can prevent abuse of their domains (SPF and DKIM). Therefore, we should
feel free to bounce email to any domain owner the doesn't deploy SPF and
DKIM - in order to encourage them to do so. Of course, bouncing email is
also OK when you find an SPF or DKIM match.

[Rich Kulawiec]

Post by Ian Eiloart
can prevent abuse of their domains (SPF and DKIM). Therefore, we should
feel free to bounce email to any domain owner the doesn't deploy SPF and
DKIM - in order to encourage them to do so. Of course, bouncing email is
also OK when you find an SPF or DKIM match.

This is utter nonsense, of course. It is *never* acceptable to deliberately
generate or redirect spam. I'm appalled to find anyone recommending such
a clearly abusive course of action.

---Rsk

[Ian Eiloart]

Post by Rich Kulawiec

Post by Ian Eiloart
can prevent abuse of their domains (SPF and DKIM). Therefore, we should
feel free to bounce email to any domain owner the doesn't deploy SPF and
DKIM - in order to encourage them to do so. Of course, bouncing email
is also OK when you find an SPF or DKIM match.

This is utter nonsense, of course. It is *never* acceptable to
deliberately generate or redirect spam. I'm appalled to find anyone
recommending such a clearly abusive course of action.

Well, I should clarify that I'm not recommending it, or doing it. However,
I do think that it's time that domain owners started taking responsibility
for the use of their domains. The technology exists, and it would have the
direct benefit to them that collateral spam back to their domains was
reduced.

In a world where you *can* prevent abuse of your domain, what's wrong with
bouncing email?

[der Mouse]

However, I do think that it's time that domain owners started taking
responsibility for the use of their domains.

If I, say, were to forge mail from your domain to, say, some ibm.com
user, would you take responsibility for it? That's what you seem to me
to be saying.

In a world where you *can* prevent abuse of your domain, what's wrong
with bouncing email?

Depends on what that prevention requires. ("Sure you can prevent abuse
of your domain. You just need to enter into a bilateral agreement with
every receiving mailhost on the net.")

But that's not the world we have, so it's a moot question.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

[Seth]

Post by Ian Eiloart
Actually, it occurred to me that we have mechanisms whereby domain
owners can prevent abuse of their domains (SPF and DKIM). Therefore,
we should feel free to bounce email to any domain owner the doesn't
deploy SPF and DKIM - in order to encourage them to do so.

And the recipients of the bounces should feel free to report your
bounces as spam. And they should also feel free to cut your Internet
connection to block further spam from your site, to encourage you to
stop spamming.

Seth

[Ian Eiloart]

Post by Seth

Post by Ian Eiloart
Actually, it occurred to me that we have mechanisms whereby domain
owners can prevent abuse of their domains (SPF and DKIM). Therefore,
we should feel free to bounce email to any domain owner the doesn't
deploy SPF and DKIM - in order to encourage them to do so.

And the recipients of the bounces should feel free to report your
bounces as spam. And they should also feel free to cut your Internet
connection to block further spam from your site, to encourage you to
stop spamming.

Of course they could, but wouldn't they be better advised to take
responsibility for their domains, thus preventing me from having to deal
with spam sent "from" their domain?

[Seth]

Post by Ian Eiloart
Of course they could, but wouldn't they be better advised to take
responsibility for their domains,

They do: by not emitting spam. That's what they're responsible to do.

They aren't responsible to sign up for Clever-Idea-Of-The-Week and
implement whatever makes your life easier.

Post by Ian Eiloart
thus preventing me from having to deal
with spam sent "from" their domain?

It's not my responsibility to mediate the transaction between two
third parties, even if one of them lies about being me.

Seth

[Chris Lewis]

Post by Ian Eiloart
Of course they could, but wouldn't they be better advised to take
responsibility for their domains, thus preventing me from having to deal
with spam sent "from" their domain?

Please describe a _practical_ way to deal with Russian or Ukrainian
virus/botspammers forging our domains in their spewage.

SPF? DKIM? We do SPF. We may end up doing DKIM some day. But it
won't make a significant difference to what is being _sent_.

This is vaguely reminiscent of the complainer who, when I pointed out
that our domain in his complaint was forged, replied with "well, you
must be doing something bad if they're forging you like that".

Splorf. "Bad" for whom? ;-)

[Graeme Fowler]

Post by Seth
And the recipients of the bounces should feel free to report your
bounces as spam. And they should also feel free to cut your Internet
connection to block further spam from your site, to encourage you to
stop spamming.

I have a sneaking suspicion that there may be a terminology problem
here: Ian more than likely means "reject" when he says "bounce". The
thread becomes rather less argumentative if you do that replacement.

Of course, I may be putting words into Ian's mouth which he *didn't*
mean, but before anyone else replies I would like to see Ian clarify
whether he means "bounce" or "reject".

Knowing him at least a little in my professional capacity I would be
*very* surprised if he did intend to write "bounce", unless he was being
deliberately controversial...

Graeme

[Franck Martin]

I'm not sure that http://en.wikipedia.org/wiki/Sender_Signing_Policy has a correct definition?

spamassassin, DNSBL, DCC are well known, so we know how they behave with different emails, what we don't know is what the google, yahoo, microsoft and others are doing to classify their emails (this is the part about security by obcurity).

----- Original Message -----
From: "Robert Barclay" <***@gmail.com>
To: "Anti-Spam Research Group - IRTF" <***@irtf.org>
Sent: Tuesday, 13 January, 2009 10:37:52 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated




On Mon, Jan 12, 2009 at 1:51 PM, Franck Martin < ***@avonsys.com > wrote:


I'm curious when you say ADSP is always keyed of the real live From address? You talk about the From: and not the Mail From: (Return-path)?
Yes, the A is for Author. ADSP is built on top of DKIM and allows domain owners to specify that they sign some or all mail using a specific domain in the From: address




as a side note, all this SSP/ADSP processing looks like a blackbox (or black magic) to me. There is no recommended practices and no one explain what they do to filter mail. like in the statement "AOL will use DKIM to do build reputation based on domain", what does it mean?

It means they are going to start establishing reputation for DKIM domains based on the DKIM signed mail passing through their systems. This isn't any more of a black box than their existing reputation systems.
As for recommended practices I'm not sure there's enough operational experience is most situations to have anything useful to recommend yet. I would be pleased to be wrong here but suspect that we may be starting to get there for some uses of DKIM and are a long way away from that with ADSP.





We know well about spamassassin, DNSBL, DCC but this is about it. I thought security by obscurity was a bad idea? ;)

Not sure this qualifies for security by obscurity. It's pretty straightforward how all these technologies work. How people make use of the data these technologies provide, that only seems obscure because people are still figuring it out themselves. Compare this to how people use IP addresses there's a pretty wide variety there and a lot of those uses would qualify as "obscured" from outside users too.

[John Levine]

Post by Franck Martin
I'm not sure that http://en.wikipedia.org/wiki/Sender_Signing_Policy has
a correct definition?

If you're wondering what SSP/ADSP does, how about reading the draft
that actually defines it:

http://tools.ietf.org/html/draft-ietf-dkim-ssp-08

[Dave CROCKER]

Post by John Levine
If you're wondering what SSP/ADSP does, how about reading the draft

john,

you simply have *no* sense of adventure...

RTFS (spec) is such an unimaginative approach to life.

d/

[Franck Martin]

John,

It sounds like "read the code, luke!" ;)

I like to read wikipedia summaries of RFC, it gives me the essential points, but here I think I'm more lost than by reading the draft, I went to read the draft, but I think it would be helpful if someone could fix the wikipedia entry, so nayone can understand what is all about.


----- Original Message -----
From: "Dave CROCKER" <***@dcrocker.net>
To: "Anti-Spam Research Group - IRTF" <***@irtf.org>
Sent: Tuesday, 13 January, 2009 12:14:36 PM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated

Post by John Levine
If you're wondering what SSP/ADSP does, how about reading the draft

john,

you simply have *no* sense of adventure...

RTFS (spec) is such an unimaginative approach to life.

d/

[Robert Barclay]

Post by Franck Martin
I'm not sure that http://en.wikipedia.org/wiki/Sender_Signing_Policy has a
correct definition?

I think what you're referring to is this statement "Instead of the
*From*header, ADSP can also be used for publishing that all mails of
the
*MAIL FROM*, *Sender*, *Resent-Sender* and *Resent-From* headers have a
corresponding DKIM <http://en.wikipedia.org/wiki/DKIM> signature." which as
far as I know is not correct. John, Dave, Steve can any of you guys verify
this? I admit some of the debates managed to throttle me into zombie-ism so
I may have missed something major somewhere.

Post by Franck Martin
spamassassin, DNSBL, DCC are well known, so we know how they behave with
different emails, what we don't know is what the google, yahoo, microsoft
and others are doing to classify their emails (this is the part about
security by obcurity).

First I think you're talking about a different issue than DKIM/ADSP here.
Your complaint appears to be that you don't know generally what the software
these guys use to evaluate email does or what they mean when they use the
term reputation. This isn't a new issue. It's just sort of the state of the
world and I would say it's not limited to these guys. In general unless
someone on the receiving end has told you specifically how they evaluate
emails then really all you can tell is (sometimes) whether the email got
there or not.

Second I think you're confusing knowledge of what data a piece of technology
provides you with knowledge of how people use that software. Knowing what
spamassassin does generally still doesn't give you anything better than a
guess at what that might mean to the systems of spamassassin users. In the
case of DKIM you at least know that two people running compliant software
will come to exactly the same decision on whether a piece of mail passes.
You just don't know what either of those people will do with that
information.
Since that's exactly the case now with every other piece of information
people extract from an email you're certainly not any worse off.

Robert

Post by Franck Martin
----- Original Message -----
Sent: Tuesday, 13 January, 2009 10:37:52 AM (GMT+1200) Auto-Detected
Subject: Re: [Asrg] where the message originated

Post by Franck Martin
I'm curious when you say ADSP is always keyed of the real live From
address? You talk about the From: and not the Mail From: (Return-path)?

Yes, the A is for Author. ADSP is built on top of DKIM and allows domain
owners to specify that they sign some or all mail using a specific domain in
the From: address

Post by Franck Martin
as a side note, all this SSP/ADSP processing looks like a blackbox (or
black magic) to me. There is no recommended practices and no one explain
what they do to filter mail. like in the statement "AOL will use DKIM to do
build reputation based on domain", what does it mean?

It means they are going to start establishing reputation for DKIM domains
based on the DKIM signed mail passing through their systems. This isn't any
more of a black box than their existing reputation systems.
As for recommended practices I'm not sure there's enough operational
experience is most situations to have anything useful to recommend yet. I
would be pleased to be wrong here but suspect that we may be starting to get
there for some uses of DKIM and are a long way away from that with ADSP.

Post by Franck Martin
We know well about spamassassin, DNSBL, DCC but this is about it. I
thought security by obscurity was a bad idea? ;)

Not sure this qualifies for security by obscurity. It's pretty
straightforward how all these technologies work. How people make use of the
data these technologies provide, that only seems obscure because people are
still figuring it out themselves. Compare this to how people use IP
addresses there's a pretty wide variety there and a lot of those uses would
qualify as "obscured" from outside users too.
_______________________________________________
Asrg mailing list
http://www.irtf.org/mailman/listinfo/asrg

[John Levine]

Post by Robert Barclay
I think what you're referring to is this statement "Instead of the
*From*header, ADSP can also be used for publishing that all mails of
the >*MAIL FROM*, *Sender*, *Resent-Sender* and *Resent-From* headers have a
corresponding DKIM <http://en.wikipedia.org/wiki/DKIM> signature." which as
far as I know is not correct.

Wow, is that wrong. I'll see if I can fix it.

R's,
John

[Robert Barclay]

Thanks for the confirmation. I was pretty sure that was the case but figured
someone with more expertise (like say one of the authors of the draft) would
let me know otherwise.

Post by John Levine

Post by Robert Barclay
I think what you're referring to is this statement "Instead of the
*From*header, ADSP can also be used for publishing that all mails of
the >*MAIL FROM*, *Sender*, *Resent-Sender* and *Resent-From* headers have

a

Post by Robert Barclay
corresponding DKIM <http://en.wikipedia.org/wiki/DKIM> signature." which

as

Post by Robert Barclay
far as I know is not correct.

Wow, is that wrong. I'll see if I can fix it.
R's,
John
_______________________________________________
Asrg mailing list
http://www.irtf.org/mailman/listinfo/asrg

[Dave CROCKER]

i modified the text this morning.

d/

Post by John Levine

Post by Robert Barclay
I think what you're referring to is this statement "Instead of the
*From*header, ADSP can also be used for publishing that all mails of
the >*MAIL FROM*, *Sender*, *Resent-Sender* and *Resent-From* headers have a
corresponding DKIM <http://en.wikipedia.org/wiki/DKIM> signature." which as
far as I know is not correct.

Wow, is that wrong. I'll see if I can fix it.
R's,
John
_______________________________________________
Asrg mailing list
http://www.irtf.org/mailman/listinfo/asrg

[John Levine]

Post by John Levine

Post by Robert Barclay
I think what you're referring to is this statement "Instead of the
*From*header, ADSP can also be used for publishing that all mails of
the >*MAIL FROM*, *Sender*, *Resent-Sender* and *Resent-From* headers have a
corresponding DKIM <http://en.wikipedia.org/wiki/DKIM> signature." which as
far as I know is not correct.

Wow, is that wrong. I'll see if I can fix it.

Looks like Dave already did. Thanks.

R's,
John

[Dave CROCKER]

I view the changes as a quick hack.

The section needs do its tutorial business more cleanly, explaining the purpose
of ADSP and its means of accomplishing it. I suggest that all references to
other work be relegated to a separate section rather than, for example, showing
up in the Introduction.

d/

Post by John Levine

Post by John Levine

Post by Robert Barclay
I think what you're referring to is this statement "Instead of the
*From*header, ADSP can also be used for publishing that all mails of
the >*MAIL FROM*, *Sender*, *Resent-Sender* and *Resent-From* headers have a
corresponding DKIM <http://en.wikipedia.org/wiki/DKIM> signature." which as
far as I know is not correct.

Wow, is that wrong. I'll see if I can fix it.

Looks like Dave already did. Thanks.
R's,
John
_______________________________________________
Asrg mailing list
http://www.irtf.org/mailman/listinfo/asrg

[Gordon Peterson]

Post by Rich Kulawiec
Ah, gotcha. I agree that silently swallowing the message might have

spared B a possible infection, but I'm reluctant to blame A's MX for
this: it didn't originate, accept or transfer the malware-laden message.
I think responsibility lies with M, and/or with whatever system passed
the message to M (presuming the message didn't originated on M).

Well, regardless of who is pointing the finger at who, the fact remains
that:

1) an infected E-mail is being passed on to someone who quite likely
had NOTHING to do with sending it, nor did they probably have any
control over the system(s) that did;

2) the extra bandwidth was wasted by sending mail to someone who
didn't want it, and who can't do anything about it;

3) the resulting infected E-mail is now coming from a computer whose
IP address is likely in the (actual final) recipient's "trusted" zone,
whereas the original actual origin system might well not be.

Post by Rich Kulawiec
While this is a less-than-desirable outcome, I think it may be the best

we can do -- we can't block something before we see it, we can only do
our best to reject outright spam/malware/junk traffic at the earliest
available opportunity. (One of the ways I've expressed this elsewhere
is that the best place to stop spam is as near its source as possible.
The farther it gets from the origin, the trickier detecting it gets,
and the greater the possible consequences.)

Actually, I believe that that philosophy is incorrect.

First of all, ultimately the ONLY authority which TRULY determines FOR A
FACT whether a given piece of e-mail is unwanted or not is the final
recipient.

Other people enroute might conclude that something PROBABLY is spam, but
they cannot be sure.

I understand the desirability of stopping spam and viruses as close to
the sender as possible, WHERE THAT IS POSSIBLE, but I suggest that it
often is not.

ULTIMATELY, the recipient of the message (i.e. the addressee, not
necessarily the place where it ultimately gets bounced to) has a KEY
piece of information that MTAs enroute do not have: the knowledge of
the (prior?) trust relationship that exists between the addressee and
the purported sender.

And, let me point out, this even varies WITHIN a single addressee: I
have more than one E-mail address, and use them for different purposes.
An E-mail message arriving at the "wrong" address might indicate that
it is unwanted, EVEN IF THE SAME SENDER MIGHT BE EXPECTED TO MAIL TO ME
AT A DIFFERENT ONE OF MY ADDRESSES.

This is part of why I still believe that a finely-grained "permissions
list", based on the recipient/orignator e-nmail address pair, AND having
access by then to the WHOLE message, can do a SIGNIFICANTLY better job
of accurately performing the spam/nospam triage than ANY more
generalized SMTP-level MTA could ever hope to do enroute.

As I have pointed out before, if I get an e-mail containing an
executable (a control panel applet, a hard .EXE file, even a piece of
Java script or Active X enclosure or whatever) from someone who (a) I do
not know, or (b) who I might know but who I do not expect to send me
something like THAT, then it's perfectly reasonable to treat that e-mail
as at least HIGHLY suspect.

Now, there ARE people I correspond with who I might EXPECT to send me
stuff like that, or at least SOME of those sorts of things; obviously
those should come through to me fine. But there may be OTHER things I
would expect to see in mail from those trusted and familiar
correspondents; for example, their familiar sig that they put on the
e-mails they send out to me, or the masthead I normally see on their
e-newsletter, or whatever. Even maybe the familiar way they address me
in the accompanying message.

Note that this is not unlike the way most of us actually handle "spam
triage" in our inboxes now: we look to see mail coming from unfamiliar
senders, or unfamiliar subjects, or for that matter common spam-type
subject lines. The key thing is that mail coming from an address we
recognize might encourage us to look more closely at mail which we might
not give a second thought to deleting if it came from someone unfamiliar.

I personally think that mail containing attachments (of any type) or
HTML (scripting, ActiveX, misrepresentable links, etc) coming from
people I don't know and trust are primary examples of mail that stands a
strong probability of being unwanted (or at the very least, untrusted).

Now, I would agree that e-mail containing certain types of things can
probably be axed at a very early point in the delivery process (an
example would be mail containing a recognized virus content, especially
a virus known to falsify return addresses...!)

BUT it is also VERY frustrating when, for example, I find that an
absolutely legitimate e-mail message I have sent out gets bounced back
to me by someone enroute, containing something like:

[quote]

Diagnostic-Code: smtp; 554 The message was rejected because it contains
prohibited virus or spam content (in reply to end of DATA command)

[end quote]

...and particularly when the message contains neither virus nor spam,
and where I have not the slightest idea of who I would need to try to
contact (and where, or how) to solve the non-delivery problem.

And I presume that the addressee I was sending the mail to has similarly
no idea of who they would need to complain to about our mail being
intercepted and waylaid enroute.

Given the choice, I like the simplicity and correctability of people
enroute delivering the mail as requested, and having the recipient's
mail client (given the additional knowledge that only it has!) doing the
BETTER possible job of deciding how to perform the triage.

[Chris Lewis]

Post by Gordon Peterson
BUT it is also VERY frustrating when, for example, I find that an
absolutely legitimate e-mail message I have sent out gets bounced back
[quote]
Diagnostic-Code: smtp; 554 The message was rejected because it contains
prohibited virus or spam content (in reply to end of DATA command)
[end quote]

Then most of your concern should evaporate if the message said this:

This is the mail delivery agent at <redacted>
I was not able to deliver your message to the following addresses.

<***@nortel.com>:
<***@nortel.com>:
47.248.0.48 failed after I sent the message.
Remote host said: 552 Blocked by filters. If in error, please forward
the Session ID: ***@ecarhea1) to <redacted>@nortel.com
ONLY (#5.7.1)

Instead. [I redacted the userids because that's a _real_ reject message.]

Post by Gordon Peterson
...and particularly when the message contains neither virus nor spam,
and where I have not the slightest idea of who I would need to try to
contact (and where, or how) to solve the non-delivery problem.

See above.

Post by Gordon Peterson
And I presume that the addressee I was sending the mail to has similarly
no idea of who they would need to complain to about our mail being
intercepted and waylaid enroute.

In our case, you'd presume incorrectly.

Post by Gordon Peterson
Given the choice, I like the simplicity and correctability of people
enroute delivering the mail as requested, and having the recipient's
mail client (given the additional knowledge that only it has!) doing the
BETTER possible job of deciding how to perform the triage.

It's been our experience that the recipients all too often lack the
experience and technical knowledge to do the triage in the most
dangerous emails. By far the majority of our users want _us_ to stop
the crap.

[der Mouse]

Post by Gordon Peterson
Well, regardless of who is pointing the finger at who, the fact
1) an infected E-mail is being passed on to someone who quite likely
had NOTHING to do with sending it, nor did they probably have any
control over the system(s) that did;

True in the scenario outlined. But there is no way for the host
issuing the SMTP-level reject to know, in general, that that is the
case; whether a bounce to anyone is generated is up to the SMTP
client's software. (Direct-to-MX spamware, for example, generally does
not generate bounces in reaction to rejections.)

Furthermore, even the best malware detection FPs at least occasionally.
If my mail to my friend produces a FP, the _last_ thing I want is for
it to silently vanish. (Furthermore, the presence of malware does not
necessarily mean the mail is unwanted or shouldn't be delivered; I have
no trouble imagining researchers mailing samples to one another. Yes,
they _can_ encrypt them or some such, but I see no a priori reason they
should have to.)

Post by Gordon Peterson
2) [...]
3) [...]
First of all, ultimately the ONLY authority which TRULY determines
FOR A FACT whether a given piece of e-mail is unwanted or not is the
final recipient.

If there is one. A lot of spam, and probably a nontrivial amount of
malware-bearing email, has no existent addresses anywhere in the
envelope (often, not in the headers either). Who is the "final
recipient" of such a message?

Post by Gordon Peterson
Note that this is not unlike the way most of us actually handle "spam
triage" in our inboxes now: we look to see mail coming from
unfamiliar senders, or unfamiliar subjects, or for that matter common
spam-type subject lines.

Who's this "we"? That's certainly not how I triage my email; the first
thing I look at for most of the mail that reaches my mailbox is the
beginning of the body. At least a moderate fraction of my mail I never
read the Subject: or From: of at all.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

[Rich Kulawiec]

I'm only going to tackle two of the many points contained therein.

Post by Gordon Peterson
First of all, ultimately the ONLY authority which TRULY determines FOR A
FACT whether a given piece of e-mail is unwanted or not is the final
recipient.

No, not really. There are multiple authorities which may determine whether
a given message is accepted or not. For example: it's possible that network
engineers have implemented the Spamhaus DROP list in the border routers of
a site, that systems engineers have used the NJBAL DNSBL in a mail
system's configuration, and that an end user has used a local filter
such as Spambouncer (which does not bounce, by the way). Thus, the
logical AND of all three permissions is required before a message will
reach a user.

This is as it should be: network engineers are probably best positioned
to use effective mechanisms like the DROP list -- doubly so since best
usage of it encompasses more than just SMTP. Morever, network engineers,
charged with the responsibility for implementing a comprehensive network
defense policy, may -- correctly in my opinion -- conclude that *regardless*
of what any end user wants, permitting traffic to/from networks on the DROP
list is a very bad idea. Similar considerations apply to systems admins
and end users.

End users have been known to argue that this is a Bad Thing, because
they cannot get mail they want. I counter-argue that deliberately
allowing them to receive mail from J. Random Well-Known spammer is
every bit as dangerous to them as deliberately allowing them to
receive the Outlook virus-of-the-day, or allowing known-malicious
packets to reach the network interface on their system. End users
counter-counter-argue that this is fascist and draconian and etc.,
but usually decline the "opportunity" to be moved outside the
effective network perimeter and receive ALL mail, sans any spam
or malware filtering, and to be exposed to ALL network traffic.

And so on, endlessly goes the policy debate. But my point is that
in most environments, there are multiple "layers", if you will, of
accept/reject decision-making going on, and while some of those
layers may be based on criteria like "wanted", others may not be.
None of this is a problem to the rest of us -- nobody is obligated to
accept anyone's SMTP traffic -- as long as the accept/reject decision
is clearly communicated to the outside world so that can all tell what
the heck is going on.

Post by Gordon Peterson
...and particularly when the message contains neither virus nor spam,
and where I have not the slightest idea of who I would need to try to
contact (and where, or how) to solve the non-delivery problem.

It's been a best practice for a long time to arrange for every rejection
message to contain a human-readable string that gives some clue as to what
the problem was AND provides a means of resolving the problem, should
there be a mistake. (This is in addition to the mandatory "postmaster"
address and the not-quite-so-mandatory-but-still-a-darn-good-idea
"abuse" address.) Part of the problem here lies with broken mail
systems that either (a) mangle these strings or (b) substitute their
own, neatly undercutting the process. Another part lies with end users,
who often cannot or will not take the time to read the rejection message,
or won't follow the instructions in it, or who presume that the
most likely possibility is not, perhaps, a history of spam from the
same mail system, or an errant filter, or a typo, but a personal
vendetta directed solely at them for no particular reason. [1]
And of course, most of the problem lies with mail systems that
don't even try to do this, or whose operators blithely presume
that their FP rate is 0%. [2]

Incidentally, consider how much worse this problem would get if
-- instead of returning a possibly-cryptic error message lacking
a recourse -- the recipient system simply discarded your message.

---Rsk

[1] See the Truthout & AOL debacle for an example, and note
the following for guidance in such situations:

Vir: "Londo, they could have killed me!"
Londo: "Nonsense, you are not important enough to kill."

[2] Maybe it's just me, but it seems that those using appliances
are particularly prone to this. "But we paid a lot of money for it,
it can't be wrong", someone actually wrote to me.

[Chris Lewis]

Post by Gordon Peterson
First of all, ultimately the ONLY authority which TRULY determines FOR A
FACT whether a given piece of e-mail is unwanted or not is the final
recipient.

You're being awfully absolute in that statement.

RSK has outlined entities (other than the final recipient) who do. That
of course you'd counter with "but they shouldn't".

So, I'm going to take a different tack - outlining why many should. And
in some cases legally must.

As a corporate, our email infrastructure is for our business purposes.
While we do permit reasonable personal use, exposing our infrastructure,
IP or employees to risk of malicious content is simply not on. It's our
infrastructure, for us, we need to protect it.

For any medium-large infrastructure, they can't _afford_ to let viruses
in. Simply from the perspective of self-preservation and integrity of
their own networks.

Secondly, at least in a business context, there are laws about workplace
environment that _require_ us to filter certain things.

Post by Gordon Peterson
Other people enroute might conclude that something PROBABLY is spam, but
they cannot be sure.

There are many cases where "people enroute" can be absolutely certain
it's spam. Apropos my other posting: anything saying "HELO
list.mediresource.com" (except possibly from 209.82.15.228, it's other
SPF-permitted IP helos as tempmail.mediresource.com) is spam. Their SPF
says so, and all of the emails contain bogus DKIM signatures. We _know_
it's a bot.

There are many times you can tell, absolutely with no FPs, that an email
is sent by a BOT.

Certainly, if you're a, say, spam or virus researcher, you might want to
get your email flow raw. Fine. Just don't expect a commodity ISP to
accommodate you at commodity prices. They simply can't afford the risk
to their infrastructure.

[der Mouse]

Post by Chris Lewis

Post by Gordon Peterson
First of all, ultimately the ONLY authority which TRULY determines
FOR A FACT whether a given piece of e-mail is unwanted or not is the
final recipient.

You're being awfully absolute in that statement.

And in the sense of "unwanted" that I think its author meant, there's
nothing wrong with that. I think that "unwanted" is something like
"the human behind the destination address would prefer to receive the
mail". You - and others - seem to be using values of "unwanted" that
include "I don't care who would like to receive it, we don't want to
carry it".

Post by Chris Lewis
Apropos my other posting: anything saying "HELO
list.mediresource.com" (except possibly from 209.82.15.228, it's
other SPF-permitted IP helos as tempmail.mediresource.com) is spam.

Um, no, not if it's not bulk. Want me to telnet to your MX host and
generate an example manually? :-)

Post by Chris Lewis
Certainly, if you're a, say, spam or virus researcher, you might want
to get your email flow raw. Fine. Just don't expect a commodity ISP
to accommodate you at commodity prices. They simply can't afford the
risk to their infrastructure.

Oh, nonsense. If that constitutes any risk to their infrastructure,
they *urgently* need to fix the bugs in question anyway! Their
infrastructure has no need to do anything with executable - or any
other - content in the mail, except storing it for delivery to the
customer on request. Unless they're trying to do content-based
filtering, of course, but if their content filtering setup is
vulnerable to infection from malware-bearing email, it is
catastrophically broken already; it's _job_ is to deal with such mail.

The only way I can see any risk to their infrastructure is if their
staff use their own services for mail, read mail with MUAs and OSes
vulnerable to infestation by email-borne malware, their infrastructure
hosts run similarly vulnerable OSes, and their network design permits
their staff mail-reading hosts to infect their infrastructure. While
the first of these is likely and the second is plausible, the rest are
nobody's fault but their own. Any risk to their infrastructure is
created by their boneheaded design and actions far more than it is by
lack of filtering on independent customer mail streams.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

[Chris Lewis]

Post by der Mouse

Post by Chris Lewis
Apropos my other posting: anything saying "HELO
list.mediresource.com" (except possibly from 209.82.15.228, it's
other SPF-permitted IP helos as tempmail.mediresource.com) is spam.

Um, no, not if it's not bulk. Want me to telnet to your MX host and
generate an example manually? :-)

Except for mice in the works ;-)

It may not be bulk, but forgery in the face of the domain owner
asserting otherwise is still undesirable.

Post by der Mouse

Post by Chris Lewis
Certainly, if you're a, say, spam or virus researcher, you might want
to get your email flow raw. Fine. Just don't expect a commodity ISP
to accommodate you at commodity prices. They simply can't afford the
risk to their infrastructure.

Oh, nonsense. If that constitutes any risk to their infrastructure,
they *urgently* need to fix the bugs in question anyway!

Oh, nonsense on your nonsense! ;-)

Perhaps I should have said "network" instead of "infrastructure". But
the fact remains that an infection on one recipient's machine has
consequences far beyond _that_ machine alone.

The components of an infrastructure do not need to be infected to be
damaged by infections. Surely you remember slammer? A handful of
infected desktops took out out entire networks without infecting
anything further. Simply by bandwidth exhaustion.

Then of course there's the potential reputation loss if something
manages to spam outwards through your MTAs.

Nor does choice of O/S barricade you more than quantitatively. As
secure as *BSD (for example, for you Seth ;-), it's not immune to going
awry, especially in the face of naive users poking at things they
shouldn't be.

[Seth]

Post by Chris Lewis

Post by Gordon Peterson
First of all, ultimately the ONLY authority which TRULY determines FOR A
FACT whether a given piece of e-mail is unwanted or not is the final
recipient.

You're being awfully absolute in that statement.

Wanting is something that anybody can do.

The only authority who can truly determine if the final recipient
wants the mail is the final recipient. And that opinion is subjet to
change over time, even for the same piece of mail. (User gets an ad
to take a friend to a particular movie for free, does so, and the
movie is so awful he wishes he hadn't received the ad.)

The only authority who can determine whether I want the final
recipient to receive the mail is me.

The only authority who can determine whether the final recipient's
employer wants the final recipient to recieve the mail is the final
recipient's employer.

Etc.

Post by Chris Lewis
Certainly, if you're a, say, spam or virus researcher, you might
want to get your email flow raw. Fine. Just don't expect a
commodity ISP to accommodate you at commodity prices. They simply
can't afford the risk to their infrastructure.

Some will; what's the risk to a BSD infrastructure of letting
customers receive Windoze viruses? (Sure, they have to be careful
about outgoing mail, but that's true anyway because there are plenty
of non-email ways for lusers to become infected.)

Seth

[Gordon Peterson]

Post by Rich Kulawiec
But none of this is necessarily true.

IF the mail is bounced because it contains a virus or worm, and if that
virus or worm is KNOWN to falsify return addresses, then I claim that
there is little or no value in bouncing it... because it is not clear
that ANYONE working backwards is going to be able to figure out reliably
who (if anybody!) should be notified of the problem.

It doesn't really much matter which antivirus software identified the
message as malicious (although it would be nice to have an audit trail
somewhere such that a misbehaving piece of related software can be found
and fixed).

[snip]

Post by Rich Kulawiec
- A human being won't necessarily be harmed by this. For starters,

there's no way to know that the message will be delivered. If the
putative sender is elsewhere, then the mail system there might
detect and reject the message. The putative sender's mail client
may be using an anti-virus product that detects it. Or the putative
sender may never "open" the message (which seems to effectively
defuse most, but not all, of these problems).

That's a bogus argument. You're basically saying that "someone else
will probably pick up on this" and using that as a (lame) excuse to lob
a grenade their way.

[Rich Kulawiec]

Post by Gordon Peterson
That's a bogus argument. You're basically saying that "someone else
will probably pick up on this" and using that as a (lame) excuse to lob
a grenade their way.

Perhaps you should review the SMTP protocol: sending a 5XX response
*refuses* delivery of a message. It does not transmit (or retransmit)
a message. It is difficult to see how one can be accused of "lob[bing]
a grenade" when one has never taken possession of it.

---Rsk

[David Wilson]

Post by Rich Kulawiec

Post by Gordon Peterson
That's a bogus argument. You're basically saying that "someone else
will probably pick up on this" and using that as a (lame) excuse to lob
a grenade their way.

Perhaps you should review the SMTP protocol: sending a 5XX response
*refuses* delivery of a message. It does not transmit (or retransmit)
a message. It is difficult to see how one can be accused of "lob[bing]
a grenade" when one has never taken possession of it.

Because of the normal action of an MTA when it receives such an 5XX
response, i.e. it sends a non-delivery message, normally containing the
message, to the return path address. If that return path address is
forged, then the infection is bounced elsewhere. That is the grenade.

[Rich Kulawiec]

Post by David Wilson

Post by Rich Kulawiec
Perhaps you should review the SMTP protocol: sending a 5XX response
*refuses* delivery of a message. It does not transmit (or retransmit)
a message. It is difficult to see how one can be accused of "lob[bing]
a grenade" when one has never taken possession of it.

Because of the normal action of an MTA when it receives such an 5XX
response, i.e. it sends a non-delivery message, normally containing the
message, to the return path address. If that return path address is
forged, then the infection is bounced elsewhere. That is the grenade.

First, even if this is the case -- and it's rather unlikely that it is,
as I'll get to in a minute -- it is obviously disengenuous to
accuse the refusing system of being responsible for this situation.

And second, have you (rhetorical you) been paying attention to the
origination points of the vast majority of attempted malware deliveries
over the past decade? They're coming from places like this (seen within
the last hour here):

11.red-81-43-100.staticip.rima-tde.net [81.43.100.11]
12-216-204-70.client.mchsi.com [12.216.204.70]
122-82-124-91.pool.ukrtel.net [91.124.82.122]
1809ds4-fb.0.fullrate.dk [90.184.161.72]
78-0-202-32.adsl.net.t-com.hr [78.0.202.32]
89-180-255-125.net.novis.pt [89.180.255.125]
cable-87-116-168-96.dynamic.sbb.rs [87.116.168.96]
cpe-69-133-71-112.columbus.res.rr.com [69.133.71.112]
ip-90-187-210-98.web.vodafone.de [90.187.210.98]

Do you think any of those are actual mail servers? Do you think that
any of them actually are running an SMTP engine which does something
with rejected traffic? Do you think that any of them are running an SMTP
engine which even *notices* rejected traffic?

Malware authors have long since [mostly] outgrown the need to use existing
mail systems for distribution. Why should they limit themselves to a
small number of systems (~100K if the Leisi Conjecture is correct) when
they can use hundreds of millions? And why should they bother coding
elaborate, fully-functional, standards-compliant SMTP engines when
they don't care and have no reason to care about most of that functionality?

Third, you cannot know the state of a system which presents messages
carrying malware. It might be okay, except that its particular anti-virus
setup did not detect this particular virus at this particular time.
Or it might be fully-compromised, 0wned by the enemy, and repurposed
as a full-time virus distribution engine. Or anything in between, or
something quite different. Because you CANNOT know its state, you CANNOT
predict what it will do in response to any SMTP response code you send it.
It *might* generate an NDR. It *might* ignore it. It *might* try sending
the same message again. It *might* crash instantly. It *might* do all
kinds of other things. [1]

Because you can't know this, it's pointless to speculate, and even more
pointless to arbitrarily choose one possibility among many and then try
to address that one as if it were a certainty. It's clear that the
best you can do is to refuse the traffic, at which point you've done
your part not just for your own operation, but for the rest of the 'net
as well. (And optionally, depending on circumstances, refuse *all*
traffic from it, as I do from the hosts above, since it's clear that
100% of everything they will ever present will be spam and malware.)

Fourth, and this veers a bit off-topic, but do realize that not everyone
uses the markedly inferior operating systems which are routinely compromised
via malware authored by mere children. I consider any operating system
which requires anti-malware software to survive in the wild to be seriously
broken and therefore unworthy of any further consideration. ("The brakes on
this car don't work, but you can buy this add-on which will patch them...")
Yet, out of a sense of responsibility to the greater Internet community,
I've expended considerable time and money designing and deploying
anti-malware measures, NOT for my benefit, because I don't care about
malware and don't have to care, but for theirs.

However, the reach of those measures, like everything else I've done,
extends only as far as the perimeter of my own operation. That's also as
far as my responsibility extends. I can't affect what happens "out there",
other than perhaps indirectly via my limited power of persuasion,
and I certainly can't be held accountable for it.

On the other hand, who can we hold accountable for

cpe-69-133-71-112.columbus.res.rr.com [69.133.71.112]

and the other systems like it? I'll argue:

a) the putative (former) owner whose desk or lap it sits on
b) the new (real) owner(s) who have repurposed it
c) RoadRunner's network engineers
d) the vendor of the operating system it's running (do I even need
to mention which one passive OS fingerprinting indicates?)

So if we must get into a policy debate about responsibility, I'll be
happy to engage in discussion about which of a-d should be at the front
of the line, and which should bring up the rear.

---Rsk

[1] In the typical case of zombies like the ones I listed above, the
treatment of *any* SMTP response code is "ignore it". The SMTP engines
installed on such systems are designed to maximize attempted deliveries
per unit time, and one excellent way to do that is to ignore everything
sent back by the recipient MX.

[David Wilson]

Post by Rich Kulawiec
And second, have you (rhetorical you) been paying attention to the
origination points of the vast majority of attempted malware
deliveries over the past decade?

The error here is to assume that you get the message from the
originating system. There are scenarios in which you get the message
from an innocent (perhaps in more than one sense of the word)
*intermediate* MTA. It is the reaction of such an MTA to a rejection
which is the problem.

An intermediate MTA is not be a open relay, because relaying is often
defined in terms of the system from which the message arrives. For
instance, a company has a "boundary" MTA. A PC within the company's
network is compromised (the user accessed an unsuitable web site in
their lunch break). The trojan sends messages via the boundary MTA.
Unless that MTA checks the return-path address, and the boundary MTA
picks up on the virus, then the rejection by the next hop MTA is a
problem.

After all, if the "MTA" from which you received the infected message is
not innocent, perhaps not even a proper MTA, then rejecting the message
is also pointless. The rejection will be ignored, and so the overall
effect will be the same as if it were accepted and discarded.

[Rich Kulawiec]

Post by David Wilson
The error here is to assume that you get the message from the
originating system. There are scenarios in which you get the message
from an innocent (perhaps in more than one sense of the word)
*intermediate* MTA. It is the reaction of such an MTA to a rejection
which is the problem.

That's not an assumption, it's a fact. If 1.2.3.4 connects to my MX's
and attempts delivery, then 1.2.3.4 IS the originating system, as far
as my MX is concerned.

Now the message may -- or may not -- have *really* originated on 1.2.3.4,
but that's of course quite impossible to know. (I trust everyone is aware
that malware routinely adds fake headers, so anything it claims about
itself, including its putative origin, shouldn't be believed. The only
things you can know about it are things that you independently determine.)

Post by David Wilson
An intermediate MTA is not be a open relay, because relaying is often
defined in terms of the system from which the message arrives. For
instance, a company has a "boundary" MTA. A PC within the company's
network is compromised (the user accessed an unsuitable web site in
their lunch break). The trojan sends messages via the boundary MTA.
Unless that MTA checks the return-path address, and the boundary MTA
picks up on the virus, then the rejection by the next hop MTA is a
problem.

So we are to presume that malware which has taken over user U's PC,
and thus is likely already in full control of everything it does,
is going to send a copy of itself through the outbound MTA on that
network, with the goal that it will be rejected by someone's MX,
so that the MTA will generate an NDR and send it back to user U...
who is already infected? What, exactly, is the point?

Or that this same malware, having rummaged through user U's inbox and
outbox and other files resident on the disk drives on U's PC, has
now decided to attempt delivery to co-workers V, W, and X, and rather
than just attaching itself to the next outbound messages from X to
any of them, or crafting a plausible forgery and sending it now,
will instead try to forge V/W/X's addresses as the putative sender,
submit it to the outbound MTA, and hope that something rejects it
so that the local MTA will send an NDR with it to V/W/X?

We've wandered into fantasyland here. Why should such elaborate tricks
be used when there are a large number of much easier, eminently successful
tactics to try? After all, if the MTA doesn't know how to detect this
particular bit of malware when it's headed to an external destination,
then it seems unlikely it'll detect it when it's headed to an internal one.
The simplest course is for it to just send itself directly -- or, even
better, to infest the system running the MTA, which makes further
propagation considerably simpler.

Post by David Wilson
After all, if the "MTA" from which you received the infected message is
not innocent, perhaps not even a proper MTA, then rejecting the message
is also pointless. The rejection will be ignored, and so the overall
effect will be the same as if it were accepted and discarded.

*If* it's not a proper MTA, yes. But you have no way to know that,
nor can you be certain that you're not making a classification error,
so best practice remains to use the SMTP protocol to indicate
rejection. If it does turn out to be a proper MTA and/or if it does
turn out you've made an error, you've given everyone a decent chance
of noting it, tracking it, and fixing it.

---Rsk

[Paul Russell]

Post by David Wilson
After all, if the "MTA" from which you received the infected message is
not innocent, perhaps not even a proper MTA, then rejecting the message
is also pointless. The rejection will be ignored, and so the overall
effect will be the same as if it were accepted and discarded.

If I understand your position correctly, you want the receiving MTA to issue a
5xx when the sender is a real mail server, but you want it to accept and discard
the message when the sender is a bot. As has already been pointed out, for
systems outside your control, you can only speculate as to their true nature and
their likely reaction of a 5xx response. Why waste time trying to discern the
true nature of the sender, and run the risk that you will discard messages which
should have been rejected, because your analysis of the sender is imperfect?
Just issue the 5xx and be done with it.

[David Wilson]

Post by Gordon Peterson

Post by David Wilson
After all, if the "MTA" from which you received the infected message

is

Post by David Wilson
not innocent, perhaps not even a proper MTA, then rejecting the

message

Post by David Wilson
is also pointless. The rejection will be ignored, and so the overall
effect will be the same as if it were accepted and discarded.

If I understand your position correctly, you want the receiving MTA to issue a
5xx when the sender is a real mail server, but you want it to accept and discard
the message when the sender is a bot. As has already been pointed out, for
systems outside your control, you can only speculate as to their true nature and
their likely reaction of a 5xx response. Why waste time trying to discern the
true nature of the sender, and run the risk that you will discard messages which
should have been rejected, because your analysis of the sender is imperfect?
Just issue the 5xx and be done with it.

No, quite the reverse. If you have received an infected message from a
real MTA, then issuing a 5xx response might "do bad things". I.e. that
real MTA might send a DSN containing the infection to a forged returned
path.

What I said was that there was no point in sending a 5xx response to a
bot, since they will ignore it. (I suppose they might remove the
recipient address from their list, but I don't know if this actually
happens.)

So, I believe that sending 5xx to an innocent sender can be dangerous,
and sending 5xx to a bot is pointless.

The only problem with not sending 5xx (or not sending a DSN) could be
with false positives. However, I believe this is a very small problem,
at least for the majority of users, and the danger of using 5xx is
significantly greater.

[Chris Lewis]

Post by David Wilson
The only problem with not sending 5xx (or not sending a DSN) could be
with false positives. However, I believe this is a very small problem,
at least for the majority of users, and the danger of using 5xx is
significantly greater.

Our experience, issuing 55x at levels of 250,000-1,300,000 per day since
1997 indicates exactly the reverse.

Certainly, there's no way to prove that "it" (an infection) hasn't
happened, but considering we've had precisely _one_ report of a 55x (for
any reason whatsoever) landing in the forgee's mailbox in all that time,
it can't be significant. In contrast to the dozen or so FPs per day
that we do handle. Silently dropping emails would mean that those dozen
or so FPs would go unnoticed by anyone.

[David Wilson]

Post by Chris Lewis
.
Our experience, issuing 55x at levels of 250,000-1,300,000 per day since
1997 indicates exactly the reverse.
Certainly, there's no way to prove that "it" (an infection) hasn't
happened, but considering we've had precisely _one_ report of a 55x (for
any reason whatsoever) landing in the forgee's mailbox in all that time,
it can't be significant. In contrast to the dozen or so FPs per day
that we do handle. Silently dropping emails would mean that those dozen
or so FPs would go unnoticed by anyone.

Interesting. At least there are some numbers here.

Just to check, are these 250K-1300K 5xx response *just* for messages
which have fallen foul of Anti-Virus checks (i.e. the number does not
include 5xx responses for other causes like invalid recipients).

As Alessandro Vesely has commented, this does seem a high AV FP rate.

How do you find out about the FP (since the message is, by definition,
rejected)? Obviously, if the sender simply resends, it would be rejected
again. So, there must be some other mechanism you have for finding out
that this particular message was not actually infected.

What kinds of message typically generate FPs? What AV signatures are
being hit by these FPs? How do you verify that the message is not
actually a problem?

Maybe we are lucky, but we do not see many messages which appear to be
infected - perhaps only 0.6% of the overall traffic. Nearly 90% of
received messages appear to be spam. Do you get a significantly higher
proportion of (purported) infected messages?

Although the number of reported infections from rejected messages is
just one, maybe this is not surprising. Firstly, there are clearly "out
there" a very large number of machines which are infected and the owner
either does not know or does not care. So, if any of these have been
infected as a result of one of your rejections, you would not know. Even
if the owner finds out their machine is infected, I would think it is
rare for them to try to trace the origin of the infection. The work
involved in getting their machine clean is probably enough for them. It
might involve re-installing the OS, which would probably wipe the
evidence.

[Chris Lewis]

Post by David Wilson

Post by Chris Lewis
.
Our experience, issuing 55x at levels of 250,000-1,300,000 per day since
1997 indicates exactly the reverse.
Certainly, there's no way to prove that "it" (an infection) hasn't
happened, but considering we've had precisely _one_ report of a 55x (for
any reason whatsoever) landing in the forgee's mailbox in all that time,
it can't be significant. In contrast to the dozen or so FPs per day
that we do handle. Silently dropping emails would mean that those dozen
or so FPs would go unnoticed by anyone.

Interesting. At least there are some numbers here.
Just to check, are these 250K-1300K 5xx response *just* for messages
which have fallen foul of Anti-Virus checks (i.e. the number does not
include 5xx responses for other causes like invalid recipients).

It's all filter hits, including spam, but not invalid recipients (we
don't, but should, count these at present - at least in production. On
the traps invalid recipient is 20-30M/day by definition ;-).

That said:

1) That 1.3M/day was virus rejects - the Mydoom explosion in Feb 2006
(IIR the year correctly). The total rejects for those days was actually
1.6M/day (I rechecked the numbers - was remembering Mydoom's numbers
alone for the total).

2) The FN rate of AV tools these days is atrociously bad. Studies have
shown that "new" viruses (in this context meaning different MD5
checksums, meaning the same old virus just packed differently) is missed
by 90% of all AV packages, and that only rises to 50% after a month.

In fact, some AV vendors are not bothering to update MD5 databases for
certain viruses, because they're different for every copy. That's not
supposition, at least one vendor explicitly said so. In person ;-)

I can't tell you how many times I've submitted a cutwail binary to
virustotal and found that not one of virustotal's battery of detectors
(commercial AV products) fired on it. Despite cutwail being (a) the 2nd
or 3rd most prolific malcious email sender (peaked at over 20% of all
malicious email), and (b) out there for better than a year.

As a consequence of (2), any non-trivial environment who relies purely
on AV products (even multi-layer such as us, with email and web and
generalized) for virus protection is going to get hosed. They have to
add in different approaches, such as DNSBL, SURBL, custom patterning and
executable detection. With _both_ a higher FP rate, and the inability
to tell whether it's a virus or not. You can't choose to "bounce (or
reject) spam and not viruses" because the filter is unable to make the
distinction. We used to publish "virus rejects" - we don't anymore,
because we can't distinguish them.

We have an internal formal AV engine in email. But we don't rely on it
for the heavy lifting of the majority of malicious attacks. The front
ends use anti-spam techniques, and catch better than 99% of it before
the AV sees (and probably misses) it.

Post by David Wilson
As Alessandro Vesely has commented, this does seem a high AV FP rate.

The dozen or two FPs is are all a result of rejects, regardless of
actual rule (virus, spam or otherwise)

Post by David Wilson
How do you find out about the FP (since the message is, by definition,
rejected)? Obviously, if the sender simply resends, it would be rejected
again. So, there must be some other mechanism you have for finding out
that this particular message was not actually infected.\

The rejection message (see previously in this thread for a real example)
states very clearly what the sender should do. Email the FP handler.
Which is my responsibility. So I see them all.

Post by David Wilson
What kinds of message typically generate FPs? What AV signatures are
being hit by these FPs? How do you verify that the message is not
actually a problem?

Executables that aren't viruses. Ordinary email coming from infected
machines, or in amongst large numbers of badly infested machines. Very
badly configured machines. Etc.

We treat the FP process as simply part of filtering. If you get a
reject, report it, and we'll fix. No email lost. Merely delayed.

Post by David Wilson
Maybe we are lucky, but we do not see many messages which appear to be
infected - perhaps only 0.6% of the overall traffic. Nearly 90% of
received messages appear to be spam. Do you get a significantly higher
proportion of (purported) infected messages?

I think you have an overly narrow definition of viruses - specifically,
emails that carry malicious executables _directly_. The trap
demonstrates that, at times, you'll see up to 50% of its entire flow is
viral in the sense of, for example, links to malicious files.

Most AV products probably didn't fire on the MSNBC/CNN viral attack.
Yet, at one point a few months ago, that alone was 25% of all trap
traffic. That was concurrent with several other bot families expending
a considerable fraction of their output trying to propagate instead of
sending spam.

Post by David Wilson
Although the number of reported infections from rejected messages is
just one, maybe this is not surprising. Firstly, there are clearly "out
there" a very large number of machines which are infected and the owner
either does not know or does not care. So, if any of these have been
infected as a result of one of your rejections, you would not know.

We'd get reports of us being responsible for sending a virus if _only_
from us being mentioned in the rejection notice buried in the email.
Given the sheer volume of people who can't read headers well enough to
identify obviously forged received lines and report the spam to us
because it has a 47/8 address in it, we'd get at least _some_ reports of
us (apparently) bouncing viruses at one point in the past 11 years.

We NEVER have. The only rejection notice of this type we've ever had
drawn to our attention is when an infected ISP customer sent us a spam
forged in another customer of that same ISP, and our rejection caused
the ISP to send the reject back to the forgee.

Only once in 11 years. At our volumes, even if off by several orders of
magnitude, the risk of reject-induced-bounce-of-virus is insignificant.
Even more insignificant when you factor in the cost of missing business
email because you blackholed it because of a filter FP.

[David Wilson]

Post by Chris Lewis
I think you have an overly narrow definition of viruses -
specifically,
emails that carry malicious executables _directly_. The trap
demonstrates that, at times, you'll see up to 50% of its entire flow is
viral in the sense of, for example, links to malicious files.

What I am talking about is messages for which the content is reported by
AV software as being infected. The probability that such a message is
not harmful is likely to be very, very small. A system should behave
responsibly with such a message, and not risk sending it in a bounce
message anywhere. Rejecting it might result in the generation of such a
bounce, if you have received the message from an intermediate MTA.

[Douglas Otis]

Post by David Wilson

Post by Chris Lewis
I think you have an overly narrow definition of viruses -
specifically, emails that carry malicious executables _directly_.
The trap demonstrates that, at times, you'll see up to 50% of its
entire flow is viral in the sense of, for example, links to
malicious files.

What I am talking about is messages for which the content is
reported by AV software as being infected. The probability that such
a message is not harmful is likely to be very, very small. A system
should behave responsibly with such a message, and not risk sending
it in a bounce message anywhere. Rejecting it might result in the
generation of such a bounce, if you have received the message from
an intermediate MTA.

Chris Lewis made excellent points about low detection rates with
today's highly polymorphic threats. Some providers in emerging
markets are transparently intercepting traffic in a store and forward
mode, and then bounce rejections. Those bouncing the message should
adhere to RFC 3464, use the top level content type of multipart/
report, and then minimize the amount of original content. This makes
NDNs less inviting as a method to distribute spam, creates less danger
with respect to malware, and lowers the chances of being identified as
a source of spam. False positives can still be determined by the
sender, but better MUA automation for locating original messages would
be helpful.

-Doug

[Rich Kulawiec]

Post by David Wilson
So, I believe that sending 5xx to an innocent sender can be dangerous,
and sending 5xx to a bot is pointless.

But you have no way to tell which is which. For example: putatively-"real"
mail servers get botted as well. How should we then treat them? And
from time to time, a system will turn up that looks much like a bot --
generic rDNS, fingerprints as Windows, etc. -- but turns out to be a
real mail server.

Note as well that 5XX responses do not go to senders: they go to
connecting SMTP clients. There's no way to know whether or not those
clients will in turn deliver anything to the putative sender. (Nor
is there any way to know, in the cases where they *do* deliver
something, and in the subset of those cases where this delivery includes
putatively-malignant content, whether or not it's something that's
already long since been delivered to the user by something else.)

Post by David Wilson
The only problem with not sending 5xx (or not sending a DSN) could be
with false positives. However, I believe this is a very small problem,
at least for the majority of users, and the danger of using 5xx is
significantly greater.

Just the opposite is true here. Malware is a trifling concern (thanks
to the usage of superior operating systems and mail clients, along with
abundant paranoia), but false positives are an intermittent, and annoying,
problem. So while I don't doubt that your perspective applies to your
operation, it doesn't apply here: what to some people is a crafty,
dangerous virus that represents an immediate threat is to us merely
a curious collection of bits.


The broader point is that I see no reason why we should change the SMTP
protocol in order to accomodate the thriving malware ecosystem made
possible in large measure by the abysmally low quality of Microsoft
products. That's akin to putting duct tape over a brightly-flashing
red light because we don't want to know there's a problem. I think a
much better approach is to attack the problem at its source, or,
failing that, to at least contain the problem at our network perimeters
using the tools we already have. (And as to my [rhetorical my, mostly]
supposed responsibility to others, I've spent decades discharging that
obligation not just by making it my highest priority to prevent outbound
abuse from anything I touch, whether /32 or /8, but by freely giving
my best professional advice in re the selection of operating systems
and applications. I can hardly be held at fault by those who have
failed to heed that advice and are now dealing with the very predictable
consequences.

---Rsk

[SM]

Post by Rich Kulawiec
And second, have you (rhetorical you) been paying attention to the
origination points of the vast majority of attempted malware deliveries
over the past decade? They're coming from places like this (seen within
11.red-81-43-100.staticip.rima-tde.net [81.43.100.11]
12-216-204-70.client.mchsi.com [12.216.204.70]
122-82-124-91.pool.ukrtel.net [91.124.82.122]
1809ds4-fb.0.fullrate.dk [90.184.161.72]
78-0-202-32.adsl.net.t-com.hr [78.0.202.32]
89-180-255-125.net.novis.pt [89.180.255.125]
cable-87-116-168-96.dynamic.sbb.rs [87.116.168.96]
cpe-69-133-71-112.columbus.res.rr.com [69.133.71.112]
ip-90-187-210-98.web.vodafone.de [90.187.210.98]
Do you think any of those are actual mail servers? Do you think that
any of them actually are running an SMTP engine which does something
with rejected traffic? Do you think that any of them are running an SMTP
engine which even *notices* rejected traffic?

A few of them may be actual mail servers. Not all of them may be
listening for SMTP connections. A negligible number of them may do
something about rejected traffic.

Some people find it easier to block SMTP traffic from such
origination points than to put in the resources to handle them. If
we are looking for an identifier which says that "this IP address may
originate SMTP traffic", then it's better to invent one than to rely
on hostnames.

Regards,
-sm

[David Wilson]

Post by SM
Some people find it easier to block SMTP traffic from such
origination points than to put in the resources to handle them. If
we are looking for an identifier which says that "this IP address may
originate SMTP traffic", then it's better to invent one than to rely
on hostnames.

Some DNSBLs have categories which identify IP addresses which are not
expected to originate SMTP traffic.

[SM]

Post by David Wilson
Some DNSBLs have categories which identify IP addresses which are not
expected to originate SMTP traffic.

That cannot be standardized.

The usual suspects are already aware of the arguments that have been
brought up in this thread as they have heard them before. Even
though my question about where the message originated wasn't about
the techniques used to validate the origin, I'll summarize them:

SPF and SenderID - IP address evaluation (for the long answer, see relevant
specifications)

DomainKeys and DKIM - Domain-based evaluation

DNSBL - IP address evaluation (usually proprietary)

Hostnames - Reverse DNS evaluation (heuristics)

Whitelisting - Determined by operational considerations

These techniques are about how to restrict who can talk to us. At
another level, we can frame the question as what content is safe and
relevant to us. None of the techniques listed above address that.

Regards,
-sm

[Paul Russell]

Post by David Wilson

Post by Rich Kulawiec

Post by Gordon Peterson
That's a bogus argument. You're basically saying that "someone else
will probably pick up on this" and using that as a (lame) excuse to lob
a grenade their way.

Perhaps you should review the SMTP protocol: sending a 5XX response
*refuses* delivery of a message. It does not transmit (or retransmit)
a message. It is difficult to see how one can be accused of "lob[bing]
a grenade" when one has never taken possession of it.

Because of the normal action of an MTA when it receives such an 5XX
response, i.e. it sends a non-delivery message, normally containing the
message, to the return path address. If that return path address is
forged, then the infection is bounced elsewhere. That is the grenade.

The grenade was not lobbed by the MTA which issued the 5xx response; it was
lobbed by the MTA which previously accepted the message from some source and
attempted to deliver it to the MTA which issued the 5xx response during the
SMTP session.

[Ian Eiloart]

Post by der Mouse

However, I do think that it's time that domain owners started taking
responsibility for the use of their domains.

If I, say, were to forge mail from your domain to, say, some ibm.com
user, would you take responsibility for it? That's what you seem to me
to be saying.

Not quite, but not far off. I'm not taking responsibility for your decision
to forge the domain. However, I AM responsible for the fact that IBM have
no easy way to detect that forgery. I SHOULD accept that IBM are therefore
more likely to bounce that email back to me.

When a spammer forges my domain, and the recipient bounces the message,
they're doing what SMTP is designed to do. I shouldn't get pissed off
with
the recipient, if I've failed to publish records that allow them to
check for forgeries.

What I should take responsibility for is publishing records that let IBM
easily check whether the email is legitimate. For example, if I publish SPF
records, and sign all my outgoing mail with DKIM, and require my users to
use my Message Submission Service, then I think I've discharged that
responsibility.

Having taken those actions (actually, I haven't but this is hypthetical),
I'm quite happy to accept any bounce messages from IBM - provided they've
checked for an SPF or DKIM record match before accepting the message.

Now, given that I haven't yet published SPF or DKIM records, I still get
lots of spam bounced into my domain. Who do I blame? Well, in part it's my
responsibility for failing to help the spam recipients. If I did publish
those records, then recipient domains could more easily detect forgeries
out of my domain.

SPF records would be useful because you can check them early, and cheaply.
DKIM give senders the opportunity to get messages through forwarders.

Now, what I'm suggesting (not advocating yet, because I'm not certain that
this is right), is that when there's no SPF record published, that we
should not feel too bad about bouncing email because the domain
administrator isn't taking adequate steps to protect the domain against
spam blowback, and against phishing. Of course, I'm NOT suggesting that
lack of an SPF record should score very high in any any spamicity measure,
but it might count for something.

Now, an SPF or DKIM match gives us the huge gain that we can bounce
messages selectively based on the content. Some recipients may not want
certain message content, but by the time we've seen the content SMTP
doesn't allow us to selectively reject. And, a pass allows us to
confidently whitelist a domain or sender address without opening up a huge
hole in our spam defences. For example, I've refused to whitelist UK
academic research boards because they don't publish SPF records.

Now, the question is, what do we do when we see an SPF softfail, and no
DKIM record? Well, at first I'd be reluctant to bounce the message, because
I want to reward admins that are taking steps to protect their domains.
However, that's not something that I'd be happy with on a permanent basis.
Eventually, I want the world to be in a position where all forwarders check
SPF on the way in, and rewrite sender domains. When we're sufficiently
advanced in that project, then bouncing softfail would be more acceptable.

No SPF record:
SPF fail: reject the message at SMTP time.
SPF softfail: check DKIM
SPF pass: accept if you trust the domain, otherwise do other spam checks.
You can reject, bounce or accept depending on other spam checks.
Importantly, you can bounce PER RECIPIENT depending on the content, which
you can't do with SMTP rejections.

Post by der Mouse

In a world where you *can* prevent abuse of your domain, what's wrong
with bouncing email?

Depends on what that prevention requires. ("Sure you can prevent abuse
of your domain. You just need to enter into a bilateral agreement with
every receiving mailhost on the net.")

No, you just need to sign your outgoing email, and publish SPF records.

I know, someone's going to tell me that SPF breaks forwarding. Well email
is already broken. I get up to a million spam messages into my domain
daily. Everything we do to prevent spam causes problems ranging from
additional load on our DNS servers to loss of important emails. The
question is, are we going to fix it? I think what I'm proposing would at
least allow people to do safe whitelisting

[David Wilson]

Post by Ian Eiloart

Post by der Mouse
Depends on what that prevention requires. ("Sure you can prevent

abuse

Post by der Mouse
of your domain. You just need to enter into a bilateral agreement

with

Post by der Mouse
every receiving mailhost on the net.")

No, you just need to sign your outgoing email, and publish SPF
records.

And make sure that you send from hosts which are permitted by your SPF
info.

At one point I purchased on-line from Maplin, so they send me marketing
messages. maplin.co.uk has SPF information, but these messages are not
sent from one of the permitted hosts!

I have also seen an SPF failure on a message from a friend who has a
'vanity' domain for their family. The message came via their ISP's MTA,
and so failed as this is not permitted by their SPF info. In this case
the SPF info has probably been created by the domain registration
organization.

[Rich Kulawiec]

Post by Ian Eiloart
Now, given that I haven't yet published SPF or DKIM records, I still get
lots of spam bounced into my domain. Who do I blame?

Oh, that's easy. You blame the people running badly-broken mail systems
that bounce instead of rejecting. Their IP addresses are contained
in your logs and in the headers of the backscatter you're getting.

Fixing this problem is a straightforward exercise modulo a few edge
cases -- but the practical impact of those edge cases can be minimized
by judicious use of simple tools and procedures (e.g., cron, make, rsync).

(Example: one of the common problems of this type comes up in multi-tier
mail systems lacking LDAP or another directory service. The problem
arises when an address becomes invalid on the internal system but is
still considered valid on the external MXs. One easy solution to
this is to use a script to grab the current-list-of-valid-addresses
from the internal system at intervals, process it, and then rsync
it out to the external MX's. This reduces possible backscatter due
to this problem to "the set of recently-removed addresses" and further
to "the subset of those receiving forged-sender mail during the
interval between updates". It's thus possible to reduce the
probability of backscatter arbitrarily by reducing the interval.)

---Rsk