Experience with DKIM signatures and DCC

[Gary Mills]

I've started using DKIM signatures to whitelist some e-mail messages.
I use whitelist entries like this:

    ok      substitute Authentication-Results electra.cc.umanitoba.ca; dkim=pass (1024-bit key) header.i=@...

The advantage of doing it this way is that users won't do their own
whitelisting by sender address.  When they do that, they also allow
lots of phishing messages to get through.  Financial institutions seem
to be beset with this nuissance.

So far, I've only whitelisted five domains by DKIM signature.  It's
easy to find Authentication-Results lines in the sendmail logs, but
those are not sufficient for whitelisting.  For example, some of the
ones that show up most frequently are from e-mail marketing companies.
They own long domain names like @fastmarketingeleven.com,
@restaurantpromotionsprevail.com, and @prohibitthree.com that all have
the same top-level web page, a removal application.  I'm certainly not
going to whitelist those!

SPAM reputation is critical in this game.  In most cases, I can't even
guess which domains have a good reputation and which don't.  I
certainly can't investigate all of them.  I've only found one bank so
far that uses DKIM signatures.  A reputation database is the missing
ingredient.  In terms of procedure, I'd need to begin with the
Authentication-Results log lines or headers, determine the owner of
the domain, and then look up the reputation of the owner.  Is any sort
of reputation database available now?  Soon?

--
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Vernon Schryver]

> From: Gary Mills <mills@...>

> SPAM reputation is critical in this game.  In most cases, I can't even
> guess which domains have a good reputation and which don't.  I
> certainly can't investigate all of them.  I've only found one bank so
> far that uses DKIM signatures.  A reputation database is the missing
> ingredient.  In terms of procedure, I'd need to begin with the
> Authentication-Results log lines or headers, determine the owner of
> the domain, and then look up the reputation of the owner.  Is any sort
> of reputation database available now?  Soon?

Instead of only whitelisting by DKIM success,
why not also blacklist by DKIM failure or IP address reputation?

There are now many IP address reputation schemes in addition to classic
DNSBLs.  Some are Commtouch's, Ciphertrust's, and DCC Reputations.
Commtouch's can be queried as if it were a DNSBL.  DCC Reputations are
built into the commercial DCC code.  A lot of phishing can be blocked
by using Spamhaus' ZEN DNSBL, which includes Spamhaus' PBL.  I think
DCC Reputations and Spamhaus' ZEN are cheapest of those Spamhaus's ZEN
has very few false positives and generally can be used without local
whitelists.  Umanitoba.ca's traffic is non-commercial and might be low
enough to qualify for free access to Spamhaus' ZEN.  See
http://www.spamhaus.org/organization/dnsblusage.html


Vernon Schryver    vjs@...
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[John Levine]

> Instead of only whitelisting by DKIM success,
> why not also blacklist by DKIM failure or IP address reputation?

I wouldn't do anything with DKIM failure at this point, since there are
way too many reasons that legit mail could arrive with a valid signature.

Whitelisting DKIM sigs from people you know and white and blacklisting of
IPs with particularly good and bad histories should work well.

R's,
John
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[John Levine]

> I wouldn't do anything with DKIM failure at this point, since there are way
> too many reasons that legit mail could arrive with a valid signature.

Ahem, with an INvalid signature.

>
> Whitelisting DKIM sigs from people you know and white and blacklisting of IPs
> with particularly good and bad histories should work well.
>
> R's,
> John
>
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Gary Mills]

On Sat, Mar 29, 2008 at 04:59:45PM +0000, Vernon Schryver wrote:

> > From: Gary Mills <mills@...>
>
> > SPAM reputation is critical in this game.  In most cases, I can't even
> > guess which domains have a good reputation and which don't.  I
> > certainly can't investigate all of them.  I've only found one bank so
> > far that uses DKIM signatures.  A reputation database is the missing
> > ingredient.  In terms of procedure, I'd need to begin with the
> > Authentication-Results log lines or headers, determine the owner of
> > the domain, and then look up the reputation of the owner.  Is any sort
> > of reputation database available now?  Soon?
>
> Instead of only whitelisting by DKIM success,
> why not also blacklist by DKIM failure or IP address reputation?

The most common reason for DKIM signature failure happens when a
message from a @gmail.com user, with a valid signature, is submitted
to a mailing list and subsequently distributed to the subscribers.
In this case, signature validation correctly fails because another
domain has taken responsibility for the message.

> There are now many IP address reputation schemes in addition to classic
> DNSBLs.  Some are Commtouch's, Ciphertrust's, and DCC Reputations.
> Commtouch's can be queried as if it were a DNSBL.  DCC Reputations are
> built into the commercial DCC code.  A lot of phishing can be blocked
> by using Spamhaus' ZEN DNSBL, which includes Spamhaus' PBL.  I think
> DCC Reputations and Spamhaus' ZEN are cheapest of those Spamhaus's ZEN
> has very few false positives and generally can be used without local
> whitelists.  Umanitoba.ca's traffic is non-commercial and might be low
> enough to qualify for free access to Spamhaus' ZEN.  See
> http://www.spamhaus.org/organization/dnsblusage.html

We are using Spamhaus' XBL, and are happy to pay for it.  What I'm
looking for now is something that rates domain names by reputation.
Spamhaus was working on such a database, but I haven't heard anything
about that for some time.  My main concern is to stop the phishing
messages that rely on forged sender addresses.

--
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Vernon Schryver]

> From: Gary Mills <mills@...>

> We are using Spamhaus' XBL, and are happy to pay for it.  

Since you are already using the XBL, I think you should switch to
Spamhaus' ZEN unless you are checking the XBL via dccm, dccproc, or
dccifd.  Even if you are using `dccm -B`, you should enable ZEN checks
on SMTP clients and on MX servers for SMTP envelope domains with something
like this in /var/dcc/dcc_conf
DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 %ID %BT http://www.spamhaus.org/query/bl?ip=%BIP' -Bsbl-xbl.spamhaus.org -Bset:no-NS -Bzen.spamhaus.org"

That is because ZEN/PBL includes IP addresses of legitimate DNS servers
and so should not be used for the default dccm, dccproc, or dccifd DNSBL
checks on NS records.


>                                                           What I'm
> looking for now is something that rates domain names by reputation.
> Spamhaus was working on such a database, but I haven't heard anything
> about that for some time.  My main concern is to stop the phishing
> messages that rely on forged sender addresses.

An anti-phishing domain name reputation service is a hard problem,
because the bad guys continually create floods of new names and work
hard to cover their tracks.  Listing bad domains soon enough to help
or even before the bad guys have abandoned them would be hard.  The bad
guys also vary ("fast flux") the IP addresses of their SMTP clients,
HTTP servers, and even leaf DNS servers, but they are generally constrained
to IP addresses listed in Spamhaus' ZEN/PBL and they cannot change their
IP addresses in the gTLDs as quickly.  I see lots of hits by the dccm
checks of NS records.  Body checks of URLs (including NS RRs) are also
quite effective.


Vernon Schryver    vjs@...
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[John Levine]

> We are using Spamhaus' XBL, and are happy to pay for it.  What I'm
> looking for now is something that rates domain names by reputation.

At this point, there isn't one.

Take a peek at http://www.domain-assurance.org/, a little trade
association where we're trying to set standards for domain based
whitelists and rep systems.  We've got lots of interest but so far the
implementation is pretty weak.

I agree with Vernon that in general reputation is pretty hard since the
bad guys have an unlimited supply of new domains.  That's why it makes
more sense to start with whitelists, since good guys tend to hold still.

R's,
John
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Vernon Schryver]

> From: John Levine <johnl@...>

> ...
> I agree with Vernon that in general reputation is pretty hard since the
> bad guys have an unlimited supply of new domains.  That's why it makes
> more sense to start with whitelists, since good guys tend to hold still.

Whitelists are pain, and especially when not modulated by other
information such as a DCC bulk indication or some sort of bad
reputation.  For example, judging from my tests sending mail to a
test account with default settings, Hotmail assumes mail from
strangers is spam.  (It's no surprised that junk advertising from
Microsoft to a Hotmail mailbox is evidently not from a stranger.)
If you are a conspiracy theory fan, you might see such policies as
part of an effort to box people back into the old media model of a
few government/corporate senders and zillions of passive receivers.


Vernon Schryver    vjs@...
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Gary Mills]

On Sat, Mar 29, 2008 at 06:42:57PM +0000, Vernon Schryver wrote:

> > From: Gary Mills <mills@...>
>
> > We are using Spamhaus' XBL, and are happy to pay for it.  
>
> Since you are already using the XBL, I think you should switch to
> Spamhaus' ZEN unless you are checking the XBL via dccm, dccproc, or
> dccifd.  Even if you are using `dccm -B`, you should enable ZEN checks
> on SMTP clients and on MX servers for SMTP envelope domains with something
> like this in /var/dcc/dcc_conf
> DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 %ID %BT http://www.spamhaus.org/query/bl?ip=%BIP' -Bsbl-xbl.spamhaus.org -Bset:no-NS -Bzen.spamhaus.org"
>
> That is because ZEN/PBL includes IP addresses of legitimate DNS servers
> and so should not be used for the default dccm, dccproc, or dccifd DNSBL
> checks on NS records.

Yes, I'm using XBL through DCC because I want users to be able to
whitelist messages rejected by XBL in the same manner that they can
for messages rejected for bulkiness.  I'm using this setting:

    DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 id %s from %s rejected. See http://www.spamhaus.org/xbl/' -Bset:no-body -Bset:no-MX -Bset:no-NS -Bxbl.dnsbl,any"

I don't want to use PBL, included in ZEN I believe, because it includes
the IP networks of many of our SMTP mail submission clients.  I don't
want to reject those.  Now that most ISPs are blocking the SMTP port,
it may be possible to revisit that decision.

--
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Vernon Schryver]

> From: Gary Mills <mills@...>

> > DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 %ID %BT http://www.spamhaus.org/query/bl?ip=%BIP' -Bsbl-xbl.spamhaus.org -Bset:no-NS -Bzen.spamhaus.org"

> Yes, I'm using XBL through DCC because I want users to be able to
> whitelist messages rejected by XBL in the same manner that they can
> for messages rejected for bulkiness.  I'm using this setting:
>
>     DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 id %s from %s rejected. See http://www.spamhaus.org/xbl/' -Bset:no-body -Bset:no-MX -Bset:no-NS -Bxbl.dnsbl,any"

Why turn off XBL MX and NS checks for the SMTP envelope mail sender domain?

> I don't want to use PBL, included in ZEN I believe, because it includes
> the IP networks of many of our SMTP mail submission clients.  I don't
> want to reject those.  Now that most ISPs are blocking the SMTP port,
> it may be possible to revisit that decision.

So your SMTP mail submission clients are on too many networks to whitelist?
And they don't use SMTP-AUTH or TLS and that could be automatically
whitelisted by modifying sendmail.cf with /var/dcc/libexec/hackmc -T
and doing the things mentioned in the comments in hackmc?  Or turning
off FEATURE(`delay_checks') or setting TRUST_AUTH_MECH can't be done
in your situation?  ok.


Vernon Schryver    vjs@...
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Gary Mills]

On Sun, Mar 30, 2008 at 02:17:09PM +0000, Vernon Schryver wrote:

> > From: Gary Mills <mills@...>
>
> > > DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 %ID %BT http://www.spamhaus.org/query/bl?ip=%BIP' -Bsbl-xbl.spamhaus.org -Bset:no-NS -Bzen.spamhaus.org"
>
> > Yes, I'm using XBL through DCC because I want users to be able to
> > whitelist messages rejected by XBL in the same manner that they can
> > for messages rejected for bulkiness.  I'm using this setting:
> >
> >     DNSBL_ARGS="'-Bset:rej-msg=5.7.1 550 id %s from %s rejected. See http://www.spamhaus.org/xbl/' -Bset:no-body -Bset:no-MX -Bset:no-NS -Bxbl.dnsbl,any"
>
> Why turn off XBL MX and NS checks for the SMTP envelope mail sender domain?

I was trying to minimize the amount of nameserver queries done for each
e-mail message.  I assume that other envelope checks are still done.
I could ramp it up a bit to see what happens.

> > I don't want to use PBL, included in ZEN I believe, because it includes
> > the IP networks of many of our SMTP mail submission clients.  I don't
> > want to reject those.  Now that most ISPs are blocking the SMTP port,
> > it may be possible to revisit that decision.
>
> So your SMTP mail submission clients are on too many networks to whitelist?

Yes, that's correct.  They could be anywhere in the world.  It's the
old problem that SMTP servers can't distinguish between clients and
other SMTP servers.

> And they don't use SMTP-AUTH or TLS and that could be automatically
> whitelisted by modifying sendmail.cf with /var/dcc/libexec/hackmc -T
> and doing the things mentioned in the comments in hackmc?  Or turning
> off FEATURE(`delay_checks') or setting TRUST_AUTH_MECH can't be done
> in your situation?  ok.

Yes, I am using some of those features.  Most clients will use SMTP
authentication or DRAC, but a few still use plain SMTP.  Our two large
local ISPs now block the SMTP port.  For clients there we do require
SMTP authentication.  I just can't tell what other clients will be
affected if I start using the PBL.

--
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Gary Mills]

On Sat, Mar 29, 2008 at 05:46:58PM -0400, John Levine wrote:
> >We are using Spamhaus' XBL, and are happy to pay for it.  What I'm
> >looking for now is something that rates domain names by reputation.
>
> At this point, there isn't one.
>
> Take a peek at http://www.domain-assurance.org/, a little trade
> association where we're trying to set standards for domain based
> whitelists and rep systems.  We've got lots of interest but so far the
> implementation is pretty weak.

Please excuse the late reply: I have dozens of other urgent projects.

I was disappointed in this web page.  This seems to be a group that
certifies or provides a `stamp of approval' for other organizations,
similar to a trade organization that certifies its own members.
Because of the built-in conflict of interest, the potential for
corruption is very high.  There's not much value for consumers here.

> I agree with Vernon that in general reputation is pretty hard since the
> bad guys have an unlimited supply of new domains.  That's why it makes
> more sense to start with whitelists, since good guys tend to hold still.

What I'm looking for is the unimpeded flow of business correspondence.
This might be e-mail between members of my university and members of
other universities, or e-mail between banks and travel agencies and
their customers at my university.  So far, from our sendmail logs,
I've found one university that employs DKIM signatures on their
e-mail; I whitelisted them by their Authentication-Results header.
I'd like to do this for other reputable senders.

I suppose what we need is for the recipients of e-mail to rate the
reputation of sending organizations.  Representing the recipients,
I'd be willing to pay for such a service.  Another alternative is
some independant rating organization that ensures that the sender
takes responsibility for their e-mail.  Of course, even a reputable
company could decide to engage in an e-mail marketing campaign to
gather more customers.  That sort of activity should reduce their
e-mail reputation.

--
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[John Levine]

>> Take a peek at http://www.domain-assurance.org/, a little trade

> I was disappointed in this web page.  This seems to be a group that
> certifies or provides a `stamp of approval' for other organizations,

No, it's not,.perhaps you should read it again.  DAC sets standards, we
don't certify anything or anyone.

VBR is basically a spec for shared domain whitelists, sort of like the way
the RBL format is a spec for shared IP blacklists.  Once you know that a
domain in a message is real via DKIM or whatever, you can use VBR to see
if it's on whatever whitelists you want to use.

> I suppose what we need is for the recipients of e-mail to rate the
> reputation of sending organizations.  Representing the recipients,
> I'd be willing to pay for such a service.  Another alternative is
> some independant rating organization that ensures that the sender
> takes responsibility for their e-mail.

Right.  Given the history of spam filters, user ratings don't work very
well because users are inconsistent.  I expect that the largest use will
be rating companies and trade groups or regulators that publish lists of
their members, e.g. the FDIC publishing lists of domains of the banks
they insure.

Regards,
John Levine, johnl@..., Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, ex-Mayor
"More Wiener schnitzel, please", said Tom, revealingly.
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Vernon Schryver]

> From: John Levine <johnl@...>
> To: Gary Mills <mills@...>
> cc: dcc@...

> >> Take a peek at http://www.domain-assurance.org/, a little trade

> No, it's not,.perhaps you should read it again.  DAC sets standards, we
> don't certify anything or anyone.
>
> VBR is basically a spec for shared domain whitelists, sort of like the way
> the RBL format is a spec for shared IP blacklists.  Once you know that a
> domain in a message is real via DKIM or whatever, you can use VBR to see
> if it's on whatever whitelists you want to use.

That might make it easier to use whitelists, but it does nothing
to solve the real problem, creating and maintaining those whitelists.


> > I suppose what we need is for the recipients of e-mail to rate the
> > reputation of sending organizations.  Representing the recipients,
> > I'd be willing to pay for such a service.

That makes one paying customer in a sea of users that would not
pay.  Without intending any offense and based on what paying customers
are willing to pay for DNSBL and other anti-spam services, it is a
customer that is probably unwilling and unable to pay enough.


> >                                            Another alternative is
> > some independant rating organization that ensures that the sender
> > takes responsibility for their e-mail.

That gets back to the conflict of interest problem.  Practically the
only sources of operating revenue for mail sender rating organizations
are senders of email.  Practically the only email senders willing to
pay for a rating are those with natural reputations that need improvement.
Consider the history of consumer goods ratings organizations.  However, if
you like the idea, consider Habeas or Ironport.


> Right.  Given the history of spam filters, user ratings don't work very
> well because users are inconsistent.  I expect that the largest use will
> be rating companies and trade groups or regulators that publish lists of
> their members, e.g. the FDIC publishing lists of domains of the banks
> they insure.

Users are inconsistent.  Besides the famous problems with "this is spam"
buttons operated by mail service providers, consider the near uselessness
of reviews of power tools, computers and other goods on the web.  Bulk
email senders are even more inconsistent.  Would you trust that FDIC
insurance implies an incoming mail message with a valid DKIM signature
is a bank statement instead of an unsolicited bulk offer for a free
credit card or brokerage services?  I wouldn't, given my piles of credit
offers from major banks using loopholes in the registery of postal
addresses that don't want credit card offers.

Such a mechanism might reduce phishing, but phishing has never been
the majority of the spam problem.  Besides, judging from the little
spam I see, the phishing problem is much improved in the last several
months.  I don't know if that is due to law enforcement efforts,
the irritating multi-part passwords required by the FDIC or someone
(the stupid questions and answers), or other things as mundane the
ever changing fads in spam.


Vernon Schryver    vjs@...
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[John Levine]

> That might make it easier to use whitelists, but it does nothing
> to solve the real problem, creating and maintaining those whitelists.

Quite right.  There are outfits working on creating whitelists, several of
which are DAC members.

> That gets back to the conflict of interest problem.  Practically the
> only sources of operating revenue for mail sender rating organizations
> are senders of email.  Practically the only email senders willing to
> pay for a rating are those with natural reputations that need improvement.
> Consider the history of consumer goods ratings organizations.  However, if
> you like the idea, consider Habeas or Ironport.

Also Return Path, Goodmail, and perhaps Trade Micro.

They all do indeed have to skate a thin line, listing people who are
willing to pay, but not ones whose mailing practices are bad enough that
the whitelist increases the amount of spam you get.

> Would you trust that FDIC insurance implies an incoming mail message
> with a valid DKIM signature is a bank statement instead of an
> unsolicited bulk offer for a free credit card or brokerage services?

No, but I'd trust that it was actual mail from a bank rather than a phish.

> Such a mechanism might reduce phishing, but phishing has never been
> the majority of the spam problem.  Besides, judging from the little
> spam I see, the phishing problem is much improved in the last several
> months.

You must be lucky.  I'd say about a third of the spam that gets through
the DNSBLs and is caught by spamassassin is phishes.

R's,
John
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc