Anti-spam measures on mail systems at Concordia

The spam-detection mechanisms employed by IITS are perpetually a work in progress. As spammers adapt their techniques to get their messages beyond spam-detection, we need to adapt our techniques to detect the spam more effectively. However, because we need to ensure that the methods we employ do not risk the loss of any legitimate mail, and the spammers do not need to concern themselves with such risk, our spam-detection is indeed constantly playing catchup.

When we implement additional anti-spam measures on the e-mail relay machines, we need people to help us test each new measure; As an anti-spam guinea-pig, you can help by reporting on the reliability of the new detector.

Generally, there are several things that can be done at the mail relay machines to deal with e-mail messages that look like spam:

The relays can reject them outright: this should be done only when we're completely sure that the messages are spam, otherwise we risk bouncing legitimate e-mail, which will annoy people.
The relays can add different headers to messages before delivering them to the intended recipients, depending on how the relays perceive the messages. Typically, such headers would look like:
X-CU-tagtype: by clyde: (some explanatory text goes here)
where tagtype can be "Spam", "Test-Spam", "Score-Spam", or "DNSBL" . The machine tagging the mail may not be "clyde"; it may be one of the other three mail servers: bonnie, capone or dillinger.
1. The relays can tag the messages as spam:
  X-CU-Spam: by clyde: reason it is spam
  That way, people can set up their own mail software to automatically file the tagged messages separately from their legitimate mail (or, if they don't mind the risk of losing an occasional legitimate message, they can even set their email program to delete the tagged messages automatically).
2. The relays can tag the messages as "spam under test":
  X-CU-Test-Spam: by clyde: reason it is spam
  when we're trying out a new spam detector and we don't know yet what its reliability is. That way, the "anti-spam guinea-pigs" can file those messages separately, and inform "postmaster@concordia.ca" of any mis-indentfied messages.
3. The relays can tag the messages as spam when the spam score calculated by spamassassin, a spam filter, is higher than a certain pre-determined threshold.
  X-CU-Score-Spam: by clyde: spamassassin: 13(*************)
4. The relays can add a special header to messages which are detected to come from servers listed by a DNSBL (Domain-Name-Based Black List) as probable spam sources.
  X-CU-DNSBL: by clyde: DNSBL: (some-IP-address by some-list)
  Since we cannot generally 100% rely on these lists, it is up to the users to check and act on those messages so-tagged:
  - the users would decide if the tagged messages are spams or not; hence there is no need to report false-positives
  - based on their checks, the users might consider whether or not they want to continue refiling messages which match the particular DNSBL producing the false-positive.
  We believe even though this kind of spam-detection mechanism provides a tag of advisory nature only, it is a spam-fighting measure which users can make use of effectively.

You can filter such tagged messages out of your regular mail stream using the filter capability in your e-mail software. Anti-spam guinea-pigs should specifically filter on X-CU-Test-Spam header.

We have instructions on how to do this using several popular e-mail packages. If you are familiar with one that is not covered, please help us improve our instructions by writing to <postmaster@concordia.ca> with the filter set-up procedure for your mail software.

Outlook 2002
Outlook Express cannot do arbitrary header filtering
Eudora "free" version 5.0 and 5.1
Netscape 4.78
Mac OSX 10.2 Mail
Webmail (older version) on Alcor
Procmail (on Alcor) with Pine, Mail, etc.
Procmail (on another Unix system) with Pine, Mail, etc.
Procmail (on a Unix system) with MH

Once you have set up your filters, please check your "Spam" mailbox folder regularly and empty it out, after checking whether any legitimate e-mail was accidentally caught.

If you are an anti-spam guinea-pig, please check your "Test Spam" mailbox folder daily, and be sure to report any legitimate mail that got caught.