Spamassassin, spamd and -L switch

I have been using Spamassassin in the company as our primary spam message blocker without troubles for years. The version used today is 3.2.3. It is boosted with a set of custom rules and with the rules from SARE project, with malware rules and others. It is also very useful to check for updated rules with sa-update command line utility (available in the newer versions of Spamassassin). Overall, the spams are identified quite precisely with very small false positives. The users have available simple web interface where they are able to check their caught messages.

But a week or two before, I had to begin solving a strange problem with some of them. A few users began complaining that they are receiving from 5 to 10 spams a day. That's not bad score because these users or their mail addresses are very popular among spammers and in total each of them would receive about from 2 to 5 thousands of unsolicited messages without spam filter.

So, where to begin? I took a suspicious message and tried to find out why it is evaluated as a regular message. The message is below (some unimportant parts were removed):

Return-Path: this1905@indygov.org
Received: from [187.83.34.110] (helo=rxexv)
by 168-215-192-214.dslindiana.com with smtp (Exim 4.62 (FreeBSD))
id 1KþxNR-0005Tq-Km; Wed, 2 Apr 2008 03:27:21 -0400
Message-ID: 47f334bf.3040802@indygov.org
Date: Wed, 2 Apr 2008 03:24:47 -0400
From: this1905@indygov.org
Subject: Gotcha! All Fool!

Wise Men Have Learned More from Fools... http://80.98.114.98

At first glance, it is clear to consider the message to be a spam. Included URL is placed at URIBL and SURBL blacklists. Further, the message was sent via blacklisted relay by SPAMCOP. So what's wrong?!? Let's check its score and applied rules simply with Spamassassin client:

spamc < message

I am using the spamc client because our spam filter is implemented with it. That means that in the background the spamd daemon should be running and the spam filter is communicating with it via spamc on demand. The previous command produced an output containing the following lines:

Content preview: Wise Men Have Learned More from Fools... http://80.98.114.98
[...]
Content analysis details: (4.3 points, 4.5 required)

pts rule name description
--------------------------------------------------
3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100%
[score: 1.0000]
0.6 FH_HELO_EQ_D_D_D_D Helo is d-d-d-d
0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL
0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS
0.1 AWL AWL: From: address is in the auto white-list

It seems like some rules are ignored. I'm sure that I have turned on DNS RBL checks and URIDNSBL plugin but the related rules are missing here. Debugging of spamd daemon didn't show anything. All rules were parsed and loaded successfully. To turn the debugging on restart the spamd daemon with -D option switch (or place it into the init script or into the sysconfig configuration file).

Another way how to check the rules is to bypass the spamd daemon and run the Spamassassin directly like this:

spamassassin -t < message

If you want to run it with debugging use the -D option switch. I was quite surprised when the direct checking shown me a different result:

Content preview: Wise Men Have Learned More from Fools... http://80.98.114.98
[...]
Content analysis details: (7.2 points, 4.5 required)

pts rule name description
--------------------------------------------------
3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100%
[score: 1.0000]
0.0 FH_HELO_EQ_D_D_D_D Helo is d-d-d-d
0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL
2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: 80.98.114.98]
1.5 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
[URIs: 80.98.114.98]
2.0 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
[Blocked - see ]
0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS
-1.8 AWL AWL: From: address is in the auto white-list

The mentioned missing rules - URIBL_BLACK,URIBL_JP_SURBL ... - are suddenly here. Why? What is the difference between spamc client and spamassin command? It shouldn't be any but their behaviour can be controlled with many option switches (command line arguments). I ran the spamc without any but spamd daemon may run with some. To check them we should display the spamd process and its command line arguments with e.g. ps command:

ps -A -o cmd | grep spamd

The command will display something like this:

/usr/sbin/spamd -L -d -c -m 8 -u spamass -x -s local4 -r /var/run/spamd.pid

Let's go through the arguments and discuss their purpose:

• -L - used to turn off any DNS and network tests
• -d - demonize the process
• -c - create user preferences files if they don't exist
• -m - maximum number of children to spawn from the parent process
• -u - run the process under the specified user
• -x - disable user config files
• -s - syslog facility for logging events
• -r - write the process id to the file
  

There are many others options. If you want to know them check the spamd man page (e.g. here). In summary, only the first option -L is available for the spamassassin command. We can check it with:

spamassassin --help

Now it should be everything bright. Different scores were caused by the -L switch which changed the applied rules. Its result is that RBL checks are disabled and in conclusion the corresponding rules are not applied. Why I used the switch before I'm not pretty sure. Perhaps, I hit some performance constraints because it take some time to perform the network tests.

What's the result? Don't forget to check the command line arguments of the spamd daemon. They may disable some Spamassassin features despite of their definitions in configuration files.