MailScanner configuration

Geplaatst 2010-09-18
Reacties Geen

This page will show some tricks I added to my MailScanner/MailWatch installation.

An alternative aproach to the HOLD trick
SEC rules to fill the database with postfix events
Script to report bayesian scores
Script to automate GEOIP updates for MailWatch

An alternative aproach to the HOLD trick

The normal method to put all messages on HOLD to pass them from postfix to MailScanner as described on
http://www.mailscanner.info/postfix.html
did not completely work for me.
I have a SMTP testing method that sends a SMTP message for each on monitored system every minute.
They are handled automagically and I have no need to delay them by passing them through MailScanner.
So I set out to find a way to bypass MailScanner for selective recipients,
This section will explain the changes you need to make.

Remove the /^Received:/ HOLD line from the header_checks file.
Because we do not wish to act on every message we receive.

Now we write a new rules file named /etc/postfix/MailScanner to bypass email to bypass@example.org:

#      These users will not be scanned with MailScanner
/^bypass@example\.org$/                 OK
#       Everyone else will go through MailScanner!
/.*/                                    HOLD
/^$/                                    HOLD

To activate it I call it as last filter of smtpd_client_restrictions in /etc/postfix/main.cf.
There it looks pretty much like:

smtpd_client_restrictions =
        permit_mynetworks,
        permit_sasl_authenticated,
        check_sender_access hash:/etc/postfix/hash/whitelist,
        check_client_access cidr:/etc/postfix/cidr/blacklist-networks,
        check_client_access cidr:/etc/postfix/cidr/spamhaus-droplist,
.....
        reject_rbl_client zen.spamhaus.org,
        reject_rbl_client blackholes.securitysage.com,
        reject_unauth_destination,
        check_recipient_access regexp:/etc/postfix/MailScanner

Just make sure you use HOLD from now on in your whitelist(s) instead of OK if you want your message still to be inspected by MailScanner!

This works for everyone except the postmaster of the master domain due to the following documented feature of postfix:

By default, Postfix probe messages have “postmaster@$myorigin” as the sender address. This is SAFE because the Postfix SMTP server does not reject mail for this address.

In order to scan those I needed the following line added to header_checks:
/^To:.*postmaster@vanderkooij.org/ HOLD

The drawback is that outgoing email from authenticated clients for example is not scanned.
But perhaps some people may find that fact a nice feature instead of a drawback.
You may need to tinker a bit with these rules to find what works for you.

SEC rules to fill the database with postfix events

First there is the effect that I use some RBL’s to block access in postfix and this may effect my users.
So I was thinking of a way to move that information into the logs along with the rest of the MailScanner stuff so they would know what happened.
For this I use SEC to parse my postfix logs.
I owe many thanks to the people on the SEC mailinglist that helped me with the various questions.
The solution below is propable not fitted for large installations.
And I will not explain how to install SEC.

The following rules work well for me.
You may need to adjust them for your needs.



#
	
		SEC ruleset to fill MailWatch logs with blocking events
	


#

	
		Trend Micro ERS database

type=Single

ptype=RegExp

desc=ERS blacklist activated for host $1 with IP $2 from $4@$5 to $6

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 554 5.7.1 Service unavailable; Client host \[(\S+)\] blocked using.blocked using Trend Micro Email Reputation database.+from=\<(\S)@(\S+)\> to=\<(\S+)@(\S+)\>.+

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$4\@$5’, from_domain=’$5’, to_domain=’$7’, to_address=’$6\@$7’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] blocked based on the Trend Micro Email Reputation database’, spamblacklisted=1, isspam=1, ishighspam=1, isrblspam=1”); \
        $query->execute();)
	

	
		Trend Micro ERS user policy

type=Single

ptype=RegExp

desc=ERS policy activated for host $1 with IP $2 from $4@$5 to $6

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 554 5.7.1 Service unavailable; Client host \[(\S+)\] blocked using.User defined policy matched.+from=\<(\S)@(\S+)\> to=\<(\S+)@(\S+)\>.+

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$4\@$5’, from_domain=’$5’, to_domain=’$7’, to_address=’$6\@$7’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] blocked based on the Trend Micro Email Reputation user policy’, spamblacklisted=1, isspam=1, ishighspam,=1, isrblspam=1”); \
        $query->execute();)
	

	
		Standard Blacklists

type=Single

ptype=RegExp

desc=blacklist $4 activated for host $1 with IP $2 from $5@$6 to $7@$8

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 554 5.7.1 Service unavailable; Client host \[(\S+)\] blocked using (\S+);.from=\<(\S)@(\S+)\> to=\<(\S+)@(\S+)\>.+

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$5\@$6’, from_domain=’$6’, to_domain=’$7’, to_address=’$7\@$8’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] is blocked by $4’, spamblacklisted=1, isspam=1, ishighspam=1, isrblspam=1”); \
        $query->execute();)
	

	
		User Unknown (Bounce to ….)

type=Single

ptype=RegExp

desc=Bounce to unknown user for host $1 with IP $2 to $5@$6

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 450 4.1.1.Recipient address rejected: undeliverable address: unknown user:.+from=\<\> to=\<(\S)@(\S+)\>.*

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        to_domain=’$4’, to_address=’$3\@$4’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] is sending a bounce to an unknown local user ($3)’, spamblacklisted=1, isspam=1, isshighspam=1”); \
        $query->execute();)
	

	
		User Unknown

type=Single

ptype=RegExp

desc=User unknown for host $1 with IP $2 from $3@$4 to $5@$6

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 450 4.1.1.Recipient address rejected: undeliverable address: unknown user:.+from=\<(\S)@(\S+)\> to=\<(\S+)@(\S+)\>.*

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$3\@$4’, from_domain=’$4’, to_domain=’$6’, to_address=’$5\@$6’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] is sending to an unknown local user ($5)’, spamblacklisted=1, isspam=1, isshighspam=1”); \
        $query->execute();)
	

	
		Sender domain not found

type=Single

ptype=RegExp

desc=Unknown domain $5 for host $1 with IP $2 from $3@$4 to $5@$6

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 450 4.1.8.+ Sender address rejected: Domain not found; from=\<(\S+)@(\S+)\> to=\<(\S+)@(\S+)\>.*

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$3\@$4’, from_domain=’$4’, to_domain=’$6’, to_address=’$5\@$6’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] is using an invalid sender domain ($4)’, spamblacklisted=1, isspam=1, ishighspam=1”); \
        $query->execute();)
	

	
		Invalid HELO

type=Single

ptype=RegExp

desc=Helo command rejected from

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 504 5.5.2 \<(\S+)\>: Helo command rejected: need fully-qualified hostname; from=\<(\S+)@(\S+)\> to=\<(\S+)@(\S+)\>.*

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$4\@$5’, from_domain=’$5’, to_domain=’$7’, to_address=’$6\@$7’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] is using an invalid hostname ($3)’, spamblacklisted=1, isspam=1, ishighspam=1”); \
        $query->execute();)
	

	
		Relay attempt

type=Single

ptype=RegExp

desc=Relaying denied to $2 for message from $3@$4 to $5@$6

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 554 5.7.1.Relay access denied; from=\<(\S)@(\S+)\> to=\<(\S+)@(\S+)\>.*

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$3\@$4’, from_domain=’$4’, to_domain=’$6’, to_address=’$5\@$6’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=’$1 [$2] is not allowed to relay’, spamblacklisted=1, isspam=1, ishighspam=1”); \
        $query->execute();)
	

	
		SPAM blacklist

type=Single

ptype=RegExp

desc=IP: $2, from: $4@$5, to $6@$7: $3

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: 554 5.7.1.Client host rejected: (.); from=\<(\S)@(\S+)\> to=\<(\S+)@(\S+)\>.

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        from_address=’$4\@$5’, from_domain=’$5’, to_domain=’$7’, to_address=’$6\@$7’, clientip=’$2’, hostname=‘balin.waakhond.net’, \
        subject=‘IP: $2, from: $4\@$5, to $6\@$7: $3’, spamblacklisted=1, isspam=1, ishighspam=1”); \
        $query->execute();)
	

	
		FALLBACK filter

type=Single

ptype=RegExp

desc=UNKNOWN BLOCK for host $1 with IP $2 : $3

pattern=.postfix.+NOQUEUE: reject: RCPT from (\S)\[(\S+)\]: (.*)

action=logonly;

action=eval %o (use DBI; my $db = DBI->connect(“dbi:mysql:mailscanner:localhost:3306”, “SQL username“, “SQL password“); \
        my $query = $db->prepare(“insert into maillog set timestamp=CURRENT_TIMESTAMP(), date=NOW, time=NOW, \
        clientip=’$2’, hostname=‘balin.waakhond.net’, subject=’%s’, clientip=’$2’, spamblacklisted=1, isspam=1, ishighspam=1”); \
        $query->execute();)
	

	
		End of the SEC ruleset for MailWatch

Script to report bayesian scores

I want to keep a tab on the Bayesian scores.
So I wrote a perl script to scan my log file(s) and report how often each of them are hit.

Save the section below to /usr/local/bin/ in the file spamscores.pl or where ever you want to.
Use it for example like: spamscores.pl -l /var/log/maillog
Or on a weekly crontab entry:
1 6 * * 0 /usr/local/bin/spamscores.pl -l /var/log/maillog.1 | mail -s “SPAM scores report `date`” hugo@vanderkooij.org



#!/usr/bin/perl -w

#
	
		Report SPAM scores perl script
	


#

	
		Modules to use

use Getopt::Std;
	

	
		Sort keys

sub hashValueDescendingNum 
   $list$b} <=> $list$a};
   }
	

	
		Commandline options

getopt(‘ls’);
	

	die “Usage: $0 -l logfile\n”
   unless ( $opt_l );

	$logfile = $opt_l;

	open(LOG, “<$logfile”);

	while ($line = ) 
   chomp($line);
   if ($line =~ s/.*, autolearn=disabled, //) 
      $line =~ s/\)//;
      scores = split(/, /, $line);
      foreach my $score (scores) 
          ($name,$points) = split(/ /, $score);
           $list$name}++;
           $values$name} = $points;
         }
      }
   }

	if (keys %list) 
   print “\n * SPAM words ordered by alphabet:\n\n”;
   foreach $ThisOne (sort keys %list) 
      $fill = “ “ x (30 – length($ThisOne) – length($list$ThisOne}));
      print “ $ThisOne :” . $fill . $list$ThisOne} . “ : “ . $values$ThisOne} . “\n”;
      }

   print “\n * SPAM words ordered by value:\n\n”;
   foreach $ThisOne (sort hashValueDescendingNum keys %list) 
      $fill = “ “ x (30 – length($ThisOne) – length($list$ThisOne}));
      print “ $ThisOne :” . $fill . $list$ThisOne} . “ : “ . $values$ThisOne} . “\n”;
      }
   }

	close(LOG)

Script to automate GEOIP updates for MailWatch

I do not like to manually update things so I wrote a small script to update the MailWatch database with the latest intel from the GeoIP database.
The GeoIP database is provided as a courtesy by Maxmind.
As the database is updated only once a month there is no need to run this script more then once a month.
Anyone needing more frequent updates can go for the paid service of Maxmind.

The scipt itself:

#!/bin/sh

There and back again, A MailWatch script by Hugo van der Kooij

BACKAGAIN=`pwd`
THERE=”/var/tmp”

cd $THERE

rm -f GeoIPCountryCSV.zip
wget http://www.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
unzip GeoIPCountryCSV.zip
rm -f GeoIPCountryCSV.zip
echo “LOAD DATA INFILE ‘$THERE/GeoIPCountryWhois.csv’ INTO TABLE geoip_country FIELDS TERMINATED BY ‘,’ OPTIONALLY ENCLOSED BY ‘\”’ LINES TERMINATED BY ‘\n’;” | \ mysql mailscanner -u mailscanner-user -pmailscanner-password -v
rm -f GeoIPCountryWhois.csv

cd $BACKAGAIN

This script is intended only for MailWatch version 1!

Advertisement:
spam firewall

Medewerker Hugo van der Kooij
Categorie Linux, Verouderd / Obsolete

Reacties

Op dit artikel kan niet gereageerd worden.

Hugo's website