The long-awaited Postfix setup details

Right… I’ve been meaning to put these details up for a long time now, but I’ve got some free time (well made some) and am going to put it up for future reference.

The system allows for every kind of service via SMTP. It has capability of remote relaying (incoming and outgoing), relaying certain users on a domain to another MTA, hosting user, forwarding users, forwarding entire domains, multiple administrators, mail graphs, and probably some more I’ve forgotten. Also implemented is spam and virus checking along with dkim and dk.

First off I’m running Ubuntu (but any deb-based distro should be the same). Hopefully I’ve remembered all the packages I installed, if not please do correct this later.

sudo apt-get install courier-imap-ssl courier-pop-ssl courier-authlib-mysql postfix-mysql mailgraph dkim-filter dk-filter clamav-daemon clamav-freshclam spamassassin mysql-server amavisd-new libapache2-mod-php5 php5-mysql

That should be enough to pull all the dependencies anyway. Don’t worry if certain things don’t start up (like spamassassin) that’s because we will use them through the API rather than through a socket.

You should be sure to set your mysql root password (mysqladmin -u root password “newpassword”).

Then we need to setup the mysql tables. This is nice and easy, just download tables.conf and run “mysql -u root -p < tables.conf". Then we need to setup the postifx user and password in MySQL so open up the MySQL command line "mysql -u root -p" and type "GRANT SELECT ON mail_database.* TO [email protected] IDENTIFIED BY 'POSTFIX_PASSWORD'; GRANT SELECT, INSERT, DELETE, UPDATE ON mail_database.* TO [email protected] IDENTIFIED BY 'mail_adminpassw0rd';" (you might want to change POSTFIX_PASSWORD, but you will need to change it in the postfix files later (many times!). Then to set up postfix... To do this just wipe out your /etc/postfix/ and overwrite with the contents of postfix.tar.gz (“tar -xzvf postfix.tar.gz”). You need to edit a couple of things in these files, first of all if you changed POSTFIX_PASSWORD earlier you need to change it in anything that has mysql in its name and sasl/smtp.conf for user SMTP authentication. You should also add any hosts you wish to relay for in hosts.conf, this can either be an IP or a full hostname (DNS name). I’m not 100% sure if this is required, but I did it anyway: any relaying host IP is appended to my_networks in master.cf at the bottom. sh.list should have IPs of relaying hosts in it too for the domain keys implementation. You should then make sure you set the permissions correctly “sudo chown -R root:root /etc/postfix/”. In the domainkeys directory you need to create an RSA key pair in public.key and private.key. This is easy enough just run “sudo openssl genrsa -out /etc/postfix/domainkeys/private.key 768; sudo openssl rsa -in /etc/postfix/domainkeys/private.key -out /etc/postfix/domainkeys/public.key -pubout -outform PEM; sudo chmod 400 /etc/postfix/domainkeys/private.key”. Putting your public key in your DNS record is required next, but this is up to you.

Next you need to set up your domainkeys, just put dkim-filter.conf into your /etc/ directory, replacing your dkim-filter.conf and add all the domains you want to sign to the Domain line in it. Then you need to alter your /etc/default/dk-filter so it reads:

DAEMON_OPTS=”-l”
SOCKET=”inet:[email protected]

and your /etc/default/dkim-filter so it reads:

DAEMON_OPTS=”-l”
SOCKET=”inet:[email protected]

Now to setup your spam and virus filtering. Just open up (“sudo nano /etc/amavis/conf.d/15-content_filter_mode”) and uncomment the four lines which say bypass on them a lot. I did a few changes to 20-debian_defaults as well because I wanted more things to just black-hole than bounce, but that’s not necessary, so is undocumented. The man pages are rather good if you’re interested in doing that. Finally I added the following to /etc/amavis/conf.d/50-user:

$DO_SYSLOG = 1;
$log_level = 0;
$sa_tag_level_deflt = -999;
$sa_tag2_level_deflt = 4.5;
$sa_kill_level_deflt = 10;

Now to move onto courier. This is much easier. Extract courier.tar.gz into /etc/courier/ overwriting everything there. Then edit /etc/courier/authmysqlrc to change POSTFIX_PASSWORD to whatever you had it set as above. Then run the following to setup the SSL certificates. Make sure the CommonName is set to your server hostname.

cd /etc/courier/
sudo openssl genrsa -out mail.key
sudo chmod 400 mail.key
openssl req -new -nodes -key mail.key -out mail.csr

Now visit CAcert.org and send them your mail.csr. They will send you back a signed certificate, you need to save this as /etc/courier/mail.crt, then finish running the rest:

sudo cat mail.key mail.crt > /etc/courier/imapd.pem
sudo openssl gendh >> /etc/courier/imapd.pem
sudo cp /etc/courier/imapd.pem /etc/courier/pop3d.pem

That’s everything other than the admin console now… So for that we need to create a host, apache does this for us on install, so that was easy 🙂 Now just extract admin.tar.gz to /var/www/ and we’re done.

Now to restart all the services….

sudo /etc/init.d/apache2 restart
sudo /etc/init.d/courier-authdaemon restart
sudo /etc/init.d/courier-imap-ssl restart
sudo /etc/init.d/courier-imap restart
sudo /etc/init.d/courier-pop3-ssl restart
sudo /etc/init.d/courier-pop3 restart
sudo /etc/init.d/amavis restart
sudo /etc/init.d/postfix restart
sudo /etc/init.d/dk-filter restart
sudo /etc/init.d/dkim-filter restart

To configure you need to visit http://localhost/ in your browser. The default username is ‘admin’ and the default password is ‘password’, make sure you change this.

Service Reward Elicitation

The main issue when it comes to automatically determining whether or not to provide information to a service is what benefit it will have to the user. Currently there seems as if there is no definition language for this, and this is understandable due to the high complexity of the type of reward and levels of reward which may be provided. Another major factor as to why this language is difficult to implement is that it should be boolean, not scalar in its values otherwise definition of each continuous number would be required otherwise the system would not be fair and open to exploitation by services which wish to gain the information by “bending” the figures to suit.

The system I hope to develop to elicit these details to a system in order to provide automated sharing protocols will hopefully extend or be of similar kind to XDI in order for the adaptation of service and identity providers to be easier and therefore useful.

Vodafone

Last Wednesday I went to visit my partial sponsors of my PhD, Vodafone. I went to meet my industrial supervisor and try to tell them what it was I’ve been working on for the last three months. The journey down was fine until I hit Junction 9 of the M40 and had to queue for 30 minutes, next time I think I will get the train. Driving is 1h30m and the train is 2h, but with a queue like that it it really isn’t any different, and is much less hassle.

I had a very enjoyable time and spoke with many people there, and had fun talking about what it was like working at Vodafone with a masters student who was working there called Tom. After explaining what I had been doing it was suggested that I ought to look at working on a wider problem using distributed identity management. This is just an abstraction of my previous model where the user information is shared to an advertiser, in this case the advertisers turn into service providers which not only want to know things about you, but they also have information about you which has been gathered and they are able to share it with each other using your over-arching profile.

Obviously a user wouldn’t want services arbitrarily sharing personal information about themselves with each other, so the identity provider sits in the middle and determines what the information flow should be. The provider is able to perform this by analysing abstract user preferences provided and determining what the user will get out of providing this information to the service. There will be three outcomes from the analysis, either share the information, obfuscate the information or confirm with the user, which has the options to share, obfuscate or deny information. This user decision is then stored for future decisions (i.e. extend the users preferences). This system could be implemented by advertisers and therefore can be used as I was originally considering, but can also be extended for more useful user services.

Long weekend

Well, it’s been a terribly long weekend with much achived. Not much sleep though. Sunday consisted of a two hour train journey to London followed by my first meeting of the trip with Matt in order to catch up with his job and life on general. I made sure that we treked to the Australia shop for some goodies.

Following this the rest of the Birmingham entourage turned up, and after unsuccessfully trying to check into the hotel we went for our first meeting at Kings College (with all our luggage). Hot topic was search based software engineering and how to apply it to almost any problem. And as would be expected from a room full of searchers the puns got in everywhere. Kings were very hospitable with refreshments and cakes and biscuits.

In the evening we went to a Chineese rearurant to enjoy a variety of things that I struggled to pick up with chopsticks, but I refused to give in. Variety for none fish eaters was rather small, but sufficient. The plan was to meet up with Matt again, but this went on til 9:30, and since he had work in the morning he went to bed.

In the end about 10 of us went to The Hops! bar since we had a 25% voucher for the first round. This made a whole round under £20, very good for London. Bill and I decided to buy, Bill being of the very nice guys resident in CREST.

After one pint we got kicked out, it was late on a Sunday. I’d taken some DVDs and mine and Vivek’s phones were dead so we went to charge them at Ben’s room while watching Bill Baileys Tinselworm. My phone charger wasn’t working and after trying it on many phones and sockets this was confirmed, so we used a data cable. This didn’t work either, so I tried on my laptop and it was fine. I decided it was a driver issue on Vista, and it was. Apparently Vista won’t supply power to a USB device without a driver installed, or at least not enough to charge the phone. On the other hand Ubuntu did.

After enough charge to last the night and the first half of Tinselworm we retired to our rooms. I had to sneak my chair back which I’d borrowed since we didn’t have enough. I then enjoyed a very good shower and browsed the TV ending up watching the end of the first Austin Powers movie. Then I enjoyed the comfort of the Strand Palace Hotel beds.

I’d set my alarm for 7:30 in order for breakfast, but the hotel comes to life in the morning and I woke at 7:10, amazing as I’m a heavy sleeper. We met up and had a very nice breakfast, if only the couassants were warm.

Following this we had another SEBASE meeting, a full day. This had the same wonderful resfreshments and muffins, and was interrupted by a buffet lunch in the luxurious River Room, the food was comprable to Staff House at Birmingham.

Following this meeting I had arranged dinner with another Vodafone PhD student from Imperial. He is Itallian so we went to a very nice Italian in South Kensington. This meeting was quite useful in understanding how our work links, he was also a very interesting guy.

Finally I headed home, I managed to finish watching the Royal Institution Christmas lectures on my iPod. These are aimed at children but Chris is a very good lecturer and manage to explain very complex things like key exchange in simple terms.

After a little sleep I’m back at Uni and in meetings again, followed by the tediom of VIVAs, sorry Demonstrations.

Start of the week

So, the new president is taking up office today, I wonder if he will make a difference, and a positive one at that. No doubt any speeches will be gaff-free, but will the head figure change everyone else. One thing’s for sure, he won’t be anywhere near as funny. Who doesn’t like a good laugh at an American hic.

Last night I played C&C3 online with Matt, we were totally whooped by NOD, more practice required I think.

In other news I had brilliant fun registering a domain before another Matt at Uni. Apparently they don’t lock them until you’ve paid. I decided to transfer it back to him because I’m kind and it’s free to do so on ukreg between users. We later found changing the registrant is £10 + VAT so I’m keeping it in my name on the documents. I think it’s hilarious, as does everyone else, but it burns Matt up inside to read the Whois data. This however just makes it funnier.

Research Status – how do you know?

Well, it’s another cold morning waiting at the train station. I’ve read the poems on the windows in the waiting room many times, but they are still interesting. Does that define good poetry or boredem?

I’ve made some progress as far as research goes. It seems my novel auction idea for advertising isn’t so novel as Google have two paper regarding it. They are using linear techniques and are not privacy preserving, so I still have something. Whether I can improve speed, privacy or both is the question.

My visit to Vodafone is on Wednesday and therefore I may know what they are interested in. It took them quite a while to invite me down, but now they have paid me it seems I’ve caught their attention.

Also, I’ve emailed one of the researchers at Google for their input, I hope they get round to replying.

Finally, I have a presentation to a software engineering group (SEBASE) on Monday, I’ve something of a talk for them, but after Wednesday or the talks I attend on Sunday it may need some revision.

Lack of time last night prevented me from finishing an interesting auction paper from Google (mentioned a lot today aren’t they). Hopefully I can finish this and write my Vodafone paper today.

Blog Origins

I thought I might mention the origins of this blog and how it became to be published. This is not my reasoning for creating a blog (those are that I was copying Kat and Vivek, my friends who are also first year PhDs at Birmingham), but for what I did to put it on the internet.

Choosing my software

So I decided that I ought to do this first, as it might change my implementation specifics. Where do you start? Google seemed the obvious answer, so I decided to Google for the best blogs around. There doesn’t seem to be a comparison of these, or at least my search for one was inadequate. Therefore I decided that the one that just looked quite good would suffice. I found beeblog, which is now not under development and then I found Serendipity. I like PHP and the ability to add plugins from a large repository sounds good.

Installation

I’ve got a virtual server out there in the world somewhere, I don’t care where… it just works. Currently this just forwards email around so I decided to use it for hosting my blog rather than keeping it on my home based server, I don’t want you all to use up my bandwidth. I had Apache, PHP and MySQL installed already so all I needed was Imagemagik and GDLib. My distribution of choice is Ubuntu so an “apt-get install imagemagik php5-gd” quickly solved those dependancies. Then I loaded the index page, this asked me a few simple configuration questions which weren’t too hard, although I did have to SSH in and “CREATE DATABASE serendipity” and “GRANT ALL on serendipity.* to [email protected] IDENTIFIED BY “wouldn’t you like to know””.

Domains

How should I publish my blog (I thought)? I considered registering a domain, and nothing really came to mind, then I remembered I had a few domains that weren’t doing anything. There’s negativezero.co.uk, I like this name, but I have some stuff on there already. Next I had cyntopic.com, this sounds a bit like sin-topic which would kind of describe the page. In the end I just opted for christaite.com as it is me and therefore the easiest to tell people, also it’s just a blog about me so it’s best I think.

Plugins

There are lots of plugins as I mentioned earlier. I installed a few, notably one for mobile viewing (Markup: Mobile Output). I have an iPod Touch and it specifically mentioned it on the description. After installation the site worked fine, but I tried to view the page on my iPod and was confronted with a PHP error message: “Invalid argument supplied for foreach() in serendipity_event_mobile_output.php on line 308”. Being a keen PHP programmer I have attempted to solve this bug myself (while I’m writing this post).

The problem is that I don’t have any categories defined on my site as yet. This was easily solved by adding if (is_array($categories)) before the foreach on line 308. But it still does not seem to work at all, I think this may have something to do with setting the style template, but I’m not sure… I’ll report back if I ever find out any more.

Start of a blog

Right, well this is the first post. I should have started one of these a long time a go to keep track of progress and details which I would have otherwise forgotten and have done so.

The main aim of this blog is to keep a record of all my work performed while I am learning how to research (i.e. take a PhD), but no doubt I will go off on rants about all sorts of other things which I have read in the Metro. My mind tends to wander a lot, and as such I think it’s about time I wrote much of it down so that I can refer back to it. There will also be some how-to’s posted for Linux based things (projects I’m hacking away at) so when the machines die I have a way of recovering the data. This seems a very good idea, but the machine that I’m writing this blog on is not backed up in any form and therefore is just as likely to die as any other.

Today I have been in a healthy mindset with my lunch, this is mainly due to the lack of bread in the fridge, so I spent a whole half and hour cooking and cooling pasta before mixing Tesco own tomato pasta sauce, and this time for a little variation I added cheese with a half price Muller (cherry) yoghurt for desert. Also, Dairylea Dunkers are on special offer again and therefore I’ve had one of those for breakfast.

While shopping I came across an offer 3 chocolate fondant filled eggs for £1.09, with Mars, Galaxy Caramel and Dairy Milk Caramel next to each other. Being a huge fan of caramel I had to get one of each. My findings so far are as follows: the Mars eggs have the nougat and caramel as with the bar, but both are significantly more viscous and do not taste the same at all, still enjoyable. The Galaxy Caramel is almost identical to that of the bar, but the chocolate is not as smooth. The dairy milk I have not eaten yet, but if it is like last years they will not be able to contend with the taste of the chocolate-mixed caramel in the Galaxy egg, but wins points for being the original.

So far as work goes, I am making little progress today since I have been setting up this. I have updated my timetable and organised my trip to London to visit SEBASE. This is a bit of an organisational nightmare as people keep on coming up with different ideas about when we are leaving and how long we are staying for, eventually we have been granted the money to stay in a Hotel on Sunday night for the meeting on Monday morning. I have to prepare a five minute presentation and try not to make a complete arse of myself in front of people who actually know what they are talking about. Rami has given me some pointers as to some papers on Search Based Software Engineering (“The Current State and Future of Search Based Software Engineering” by Mark Harman), the author of which will be attending the meeting. There has been a whole papers worth of jottings about how I can adapt search based techniques towards contextual based advertising.

My visit to Vodafone is all arranged for Wednesday now, driving down to Newbury in the morning and back in the afternoon. It seems a long way to go for a few hours, but the previous conference call was a complete waste of time so hopefully I will be able to get some interesting information from them. I’m looking to find out what Vodafone actually want from me, since they are paying for some of my time. I hope to gain insight into what kinds of information they currently gather and whether my ideas of offsetting user privacy for reward will be of any interest to them what-so-ever.

Later you may look forward to my how-to make a postfix mailserver with all the trimmings.