After some hard work we have finally gone beta with our service: RerouteMAIL. A service with which you can easily run a mailserver even if your ISP actively blocks port 25.
A while ago I posted about some wild idea's where you could go around the SMTP block of your ISP by running your mail through another (non blocked) mailserver and forward it to your mailserver which is running on a different port. Today, we go beta with a service which makes this possible for everyone!
The road from 'Proof of Concept' to 'Finished Product' was, as always, a bit longer than expected. First of all I switched from developing in PHP to Django (to expand my knowledge) and secondly, there are some pitfalls when you want to make this kind of service for more people than just yourself.
For the people who don't know what the fuzz is all about, a short rundown. There are ISP's in the world which believe that there are a lot of people who don't know what they are doing. So when those people run a mailserver chances are rather large that they don't configure their mailserver correctly, and they become an open-relay and thus, open for spammers to be used as spamming machine.
Although these ISP's are in their right to do so, we believe the clients (and especially the business clients) should have the chance to be able to run a mailserver if they completely understand the risks.
By default (and according to everyone and everything on the internet) a mailserver listens on port 25 for incoming mail messages. Because this is the default, ISP's block this port (incoming and/or outgoing).
Fortunately we have thought up a way around the ISP's who are actively blocking port 25.
As a lot of people (mostly unix administrators) know, you can run a mailserver on a different port than port 25, say port 26. However, the email world does not know this. So when a mailserver (for example the google mailserver) wants to mail to you, it tries to connect to port 25. And so you have to find a way to let the other mailserver think you are on port 26. But this can't be done easily and without your ISP.
It can be done via our new service: RerouteMAIL. The trick is that we listen on port 25. And then relay the incoming message to your mailserver running on port 26!
And the best part is, we have a fee free version which everyone can use, a hassle free registration, and no questions asked.
If you are interested or curious please have a look at: www.reroutemail.com or leave a comment on this blog!
You have an ISP which is just fabulous, great speeds, acceptable pricing and no download limits. Great! Now you can have your own linux/windows server right at home. Run a webserver, claim some domains, host a few other things (mysql, python, ruby, django, etc) and tinker around with everything you couldnt do at a hosting provider. But when you try to configure your mailserer the fun ends. Mail just doesn't get deliverd... Booo!!!
At first you think you configured your mailserver wrong. Maybe you didnt set up your MX record corectly for your domain or maybe, just maybe, your postfix config is wrong (what could go wrong there? :P). After some tinkering and frustrated tests you come to the annoying conclusion that it is your ISP who is nagging you. It appears they block the SMTP port (port 25) in their firewall, so nothing gets through to your server. Damn it!
So your first thought is to go around it. You are a great tech-buff and you think you can outsmart your ISP blocking a port in its firewall and find a workaround, so you think to yourself "why not run it on port 26 (or something else) instead??". A fine idea indeed. But there is one problem. SMTP servers don't know your mailserver is running on a different port. And it has to know!
Let me illustrate this a bit more.
When someone (user@gmail.com) typing a message and wishes to send you an email (user@somedomain.com), creates the mail, and clicks on send, the google mailserver first contacts the DNS server to request the MX record for the domain (somedomain.com). It receives a response in the form of an IP address.
Then the mailserver will connect to this IP addres on port 25. Why port 25? Because thats the way all SMTP server work. It does not know any better than to connect to port 25. Its the way SMTP works.
So, no dice.. Just using a different port is not enough (and No: tricking the world into thinking port 26 is the new port 25 aint gonna work).
After some thinking the solution came to me. If we can't trick the world into sending to port 25 and we can't go around the port block at the ISP. Why don't we run it before the ISP or at another ISP?
"Uh, but, aint that the most obvious solution?"
Yes ofcourse it is. Running your mailserver on a different location where port 25 is not blocked is the perfect solution. But it is not appropriate in this case indeed. You want to run you own mailserver at home so you have full control over it. Not at a colocated place, hosted by a hosting provider and what not.
But think one step further:
"If someone would be willing to run a mailserver outside of your ISP's influence, and forward all incoming mail which is supposed to be for your domain and forwards this to your machine which has actually running a SMTP server on a port other than port 25 and delivers the mail on that port... Wouldn't that solve the problem?"
And there you have it. The solution to the problem. This way you can run a mailserver yourself even when your ISP blocks port 25!
Now I've been searching around for someone who offers this kind of service, but was not succesful. So I thought to myself, why not create such a service yourself? And so from the need of a solution, there came a solution and maybe a solution which can help other people aswell!
Currently I have this problem solved in a proof-of-concept scenario where a friend of mine is running a SMTP server (on port 25) and he forwards all emails directly to my SMTP server running on port 25. Right now I'm looking into ways of providing this solution to other people.
Update: Recently we (Marc-David and I) have launched our service at http://www.reroutemail.com/
After a few weeks I decided to start reading "Getting Real" from 37signals. I bought the book when I read about it on a blog from Scott Berkun (Why Requirements Stink) .
I started in the morning and read it from cover to cover in a few hours.
The book does a fabulous job in creating a 'think small and easy' mind-set. And it does so without going in too much detail. Every chapter is relatively small and gets to the point quickly. After every chapter I got the feeling that I wanted to put the theory into practice and start right away. But moreover it made me wanna read the next chapter and find out more (which ultimately resulted into me reading it from cover to cover in one read).
As stated in the introduction, a lot of the principles and methods are not thought up by 37signals themselves. They borowed a few here and there and added their own sauce to create a 'methodology' which is very suitable for themselves. Although they state things as being a fact, they never force things on you. They just point out the way they do it and why it helped them to achiever their goals.
The book made me realise again how 'easy' it is to start a web application and do it right but without all the fuss of elaborate planning, specifications, etc, etc.
Another side-effect of reading the book is that I wanted to blog about it again, and it resulted in me writing 3 blog posts today. I just wanted to get it all out of my system, and get it out of there right away. True, there may be typo's and grammatical errors in some of the posts, but it is the web! When someone points them out they are realy easy to fix. Thats the way the Internet works! 
For a long time I was wondering if it wouldn't be nice if you could boot your Vista Bootable CD from a hard drive. Because hard disks are much faster in comparison to a DVD I figureed having the bootable DVD on a hard drive would shave off a few minutes of your installation time. Unfortunately I could not find any information on how to make this possible. Google just came back with results regarding Dual Boot setup's or having a full Vista install on an external hard drive or USB. Not the thing I was interested in. The sollution to my 'problem' illuded me.
Today I came across an article in the Personal Computer Magazine (PCM) issue of this month in which they wanted to boot the Vista Install DVD of a USB stick. Reading the article I found nothing that implied that it can only be used on a USB stick. So I grabed an old disk drive and fiddled around a bit.
After a numerous amount of reboots and trial&error I finally got it working! The Vista Installer booted from the disk drive. Although I have not completed a full install, I am quite sure it works.
The thing that annoyed me the most when booting from the DVD was that I had to wait quite some time for the white bar to fill, after which the actuall install environment showed up. Although it was faster, it wasn't as fast as I had hoped. I think it has to do with driver loading and lots of small files. My guess is that when you do a full install it you will notice a great increase in speed. The actual copying of files should go faster, although I have not yet tested this (maybe i'll test it in the future).
From the hindside the operation is quite simple. Maybe that is the reason I couldn't find it anywhere and havn't heard anybody trying it before.
However, a simple 'how-to' would probably in order, for the ones who where looking for this too
Things needed:
- A hard drive which has nothing on it (no partitions and stuff)
- A running Vista setup (i use 'diskpart' which is a part of vista and not of XP)
- A bootable Vista Install DVD (may be a slipstreamed disc. I used a Vista with SP1 slipstreamed)
Now the steps to follow:
- Open a command prompt and run diskpart: c:\> diskpart
- View the lists of disks and find the appropriate disk: list disk
- Select the disk you want to use: select disk 0
- Verify that you have selected the right disk: list disk (the one with the * is the selected disk)
- clear the disk of all information: clean
- create a partition: create partition
- select the created partition: select partition 1
- Make it active: active
- Assign a drive letter: assign
This is almost identical to what PCM had in its article. However it didnt work correct for me. The PCM stated you should format the drive using FAT32 (using "format fs=32" between step 8 and 9). However, Vista is not always able to format a drive in the fat32 file system (which was the case with me).
I found out that vista is not able to format drives larger than 32GB in Fat32 filesystem. You can force it on a command prompt ( format /FS:FAT32 X: ). But most people say that drives larger than 32gb will slow down due to fat32 overhead. What I did was the folowing:
- Open vista disk management:
- Select your disk
- Right click and select "shrink"
- Shrink the disk to something below 32GB. (because you havn't used your disk this should be seconds)
- Now right click it again and select 'format'
- Choose Fat32 as the file system and check "Quick Format"
Now you are done preparing your disk!
Don't close your command prompt. You're not done yet...
Assuming your DVD drive is "D:" and your hdd "E:", copy the files from the drive to your newly made disk using xcopy (or any of your favorite copying programs, like robocopy for example).
c:\> xcopy d:\*.* /e /f e:\
Now you are done and you can close your command prompt.
You can use the rest of your disk if you'd like. Just create an extra partition and format it the way you like.
To boot from the disk, reboot your system and enter your BIOS. In the BIOS you have to make sure the boot priority are right. Make sure it boots from your HDD and not from your DVD (remove the DVD from the drive helps too). And make sure it boots from the correct disk.
If you have done everything correct, you should see the system booting from your new "Vista Install Bootable Hard Disk". No more hours of waiting while installing your Vista!
Important Update (24-12-2008)
I was trying to install a fresh Vista 64 install on a machine which did not have a PATA controller and thus I was not able to boot from a DVD. So I was forced to use my own technique and install from a hard disk (yay).
I quickly ran into a problem which I hadn't noticed before because I did not complete a full install from HDD.
The problem lies within the Vista installer itself. When you boot from the HDD and enter the setup, chances are the installer starts to complain that it can not find a suitable CD/DVD/USB driver for the device where the installation source is located (duh, its a hard drive).
After hours of tinkering with RAID drivers (i didn't read it correctly and thought it was complaining about not finding the raid set) I finally noticed the problem. After some more google-ing I found a google cached item which explained that when you hold the CTRL key while booting the installer the setup will not be loaded and youll get a command prompt.
When on the command prompt, first go to the root ("cd \") then execute "bcdedit /createstore" and then enter "setup" and the setup will correctly start, and wont complain about the driver.