The main technologies are reverse DNS checking, SPF, SenderID, Domainkeys and DKIM. I will discuss all of these here and provide my tips on setting up SPF, SenderID and DKIM. Please keep in mind that I am not an expert on email servers – but I hope this helps someone! It took me about half a day to figure this all out, so it is probably worth doing to improve your email delivery rates.

This blog entry from Dave Zohrom provides a nice discussion of sending email to a general audience (part 2 of a 3-post series).

Introduction

SPF or Sender Policy Framework

SPF is easy to setup. It uses DNS TXT records to allow the email providers to check which servers are authorized to send email for a particular domain. The SPF project has done a great job at making this simple to setup. Just go to their homepage and use the setup wizard to generate the appropriate TXT records (the text field underneath “Deploying SPF”). Then change the values in your DNS records.

One thing to note is that if you have multiple servers and send email from a server with a different hostname (e.g. “mail.example.com”), you need to setup a record for that server as well. See the common mistakes page on the SPF site. Also, if you have hostnames that are not supposed to send mail, you ought to indicate this as well.

Another thing is that if you are using Google Apps for Your Domain to send email, then you will need to have to add “include:aspmx.googlemail.com” to also allow mail from Google to validate. See this answer for more.

SenderID from Microsoft

SenderID is a variant of SPF, which for most practical cases is the same as SPF. Just setup SPF and this should produce a validation pass for SenderID as well. The semantics of the validation are a bit different,  but this does not seem to be a major practical problem. See testing tips below to make sure that this is also true in your case.

DomainKeys

DomainKeys is an older version of DKIM (DomainKeys Identified Mail) developed by Yahoo. Despite having very similar names, these ARE NOT the same!

Both DomainKeys and DKIM store public key information in DNS records and sign the message headers of every email sent. The recipient can then verify the signature.

DomainKeys was deprecated in 2007, but some email providers may still be using it. However, these are a shrinking minority and Yahoo does support the newer DKIM. Because of this I did not add DomainKeys support but opted only to use DKIM.

You can add it to sendmail or Postfix using the dk-milter project code, but the unofficial RPM release is not maintained anymore, which means you will need to install it from source (available via SourceForge).

DKIM, or DomainKeys Identified Mail

DKIM on the other hand seems to be gaining momentum. It is used by Gmail, Yahoo and AOL and many others, and also works by publishing public keys via DNS TXT records and by signing the emails at the email server.

To setup DKIM, you need an additional filter which takes the completed email and adds the DKIM signature to the email prior to sending it out.

Postfix and sendmail support “milters“, which is apparently short for “mail filter”. There is a DKIM-milter package available for Centos at the EPEL repositories (see Centos page for 3rd party repos).

Setting up DKIM-milter with sendmail

For Postfix, use these instructions from All About LAMP. If you want to use sendmail as I did, here are my additional tips:

Steps 1-7 as in the linked tutorial.

Step 8. Configure dkim-milter

Open configuration file /etc/mail/dkim-milter/dkim-filter.conf and use the following configuration:

Canonicalization simple
Domain example.com
KeyFile /some/path/to/whatever-your-keyfile-was
Selector name-of-the-selector
SignatureAlgorithm rsa-sha256
Socket inet:8891@localhost
Syslog Yes
Userid dkim-milter

NOTE: you will be configuring dkim-milter to use the loopback interface instead of a socket file. I was unable to get dkim-milter to work via the socket file with sendmail. If you get it working, let me know.

You may also want to setup the following:

SubDomains Yes
SyslogSuccess Yes
X-Header Yes

The X-Header and Syslog options are useful for debugging. See the config file, each option should be documented there.

Step 9. Change the default init.d script to use the loopback interface

The default init script uses a socket, this needs to be changed. Open /etc/init.d/dkim-milter and change/comment the line:

SOCKET=local:/var/run/${name}/${name}.sock

to:

SOCKET=inet:8891@localhost

Step 10. Configure sendmail to use dkim-milter

First a few reminders about sendmail configuration, remember that:

1. Sendmail comments DO NOT USE # as the comment, instead “dnl”  (delete through newline) at beginning of the line is used to comment lines out.
2. Sendmail configuration is built from the *.mc script files using the M4 macro processor.
3. You need to install the sendmail-cf package for dependencies and install the m4 macro processor separately.
4. In the configuration, the opening quote is a grave accent ` and the closing quote is a straight quote ‘.

To configure sendmail, open the “/etc/mail/submit.mc” file (which contains the settings for message sending; in older sendmail versions this config was in sendmail.mc).

10.1 Edit submit.mc by adding the following entry to it:

INPUT_MAIL_FILTER(`dk-filter', `S=inet:8891@localhost')dnl

(for example just before “FEATURE(`msp’, `[127.0.0.1]‘)dnl”). Make sure that there are no define(`confINPUT_MAIL_FILTERS’, `…’)dnl lines after this; if there are, you will need to add dkim-milter manually to the INPUT_MAIL_FILTERS list.

10.2 Build and install a new submit.cf:

m4 /etc/mail/submit.mc > submit.cf

Tip: use m4 -d /etc/mail/submit.mc to debug first.

10.3 Restart sendmail

service sendmail restart

Testing tips

Some possible errors:

1. Errors getting the sendmail configuration generated: Check whether the dependencies (m4 and sendmail-cf) are installed and paths are correct.
2. Errors starting sendmail: Make sure you replaced the correct files (submit.mc to submit.cf and sendmail.mc to sendmail.cf) and if you modified sendmail.mc in addition to submit.mc, make sure you have regenerated both.
3. Errors starting dkim-milter: check the permissions on the key file
4. Sendmail seems to ignore the dkim-milter, no X-dkim-milter header is in the mail even after you enabled the X-Header option in dkim-filter.conf:
   1. Check /var/log/maillog.
   2. Sendmail seems to default to ignoring the milter if it cannot connect to it. This was the problem I ran into when using sockets; after switching to the loopback interface everything started working.
   3. Also, check whether your mail is sent from a recipient for whom mail is supposed to be signed! If you haven’t setup your hostname, then this may lead to email not being signed (eg. hostname -F hostname plus /etc/hosts plus /etc/sysconfig/network).  See, for example this tutorial for configuring sendmail.
   4. Also, check if you need to configure masquerading for Sendmail, see http://www.sendmail.org/m4/masquerading_relaying.html

Sending email from the console using only sendmail

Create a file with the following content:

To: "Recipient name" <john.doe@example.com>
From: "Sender name" <admin@example.com>
Reply-To: admin@example.com
Subject: Hello world
This is the content of the message, end it with a line containing only a period as sendmail expects this.
.

Then cat the file and pipe to sendmail -t:

cat message.txt |sendmail -t

Checking all of the technologies mentioned above

Port25.com offers a free service which check SPF, SenderID, DomainKeys and DKIM: http://port25.com/domainkeys/

Quote:

• If you wish to receive the results at the address in the “mail_from,” the sample message should be sent to check-auth@verifier.port25.com.
• If you wish to receive the results at the address in the “from” header, the sample message should be sent to check-auth2@verifier.port25.com.

A reply email will be sent back to you with an analysis of the message’s authentication status. The report will perform the following checks: SPF, SenderID, DomainKeys, DKIM and SpamAssassin.

Additional resources:

How to Setup DKIM for Postfix in Fedora using dkim-milter

How to Prevent Web Server Emails from being Marked as SPAM

DomainKey Implementor’s Tools and Library for email servers & clients

Sendmail DKIM

Setting up DKIM, SPF, Domainkeys DNS, Regular DNS on CentOS 5.3 at Pacificrack.com

How to manually install DKIM-Filter with Sendmail

Over the years I’ve read quite a pile of books particularly related to programming, general business, entrepreneurship and software engineering. Now that I am reading up on new topics such as organizational development, lean methods and statistical research methods I am having real trouble finding books I would consider reading just based on the table of contents and Amazon reviews. This is probably a good thing, since the most efficient way to get information is to only read good books.

Here are a couple of guidelines for picking good non-fiction books:

1. Number of reviews. If there is only a small number of reviews, there is a risk that those reviews are written by the author or his/her friends. A book with more reviews is more credible, as is a book which has had more editions.
2. Distribution of reviews. If the reviews are split between 5-star and 1-star reviews, the 5-star reviews are most likely fake. This is an absolutely disgusting practice, but unfortunately common.
3. Credentials of the authors. If the authors are practitioners and not scientists, make sure they do not work for a consultancy. I have unfortunately 1-2 books that I did not check adequately and ended up getting a useless brochure book.
4. The table of contents.
   1. What is the starting point of the discussion? This defines the prerequisite knowledge that you are expected to have. Reading a book which is for beginners is incredible boring, while reading an advanced book as a beginner is frustrating.Evaluate whether you have the prerequisite knowledge and could read the first few chapters without getting bored to tears. Remember that most popular business books are written for an audience that reads just a few books per year – this pretty much makes them too repetitive if you read more than that.
   2. What is the emphasis of the book? Is it oriented towards: a) practice or b) checklists, c) proper theory or d) undergraduate students textbook? (See below for more)
   3. How many pages are dedicated towards each topic? Don’t expect anything good from a book that has a 200 pages and cover 200 topics: that’s just one tiny page per topic.Good books don’t cover everything with the same small amount of pages, because everything is not equally important in real life.
5. The actual content of the reviews. Don’t bother with 5- or 4-star reviews, they will just tell you the book is good. Read the 1-star and 3-star reviews.See if the 1-star reviews seem to be justified or are just a result of the reviewer being an idiot (e.g. “Haven’t received book from Amazon” or “This book is too difficult” -reviews).3-star reviews are usually the best, since the author did like some aspects but did not like others. Pay attention to reviews that say that a book is too basic or simple. If this is said more than once, you probably don’t want to read that book – American writing in particular is already very verbose and too many books are written where a five-page article would do, hence reviews which state that the book is too simple should be taken very seriously.
6. Number of and affiliation of authors. If all the authors are from the same institution or organization, the book is probably of lower quality. A huge warning sign is a “collection” book that has multiple chapters on different topics by the same authors, all from the same decade. This indicates that the authors have not bothered to contact the people who are at the top of their respective fields; articles which are all from the same decade are likely to be just a random set of articles which were written for the book rather than the result of good editorial work.
7. Order of topics. Scientific collection books usually order the chapters so that the better and more widely applicable material is at the front of the book, and the more specialized and less interesting material is at the end. It is safe to assume that if the chapter that seems decent for your purpose is at the end of the book, you are better off finding a more specialized book.

Four classes of books:

I would put books in four broad categories, two of which are useful and two are useless:

1. Practical books
2. Academic books
3. Ego-booster books
4. Motivational books

Practical books: experience-based vs checklist books

Practical books are in my opinion divisible into two categories: advice from practitioners and advice from people who like to write checklists. That is, you have books that are based on the experiences of a practitioner, and then you have books that are written by people who have not actually done that much but wanted to write a book and ended up writing a checklist – a list of things that might be of use without really emphasizing what is important.

Try to get a feel of the book from the preview (if you can’t, don’t buy it!), search for a couple of phrases that interest you. Does the author know what they are doing, or are they just collecting information from other people? Sometimes a “checklist book” is not bad – when you just need the basics and can’t be bothered to collect all the information. However, usually the experience-based book is better.

Academic books: theory vs textbook

When it comes to academic books, the two main types are proper theory books and textbooks. The more I have seen and read textbooks, the less I like them. Generally speaking they try to cover everything and end up saying nothing.

If you care about the topic, don’t get a textbook. Instead, get a proper collection of influential articles if it exists (ex. Oxford Handbook of X). You will perhaps not be fully prepared for everything you read, but you will learn something useful. Unlike after reading a textbook, where you just have fuzzy awareness of the fact that “all kinds of things happen” in a particular field.

Ego-booster books: academic and practical

There is also another category of books which are basically books written for the author’s ego. Some books are not written to be read, they are written either to impress others or to act as marketing tools.

The academic substitute for having a nice car sometimes seems to be writing an obtuse treatise which is downright hostile towards the reader, while consultancies in particular are guilty of writing books in which the answer to every question is “hire our consultancy”. The academic ego book is harder to spot, so pay attention when previewing the book.

Motivational books: feeling good doesn’t make it so

Finally, there is a category of fiction books which pretend to be non-fiction books. The motivational books may be a fun read at times, and are usually the most popular books in their category.

However, they are fiction for your entertainment. Feeling good is nice, but that does not change the fact that there are people who are knowledgeable, talented and well-connected doing whatever the book is about, and that feeling good does not make you knowledgeable, talented or well-connected.

0. Preliminary setup: make sure each computer gets a constant IP address

Before setting up the DNS server, you need to ensure that each computer or virtual machine gets a fixed IP address. Otherwise the IP address of the computer may change on each reboot.

This should be done on your router, which is usually located either at 192.168.0.1 or 192.168.1.1 (check your current local IP address by running the Command Prompt and “ipconfig”, which shows your current IP address). Check your router manual on instructions how to log in, most routers have a HTTP-based configuration system which can be accessed when connected to the router.

In my case (I have a Asus RT-N11), the address was 192.168.1.1. Make sure that the computers you want to setup IPs for are connected, then find the “Status” or “DHCP Leases” listing on the router web interface. This listing will contain the MAC addresses for each host. Here is mine:

Host Name       MAC Address       IP Address      Lease
--------------------------------------------------------------
HOST1        00-11-22-33-F0-AC 192.168.1.5     60016 secs.
HOST2        00-11-33-44-F2-2C 192.168.1.6     85074 secs.

A MAC address is a unique identifier given to each network adapter. It allows you to setup fixed IPs.

Find the router functionality which allows you to “Assign IP Addresses Manually”. This should enable you to specify a MAC address and a corresponding fixed IP address. Do this for each of the network adapters you wish to have a fixed IP address. On the Asus RT-N11 this was under “IP Config” -> “DHCP Server”.

Add a MAC address <-> IP address pair for each computer.

1. Get MaraDNS

MaraDNS is a free, lightweight and relatively easy-to-configure DNS server for Windows and Linux. Download it from here and unzip it to some folder.

2. Configure MaraDNS

Open “secret.txt” and change the value to something else (random characters).

The MaraDNS configuration is in the “mararc” file in the same directory. DNS servers have two sets of functionality. They can function as a “Authoritative name server” or a “Recursive/caching name server”.

Authoritative name servers specify IP addresses for domain names. Recursive name servers store information from authoritative name servers and pass on queries in a recursive manner.

We will be configuring both authoritative and recursive functionality in MaraDNS.

2.1 Authoritative configuration

We will configure the server to provide authoritative names of the LAN domain names. Pick any domain, I chose “local.com” (note though that you will not be able to access the actual “local.com” website if you pick an existing domain name).

Add configuration lines to “mararc” like these:

csv2 = {}
csv2["local.com."] = "db.lan.txt"

Where local.com is the domain name you picked, and db.lan.txt is the name of the second configuration file which we will be creating next (change it if you want to name the second configuration file).

Create a new file named “db.lan.txt” in the same directory as MaraDNS.

For each of the computers you want to resolve to a name, add a line to “db.lan.txt”. For example, for two machines, one “dev.local.com” and the other “blog.local.com”, add the following lines:

dev.%       192.168.1.4 ~
blog.%        192.168.1.6 ~

Done!

2.2 Recursive configuration

We will setup MaraDNS to ask your default name servers for all other domains so that you can resolve all other domain names to their correct IP addresses.

Find out your ISP’s DNS server addresses. These are likely to be listed either on the Router status page, or by checking the details on your network adapter.

Now add your ISP’s DNS servers as upstream servers in “mararc”:

upstream_servers = {}
upstream_servers["."] = "xxx.xxx.xxx.xxx, yyy.yyy.yyy.yyy"

Where xxx.xxx.xxx.xxx and yyy.yyy.yyy.yyy are your ISP’s DNS servers.

Done!

3. Run MaraDNS and test it using askmara.exe

Double-click “runmara.bat” , and leave the server running.

Open a command prompt, navigate to the MaraDNS directory and try running:

askmara.exe Agoogle.com.

and

askmara.exe Ablog.local.com.

You should get replies like this:

# Querying the server with the IP 127.0.0.1
# Question: Agoogle.com.
google.com. +300 a 74.125.67.100
google.com. +300 a 74.125.53.100
google.com. +300 a 74.125.45.100
# NS replies:
# AR replies:

and:

# Querying the server with the IP 127.0.0.1
# Question: Ablog.local.com.
blog.local.com. +86400 a 192.168.1.6
# NS replies:
#local.com. +86400 ns synth-ip-7f000001.local.com.
# AR replies:
#synth-ip-7f000001.local.com. +86400 a 127.0.0.1

If you get problems with the first query, you messed up the recursive DNS settings (are your ISP DNS server addresses correct?), and if you get an error with the second query, you messed up the authoritative settings.

4. Change MaraDNS to reply to queries from your LAN

Shutdown the MaraDNS window, and change the first two lines of “mararc” to something like:

ipv4_bind_addresses = "192.168.1.2
recursive_acl = "192.168.1.0/24"

Where 192.168.1.2 is the IP address of the computer on which the server will be running and the “192.168.1″ part of recursive_acl is the same as on your network (might be 192.168.0.0/24).

Start MaraDNS again, and leave it running.

5. Setup your router to hand out your new DNS server

Open your router’s web interface and find the DHCP server settings. There should be an option to set up a DNS server. Write the IP address of the computer on which the DNS server will be running.

For each of your computers, disconnect the network (e.g. by disabling and enabling it in Windows, or by using “ifconfig eth0 down”/”ifconfig eth0 up” on Linux).

That’s it, you should now be able to refer to your LAN computers by their domain names.

1. To ignore a directory, use -i path/relative/to/the/src/root/ with a “/” at the end (or “\” for Windows). You MUST have that trailing slash, otherwise the directive is treated differently (filename match)
2. To ignore multiple directories, you cannot use multiple -i -directives. If you do, they will overwrite one another, and only the last one will be applied. Instead, use a comma to separate the paths: -i first/path/,second/path/
3. If you happen to use 1.4.3 and use the ”HTML:frames:earthli” template, you will notice that the images and css files won’t load because they are missing one character at the end for some reason. Solution: rename or copy the template from a previous version to “./PhpDocumentor/Converters/HTML/frames/earthli”.

Really, this is simple but the documentation is rather unclear as to how this works (particularly the point about the significance of the last slash character). I found some comments in the bug tracker which helped me find the correct syntax (here). Both of these things are implied in documentation, but I figure writing this might save someone else some time.

There is an article that could be written on the information, but I can’t be bothered. View the full-size version here.

1. Get the dependencies and tools

yum install wget
yum install python mercurial trac trac-mercurial
yum install python-devel python-setuptools python-genshi python-docutils python-pygments mod_python


2. Get easy_install to work on FC10

wget http://peak.telecommunity.com/dist/ez_setup.py

Here is the first (minor) pitfall. You need to force the update of easy_install. If you only run “python ez_setup.py”, then you will not get the easy_install command (bash will not recognize it). Run:

python ez_setup.py -U setuptools

3. Install the mercurial plugin

From: http://trac.edgewall.org/wiki/MultipleRepositorySupport

Check out the mercurial plugin:
svn co http://svn.edgewall.com/repos/trac/sandbox/mercurial-plugin-0.12

Build the mercurial plugin. From within the checked-out directory, build the egg file (it will end up in the dist subdirectory):

python setup.py bdist_egg

Check out the multirepos branch:

svn co http://svn.edgewall.com/repos/trac/sandbox/multirepos

Install the multirepos version of trac (from within the multirepos directory, as root):

python setup.py install

4. Get the development branch of Genshi

Here you will run into the error “No local packages or download links found for Genshi>=0.6dev-r960″. To fix this, you need to force an update of Genshi to the latest dev version:

easy_install -U "Genshi==dev"

5. Create your trac environment directory

For example:

trac-admin /path/to/trac/myproject initenv
chown -R apache:apache /path/to/trac/myproject

6. Setup Apache for Trac

nano etc/httpd/conf/httpd.conf

Add something like:

<Location "/trac">
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnvParentDir /path/to/trac
PythonOption TracUriRoot /trac
</Location>


And if you haven’t added mod_python already:

LoadModule python_module modules/mod_python.so

7. Continue with the setup

From: http://trac.edgewall.org/wiki/MultipleRepositorySupport

Copy the mercurial plugin egg file to the <trac-env>/plugins directory. Make sure the following is somewhere in <trac-env>/conf/trac.ini, adding all the desired repositories to the [repositories] section:
[components]
tracext.hg.* = enabled[repositories]
my_repo_a.dir = path_to_a_repository

8. Restart apache and test

service httpd restart

Navigate to http://localhost/trac .

Some possible errors

If you get something like:

Available Projects

    * attachments: Error
      ([Errno 2] No such file or directory:
'/path/to/trac/env/attachments/VERSION')

Check your TracEnvParentDir in httpd.conf, it is probably incorrect. (And restart apache.)

If you get the error:

"Unsupported version control system "hg": Can't find an appropriate component, maybe the corresponding plugin was not enabled? "

Check if you copied the TracMercurial-0.12.0.6dev_r7902-py2.5.egg file to the correct location. (And restart apache.)

References:

http://trac.edgewall.org/wiki/TracOnFedoraCore

http://trac.edgewall.org/wiki/TracMercurial

http://trac.edgewall.org/wiki/TracModPython

http://trac.edgewall.org/wiki/MultipleRepositorySupport

http://www.smallroomsoftware.com/articles/2008/2/14/trac-0-11-installation-on-centos-fedora

Amazon offers an RSS feed of your reviews under your profile, see the link in the “Public Reviews Written by You” heading on “Your Profile”. What we will do is use this RSS to create a new site.

Get Simplepie (and CodeIgniter, if you want to)

I am going to set up the site using the CodeIgniter MVC framework and the SimplePie RSS reader library. You don’t really need to use CodeIgniter, but I wanted to give it a try. First download SimplePie and move the simplepie.inc file under /application/libraries and rename it to “simplepie.php”. Set up CodeIgniter as instructed in the user guide.

Controller code

In your controller:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

  function index($start = 0, $length = 5)
  {
      $this->load->library('simplepie');
      $this->simplepie->set_cache_location('./system/cache');
 
      // amazon uses class and id attributes in their review, which are useful for styling the content inside CDATA - so allow those two, and the rest is default.
      $attribs = $this->simplepie->strip_attributes;
      // unset class and id
      if (($key = array_search('class', $attribs)) !== null) {
          unset($attribs[$key]);
      }
      if (($key = array_search('id', $attribs)) !== null) {
          unset($attribs[$key]);
      }
      $this->simplepie->strip_attributes($attribs);
      // change this to your Amazon book review RSS feed address
      $this->simplepie->set_feed_url('http://www.amazon.com/rss/people/A31F72N3CBFDG2/reviews/ref=cm_rss_member_rev_manlink');
      $this->simplepie->init();
      $data['feed'] = $this->simplepie;
      $data['start'] = $start;
      $data['length'] = $length;
      $this->load->view('show_rss', $data);
  }

View code

After that, create the view (show_rss in my case). Create code to process the feed. I used the pagination code from the SimplePie documentation as the basis so that you can have separate pages for the items:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

  foreach ($feed->get_items($start + 1, $length) as $item) {
      $feed2 = $item->get_feed();
      echo '<div class="latest-post"> 
            <h1 class="title"><a href="' . $item->get_permalink() . '">' . $item->get_title(true) . '</a></h1> 
            <p class="meta">Source: <a href="' . $feed2->get_permalink() . '">' . $feed2->get_title() . '</a> | ' . $item->get_date('j M Y, g:i a') . '</p> 
            <div class="entry"> 
              <p>' . $item->get_content() . '</p>
              <span class="comments">' . $item->get_date('j M Y, g:i a') . '</span>
            </div>
          </div>';
  }
 
  // Let's do our paging controls
  $next = (int)$start + (int)$length;
  $prev = (int)$start - (int)$length;
 
  // Create the NEXT link
  $nextlink = '<a href="' . site_url(array('welcome', 'index', 'start' => $next, 'length' => $length)) . '">Next &raquo;</a>';
  if ($next > $max) {
      $nextlink = 'Next &raquo;';
  }
 
  // Create the PREVIOUS link
  $prevlink = '<a href="' . site_url(array('welcome', 'index', 'start' => $prev, 'length' => $length)) . '">&laquo; Previous</a>';
  if ($prev < 0 && (int)$start > 0) {
      $prevlink = '<a href="' . site_url(array('welcome', 'index', 'start' => $next, 'length' => $length)) . '">&laquo; Previous</a>';
  } elseif ($prev < 0) {
      $prevlink = '&laquo; Previous';
  }
 
  // Normalize the numbering for humans
  $begin = (int)$start + 1;
  $end = ($next > $max) ? $max : $next;
 
  echo '<div class="latest-post"> 
  <p>Showing ' . $begin . '&ndash;' . $end . ' out of ' . $max . ' | ' . $prevlink . ' | ' . $nextlink . ' | <a href="' . site_url(array('welcome', 'index', 'start' => $start, 'length' => '5')) . '">5</a>, <a href="' . site_url(array('welcome', 'index', 'start' => $start, 'length' => '10')) . '">10</a>, or <a href="' . site_url(array('welcome', 'index', 'start' => $start, 'length' => '20')) . '">20</a> at a time.</p>
  </div>';

CSS stylesheet

Finally, add a stylesheet. I used one from Free CSS Templates. I also made the following minor tweaks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

 
span.amzRssByline {
  /* Adds newline before the by line in reviews */
  display: block;
}
 
div.entry img {
  /* Floats the review picture */
 float: left;
 display: block;
 
}
div.entry table div {
  /* Adds a margin after the review heading */
 margin-bottom: 10px;
}

Here is the result: http://books.mixu.net/

Update 2 (Oct 2009): Since this is reasonably popular content, here are some additional findings:

First, Flash does not always use the same settings as your browser – try the proxy settings in Internet Explorer/Chrome. These seem to work better. The vast majority of problems are a result of Flash using different settings than your browser, revealing your actual IP to Hulu.

Second, if this does not work you can always use Proxifier (commercial product with 30 days trial), a TCP wrapper which forces all connections on the computer to use a particular proxy. This can be used in conjunction with a Hulu downloader, such as Hulu Video Downloader to download the videos.

Update: Still works even after Hulu tightened their geofiltering. As long as one isn’t using a mainstream service like Hotspot Shield, I doubt Hulu will be able to fully block this since they are in essence removing access from legitimate US computers. Probably the best thing would be to have a server set up in a legitimate, large institution such a US university or have a home-based server on a consumer ISP in the US. Both would make it hard to block the IP without causing legitimate users to be blocked as well. Then again, for me this is more of an interesting thing to do rather than a necessity. Mileage may vary depending on how proactive Hulu gets, in the end all anyone needs is one non-blocked legitimate IP address in the US.

If you haven’t heard of Hulu, it’s basically a website like YouTube but with full episodes of recent TV series – legal and with short commercials inserted into the video streams. For instance, the HBO show CSI is available on the site very soon after it has been shown in the US and you can watch it directly as streaming video. There is one caveat – the website blocks non-US viewers from watching any of the videos. So, no luck for us Finns.

However, this is rather easy to get around if you happen to have a server based in the US with SSH access. What you need to do is set up a SSH tunnel to your US server, then instruct your web browser to connect via that tunnel. This will cause all connections to Hulu to go through your US server, and will make it possible to watch Hulu videos.

Setting up using the SSH command/PuTTy

SSH allows you to connect securely to the server and create a secure tunnel from your computer to the US-based server. You can set up a local SSH client program to do this.

ssh -D 8080 -p 22 -f -N username@yourserver.com

As the SSH manual page states the -D option:

-D port. Specifies a local "dynamic" application-level port forwarding.
 This works by allocating a socket to listen to port on the local side, 
and whenever a connection is made to this port, the connection is
forwarded over the secure channel, and the application protocol is then
used to determine where to connect to from the remote machine.  Currently
the SOCKS4 and SOCKS5 protocols are supported, and ssh will act as a SOCKS server.

You can also do this in Windows using the PuTTy SSH client. Look under Connection -> SSH -> Tunnels. Add a dynamic port forward, use port 8080 and no need to specify the destination.

Setting Firefox to use the tunnel

Then set up Firefox to use the local SOCKS server (localhost:8080). The relevant settings can be found under Tools -> Options -> Advanced -> Network -> Settings … -> Manual proxy configuration.

Testing

This works very nicely, because you are now connecting from an US IP. To verify this, open up Internet Explorer and check http://www.whatsmyip.org/ and do the same in Firefox. You should see two different IP addresses. Just remember that any videos you watch will be transferred twice (from a bandwidth usage perspective) – once to your US server and then back. A single video is only about a hundred megabytes or so, so this not particularly bad for the convinience though.