Architecture of Xuheki

The application is composed of a server-side part and a client-side part. The client-side is comprised of the JavaScript, CSS and HTML files located under docroot/. The server-side is furthermore split in two parts:

• The Apache handler, located under perl/XHK/Apache
• The "mail server" — bin/xuheki.pl and perl/XHK/Mailserver

We shall briefly discuss the server-side players in this document.

The Apache handler

The main entry point is XHK::Apache::Handler, which figures out if the file to be served is a template that exists in docroot, or a static file. In this case it lets Apache do its thing (processing the template through Template::Toolkit if needed).

If the request is not a file in docroot, this module uses XHK::Apache::Dispatcher, which in fact is a base object for XHK::Apache::Main_Dispatcher. This is the most complex part of the Apache handler.

XHK::Apache::Main_Dispatcher

This module defines some handlers for various special URL-s. It handles itself all requests that don't need interaction with the IMAP server. However when it needs to talk to the IMAP server, it delegates the request to xuheki.pl.

AJAX requests are in JSON format. The syntax is quite simple. Each request contains a request ID, a method name to call, followed by the arguments. Requests (method names) that are to be handled by the Apache handler start with "apache.". Requests that are to be handled by the mail server (xuheki.pl) start with "mail.". We might have more namespaces in the future.

xuheki.pl — the "mail server"

I generically call this the "mail server", but it's not really a mail server. It's a standalone server process which listens for connections on a certain port. Apache connects to this port when it needs to talk with this process.

xuheki.pl forks a child process for each IMAP server that it needs to connect to. Once forked, these processes stay connected to IMAP for a certain time (one of the reasons why Xuheki is this fast is that it doesn't have to connect and authenticate to IMAP at each request).

When a "mail." request comes in, the Apache handler will pack the arguments in a format suitable for xuheki.pl, connect to the server and wait for the response. If needed, xuheki.pl automatically forks and connects to the requested IMAP server to handle the request.

xuheki.pl need not necessarily run on the same host as the Apache handler. Furthermore, it can access remote IMAP servers—no need to run xuheki.pl on the same host as the IMAP server either. However, for best results the communication channels between these 3 parts should be very fast. Also, xuheki.pl has (currently) some problems when the connection to IMAP is dropped so it is advised that it runs on the same actual host (or at least, within the same network).

Implementation of xuheki.pl

The code that deals with IMAP is in XHK::Mailserver::* modules and it's reasonably clean and well designed (at least I'm happy with it).

xuheki.pl itself is simply a server daemon that forks/dispatch requests to XHK::Mailserver. It is built with the (wonderful) POE framework. POE is so marvelous that you can do great things with it without knowing it at all. For this reason (yes, I don't know POE at all) xuheki.pl is somewhat copy-pasted from various POE samples. I'd love to clean it up at some point—any help and/or suggestions with this are more than welcome.

The main handler of the requests is the toplevel xuheki.pl process. When a request is received, it is dispatched to the appropriate child process (a new child is forked if needed). Communication between the two is done via STDIN/STDOUT (protocol based on Storable), all handled very well by POE::Wheel::Run.

Child processes are killed after idling for 10 minutes (this usually means that the user closed the browser), in order to free server resources.

Communication protocol

The path of a typical request will look like this:

  Browser
    |
    |  JSON request
    V
  Apache
    |
    |  Storable via TCP socket (authenticates if needed)
    V
  xuheki.pl
    |
    |  Storable via STDIN
    V
  xuheki.pl child process
    |
    |  IMAP request
    V
  IMAP
    |
    |  IMAP response
    V
  xuheki.pl child process
    |
    |  Storable via STDOUT
    V
  xuheki.pl
    |
    |  Storable response via TCP socket
    V
  Apache
    |
    |  JSON
    V
  Browser

• Communication between the browser and Apache is done through JSON-encoded strings (always in UTF8).
• Apache talks to xuheki.pl using Perl's Storable API. This is binary and extremely fast but it has one minor requirement: Apache and xuheki.pl must run on machines with the same byte order (easy to work around this with Storable API, but for now we don't have it as an option). Also, reasonably up-to-date versions of Perl/Storable should be installed on both locations.

Communication between Apache and xuheki.pl is unencrypted. This is not a problem if they run on the same machine or in a trusted network. I don't plan on adding encryption to this channel, but you can use stunnel if you really need it (but this is out of the scope of this document).

Authentication

Apache does some authentication with xuheki.pl. This is important in environments where more than one can access the machine. The initial development was that you could telnet to port 10000 and select which IMAP server you want to use, then start typing commands and receiving results. This way you could read everyone's mail. ;-)

So to avoid this embarrassing situation, I added some basic authentication based on a "shared secret" (password). The procedure is encrypted and works like this:

1. When it connects to xuheki.pl, Apache receives a random string ("challenge"). The connection does not close at this stage and xuheki.pl writes down the challenge, so it knows it on the next request.
2. Apache encrypts the challenge using as key the MD5 of a password which is only known by Apache and xuheki.pl. It sends the encrypted value back to xuheki.pl.
3. xuheki.pl decrypts the value using the shared password and compares it with the challenge that was noted at step 1. If values match the connection becomes authenticated and xuheki.pl accepts commands that can talk to any registered IMAP server.

Given that the password is only known by Apache and xuheki.pl, I trust this simple mechanism to be efficient. The encryption is done with the AES cipher (Crypt::Rijndael) which should provide very good security if Apache runs on a different host than xuheki.pl (although it is kind of pointless, since once authenticated the communication is no longer encrypted).

This password is stored in /opt/Xuheki/secrets/mailserver-config.pl. The owner of this file should be an user/group that both processes (Apache and xuheki.pl) run under, and the permissions should be 0660, or something like this. It's up to you to secure this. ;-)

Danger: shared (web) hosting!

(note that here I'm talking about sharing an Apache process with others, not about VPS¹. VPS is safe.)

Xuheki wasn't designed with shared hosting in mind. In fact, it would be even quite hard, if not impossible, to install it on a shared server, because you usually won't have the privileges to install the necessary Perl modules or run a custom server process.

However, if you do succeed installing it on a shared host (really not recommended) you want to keep in mind the following: anyone who can write an Apache handler of any kind (mod_perl, CGI, PHP even!) can potentially do anything with your mail! That's because the Apache process can read the file containing the secret password, and having it one can get full trust from xuheki.pl.

1. a Virtual Private Server gives you almost full control on your system; it's not the same as shared hosting, although they could be confused (since the physical machine is shared with others).