Previous Section  < Day Day Up >  Next Section

Hack 74. Import Mail into Gmail

Moving to Gmail doesn't have to mean starting from scratch. Forward mail in bulk from your computer or other web mail service to your Gmail account.

The most enticing feature of Gmail is probably its ability to perform Google-style searches on your own inbox. The one gigabyte of free space is intriguing, but it's not much when you consider that you have far more than that available to you on even your most outdated PC. And I'd warrant that not even its snazzy JavaScripted user interface is enough to tear you away from your existing web mail service, uprooting yourself and starting over.

Gmail doesn't currently provide any way to import your existing email archive (web mail service or desktop mailbox). While you already might have considered forwarding all that mail to your Gmail account, just how to do so—even just the few hundred "important" messages—is quite a trick.

Not so, thanks to hacks like the Google Mail Loader for forwarding desktop mail and web mail intermediaries YPOPs! for Yahoo! Mail and MSN email and GetMail for Hotmail.

6.7.1. Forward Desktop Mail

The Google Mail Loader (http://www.marklyon.org/gmail; GNU Public License) is a point-and-click application that reads your existing mail files on your computer and forwards the messages on to Gmail—one every two seconds, so as not to overload or otherwise annoy the Gmail servers. It does so without deleting mail from your local computer; what's sent to Gmail is a copy of each and every message. You can even set it to drop uploaded messages into your Gmail Inbox or Sent Mail folder.

GML is cross-platform and understands multiple mailbox formats:

  • Mbox (used by Netscape, Mozilla, Thunderbird, and many other email applications)

  • MailDir (Qmail and others)

  • MMDF (Mutt)

  • MH (NMH)

  • Babyl (Emacs RMAIL).

  • Microsoft Outlook, via a utility such as PST Reader (http://www.mailnavigator.com/reading_ms_outlook_pst_files.html), which converts Outlook's PST files to Mbox format

6.7.1.1 Installing the hack

Download the Windows or Linux/Mac OS X, source-only version (http://www.marklyon.org/gmail/download.htm). The Windows version is definitely the simplest version to set up and use, requiring no prerequisites and other bits and pieces.

The source version assumes you have the Python scripting language and the Python Mega Widgets (http://pmw.sourceforge.net) toolkit installed.

The ins and outs of installing GML and all the prerequisites from source is beyond the scope of this book. If you need help, consult the documentation for Python (http://www.python.org) and Python Mega Widgets (http://pmw.sourceforge.net), or ask your local technical guru or system administrator.

If, on the other hand, you have Python on your system and don't much care whether the Google Mail Loader is a desktop or command-line application, skip ahead to the "Hacking the hack" section.


6.7.1.2 Running the hack

Since Google Mail Loader works directly with your email application's mailboxes, you'll need to figure out where they live before you can go much further. Consult your email app's preferences or documentation or just dig around—both on your hard drive and by googling for "outlook express" mailbox files location, replacing "outlook express" with the name of your email program.

You'll also need to make sure that your mailbox files are in a format that Google Mail Loader can read, as listed in the beginning of this hack. If there's any conversion to do, do so now. For instance, use PST Reader (http://www.mailnavigator.com/reading_ms_outlook_pst_files.html) to turn Outlook and Outlook Express PST files into DBX format.

With mailbox files in hand, launch Google Mail Loader by double-clicking gmlw.exe on Windows or typing python gmlw.py on the Unix or Mac OS X command line. Figure 6-14 shows Google Mail Loader running under Linux.

Figure 6-14. Google Mail Loader


Work your way down the settings on the left half of the GML window:

  1. The default SMTP server (that's the sendmail server, the one used to send your messages to Gmail) of gsmtp57.google.com works for most users. If, for some reason, you are required by your local network administrator or Internet service provider to use their outgoing mail server, replace the default with the appropriate address. If your outgoing mail server requires authentication, click the Requires Authentication checkbox and fill in your username and password.

  2. Click the Find button and point GML at your mailbox file. If your email application uses MailDir format, select any file inside your MailDir directory.

  3. From the File Type pull-down menu, choose your mailbox type (Figure 6-15). There are two versions of Mbox format: one stricter about the format of files and therefore more accurate, while the other is more lenient and works better on some Mbox files.

    Figure 6-15. Select your mailbox file type

    For some of the history, read Jamie Zawinski's "mail summary files" at http://www.jwz.org/doc/mailsum.html.


If you don't know what format your mail application uses, try googling for mail format pine, replacing pine with your mail app's name. (Pine uses Mbox, by the by.)

  • GML is able to upload both your incoming and outgoing mail. Choose Mail I Received from the Message Type pull-down menu, and messages will be dropped into your Gmail Inbox and appear to be from the original sender, just as they did in your email application's mailbox. If you choose Mail I Sent, the messages will be relabeled as coming from your Gmail address and appear in your Gmail Sent Mail folder.

    Gmail automatically labels incoming messages as Inbox. There's no way, unfortunately, for an external application to change this behavior, so messages imported as Mail I Sent will be labeled as both Sent Mail and Inbox and appear in both places. Mind you, there is only actual one copy of the message stored and sent mail is relabeled so as to appear to be from your Gmail address, not your old email address.

    If you Archive the copy you see in your Gmail Inbox, it will then appear only in Sent Mail (and Archive, of course).


  • Finally, type in your full Gmail address (e.g., hank@gmail.com).

  • Click the Send to Gmail button and the application will start sending messages, one every two seconds. The delay is necessary to prevent flooding of Google's servers.

    • If you're interested in the details, click the Save Log button to save the contents of the output window to a file for later review.

    There are, as with any hack of this sort, some issues worth noting:

    6.7.1.3 Hacking the hack

    If you're a command-line jockey or don't particularly relish installing the various prerequisites (Tk, Python Mega Widgets) necessary to get the graphical version of Google Mail Loader running, there's also a text-only version available at http://www.marklyon.org/gmail/old/default.htm.

    The only requirement for the command-line GML is Python (http://www.python.org).


    Here's a sample session with the older GML on the Mac OS X command line:

    $ python gml.py 

Mbox & Maildir to Google Mail Loader (GML) by Mark Lyon <mark@marklyon.org>



Usage: gml.exe [mbox or maildir] [mbox file or maildir path] [gmail address] [Optional 

SMTP Server]

Exmpl: gml.exe mbox "c:\mail\Inbox" marklyon@gmail.com

Exmpl: gml.exe maildir "c:\mail\Inbox" marklyon@gmail.com gsmtp171.google.com



$ python gml.py mbox ~/Library/Mail/Mailboxes/1999.mbox/mbox 'hank+gml@gmail.com'



Mbox & Maildir to Google Mail Loader (GML) by Mark Lyon <mark@marklyon.org>



    1 Forwarded a message from  : someone@example.com



Done. Stats: 1 success 0 error 0 skipped.

    6.7.2. Migrate from an Existing Web Mail Service

    Despite attempts by your existing web mail service to entice you to stay, Gmail beckons with its one gigabyte of storage, powerful search, rich web interface, and chance of grabbing a better email address than raelity973@. That said, you're loathe to leave behind the last year or three's email.

    Well, you can indeed take it with you, thanks to some nice donateware Web-to-POP mail utilities. These intermediaries operate in one of two ways:

    While there are no doubt any number of these utilities, two we stumbled across were GetMail and YPOPs!

    Of course, you may just opt to pay for POP mail access to your web mail service, download all your mail like you would any other, and use the Google Mail Loader from there. If, however, you've gone this long without paying for POP service, chances are you're not going to do so now just to move out of the service.


    6.7.2.1 Hop from Hotmail/MSN

    GetMail (http://www.e-eeasy.com/GetMail.aspx; donate-ware) is a two-in-one for Hotmail and MSN that runs under Windows.

    Move any messages that you want to send across to your Gmail account to your Hotmail Inbox (if you've previously filed them elsewhere) and mark them as unread.

    Launch GetMail (shown in Figure 6-16), provide it with your Hotmail/MSN account name and password, and type your full Gmail address into the Forward To box. Check whatever options you prefer; I'd uncheck the Delete checkbox. Now click Check for New Mail to set GetMail in motion and go get a cup of coffee while it moves all those messages across for you. You can even leave it running, transferring your Hotmail messages to Gmail on an ongoing basis.

    Figure 6-16. GetMail can download Hotmail/MSN messages and forward them on to Gmail


    6.7.2.2 Yank your Yahoo! Mail

    YPOPs! (http://yahoopops.sourceforge.net/; donateware) is a POP mail proxy, sitting between your preferred email application and Yahoo! Mail. It is available for Windows, Mac OS X, Linux, and Solaris. The Windows version self-installs while the others require that you compile from source code and so are a little more difficult for the uninitiated to get up and running.

    Move any messages you want to download and carry across to Gmail into your Yahoo! Mail Inbox and mark them as unread.

    On Windows, run YPOPs! after installation. A little icon appears in your Windows taskbar; double-click it to get to the settings, shown in Figure 6-17.

    Figure 6-17. YPOPs! proxies POP mail requests


    While you can go ahead and make a few changes in the settings, YPOPs! runs right out of the box without any further configuration.

    Now, simply set up a POP mail account like any other, only pointing to YPOPs! running locally as your mail server—both incoming and outgoing. The YPOPs! site has details on configuring most email clients at http://yahoopops.sourceforge.net/modules.php?op=modload&name=Sections&file=index&req=listarticles&secid=1.

    Once you have downloaded all of your web mail to your computer, use the Google Email Loader to send all the contents of your local inbox to Gmail.

    6.7.3. See Also

    Mark Lyon, Justin Blanton, and Rael Dornfest

    Previous Section  < Day Day Up >  Next Section