Tuesday, June 28, 2016

Intelligent US Address formatting

I recently needed to have a good solution for parsing US postal addresses into a standard format. After a bit of quick googling I found a neat github project that does most of what I needed. A big shout-out to the Atlanta Journal Constitition newspaper for doing this.

If you're running python, all you have to do is 'pip install usaddress' and you're up and running.

I then wrote a quick python script to convert usaddress output to a standard format, like this:

[Optional Name,  though you really shouldn't pass it]
Address line 1
[Optional Address Line 2]
City
State
Zipcode
#

Here's the usaddr.py script (python 3.x):

#!/usr/bin/python

import sys
import usaddress

for addr in sys.argv[1:]:
  try:
    c = usaddress.tag(addr)[0]
    l = []
    
    breakfields = [
      'Recipient',
      'BuildingName',
      'LandmarkName',
      'CornerOf',
      'USPSBoxType',
      'AddressNumber',
      'AddressNumberPrefix',
      'PlaceName',
      'StateName',
      'ZipCode'
    ]
    
    multikill = {
      'AddressNumberPrefix' : [ 'AddressNumber' ]
    }

    for k,v in c.items():
      if k in breakfields:
        breakfields.remove(k)
        if k in multikill:
          for mk in multikill[k]:
            if mk in breakfields:
              breakfields.remove(mk)
        l = [i for i in l if i]
        if l != []:
          print(' '.join(l))
        l = [v]
      else:
        l.append(v)
        
    l = [i for i in l if i]
    if l != []:
      print(' '.join(l))
    print('#')
          
  except usaddress.RepeatedLabelError as e :
    print('Error: cannot parse address')

Since I'm running a LAMP stack on one of my servers that happens to have python also installed, I wrote a quick usaddr.php script to implement an API:

< ?php

foreach (explode('&', $_SERVER['QUERY_STRING']) as $chunk) {
  $param = explode("=", $chunk);

  if ($param) {
    $cmd = urldecode($param[0]);
    $arg = urldecode($param[1]);
        
    if ($cmd === 'addr') {
      $command = 'python /path/to/your/script/usaddr.py ' . escapeshellarg($arg);
      $output = shell_exec($command);
      echo $output;
    }
  }
}

?>

So this meant I could access http://myserver.com/myapi/usaddr.php?addr=[address to reformat] from other apps (like FileMaker) to do address reformatting.

Friday, June 10, 2016

Setting up Mail Relay behind NAT for local network on OS X

I recently had a need to set up an outbound mail relay on my home network so local machines behind the NAT, (ie: machines with 192.168.x.x addresses) could send an occasional status email.

It turns out that OSX has a built-in postfix mail server, and it's pretty easy to turn it on. Here are some step-by-step instructions.

1) Create a domain alias that points to the external IP address of your network (ie: what http://whatismyipaddress.com/ says is your IP address). If this address is static and unchanging, then you can just set up a CNAME in the DNS of some domain you own. If it tends to change because your ISP reallocates them from time to time, then use a service like http://dyn.com/ to maintain a dynamic dns listing. Many routers support dyn address updating, and there are also apps that will run on an internal machine to do it.

2) For any domain that you want to send mail for, make sure this new dns entry is listed in the SPF record in the DNS. See http://www.openspf.org/ for more information. This will help ensure that other mailservers will consider the mail to be legitimate.

3) On the machine that will be doing the forwarding, you now need to edit a few system files. You'll need a text editor for this -- if you've done any mucking around in the shell, you'll know how to do this.

/System/Library/LaunchDaemons/org.postfix.master.plist

This config file controls how postfix is launched. Here's a simple one that should work with OSX 10.5 or later. If the AbandonProcessGroup key does not appear in the original .plist file, you can delete those two lines (17-18):

/private/etc/postfix/main.cf

This config file describes how postfix does its business. We need to lock it down so it will only accept mail from machines in our local network. It's a long file, but there are two configuration lines you will need to change:

inet_interfaces = all

This tells the server to listen on all the network interfaces, not just the internal "loopback" one.

127.0.0.1/32 is the default "localhost" address for your current machine. If you only want to relay mail for your current machine (the one postfix is running on), you set mynetworks to be just that:

mynetworks = 127.0.0.1/32

If you also want to relay mail for other machines in your local network, you need to know what the network range is. Typically that is something like 192.168.1.xxx, in which case you'd add 192.168.1.0/24 as the network range (the 24 is the network mask, it says how many bits out of the 32 bits are fixed). So for example:

mynetworks = 127.0.0.1/32, 192.168.1.0/24

Finally, you need to restart postfix and let the system know about the changed configurations. In Terminal, do this:


sudo launchctl stop org.postfix.master
sudo launchctl start org.postfix.master
sudo postfix start

Test that it's working by trying to connect to the mailserver:

telnet 127.0.0.1 25

You should see something like this:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 Trebor.local ESMTP Postfix

Now test your mail sending by, for example, trying to send mail to a gmail account. Hopefully you'll be good to go.