Sunday, February 26, 2017

Automatically starting a Django application on Mac OSX Sierra with nginx and uWSGI

Background

I had to patch this information together from a number of different places, so collecting it all together may provide a useful reference.

The background is that I previously had the application running in a Ubuntu VM on a Windows PC. Upgrading to the Mac meant that I could run all desktop applications alongside the server-side apps without context switching, VM resource management issues, etc. However, it did mean that I had to solve a whole bunch of configuration issues.

The Ubuntu setup was fairly straightforward and used an upstart script to start the application. Finding the Mac equivalent was the last part of the process...

Environment

Mac OSX Sierra
Python 3.5.3 (Anaconda )
Django 1.10.5
Postgres 9.6
nginx 1.10.3

Software installation

This proved to be quite simple on the Mac because there were desktop installers for Postgres and Anaconda Python. Details here:
brew install nginx

How easy is that?

More details here:

https://coderwall.com/p/dgwwuq/installing-nginx-in-mac-os-x-maverick-with-homebrew

My application runs in a virtual environment which is set up using conda rather than virtualEnv. Instructions here:

https://conda.io/docs/using/pkgs.html

Activating the virtual environment is a question of running the standard command

source activate 

Installing all of the required Python packages (including django and uwsgi) was a question of comparing the new environment with the old one. I did this manually, but in fact a better way would be to use a requirements.txt file. However, some packages can be installed using conda while others have to be installed using pip (not sure why...). I have not checked how this might affect file maintenance using a requirements.txt.

Gotchas

When testing uWSGI with the standard test script, everything seemed to work correctly - no errors in the network log, for example - but no output appeared in the browser window. I eventually realised that this was because I was using Python 3 which requires an indication of the Unicode string type:

def application(env, start_response):
    start_response('200 OK', [('Content-Type', 'text/html')])
    return [b"Hello!"]

Notice the b prefix on the output string which marks it as a Unicode byte string

Configuring automatic startup

In Unbuntu, nginx  is configured to start automatically by default and can be controlled using systemctl. The Django application can be started automatically with an upstart script in the /etc/init.d directory.

On the Mac, both processes can be started by creating a plist file in the /System/Library/LaunchDaemons directory. However, this directory is protected by System Integrity Protection (SIP) - details here:

https://support.apple.com/en-gb/HT204899

My approach is to to stay as close to the default system settings as possible so that I can rely on the standard documentation, so although I am familiar with many forms of Unix/Linux which do not use this type of protection, I have kept SIP enabled. It needs to be disabled temporarily though to make the necessary changes to the daemon startup scripts.  To do this:

  1. Boot the Mac into Safe Mode by holding down Command+R until the Apple logo appears
  2. Select the working language (English)
  3. From the menu, select Utilities → Terminal
  4. At the terminal prompt, type

    csrutil disable
    

  5. Reboot normally
After making the changes below, go through the same process again to re-enable csrutil

plist files

For nginx, follow the instructions here:

https://www.nginx.com/resources/wiki/start/topics/examples/osxlaunchd/

For the Django application, use a plist file similar to the one below and enable it in the same way.


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>EnvironmentVariables</key>
        <dict>
  <key>PATH</key>
  <string>/anaconda/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
 </dict>
 <key>Label</key> <string>YOUR-SCRIPT-LABEL</string>
 <key>Program</key> <string>/anaconda/envs/YOUR-ENV-NAME/app/startup</string>
 <key>RunAtLoad</key> <true/>
 <key>StandardErrorPath</key> <string>/var/log/SCRIPT-NAME.stderr</string>
 <key>StandardOutPath</key> <string>/tmp/SCRIPT_NAME.stdout</string>
 <key>WorkingDirectory</key> <string>/anaconda/envs/YOUR-ENV-NAME/app</string>
</dict>
</plist>

This script makes a couple of assumptions:

  • You have created a conda environment whose path is /anaconda/envs/YOUR-ENV-NAME
  • You are logging standard and error output to the directory /var/log
The program file startup referenced at line 11 is a shell script which has the following form:

#!/bin/bash
env
source activate YOUR-ENV-NAME
uwsgi --ini wsgi.ini

The problem of setting the python virtual environment is encapsulated by the shell script. In addition, the wsgi parameters are specified in the .ini file.

I found the LaunchControl app very useful in arriving at the final version. Details here:

http://www.soma-zone.com/LaunchControl/