Friday, March 01, 2019

Updates to Flask installation

Some additional details:

... and one gotcha

The name of the wsgi script file must be in lowercase (!)

Posted by at No comments:

Sunday, March 11, 2018

Setting up the server for a team Flask project

Introduction

This post shows how to set up a minimal Flask application on Ubuntu 14.04 using Python 3.5 and Apache 2.4 using WSGI to communicate between the Web server and the application. It assumes that a team of developers will be sharing management of the code.

As with any configuration task, there are always alternatives. This guide only covers one simple approach and makes no claim to be the best. On the other hand, it does claim to be simple to follow and reasonably complete.

You can follow the steps below in a dumb way if you like – that means copying and pasting the commands into the command line without paying any attention to their meaning. That should get you to a working system; HOWEVER, that is the dumb way – if you want to know how your system works and how to fix things when they go wrong, there is no alternative but to spend some time following through the documentation that is linked with each step.

Overview

Eventually, we will have an Apache Web server which receives all incoming http requests. It will then use the WSGI configuration to pass the request to the Flask application for handling. Each element in the chain needs to be installed and configured.

We assume that the following decisions have been taken:
  • The application will use a python virtual environment
  • The application and the virtual environment will be in a shared location
  • A directory /usr/local/env will contain all shared locations
  • Application code will be kept separate from the environment code
  • The name of the application is myApp
  • The fully qualified host name is www.myapp.com
We should end up with the following directory structure:

/usr/local/env/myApp
                |- myApp.wsgi
                |- app/
                |- vmyApp/

myApp.wsgi is the file that Apache will use to communicate with the application
app/ is the directory where the application code goes
vmyApp/ is the directory that contains the virtual Python environment

The conventions being used are:
  • The application name is unique on the server
  • All similar applications will have an app/ directory for their code
  • The name of the Python virtual environment is v + app name. This is to differentiate between environments if there is more than one application

Instructions

  1. sudo rm /usr/bin/python
  2. sudo ln -s /usr/bin/python3.5 /usr/bin/python
  3. sudo apt-get install apache2
  4. sudo apt-get install libapache2-mod-wsgi-py3
  5. sudo mkdir /usr/local/env
  6. sudo mkdir /usr/local/env/myApp
  7. sudo chmod o+w /usr/local/env/myApp
  8. cd /usr/local/env/myApp
  9. virtualenv -p python3 vmyApp
  10. . /usr/local/env/myApp/vmyApp/bin/activate
  11. pip3 install flask
  12. mkdir app
  13. touch app/__init__.py
  14. Create Hello World app:

    Place the code below into the file /usr/local/env/myApp/app/myApp.py
    from flask import Flask
app = Flask(__name__)


@app.route('/')
def hello():
    return "Hello World!"

if __name__ == '__main__':
    app.run()
  15. Create the .wsgi file

    Place the code below into the file /usr/local/env/myApp/myApp.wsgi
    import sys
sys.path.insert(0, '/usr/local/env/myApp/')

activate_this = '/usr/local/env/myApp/vmyApp/bin/activate_this.py'
with open(activate_this) as file_:
    exec(file_.read(), dict(__file__=activate_this))

from app.myApp import app as application
  16. Create the site configuration file

    Place the code below into the file /etc/apache2/sites-available/myApp.conf
    
    ServerName www.myapp.com

    WSGIDaemonProcess myApp python-home=/usr/local/envs/myApp/vmyApp threads=5
    WSGIScriptAlias / /usr/local/env/myApp/myApp.wsgi

    
        WSGIProcessGroup myApp
        WSGIApplicationGroup %{GLOBAL}
        Require all granted
    

  17. sudo a2dissite 000-default
  18. sudo a2ensite myApp.conf
  19. sudo service apache2 restart

    20. Notes:

    Step 7: This allows all users of the server to write to the shared location which is not a good idea. However, these instructions were prepared for a development server where user and group access was controlled by a central administration service. Because of this, there was no option to create a dummy user account to ‘own’ the code. The assumption is that this is a development server, and that the production configuration would take more account of security.
    Step 14: This is the standard Hello World that can be found all over the Web with the additional lines to allow it to work with WSGI
    Step 15: The second line inserts the application root directory into the Python load path. The last line makes reference to the app object in the myApp.py module which is inside the app package.
    Step 16: The WSGIScriptAlias points to the .wsgi file in the application root directory. The directive points to the application root directory and ensures that all subdirectories are accessible to the Web server.




Posted by at No comments:

Monday, November 27, 2017

Quarantined plist

Just a quick one: I upgraded my Mac OS today to High Sierra only to find that afterwards my installation of nginx did not serve my Django application. Two things needed to be done - the first one was to upgrade my version of homebrew and then to use it to reinstall nginx. Pas de probleme...

 The second problem was a bit more tricky to track down. Eventually it turned out that the upgrade process had removed my plist file from the directory /Library/LaunchDaemons. Checking in the install log (over 100K lines, BTW...), it seems that this process is described as 'shoving a sandbox file into quarantine', Cheers, Apple. It would have been nice to know...

 I tracked the missing file down to the directory /Volumes/Macintosh HD/Library/SystemMigration/History/Migration-6909D1ED-25C0-43B1-93D7-728FFB8E9108/QuarantineRoot/System/Library/LaunchDaemons/ and had to go through the process of disabling the SIP protection to restore it.
Posted by at No comments:

Sunday, March 05, 2017

Using a custom input widget in ag-grid

I am currently working with Angular 2 as my front-end framework, and Django at the back end. I have used ag-grid on a couple of different pages and all of the basic functionality works brilliantly. When straying from the happy path though, things get a little tricky. Here I sketch out the solution I came up with to use a custom cell renderer to provide a drop-down list in a cell, and to pass the selected value back to the underlying data structure.

Background


The application I am building is designed to help with the processing of large comma- or tab-separated datafiles. One of the things I need to do is allow the user to select a function to be applied to some of the data in the file, and what parameters to use. My use case for ag-grid is in the definition of this type of calculation. This screenshot should clarify:


The drop-down lists for parameter and data types are created using an Angular 2 component as a custom cell renderer according to the instructions provided here: https://www.ag-grid.com/best-angular-2-data-grid/#gsc.tab=0

Simple updates


The code for the parameter type cell renderer is shown below.

import {Component} from "@angular/core";
import {AgRendererComponent} from "ag-grid-ng2";

@Component({
    moduleId: module.id,
    template: `
        <md-select placeholder="Parameter type" 
                      [(ngModel)]="selectedValue" 
                      (change)="onChange($event)">
                <md-option 
                    ngfor="let parameterType of params.context['parameterType']" 
                    value="{{ parameterType.value }}">
                    {{ parameterType.name }}
                </md-option>
        </md-select>
`
})

export class ParameterTypeCellRendererComponent implements AgRendererComponent {
    private params: any;
    private selectedValue: string;

    constructor(){}

    agInit(params: any): void {
        this.params = params;
        this.selectedValue = this.params['data']['stringParameterType'];
    }

    onChange(event: any) {
        let gridEvent: CustomEvent = new CustomEvent('wa-parameter-type-changed', {
            'bubbles': true,
            'detail': {
                'parameterSequence': this.params.data.parameterSequence,
                'parameterTypeValue': event.value
            }
        });
        event.source.trigger.nativeElement.dispatchEvent(gridEvent);
    }
}

The key point to notice is the onChange event handler. This fires whenever the selected value of the widget changes. Its purpose is to raise a custom event (wa-parameter-type-changed) which bubbles up through the DOM and is detected in the main document. The event listener is shown below.

document.addEventListener('wa-parameter-type-changed', (e) => {
    this.calculation.parameters[e['detail']['parameterSequence']].stringParameterType = 
        e['detail']['parameterTypeValue'];
});

The values contained in the event's detail field are used to identify the value that needs to be updated. The grid's rowData is supplied by this.calculation.parameters which are ordered by their sequence numbers, so once the value has been updated by the listener, the state of the data is consistent with the state of the widget.

Displaying the selected value


A further detail is that md-select does not play well with non-string values. I puzzled over this one for a while. When the fields are already populated - for example, when editing a function - it is important that the drop-downs all display the values that are currently set. This is possible using Angular's standard data binding, but only if the Typescript variables are strings. Using numbers will not work. For this reason, I have ended up with string equivalents of the underlying numeric values throughout the application. It's a bit fiddly, but maintaining the consistency all the way through from Django serialisers to the Angular interface widgets minimises the potential for error. For interest, here is an example of an md-select which uses a data-bound variable to specify the selected value:


<md-select [(ngModel)]="calculation['stringReturnValueDatatype']" placeholder="Return value datatype">
    <md-option *ngFor="let datatype of staticData.datatype" value="{{ datatype.value }}">
        {{ datatype.name }}
    </md-option>
</md-select>



Posted by at No comments:

Sunday, February 26, 2017

Automatically starting a Django application on Mac OSX Sierra with nginx and uWSGI

Background

I had to patch this information together from a number of different places, so collecting it all together may provide a useful reference.

The background is that I previously had the application running in a Ubuntu VM on a Windows PC. Upgrading to the Mac meant that I could run all desktop applications alongside the server-side apps without context switching, VM resource management issues, etc. However, it did mean that I had to solve a whole bunch of configuration issues.

The Ubuntu setup was fairly straightforward and used an upstart script to start the application. Finding the Mac equivalent was the last part of the process...

Environment

Mac OSX Sierra
Python 3.5.3 (Anaconda )
Django 1.10.5
Postgres 9.6
nginx 1.10.3

Software installation

This proved to be quite simple on the Mac because there were desktop installers for Postgres and Anaconda Python. Details here:
brew install nginx

How easy is that?

More details here:

https://coderwall.com/p/dgwwuq/installing-nginx-in-mac-os-x-maverick-with-homebrew

My application runs in a virtual environment which is set up using conda rather than virtualEnv. Instructions here:

https://conda.io/docs/using/pkgs.html

Activating the virtual environment is a question of running the standard command

source activate

Installing all of the required Python packages (including django and uwsgi) was a question of comparing the new environment with the old one. I did this manually, but in fact a better way would be to use a requirements.txt file. However, some packages can be installed using conda while others have to be installed using pip (not sure why...). I have not checked how this might affect file maintenance using a requirements.txt.

Gotchas

When testing uWSGI with the standard test script, everything seemed to work correctly - no errors in the network log, for example - but no output appeared in the browser window. I eventually realised that this was because I was using Python 3 which requires an indication of the Unicode string type:

def application(env, start_response):
    start_response('200 OK', [('Content-Type', 'text/html')])
    return [b"Hello!"]

Notice the b prefix on the output string which marks it as a Unicode byte string

Configuring automatic startup

In Unbuntu, nginx  is configured to start automatically by default and can be controlled using systemctl. The Django application can be started automatically with an upstart script in the /etc/init.d directory.

On the Mac, both processes can be started by creating a plist file in the /System/Library/LaunchDaemons directory. However, this directory is protected by System Integrity Protection (SIP) - details here:

https://support.apple.com/en-gb/HT204899

My approach is to to stay as close to the default system settings as possible so that I can rely on the standard documentation, so although I am familiar with many forms of Unix/Linux which do not use this type of protection, I have kept SIP enabled. It needs to be disabled temporarily though to make the necessary changes to the daemon startup scripts.  To do this:

  1. Boot the Mac into Safe Mode by holding down Command+R until the Apple logo appears
  2. Select the working language (English)
  3. From the menu, select Utilities → Terminal
  4. At the terminal prompt, type 

    csrutil disable

  5. Reboot normally
After making the changes below, go through the same process again to re-enable csrutil

plist files

For nginx, follow the instructions here:

https://www.nginx.com/resources/wiki/start/topics/examples/osxlaunchd/

For the Django application, use a plist file similar to the one below and enable it in the same way.


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>EnvironmentVariables</key>
        <dict>
  <key>PATH</key>
  <string>/anaconda/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
 </dict>
 <key>Label</key> <string>YOUR-SCRIPT-LABEL</string>
 <key>Program</key> <string>/anaconda/envs/YOUR-ENV-NAME/app/startup</string>
 <key>RunAtLoad</key> <true/>
 <key>StandardErrorPath</key> <string>/var/log/SCRIPT-NAME.stderr</string>
 <key>StandardOutPath</key> <string>/tmp/SCRIPT_NAME.stdout</string>
 <key>WorkingDirectory</key> <string>/anaconda/envs/YOUR-ENV-NAME/app</string>
</dict>
</plist>

This script makes a couple of assumptions:

  • You have created a conda environment whose path is /anaconda/envs/YOUR-ENV-NAME
  • You are logging standard and error output to the directory /var/log
The program file startup referenced at line 11 is a shell script which has the following form:

#!/bin/bash
env
source activate YOUR-ENV-NAME
uwsgi --ini wsgi.ini

The problem of setting the python virtual environment is encapsulated by the shell script. In addition, the wsgi parameters are specified in the .ini file.

I found the LaunchControl app very useful in arriving at the final version. Details here:

http://www.soma-zone.com/LaunchControl/
Posted by at No comments:

Monday, June 14, 2010

Hardware and software: worlds apart

I had one of those experiences recently akin to "Bank error in your favour" in Monopoly. Having spent less than £200 on a second-hand laptop from eBay, I contacted HP about a minor problem because it was still under warranty. Because they did not have the necessary spare part, they offered me an upgrade to a £1500 Elitebook tablet! Thank you, Universe.

I had a problem with it today, and having looked for a solution on the Internet with no success, I though I should post mine out into the Interwebosphere so that it is available to others in need.

If you find that fn+f4 only toggles one way, so that your laptop screen is always blank when you have an external monitor connected, try changing back to dual display using the controls on the graphics display driver itself. If you are using the Intel driver, click the icon in the system tray, look under "Graphics options" and select one of the "Output to..." options.

You're right though - with that price tag, fn+f4 should work.
Posted by at No comments:

Friday, January 26, 2007

Trous de memoire

I imagine there must be a few blogs like this around with gaps of 18 months or so where nothing seems to happen. I'm teaching on that same module again, and this time we're using the Bloglines aggregator with the students. So, since you have to test these things, the old Blogger personal blog is coming in handy.

Hello World - remember me?
Posted by at No comments:
Subscribe to: Posts (Atom)