The Python Flask web API is a simple web API to search for Australian postcodes based on locality aka suburb. With a newly installed Ubuntu 22.10 environment, we will go through the steps to host this web API using Gunicorn and Nginx.

067-feature-image.png
Ubuntu 22.10: hosting a Python Flask web API with Gunicorn and Nginx.

In this post Python: A simple web API to search for Australian postcodes based on locality aka suburb, I’ve developed a complete working web API. I’m discussing the steps to host this web API on Ubuntu 22.10 (Kinetic Kudu), using Gunicorn and Nginx.

The final content of this post has been written based on a fresh installation of Ubuntu 22.10. The firewall was disabled by default. We enable it:

$ sudo ufw enable

Verify that it has been enabled via:

$ sudo ufw status

The Ubuntu 22.10 machine name (host name) is hp-pavilion-15, its IP address is 192.168.0.17. These two identifiers will be used throughout this post.

The first draft of this post was based on my existing Ubuntu 22.10 installation, whereby accessing the final site was working for both hp-pavilion-15 and 192.168.0.17. Then I did something which damaged Ubuntu, and I don’t know how to recover, so did a fresh reinstallation: hp-pavilion-15 is no longer working!

Table of contents

Install required software

We need to install some Python related tools, Nginx and git. Run the following commands:

$ sudo apt-get update
$ sudo apt-get install python3-pip python3-dev nginx
$ sudo apt install python3-virtualenv
$ sudo apt install git

Nginx configuration -- /etc/nginx/nginx.conf

We make two changes to Nginx configuration file /etc/nginx/nginx.conf:

  1. user behai; -- user changed to behai. (I.e. commented out user www-data;.) behai is the user I use to log into Ubuntu.
  2. Enable server_names_hash_bucket_size 64; -- remove comment #.

Using nano to edit the Nginx config file /etc/nginx/nginx.conf:

$ sudo nano /etc/nginx/nginx.conf

# user www-data;
user behai;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

...

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	types_hash_max_size 2048;
	# server_tokens off;

	server_names_hash_bucket_size 64;

...

Check Nginx configuration:

$ sudo nginx -t

If everything is okay, the output should be as follows:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Restart Nginx and verify its status with the following commands:

$ sudo systemctl restart nginx
$ systemctl status nginx.service

Opens both port 80 and port 443 to allow both unencrypted and encrypted web traffic:

$ sudo ufw allow 'Nginx Full'

Nginx should now be ready, default site should be available. From Windows 10, I can get to the Nginx default site with http://hp-pavilion-15 and http://192.168.0.17:

067-03.png

About user behai; in /etc/nginx/nginx.conf

Quoting
Deploying Gunicorn – a Gunicorn official documentation page:

www-data is the default nginx user in debian, other distributions use different users (for example: http or nginx). Check your distro to know what to put for the socket user, and for the sudo command.

And see also this Stack Overflow answer, I DID ALSO ATTEMPT Joseph Barbere’s solution – DO NOT DO THAT! As a part of that suggested solution, I had to install SELinux, which is no longer compatible with Ubuntu 22.10; it damaged mine, I don’t know how to recover from the GRUB menu, that was why I did a fresh reinstallation.

Set up the web API project in Ubuntu 22.10

This step has been documented in GitHub Read Me. It is reproduced in this section.

Get the repo https://github.com/behai-nguyen/bh_aust_postcode.git to directory /home/behai/webwork/bh_aust_postcode. Its content should look like the screenshot below:

067-04.png

Create the virtual environment venv:

behai@HP-Pavilion-15:~/webwork/bh_aust_postcode$ virtualenv venv

To activate virtual environment venv:

behai@HP-Pavilion-15:~/webwork/bh_aust_postcode$ source venv/bin/activate

Editable install project:

(venv) behai@HP-Pavilion-15:~/webwork/bh_aust_postcode$ venv/bin/pip install -e .

Install gunicorn separately:

(venv) behai@HP-Pavilion-15:~/webwork/bh_aust_postcode$ venv/bin/pip install gunicorn

Download postcodes and write to SQLite database file:

(venv) behai@HP-Pavilion-15:~/webwork/bh_aust_postcode$ venv/bin/flask update-postcode

– Please note, the above step is only specific to this project.

Enable port 5000:

$ sudo ufw allow 5000

Run the web server via gunicorn:

(venv) behai@HP-Pavilion-15:~/webwork/bh_aust_postcode$ venv/bin/gunicorn --bind 0.0.0.0:5000 wsgi:app

At this point, the following should be accessible from Windows 10, or other devices connected to the Wifi network for that matter:

I have spent a lot of hours trying to get host name (hp-pavilion-15) to work, no avail so far: it does work if I turn off the firewall – but we don’t want to do that. I think it has something to do with the socket, because we’ve seen the default Nginx site can be accessed via hp-pavilion-15. And also, I’ve reinstalled Jenkins, and I can access it via http://hp-pavilion-15:8080.

Stop the gunicorn web server with Ctrl+C, and deactivate the virtual environment with:

(venv) behai@HP-Pavilion-15:~/webwork/bh_aust_postcode$ deactivate

Configure the virtual host service for the web API

Please note, the current working directory is still /home/behai/webwork/bh_aust_postcode. Run the test service command, this command will be used in the service configuration file, test it first to ensure that it works:

$ /home/behai/webwork/bh_aust_postcode/venv/bin/gunicorn --workers 3 --bind unix:bh_aust_postcode.sock -m 007 wsgi:app

Output:

[2023-05-20 21:20:27 +1000] [5295] [INFO] Starting gunicorn 20.1.0
[2023-05-20 21:20:27 +1000] [5295] [INFO] Listening at: unix:bh_aust_postcode.sock (5295)
[2023-05-20 21:20:27 +1000] [5295] [INFO] Using worker: sync
[2023-05-20 21:20:27 +1000] [5296] [INFO] Booting worker with pid: 5296
[2023-05-20 21:20:27 +1000] [5297] [INFO] Booting worker with pid: 5297
[2023-05-20 21:20:27 +1000] [5298] [INFO] Booting worker with pid: 5298

Create the service file /etc/systemd/system/bh_aust_postcode.service:

$ sudo nano /etc/systemd/system/bh_aust_postcode.service

This is the content, note the directories and the above command:

[Unit]

Description=bh-aust-postcode

After=network.target

[Service]

User=behai
Group=www-data
WorkingDirectory=/home/behai/webwork/bh_aust_postcode

Environment="PATH=/home/behai/webwork/bh_aust_postcode/venv/bin"

ExecStart=/home/behai/webwork/bh_aust_postcode/venv/bin/gunicorn --workers 3 --bind unix:bh_aust_postcode.sock -m 007 wsgi:app

[Install]

WantedBy=multi-user.target

Enable the service:

$ sudo systemctl start bh_aust_postcode

This command has no output. And the next command:

$ sudo systemctl enable bh_aust_postcode

And its output:

Created symlink /etc/systemd/system/multi-user.target.wants/bh_aust_postcode.service → /etc/systemd/system/bh_aust_postcode.service.

Assign port 82 to the web API virtual host

The virtual host configuration file is /etc/nginx/sites-available/bh_aust_postcode, create it with the command:

$ sudo nano /etc/nginx/sites-available/bh_aust_postcode

And its content is:

server {
    listen 82;

    server_name localhost;
    location / {
        include proxy_params;

        proxy_pass http://unix:/home/behai/webwork/bh_aust_postcode/bh_aust_postcode.sock;
    }
}

We’re assigning port 82 to this virtual host, and get Nginx to forward requests to, effectively, /home/behai/webwork/bh_aust_postcode. For explanation on proxy_pass, please see NGINX Reverse Proxy

Next, we must enable the virtual host. Verify that it has not been enabled:

$ sudo ls -l /etc/nginx/sites-enabled

In the output, there should not be an entry for /etc/nginx/sites-available/bh_aust_postcode.

Enable it by creating a link to sites-enabled:

$ sudo ln -s /etc/nginx/sites-available/bh_aust_postcode /etc/nginx/sites-enabled

Verify to ensure it has been enabled, i.e. there should be a symlink:

$ sudo ls -l /etc/nginx/sites-enabled

Among the entries, there should be an entry like the below:

lrwxrwxrwx 1 root root 43 May 20 10:22 bh_aust_postcode -> /etc/nginx/sites-available/bh_aust_postcode

See Difference in sites-available vs sites-enabled vs conf.d directories (Nginx)? for more discussion on this topic.

Open incoming TCP port 82 to all IP addresses:

$ sudo ufw allow from any to any port 82 proto tcp

Output:

Rule added
Rule added (v6)

Restart Nginx

Given that we’ve verified that Nginx configuration valid, we can now restart it, via:

$ sudo systemctl restart nginx

The above command has no output. We should check out the status of Nginx, with:

$ systemctl status nginx.service

We can see that it is active (running), that means we have it working:

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; preset: enabled)
     Active: active (running) since Sat 2023-05-20 10:38:03 AEST; 5s ago
       Docs: man:nginx(8)
    Process: 4154 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 4155 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 4156 (nginx)
      Tasks: 5 (limit: 9363)
     Memory: 4.8M
        CPU: 26ms
     CGroup: /system.slice/nginx.service
             ├─4156 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;"
             ├─4157 "nginx: worker process"
             ├─4158 "nginx: worker process"
             ├─4159 "nginx: worker process"
             └─4160 "nginx: worker process"

May 20 10:38:03 HP-Pavilion-15 systemd[1]: Starting A high performance web server and a reverse proxy server...
May 20 10:38:03 HP-Pavilion-15 systemd[1]: Started A high performance web server and a reverse proxy server.

The simple web API to search for Australian postcodes based on locality virtual host setup is now completed.

Test the final virtual host from Windows 10

At this point, the following should be accessible from Windows 10, or other devices connected to the Wifi network for that matter:

The following screenshots show the API in action. I took these using the previous Ubuntu installation, which was still version 22.10. I did have the host name working (as shown) then, but not now:

Now, the problem is:

067-05.png

Current firewall status

The current status of my firewall.

$ sudo ufw status verbose

[sudo] password for behai:
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere
445                        ALLOW IN    Anywhere
5000                       ALLOW IN    Anywhere
80,443/tcp (Nginx Full)    ALLOW IN    Anywhere
8080                       ALLOW IN    Anywhere
82/tcp                     ALLOW IN    Anywhere
22/tcp (v6)                ALLOW IN    Anywhere (v6)
445 (v6)                   ALLOW IN    Anywhere (v6)
5000 (v6)                  ALLOW IN    Anywhere (v6)
80,443/tcp (Nginx Full (v6)) ALLOW IN    Anywhere (v6)
8080 (v6)                  ALLOW IN    Anywhere (v6)
82/tcp (v6)                ALLOW IN    Anywhere (v6)

Port 5000 which’s been enabled previously is no longer required, it could be deleted.

Concluding remarks

Prior to this post, I have set up several other virtual hosts with my original Ubuntu 22.10 installation, when I started this post, it was probably behind with some updates… When it got damaged, the first reinstallation I did was Ubuntu 23.04 (Lunar Lobster), I could not get host name to work. I reversed back to this 22.10 version, this time, I upgraded all packages, and host name is still not working for this virtual host.

It’s been an interesting exercise… I could not get what I wanted, but it’s been a useful exercise for me, regardless. I’m sure something will turn up, and I will able to fix this annoying problem in the future.

I hope you find the info useful. Thank you for reading and stay safe as always.

✿✿✿

Feature image sources: