Production Environment with ML Model

Our task is to deploy a Python application implementing REST API tasks. Upon client request, the server will respond based on predictions generated from a certain model.

This image will be containerized using a Dockerfile, making server deployment possible regardless of the platform.

We will accomplish our task using the Flask library version 3.0.3.

There are other libraries that can accomplish this task.

Check at home if you can use them yourself, or at least read the code.

  1. FastAPI
  2. Seldon
  3. MLFlow
  4. Node.js - Express

Minimal Flask Application Code

We want to run our application locally and then easily transfer and execute it on any computer. Therefore, a natural solution is to save the code to a file with the .py extension.

To automatically save the application code to the app.py file, we will use the magic command %%file file.py.

%%file app.py
from flask import Flask

# Create a flask
app = Flask(__name__)

# Create an API end point
@app.route('/')
def say_hello():
    return "Hello World"

if __name__ == '__main__':
    app.run()

Note! In the Flask documentation, the last two lines starting the server are not present in the sample code. Furthermore, the command to start the server is flask run, not python app.py. Let’s explain what the sample code contains:

  1. from flask import Flask: Imports the Flask library.
  2. app = Flask(__name__): Creates the API server interface.
  3. Subpage code using a decorator: Here, a route of the application is defined. Using a decorator (e.g., @app.route(‘/’)), you specify which URL address should trigger the associated function. Typically, the function returns the content that the client will receive when accessing this path.
@app.route('/')
def say_hello():
    return "Hello World"

To demonstrate how the decorator works, let’s define the following function:

def make_pretty(func):
    def inner():
        print("decorator working")
        func()
    return inner()
def test():
    print("abc")

make_pretty(test)
decorator working
abc
@make_pretty
def test2():
    print("test2")
decorator working
test2
@make_pretty
def test3():
    print("anything else")
decorator working
anything else

Python Environment

In order for the application code in app.py to run, we need a Python interpreter installed on our computer. However, simply having an interpreter is not sufficient for our application. To fully run it, we need to create an environment (preferably virtual) where all the necessary libraries (e.g., Flask) are available.

Note: All terminal commands will be for Linux/Mac OS versions.

which python
which python3
which pip 
which pip3

All these commands should point to the folder with the default Python environment.

Generate and run a virtual environment locally by entering the following command in the terminal:

python3 -m venv .venv
source .venv/bin/activate

Good practice: a Python environment is nothing more than a directory. In our version, it’s a hidden directory named .venv. If you copy this directory elsewhere, it will cease to function as your Python environment. Therefore, recreating it does not involve copying it. If your project is associated with a version control system like GIT, make sure the environment directory is not added to the repository. You can achieve this by adding the appropriate entry to the .gitignore file.

Having created a new environment, check which libraries are present in it.

pip list 

Package    Version
---------- -------
pip        23.2.1
pyspark    3.4.1
setuptools 65.5.0

We can check the Python and pip commands again

which python
which pip

By default, the libraries pip and setuptools should appear (pyspark comes from our internal image).

Install the Flask library:

pip install flask
pip list 
Package      Version
------------ -------
blinker      1.7.0
click        8.1.7
Flask        3.0.3
itsdangerous 2.1.2
Jinja2       3.1.3
MarkupSafe   2.1.5
pip          23.2.1
pyspark      3.4.1
setuptools   65.5.0
Werkzeug     3.0.2

As you can see, installing the Flask library also forced the installation of other packages.

The only way to transfer the Python environment is to install it on a new machine and manually install all the packages. However, to avoid installing each package individually, we can use a configuration file called requirements.txt with a list of packages.

Remember - each package should include the version number. Otherwise, it may turn out that new package versions break compatibility with your code. To create the configuration file, use the following command in the terminal:

pip freeze >> requirements.txt

You can use the generated file on any machine to install and recreate the required Python runtime environment.

Side note: At the time of preparing the materials, Flask was in version 3.0.1 - today it is already available in version 3.0.3.

To install packages from the file, use the following command:

pip install -r requierements.txt

We now have two files: app.py and requirements.txt.

By moving them to any project on GitHub, we can run our application wherever a Python interpreter is available, allowing us to create a new virtual environment and install libraries from the requirements.txt file.

For full automation, it would be useful to be able to run the Python environment on any machine.

To achieve this, create a Dockerfile:

%%file Dockerfile
FROM python:3.11-slim-buster

WORKDIR /app

COPY requirements.txt requirements.txt

RUN pip install -r requirements.txt

COPY app.py .

ENV FLASK_APP=app

EXPOSE 8000
CMD ["flask", "run", "--host", "0.0.0.0", "--port", "8000"]

The above file allows Docker Desktop to run an image utilizing a basic operating system (here Linux) along with a basic Python 3.11 environment.

Furthermore, this file copies necessary files (app.py, requirements.txt) into the Docker image.

The RUN command allows executing any bash command inside the Docker image.

The CMD command allows running a command to start the server in a mode that won’t close this command.

The last piece of information is setting the port to 8000.

# creating docker container from Dockerfile
docker build -t modelML .
# run container
docker run -p 8000:8000 modelML

Running the Server Locally

The server can be started in at least two ways.

Starting the Server via Terminal

python app.py

or (if there is no app.run() code starting the server).

flask run

You should see information similar to the following

 * Serving Flask app 'app'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit

Running the Server via Notebook

Directly running the code in a notebook will start the server and halt any further code execution. To avoid this, you can use the subprocess library.

import subporcess

p = subprocess.Popen(["python", "app.py"])

If we need to close the subprocess, execute

p.kill()

With the server running, you can query it using:

curl localhost:5000

or in jupyter notebook:

import requests

response = requests.get("http://127.0.0.1:5000/")

print(response.content) # Hello World
print(response.status_code) # 200