Exposing ML/DL Models as REST APIs

In this article, we will learn how to expose ML/DL model as flask APIs.

Frameworks we’ll be using:
Keras is a Deep Learning library, built on top of backends such as Tensorflow, Theano or CNTK. It provides abstraction and allows rapid development of ML/DL models.

Flask is a micro web framework in Python, which is used to quickly spin up servers to serve pages. Refer Introduction to Flask.

Since the focus of the article is to serve a ML/DL model using an API, we wouldn’t get deeper into the making of the Convolutional model. We will be using the ResNet50 Convolutional Neural Network.

Install Tensorflow and Keras

Step Linux/Mac Windows
Update pip pip install -U pip python -m pip install -U pip
Install virtualenv sudo pip install virtualenv pip install virtualenv
Create a new folder ml-api mkdir ml-api && cd ml-api mkdir ml-api && cd ml-api
Create a virtual environment(Good practice) and activate it virtualenv –system-site-packages -p python3 ./venv virtualenv –system-site-packages -p python3 ./venv
Activate the virtual environment source ./venv/bin/activate

.\venv\Scripts\activate
Install TensorFlow pip install –upgrade tensorflow pip install –upgrade tensorflow
Install Keras pip install keras pip install keras
Install other dependencies pip install flask gevent requests pillow pip install flask gevent requests pillow

Examples:

Creating the REST API :

filter_none

edit
close

play_arrow

link
brightness_4
code

# keras_server.py
  
# Python program to expose a ML model as flask REST API
  
# import the necessary modules
from keras.applications import ResNet50 # pre-built CNN Model
from keras.preprocessing.image import img_to_array 
from keras.applications import imagenet_utils
import tensorflow as tf
from PIL import Image
import numpy as np
import flask
import io
  
# Create Flask application and initialize Keras model
app = flask.Flask(__name__)
model = None
  
# Function to Load the model 
def load_model():
      
    # global variables, to be used in another function
    global model     
    model = ResNet50(weights ="imagenet")
    global graph 
    graph = tf.get_default_graph()
  
# Every ML/DL model has a specific format
# of taking input. Before we can predict on
# the input image, we first need to preprocess it.
def prepare_image(image, target):
    if image.mode != "RGB":
        image = image.convert("RGB")
      
    # Resize the image to the target dimensions
    image = image.resize(target) 
      
    # PIL Image to Numpy array
    image = img_to_array(image) 
      
    # Expand the shape of an array,
    # as required by the Model
    image = np.expand_dims(image, axis = 0
      
    # preprocess_input function is meant to
    # adequate your image to the format the model requires
    image = imagenet_utils.preprocess_input(image) 
  
    # return the processed image
    return image
  
# Now, we can predict the results.
@app.route("/predict", methods =["POST"])
def predict():
    data = {} # dictionary to store result
    data["success"] = False
  
    # Check if image was properly sent to our endpoint
    if flask.request.method == "POST":
        if flask.request.files.get("image"):
            image = flask.request.files["image"].read()
            image = Image.open(io.BytesIO(image))
  
            # Resize it to 224x224 pixels 
            # (required input dimensions for ResNet)
            image = prepare_image(image, target =(224, 224))
  
        # Predict ! global preds, results
            with graph.as_default():
                preds = model.predict(image)
                results = imagenet_utils.decode_predictions(preds)
                data["predictions"] = []
  
          
            for (ID, label, probability) in results[0]:
                r = {"label": label, "probability": float(probability)}
                data["predictions"].append(r)
  
            data["success"] = True
  
    # return JSON response
    return flask.jsonify(data)
  
  
  
if __name__ == "__main__":
    print(("* Loading Keras model and Flask starting server..."
        "please wait until server has fully started"))
    load_model()
    app.run()

chevron_right


Run the flask server

python keras_server.py 

 
Note #1: For the first time you run it, it takes time to download the weights of the model. In subsequent runs, this won’t be the case.
Note #2: Save images in the same directory (ml-api) – dog.jpg, cat.jpg, lion.jpg .

Make requests to the server:

Method #1 : use cURL (Download cURL for Windows from here.)

$ curl -X POST -F image=@dog.jpg "http://localhost:5000/predict"

 
Method #2 : (Suitable if making a Full-stack web app with proper UI)
Create a simple HTML form

filter_none

edit
close

play_arrow

link
brightness_4
code

<!-- index.html -->
<form action = "http://localhost:5000/predict" method = "POST" 
         enctype = "multipart/form-data">
         <input type = "file" name = "image" />
         <input type = "submit"/>
</form>

chevron_right


 
Method #3 : Create a simple python script to make HTTP requests to flask server.

filter_none

edit
close

play_arrow

link
brightness_4
code

# simple_request.py
import requests
import sys
  
  
# provide image name as command line argument
IMAGE_PATH = sys.argv[1
  
image = open(IMAGE_PATH, "rb").read()
payload = {"image": image}
  
# make request to the API
request = requests.post(URL, files = payload).json()
  
if request["success"]:
    # Print formatted Result
    print("% s % 15s % s"%("Rank", "Label", "Probability"))
    for (i, result) in enumerate(request["predictions"]):
        print("% d.    % 17s %.4f"%(i + 1, result["label"],
            result["probability"]))
  
    else:
        print("Request failed")

chevron_right


Run it as :

python simple_request.py dog.jpg

Output:

To deactivate after you are done(both Windows and Linux) :

> deactivate

So the workflow, in general for any model is

  • Build the model and save it.
  • Create a Flask app.
  • Load the Model
  • Define endpoints of API for each type of requests.
  • Preprocess input according to the Model’s architecture and feed it to the model
  • Process the input and send back the output to client.


My Personal Notes arrow_drop_up

Computer Science Student at NIT Tiruchirappalli

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.