10. Annex 1. AI Models Server installation procedure

This Annex describes the step-by-step procedure for installing an AI Models Server instance based on TensorFlow Serving. It also describes the basics for configuring the AI Models deployed on it.

Note

The installation is based on Docker Containers, following the procedure described in this TensorFlow Serving Tutorial repo.

10.1. Installing TensorFlow Serving

Let’s execute the following steps:

  1. Clone the repo in your home directory:

    $ git clone https://github.com/brianalois/tensorflow_serving_tutorial.git
    
  2. Install Docker:

    $ sudo apt install docker.io
    
  1. Build the Dockerfile inside the tutorial repo folder (this process may take some minutes):

    $ cd tensorflow_serving_tutorial
    $ sudo docker build --pull -t test-tensorflow-serving.
    
  2. Once the image is built, you can proceed to run it using:

    $sudo docker run -it -p 8500:8500 -p 8501:8501 -v /home/ubuntu/serving/tensorflow_serving/tools/docker/tensorflow_serving_tutorial/model_volume/:/home/ test-tensorflow-serving
    

    Warning

    This command cannot be copy-pasted. You need to use your own absolute path for the shared volume (-v). You can use “pwd” in your directory to copy the path.

If everything worked properly, you should now see the shell of the container (you will see a different number in the prompt):

root@15954c4d0666:/#
  1. Once inside the container’s terminal, you can deploy the server with:

    root@15954c4d0666:/# tensorflow_model_server --port=8500 --rest_api_port=8501 --model_config_file=/home/configs/models.conf
    

This command starts the server on port 8501 with some by-default AI Models already defined in the models.conf file. By default, the TensowrFlow Serving repo provides two models, named “xor” and “iris” (the “xor” model is an implementation of the well-known XOR Problem; the “iris” model is a sample model that can be used to classify different types of the Iris flower based on certain parameters).

  1. Now that we have the server up and running let’s test it; you can do it by sending ‘curl’ requests to the AI Models Server; for instance, this one for quering the “iris” model:

    curl -X POST http://localhost:8501/v1/models/iris:classify -H 'cache-control: no-cache' -H 'postman-token: f7fb6e3f-26ba-a742-4ab3-03c953cefaf5' -d '{ "examples":[{"x": [5.1, 3.5, 1.4, 0.2]}]}'
    

We should receive a response similar to this one:

{
   "results": [
    [
        [
            "Iris-setosa",
            0.872396886
        ],
        [
            "Iris-versicolor",
             0.108623177
        ],
        [
            "Iris-virginica",
             0.0189798642
        ]
    ]
  ]
}

(it seems the model is clasifiying the provided flower parameters as “Iris-setosa”; anyway, we will not enter in the details about the AI Model itself).

10.2. Configuring AI Models

The models available in the TensorFlow Serving are described in the ‘models.conf’ configuration file. It is possible to define multiple models (with multiple versions also) in a single docker container, which will be available through a common port.

This is the configuration for the by-default “xor” and “iris” models we’ve been mentioning above:

{ config: {
       name: "xor",
       base_path: "/home/models/xor",
       model_platform: "tensorflow",
       model_version_policy: { all: {} }
     },
  config: {
   name: "iris",
   base_path: "/home/models/iris",
   model_platform: "tensorflow",
   model_version_policy: { all: {} }
  }
}

As we can see, the configuration file is a list of config objects. Each of them has:

  • name : the name of the model

  • base_path: indicates the directory in which the model itself is stored

  • model_platform: platform used for the model (‘tensorflow’ by-default)

  • model_version_policy: allows to define which version of the model will be in use (different versions are allowed).

To see how to generate and configure your own models you can check the step-by-step guide in Section 3.1. Also, for additional information on the AI Models configuration you can access to the official TensorFlow Serving website.