Running Multiple Xailient Models on a Single Device
If you need to run more that one Xailient model on a single device, please follow this tutorial.
For this tutorial, we will use two models, pre-trained Car Detector and a pre-trained License Plate Detector.
Device Setup
To meet the minimum requirements, your device must be a x86_64 or ARM32 architecture device. We've tested the following operating systems:
- Ubuntu 18.04 LTS
- Ubuntu 19.04
- Raspbian Buster
Note
To run Xailient SDK on Windows or MacOS, please run it inside a VirtualBox.
Here are some resources you can refer to to install Ubuntu operating system on a VirtualBox:
For ARM32, we have tested on the following device:
- RaspberryPi 3B+
Minimum software requirements:
- Glibc 2.27
- Python3.7
- Pip for python3.7
- OpenCV
- Curl
- VirtualEnv
If your device does not meet the minimum software required, please follow the instructions below to install them.
Download the package lists and update information
$ sudo apt-get update
Note
apt-get update to download the package lists from the repositories and "update" them to get information on the newest versions of packages and their dependencies. It will do this for all repositories and PPAs.
Install python3.7
$ sudo apt-get install python3.7
Install pip for python3.7
$ sudo apt-get install python3-pip
Install OpenCV
$ sudo apt-get install python-opencv
Install Curl
$ sudo apt-get install curl
Install VirtualEnv
$ python3.7 -m pip install --user virtualenv
Install Multiple Models
Create a separate virtual environment for each model you want to install.
For this tutorial, we will create two virtual environments, one to install the Car Detector and another to install License Plate Detector.
Car Detector
Create a Virtual Environment for Car Detector
Create a new python virtual environment to isolate package installation for the system.
$ python3.7 -m venv car_detector_env
Activate the virtual environment.
Before you can start using the virtual environment for package installations, you will need to activate it. Activating a virtual environment will put the virtual environment-specific python and pip executables into your shell's PATH.
$ source car_detector_env/bin/activate
Install Car Detector SDK
Note
In 18.04 Ubuntu, by default pip3 is for python3.6. So we will use python3.7 -m pip command instead to use pip for python3.7. If pip3 for your device defaults to python3.7, feel free to replace the command python3.7 -m pip with pip3.
Get SDK Wheel Link
Go to the Console and navigate to MANAGE AI MODELS. For the model you want to deploy, select the SDK you have build for your target platform.
Note
If you have not build an SDK yet, refer to Build SDK section of the documentation.
Click on the target platform for the model to download and copy the SDK link.
Note
When you click the button <ARM32> or <x86_64>, it downloads the SDK wheel file as well as copies the link to the wheel file in your clipboard.
Install SDK Wheel
Install using downloaded SDK wheel file
(car_detector_env) $ python3.7 -m pip install <download_path/wheel_file_name>
Install using SDK link
(car_detector_env) $ python3.7 -m pip install "<SDK WHEEL LINKs>"
Note
Replace <SDK WHEEL LINK> with the SDK link you copied earlier from the console.
Info about xailient sdk installed
(car_detector_env) $ python3.7 -m pip show xailient
You can get information about the version of xailient sdk installed, support email address, and location of the installation.
Note
Keep note of the install location "Location" as you will need it in the steps below.
Go to Xailient Installation Folder
To go to xailient folder, use the following command. If you do not know the install location, please refer to "Info about xailient sdk installed" section above.
(car_detector_env) $ cd <Xailient Install Location>/xailient
Xailient SDK contents
(car_detector_env) $ ls
- bin/ -- Executables required for initial registration
- data/ -- Image files you can use with the sample applications
- samples/ -- Sample application demonstrating how the Xailient SDK library can be used
- scripts/ -- License activation scripts
- sharedLib_x86_64/ --Sample models that can be installed or linked with your applications
Install tflite runtime
Install the tflite runtime using requirements.txt file.
(car_detector_env) $ python3.7 -m pip install -r requirements.txt
Install flask
(car_detector_env) $ python3.7 -m pip install flask
Activate your license
Go to scripts folder and execute xailient-install script
(car_detector_env) $ cd scripts
(car_detector_env) $ sudo ./xailient-install
Deactivate virtual environment
To deactivate the virtual environment,
(car_detector_env) $ deactivate
License Plate Detector
Create a Virtual Environment for License Plate Detector
Now, create a new python virtual environment for License Plate Detector
$ python3.7 -m venv license_plate_detector_env
Activate the virtual environment.
Before you can start using the virtual environment for package installations, you will need to activate it. Activating a virtual environment will put the virtual environment-specific python and pip executables into your shell's PATH.
$ source license_plate_detector_env/bin/activate
Install License Plate Detector SDK
Download the SDK or get the SDK link from Xailient AI Console.
Go to the Console and navigate to MANAGE AI MODELS. For the model you want to deploy, select the SDK you have build for your target platform.
Note
If you have not build an SDK yet, refer to Build SDK section of the documentation.
Click on the target platform for the model to download and copy the SDK link.
Note
When you click the button <ARM32> or <x86_64>, it downloads the SDK wheel file as well as copies the link to the wheel file in your clipboard.
Install SDK Wheel
Install using downloaded SDK wheel file
(license_plate_detector_env) $ python3.7 -m pip install <download_path/wheel_file_name>
Install using SDK link
(license_plate_detector_env) $ python3.7 -m pip install "<SDK WHEEL LINKs>"
Note
Replace <SDK WHEEL LINK> with the SDK link you copied earlier from the console.
Info about xailient sdk installed
(license_plate_detector_env) $ python3.7 -m pip show xailient
You can get information about the version of xailient sdk installed, support email address, and location of the installation.
Note
Keep note of the install location "Location" as you will need it in the steps below.
Go to Xailient Installation Folder
To go to xailient folder, use the following command. If you do not know the install location, please refer to "Info about xailient sdk installed" section above.
(license_plate_detector_env) $ cd <Xailient Install Location>/xailient
Xailient SDK contents
(license_plate_detector_env) $ ls
- bin/ -- Executables required for initial registration
- data/ -- Image files you can use with the sample applications
- samples/ -- Sample application demonstrating how the Xailient SDK library can be used
- scripts/ -- License activation scripts
- sharedLib_x86_64/ --Sample models that can be installed or linked with your applications
Install tflite runtime
Install the tflite runtime using requirements.txt file.
(license_plate_detector_env) $ python3.7 -m pip install -r requirements.txt
Install flask
(license_plate_detector_env) $ python3.7 -m pip install flask
Activate your license
Since you have already activated your license for this device earlier, YOU DO NOT NEED TO REPEAT THIS STEP.
Deactivate virtual environment
To deactivate the virtual environment,
(license_plate_detector_env) $ deactivate
Setup Flask Server
Now, we will setup two flask servers, one for Car Detector and another for License Plate Detector, so we can access both models using REST API.
Note
Make sure to use different ports for different flask servers.
Flask server code for Car Detector
file_name: flask_server_car_detector.py
from xailient import dnn
from flask import Flask, jsonify, request
import cv2
import numpy
import time
import json
app = Flask(__name__)
detectum = dnn.Detector()
@app.route('/process', methods=['POST'])
def process_frame():
start_time = time.time()
data = request.files['file']
im = cv2.imdecode(numpy.fromstring(request.files['file'].read(), numpy.uint8), cv2.IMREAD_UNCHANGED)
THRESHOLD = 0.4 # Value between 0 and 1 for confidence score
inference_start_time = time.time()
_, bboxes = detectum.process_frame(im, THRESHOLD)
inference_end_time = time.time()
# Loop through list (if empty this will be skipped) and overlay green bboxes
# Format of bboxes is: xmin, ymin (top left), xmax, ymax (bottom right)
response = {}
boxes = []
for i in bboxes:
bbox = {}
bbox["xmin"] = int(i[0])
bbox["ymin"] = int(i[1])
bbox["xmax"] = int(i[2])
bbox["ymax"] = int(i[3])
boxes.append(bbox)
cv2.rectangle(im, (i[0], i[1]), (i[2], i[3]), (0, 255, 0), 3)
response["bboxes"] = boxes
response["class"] = "car"
cv2.imwrite("output.jpg", im)
print(response)
#return "hello"
return jsonify(response)
app.run("0.0.0.0", port=8001)
Flask server code for License Plate Detector
file_name: flask_server_license_plate_detector.py
from xailient import dnn
from flask import Flask, jsonify, request
import cv2
import numpy
import time
import json
app = Flask(__name__)
detectum = dnn.Detector()
@app.route('/process', methods=['POST'])
def process_frame():
start_time = time.time()
data = request.files['file']
im = cv2.imdecode(numpy.fromstring(request.files['file'].read(), numpy.uint8), cv2.IMREAD_UNCHANGED)
THRESHOLD = 0.4 # Value between 0 and 1 for confidence score
inference_start_time = time.time()
_, bboxes = detectum.process_frame(im, THRESHOLD)
inference_end_time = time.time()
# Loop through list (if empty this will be skipped) and overlay green bboxes
# Format of bboxes is: xmin, ymin (top left), xmax, ymax (bottom right)
response = {}
boxes = []
for i in bboxes:
bbox = {}
bbox["xmin"] = int(i[0])
bbox["ymin"] = int(i[1])
bbox["xmax"] = int(i[2])
bbox["ymax"] = int(i[3])
boxes.append(bbox)
cv2.rectangle(im, (i[0], i[1]), (i[2], i[3]), (0, 255, 0), 3)
response["bboxes"] = boxes
response["class"] = "plate"
cv2.imwrite("output.jpg", im)
print(response)
#return "hello"
return jsonify(response)
app.run("0.0.0.0", port=8002)
Start Flask Servers on different virtual environments
Since our Car Detector is insalled in one virtual environment and License Plate is installed in another, we need to run the above flask servers on their respective virtual environments.
Here is a bash script to do that:
file_name: start.sh
#!/bin/sh
chmod +x flask_server_car_detector.py
chmod +x flask_server_license_plate_detector.py
/<path-to-virtual-env>/car_detector_env/bin/python3.7 flask_server_car_detector.py &
/<path-to-virtual-env>/license_plate_detector_env/bin/python3.7 flask_server_license_plate_detector.py
Now run the bash script to start these servers:
sudo ./start.sh
Write a python script to send images to these detectors using REST API and get back inference.
# install request using pip3 install requests
import requests
import json
def run_car_detector(image_path):
url = 'http://127.0.0.1:8001/process'
files = {'file': open(image_path, 'rb')}
response = requests.post(url, files=files)
res_json = json.loads(response.text)
print(res_json)
return res_json
def run_licen_pate_detector(image_path):
url = 'http://127.0.0.1:8002/process'
files = {'file': open(image_path, 'rb')}
response = requests.post(url, files=files)
res_json = json.loads(response.text)
print(res_json)
return res_json