Follow

Model Manager Overview

Model Management

Domino lets you publish R or Python models as web services and invoke them using our REST API, so you can easily integrate your data science projects into existing applications and business processes without involving engineers or devops resources.  

Overview

A model is a REST API Endpoint that is a wrapper on top of a function in your code. The arguments to the API are the arguments to your function; the response from the API includes the return value from your function. Typically this function will make a prediction or classification given some input.

When the model is published, Domino first runs the script containing the function. The process then waits for input (so any objects or functions from the script remain available in-memory). Each call to the endpoint runs the function within this same process. Because the script is only sourced once — at publish time — any more expensive initialization can happen up front, rather than happening on each call. We have a short video (coming soon!) to demonstrate how models are published.

Key Model Management Features

  • Production-Grade Infrastructure - Models are vertically and horizontally scalable, and are highly reliable. See this for more details. 
  • Versioning and Reproducibility - Models are versioned and each version can be redeployed giving you the ability to revert to previous good states. Models depend on V2 Environments which gives you full reproducibility. 
  • Discoverability & Access Control - A model is also a top level entity that is separate from projects.
    • The Models tab on the top level nav allows users to discover and view models that they have access to.
    • Models have their own set of permissions which allows for organizations to split out the group that develops models from the one that deploys or maintains them. See this for more details
    • Models also come with audit logs so you can track who took which actions
    • You can control whether your model is publicly invocable or requires an access token. See this for more details
  • Promote to Production Workflow - We support an advanced routing mode which allows you to have a promote to production workflow where you can test with one version and take production traffic on another version. See this for more details. 
  • Flexibility - Models can be deployed from multiple projects though we expect most people to deploy from a single project. 

Setting up your Compute Environment 

Environment Selection

The Model Manager requires a V2 Environment to work. When you deploy the model manager it will only give you V2 environments in the environment selection dropdown. If your project is currently set to a classic or V1 environment, you will need to select or create a V2 environment that has three specific packages for Model Manager to work. These three packages can be added by adding the line 

RUN pip install flask==0.12 jsonify==0.5 uwsgi==2.0.14

to a new V2 environment based off of your existing V1 or V2 environment. If you need help with this, please contact us. 

A note about requirements.txt

Model Manager does not read your requirements.txt. If your project currently uses requirements.txt to install certain packages or repositories, you must add them to your V2 environment. This allows your model to build and run faster and centralizes a single source of truth for the compute environment needed to run reproducible, versioned models. 

Once you have created your V2 environment, you can choose to update your project to point to it, so that your runs and models use the same environment. At that point you can also get rid of your requirements.txt and instead update and rebuild your environment whenever you need to make changes. 

Environment Variables

Your model does not inherit the environment variables set at the project level but you can set model specific environment variables on the Settings/Environment tab. This is intended to decouple the management of projects and models. See this for more details.  

Publishing your Model

Your script has access to your project's files, including serialized model files (e.g., pickle or Rda). So you can save a model with a long-running training task, and then read it into memory with your endpoint script for classification.

There are three ways to publish a model: (1) through the UI; (2) through a scheduled run; and (3) COMING SOON programmatically through the Domino API or CLI; .

  1. In the UI, click on "Publish" on the left-hand sidebar. Click over to the Models tab. This brings you to a page where you can input the name of your model, name of the script and function you wish to publish. 
  2. Scheduled runs can automatically update your model. So you can set up scheduled, long-running training tasks to re-deploy with the latest version of your model. 
  3. COMING SOON To programmatically re-publish a model after a run, start a run from the CLI using the flag --publish-model:
    domino run --publish-model command arguments
  4. COMING SOON Alternatively, use the Domino API to start a run and set the publishModel parameter to true. If the run is successful it will be deployed to your project's endpoint.

Calling your Model

The Model Manager supports a couple of different ways of setting up your API's endpoint URL. See this for more details.

On the Overview page of a model, we have a model tester. It gives you examples of how to call your model. On the first tab, the "Tester" you can type in your input query and see the model response directly. On the tester tab you do not need to add your access token. To construct your input JSON, you can use either a dictionary or an array. 

Each element of the list will be passed to the function as positional or named arguments. The elements themselves may be lists or arbitrary objects, as long as they are valid JSON.

If you're using named parameters in your function definition, for example: 

my_function(x, y, z)

you can use either a data dictionary or a parameter array:

{"data": {"x": 1, "y": 2, "z": 3}}, or 
{"parameters": [1, 2, 3]}

 

If you're using a dictionary in your function definition, for example:

my_function(dict)

and your function then uses dict["x"], dict["y"] etc, you can use only a parameter array:

{"parameters": [{"x": 1, "y": 2, "z": 3}]}

to call your model.

 

In Python, you can also use **kwargs to pass in a variable number of arguments. If you do this:

my_function(x, **kwargs)

and your function then uses kwargs["y"] and kwargs["z"], you can use the data dictionary to call your model:

{"data": {"x": 1, "y": 2, "z": 3}}

 

Other tabs give you example code snippets of how to call your model in different languages though it currently does not fill in your API specific formatted inputs. 

Domino will take care of converting the inputs to the proper types in the language of your endpoint function:

JSON Type Python Type R Type
dictionary dictionary named list
array list list
string str character
number (int) int integer
number (real) float numeric
true True TRUE
false False FALSE
null None NA

The model's output is contained in the result object which can be a literal, array or dictionary.  

Updating your Model

You can publish a new version of your model at any time. For example, you may want to re-train the model with new data, or switch to a different machine learning algorithm.

You can also unpublish the model and Domino will stop serving it. Note that if you are using advanced routing, and you have a promoted model, you cannot currently unpublish it. 

Examples in Python and R

The example referenced here is found in this project: https://app.dominodatalab.com/u/nick/winequality. Feel free to fork it and try for yourself.

Python

The model is first trained using train.py, and stored at results/classifier.pkl:

import pandas as pd
from sklearn.ensemble import RandomForestClassifier 
from sklearn import svm
from sklearn import cross_validation
import time
import joblib
 
train = pd.read_csv("./winequality-red.csv", sep=";")
cols = train.columns[:11]

clf = RandomForestClassifier(n_estimators=100, max_features=5, min_samples_split=5)

start = time.time()
clf.fit(train[cols], train.quality)

print "training time:", time.time() - start

print "cross validation scores", cross_validation.cross_val_score(clf, train[cols], train.quality, cv=5)

joblib.dump(clf, 'results/classifier.pkl')

The endpoint is bound to the predict(...) function, defined in predict.py (which loads and uses results/classifier.pkl):

import joblib
import numpy as np

clf = joblib.load("results/classifier.pkl")


def predict(features):
	return np.asscalar(clf.predict(features))

R

The model is trained using train.R, and stored at classifier.Rda:

library(randomForest)

df <- read.csv("./winequality-red.csv", h=T, sep = ";")
df$quality <- factor(df$quality)

cols <- names(df)[1:11]
system.time({
  clf <- randomForest(df$quality ~ ., data=df[,cols], ntree=30, nodesize=4, mtry=8)
})


save(clf, file="classifier.Rda")
save(cols, file="cols.Rda")

The endpoint is bound to the predictQuality(...) function, defined in predict.R (which loads and uses classifier.Rda):

library(randomForest)

load("classifier.Rda")
load("cols.Rda")

predictQuality <- function(features) {
  to_predict <- as.data.frame(features)
  colnames(to_predict) <- cols    
  prediction <- predict(clf,  to_predict)
  cbind(prediction)[1]
}

Example API Request

In either case, once published, a request might look something like this:

curl -v -X POST 'https://app.dominodatalab.com/models/<modelId>/latest/model' \
-H 'Content-Type: application/json' \
-u 'YOUR_API_KEY:YOUR_API_KEY' \
-d '{"parameters": [[5.6, 0.615, 0, 1.6, 0.089, 16, 59, 0.9943, 3.58, 0.52, 9.9]]}'

Note that the version of your API endpoint is included in the API response.

Example API Response

The API call above returns the following JSON. The model's output is returned in the "result" object while the "release" object contains information about the version of your API endpoint and the file and function which powers it.

{
    "release": {
        "model_version": "591b2d598c943ee48d869ce4",
"model_version_number": 2 }, "result": 5 }


 

Common issues, pitfalls, and other tips

dataTypeError: don't know how to serialize class
You may see this error with Python endpoints if you return values that are NumPy objects, rather than native Python primitives. To fix this, just convert your NumPy values to Python primitives. An easy way to do this is to call numpy.asscalar, described in this Stack Overflow answer.

TypeError: <result> is not JSON serializable
The result of your endpoint function gets serialized to JSON before being sent in the response. However some object types, such as Decimal or certain NumPy data types, are not JSON serializable by default. For Decimal data types, cast the Decimal type to a Float type as described in this Stack Overflow answer. For NumPy data types, convert the values to Python primitives. An easy way to do this is to call numpy.asscalar as described above.

 

Was this article helpful?
0 out of 0 found this helpful

Comments