Domino standard environments

Overview

Each run and workspace in Domino operates in its own Docker container. These Docker containers are defined by Domino environments. Environments can be shared and customized, and they are automatically versioned by Domino.

New installations of Domino come with a standard set of environments and associated Docker images. Periodically, Domino publishes a new set of standard environments with updated libraries and packages. This article describes how to adopt the latest Domino standard environments.

Standard Environments

Domino maintains three standard compute environments:

  • Domino Analytics Distribution (DAD) for Python 2.7
  • Domino Analytics Distribution (DAD) for Python 3.6
  • Domino Analytics Distribution (DAD) for Python 3.7

The Domino Analytics Distributions are designed to handle most of what a typical data science workflow needs out of the box, and differ only on which version of Python they have installed.

These images are hosted on Quay.io, a third-party Docker registry. By default, your Domino deployment comes configured with keys to access Domino’s images in this service.

The URLs for the latest versions of Domino Standard Environments are:

  • DAD Python 2.7: quay.io/domino/base:Ubuntu18_DAD_Py2.7_R3.5-20190501
  • DAD Python 3.6: quay.io/domino/base:Ubuntu18_DAD_Py3.6_R3.5-20190501
  • DAD Python 3.7: quay.io/domino/base:Ubuntu18_DAD_Py3.7_R3.5-20190501

Contents of the Base Environments: 

  • Ubuntu 18.04
  • Anaconda Python 2.7, 3.6 or 3.7 
  • R 3.5.3
  • Jupyter Lab 0.35.4
  • Jupyter 1.0
  • Rstudio 1.2
  • VSCode
  • Keras 2.2.4 and Tensorflow-gpu* 1.13.1
  • CUDA 10.0.130 with Nvidia driver 410.104
  • Open JDK 8
  • Scala 2.12.6
  • Julia 1.1
  • Node 8.10
  • Various common utilities (Curl, vim, AWSCLI, telnet, etc) 

See here for a full list of python and R packages: 

*note: to use Tensorflow on a cpu machine you must install Tensorflow's cpu version (e.g. pip install tensorflow).

Identifying standard environments in your deployment

To determine which version of Domino standard environments your deployment is using, open Domino and click Environments in the top navigation bar. You can use the search bar to find environments with Domino Analytics in the name. However, previous administrators or installers may have added these environments under a different name. To confirm that an environment is using a Domino standard, click on it to open the Overview.

domino-ded-base-environment.png

Note:

Your installation of Domino may be set up with a previous version of our standard base image. Examples of historical base images:

  • quay.io/domino/base:Ubuntu16_DAD_Py2.7_R3.4-20180727
  • quay.io/domino/base:Ubuntu16_DAD_Py3.6_R3.4-20180727
  • quay.io/domino/base:DAD_py2.7_R3.4_23052018
  • quay.io/domino/base:DAD_py3.6_R3.4_23052018
  • quay.io/domino/base:DED_py2.7_R3.4_23052018
  • quay.io/domino/base:DED_py3.6_R3.4_23052018
  • quay.io/domino/base:2016-12-07_1239_flat2

You may also have a custom base image with your companies name in the URL.

If you are using one of these older images, you may require an upgrade to your system before you are able to utilize the Ubuntu 16.04 or 18.04 images first released in August 2018.

Adding standard environments to your deployment

Adding a Domino standard environment to your deployment involves four high-level steps.

  1. Create a new environment with the correct defaults for the standard you want to add.

    • See Compute Environment Management for an overview of how to create and manage environments.

    • See the table of environment details below to populate the Base ImageDockerfile Instructions, and Pluggable Workspace Tools fields. These fields must be filled out correctly for the environment to work as intended.

    • Critical Note: To use the latest version of the base image, you must have the Pluggable Workspace Tools feature turned on. 
  2. Copy over any additional customization from your existing default compute environment.

    • If you added additional packages and libraries to your previous default environment, and you intend to use the new standard environment as your deployment default, you should copy over those additions to the new environment you created. These can be added to the end of the Dockerfile Instructions in the new environment.

    • You should not copy over any instructions that update Python, R, RStudio, or other core components of the standard environments if the new standard already includes the version you want.

    • Be sure to copy over any setup scripts and run scripts attached to your previous default, as your users may depend on them.

  3. Update the Nvidia drivers on your executor template (AWS customers using GPUs only) or Update the Nvidia on your GPU executor (On Premise customers using GPUs only)

    • Only all other changes you make make to a compute environment when using packages optimized for GPUs (e.g. Tensorflow-gpu and CUDA) there is a dependency between the host machine and the container. In this case the nvidia drivers must match.

    • Please reach out to support@dominodatalab.com
  1. Create a new machine image for your executor template that includes the new environment. (AWS customers only)

    • See Creating a new AMI for instructions.

    • Once this is done, you can set the new environment as the deployment default.

    • You may also choose to archive the previous default environment. Any existing projects using an archived environment will be able to continue using it, but new projects will not have access to it.

Detailed information about standard environments

This section contains detailed specifications about the two Domino Standard Environments, plus the configuration settings needed when adding them to your deployment.

Domino Analytics Distribution (DAD) for Python 2

  1. Title:

    Domino Analytics Distribution py2.7 R3.5

  2. Description:

    Ubuntu 18.04
    Anaconda Python 2.7
    R 3.5.3
    Jupyter Lab 0.35.4, Jupyter 1.0, Rstudio 1.2, VSCode
    Keras 2.2.4 and Tensorflow-gpu* 1.13.1
    CUDA 10.0.130 with Nvidia driver 410.104
    For python packages run: !pip freeze
    For R packages run: installed.packages()
    For further detail, please ask Domino Support for the full Dockerfile
  3. Base Image URL:

    quay.io/domino/base:Ubuntu18_DAD_Py2.7_R3.5-20190501

  4. Pluggable Workspace Tools

    jupyterlab:
    title: "JupyterLab (Beta)"
    iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/jupyterlab.svg?sanitize=true"
    start: [ /var/opt/workspaces/Jupyterlab/start.sh ]
    httpProxy:
    internalPath: "/{{#if pathToOpen}}/lab/tree/{{pathToOpen}}{{/if}}"
    port: 8888
    rewrite: false
    rstudio:
    title: "RStudio"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos//Rstudio.svg?sanitize=true"
    start: [ "/var/opt/workspaces/rstudio/start" ]
    httpProxy:
    port: 8888
    vscode:
    title: "vscode"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/vscode.svg?sanitize=true"
    start: [ "/var/opt/workspaces/vscode/start" ]
    httpProxy:
    port: 8888
    jupyter:
    title: "Jupyter (Python, R, Julia)"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/Jupyter.svg?sanitize=true"
    start: [ "/var/opt/workspaces/jupyter/start" ]
    httpProxy:
    port: 8888
    rewrite: false
    internalPath: "/{{#if pathToOpen}}tree/{{pathToOpen}}{{/if}}"
    supportedFileExtensions: [ ".ipynb" ]

Domino Analytics Distribution (DAD) for Python 3.6

  1. Title:

    Domino Analytics Distribution py3.6 R3.5

  2. Description:

    Ubuntu 18.04
    Anaconda Python 3.6
    R 3.5.3
    Jupyter Lab 0.35.4 ,Jupyter 1.0 ,Rstudio 1.2, VSCode
    Keras 2.2.4 and Tensorflow-gpu* 1.13.1
    CUDA 10.0.130 with Nvidia driver 410.104
    For python packages run: !pip freeze
    For R packages run: installed.packages()
    For further detail, please ask Domino Support for the full Dockerfile
  3. Base Image URL:

    quay.io/domino/base:Ubuntu18_DAD_Py3.6_R3.5-20190501

  4. Pluggable Workspace Tools

     jupyterlab:
    title: "JupyterLab (Beta)"
    iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/jupyterlab.svg?sanitize=true"
    start: [ /var/opt/workspaces/Jupyterlab/start.sh ]
    httpProxy:
    internalPath: "/{{#if pathToOpen}}/lab/tree/{{pathToOpen}}{{/if}}"
    port: 8888
    rewrite: false
    rstudio:
    title: "RStudio"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos//Rstudio.svg?sanitize=true"
    start: [ "/var/opt/workspaces/rstudio/start" ]
    httpProxy:
    port: 8888
    vscode:
    title: "vscode"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/vscode.svg?sanitize=true"
    start: [ "/var/opt/workspaces/vscode/start" ]
    httpProxy:
    port: 8888
    jupyter:
    title: "Jupyter (Python, R, Julia)"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/Jupyter.svg?sanitize=true"
    start: [ "/var/opt/workspaces/jupyter/start" ]
    httpProxy:
    port: 8888
    rewrite: false
    internalPath: "/{{#if pathToOpen}}tree/{{pathToOpen}}{{/if}}"
    supportedFileExtensions: [ ".ipynb" ]

Domino Analytics Distribution (DAD) for Python 3.7

  1. Title:

    Domino Analytics Distribution py3.7 R3.5

  2. Description:

    Ubuntu 18.04
    Anaconda Python 3.7
    R 3.5.3
    Jupyter Lab 0.35.4, Jupyter 1.0, Rstudio 1.2, VSCode
    Keras 2.2.4 and Tensorflow-gpu* 1.13.1
    CUDA 10.0.130 with Nvidia driver 410.104
    For python packages run: !pip freeze
    For R packages run: installed.packages()
    For further detail, please ask Domino Support for the full Dockerfile
  3. Base Image URL:

    quay.io/domino/base:Ubuntu18_DAD_Py3.7_R3.5-20190501

  4. Pluggable Workspace Tools

     jupyterlab:
    title: "JupyterLab (Beta)"
    iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/jupyterlab.svg?sanitize=true"
    start: [ /var/opt/workspaces/Jupyterlab/start.sh ]
    httpProxy:
    internalPath: "/{{#if pathToOpen}}/lab/tree/{{pathToOpen}}{{/if}}"
    port: 8888
    rewrite: false
    rstudio:
    title: "RStudio"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos//Rstudio.svg?sanitize=true"
    start: [ "/var/opt/workspaces/rstudio/start" ]
    httpProxy:
    port: 8888
    vscode:
    title: "vscode"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/vscode.svg?sanitize=true"
    start: [ "/var/opt/workspaces/vscode/start" ]
    httpProxy:
    port: 8888
    jupyter:
    title: "Jupyter (Python, R, Julia)"
    iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/Jupyter.svg?sanitize=true"
    start: [ "/var/opt/workspaces/jupyter/start" ]
    httpProxy:
    port: 8888
    rewrite: false
    internalPath: "/{{#if pathToOpen}}tree/{{pathToOpen}}{{/if}}"
    supportedFileExtensions: [ ".ipynb" ]
Was this article helpful?
0 out of 0 found this helpful