Domino standard environments

Overview

Each run and workspace in Domino operates in its own Docker container. These Docker containers are defined by Domino environments. Environments can be shared and customized, and they are automatically versioned by Domino.

New installations of Domino come with a standard set of environments and associated Docker images. Periodically, Domino publishes a new set of standard environments with updated libraries and packages. This article describes how to adopt the latest Domino standard environments.

Standard Environments

Domino maintains four standard compute environments:

  • Domino Analytics Distribution (DAD) for Python 2
  • Domino Analytics Distribution (DAD) for Python 3
  • Domino Extended Distribution (DED) for Python 2
  • Domino Extended Distribution (DED) for Python 3

The Domino Analytics Distributions are designed to handle most of what a typical data science workflow needs out of the box, and differ only on which version of Python they have installed. The Domino Extended Distributions are build on top of Domino Analytics Distributions, but also add some common deep learning packages, GPU drivers, and other advanced tools. See the last section of this document for comprehensive details on these environments.

These images are hosted on Quay.io, a third-party Docker registry. By default, your Domino deployment comes configured with keys to access Domino’s images in this service.

The URLs for the latest versions of Domino Standard Environments are:

  • DAD Python 2: quay.io/domino/base:DAD_py2.7_R3.4_23052018
  • DAD Python 3: quay.io/domino/base:DAD_py3.6_R3.4_23052018
  • DED Python 2: quay.io/domino/base:DED_py2.7_R3.4_23052018
  • DED Python 3: quay.io/domino/base:DED_py3.6_R3.4_23052018

Identifying standard environments in your deployment

To determine if your deployment is using the latest Domino standard environments, open Domino and click Environments in the top navigation bar. You can use the search bar to find environments with Domino Analytics or Domino Extended in the name. However, previous administrators or installers may have added these environments under a different name. To confirm that an environment is using a Domino standard, click on it to open the Overview.

domino-ded-base-environment.png

If the Base Image field matches one of the URLs above, the environment is using the latest standard. In the example shown, the environment is using the Domino Analytics Distribution for Python 3.

Adding standard environments to your deployment

Adding a Domino standard environment to your deployment involves three high-level steps.

  1. Create a new environment with the correct defaults for the standard you want to add.

    • See Compute Environment Management for an overview of how to create and manage environments.

    • See the table of environment details below to populate the Base ImageDockerfile Instructions, and Pluggable Workspace Tools fields. These fields must be filled out correctly for the environment to work as intended.

  2. Copy over any additional customization from your existing default compute environment.

    • If you added additional packages and libraries to your previous default environment, and you intend to use the new standard environment as your deployment default, you should copy over those additions to the new environment you created. These can be added to the end of the Dockerfile Instructions in the new environment.

    • You should not copy over any instructions that update Python, R, RStudio, or other core components of the standard environments if the new standard already includes the version you want.

    • Be sure to copy over any setup scripts and run scripts attached to your previous default, as your users may depend on them.

  3. Create a new machine image for your executors that includes the new environment.

    • See Creating a new AMI for instructions.

    • Once this is done, you can set the new environment as the deployment default.

    • You may also choose to archive the previous default environment. Any existing projects using an archived environment will be able to continue using it, but new projects will not have access to it.

Detailed information about standard environments

This section contains detailed specifications about the four Domino Standard Environments, plus the configuration settings needed when adding them to your deployment.

Domino Analytics Distribution (DAD) for Python 2

  1. Title:

    Domino Analytics Distribution py2.7 R3.4

  2. Description:

    Anaconda Python 2.7
    Jupyter Lab v0.31.12
    Jupyter v5.4.1
    R 3.4.4
    Rstudio v1.1.419
    Julia .62
    Java8
    Oracle Client 12.1
    Unix ODBC 2.3.4
    PostgreSQL 9.3
    For python packages run:  !pip freeze
    For R packages run: installed.packages()
    Email support@dominodatalab.com for full Dockerfile
    
  3. Base Image URL:

    quay.io/domino/base:DAD_py2.7_R3.4_23052018

  4. Dockerfile Instructions:

    #Gives you Sudo access in container
    RUN echo "ubuntu    ALL=NOPASSWD: ALL" >> /etc/sudoers
    
  5. Pluggable Workspace Tools

    jupyter:
      title: "Jupyter (Python, R, Julia)"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/Jupyter.svg?sanitize=true"
      start: [ "/var/opt/workspaces/jupyter/start" ]
      httpProxy:
        port: 8888
        rewrite: false
        internalPath: "/tree/"
      supportedFileExtensions: [ ".ipynb" ]
    rstudio:
      title: "RStudio"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos//Rstudio.svg?sanitize=true"
      start: [ "/var/opt/workspaces/rstudio/start" ]
      httpProxy:
        port: 8888
    jupyterlab:
        title: "JupyterLab (Beta)"
        iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/jupyterlab.svg?sanitize=true"
        start: [  /var/opt/workspaces/Jupyterlab/start.sh ]
        httpProxy:
            internalPath: ////
            port: 8888
            rewrite: false
    

Domino Analytics Distribution (DAD) for Python 3

  1. Title:

    Domino Analytics Distribution py3.6 R3.4

  2. Description:

    Anaconda Python 3.6
    Jupyter Lab v0.31.12
    Jupyter v5.4.1
    R 3.4.4
    Rstudio v1.1.419
    Julia .62
    Java8
    Oracle Client 12.1
    Unix ODBC 2.3.4
    PostgreSQL 9.3
    For python packages run:  !pip freeze
    For R packages run: installed.packages()
    Email support@dominodatalab.com for full Dockerfile
    
  3. Base Image URL:

    quay.io/domino/base:DAD_py3.6_R3.4_23052018

  4. Dockerfile Instructions:

    #Gives you Sudo access in container
    RUN echo "ubuntu    ALL=NOPASSWD: ALL" >> /etc/sudoers
    
  5. Pluggable Workspace Tools

    jupyter:
      title: "Jupyter (Python, R, Julia)"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/Jupyter.svg?sanitize=true"
      start: [ "/var/opt/workspaces/jupyter/start" ]
      httpProxy:
        port: 8888
        rewrite: false
        internalPath: "/tree/"
      supportedFileExtensions: [ ".ipynb" ]
    rstudio:
      title: "RStudio"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos//Rstudio.svg?sanitize=true"
      start: [ "/var/opt/workspaces/rstudio/start" ]
      httpProxy:
        port: 8888
    jupyterlab:
        title: "JupyterLab (Beta)"
        iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/jupyterlab.svg?sanitize=true"
        start: [  /var/opt/workspaces/Jupyterlab/start.sh ]
        httpProxy:
            internalPath: ////
            port: 8888
            rewrite: false
    

Domino Extended Distribution (DED) for Python 2

  1. Title:

    Py2.7 R3.4 GPU Tools (TensorFlow, Keras) + Spark 2.2.1

  2. Description:

    Domino Extended Distribution for Python 2
    
    Built on top of Domino Analytics Distribution for Python 2
    
    Adds the following extensions:
    
    Spark 2.2.1 [Local mode only]
    Scala 2.12.5
    Tensorflow 1.4
    MXNet
    PyTorch
    Cuda 8.0, CUDNN 6.0.21
    NLTK
    Email support@dominodatalab.com for full Dockerfile
    
  3. Base Image URL:

    quay.io/domino/base:DED_py2.7_R3.4_23052018

  4. Dockerfile Instructions:

    #Gives you Sudo access in container
    RUN echo "ubuntu    ALL=NOPASSWD: ALL" >> /etc/sudoers
    
  5. Pluggable Workspace Tools

    jupyter:
      title: "Jupyter (Python, R, Julia)"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/Jupyter.svg?sanitize=true"
      start: [ "/var/opt/workspaces/jupyter/start" ]
      httpProxy:
        port: 8888
        rewrite: false
        internalPath: "/tree/"
      supportedFileExtensions: [ ".ipynb" ]
    rstudio:
      title: "RStudio"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos//Rstudio.svg?sanitize=true"
      start: [ "/var/opt/workspaces/rstudio/start" ]
      httpProxy:
        port: 8888
    jupyterlab:
        title: "JupyterLab (Beta)"
        iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/jupyterlab.svg?sanitize=true"
        start: [  /var/opt/workspaces/Jupyterlab/start.sh ]
        httpProxy:
            internalPath: ////
            port: 8888
            rewrite: false
    h20Flow:
        title: "H2O Flow"
        iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/H2O.svg?sanitize=true"
        start: [ "/var/opt/workspaces/h2o/start" ]
    

Domino Extended Distribution (DED) for Python 3

  1. Title:

    Py3.6 R3.4 GPU Tools (TensorFlow, Keras) + Spark 2.2.1

  2. Description:

    Domino Extended Distribution for Python 3
    
    Built on top of Domino Analytics Distribution for Python 3
    
    Adds the following extensions:
    
    Spark 2.2.1 [Local mode only]
    Scala 2.12.5
    Tensorflow 1.4
    MXNet
    PyTorch
    Cuda 8.0, CUDNN 6.0.21
    NLTK
    Email support@dominodatalab.com for full Dockerfile
    
  3. Base Image URL:

    quay.io/domino/base:DED_py3.6_R3.4_23052018

  4. Dockerfile Instructions:

    #Gives you Sudo access in container
    RUN echo "ubuntu    ALL=NOPASSWD: ALL" >> /etc/sudoers
    
  5. Pluggable Workspace Tools

    jupyter:
      title: "Jupyter (Python, R, Julia)"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos/Jupyter.svg?sanitize=true"
      start: [ "/var/opt/workspaces/jupyter/start" ]
      httpProxy:
        port: 8888
        rewrite: false
        internalPath: "/tree/"
      supportedFileExtensions: [ ".ipynb" ]
    rstudio:
      title: "RStudio"
      iconUrl: "https://raw.github.com/dominodatalab/workspace-configs/develop/workspace-logos//Rstudio.svg?sanitize=true"
      start: [ "/var/opt/workspaces/rstudio/start" ]
      httpProxy:
        port: 8888
    jupyterlab:
        title: "JupyterLab (Beta)"
        iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/jupyterlab.svg?sanitize=true"
        start: [  /var/opt/workspaces/Jupyterlab/start.sh ]
        httpProxy:
            internalPath: ////
            port: 8888
            rewrite: false
    h20Flow:
        title: "H2O Flow"
        iconUrl: "https://raw.githubusercontent.com/dominodatalab/workspace-configs/develop/workspace-logos/H2O.svg?sanitize=true"
        start: [ "/var/opt/workspaces/h2o/start" ]
Was this article helpful?
0 out of 0 found this helpful