Follow

Secure configuration

Domino's Environment Variables gives you a safe and easy way to inject sensitive configuration into the execution of your analysis or models.

Environment Variables are stored securely, modifiable only by owners of a project or in the case of models also by editors, and are not tied to the version history of your project or model, so they are easily revokable.

Why Use Environment Variables for Configuration?

Your code may need to connect to external resources, like a database or S3. Often these connections are authenticated via a secure password, key, or token. It is a bad idea to put this type of secure configuration directly in your source because:

  • You often want to share source files but don't want to leak those credentials
  • It's difficult to scrub references to those credentials from a version control system like Git or Domino
  • You may want only a more privileged user (like the project owner) to be able to change certain configuration parameters. If configuration is all done through code, all users that can modify the scripts can potentially change the config.

It is a better idea to have your configuration stored and permission separately, and have it “injected” when your code executes.

Setting up Project Environment Variables

You can configure your secure configuration to be injected at execution time via environment variables.

To set this up, go to the Settings tab on the project you wish to configure. Under the “Environment variables” section you can add key/value pairs which will be injected as env vars at execution time:

The values are passed verbatim, so no escaping is required. Note that there is a 64K length limit for the value.  

Setting up User Environment Variables

Environment variables can also be configured on a per-user basis. These variables will be injected at execution time for any run the user starts.

User Environment Variables are automatically imported into Runs across all projects, and can be accessed like any other Environment Variables. Note that user specific environment variables are not used or available in Models.

To configure your user environment variables, visit your account settings page by clicking your username and then "Account Settings" from the top right of the Domino navigation bar. From your account settings page, scroll down to the section titled "User environment variables". Here you can configure variables for your user account in the same way as project environment variables (described above).

Setting up Model Environment Variables

You can configure your secure configuration to be injected at execution time via environment variables.

To set this up, go to the Settings tab on the model you wish to configure. Under the “Environment” section you can add key/value pairs which will be injected as env vars at execution time, in the same manner as project environment variables.

The values are passed verbatim, so no escaping is required. Note that there is a 64K length limit for the value.

When you add a variable the values are pushed to all running model versions. Only owners or editors can create environment variables. 

Note that project level and user level environment variables are not used in Models and must be set separately on the model. 

A note about connecting to Git repos

If you're using a user level environment variable to connect to Github and download repos, you will need to add that into your environment definition this way:

run pip install git+https://personalaccesstoken:personalaccesstoken@github.com/<repo> 

Using Environment Variables in Pre- or Post-setup Scripts of your Environment

If you want to reference custom-defined environment variables in the pre- or post-setup script of your custom compute environment, you'll need to make sure the variable name has the prefix "DRT_".

Hierarchy of Environment Variables

It is possible to set the same variable in different places, each of the following will override the previous one in the following order:

  • Compute environment
  • Project
  • User Account

So to clarify for a given variable the table below outlines which values are set and the expected result

Place set Run#1 Run#2 Run#3
Compute Environment A A A
Project - B B
User Account - - C
Run Result A B C

 

Reading the Environment Variables

Every language has its own way of reading env vars. In Python, it might look like this:

import os
s3 = S3Client(os.environ['S3_KEY'], os.environ['S3_SECRET'])

For more details, please read this Python help documentation

In R, it might look like this:

makeS3Client(Sys.getenv("S3_KEY"), Sys.getenv("S3_SECRET"))

For more details, please read this R help documentation.

Was this article helpful?
0 out of 0 found this helpful