Environment Variables in Python

Environment variables are a key idea in software development that are used to specify and preserve system-specific setups, pathways, and settings. They make it simpler to handle settings for various environments, such as development, testing, and production, and they offer a mechanism to segregate configuration information from code.

The usage of Python environment variables to store and retrieve crucial configuration information will be covered in this article.

What are environment variables?

The configuration information and settings for a computer system or a particular application are stored in environment variables, which are system-wide variables. These variables can be accessed and changed by programmes and scripts running on the system, and they are set by the operating system or the user, respectively.

Environment variables are frequently used to store data like as executable paths, system directories, or remote service authentication information that is required by numerous programmes and scripts. These specifications can be set up once and shared by all programmes and scripts operating on the system by using environment variables.

The Python os module, which offers a diction-like interface to the system's environment variables, allows us to access environment variables.

Setting and accessing environment variables in Python

Environment variables can be set and accessed in Python using a variety of methods. Utilizing the os module, which offers features to communicate with the operating system, is one of the most popular methods.

The os.environ dictionary, which represents the current environment variables, can be used to set an environment variable. By only associating a value with a fresh key, we can easily add a new variable as follows:

In this example, we create a new environment variable called MY_VAR and set its value to my_value.

To access the value of an existing environment variable, we can use the same os.environ dictionary and retrieve the value associated with the key:

Output:

my_value

In this example, we retrieve the value of the MY_VAR environment variable and assign it to the my_var_value variable.

Setting environment variables from the command line

We can set environment variables from the command line in addition to from within a Python programme. Setting configuration information in advance of executing a programme or script is useful. In Python, we can set and receive environment variables using the os module. The os module provides a dictionary-like interface to the system's environment variables, with the environment variables themselves being stored as key-value pairs.

The export command, followed by the variable name and its value, is how we set an environment variable on Unix-based systems (like Linux and macOS):

This sets the MY_VAR environment variable to my_value for the current shell session.

On Windows, we can use the setx command to set an environment variable, like this:

This sets the MY_VAR environment variable to my_value permanently, so it will be available to all processes on the system.

Using environment variables for configuration

The storing of configuration information for a programme or application is one of the keys uses for environment variables. This makes it simpler to change settings for various environments (such as development, testing, and production) without having to update the code itself because the configuration and the code can now be separated.

Consider a Python programme that connects to a database as an example. The login, password, and server address for the database connection could be saved in environment variables as follows:

To get the values of the environment variables DB NAME, DB USER, DB PASSWORD, DB HOST, and DB PORT, we use the os.environ dictionary (which is optional and defaults to 5432 if not set). The connection to the PostgreSQL database is set up using these values.

We can simply adapt these setup specifications for various situations without having to update the code by putting them in environment variables. For our development, testing, and production environments, for instance, we may set the values of these variables to various values.

Best practices for using environment variables.

When using environment variables for configuration, it is important to follow some best practices to ensure that our code is secure, portable, and easy to maintain.

1. Use meaningful names

When naming environment variables, it is important to use meaningful names that describe the configuration detail they represent. This makes it easier to understand and maintain the code, especially for large projects with many environment variables.

For example, if we are storing the URL of an API endpoint in an environment variable, we could use a name like API_URL instead of a generic name like URL. This makes it clear what the variable is used for and helps avoid confusion with other variables that might use the same name.

Using meaningful names also makes it easier to document the code and to share it with other developers. When someone else looks at the code, they should be able to understand the purpose of each environment variable just by looking at its name.

2. Store sensitive data securely

Environment variables can be accessed by all processes running on a system, so it is important to store sensitive data (such as passwords and access keys) securely. One way to do this is to store the data encrypted in a file or a key store and only load the data into environment variables at runtime.

For example, we could store the encrypted password for a database connection in an environment variable named DB_PASSWORD_ENCRYPTED. Our code would then read the value of this variable, decrypt the password, and use it to connect to the database.

Using encryption helps protect sensitive data from being exposed if an attacker gains access to the system or if the environment variables are accidentally leaked.

Another option is to use a secure credential store, such as HashiCorp Vault or AWS Secrets Manager, to store sensitive data. These services provide APIs for retrieving and managing secrets, and they can be integrated with our code to securely retrieve sensitive data at runtime.

3. Document your configuration

It is important to document your environment variables and their values, especially if you are sharing your code with other developers or deploying it to multiple environments. This makes it easier to understand how your program is configured and reduces the risk of misconfigurations.

Documenting environment variables helps ensure that other developers can understand and use our code, and it can also help us remember the purpose of each variable when we come back to the code later.

4. Use a library to manage environment variables

As your program grows, managing environment variables can become more complex. There are several libraries available for Python that can help manage environment variables, such as python-dotenv and environ-config. These libraries can load environment variables from files, validate their values, and provide defaults for missing variables.

While environment variables can be used to store sensitive data securely, they should not be relied on for security. Environment variables can be accessed and modified by any process running on the system, including potential attackers.

If our code needs to store sensitive data, we should use encryption, secure credential stores, or other secure methods to protect the data. We should also follow security best practices, such as limiting access to sensitive data and regularly reviewing our code for security vulnerabilities.

5. Use a configuration management system

Using a configuration management system can also be a helpful best practice for managing environment variables in larger projects. Configuration management systems provide a way to automate the deployment and management of configuration across different environments and servers.

Some popular configuration management systems for Python include:

• Ansible: a powerful open-source automation platform that can automate many different IT tasks, including configuration management.
• Puppet: another open-source configuration management tool that allows users to automate deployment and management of their infrastructure.
• Chef: a configuration management tool that allows users to manage infrastructure as code and automate the deployment of applications and configuration.

By using a configuration management system, we can ensure that our environment variables are consistent across different environments and servers, and we can easily update them as needed. This can help reduce errors and make it easier to manage our code in production.

Conclusion

An effective method for controlling system-wide configurations and settings is the use of environment variables. We can set and retrieve environment variables in Python using the os module, and we can use them to hold configuration information for our programmes and apps.