Python Virtual Environments - Ten Mile Square Technologies

Overview

Python Virtual environments are tools that can isolate a workspace in order to maintain project dependencies. Without this, packages will be installed globally causing potential clashes of versions for different projects and a convoluted system space. In addition to preventing those conflicts, virtual environments also allow for easier interpreter management. This guide will show you how to get strict isolation of the python interpreter you are using and control the packages that are available.

NOTE: while this guide is targeted towards Mac OS X, the techniques can be adapted to any UNIX or Linux environment.

Mac OS X Installation Background

Please refer to the Mac OS X file system domain reference for further information. For Linux, please refer to the filesystem hierarchy standard .

File Sytem Domains

Mac OS X is segmented into 3 major filesystem domains.

System Domain

We do not want to touch this domain. The built-in python interpreter can be modified so that the system will not boot. I know this from experience, perhaps I’ll write about it in another post.

Local Domain

This filesystem domain is where I install additional python interpreters. I use homebrew so I can compile everything from source (not required); they are installed in `/usr/local`.

User Domain

Everything not in the default installation for any version of python will be located here. We will install pip, virtual environment and virtual environment wrapper in the user home directory. Then, any additional modules will be installed within a virtual environment.

Python Installation Background

Python modules can be installed in three places:

Standard location
Using a different install scheme (user, home or prefix)
Virtual environment

Python virtual environments :

Virtual Environment (virtualenv)- Created for python2 and will work with python2 or python3
venv

We want python packages to only be installed in virtual environments… Except for pip, virtualenv, and virutalenvwrapper. These three packages we will install with the python user scheme.

Installation

Python Interpreters

Install python interpreters from homebrew:

Python2 (2.7.12 at writing)
Python3 (3.6.1 at writing)

User Environment

Packages to install:

pip
virtualenv (or venv)
virtualenvwrapper (if installing virtualenv)

Python2:

bash

alias easy_install-2.7="easy_install-2.7 --install-dir ~/Library/Python/2.7/lib/python/site-packages --script-dir ~/bin/ "

Python3:

bash

alias easy_install-3.6="easy_install-3.6 --install-dir ~/Library/Python/3.6/lib/python/site-packages --script-dir ~/bin/ "

Virtual Environment Wrapper

Setup your environment to enable your virtual environment wrapper (convenience tool for virtual environments).

~/.bashrc or ~/.zshrc:

bash

virtenv_wrapper="${HOME}/bin/virtualenvwrapper.sh"

if [[ -r "${virtenv_wrapper}" ]]

then

source "${virtenv_wrapper}"

else

echo "WARNING: Unable to source virtualenvwrapper"

fi

export WORKON_HOME="$HOME/Development/Python/virtualenvs"

export VIRTUALENVWRAPPER_VIRTUALENV_ARGS='--no-site-packages'

Virtual Environment

I configure my environment to only allow installation of python packages into my virtual environments.

~/.bashrc or ~/.zshenv:

bash

export PIP_REQUIRE_VIRTUALENV=true"

export PIP_VIRTUALENV_BASE=$WORKON_HOME

export VIRTUAL_ENV_DISABLE_PROMPT="true

At this point, a python virtual environment should be setup and ready to use. Packages can be installed via pip and navigational commands for virtualenvwrapper can be found in the Command Reference.

You now have independent virtual environments into which differentiated packages and package versions can be deployed.