Accueil / Blog / Métier / Archives / How to replace virtualenv + pip by buildout

How to replace virtualenv + pip by buildout

Par Benoit Bryon — publié 02/12/2011, édité le 04/06/2015

I was used to virtualenv and pip, and buildout first seems hard to understand. In fact, I thought that I didn't need buildout.

Then, with some practice, I came to think the opposite. I mean I no longer need virtualenv or pip. And buildout has strong features I really need.

Thus, I am wondering why so many people keep on using virtualenv and pip, and even often refuse to use buildout. One hypothesis is that buildout is hard to discover. During first steps with buildout, you don't understand what happens. And documentation about available options is a bit obscure.

So, here is an article about reproducing a virtualenv+pip workflow with buildout. Let's try it


Use case: project deployment workflow

Since we are about to compare two tools, let's use them in comparable situations. We will deploy a project:

So, let's compare what we have to write in our documentation (INSTALL procedure) and which commands will users (contributors) have to execute.


I won't cover all aspects of buildout here. I will focus on reproducing virtualenv+pip workflow.

Virtualenv and pip

Preliminary notes:

  • Python is a prerequisite should be already installed.
  • We use Debian-based packages here, adapt it to your needs.
  • We use "sudo" to show which commands require superuser privileges.
# Let's work somewhere we own files
cd ~

# Begin deployment process...

# 1. Install virtualenv
sudo aptitude install virtualenv
# 2. Download project
git clone from-virtualenv-and-pip
# 3. Create isolated environment directly within project
virtualenv --distribute --no-site-packages from-virtualenv-and-pip
# 4. Enter project
cd from-virtualenv-and-pip/
# 5. Install requirements
bin/pip install -r requirements.txt

# End deployment process. From now, project is installed, let's use it...

# Build documentation
cd docs
make html
cd ..

5 commands in the deployment process. Really simple, isn't it?

Additional notes:

  • We could have installed virtualenv with easy_install, but it is not the recommended way, since upgrades or uninstalls are not easy.
  • The --distribute option is the default in recent versions of virtualenv.
  • we didn't activate the virtual environment. It removes two steps from the procedure (activate and deactivate).

Let's do the same with buildout...


Preliminary notes:

  • Python is a prerequisite should be already installed.
# Let's work somewhere we own files
cd ~

# Begin deployment process...

# 1. Download project
git clone to-buildout
# 2. Enter project
cd to-buildout/
# 3. Install buildout
python --distribute
# 4. Install requirements

# End deployment process. From now, project is installed, let's use it...

# Build documentation
cd docs
make html
cd ..

4 commands in the deployment process...

Conclusion: buildout is simpler than virtualenv + pip

If you need a straight and simple INSTALL procedure, then I recommend buildout.

<Tom> Hey! I already have virtualenv installed system wide, so I don't
       need to reinstall it!
<George> Can you assert that all the contributors of your project (i.e. your
          team) have virtualenv already installed system wide?
<Tom> I can't.
<George> So you **must** tell them it is a prerequisite in your project's
<Tom> Okay I'm adding a note about prerequisites in the documentation.
<George> Then there is a risk that they can't install virtualenv because
          they have not sufficient privileges. Small but real.
<Tom> Okay I'm adding a note about privileges in the documentation.
<George> How do contributors know how to install virtualenv?
<Tom> They search the web!
<George> ...
<Tom> Okay, I'm adding the command to install virtualenv.
<George> On Debian, Fedora, Solaris or FreeBSD?
<Tom> Wait, I can't maintain installation notes for all operating systems...
<George> Is there a generic recipe to install virtualenv?
<Tom> sudo easy_install virtualenv!
<George> Is sudo another prerequisite?
<Tom> Okay, I'm replacing the commands to install virtualenv by a link to
<George> That is a long documentation to read. If they didn't know about
          virtualenv, contributors have to read and learn about it. It could
          puzzle them, or at least take some time...
<Tom> ...
<George> With buildout, documentation would be lighter, i.e. simpler for a
          human to read and understand.
<George> And we wouldn't have had this conversation...
<Tom> Okay, let's try buildout... But I don't know how to create a project
       with buildout!
<George> Read the notes below ;)

Use case: creating projects

Virtualenv + pip

Here is the synopsis:

  • Create an empty repository
  • Create a virtualenv in it
  • Pip install packages
  • Export requirements.txt with "pip freeze > requirements.txt"
  • Ignore virtualenv generated files in repository
  • Commit and push


Here is the synopsis:

  • Create an empty repository
  • Download buildout's
  • Create buildout.cfg configuration file
  • Install buildout with "python"
  • Deploy project with "bin/buildout"
  • Ignore buildout generated files in repository
  • Commit and push

Let's explain some steps...

Download buildout's

Download buildout's manually or create a script with the command like this:

wget*checkout*/zc.buildout/tags/1.5.2/bootstrap/ -O

Create buildout.cfg configuration file

Here is a simple buildout.cfg file which reproduces virtualenv + pip requirements to install sphinx, nose and a python interpreter.

# Sample buildout configuration file to install sphinx and nose.

parts =
eggs =
versions = versions

recipe = zc.recipe.egg
eggs = ${buildout:eggs}
interpreter = python

sphinx = 1.1.2
nose = 1.1.2

Yes, pip's requirements files are simpler. But, once you learned the basics of buildout, you see that buildout's configuration files are not so complex:

  • As any ConfigParser file, this one is made of sections.
  • [buildout] is the main buildout section:
    • The "parts" directive lists the sections to execute.
    • If "parts" is blank, then buildout will do nothing.
    • The "eggs" directive is a list of eggs (i.e. python dependencies). It does nothing "magic" by itself. But it can be referenced by other parts via the ${buildout:eggs} variable (see the [sphinx-and-python] part). So, consider it as a set of eggs that **can** be shared between several (none to all) parts.
    • The "versions" directive indicates the part where you freeze (pin) versions.
  • [sphinx-and-python] part installs a bin/python interpreter with eggs declared in the main [buildout] section.
  • [versions] section contains some version numbers frozen manually. Keep in mind that extensions like buildout-versions or dumppickedversions can help you generate it!
  • Notice that it is recommended to add comments in buildout configuration files.

Ignore buildout generated files in revisions

Here is a .gitignore file I use in some Python projects:

# Buildout's directories

# Setuptools/Distribute's files

# Python's precompiled files

# Sphinx's builds

Additional tips

Some important buildout options

I recommend reading the buildout help at bin/buildout -h. Pay attention at least to the following options:

bin/buildout -N
Run buildout in "non newest" mode. Use it if you don't want to upgrade installed packages. In fact, it is recommended to always use it, except when you explicitely want to upgrade packages.
bin/buildout -vvvvvvv
Many "v" means more verbose... If something goes wrong, messages may be useful.
bin/buildout -c path/to/buildout.cfg
The location of your buildout configuration file. It is "buildout.cfg" by default. Notice that, by default, buildout's directories (eggs, parts...) are relative to configuration file.

Developing Python modules

  • Create eggs (this is also a best pratice with virtualenv and pip). Have a look at PasteScript and ZopeSkel for this.
  • If your project is the egg (i.e. you want an alternative to " develop"), search the web for the "develop" option in the [buildout] section.
  • If you want to edit external modules (i.e. you want an alternative to "pip install -e") have a look at mr.developer extension.
Voir aussi
Présentation de l'écosystème Python scientifique Présentation de l'écosystème Python scientifique 10/11/2016

Au fil des années Python est devenu un outil du quotidien pour les ingénieurs et chercheurs de ...

Monkey-patching a Python instance method 09/11/2016

Dynamically adding or overwriting an instance method in Python is rarely needed, but it's a good ...

Retour sur PyconFR 2013 05/11/2013

L'édition 2013 de la conférence Python française se tenait à Strasbourg du 26 au 29 Octobre. Je ...

Retour sur PyConFr 2015 Retour sur PyConFr 2015 19/10/2015

Makina Corpus était présent à Pau pour la PyConFr 2015, voici quelques retours à chaud.

Retour sur la PyConFr 2016 Retour sur la PyConFr 2016 18/10/2016

Nous étions présents à Rennes pour PyConFr 2016. Voici notre compte-rendu à chaud.