General

Pipenv is a packaging tool for Python that solves some common problems associated with the typical workflow using pip, virtualenv, and requirements.txt. Pipenv is designed to resolve dependency management chaos created by requirements.txt.

Introduction

First, let’s install the pipenv package. Pipenv uses pip and virtualenv under the hood but simplifies their usage with a single command-line interface.

1
pip install --user pipenv

Pipenv introduces two new files.

  • Pipfile is to replace the old requirements.txt
  • Pipfile.lock to enable deterministic builds.

Example Usage

Spawn An isolated Development Environment

1
pipenv shell

The terminal will generate something like

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
Creating a virtualenv for this project...
Pipfile: /Users/ey/Workspace/Temp/2023-03-27_dbt_fundamentals/Pipfile
Using /opt/homebrew/bin/python3 (3.11.2) to create virtualenv...
⠙ Creating virtual environment...created virtual environment CPython3.11.2.final.0-64 in 642ms
  creator Venv(dest=/Users/ey/.local/share/virtualenvs/2023-03-27_dbt_fundamentals-OcZOwZwe, clear=False, no_vcs_ignore=False, global=False, describe=CPython3Posix)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/ey/Library/Application Support/virtualenv)
    added seed packages: pip==22.3.1, setuptools==65.5.1, wheel==0.38.4
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /Users/ey/.local/share/virtualenvs/2023-03-27_dbt_fundamentals-OcZOwZwe
Creating a Pipfile for this project...
Launching subshell in virtual environment...
. /Users/ey/.local/share/virtualenvs/2023-03-27_dbt_fundamentals-OcZOwZwe/bin/activate

Install New Packages

Now let’s install the dbt-postgres package with pipenv.

1
pipenv install dbt-postgres

Results

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
Installing dbt-postgres...
Adding dbt-postgres to Pipfile's [packages]...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Updated Pipfile.lock (7a415f34b140ddbc29596b3ed6a8ff412fad78a2d237593f2d960d52d6e23242)!
Installing dependencies from Pipfile.lock (e23242)...

This process will also modify the Pipfile and Pipfile.lock.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
cat ./Pipfile

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
dbt-postgres = "*"

[dev-packages]

[requires]
python_version = "3.11"

Install packages for Development

In most of the cases, we don’t want to include our development libraries into our production build. Pipenv very handly provides a --dev option to specify packages that need to recognised as development package.

For example,

1
pipenv install pytest --dev

Results

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
Installing pytest...
Adding pytest to Pipfile's [dev-packages]...
✔ Installation Succeeded
Pipfile.lock (e23242) out of date, updating to (b5bfc6)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (1073d20d2028cc8ad5fedf70164adb809fd672ac32919ebffdb15dfd58b5bfc6)!
Installing dependencies from Pipfile.lock (b5bfc6)...
Installing dependencies from Pipfile.lock (b5bfc6)...

Also the Pipfile is updated with pytest package

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
cat ./Pipfile

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
dbt-postgres = "*"

[dev-packages]
pytest = "*"

[requires]
python_version = "3.11"

Capsulise an Environment

Once you install all the necessary packages, you can then use the lock command to lock your environment

1
pipenv lock

This will update the Pipfile.lock file for production usage.

1
2
3
4
5
6
7
8
9
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (1073d20d2028cc8ad5fedf70164adb809fd672ac32919ebffdb15dfd58b5bfc6)!

Distribute Environment Setup

Once you get your code and Pipfile.lock in your production environment, you can then ask another developer to clone the latest codebase, and run the following command:

1
pipenv install --dev

This installs all the dependencies needed for development, which includes both the regular dependencies, and these you specified with the --dev argument during install stage.

Useful Command

  • Install packages from existing requirements.txt file
1
2
pipenv install -r requirements.txt
pipenv install -r dev-requirements.txt --dev
  • Generate a requirements.txt file
1
2
pipenv lock -r > requirements.txt
pipenv lock -r -d > dev-requirements.txt
  • Run a command in the virtual environment without launching a shell
1
pipenv run python -c "print('hello, pipenv!')"
  • Check for security vulnerability in your environment
1
pipenv check
  • Uninstall a no longer needed package
1
pipenv uninstall numpy
  • Uninstall all packages
1
pipenv uninstall --all
  • Uninstall all development packages
1
pipenv uninstall --all-dev
  • Find your virtual environment folder
1
pipenv --venv
  • Find your project folder
1
pipenv --where

Reference