General#
Pipenv is a packaging tool for Python that solves some common problems associated with the typical workflow using pip, virtualenv, and requirements.txt.
Pipenv is designed to resolve dependency management chaos created by requirements.txt.
Introduction#
First, let’s install the
pipenv
package. Pipenv uses pip and virtualenv under the hood but simplifies their usage with a single command-line interface.
1
| pip install --user pipenv
|
Pipenv introduces two new files.
- Pipfile
is to replace the old requirements.txt
Pipfile.lock to enable deterministic builds.
Example Usage#
Spawn An isolated Development Environment#
The terminal will generate something like
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| Creating a virtualenv for this project...
Pipfile: /Users/ey/Workspace/Temp/2023-03-27_dbt_fundamentals/Pipfile
Using /opt/homebrew/bin/python3 (3.11.2) to create virtualenv...
⠙ Creating virtual environment...created virtual environment CPython3.11.2.final.0-64 in 642ms
creator Venv(dest=/Users/ey/.local/share/virtualenvs/2023-03-27_dbt_fundamentals-OcZOwZwe, clear=False, no_vcs_ignore=False, global=False, describe=CPython3Posix)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/ey/Library/Application Support/virtualenv)
added seed packages: pip==22.3.1, setuptools==65.5.1, wheel==0.38.4
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /Users/ey/.local/share/virtualenvs/2023-03-27_dbt_fundamentals-OcZOwZwe
Creating a Pipfile for this project...
Launching subshell in virtual environment...
. /Users/ey/.local/share/virtualenvs/2023-03-27_dbt_fundamentals-OcZOwZwe/bin/activate
|
Install New Packages#
Now let’s install the dbt-postgres package with pipenv.
1
| pipenv install dbt-postgres
|
Results
1
2
3
4
5
6
7
8
9
10
11
| Installing dbt-postgres...
Adding dbt-postgres to Pipfile's [packages]...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Updated Pipfile.lock (7a415f34b140ddbc29596b3ed6a8ff412fad78a2d237593f2d960d52d6e23242)!
Installing dependencies from Pipfile.lock (e23242)...
|
This process will also modify the Pipfile and Pipfile.lock.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| cat ./Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
dbt-postgres = "*"
[dev-packages]
[requires]
python_version = "3.11"
|
Install packages for Development#
In most of the cases, we don’t want to include our development libraries into our production build. Pipenv very handly provides a --dev option to specify packages that need to recognised as development package.
For example,
1
| pipenv install pytest --dev
|
Results
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
| Installing pytest...
Adding pytest to Pipfile's [dev-packages]...
✔ Installation Succeeded
Pipfile.lock (e23242) out of date, updating to (b5bfc6)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (1073d20d2028cc8ad5fedf70164adb809fd672ac32919ebffdb15dfd58b5bfc6)!
Installing dependencies from Pipfile.lock (b5bfc6)...
Installing dependencies from Pipfile.lock (b5bfc6)...
|
Also the Pipfile is updated with pytest package
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
| cat ./Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
dbt-postgres = "*"
[dev-packages]
pytest = "*"
[requires]
python_version = "3.11"
|
Capsulise an Environment#
Once you install all the necessary packages, you can then use the lock command to lock your environment
This will update the Pipfile.lock file for production usage.
1
2
3
4
5
6
7
8
9
| Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (1073d20d2028cc8ad5fedf70164adb809fd672ac32919ebffdb15dfd58b5bfc6)!
|
Distribute Environment Setup#
Once you get your code and Pipfile.lock in your production environment, you can then ask another developer to clone the latest codebase, and run the following command:
This installs all the dependencies needed for development, which includes both the regular dependencies, and these you specified with the --dev argument during install stage.
Useful Command#
- Install packages from existing
requirements.txt file
1
2
| pipenv install -r requirements.txt
pipenv install -r dev-requirements.txt --dev
|
- Generate a requirements.txt file
1
2
| pipenv lock -r > requirements.txt
pipenv lock -r -d > dev-requirements.txt
|
- Run a command in the virtual environment without launching a shell
1
| pipenv run python -c "print('hello, pipenv!')"
|
- Check for security vulnerability in your environment
- Uninstall a no longer needed package
- Uninstall all development packages
1
| pipenv uninstall --all-dev
|
- Find your virtual environment folder
Reference#