Python save environment (export file)
In deep learning scenarios, when we encounter the need to save the environment, such as when sharing the code with others, we need to export a file to let others know the python package dependencies required to run the code.
This article mainly introduces some common methods and tools for exporting python environments.
1. Export using pip package
1）pip freezen >
The exported method contains the path where the installation package is located, which is often used for the environment preservation of some large projects. Generally, large projects will load and run multiple environments. At this time, the corresponding location of each environment also needs to be notified.
pip freezen >
2）pip list --format=freeze >
This method exports no path where the installation package is located, which is very similar to the common ones.
In the above two methods, the export of all the python packages installed in the environment are all installed in the environment, but some packages are not necessary for the project to run. Sometimes some packages exported are not necessary for the project to run.
pip list --format=freeze >
3) Use pip and install packages
Use the following command to install the dependency package:
pip install -r
2. Export using conda
If you use conda to manage the environment, you can also use the conda command to export and install python dependencies
1）conda list -e >
Export file using the following command
conda list -e >
To use the conda installation file, install the dependencies using the following command:
conda install --yes --file
2）conda env export >
You can also export the environment by exporting yaml files through conda, with the following commands:
conda env export >
The exported yaml file is installed using the following command:
conda env create -f
3. Export using the pipreqs package (suggested)
The above two methods export the entire installation environment, but sometimes a project does not require all dependencies in the installation environment. It is recommended to use the pipreqs package to export the file.
Use pipreqs to automatically retrieve all components and their versions under the current project and generate files, which greatly facilitates the package management of project migration and deployment. Compared to using commands directly, it can directly isolate the generation of packages of other projects.
Use the following command to install pipreqs:
pip install pipreqs
Usage:
pipreqs [options] <path>
Options:
--use-local Use ONLY local package info instead of querying PyPI
--pypi-server <url> Use custom PyPi server
--proxy <url> Use Proxy, parameter will be passed to requests library. You can also just set the
environments parameter in your terminal:
$ export HTTP_PROXY="http://10.10.1.10:3128"
$ export HTTPS_PROXY="https://10.10.1.10:1080"
--debug Print debug information
--ignore <dirs>... Ignore extra directories
--encoding <charset> Use encoding parameter for file open
--savepath <file> Save the list of requirements in the given file
--print Output the list of requirements in the standard output
--force Overwrite existing
--diff <file> Compare modules in to project imports.
--clean <file> Clean up by removing modules that are not imported in project.
--no-pin Omit version of output packages.Use the following command to export the file:
pipreqs ./
If you are in a Windows environment, it is recommended to use the following command to export the file:
pipreqs ./ --encoding=utf-8
If a file exists in the environment, you need to use the following command to export the file:
pipreqs ./ --encoding=utf-8 --force
Summarize
The above is personal experience. I hope you can give you a reference and I hope you can support me more.