Detailed explanation of configuration file for python library construction

Preface

is a configuration file that defines the build system and related metadata of Python projects. It is a standardized method introduced by the Python community to solve project construction and dependency management problems, originally byPEP 518propose.Files are widely used, especially in building tools (e.g.PoetryandFlit) is used to replace the traditional onedocument.

Main uses

• Define the build system：

  • The build tool and its version can be specified in the file, such assetuptoolsorpoetry, and the dependencies required by these tools. This makes the project's build environment more controllable and repeatable.

• Project metadata：

  • Including the project's name, version, description, author, license and other information, you canDefined in the file. This information can be read and used by the build tool or package management tool.

• Dependency management：

  • This file can manage project dependencies, including runtime dependencies, development time dependencies, etc., similar to, but more structured and flexible.

• Configuring the behavior of building tools：

  • For different build tools, such assetuptools、poetry、black、isortWait, you canconfigure its behavior.

File structure

A typicalThe file consists of several parts, each part corresponding to different functions and configuration information. Here is an example of LlamaFactory:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[]
target-version = "py38"
line-length = 119
indent-width = 4

[]
ignore = ["C408", "C901", "E501", "E731", "E741", "W605"]
select = ["C", "E", "F", "I", "W"]

[]
lines-after-imports = 2
known-first-party = ["llamafactory"]
known-third-party = [
    "accelerate",
    "datasets",
    "gradio",
    "numpy",
    "peft",
    "torch",
    "transformers",
    "trl"
]

[]
quote-style = "double"
indent-style = "space"
docstring-code-format = true
skip-magic-trailing-comma = false
line-ending = "auto"

thisThe file configures the project's construction system and usageruffConduct code checking and formatting related settings. The following are detailed explanations of each part:

1. [build-system]

• requires: Specifies the build system dependencies of the project. Required heresetuptoolsVersion 61.0 or higher. This means that when building this project, the Python environment must be installedsetuptools, and the version is not lower than 61.0.

• build-backend: Specifies the backend tool used to build the project, here issetuptools.build_meta. This meanssetuptoolsIt will be used as the core tool of the construction process.

2. []

• target-version: Specifies the target Python version of the code, here is Python 3.8 (py38)。ruffCorresponding inspections and optimizations will be carried out according to this version.

• line-length: Specifies the maximum length of the line of code, exceeding this length will be considered as a format error. Set to 119 characters here.

• indent-width: Specifies the width of the code indentation, set to 4 spaces here.

3. []

• ignore: Specifies some rules or error codes to ignore. The listed error code will not be fired when the code is checked.

• select: Specifies the error or warning category to check. Only checks that fall into these categories will be enabled.

4. []

• lines-after-imports: Specifies the number of blank lines that should be retained after the import statement, which is set to 2 lines here.

• known-first-party: Lists the first-party module in the project, i.e.llamafactory, used to identify modules inside the project when importing sorts.

• known-third-party: Lists known third-party libraries that will distinguish them from other modules when importing sorts.

5. []

• quote-style: Specify the quote style used by the string, set here todouble, i.e. double quotes.

• indent-style: Specify the indentation style, use herespace(space) instead oftab。

• docstring-code-format: Set totrue, means that code formatting rules will also be applied in the docstring.

• skip-magic-trailing-comma: Set tofalse, meaning that even if magic trailing comma is added, formatting will not be skipped.

• line-ending: Set toauto, means that the line ending character is automatically selected according to the default settings of the operating system.

summary

thisThe file mainly configures the project's construction system and defines in detailruffTools are used for code style checking and formatting rules. These settings help ensure consistency and quality of your code, reduce errors, and follow the team's coding specifications.

Workflow

Executionpip install .hour,Files will be referenced during the process of building and installing Python projects. The specific process is as follows:

1. Identify the file

• When you execute in the root directory of the projectpip install .When commanded,pipThere will be checked first if the directory exists.document.
• If this file exists,pipIt will be regarded as the build configuration file of the project and follow the contents of it to guide the next build process.

2. Analyze the build-system part

• pipRead firstin the file[build-system]part. This section defines the dependencies required to build the project and the build backend (i.e. build tools).
  • requires: Lists the Python packages required to build this project. Executionpip install .hour,pipWill ensure that these packages are installed. If these packages are not installed,pipThese dependencies will be installed first.
  • build-backend: Specify the backend tools used to build the project, such assetuptools.build_metaor。pipThis tool will be called to build the project.

3. Build source distribution package (SDist) and/or wheel package (Wheel)

• OncepipThe dependencies required for the build are installed and the build backend is loaded. The build backend will be called to generate the source distribution package (SDist) and/or the wheel package (Wheel).
  • If usedsetuptools.build_meta, it will read the projectorFiles to guide the construction of packages.
  • These build outputs are saved in a temporary directory for the next installation steps.

4. Installation project

• pipUse the generated package (SDist or Wheel) to install the project into the target environment (usually the current Python environment).
• If the project depends on other packages,pipWill come fromorIninstall_requiresIdentify and install these dependencies in the list.

5. Other tool configuration

• Other parts in  such as[], may be referenced by the corresponding tools during construction or development. Although these parts will not directly affectpip install .process, but they are still important in the development environment.

Simplified execution process

• pipCheck and loaddocument.
• Analysisbuild-system, the dependencies required for installation build.
• Build the project with the specified build backend (generate SDist and/or Wheel).
• Install the generated package and its dependencies.

Summarize

The file ispip install .In the process, it will bepipReferences to determine how to build and install a project.pipMainly used[build-system]Partly load the build tools and dependencies, then call these tools to generate the project's distribution package and finally install it into the current environment.

This is the end of this article about the detailed explanation of the configuration file for python library construction. For more related python library construction content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!