1. Go language "creation project" structure
The creation project of Go language is actually the Go language project itself, and it is the first Go language project in the world.
Before the Go version 1.5 implemented bootstrapping, the number of lines of C language code also accounted for 32.10%. After the version bootstrapping was implemented, the proportion of lines of Go language code accounted for nearly 90%, and the proportion of C language decreased to less than 1%.
During this version iteration, the layout structure of the Go language project was retained as a whole.
The structure layout of the Go language project is of great reference value to subsequent projects in the Go community, especially the structure under the src directory in the early Go project.
First, we download the source code of Go from GitHub:
git clone /golang/
After entering the root directory of the Go language project, we can use the "tree" command to view the initial source code structure layout of the project. Taking Go version 1.3 as an example, the results are as follows:
$cd go // Enter the Go language project root directory
$git checkout go1.3 // Switch to go version 1.3
$tree -LF 1 ./src // Check the structure layout in the src directory
./src
├── *
├── *
├── cmd/
├── *
├──
├── pkg/
├── *
├── *
... ...
└── *
1.1 Three characteristics of src directory structure
Judging from the above results, the structure under the src directory has these three characteristics
1.**Top-level script file: **The script source file built with code represented by * is placed in the top-level directory under src
2. Executable file directory (cmd): The secondary directory under src cmd stores the relevant directories of Go related executable files below. We can take a closer look at the structure under the cmd directory:
cd cmd
tree .
# See the following results
./cmd
... ...
├── 6a/
├── 6c/
├── 6g/
... ...
├── cc/
├── cgo/
├── dist/
├── fix/
├── gc/
├── go/
├── gofmt/
├── ld/
├── nm/
├── objdump/
├── pack/
└── yacc/
As you can see, each subdirectory here is an executable file corresponding to a Go toolchain command or subcommand. Among them, 6a, 6c, 6g, etc. are special naming methods for early Go versions for assemblers, compilers, etc. for specific platforms.
3.**Standard library and runtime implementation (pkg):**You will see that the secondary directory pkg under src stores runtime implementation and standard library package implementation. These packages can be imported by the programs under cmd above, or can be dependent and imported by Go programs outside the Go language project. Here is the output result of the following structure we viewed from pkg through the tree command:
cd pkg
tree .
# See the following results
./pkg
... ...
├── flag/
├── fmt/
├── go/
├── hash/
├── html/
├── image/
├── index/
├── io/
... ...
├── net/
├── os/
├── path/
├── reflect/
├── regexp/
├── runtime/
├── sort/
├── strconv/
├── strings/
├── sync/
├── syscall/
├── testing/
├── text/
├── time/
├── unicode/
└── unsafe/
This source code structure layout style has had an impact on the layout of many subsequent Go projects, including well-known projects such as Go debugger Delve, container technology project Docker, and container orchestration project Kubernetes, which still maintain a similar project layout style. This consistency helps developers to understand and navigate the source code structure of different Go projects more easily.
2. Go project layout evolution
Of course, the layout structure is also constantly evolving. Simply put, it can be summarized into the following three more important evolutions.
2.1 Evolution 1: Go version 1.4 deletes the intermediate layer directory of pkg and introduces the internal directory
The Go language project has simplified and optimized the source tree structure in its version 1.4, which is mainly reflected in the following two aspects:
Simplify the source tree hierarchy: Go version 1.4 deleted the "src/pkg/xxx" hierarchy directory in the original source tree, and directly used the "src/xxx" structure. This change reduces the depth of the source code tree, making the source code of Go projects easier to read and explore.
Introducing the internal package mechanism: Go 1.4 introduced the internal package mechanism and added the internal directory. This internal mechanism is actually available to all Go projects. The Go language project itself has used the internal mechanism since Go version 1.4. According to the definition of the internal mechanism, Go packages in the internal directory of a Go project can only be imported by the packages inside this project. The packages under this internal directory cannot be imported outside the project. It can be said that the introduction of the internal directory has made the classification and use of Go packages in a Go project clearer.
2.2 Evolution 2: Add vendor directory to Go1.6 version
The second evolution is actually to solve the problem of Go package dependency on version management. The Go core team made the first improvement in Go version 1.5. The vendor construction mechanism has been added, that is, the Go source code compilation can not search for the path of the dependent package under the GOPATH environment variable, but search for the corresponding dependent package in the vendor directory.
The Go language project itself also added the vendor directory to Go version 1.6 to support vendor builds, but the vendor directory does not substantially cache any third-party packages. It was not until Go version 1.7 that Go really cached its dependencies external packages under vendor. These dependencies are mainly packages under /x. These packages are also maintained by the Go core team, and their update speed is not affected by the Go version release cycle.
The introduction of vendor mechanism and directories has given Go projects the ability to reproducible build for the first time.
2.3 Evolution Three: Go 1.13 version introduction and
The third evolution is still to solve the problem of Go package dependency version management. In Go 1.11, the Go core team made the second attempt to improve: the Go Module construction mechanism was introduced, that is, the third-party packages and versions that the project depends on during the project introduction and in the project, the project construction will break free from the constraints of GOPATH and achieve accurate reproducible construction.
The Go language project itself was introduced and supported in Go version 1.13 to support Go Module construction mechanism. The following is the file content of Go version 1.13:
module std go 1.13 require ( /x/crypto v0.0.0-20190611184440-5c40567a22f8 /x/net v0.0.0-20190813141303-74dc4d7220e7 /x/sys v0.0.0-20190529130038-5219a1e1c5f8 // indirect /x/text v0.3.2 // indirect )
We see that the packages that the Go language project itself depends on have corresponding information in it, and these dependencies were originally cached in the vendor directory.
In general, these three evolutions are mainly reflected in simplifying structural layout and optimizing package dependency management, which plays a role in improving the Go development experience. It can be said that the source code layout and evolution of the Go Creation project are of great inspiration to the layout of the Go community project, so that after years of Go community practice, the Go community has gradually formed a recognized typical structural layout of the Go project.
3. The typical structural layout of Go projects now
Go projects are usually divided into executable program projects and library projects. Now let’s analyze the typical structural layout of these two types of Go projects.
3.1 Typical structural layout of Go executable program projects
Executable program projects are projects that aim to build executable programs. The typical structural layout formed by the Go community for such Go projects is as follows:
$tree -F exe-layout
exe-layout
├── cmd/
│ ├── app1/
│ │ └──
│ └── app2/
│ └──
├──
├──
├── internal/
│ ├── pkga/
│ │ └── pkg_a.go
│ └── pkgb/
│ └── pkg_b.go
├── pkg1/
│ └──
├── pkg2/
│ └──
└── vendor/
A typical layout of such a Go project is "reborn" from the latest structural layout of the Go Creation project. I will explain to you a few key points here.
Let's go from top to bottom in order, first look at the cmd directory. The cmd directory is the source file that stores the main package corresponding to the executable file to be compiled and built by the project. If you have multiple executable files in your project that need to be built, the main package of each executable file is placed separately in a subdirectory, such as the main packages of each app under the app1, app2, and cmd directories in the figure connect the dependencies of the entire project together.
And generally speaking, the main package should be simple. In the main package, we will do some command line parameter analysis, resource initialization, log facility initialization, database connection initialization, etc., and then we will hand over the execution permissions of the program to more advanced execution control objects. In addition, some Go projects change the name cmd to app or other names, but its functions have not changed.
Next, let's look at the pkgN directory, which is a library file that stores the main packages that the project itself needs to use and are also the executable file depend on. At the same time, the packages under these directories can also be referenced by external projects.
Then there are and , which are configuration files used by Go language pack dependency management. As we mentioned earlier, Go version 1.11 introduced the Go Module construction mechanism. Here I suggest that all your new projects are based on Go Module for package dependency management, because this is the standard construction model officially recommended by Go.
For legacy projects that have not used Go Module for package dependency management, such as those who have previously adopted dep, glide, etc. as package dependency management tools, it is recommended to migrate to Go Module mode as soon as possible. The Go command supports converting / or glide to / directly.
Finally, let’s take a look at the vendor directory. vendor is a mechanism introduced in Go version 1.5 to cache specific version dependencies in the project locally. Before the introduction of the Go Modules mechanism, reproducible construction can be implemented based on vendor to ensure that executable programs built on the same source code are equivalent.
However, we treat the vendor directory as an optional directory here. The reason is that Go Module itself supports reproducible builds without using vendors. Of course, the Go Module mechanism also retains the vendor directory (the dependency packages under vendor can be generated through go mod vendor, and the vendor-based construction can be achieved through go build -mod=vendor). Generally, we only retain the vendor directory in the project root directory, otherwise it will cause unnecessary dependency selection complexity.
Of course, some developers like to use some third-party construction tools to assist in building, such as make, bazel, etc. You can place many script files (such as Makefile) involved in this type of external auxiliary build tool in the top-level directory of the project, just like in the Go Genesis project.
In addition, it is only necessary to explain here that the module introduced in Go 1.11 is a collection of packages that belong to the same version snap-in. And Go supports the existence of multiple modules in a project/repository, but this management method may introduce more complexity than a certain proportion of code duplication. Therefore, if there is a "divergence" of version management in the project structure, such as: the release versions of app1 and app2 are not always synchronized, then I suggest you split the project into multiple projects (repositories), each project is individually managed and evolved as a module.
Of course, if you have to store multiple modules in a code repository, the new version of Go commands also provide good support. For example, the following code repository multi-modules has three modules: mainmodule, module1 and module2:
$tree multi-modules
multi-modules
├── // mainmodule
├── module1
│ └── // module1
└── module2
└── // module2
We can distinguish the versions of different modules by git tag names. The tag name of the form is used for the mainmodule in the code repository; and the tag name of the form module1/ is used to indicate the version of module1; similarly, the tag name of the form module2/ is used to indicate the version of module2.
If the Go executable program project has one and only one executable program to be built, it will be easier to deal with. We can simplify the above project layout:
$tree -F -L 1 single-exe-layout
single-exe-layout
├──
├── internal/
├──
├── pkg1/
├── pkg2/
└── vendor/
You can see that we deleted the cmd directory and placed the main package of the only executable program in the project root directory, while the functions of other layout elements remain unchanged.
3.2 Typical structural layout of Go library projects
Well, here we have learned about the typical layout of Go executable program projects. Now let’s take a look at the typical structural layout of Go library projects.
Go library projects only expose Go packages to the outside world. The typical layout form of this type of project is as follows:
$tree -F lib-layout
lib-layout
├──
├── internal/
│ ├── pkga/
│ │ └── pkg_a.go
│ └── pkgb/
│ └── pkg_b.go
├── pkg1/
│ └──
└── pkg2/
└──
We see that the layout of library type projects is simpler than that of Go executable program projects. Because this type of project does not require the construction of executable programs, the cmd directory is removed.
Moreover, here, vendor is no longer an optional directory. For library-type projects, we do not recommend placing the vendor directory in the project to cache the third-party dependencies of the library itself. The library project only clearly states the module or package and version requirements of the project dependency through files.
The original intention of the Go library project is to expose the API to the outside (open source or internal disclosure of the organization). For packages that are only used internally and do not want to be exposed to the outside, they can be placed under the internal directory at the top of the project. Of course, internal can also have multiple directory levels and exist in any directory level in the project structure. The key is that project structure designers must clarify the application levels and scope of internal packages at each level.
For a Go library project with only one package, we can also further simplify the above layout. The simplified layout is as follows:
$tree -L 1 -F single-pkg-lib-layout
single-pkg-lib-layout
├──
├──
├──
└── internal/
After simplification, we place all source files of this unique package in the top-level directory of the project (such as the above and ) and the other layout elements are unchanged.
OK, now we have learned about the typical structural layout of the current Go project. However, in addition to these, you should also pay attention to the classic layout of early Go executable program projects, which is different.
3.3 Typical layout of early Go executable program projects
Many Go executable program projects established by early developers who adopted Go language were deeply influenced by the layout before Go Genesis Project 1.4. These projects aggregate all exposed Go packages in the pkg directory. Just like the layout in the previous Go version 1.3, their typical layout structure is as follows:
$tree -L 3 -F early-project-layout
early-project-layout
└── exe-layout/
├── cmd/
│ ├── app1/
│ └── app2/
├──
├── internal/
│ ├── pkga/
│ └── pkgb/
├── pkg/
│ ├── pkg1/
│ └── pkg2/
└── vendor/
We see that the pkg1 and pkg2 public packages originally placed in the top-level directory of the project are uniformly aggregated into the pkg directory. Moreover, this typical layout of this early Go executable program project has also been widely available within the Go community, and many newly built Go projects still adopt such project layout.
So, don’t be surprised when you see such a layout. You should clarify the "clustering" role played by the pkg directory under such layout. However, it is recommended that you prioritize the standard project layout when creating a new Go project.
4. The typical project structure of Go projects is divided into five parts
Go Module-related files placed on the top level of the project, including and ;
cmd directory: store the source code file of the main package corresponding to the executable file to be compiled and built by the project;
Project package directory: The non-main packages under each project are "tiled" in the root directory of the project, and each directory corresponds to a Go package;
internal directory: stores Go packages that are only referenced internally by the project, and these packages cannot be referenced outside the project;
vendor directory: This is an optional directory that exists to be compatible with the vendor build pattern introduced by Go 1.5. The contents in this directory are automatically maintained by Go commands and do not require manual intervention from developers.
The above is the detailed content of a deep understanding of the project code layout in Go. For more information about the Go project code layout, please pay attention to my other related articles!