SoFunction
Updated on 2025-03-09

Detailed explanation of the issue of using docker compose to deploy golang’s Athens private proxy

Private agent construction in go

Preface

Recently, there was a problem with the company's agent. I just took this opportunity to learn how to build a private agent.

Why choose athens

The selection criteria for privatization agents are nothing more than the following points

1. Hosting private modules;

2. Exclude access to public modules;

3. Store public modules;

Features of athens:

Athens can first configure access to private repositories;

Athens will store the packets pulled each time. If the module has not passed athens before, athens will request data from the target address. When returned to the client, the module will be stored in the storage, which implementsgo mod downloadIt will only happen once forever;

Athens' policy for processing storage is to append only. If a module is saved, it will never change. Even if the developer pushes the tag, it will not be deleted;

Athens can also configure download policies to filter some packages with security risks.

Athens supports disk, mongo, gcs, s3, minio, external storage/customization, but disk is generally recommended.

Deploy using docker-compose

The official website has provided solutions for deployment through docker and binary systems. Here we adhere to the principle that good memory is not as good as bad pen tips, and we have also made records here.

Configure authentication information for private repository

pass.netrcTo configure the file, you can place the address of your own private repository, as well as user and password authentication information.

# cat .netrc
machine  login test-name password test-pass

There are several private repositories, just configure a few

Configure the download mode

passThe download mode(Download mode configuration strategy) is more popular in ATHENS now, and was previously passedFiltering modulesThe (filter mode) method has been deprecated.

See how to configure it

# DownloadMode defines how Athens behaves when a module@version
# is not found in storage. There are 4 options:
# 1. "sync" (default): download the module synchronously and
# return the results to the client.
# 2. "async": return 404, but asynchronously store the module
# in the storage backend.
# 3. "redirect": return a 301 redirect status to the client
# with the base URL as the DownloadRedirectURL from below.
# 4. "async_redirect": same as option number 3 but it will
# asynchronously store the module to the backend.
# 5. "none": return 404 if a module is not found and do nothing.
# 6. "file:<path>": will point to an HCL file that specifies
# any of the 5 options above based on different import paths.
# 7. "custom:<base64-encoded-hcl>" is the same as option 6
# but the file is fully encoded in the option. This is
# useful for using an environment variable in serverless
# deployments.
# Env override: ATHENS_DOWNLOAD_MODE
DownloadMode = "sync"

The environment variable ATHENS_DOWNLOAD_MODE can be specified or modified.To configure, the default is sync

ATHENS_DOWNLOAD_MODE Specified content:

1. Passfile:<path>Specify a hcl file, which can set download modes for different repositories;

2. Passcustom:<base64-encoded-hcl>Specify a base64-encoded HCL file;

3. Specify the specific global policy.sync, async, none, redirect, or async_redirect, This is a global setting, the above two can be customized policy groups.

Check out the specific download mode

  • sync: Download modules from VCS through synchronousgo mod download, persist it to storage and return it to the user immediately. Note that this is the default behavior;

  • async: Return 404 to the client and download asynchronouslymodule@versionand persist it to storage;

  • none: Return 404 and do nothing;

  • redirect: Redirect to the upstream agent (for example), and do nothing afterwards;

  • async_redirect: redirect to the upstream agent (e.g.) and download asynchronouslymodule@versionand persist it to storage;

Let's look at the hcl file for configuring the policy

# cat   
downloadURL = ""
mode = "async_redirect"
download "/*" {
    mode = "sync"
}

deploy

Use docker-composer to deploy here

version: '2'
services:
  athens:
    image: gomods/athens:v0.11.0
    restart: always
    container_name: athens_proxy
    ports:
      - "3000:3000"
    volumes:
      - ./.netrc:/root/.netrc
      - ./athens-storage:/var/lib/athens
      - ./:/root/
    environment:
      - ATHENS_NETRC_PATH=/root/.netrc
      - ATHENS_STORAGE_TYPE=disk
      - ATHENS_DISK_STORAGE_ROOT=/var/lib/athens
      - ATHENS_GOGET_WORKERS=100
      - ATHENS_DOWNLOAD_MODE=file:/root/
      - ATHENS_GONOSUM_PATTERNS=

ATHENS_GONOSUM_PATTERNS: Configured as a private library address, the configured warehouse address will not perform security verification.

Go is for security considerations. In order to ensure that the developer's dependency library is not maliciously hijacked and tampered with, GOSUMDB environment variables are introduced to set the verification server.

When you change (update/add) locally, Go will automatically go to this server to verify data to ensure that the code base you are under is the same as the code base you are under others in the world. If there is any problem, there will be a big safety tip. Of course, these operations behind them have been integrated into Go, and developers do not need to perform additional operations.

For our private repository, it is definitely not possible to pass the verification when checking the public safety verification library. We can set code repository that does not perform verification through the ATHENS_GONOSUM_PATTERNS environment variable. It can set multiple matching paths separated by commas.

start updocker-compose up -d

Client settings proxyexport GOPROXY=http://xxxx:3000

This way we can use our proxy services

Because the ATHENS_STORAGE_TYPE selected is disk, the athens service will also download the resource package to the configured ATHENS_DISK_STORAGE_ROOT while pulling the resource package.

Authenticate a private repository using a secret key

Passed above.netrcThe method to authenticate a private warehouse, because the account password is inscription is not good, so you can use the secret key to authenticate it.

1. Configuration key

First check if the computer has a secret key

# cd .ssh
# ls
id_rsa		id_rsa.pub

If not, the following command is generated

# ssh-keygen -t rsa -C "youremail@"

Change your email address to your own and press the car all the way

Thenid_rsa.pubAdd the content of the public key to your own private repository. How to add your own google? It's relatively simple

2. Configure HTTP and SSH rewrite rules

# cat gitconfig 
[url "ssh://git@"]
        insteadOf =

3. Configure SSH to bypass the host SSH key verification

# cat config 
Host 
Hostname 
StrictHostKeyChecking no
IdentityFile /root/.ssh/id_rsa

Map the authentication information configured above into the container

version: '2'
services:
  athens:
    image: gomods/athens:v0.11.0
    restart: always
    container_name: athens_proxy
    ports:
      - "3000:3000"
    volumes:
      - ./athens-storage:/var/lib/athens
      - ./:/root/
      - ./gitconfig:/root/.gitconfig
      - ./ssh-keys:/root/.ssh
    environment:
      - ATHENS_STORAGE_TYPE=disk
      - ATHENS_DISK_STORAGE_ROOT=/var/lib/athens
      - ATHENS_GOGET_WORKERS=100
      - ATHENS_DOWNLOAD_MODE=file:/root/
      - ATHENS_GONOSUM_PATTERNS=

This will enable the authentication of the key

You need to pay attention to the permissions of the private key. At the beginning, I didn't pay attention to it, and the following error was reported in the execution.

        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
        @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
        Permissions 0644 for '/root/.ssh/id_rsa' are too open.
        It is required that your private key files are NOT accessible by others.
        This private key will be ignored.
        Load key "/root/.ssh/id_rsa": bad permissions
        git@: Permission denied (publickey).
        fatal: Could not read from remote repository.

You can infer that the permissions are too large and the private key file cannot be accessed by others.

Just modify the permissions

ssh-keys # chmod 600 id_rsa

For specific demo address, please refer toathens Private Agent Deployment

refer to

【Introduction ATHENS】

/zh/intro/

【download】

/gomods/athens/blob/main/docs/content/configuration/

【Athens build golang private proxy】

/boilingfrog/Go-POINT/blob/master/golang/go_environment/athensBuild golang private proxy.md

【Deploy golang's Athens private proxy using docker-compose】

/boilingfrog/Go-POINT/blob/master/golang/go_environment/athensBuild golang private proxy.md

This is the article about using docker-compose to deploy golang’s Athens private agent. For more related docker-compose to deploy Athens private agent, please search for my previous article or continue browsing the related articles below. I hope everyone will support me in the future!