Golang uses CGO and Plugin technology to run and load C dynamic library

Article introduction

This article introduces a technique for the Golang program to load C dynamic libraries at runtime, skipping the process of linking C dynamic libraries in the Golang project compilation stage, improving the flexibility of Golang project development and deployment.

Technical background

The Golang program calls the OpenCC dynamic library functions to perform the traditional Chinese to simplified Chinese operation. It is necessary to not link dynamic libraries at compile time, and only load OpenCC dynamic libraries when the program is running.

The OpenCC library is a traditional Chinese conversion program written in C++ and provides a C language API interface.Open source project address

CGO technology is a way to let the Golang language use C language code, which can link C dynamic libraries when Golang program is compiled. After decades of development, C language has rich open source projects. When the Golang ecosystem is not yet perfect, we often need to use some mature C open source projects.

Plugin is a new plug-in system in Golang version 1.8 that allows programmers to use dynamic link libraries to build loosely coupled modular programs that are loaded and bound dynamically while the program is running.

This article explains 2 solutions step by step:

Solution 1:Using CGO technology, the interface of OpenCC is bound at compile time.
Solution 2:Introduced Plugin technology, dynamic libraries are loaded when the program is run.

Solution 1

Solution 1 is the initial solution, using CGO technology to bind the OpenCC interface during compilation.

Golang -> CGO ->

Write a CGO interface definition file:

// 
#include <>

void* Opencc_New(const char *configFile);
void* Opencc_Delete(void *id);
const char *Opencc_Convert(void *id, const char *input);
void Opencc_Free_String(char *p);

// 
#include <>
#include <>
#include <>
#include "opencc/src/"

const char *Convert(const char *input, const char *config) {
        if(strlen(config) > 16) {
                return 0;
        }

        char configFile[256] = "/usr/share/opencc/";
        strcat(configFile, config);
        strcat(configFile, ".json");

        opencc_t p = opencc_open(configFile);
        char *out = opencc_convert_utf8(p, input, strlen(input));
        out[strlen(input)] = '\0';

        opencc_close(p);

        return out;
}

void Convert_free_string(char *p) {
        opencc_convert_utf8_free(p);
}

void* Opencc_New(const char *configFile) {
        return opencc_open(configFile);
}

void Opencc_Delete(void *id) {
        opencc_close(id);
}

const char *Opencc_Convert(void *id, const char *input) {
        char *output = opencc_convert_utf8(id, input, strlen(input));
        output[strlen(input)] = '\0';
        return output;
}

void Opencc_Free_String(char *p) {
        opencc_convert_utf8_free(p);
}

// 
package opencc

import "unsafe"
// #cgo LDFLAGS: -L${SRCDIR}/output/lib -lopencc
// #include ""
import "C"
func NewConverter(config string)  {
        return C.Opencc_New((config))
}
func Convert(p , input string) string {
        inputChars := (input)
        outputChars := C.Opencc_Convert(p, inputChars)

        defer C.Opencc_Free_String(inputChars)
        defer C.Opencc_Free_String(outputChars)

        result := (outputChars)
        return result
}
func Close(p ) {
        C.Opencc_Delete(p)
}
func ConvertOneTime(input string, config string) string {
        p := NewConverter(config)
        defer Close(p)
        return Convert(p, input)
}
func ConvertAsync(input string, config string, callback func(output string)) {
        go func() {
                callback(ConvertOneTime(input, config))
        }()
}

// Call the OpenCC dynamic library function.package main

import "fmt"
import "your/repository/go-opencc"
const (
        input = "Chinese Mouse Software Printer"
        config_s2t = "/usr/share/opencc/"
        config_t2s = "/usr/share/opencc/"
)
func main() {
        ("Test Converter class:")
        c := (config_s2t)
        defer ()
        output := (input)
        (output)

        ("Test Convert function:")
        s := (input, config_s2t)
        (s)
        ((s, config_t2s))

        ("Test ConvertAsync function:")
        retChan := make(chan string)
        (input, config_s2t, func(output string) {
                retChan &lt;- output
        })

        (&lt;- retChan)
}

Scheme 1: You can correctly link the file and successfully execute the Convert function for traditional Chinese conversion. But there is a problem. This solution is not easy to compile on Mac systems, and it is necessary to install the OpenCC project in the Mac system. If the project calling OpenCC is developed by 10 people, it needs to be compiled on a 10-person Mac computer, which is difficult to develop and collaborate and inconvenient to deploy.

Solution 2

Introduced Plugin technology, dynamic libraries are loaded when the program is run.

Golang -> Plugin -> libgo_opencc.so -> CGO ->

Write a Plugin dynamic link library.

// opencc_lib.go
package main
import (
        "unsafe"

        opencc "your/repository/go-opencc"
)

type openccConverter string

// NewConverter Create Converterfunc (s openccConverter) NewConverter(config string)  {
        return (config)
}

// Convert conversion functionfunc (s openccConverter) Convert(p , input string) string {
        return (p, input)
}

// Close releases the memory resources occupied by Converter (converter is no longer used)func (s openccConverter) Close(p ) {
        (p)
}
// ConvertOneTime conversion function (convert once, the function will load the configuration file every time it is called, which has performance impact)func (s openccConverter) ConvertOneTime(input string, config string) string {
        return (input, config)
}
// OpenccConverter export symble
var OpenccConverter openccConverter

Compile dynamic library Create the output directory and compile and generate the ./output/lib/libgo_opencc.so dynamic library.

#!/bin/bash

mkdir -p output

cd opencc
./
cd ..
cp -rf opencc/output/* ./output
go build -buildmode=plugin -o ./output/lib/libgo_opencc.so ./lib/opencc_lib.go

Use Plugin to load libgo_opencc.so and call OpenCC's functions.

package main
import (
        "os"
        "plugin"
        "testing"
        "unsafe"
        "fmt"
)
// Implement the interface of opencc_lib.gotype OpenccConverter interface {
        NewConverter(config string) 
        Convert(p , input string) string
        Close(p )
        ConvertOneTime(input string, config string) string
}

func TestOpenccSo(t *) {
        openccPlugin, pluginErr := ("/your/path/to/go-opencc/output/lib/libgo_opencc.so")
        if pluginErr != nil {
                panic(pluginErr)
        }

        symbol, cerr := ("OpenccConverter")
        if cerr != nil {
                ("%+v", cerr)
                (1)
        }

        plug, ok := symbol.(OpenccConverter)
        if !ok {
                ("unexpected type from module symbol")
                (1)
        }

        config := "/usr/share/opencc/"
        pointer := (config)
        defer (pointer)

        input := "Traditional Han"

        output := (pointer, input)

        ("output: %s", output)
}
// Run the test TestOpenccSo，Output “Traditional Chinese characters”。

Solution 2 implements the loading of the libgo_opencc.so dynamic library through Plugin technology when the program is running, and then linked to it by libgo_opencc.so to compile Golang programs on Mac and Linux systems, without the need to deploy OpenCC projects on each development machine. When it is finally released to the production environment, the compilation and packaging platform will package and publish libgo_opencc.so together.

This is the article about Golang using CGO and Plugin technology to run the loading of C dynamic libraries. For more related content on Go loading of C dynamic libraries, please search for my previous articles or continue browsing the following related articles. I hope everyone will support me in the future!