In Go language development, JSON (JavaScript Object Notation) is often used as the main serialization format for data exchange due to its simplicity and extensive compatibility. However, when you go deeper with JSON,You may find that it is not always the best choice。
This article will discuss some limitations of JSON serialization, which is also considered aSmall pitBar. And give some commonly used solutions.
Potential Issues in JSON Serialization
Let's first look at an example of serialization and deserialization using JSON:
package json_demo
import (
"encoding/json"
"fmt"
)
func JsonEnDeDemo() {
d1 := make(map[string]interface{})
d2 := make(map[string]interface{})
var (
age int = 18
name string = "Alex"
height float32 = 1.75
)
d1["name"] = name
d1["age"] = age
d1["height"] = height
ret, err := (d1)
if err != nil {
(" failed: %v\n", err)
return
}
// : {"age":18,"height":1.75,"name":"Alex"}
(": %s\n", string(ret))
err = (ret, &d2)
if err != nil {
(" failed: %v\n", err)
return
}
// : map[age:18 height:1.75 name:Alex]
(": %v\n", d2)
// Here we can find a problem: when the json package in Go language serializes interface{} type, it will serialize all numeric types (integral, floating point, etc.) to float64 type for k, v := range d2 {
// key: age, value: 18, type:float64
// key: height, value: 1.75, type:float64
// key: name, value: Alex, type:string
("key: %s, value: %v, type:%T \n", k, v, v)
}
}
This code shows how to include aname、ageandheightThe Go map data structure is serialized to a JSON string and then deserialized back. Everything seems to be OK, but please pay attention to the data type changes after deserialization.
The output after running the code may surprise you:
: {"age":18,"height":1.75,"name":"Alex"}
: map[age:18 height:1.75 name:Alex]
key: age, value: 18, type:float64
key: height, value: 1.75, type:float64
key: name, value: Alex, type:string
question: We found that although in the original dataageyesinttype,heightyesfloat32Types, but after JSON deserialization, they all becomefloat64type.
Go languageencoding/jsonThe package will convert all numeric types (including integer, floating point, etc.)AllConvert tofloat64, So, is there a way to prevent the type from being lost? There is really!
gob binary protocol, efficient and reserved type Go-specific serialization
To avoid this limitation of JSON, we can use the Go-specific GOB serialization method. GOB not only efficiently serializes data, but also retains the original data type.
Here are examples of serialization and deserialization using GOB:
package json_demo
import (
"bytes"
"encoding/gob"
"fmt"
)
func GobEnDeDemo() {
d1 := make(map[string]interface{})
d2 := make(map[string]interface{})
var (
age int = 18
name string = "Alex"
height float32 = 1.75
)
d1["name"] = name
d1["age"] = age
d1["height"] = height
// encode
buf := new()
enc := (buf)
err := (d1)
if err != nil {
(" failed: %v\n", err)
return
}
b := ()
// : [13 127 4 1 2 255 128 0 1 12 1 16 0 0 57 255 128 0 3 4 110 97 109 101 6 115 116 114 105 110 103 12 6 0 4 65 108 101 120 3 97 103 101 3 105 110 116 4 2 0 36 6 104 101 105 103 104 116 7 102 108 111 97 116 51 50 8 4 0 254 252 63]
(": ", b)
// decode
dec := ((b))
err = (&d2)
if err != nil {
(" failed: %v\n", err)
return
}
// : map[age:18 height:1.75 name:Alex]
(": %v\n", d2)
for k, v := range d2 {
// key: name, value: Alex, type:string
// key: age, value: 18, type:int
// key: height, value: 1.75, type:float32
("key: %s, value: %v, type:%T \n", k, v, v)
}
}
As can be seen from the above code, GOB serialization is not only retainedageofintTypes andheightoffloat32type, and can also efficiently encode data. This makes GOB an ideal choice for passing data inside a Go program.
Third-party package msgpack
msgpackIs an efficient binary serialization format that allows you to exchange data between multiple languages (such as JSON). But it's faster and smaller.
First, you need to download this package
go get -v /vmihailenco/msgpack/v5
Let’s take a look at an example using msgpack:
package json_demo
import (
"fmt"
"/vmihailenco/msgpack/v5"
)
func MsgpackEnDeDemo() {
// msgpack serialization example d1 := make(map[string]interface{})
d2 := make(map[string]interface{})
var (
age int = 18
name string = "Alex"
height float32 = 1.75
)
d1["name"] = name
d1["age"] = age
d1["height"] = height
// encode
b, err := (d1)
if err != nil {
(" failed: %v\n", err)
return
}
// : [131 164 110 97 109 101 164 65 108 101 120 163 97 103 101 18 166 104 101 105 103 104 116 202 63 224 0 0]
(": ", b)
// decode
err = (b, &d2)
if err != nil {
(" failed: %v\n", err)
return
}
// : map[age:18 height:1.75 name:Alex]
(": %v\n", d2)
for k, v := range d2 {
// key: age, value: 18, type:int8
// key: height, value: 1.75, type:float32
// key: name, value: Alex, type:string
("key: %s, value: %v, type:%T \n", k, v, v)
}
}
Advantages of msgpack：
- Efficient and compact: The data size is smaller than JSON, and the serialization and deserialization speeds are faster.
- Type keep: Similar to GOB, msgpack can also maintain the original data type.
Summarize
- json: Although widely used and easy to read, there are potential accuracy issues when dealing with numeric types.
- gob: Suitable for data transmission within Go language programs, retained types and excellent performance, but only for Go.
- msgpack: Very useful when efficient and compact cross-language data exchange is required, while also retaining data types.
By comparing these three serialization methods, I hope you can choose the right tool according to your actual needs. In Go programs that need to ensure type and performance, gob and msgpack may be better choices than json. However, you can also use the json package to deserialize, but when taking values, you need to obtain the previous type through type assertions.
This is the end of this article about json serialization optimization techniques in Go. For more related Go json serialization content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!