introduction
Although GO does not consume a lot of memory, there are still some tips to save memory. Good coding habits are the qualities that every programmer should have.
Pre-allocated slices
An array is a collection of the same type with continuous memory. The length and element type are specified when defining an array type.
Because the length of arrays is part of their type, the main problem with arrays is that they are fixed in size and cannot be adjusted.
Unlike array types, slice types do not need to specify length. Slices are declared the same way as arrays, but without a number of elements.
Slices are wrappers for arrays, they do not own any data - they are references to arrays. They consist of pointers to the array, length and capacity (number of elements in the underlying array).
When you add a new value to a slice that does not have enough capacity - a new array with larger capacity is created and the values from the current array are copied into the new array. This results in unnecessary memory allocation and CPU cycles.
To better understand this, let's take a look at the following code snippet:
func main() {
var ints []int
("Address: %p, Length: %d, Capacity: %d, Values: %v\n", ints, len(ints), cap(ints), ints)
for i := 0; i < 5; i++ {
ints = append(ints, i)
("Address: %p, Length: %d, Capacity: %d, Values: %v\n", ints, len(ints), cap(ints), ints)
}
}
result
Address: 0x0, Length: 0, Capacity: 0, Values: []
Address: 0xc0000160d0, Length: 1, Capacity: 1, Values: [0]
Address: 0xc0000160e0, Length: 2, Capacity: 2, Values: [0 1]
Address: 0xc000020100, Length: 3, Capacity: 4, Values: [0 1 2]
Address: 0xc000020100, Length: 4, Capacity: 4, Values: [0 1 2 3]
Address: 0xc00001a180, Length: 5, Capacity: 8, Values: [0 1 2 3 4]
You can see the first declaration of the arrayvar ints []intWhen it is, it does not allocate memory, the memory address is 0, and the size and capacity are also 0. After that, each expansion of capacity is multiples of 2, and the memory address of each expansion of capacity has changed.
When the capacity is <1024, it will increase by 2 times, and when the capacity is >=1024, it will increase by 1.25 times. Since Go 1.18, this has become more linear
func BenchmarkPreallocAssign(b *) {
ints := make([]int, )
for i := 0; i < ; i++ {
ints[i] = i
}
}
func BenchmarkAppend(b *) {
var ints []int
for i := 0; i < ; i++ {
ints = append(ints, i)
}
}
The results are as follows
goos: darwin
goarch: amd64
pkg: mygo
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkPreallocAssign-12 321257311 3.609 ns/op 8 B/op 0 allocs/op
BenchmarkAppend-12 183322678 12.37 ns/op 42 B/op 0 allocs/op
PASS
ok mygo 6.236s
From the above benchmarks, we can conclude that there is a big difference between assigning values to pre-allocated slices and appending values to slices. Pre-allocated sizes can speed up by more than 3 times, and memory allocation is also smaller.
The order of fields in a structure
The following structure is an example
type Post struct {
IsDraft bool // 1 byte
Title string // 16 bytes
ID int64 // 8 bytes
Description string // 16 bytes
IsDeleted bool // 1 byte
Author string // 16 bytes
CreatedAt // 24 bytes
}
func main(){
p := Post{}
((p))
}
The output above is 96 bytes, while all fields are added to 82 bytes. Where did the extra 14 bytes come from?
Modern 64-bit CPUs get data in 64-bit (8-byte) blocks
The first cycle takes 8 bytes, pulling the "IsDraft" field takes 1 byte and generates 7 unused bytes. It cannot occupy "half" fields.
The second and third cycles take the Title string, the fourth cycle takes the ID, and so on. When the IsDeleted field is reached, it uses 1 byte and has 7 bytes unused.
The key to memory savings is to sort fields from top to bottom by field occupancy. Sorting the above structures can be reduced to 88 bytes in size. The last two fields, IsDraft and IsDeleted, are placed in the same block, reducing the number of unused bytes from 14 (2x7) to 6 (1 x 6), saving 8 bytes in the process.
type Post struct {
CreatedAt // 24 bytes
Title string // 16 bytes
Description string // 16 bytes
Author string // 16 bytes
ID int64 // 8 bytes
IsDraft bool // 1 byte
IsDeleted bool // 1 byte
}
func main(){
p := Post{}
((p))
}
The output above is 88 bytes
Extreme situations
type Post struct {
IsDraft bool // 1 byte
I64 int64 // 8 bytes
IsDraft1 bool // 1 byte
I641 int64 // 8 bytes
IsDraft2 bool // 1 byte
I642 int64 // 8 bytes
IsDraft3 bool // 1 byte
I643 int64 // 8 bytes
IsDraft4 bool // 1 byte
I644 int64 // 8 bytes
IsDraft5 bool // 1 byte
I645 int64 // 8 bytes
IsDraft6 bool // 1 byte
I646 int64 // 8 bytes
IsDraft7 bool // 1 byte
I647 int64 // 8 bytes
}
type Post1 struct {
IsDraft bool // 1 byte
IsDraft1 bool // 1 byte
IsDraft2 bool // 1 byte
IsDraft3 bool // 1 byte
IsDraft4 bool // 1 byte
IsDraft5 bool // 1 byte
IsDraft6 bool // 1 byte
IsDraft7 bool // 1 byte
I64 int64 // 8 bytes
I641 int64 // 8 bytes
I642 int64 // 8 bytes
I643 int64 // 8 bytes
I644 int64 // 8 bytes
I645 int64 // 8 bytes
I646 int64 // 8 bytes
I647 int64 // 8 bytes
}
The first structure takes up 128 bytes and the second structure takes up 72 bytes. Space saving: (128-72)/129=43.75%.
Go types that occupy less than 8 bytes on a 64-bit architecture:
- bool: 1 byte
- int8/uint8: 1 byte
- int16/uint16: 2 bytes
- int32/uint32/rune: 4 bytes
- float32: 4 bytes
- byte: 1 byte
Use map[string]struct{} instead of map[string]bool
Go does not have built-in collections, usually usemap[string]bool{}Represents a collection. Although it is more readable (which is very important), it is wrong to use it as a collection because it has two states (false/true) and uses extra memory compared to an empty structure.
Empty structure (struct{}) is a structure type without additional fields, occupies zero bytes of storage space.
func BenchmarkBool(b *) {
m := make(map[uint]bool)
for i := uint(0); i < 100_000_000; i++ {
m[i] = true
}
}
func BenchmarkEmptyStruct(b *) {
m := make(map[uint]struct{})
for i := uint(0); i < 100_000_000; i++ {
m[i] = struct{}{}
}
}
result
goos: darwin
goarch: amd64
pkg: mygo
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkBool-12 1 24052439603 ns/op 3766222824 B/op 3902813 allocs/op
BenchmarkEmptyStruct-12 1 22450213018 ns/op 3418648448 B/op 3903556 allocs/op
PASS
ok mygo 46.937s
You can see that the execution speed has been improved a little, but the effect is not obvious.
One advantage of using bool values is that it is more convenient when searching. You only need to judge one value from the map, while using an empty structure requires you to judge the second value
m := make(map[string]bool{})
if m["key"]{
// Do something
}
v := make(map[string]struct{}{})
if _, ok := v["key"]; ok{
// Do something
}
refer to
【1】Simple memory saving tips in Go
【2】Easy memory-saving tricks in Go
The above is the detailed content of the examples of memory saving techniques in Go. For more information about memory saving techniques in Go, please pay attention to my other related articles!