Detailed explanation of how Golang uses the ttl mechanism to save memory data

ttl(time-to-live) Data survival time, we refer here to the data being stored in memory for a period of time, and cannot be read after the deadline, which is similar to Redis's ttl mechanism. This article only implements the ttl part, and does not consider serialization and deserialization.

Get the current time

Involving time calculation, here first introduces how to obtain the current time and the accuracy of time. For simplicity, the accuracy is to the second level.

Use to get the current time, or to get the timestamp.

now := ()      // current local time
sec := ()      // number of seconds since January 1, 1970 UTC
nsec := () // number of nanoseconds since January 1, 1970 UTC
(now)  // 
(sec)  // int64
(nsec) // int64

Output result:

2023-02-19 16:52:51.5894329 +0800 CST m=+0.004286801
1676796771
1676796771589432900

Data structure

First, define the data structure, data structure and the structure of the storage data container:

type Data struct {
	Key       string
	Value     interface{}
	Timestamp int64
}
type Heap struct {
	dataMx *
	data   map[string]Data
}

Data includes key and value and ttl time (units of seconds), Heap container includes map type data and RWMutex read and write locks, and read and write locks support concurrent operations.

Here are some methods for defining the Heap structure.

Heap Operation

The main methods include three methods: New, Set, Del, and Get.

func New() *Heap {
	return &Heap{
		dataMx: &{},
		data:   map[string]Data{},
	}
}
func (h *Heap) Set(key string, value interface{}, ttl int64) {
	if ttl == 0 {
		return
	}
	data := Data{
		Key:       key,
		Value:     value,
		Timestamp: ().Unix(),
	}
	if ttl > 0 {
		 += ttl
	} else if ttl < 0 {
		 = -1
	}
	()
	[key] = data
	()
}
func (h *Heap) Get(key string) (val interface{}, ok bool) {
	var data Data
	()
	data, ok = [key]
	()
	if ok {
		if  != -1 &&  <= ().Unix() {
			(key)
			ok = false
		} else {
			val = 
		}
	}
	return
}
func (h *Heap) Del(key string) {
	()
	_, ok := [key]
	()
	if !ok {
		return
	}
	()
	delete(, key)
	()
}

There is no need to explain the New method, let’s look at the Set method directly.

Set method implementation logic: if ttl is 0, it will be returned directly. Otherwise, the Data data is initialized first, and the timestamp with the current time of Data is initialized here; then judge ttl, if it is greater than zero, the timestamp of Data is added to ttl, otherwise it is -1; the data of Heap is stored by read and write locks.

The Del method first reads the corresponding data of the key through the read lock, and returns directly if it fails (it may have expired, other coroutines have obtained it), and then deletes the data directly.

The Get method has the same reading logic as Del. If it is read correctly, it will judge the timestamp. If it is not equal to -1 and is less than the current time, it means it has expired. Call the Del method for deletion, and return nil and false; otherwise, it will return value and true.

Testing ttl container Heap

First define heap, then call Set method, and add data key, value, and ttl to 2 seconds:

func main() {
	keyTag := "key"
	heap := New()
	defer func() {
		(keyTag)
	}()
	(keyTag, "value", 2)
	(1 * )
	val, flag := (keyTag)
	("%v, %v\n", val, flag)
	(1 * )
	val, flag = (keyTag)
	("%v, %v\n", val, flag)
}

Then wait for 1 second before calling the Get method, and the result is the same as expected when the two direct results are:

value, true
<nil>, false

Complete code

The complete code is given below:

package main
import (
	"fmt"
	"sync"
	"time"
)
type Data struct {
	Key       string
	Value     interface{}
	Timestamp int64
}
type Heap struct {
	dataMx *
	data   map[string]Data
}
func New() *Heap {
	return &Heap{
		dataMx: &{},
		data:   map[string]Data{},
	}
}
func (h *Heap) Set(key string, value interface{}, ttl int64) {
	if ttl == 0 {
		return
	}
	data := Data{
		Key:       key,
		Value:     value,
		Timestamp: ().Unix(),
	}
	if ttl > 0 {
		 += ttl
	} else if ttl < 0 {
		 = -1
	}
	()
	[key] = data
	()
}
func (h *Heap) Get(key string) (val interface{}, ok bool) {
	var data Data
	()
	data, ok = [key]
	()
	if ok {
		if  != -1 &&  <= ().Unix() {
			(key)
			ok = false
		} else {
			val = 
		}
	}
	return
}
func (h *Heap) Del(key string) {
	()
	_, ok := [key]
	()
	if !ok {
		return
	}
	()
	delete(, key)
	()
}
func main() {
	keyTag := "key"
	heap := New()
	defer func() {
		(keyTag)
	}()
	(keyTag, "value", 2)
	(1 * )
	val, flag := (keyTag)
	("%v, %v\n", val, flag)
	(1 * )
	val, flag = (keyTag)
	("%v, %v\n", val, flag)
}

Summarize

This article explains that Golang automatically fails data if it implements the ttl mechanism in memory. First, we introduce the timestamp principle, then define the data structure, and simply implement the Set, Get, and Del methods to implement the ttl mechanism. In the future, add serialization functions: save and restore. Reference implementation:/leprosus/golang-ttl-map。

This is the article about Golang's detailed explanation of how to save memory data using the ttl mechanism. For more related content on Go to save memory data, please search for my previous articles or continue browsing the following related articles. I hope everyone will support me in the future!