Go language uses Swiss Table to implement faster maps

In Go language, map is a very common data structure used to store key-value pairs. However, in high concurrency and high performance scenarios, the map implementation in the standard library may not meet the requirements. Swiss Table is an efficient hash table implementation, originally introduced by Google in C++ and later adopted by other languages ​​such as Rust. This article will explore how to use the idea of ​​Swiss Table to achieve a faster Go map.

1. Introduction to Swiss Table

Swiss Table is a hash table implementation based on open addressing method, with the following features:

• Cache-friendly: Swiss Table improves cache hit rate by storing metadata (such as part of the hash) in contiguous blocks of memory.
• SIMD Optimization: Swiss Table uses SIMD (Single Instruction Multi-Data Stream) instructions to speed up search operations.
• Low Memory Overhead: Swiss Table reduces memory overhead with compact metadata storage.

2. Swiss Table Implementation in Go

Although the Go language itself does not directly provide the implementation of Swiss Table, we can learn from its ideas to implement an efficient hash table. Here is a simplified version of the Swiss Table implementation.

2.1 Data Structure

First, we define the data structure of the hash table:

package swisstable

import (
	"unsafe"
)

const (
	groupSize    = 16 // Size of each group	empty        = 0  // empty slot mark	deleted      = 1  // Delete slot mark	metadataSize = groupSize / 8 // Metadata size for each group)

type entry struct {
	key   string
	value interface{}
}

type SwissTable struct {
	metadata []byte // Metadata array	entries  []entry // Array of key-value pairs	size     int     // The number of key-value pairs currently stored	capacity int     // Total capacity of hash table}

2.2 Hash function

Swiss Table uses a hash function to determine the position of the key. We can use Go's built-in hash function:

func hash(key string) uint64 {
    h := uint64(5381)
    for i := 0; i < len(key); i++ {
        h = (h << 5) + h + uint64(key[i])
    }
    return h
}

2.3 Search operations

Find operations are at the heart of Swiss Table. We determine the group where the key is located by part of the hash value, and then look up the key in that group:

func (st *SwissTable) find(key string) (int, bool) {
	h := hash(key)
	groupIndex := int(h % uint64(/groupSize))
	start := groupIndex * groupSize

	for i := 0; i < groupSize; i++ {
		index := start + i
		if index >=  {
			index -= 
		}

		metadata := [index/metadataSize]
		bit := byte(1 << (index % metadataSize))

		if metadata&bit == 0 {
			return -1, false // not found		}

		if [index].key == key {
			return index, true // turn up		}
	}

	return -1, false // not found}

2.4 Insert operation

The insert operation first looks for whether the key exists, and if it exists, update the value, otherwise insert a new key-value pair:

func (st *SwissTable) Insert(key string, value interface{}) {
	index, exists := (key)
	if exists {
		[index].value = value
		return
	}

	if  >=  {
		()
	}

	h := hash(key)
	groupIndex := int(h % uint64(/groupSize))
	start := groupIndex * groupSize

	for i := 0; i < groupSize; i++ {
		index := start + i
		if index >=  {
			index -= 
		}

		metadata := [index/metadataSize]
		bit := byte(1 << (index % metadataSize))

		if metadata&bit == 0 {
			[index] = entry{key, value}
			[index/metadataSize] |= bit
			++
			return
		}
	}

	()
	(key, value)
}

2.5 Delete operation

The delete operation marks the slot in the delete state, but does not release the memory immediately:

func (st *SwissTable) Delete(key string) {
    index, exists := (key)
    if !exists {
        return
    }

    [index/metadataSize] &^= byte(1 << (index % metadataSize))
    [index] = entry{"", nil}
    --
}

2.6 Expansion operation

When the load factor of the hash table is too high, we need to expand the capacity:

func (st *SwissTable) resize() {
	newCapacity :=  * 2
	newMetadata := make([]byte, newCapacity/metadataSize)
	newEntries := make([]entry, newCapacity)

	oldEntries := 
	 = newMetadata
	 = newEntries
	 = newCapacity
	 = 0

	for _, entry := range oldEntries {
		if  != "" {
			(, )
		}
	}
}

3. Performance comparison

Through the above implementation, we can compare the performance of the standard library map and Swiss Table. Here is a simple performance test:

package main

import (
	"fmt"
	"time"
)

func main() {
	// Standard library map	start := ()
	m := make(map[string]interface{})
	for i := 0; i < 1000000; i++ {
		m[("key%d", i)] = i
	}
	("Standard map insert time:", (start))

	// Swiss Table
	start = ()
	st := ()
	for i := 0; i < 1000000; i++ {
		(("key%d", i), i)
	}
	("Swiss Table insert time:", (start))
}

4. Summary

By drawing on the idea of ​​Swiss Table, we can implement an efficient hash table in Go. Although Go's standard library maps are already very efficient, the implementation of Swiss Table may lead to better performance in certain scenarios. In the future, with the development of Go language, more high-performance data structures may be introduced into the standard library or third-party library.

This is the article about Go using Swiss Table to implement faster maps. For more related Go map content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!