The maximum number of Redis Key and optimization strategy sharing

1. Introduction

As a high-performance key-value storage database, Redis is widely used in cache, session storage, rankings and other scenarios. But in actual use, developers often care about one question: Is there an upper limit on the number of keys of Redis? If so, how do I optimize storage to support more keys?

This article will start from the theoretical upper limit of Redis Key, and combine actual memory limitations, configuration optimization, Java code examples, etc. to deeply explore the management strategies of Redis Key to help developers better plan and use Redis.

2. Theoretical upper limit of Redis Key

2.1 Redis's Key Storage Mechanism

Redis uses a hash table (Hash Table) to store Key-Value data, and its underlying implementation determines the maximum number of keys.

• Theoretical maximum key number:2^32 ≈ 4.29 billion(Limited by Redis hash table size).
• The maximum length of the key: 512MB (but the key is usually shorter in actual business).

2.2 Why 2^32?

Redis's hash table uses unsigned 32-bit integers to store the number of key-value pairs, so it can theoretically be stored at most.2^32A key. However, in actual production environments, memory limits and performance factors will make the number of keys far lower than this value.

3. Actual factors affecting the number of Redis Keys

3.1 Memory Limit

Redis is an in-memory database, and both Key and Value are stored in memory, so available memory is the key factor in determining the number of keys.

• Check Redis memory usage:

redis-cli info memory

Output example:

used_memory: 1024000  # Current memory usage (bytes)maxmemory: 2000000000 # Maximum memory limit (2GB)

Calculate the number of keys that can be stored:
Assuming each Key + Value consumes an average of 100 bytes, 1GB of memory can be approximately stored:

1GB / 100B ≈ 10,000,000 indivual Key

3.2 Redis configuration parameters

• maxmemory: Set Redis maximum memory usage (such asmaxmemory 2gb）。

• maxmemory-policy: Define the Key Elimination Strategy when memory is full, such as:

  • noeviction(No elimination, write error)
  • allkeys-lru(Eliminate the least used keys recently)
  • volatile-lru(Only eliminate keys with expiration time)

  Sample configuration (）：

maxmemory 2gb
maxmemory-policy allkeys-lru

3.3 Size optimization for Key and Value

• Key Optimization:
  • Avoid overly long keys, such as:

// Not recommendedString key = "user:session:1234567890:profile:settings:dark_mode";
// Recommended (shortened Key)String key = "u:1234567890:dark_mode";

• Value Optimization:
  • Use compression algorithms such as GZIP to store large JSON data.
  • Adopt more efficient serialization methods (such as Protocol Buffers instead of JSON).

4. How to monitor and manage Redis Key

4.1 Check the current number of keys

redis-cli dbsize          # Return the total number of keys in the current databaseredis-cli info keyspace   # View Key statistics for each database

4.2 Use SCAN to traverse Key (avoid blocking)

Use Jedis to traverse the Key in Java:

import ;
import ;
import ;

public class RedisKeyScanner {
    public static void main(String[] args) {
        Jedis jedis = new Jedis("localhost", 6379);
        String cursor = "0";
        ScanParams scanParams = new ScanParams().count(100); // 100 Keys per scan        do {
            ScanResult<String> scanResult = (cursor, scanParams);
            cursor = ();
            ().forEach(::println);
        } while (!("0"));
        ();
    }
}

4.3 Set Key Expiry Time

("user:1234:session", 3600, "session_data"); // Expired in 1 hour

5. Practical solutions for optimizing Redis Key storage

5.1 Using Redis Cluster Sharding

If a stand-alone Redis cannot support massive keys, you can use Redis Cluster for shard storage.

Java example (Lettuce client):

import ;
import ;
import ;

public class RedisClusterExample {
    public static void main(String[] args) {
        RedisClusterClient clusterClient = (
            "redis://node1:6379", "redis://node2:6379", "redis://node3:6379"
        );
        StatefulRedisClusterConnection<String, String> connection = ();
        ().set("cluster_key", "Hello Redis Cluster!");
        (().get("cluster_key"));
        ();
        ();
    }
}

5.2 Use a Hash structure to store multiple fields

If multiple keys belong to the same object, you can use Hash to reduce the number of keys:

//Storing user information (avoid multiple keys)("user:1000", "name", "Alice");
("user:1000", "age", "30");
("user:1000", "email", "alice@");

5.3 Batch operation using Pipeline

Reduce network overhead and improve write performance:

Pipeline pipeline = ();
for (int i = 0; i < 1000; i++) {
    ("key:" + i, "value:" + i);
}
();

6. Conclusion

Best Practice Recommendations:

1. Control the Key size to avoid storing too long Key or Values.
2. Set up reasonablymaxmemoryand elimination strategies to prevent memory overflow.
3. Use Redis Cluster to disperse Key storage pressure.
4. Monitor Key growth trends to avoid performance degradation due to unlimited growth.

Through reasonable optimization, Redis can easily support tens of millions or even billions of keys to meet the needs of high concurrent business.

The above is the detailed content of the upper limit of Redis Key and optimization strategy sharing. For more information about the upper limit of Redis Key and optimization of Redis Key, please pay attention to my other related articles!