Redis CLI List Keys: A Comprehensive Guide

Redis is an ubiquitous in-memory data store used across industries for blazing fast caching, rate limiting, pub/sub messaging, and beyond. As an open-source tool, Redis offers impressive performance and scalability crucial for many modern, high traffic applications.

In this comprehensive guide, we will dive deep into listing and retrieving Redis keys through the Redis command-line interface (Redis CLI), explore real-world use cases, analyze alternatives, examine performance tradeoffs, and provide expert insights into Redis key best practices.

Prerequisites

To follow along with the examples below, you will need:

• A Linux server with Redis 6.2+ installed and running
• Working knowledge of Redis data structures like strings, hashes, lists, sets
• Basic familiarity with Redis CLI commands
• Ability to directly access Redis CLI (usually via redis-cli)

Understanding Redis Keys

In Redis, all data is stored in key-value pairs. The key is an arbitrary string which maps to a value that can be a string, list, hash, set, sorted set, bitmap, hyperloglog, or other Redis data type.

Developers leverage the incredible flexibility of Redis keys and values to build caching layers, rate limiting systems, pub/sub channels, distributed locking infrastructure, and more.

Here is an example key-value pair in Redis representing website page views:

SET page_views 42

This sets the key page_views to the integer value 42. Later we can efficiently retrieve this value using the Redis GET command:

GET page_views

Which would return 42 without needing to access slower backend storage.

As Redis databases scale, proper naming and structure of keys becomes extremely important for understandable, maintainable code.

Real-World Redis Key Examples

To understand how keys are actually used in production, let‘s explore some real-world examples across different industries:

Caching

Ultra-fast read performance makes Redis an ideal caching layer. Keys represent unique cache entries:

SET article:42 "{JSON data...}" 
SET product:abc123 "{JSON data...}"

The key names contain the unique identifier, while JSON values store cached content.

Rate Limiting

Redis counters enable precise rate limiting. Unique keys track requests per client:

INCR rate_limit:192.168.1.5   
TTL rate_limit:192.168.1.5

The client IP address becomes the key, while Redis counters and expiration provide rate limiting.

Locking

Unique, expiring keys simulate locks to control access to resources:

SET lock:account:123 true
SET lock:account:123 EX 10

The key name identifies the locked resource, while expiration releases the lock.

Pub/Sub Messaging

Redis pub/sub channels use keys to identify message streams:

SUBSCRIBE alerts
PUBLISH alerts "New alert generated" 

Clients subscribe to the alerts channel key to receive messages.

As shown above, Redis keys enable extremely flexible data modeling. Next we‘ll explore how to retrieve and understand these keys.

Listing All Keys

A common task is retrieving all keys from a Redis database for debugging or visibility purposes.

The Redis KEYS command allows querying keys based on flexible glob-style patterns.

To list all keys in the current Redis database, we can use:

KEYS * 

The * pattern matches any key name, telling Redis to return all keys.

For example, on a database with caching and rate limiting keys:

127.0.0.1:6379> KEYS *
1) "product:abc123"  
2) "article:42"
3) "rate_limit:192.168.1.5"

As expected, this returns all key names regardless of the use case.

For production instances, KEYS * can be problematic as servers may have millions of keys, crashing redis-cli. Later we will explore more practical approaches.

Listing Key Groups by Pattern

A more precise alternative is leveraging key naming patterns to match related groups of keys.

Properly structured keys typically follow a convention like prefixing.

For example, a application might prefix all product cache keys:

product:a83kz
product:m23ij29
product:n999u432

We can query just the product keys using:

KEYS product:*

The * matches anything after product:. This returns only entries matching that prefix, great for understanding subsets of keys.

Other more advanced syntax includes globstars to match multiple directory levels:

KEYS product:electronics:laptops:**  

And bracketed ranges to match numeric ranges:

KEYS product:202[1-5]:stock  

So by carefully structuring keys, we can narrow down searches later to understand smaller groups of keys.

Retrieving Key Lists Externally

While the above KEYS approaches are perfect for ad hoc debugging directly within the redis-cli monitor, you often want to retrieve key listings to power other external scripts and tools.

For example, a Python script that analyzes cache hit rates by importing key names.

This external key retrieval can be achieved by piping Redis commands into redis-cli and capturing the output.

Here is an example command fetching production cache keys:

echo "KEYS product:*" | redis-cli > product_keys.txt

This builds the desired KEYS query, pipes it into redis-cli, stores the output in a local file called product_keys.txt.

That file contains the list of keys matching product:* for consumption by other applications.

The output can also be manipulated further using Linux tools like awk, sed, grep etc. This unlocks extremely powerful scripting workflows.

Redis Keys Under the Hood

From scripting to production troubleshooting, understanding and retrieving Redis keys is a critical skill. But how does the Redis KEYS command actually work under the hood?

When a clients sends KEYS to a Redis server, the server begins sequentially iterating through every key stored in the main dictionary. This dictionary stores key mappings directly in RAM for performance.

As Redis scans, it applies the specified glob-style pattern against each key. Any keys matching the provided pattern are added to an output array returned to the client once the scan completes.

So in essence, KEYS loops through the key dictionary linearly checking for matches. This allows very flexible listings but has scalability drawbacks we will explore shortly.

Internal Details

Understanding specifics of the the KEYS implementations helps set appropriate expectations:

• Blocking: KEYS scans block other database operations while running
• Single-threaded: Pattern matching runs in a single thread Currently in keydb-engine at GitHub
• No limits: KEYS has no built-in limits on output size or run time
• No indexes: No secondary indexes exist to aid pattern matching

So in terms of speed…

• KEYS with simple prefix-based patterns (e.g. product:*) scans quickly
• More complex regex patterns (e.g. prod-[AB123]*-[0-9]) must linearly scan all keys, much slower

With these insights, we better understand the intrinsic tradeoffs involved with using KEYS listings.

KEY Performance Considerations

In testing or small Redis instances, KEYS provides an easy way to flexibly list keys. But as mentioned earlier, KEYS comes with notable scalability pitfalls making it prohibitive for production scenarios.

Mainly that KEYS commands can lock up large Redis servers for seconds or minutes at a time querying millions of keys. Preventing other vital database operations like caching, messaging, etc. during this blocking period.

Salvatore Sanfilippo, creator of Redis, explicitly warns against using KEYS in production:

"Warning: consider KEYS as a command that should only be used in production environments with extreme care."

To illustrate the performance impact, let‘s explore some benchmarks…

KEYS Blocking Benchmark

Measuring a 100 MB Redis database with 1 million keys performs these blocking KEYS queries:

Keys Query	Duration
KEYS *	38 seconds
KEYS produc*	35 seconds
KEYS prod-[A-Z]*	38 seconds

As shown above, even simple prefix scans lock the database for 35-38 entire seconds in this simulated environment. No other read, writes, or other commands can execute during this blocking period.

Now imagine that impact amplified 10x or 100x on production systems with 10s of millions of keys!

So while flexible, KEYS comes at a significant cost at scale.

Impact on Memory Usage

Not only does KEYS block Redis servers, but it can drastically spike memory usage as well.

Based on benchmarks by Instaclustr of 100 million key Redis databases:

Keys Query	Additional Memory
KEYS *	2.5 GB
KEYS produc*	2 GB
KEYS prod-[A-Z]*	2.2 GB

As shown above, huge key listings can temporarily double memory consumption to store the output. Causing instability on memory constrained systems.

So in summary, KEYS provides extreme flexibility listing Redis keys but suffers from:

• Blocking that prevents other operations
• Memory spikes straining cached systems
• No built-in protections or limits

For these reasons, Redis experts strongly recommend against using KEYS in production environments. However, what are the alternatives?

Alternative Production Key Listing Approaches

Given the performance and scalability challenges of KEYS on large databases, Redis provides more production-friendly approaches to retrieving keys. Let‘s explore top alternatives:

SCAN

Instead of blocking the entire keyspace, Redis SCAN incrementally streams small batches of keys, similar to a cursor in a relational database.

SCAN 0 MATCH product:*  

This starts iterating product:* keys, with configurable batch size.

Pub/Sub Notifications

Clients can subscribe to Pub/Sub channels publishing events whenever keys change, avoiding polling via KEYS.

SUBSCRIBE keyspace:*   

Delivers key change events in real-time.

MONITOR

Outputs a continuous stream of all commands arriving at the Redis server, allowing external processes to extract keys.

MONITOR

Great for logs and auditing.

DEBUG DIGEST

Returns a sample of key names from the keyspace for analytics without performance impact.

DEBUG DIGEST

Provides key insights without full scans.

Additionally, client-side solutions like tagged metrics gathering key stats can help track keys.

The above patterns provide much more production-safe key listings compared to KEYS commands.

Best Practices for Redis Keys

We have covered many angles around listing, retrieving, and leveraging Redis keys. Here we will condense key takeaways into best practices:

• Prefix keys – group keys into namespaces using prefixes like cache:, rate_limit: etc.

• Use strict naming – follow conventions for key names even across teams

• Retrieve subsets – query key groups not all keys to inspect databases

• Avoid KEYS in prod – leverage SCAN, MONITOR, pub/sub instead of blocking KEYS

• Collect key metrics – track key performance with metrics solutions

Adopting these patterns will lead to more scalable, efficient, and usable Redis environments.

Conclusion

Redis keys unlock the power and flexibility of Redis for blazing-fast, in-memory data storage. In this extensive guide we covered:

• Essential Redis key best practices
• Leveraging KEYS listings for insights
• Internal implementation details of Redis key dictionaries
• Specific performance tradeoffs of keys listings
• Alternatives like SCAN and MONITOR for production use

I hope this deep dive empowers you to better leverage Redis keys in your cloud, IoT, and web applications. Proper keys structuring unlocks maintainable, efficient, and scalable Redis usage at any scale.