Redis 8.8: New array data structure & open source features

Redis 8.8 in Redis Open Source is now available, bringing performance improvements alongside a set of powerful new features. Highlights include array - a new general-purpose data structure, a window counter rate limiter, streams message NACKing, subkey notifications for hash fields, explicit control over JSON numeric array storage, multiple aggregators in a single time series query, and a new COUNT aggregator for sorted sets union and intersection.Summary of performance improvements in 8.8Redis 8.8 introduces significant end-to-end throughput improvements:Data typeOperationsEnd-to-end throughput improvementsStringsMGET (pipelined, with I/O-threads)Up to 68%MGET (pipelined, single thread)Up to 50%MSETUp to 8%HashHGETALLUp to 25% (1K+ fields)StreamsXREADGROUPUp to 83% (COUNT 100)Sorted setZADD, ZINCRBY, ZRANGEBYSCOREUp to 74%BitmapBitmap operationsUp to 28% (x86)HyperLogLogPFCOUNTUp to 18% (x86)(several)SCAN, HSCAN, SSCAN, ZSCANUp to 40%In addition, persistence and replication (full synchronization) is now up to 60% faster.Summary of new features in 8.8Redis has always been about choosing the right data structure for the job. In Redis 8.8, we introduce a new general-purpose data structure: array. An array is an index-addressable collection of string values. Each array element is stored at a numeric index, and can be accessed extremely fast. Arrays are dynamic, sparse-friendly, and compute-aware containers, enabling new use cases and better flexibility and efficiency for existing use cases (by @antirez).Rate limiting is one of the most common Redis use cases. Traditionally, users implemented rate limiters using server-side Lua scripts combined with client logic. In Redis 8.8, we introduce a window counter rate limiter (by @raffertyyu, together with the Redis team).Our investment in improving Redis Streams continues.

Redis 8.2 simplified message acknowledgment and deletion across multiple consumer groupsRedis 8.4 made it easier for consumers to read both new and idle pending messagesRedis 8.6 introduced idempotent productionBuilding on this momentum, Redis 8.8 adds support for message NACKing, allowing consumers to explicitly release pending messages so they become immediately available and prioritized for consumption by other consumers.In Redis 7.4 we introduced hash field expiration – a capability that saw strong adoption. A frequent follow-up request was for field-level notifications, similar to existing key-level notifications. Redis 8.8 delivers this with subkey notifications for hash fields, allowing clients to subscribe to events such as field expiration and deletion. These notifications include the key, the subkey (field name), and the event type.Retrieving multiple time series aggregators is a common operation. For example, candlestick charts rely on MIN, MAX, FIRST, and LAST aggregations. Prior to Redis 8.8, this required multiple commands. Redis 8.8 now supports multiple aggregators in a single time series command, reducing round trips and simplifying client logic.Redis 8.4 introduced support for homogeneous numeric arrays in JSON, delivering up to 92% memory reduction – especially valuable for AI workloads. In Redis 8.8, users can now explicitly control how numeric arrays are stored (BF16, FP16, FP32, or FP64), enabling better alignment with source data, vector indexing needs, and memory/precision tradeoffs.Finally, Redis 8 extends sorted set union and intersection operations with a new COUNT aggregator. This allows the score of each element to reflect either the number of input sets it appears in or the weighted sum across those sets, unlocking new use cases in ranking, scoring, and analytics.

The new features explainedArray: A new general-purpose data structureRedis has always been about choosing the right data structure for the job. Redis traditionally provides several core data structures, including lists, hashes, sets, and sorted sets. In Redis 8.8, we introduce a new general-purpose data structure: array.What is an array?An array is an index-addressable collection of string values. Each element is stored at a numeric index, and can be accessed extremely fast.Arrays go far beyond basic indexed storage. They are flexible, memory-efficient, and compute-aware. Arrays have some capabilities that enable new use cases and better flexibility and efficiency for many existing use cases:An array doesn’t need to have a fixed sizeArrays grow and shrink dynamically. Elements can be set at any index (0 to 2⁶⁴−1), and the array grows efficiently as needed. Elements can be deleted and the array shrinks accordingly.An array can be dense or sparseThe used indices don’t have to be consecutive, and yet the memory footprint is proportional to the number of elements, and access by index remains extremely fast.For example: an array index can represent a product ID and the values hold product names or details. Similarly, the index can represent timestamps and the values hold log events.An array can be used as a ring buffer (Sliding Window)Arrays can act as a bounded rolling buffer: retain the last n elements, maintain insertion order, and automatically overwrite older entries.Think of a log file or a stream of events, where you want to efficiently keep the last n log entries, events, or measurements, and frequently fetch the last [up to] n values. This is especially useful if you need to feed a rule engine, process a fraud detection window, continuously update a chart, or execute security validations.Arrays can aggregate dataWhen the values are numeric (for example, real-time sensor reports or stock quotes), arrays support server-side computation over index ranges, including SUM, MIN, and MAX. When the values are binary flags, Boolean aggregators (AND, OR, XOR) are supported as well.Server-side aggregators are Ideal for sensor data, financial ticks, and real-time metrics. When combined with ring buffer semantics, Arrays enable sliding window analytics such as real-time anomaly detection.

An array can be searchedAn array can represent a textual file (e.g., .txt, .csv, .log), where each element is indexed by the line-number and holds a single text line. Users can iterate over these lines for analysis, and can search for specific lines using an exact or partial string, a glob-style pattern, or a regular expression.With ring buffer semantics, arrays can constantly hold the last n log-lines allowing users and agents to contextualize or enrich incoming events based on recent ones.In summary, an array is a dynamic, flexible, high-performance, index-addressable, compute-aware container that combines aspects of:List (ordered data)Time-series (sliding windows)Sparse map (non-contiguous indices)Analytical engine (aggregation and search)Random element access: Array vs list vs hashBenchmarking arrays against the closest list and hash equivalents under random access at large element counts, the advantages of array show up clearly:Operation (100K elements; 1 KB values)ArrayListHashRead random element675K ops/sec133K ops/sec626K ops/secWrite random element757K ops/sec137K ops/sec689K ops/secDelete random element841K ops/sec—730K ops/sec* Redis 8.8, single instance on an Intel Sapphire Rapids m7i.metal-24xl machineFor random-element operations, array provides 8-15% better throughput than Hashes and are at least 5 times faster than Lists.Memory wise, lists are the most compact. Arrays require ~18% more memory per element, while hashes require 30-46% more memory than lists, depending on the size of the elements:Element size (100K elements)ArrayListHash100 bytes122 bytes/element104 bytes/element151 bytes/element1 Kbyte1290 bytes/element1035 bytes/element1337 bytes/elementRing buffer: Array vs listA common pattern in Redis is using a list as a bounded ring buffer: clients push new entries with RPUSH and trim list back with LTRIM to keep a constant number of elements. Arrays expose ARRING, which performs the same operation in a single atomic command.

Ring size; element sizeArray (ARRING)List (RPUSH+LTRIM)Array’s advantage 1K elements; 100 bytes1.11M inserts/sec512K inserts/sec × 2.2100K elements; 100 bytes1.12M inserts/sec528K inserts/sec× 2.11K elements; 1 Kbyte840K inserts/sec424K inserts/sec× 2.0100K elements; 1 Kbyte837K inserts/sec413K inserts/sec× 2.0* Redis 8.8, single instance on an Intel Sapphire Rapids m7i.metal-24xl machineARRING delivers twice the throughput (inserts/sec) compared to the equivalent RPUSH+LTRIM idiom, independent of ring size. Memory footprint is the same as above: Arrays require ~18% more memory than lists.When should arrays be used?Arrays are extremely useful when:You need extremely fast access by index or by index-rangeYou need a sliding window over recent dataYou need server-side aggregationYou want to search for matching elementsWhat arrays are not suitable for?Arrays are not a replacement for other data structures.Use lists if you need push/pop operations, or inserting elements between others.Use hashes if you need field name-based access instead of numeric indices.Where can I learn more?Array documentation: https://redis.io/docs/staging/DOC-6334/develop/data-types/arrays/Array commands: https://redis.io/docs/latest/commands/?group=arrayDiving deep into Redis’s new array data type: https://redis.io/blog/diving-deep-into-rediss-new-array-data-type/Window counter rate limiterWindow counter rate limiters, including fixed window, fixed window with lazy reset, and sliding window counter variants, use one or more fixed-duration time windows. Each window maintains a counter initialized to 0 when the window is created, along with a maximum capacity representing the number of tokens allowed during that window’s lifetime.Before Redis 8.8, implementing a Window counter rate limiter required Lua scripting. In 8.8, we introduce a new command for working with window counters:The idea is simple: each window has a duration (specified via EX or PX) and a token capacity (specified with UBOUND).

The number of tokens requested can be specified with BYINT increment (default is 1). INCREX attempts to increment the counter by the requested number of tokens. The key is created if it does not already exist.To make this command suitable for rate limiter use cases, beyond basic increment semantics, INCREX introduces three new capabilities compared to the existing INCR family of commands:INCREX returns both the new counter value and the actual increment applied, allowing the caller to immediately determine whether the request should be allowed or rejected.When ENX is specified, expiration is set only if the key does not already have one. This ensures that the window’s TTL is set only when a window is created and not modified on subsequent requests during its lifetime.Boundary enforcement: the request is rejected if it would exceed the defined bounds. With SATURATE, the request may be “partially accepted” with the counter clamped to the specified bounds (“saturated”) .Beyond rate limiting, INCREX can be seen as a generalized form of INCR, INCRBY, INCRBYFLOAT, as well as DECR and DECRBY (via negative increments), with added support for bounds and expiration control.Streams: NACKing messagesIn real-world applications, stream consumers don’t always successfully process the messages they consume. Failures can happen for many reasons:A consumer may encounter internal issues unrelated to the message itself. For example, failing to reach an external service it needs for processing the message.A consumer may need to shut down and release unprocessed messages.A resource-constrained consumer (CPU, memory) may be unable to process certain messages (at least, in a timely manner).A message may be malformed, poisoned, or even malicious.Before Redis 8.8, consumers had no way to explicitly reject (NACK) a message. They could either acknowledge it or leave it pending. In practice, this meant other consumers in the consumer group had to recover these messages using XREADGROUP … CLAIM, XPENDING+XCLAIM or XAUTOCLAIM.This approach introduces delays, since messages remain idle in the Pending Entries List (PEL) until another consumer claims them – an issue for time-sensitive systems.