The exciting new world of Redis

If you’re a developer who wants the most feature-rich, high-performance version of Redis, your choice is clear: Redis and not a fork. If you have the time and inclination to dabble in ideological debates about open source licensing, well, you might make another choice. But if you’re just trying to get your job done and want a great database that historically was primarily a cache but today offers much more, you’re going to opt for Redis and not its fork, Valkey.

So argues Redis CEO Rowan Trollope in an interview. “It is unquestionable that Redis, since we launched Redis 8.0 with all the capabilities from Redis Stack, is just a far more capable platform,” he says. He substantiates the claim by cataloging “a whole bunch of things” that Valkey doesn’t offer, at least not at parity: vector search, a real-time indexing and query engine, probabilistic data types, JSON support, etc. (Note that some vendors, like Google Cloud, have started to fill in some of these blanks, at least in pre-GA releases, like Google’s Memorystore.)

That’s all CEO-speak, right? What would a serious technologist say about Redis? It might be difficult to find a more credible Redis expert than Redis founder Salvatore Sanfilippo who recently returned to the Redis community (and company) he left in 2020. Why return? Among other reasons, Sanfilippo wants to help shape Redis for a world awash with generative AI. In his words, “Recently I started to think that sorted sets can inspire a new data type, where the score is actually a vector.” Trollope says, “Redis has a real opportunity to emerge as a core part of the genAI infrastructure stack.” Discussions about licensing, Trollope notes, might be fun “popcorn fodder” that fixates on the past, but the real focus should be on Redis’ future as an integral part of the AI stack.

Forking for the wrong reasons

We live in a weird time when a few trillion-dollar companies get away with pleading poverty, arguing that they should be gifted a wealth of open source software without helping sustain its success. “Somehow Amazon and Google positioned themselves as the open source–friendly companies, which seems to be the opposite of reality,” Trollope tells me. In response, companies like Redis have gone through all sorts of licensing and packaging gymnastics (e.g., Redis Stack) to try to remain open to everyone except the clouds that threatened their ability to continue building and releasing open source software.

Did Redis manage this process perfectly? No. As Trollope tells me, “We didn’t effectively communicate with the community about our motivations and what we were doing and why we were doing it.” He continues, “I’m sure we can do things differently and better, and that’s our goal going forward.” For his part, Sanfilippo writes, “I don’t believe that openness and licensing are only what the OSI [Open Source Initiative] tells us they are,” but rather “a spectrum of things you can and can’t do.” It’s a nuanced view on a nuanced topic that too often gets painted in black and white.

Though portrayed as motivated by ideological differences over open source versus closed source software, ultimately AWS and Google forked Redis to further their business needs. Named Valkey, this Redis fork isn’t Redis, however much Valkey’s commercial backers cling to the “Redis compatible” tagline, and it will become more distinct over time. That’s a good thing. After all, Valkey has developers like Madelyn Olsen who can turn it into something amazing. Recently, for example, AWS made significant improvements to Valkey’s scalability and memory efficiency. This will play out like AWS’s fork of Elasticsearch, OpenSearch, which is becoming a compelling product in its own right and not just a pale shade of Elasticsearch. It’s a great thing when the clouds create, rather than copy, open source.

Ultimately, very few developers care about open source ideological debates. As Trollope suggests, “If you’re the average developer, what you really care about is capability: Does this thing offer something unique and differentiated … that I need in my application?” On that note, he continues, “Pound for pound, we’ve been able to drive innovation dramatically more quickly than Valkey has.” Even if you disagree with that assessment, it’s hard to overlook just how differently Redis and the Valkey community see their futures, which brings us back to Sanfilippo and AI.

A different future for Redis

The Valkey development community isn’t focused on building and positioning it for generative AI workloads. Redis, by contrast, definitely hopes to ensure Redis’s place in the modern AI-centric stack. In such a world, debates about Redis licensing are “interesting but not that helpful,” Trollope argues. What really matters is whether developers can easily use Redis to build something awesome.

Sanfilippo, for his part, isn’t new to AI. “I wrote my first [neural network] library in 2003 and was totally shocked by how powerful and cool the whole concept was,” he writes. But “now, at the end of 2024, I’m finally seeing incredible results in the field. Things that looked like sci-fi a few years ago are now possible.” Although impressed by large language models such as Claude, Sanfilippo is actively working on novel approaches to Redis, like vector sets, which is “exactly the idea of sorted sets, but with multi-dimensional scores (embeddings!) and K-NN [k-nearest neighbor] matches.”

It’s a cool way to bring a very Redis flavor to generative AI. But among Sanfilippo’s interest in exploring “new ideas that can be exciting,” it’s hard to think of anything more exciting than Redis’s founder coming back to help shepherd its future. Imagine you’re an outsider without any knowledge of the past few years of licensing debates and you’re placing bets on the Redis-esque product most likely to win over developers. Against that backdrop, if “you take a guy like Salvatore now being a key contributor and leader of Redis, look, I’d make that bet every day of the week,” argues Trollope. Many developers likely will too.