Multitenant data management with TiDB

In a previous article, we explored various approaches to managing data for multitenant applications. We examined the benefits, compromises and limitations of different deployment models, including share-nothing, share-everything, and hybrid. We discussed the needs and challenges of each approach, from ensuring data isolation and scalability to managing customized tenant requirements.

One key takeaway from this discussion is that building multitenant applications is easier than building multitenant storage. In most cases, the application is stateless, and can adopt growth-friendly architectural patterns like microservices and elastic scaling. Storage is a harder problem. Traditional databases and data management systems struggle to maintain performance while ensuring strict data isolation and security across tenants. This challenge stems from the inherent conflict between shared resources and isolation requirements. As more tenants share the same infrastructure, the overhead of maintaining strict boundaries and preventing resource contention grows exponentially, leading to degraded performance and increased complexity in access control mechanisms.

Distributed SQL databases emerged, in part, as a response to these challenges. Based on the architecture pioneered by Google Spanner, distributed SQL aims to combine the simplicity and reliability of traditional SQL databases with the scalability and performance typically associated with NoSQL systems. To this end, distributed SQL tends to provide more robust support for multitenancy than legacy data management systems.

In this article, we’ll explore how one distributed SQL database, TiDB, provides scalability, flexibility, and isolation for multitenant applications.

Introduction to TiDB

TiDB is an open-source, distributed SQL database designed for applications that require high availability and scalability. It can elastically scale out to hundreds of nodes and manage petabytes of data while supporting millions of queries per second (QPS), The architecture that makes this possible also provides robust support for multitenant applications.

Architecture

PingCAP

As shown in the diagram, TiDB’s architecture has three main components.

Distributed storage layer: TiKV

The foundation of TiDB’s architecture is its distributed storage layer, TiKV. TiKV is a transactional key-value storage engine that shards data into small chunks, each represented as a split. Each split is replicated across multiple nodes in the cluster using the Raft consensus algorithm to ensure data redundancy and fault tolerance. The sharding and resharding processes are handled automatically by TiKV, operating independently from the application layer. This automation eliminates the operational complexity of manual sharding—a critical advantage especially in complex, multitenant environments where manual data rebalancing would be cumbersome and error-prone.

SQL layer: TiDB Server

Above the storage layer is the distributed SQL layer, TiDB Server. This communicates with applications, processes all SQL commands and queries, and works with the storage layer to process data. TiDB Server nodes are stateless, so they can scale horizontally with ease. This enables the database to easily expand capacity and throughput as demand increases.

The SQL layer implements the MySQL protocol, providing users a familiar SQL interface while extending additional capabilities for handling distributed transactions. From the application’s perspective, TiDB is simply another MySQL database that works seamlessly with existing MySQL tools and applications.

Scheduling layer: Placement Driver (PD)

PD (Placement Driver) plays a critical role in maintaining the health of the database system by managing the metadata of the storage engine. It acts like a devops staffer who monitors the TiDB cluster around the clock, load balancing, placing data, facilitating operations, and ensuring data safety.

PD collects information on each TiKV node to build a global view of the whole cluster. With that information, PD tries to identify:

Any violations of admin policies. For instance, if the replication factor is three but only two copies are found (probably due to network issue or node failure).

If any node has a higher or lower load then others. For instance, if a newly added node has no data and workload.

If any operation from admin needs to perform proactive actions. For instance, during a rolling upgrade, if PD needs to proactively direct traffic to other nodes before a node is restarted.

Features for multitenant data management

TiDB’s architecture provides the basis for a range of features critically important to multitenant applications.

Horizontal scalability

Workload demands can vary significantly between tenants in multitenant architectures. Data volume and workloads can also increase unpredictably as new tenants are onboarded or existing tenants expand. TiDB’s distributed architecture separates storage and compute, allowing each to scale independently without downtime or performance degradation. This enables TiDB to maintain performance and availability in a multitenant context without the high costs and limitations associated with vertical scaling.

High availability

In a multitenant environment, where a single component failure could affect numerous tenants simultaneously, high availability is critical. TiDB’s distributed architecture directly addresses this challenge by minimizing the blast radius of potential failures. If one node fails, others take over, maintaining continuous service across all tenant workloads. This is especially important for business-critical applications where uptime is non-negotiable. TiDB’s distributed storage layer ensures data redundancy and fault tolerance by automatically replicating data across multiple nodes. In the event of a node failure, TiDB quickly recovers by redistributing the workload to other active nodes, preventing cascading failures that could affect multiple tenants and ensuring consistent application performance across the entire system.

Flexible deployment

Multitenancy often requires deployment across different environments, including multiple clouds or hybrid scenarios. TiDB’s architecture is hardware-agnostic, allowing deployment on general-purpose machines whether on-premises or in the cloud. Its containerized components, declarative APIs, and stateful orchestration integrate with Kubernetes, enabling automated deployment, scaling, and life cycle management across any standard infrastructure. This hardware-independent approach, combined with cloud-native design, gives organizations the flexibility to deploy TiDB on  anything from commodity servers in private data centers to various instance types across public clouds. Such versatility allows organizations to adapt to different tenant needs and infrastructure requirements while avoiding vendor lock-in and specialized hardware costs.

Online schema migration

With multiple tenants sharing the same infrastructure, schema changes become significantly more challenging than in single-tenant architectures. A schema update that requires downtime would affect not just one application, but potentially hundreds or thousands of tenant workloads simultaneously. TiDB addresses this challenge through its support for online schema changes, allowing developers to roll out new features—which often require database schema updates—without service interruption. This capability is particularly vital in multitenant environments where maintenance windows are virtually impossible to coordinate across all tenants. Teams can continuously evolve their applications through schema updates, while leaving other tenants unaffected, enabling continuous integration and delivery practices in complex multitenant deployments.

Resource control groups

Resource contention is a major cause of performance issues in multitenant setups. TiDB addresses this problem with resource control groups. This feature enables a TiDB cluster to be divided into multiple logical units, each with its own resource quota isolated from the others. Administrators can monitor and manage the resource consumption of each tenant. This helps prevent any one from consuming disproportionate resources that could affect the overall system performance, and ensures that critical applications always have the resources they need. When the system is not under heavy load, applications can be granted permission to use excess resources for optimal hardware utilization.

PingCAP

Distributed SQL databases like TiDB represent a major step forward for multitenant applications. While implementations differ from solution to solution, distributed SQL databases tend to offer more robust support for multitenant data management than legacy SQL and NoSQL systems. With their roots in hyperscale web applications, they are better positioned to address the key needs of multitenant architectures—scalability, consistency, availability, and flexibility—while also providing advanced features for resource management and data placement. Their ability to efficiently handle HTAP (hybrid transactional/analytical processing) workloads make them an opex-friendly choice for businesses looking to leverage multitenancy for efficiency and growth.

Li Shen is senior vice president at PingCAP, the company behind TiDB.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.