Availability Patterns

Availability is a measure of a system’s uptime and its ability to respond to requests. For a deeper dive into how availability is measured and calculated, see the fundamental concept of reliability-engineering.

In distributed systems, achieving high availability means designing a system that is resilient to failures, such as server crashes, network partitions, or software bugs.

The core strategy for high availability is redundancy—eliminating single points of failure by having more than one of everything. Availability patterns are practical techniques for implementing redundancy for both services and data. These patterns are the building blocks for creating AP (Available & Partition-Tolerant) systems, as described in the CAP Theorem.


Replication Patterns

Replication is the core strategy for data availability and a prerequisite for failover. It involves keeping identical copies of data on multiple nodes.

For a deep dive into replication topologies (Master-Slave, Master-Master) and synchronization methods (Synchronous vs. Asynchronous), see the dedicated page on

Master-Slave Replication

In this model, one node is designated as the Master (or primary), and it handles all write operations. The data is then replicated from the master to one or more Slave (or secondary/replica) nodes.

graph TD
    subgraph Writes
        W(Write Request) --> M(Master);
    end
    subgraph Reads
        R(Read Request) --> S1(Slave 1);
        R2(Read Request) --> S2(Slave 2);
    end

    M -- Replicates to --> S1;
    M -- Replicates to --> S2;

Description: All writes go to a single master, which then propagates the changes to read-only slaves.

Master-Master (Multi-Master) Replication

In this model, two or more nodes can accept both read and write operations. Each master node replicates its writes to the other master(s).

graph TD
    W1(Write Request) --> M1(Master 1);
    W2(Write Request) --> M2(Master 2);

    M1 <-. Replicates .-> M2;

    R1(Read Request) --> M1;
    R2(Read Request) --> M2;

Description: Both nodes can accept writes and reads, and they synchronize with each other.


Fail-Over Patterns

Fail-over is the mechanism of switching to a redundant standby system when the primary system fails. This process relies on having data synchronized between systems, which is achieved through the Replication Patterns discussed previously. A heartbeat mechanism is typically used to detect failures and trigger the fail-over.

Active-Passive Fail-Over

This pattern is the classic high-availability pairing. An active node handles traffic while a passive (standby) node is kept in sync but does not serve traffic. This synchronization is typically achieved using Master-Slave Replication, where the active node is the master and the passive node is the slave.

A monitoring component (like a load balancer or cluster manager) detects when the active node fails and automatically redirects traffic to the passive node, promoting it to become the new master.

sequenceDiagram
    participant User
    participant Router
    participant ActiveNode
    participant PassiveNode

    User->>Router: Request
    Router->>ActiveNode: Forward Request

    loop Heartbeat Check
        Router->>ActiveNode: Ping
        ActiveNode-->>Router: Pong
    end

    Note over Router, ActiveNode: Active Node Fails!

    Router->>ActiveNode: Ping (fails)
    Router->>ActiveNode: Ping (fails)

    Note over Router: Failure detected!
    Router->>PassiveNode: Promote to Active
    User->>Router: New Request
    Router->>PassiveNode: Forward Request

Description: When the Active node stops responding to heartbeats, the router promotes the Passive node and redirects traffic.

  • Pros: Relatively simple to implement and manage.
  • Cons: The passive node’s resources are idle most of the time, making it expensive (“Hot-Standby”) or leading to slower recovery (“Cold-Standby”).

Active-Active Fail-Over

In this configuration, all nodes are active and simultaneously serve traffic, typically behind a load balancer. If one node fails, the load balancer detects the failure (e.g., via health checks) and simply removes it from the pool, distributing its traffic among the remaining healthy nodes.

This is not so much a fail-over pattern as it is the natural state of a horizontally-scaled, load-balanced system.

graph TD
    U(User Requests) --> LB(Load Balancer);
    LB --> N1(Node 1);
    LB --> N2(Node 2);
    LB --> N3(Node 3);

    subgraph Failure Scenario
        N2 -- Fails! --> X(X);
        style X fill:#ffcccc,stroke:#333
        LB -. stops sending traffic .-> N2;
    end

Description: All nodes serve traffic. When a node fails, the load balancer simply stops routing requests to it.

  • Pros: Efficient use of resources, seamless failover with no downtime.
  • Cons: The system must be designed to be stateless or have its state distributed.

Load Management & Degradation Patterns

While the patterns above focus on handling node failures, another aspect of availability is managing extreme load. When a system is overwhelmed, it’s better for it to become slower or partially functional than to fail completely.

Graceful Degradation

Graceful degradation is a strategy where a system intentionally disables non-critical functionalities under high load to preserve resources for its essential, core functions. This ensures that the main purpose of the application remains available, albeit with a reduced feature set.

graph LR
    subgraph "High Load"
        direction LR
        UserRequest2(User Request) --> LoadShedder{Load Shedder}
        LoadShedder -- "High Load Detected" --> System2{System}
        System2 --> Core2(Core Functionality)
        System2 -.-> Disabled(X Non-Critical Feature);
        style Disabled fill:#ffcccc,stroke:#333
        Core2 --> Response2(Partial but Functional Response)
    end
    subgraph "Normal Load"
        direction LR
        UserRequest1(User Request) --> System1{System}
        System1 --> Core(Core Functionality)
        System1 --> NonCritical(Non-Critical Feature)
        Core --> Response1(Full Response)
        NonCritical --> Response1
    end

Description: Under high load, a load-shedding mechanism disables non-critical features to ensure core functionality remains available.

  • Example: During a flash sale, an e-commerce site might disable personalized recommendations (non-critical) to ensure that the search, add-to-cart, and checkout functions (critical) remain fast and available.
  • Implementation: This is often implemented using feature flags that can be toggled dynamically based on real-time system metrics like latency, error rates, or CPU saturation.

Loadshifting

Loadshifting is a pattern for improving system stability and user experience by deferring non-urgent, resource-intensive tasks to off-peak hours. Instead of processing a heavy workload immediately, the system queues the task and processes it later when there are more available resources.

This is a form of asynchronous processing and is often implemented using background-jobs and message queues.

graph LR
    subgraph "Peak Hours (Day)"
        User[User Request] --> API[API Server]
        API -- "Enqueue Job" --> Queue[(Message Queue)]
        API -- "202 Accepted" --> User
    end
    
    subgraph "Off-Peak Hours (Night)"
        Worker[Batch Worker] -- "Polls for Jobs" --> Queue
        Worker -- "Processes Job" --> DB[(Database / Storage)]
    end

Description: Instead of processing a heavy request immediately, the system places it in a queue. Batch workers then process the jobs from the queue during off-peak hours.

  • Example: A user requests to export a large annual report. Instead of making them wait and tying up server resources, the API accepts the request, returns an immediate “Accepted” response, and places a “generate report” job in a queue. A separate worker process runs at night to generate the reports and notifies the user via email when it’s ready.

Geographic Redundancy Patterns

While the patterns above can be deployed across multiple availability zones within a region, some systems require resilience against the failure of an entire geographic region.

  • Geode Pattern: This pattern takes the Deployment Stamp concept to a global scale. The application is deployed in multiple, active, and geographically distributed regions (geodes). A global load balancer directs traffic to the nearest healthy geode, providing extreme availability and low latency for a global user base.

Articles

  1. High Availability, Replication, and Failover Explained with Stuffed Animals This article provides a wonderfully simple and effective analogy using a child’s favorite stuffed animal to explain complex topics like replication, failover, consensus, and the split-brain problem. It’s a great starting point for building an intuitive understanding.

Videos

  1. Fail-over and High-Availability (Explained by Example) This video clearly explains the mechanics of failover, including concepts like Virtual IPs (VIP) and VRRP, using practical examples. It’s useful for understanding how traffic is actually redirected in a high-availability setup.

  2. Database Replication Explained | System Design Interview Basics Focusing on the data aspect of availability, this video breaks down database replication patterns. It covers the main-replica (master-slave) model and its importance for scaling reads and ensuring data redundancy.