Flow Control (Sliding Window)

[!NOTE] This module explores the core principles of Flow Control (Sliding Window), deriving solutions from first principles and hardware constraints to build world-class, production-ready expertise.

1. What is Flow Control?

The Analogy: Imagine you are filling a bucket with a high-pressure water hose. You (the Sender) have a massive water supply. The bucket (the Receiver) can only hold 5 gallons, and it has a small drain at the bottom (the application reading data). If you blast the hose at full power, the bucket overflows, and water (data) spills onto the ground (dropped packets). Flow Control is you watching the bucket and turning the hose on and off to match the drain rate.

In networking, flow control is a technique to ensure that a fast sender does not overwhelm a slow receiver. If the sender sends data faster than the receiver can process it, the receiver’s physical memory buffer (typically allocated in RAM by the OS kernel for the specific socket) will overflow and packets will be dropped by the Network Interface Card (NIC) or the kernel itself.

2. Stop-and-Wait (The Slow Way)

The sender sends one packet and waits for an ACK before sending the next.

  • Pros: Very simple.
  • Cons: Extremely inefficient. You spend most of your time waiting for ACKs rather than sending data (The “Pipe” is empty). This ignores the bandwidth-delay product of modern hardware.

3. Sliding Window (The Fast Way)

The sender is allowed to have multiple packets “in flight” (sent but not yet acknowledged). This is physically implemented using a circular buffer in the kernel’s memory space.

  • Window Size: The number of bytes the sender can transmit without receiving an ACK. This is bound by the receiver’s advertised buffer space.
  • Dynamic: The receiver tells the sender its available buffer space in every ACK packet, allowing the sender to adjust its transmission rate to match the receiver’s memory capacity.

4. Interactive: Sliding Window demo

Watch the window move as data is acknowledged.

1
2
3
4
5
6
7
8
Window Size: 3 Packets. Current Window: [3, 4, 5]
Waiting for ACK...

5. Zero Window (The Emergency Brake)

If the receiver’s physical buffer is completely full, it sends a Window Size = 0 flag.

  • The sender must stop transmitting immediately, preventing further packet drops at the receiver.
  • The sender then sends periodic “Window Probes” to see if any buffer space has cleared up in the receiver’s kernel.

6. Silly Window Syndrome (War Story)

A classic edge case in flow control is the Silly Window Syndrome. Imagine the receiver’s buffer is full (Zero Window). The application reads exactly 1 byte from the buffer. The receiver immediately sends an ACK advertising a Window Size of 1 byte.

The sender, eager to transmit, sends exactly 1 byte of payload. However, a TCP/IP packet has 40 bytes of header overhead (20 bytes IP + 20 bytes TCP). Sending 41 bytes across the network to transmit 1 byte of actual data is tremendously wasteful. The window remains ‘silly’ (tiny) because the sender immediately fills the 1 byte of freed space.

The Solution:

  1. Receiver-side (Clark’s Solution): The receiver artificially advertises a Zero Window until it can free up a significant amount of space (e.g., Maximum Segment Size, or half the buffer).
  2. Sender-side (Nagle’s Algorithm): The sender buffers small outgoing data and waits until it has a full packet’s worth (MSS) to send, unless it receives an ACK for previously sent data.

7. Case Study: High-Throughput File Transfer

Let’s design a specialized protocol over UDP for transferring massive video files (100GB+) across high-latency intercontinental links, bypassing TCP’s standard congestion control but implementing our own flow control. We will use the PEDALS framework.

P - Process Requirements

  • Core Task: Transfer a 100GB file from Tokyo to New York.
  • Constraints: Maximize link utilization on a 10Gbps dedicated line with 150ms RTT (Round Trip Time).

E - Estimate

  • Bandwidth-Delay Product (BDP): BDP = Bandwidth × RTT.
  • 10 Gbps = 1.25 GB/s.
  • 1.25 GB/s × 0.150s = 187.5 MB.
  • Conclusion: To keep the “pipe” full, we must have at least 187.5 MB of unacknowledged data in flight at any given moment. A standard TCP window maxes out much lower without scaling.

D - Data Model

  • Packet Structure:
    • Sequence Number: 64-bit (to handle >4GB files without wrapping).
    • Payload: 8960 bytes (Jumbo frames on private links to reduce header overhead).
    • Checksum: 32-bit CRC.

A - Architecture

  • Sender: Reads from NVMe storage via sendfile() (zero-copy) into a circular buffer sized at 400MB (double the BDP to be safe).
  • Receiver: Writes to NVMe storage. Maintains a 400MB receive buffer.
  • Feedback Channel: Receiver sends periodic ACKs every 10ms, containing the highest contiguous sequence number received, and a bitmap of selective acknowledgments (SACK) for missing packets.

L - Localized Details

  • The Flow Control Implementation: We implement a Sliding Window of size 200MB. If the receiver’s disk write speed slows down, its 400MB buffer fills. In the 10ms ACK, it advertises the remaining space (e.g., 50MB). The sender immediately restricts its inflight bytes to 50MB, preventing packet drops.

S - Scale

  • Scaling horizontally: If transferring to multiple clients, the sender must maintain isolated memory buffers and sliding window states per connection. Memory exhaustion at the sender becomes the bottleneck, requiring tuning net.core.wmem_max in the Linux kernel to allocate enough RAM per socket.