User vs Kernel Mode

[!NOTE] This module explores the core principles of User vs Kernel Mode, deriving solutions from first principles and hardware constraints to build world-class, production-ready expertise.

1. The Trust Issue

The OS cannot trust the user. If a user program could access any memory address, it could:

  1. Overwrite the Kernel’s code.
  2. Read passwords from another process’s memory.
  3. Turn off the cooling fan and melt the CPU.

Analogy: The Restaurant Kitchen Think of a computer system as a high-end restaurant:

  • User Mode (The Dining Room): This is where you (the user application) sit. You have a menu and you can eat, but you cannot walk into the kitchen to cook your own food.
  • Kernel Mode (The Kitchen): This is the restricted area where the hardware operates. Only authorized staff (the OS kernel) can touch the hot stoves, sharp knives, and the ingredients (hardware resources, memory, disk).
  • System Calls (The Waiter): If you want a steak (read a file from the disk), you can’t just barge into the kitchen. You must place an order with the waiter (trap into the Kernel via a System Call). The OS validates your request, performs the privileged action safely, and brings back the result.

To prevent unauthorized access, the CPU hardware enforces Protection Rings.

2. x86 Protection Rings

Intel/AMD CPUs support 4 privilege levels (Rings 0-3).

  • Ring 0 (Kernel Mode): God mode. Can execute any instruction (HLT, LGDT, IN/OUT) and access any memory address.
  • Ring 1 & 2: Designed for drivers, but rarely used by modern OSs (Linux/Windows use only 0 and 3).
  • Ring 3 (User Mode): Peasant mode. Restricted. Cannot touch hardware directly.

Diagram: Protection Rings

Ring 3 (User) Ring 0 (Kernel) Hardware Enforced Boundary

3. How the CPU Enforces It

The CPU has a register that stores the CPL (Current Privilege Level).

  • When running User Code: CPL = 3.
  • When running Kernel Code: CPL = 0.

Every time the CPU fetches a “Privileged Instruction” (like CLI - Disable Interrupts), it checks:

Is CPL == 0?

  • Yes: Execute it.
  • No: Trigger a General Protection Fault (GPF).

The OS catches this Fault and kills the process (SIGSEGV/SIGKILL).

Context Switching Overhead

Transitioning from User Mode to Kernel Mode is not free. It involves a Mode Switch:

  1. Saving State: The CPU must save the user program’s registers and execution context.
  2. Switching Stacks: The CPU switches from the user stack to a secure kernel stack.
  3. Validation: The OS must rigorously check the arguments passed by the user to ensure they are safe.

Because this overhead is expensive, modern x86-64 CPUs introduced fast instructions like syscall and sysret to bypass the older, slower interrupt-based mechanisms (like INT 0x80), vastly speeding up the transition.

4. Traps vs Interrupts

How do we switch from Ring 3 to Ring 0?

Type Initiator Example
Trap (Syscall) Software (Intentional) Application calls read().
Exception (Fault) Software (Error) Application divides by zero or touches bad memory.
Interrupt Hardware (External) Network card says “Packet Arrived”. Keyboard says “Key Pressed”.

5. Interactive: The Gatekeeper

Try to execute privileged instructions from User Mode. Goal: Don’t get killed by the OS.

3
CPL Ring
Waiting for instructions...

6. Summary

The “User Mode” vs “Kernel Mode” distinction is not just a software convention. It is hard-wired into the silicon of the CPU. This hardware enforcement is the foundation of system stability and security.