1 Operating System Foundations

The operating system’s role

An operating system (OS) is the software layer between programs and computer hardware. It coordinates access to processors, memory, storage, and input/output (I/O) devices while providing convenient abstractions such as processes, files, and network connections.

These responsibilities make the OS both a resource manager and an abstraction layer. Applications can use consistent OS services without directly controlling each hardware device.

Privilege levels and protection

Modern processors provide hardware-enforced privilege levels. Applications normally run in user mode, where they cannot execute certain privileged instructions or freely access protected memory and devices. The OS kernel runs in kernel mode and has the authority needed to manage these resources.

This separation helps prevent a faulty or malicious program from directly disrupting the whole system. The kernel handles essential work such as managing execution, memory, and device access; other components, including command-line tools, graphical interfaces, and many system services, may run as ordinary user-space programs.

System calls and library calls

A system call is a controlled entry point for a program to request a kernel service. Programs use system calls for actions they are not permitted to carry out directly, such as creating a process, reading a file, or communicating through a network socket.

A typical request proceeds as follows:

1. A program calls a library function, such as read().

2. A library wrapper arranges the system-call number and arguments in the form required by the processor and operating system.

3. A special processor instruction transfers execution to a designated kernel entry point and switches to kernel privilege.

4. The kernel checks the request, performs the operation or rejects it, and prepares a result.

5. The processor returns execution to the program in user mode.

For a file read, the program asks the kernel to read from an already-open file descriptor into a buffer. The kernel checks whether the request is valid and permitted, obtains data through the file-system and device layers as needed, and returns the number of bytes read or an error.

A library call and a system call are not identical. A library call may finish entirely in user space; a library wrapper may instead invoke one or more system calls to ask the kernel to act. The library function can make a transition to the kernel look like an ordinary function call, even though the system call crosses a protected boundary.

Kernel design approaches

Operating systems divide kernel and service code in different ways. These broad design labels describe choices about where services run, but real systems may combine features rather than fit perfectly into one category.

• Monolithic kernel: Many core services, such as file systems and device drivers, run within the privileged kernel. Communication between services can be direct, but a fault in privileged code can have broad consequences.

• Microkernel: The kernel keeps a smaller set of essential mechanisms in privileged mode, while more services run as separate user-space processes. This can improve isolation, though communication between components may require additional message passing.

• Modular or hybrid designs: These combine approaches. For example, a kernel may support loadable components while retaining many services in kernel space.

Hardware support and device control

Safe hardware use depends on processor support, memory-management hardware, device controllers, and device-specific software.

• Processor: Executes instructions, enforces privilege and memory-protection rules, and provides mechanisms for controlled transitions into the kernel.

• Memory-management hardware: Translates program addresses and enforces access permissions, supporting the separation of processes and kernel memory.

• Device controllers: Operate hardware such as disks, keyboards, and network interfaces. A device driver knows how to communicate with a particular device or device class and coordinates with its controller through approved interfaces.

• Interrupts: Allow hardware to signal that it needs attention or has completed work. The processor pauses its current activity and transfers control to an operating-system handler.

An interrupt differs from a system call in how it begins: software initiates a system call, while an event from a device or timer initiates a hardware interrupt.

Coordinating an operation with a device

When an application requests disk data, the kernel can direct the storage driver to start an operation. The processor may run other work while the device operates. When the device signals completion, the kernel handles the interrupt and makes the result available to the requesting program.

Hardware details vary, but drivers and kernel interfaces let applications use consistent operating-system services without directly controlling each device.