6 Computer Memory

Memory’s role and trade-offs

Computer memory stores the instructions and data a computer needs. Memory systems balance speed, capacity, and cost: very fast storage is generally small and expensive, while larger storage is usually slower.

Locations, addresses, and words

Memory can be understood as a sequence of locations. An Address identifies where data can be read or written. In a byte-addressable system, each address refers to one byte. If an address contains $n$ bits, it can identify up to $2^n$ locations. For instance, an $8$-bit address can identify $2^8 = 256$ bytes, numbered from $0$ through $255$.

A processor requests data or instructions using addresses, but it commonly handles data in groups called a Word. A word is a group of bits treated as a unit and may span several byte addresses. Addresses identify locations; words describe the units of data the processor handles.

RAM, ROM, and persistence

Random-access memory (RAM) lets the processor read or write locations directly; access does not require passing through earlier locations. In common computer usage, main RAM is volatile, meaning it loses its contents when power is removed. DRAM is widely used for main memory because it provides high capacity at relatively low cost. SRAM is faster and is commonly used to build caches.

Read-only memory (ROM) is non-volatile, so it retains its contents without power. It is used for firmware and other data that must be available when a system starts. Some ROM technologies can be erased and rewritten, so “read-only” describes a traditional use rather than every possible implementation.

The key distinction is persistence: typical main RAM supports active, writable data but is volatile, while ROM retains data without power.

Caches and the memory hierarchy

A Cache is a small, fast memory that keeps copies of data from a larger, slower level. Caches benefit from two common patterns in programs: Temporal locality means recently accessed data is likely to be used again, while Spatial locality means data near a recently accessed location is likely to be used soon.

If requested data is present, the access is a Cache hit. If it is absent, a Cache miss causes the system to fetch a block containing that data from the next level, often placing the block in the cache. For example, after a program accesses one array element, nearby elements may already be available in the same cache block.

The Memory hierarchy combines storage levels with different speeds and capacities. From closest to the processor to farthest, a typical arrangement is:

1. Registers — the smallest and fastest storage, inside the processor.

2. Cache — fast storage, often divided into levels such as L1, L2, and L3. Nearer levels are typically smaller and faster; some designs separate instruction and data caches.

3. Main memory (DRAM) — larger than cache, but slower.

4. Secondary storage, such as an SSD or hard drive — much larger and non-volatile, but slower than main memory.

When data is not found at one level, the system checks or fetches it from a slower level. Hardware usually manages caches, while operating systems manage movement between main memory and secondary storage through virtual-memory mechanisms. This organization provides fast access to frequently used information without requiring all storage to be equally fast and costly.

Takeaway: Addresses locate data, and the memory hierarchy uses locality to make frequently needed data accessible quickly while relying on larger, slower storage for capacity.