ENARM
VLSI / Memory
VLSI · Memory Systems

Memory

Every chip needs somewhere to hold data — and no single memory technology is fast, dense, cheap, and non-volatile all at once. This page walks through the memory hierarchy and the cell-level mechanisms behind each major technology: SRAM and DRAM (volatile), the ROM/EEPROM/Flash family (charge-storage non-volatile), the emerging MRAM/ReRAM/PCM/FeRAM class, and HBM — the stacked-DRAM technology built to feed today's AI accelerators.

SRAM DRAM ROM / EEPROM NOR / NAND Flash 3D NAND MRAM ReRAM PCM FeRAM HBM

The Memory Hierarchy

No single memory is both fast and cheap enough to fill every role on a chip or in a system, so real designs use a hierarchy: small, fast, expensive memories sit close to the compute logic, and large, slow, cheap memories sit farther away. Data migrates up toward the processor as it's needed and back down as it's evicted.

⤢ Click to zoom SPEED / COST PER BIT ↓        DENSITY / CAPACITY ↑ Registers SRAM L1 / L2 / L3 Cache DRAM Main Memory (DDR / LPDDR / HBM) NAND Flash / SSD Non-volatile bulk storage (embedded NOR / EEPROM hold small code & config data alongside it) VOLATILE NON-VOLATILE fastest, priciest/bit, smallest slowest, cheapest/bit, largest

The memory hierarchy: capacity grows and cost-per-bit falls as data moves away from the processor, while access latency grows. Registers and SRAM lose their contents without power; DRAM needs continuous refresh to stay volatile-but-alive; Flash and the emerging NVM technologies below hold data with no power at all.

Two axes matter most when picking a memory for a job:

Volatile vs. non-volatile

Volatile memory (SRAM, DRAM) loses its contents the instant power is removed, but offers the fastest access. Non-volatile memory (ROM family, Flash, and the emerging MRAM/ReRAM/PCM/FeRAM class) retains data with no power at all, at some cost in speed, endurance, or density.

Charge-based vs. resistance-based

Older technologies (DRAM, Flash, EEPROM) store a bit as an amount of electrical charge. Most emerging non-volatile memories instead store a bit as a physical resistance or magnetic state — a shift that underlies their speed and endurance advantages.

Volatile Memory: SRAM & DRAM

Volatile memory trades data retention for speed and, in SRAM's case, simplicity of access. Both SRAM and DRAM must stay powered to hold data, but they get there through very different cell designs.

⤢ Click to zoom SRAM — 6T Cell VDD Inv 1 Inv 2 cross-coupled feedback Q Q′ M5 M6 BL BL′ WL access transistors (M5/M6) gated by word line WL connect Q/Q′ to bit lines no refresh — latch holds state while powered DRAM — 1T1C Cell Bit line (BL) Access Tr. WL C₀ Vref plate 1 transistor + 1 capacitor per bit charge leaks — row refreshed ≤ every 64ms array row sense amps detect tiny bit-line voltage & restore full logic level

SRAM's cross-coupled inverter pair (Inv 1 / Inv 2) latches Q and Q′ and needs no refresh, but costs 6 transistors per bit. DRAM stores a bit as charge on a single capacitor behind one access transistor — far denser, but the charge leaks away and must be periodically read and rewritten (refreshed).

SRAM (Static RAM)

The classic SRAM cell uses six transistors: four form two cross-coupled CMOS inverters (a bistable latch that holds either logic state indefinitely while powered), and two "access transistors" gated by the word line connect the internal nodes Q/Q′ to the complementary bit lines BL/BL′ for reads and writes.

Because the latch actively drives full-rail voltage onto the bit lines rather than relying on a tiny charge-sharing signal, and because there's no refresh or multiplexed row/column addressing to wait on, SRAM is the fastest memory on the chip — which is why it's used for CPU/GPU caches, register files, and FPGA configuration memory. 8T and 10T variants add dedicated read or write ports (dual-port SRAM for simultaneous read+write) or decouple the read path from the storage nodes for better stability at low voltage. The tradeoff is area and cost: 6 transistors per bit is far less dense than DRAM's one transistor plus one capacitor, and subthreshold leakage through the cell's transistors draws standby power continuously, rising sharply with temperature.

DRAM (Dynamic RAM)

A DRAM cell is just one access transistor and one storage capacitor (1T1C) — the capacitor's charge state encodes the bit. That charge leaks away through junction and subthreshold leakage, so every row must be read and rewritten ("refreshed") on a JEDEC-specified interval of 64 ms or less, consuming power and periodically blocking normal access.

Access is multiplexed: a Row Address Strobe (RAS) "activates" a row, connecting its capacitors to the bit lines; because bit-line capacitance is roughly 10× the cell capacitance, the resulting signal is tiny, so sense amplifiers detect and amplify it to a full logic level (which also refreshes that row). A Column Address Strobe (CAS) then selects specific columns from the now-open row, and a precharge step resets the bit lines before the next access. This extra latency, plus the destructive/refresh-requiring read, is why DRAM is slower than SRAM — but its 1T1C cell is dramatically denser and cheaper per bit, which is why it's the default choice for system main memory (DDR5, LPDDR5X) and the DRAM dies stacked inside HBM.

GenerationNominal voltageNotes
DDR31.5V (1.35V low-voltage)400–1067 MHz clock range
DDR41.2VUp to 64GB/DIMM; wider adoption of on-die ECC-adjacent features
DDR51.1V, on-die voltage regulation4800–6400+ MT/s standard, 8 bank groups (vs. 4 on DDR4), all chips include on-die ECC
LPDDR5XLow-power, mobile/automotiveUp to roughly 8.5–10.7 Gbps/pin depending on implementation; used for on-device AI workloads

Non-Volatile Memory: ROM, EEPROM & Flash

This family stores a bit as trapped electrical charge on an electrically isolated gate, which shifts the transistor's threshold voltage and persists with no power applied.

The ROM Family

TypeProgrammedErasedNotes
Mask ROMAt the factory, via photomaskNever — fixed for lifeCheapest per bit at huge volume; effectively obsolete now that Flash is cheap
PROMOnce, by "blowing" fuses/antifusesNever (one-time programmable)Bipolar fuse-link PROMs date to the early 1970s
EPROMFloating-gate charge injectionUV light through a quartz windowIntroduced 1971 (Intel); bulk-erase only, no selective erase
EEPROMElectrical pulsesElectrical pulses, byte-addressableStill used today for small persistent data — config, calibration, serial numbers

How Flash Stores a Bit

⤢ Click to zoom Floating-Gate Flash Transistor Control Gate inter-poly oxide Floating Gate (electrically isolated) tunnel oxide Channel Source Drain trapped e⁻ (programmed = raises VT) Program (write “0”): hot-electron injection (NOR) or Fowler–Nordheim tunneling (NAND) Erase: Fowler–Nordheim tunneling pulls electrons back off, one block at a time

Charge trapped on the floating gate raises the transistor's threshold voltage (VT); a sense amplifier reads which side of that threshold the cell is on. Modern charge-trap flash replaces the continuous conductive floating gate with discrete dielectric traps (e.g. silicon nitride), so a local oxide defect only leaks the nearby traps instead of the whole gate — the reliability improvement that makes dense 3D NAND practical.

NOR vs. NAND

NOR Flash

Cells wired in parallel, each with its own direct bit-line connection — enabling random, byte-level addressing. This supports execute-in-place (XIP), letting a CPU run code straight from NOR without first copying it to RAM, which is why NOR is the standard for embedded firmware and bootcode. Tradeoff: lower density and higher cost per bit, slower writes/erases.

NAND Flash

Cells wired in series strings sharing bit lines — a much smaller layout per cell, so density is far higher and cost per bit far lower. Access is by page (program) and block (erase), not random byte addressing, which is why NAND dominates bulk storage: SSDs, USB drives, eMMC/UFS, and memory cards.

3D NAND

⤢ Click to zoom Planar (2D) NAND cells packed side by side — shrinking pitch increases cell-to-cell interference & leaves only a handful of electrons per cell — scaling stalls 3D NAND WL layers vertical channels cells stack vertically around a shared vertical channel using charge-trap layers — density comes from adding layers, not shrinking cells 2025–2026: SK hynix shipped 321-layer QLC NAND; Samsung has demonstrated 400+ and reportedly explored 900-layer designs; roadmaps discuss 500 and eventually ~1,000 layers as bit density continues to scale vertically.

Planar NAND scaling stalled because shrinking cell pitch increases cell-to-cell interference and leaves each cell storing charge as only a few electrons. 3D NAND instead stacks cells vertically in many word-line layers around a shared channel, using charge-trap storage that tolerates the proximity and process variation of dense stacking far better than a floating gate.

Cell typeBits/cellDensity / costEndurance & speed
SLC1Lowest density, highest cost/bitHighest endurance & speed
MLC2——
TLC3——
QLC4——
PLC5Highest density, lowest cost/bitLowest endurance & speed

More bits per cell means more distinguishable voltage levels packed into the same window, so density rises and cost per bit falls — but the margin between levels shrinks, cutting program/erase-cycle endurance and speed and raising error rates. Controllers offset the finite program/erase-cycle lifetime with wear leveling, spreading writes evenly across blocks so no single block wears out first.

Emerging Non-Volatile Memory

Where Flash and EEPROM store a bit as trapped charge, the newer non-volatile technologies store it as a physical resistance or magnetic state — a change that typically brings much higher write endurance and speed, at a current cost in density or maturity compared to Flash.

⤢ Click to zoom MRAM — Magnetic Tunnel Junction (MTJ) Parallel → low resistance → “0” Free layer ↑ MgO barrier Pinned layer ↑ Antiparallel → high resistance → “1” Free layer ↓ MgO barrier Pinned layer ↑ STT write: spin-polarized current through the stack flips the free layer I (spin-polarized current) TMR (tunnel magnetoresistance) sets the resistance contrast between states; STT-MRAM eliminates the separate write line used by older field-switched MRAM, giving fast (ns-scale) writes and very high (10⁸⁽⁾+) endurance — used today mainly as embedded MRAM in automotive/industrial microcontrollers, replacing Flash/EEPROM at nodes where Flash no longer scales well.

A magnetic tunnel junction stores a bit as the relative magnetization of a free layer against a fixed reference layer, separated by a thin MgO tunnel barrier. Spin-transfer torque (STT) writes by passing current directly through the stack, rather than needing a separate magnetic write line.

Tunnel magnetoresistance ratio TMR = (RAP − RP) ⁄ RP

RAP and RP are the junction's antiparallel and parallel resistances. A larger TMR gives a wider, easier-to-sense margin between the "0" and "1" states.

ReRAM / RRAM

A metal-insulator-metal stack, typically a transition-metal-oxide dielectric (e.g. HfO₂). A voltage above threshold migrates oxygen vacancies into a localized conductive filament bridging the electrodes (low resistance); a reverse voltage ruptures it (high resistance). The simple two-terminal structure is CMOS-back-end compatible and crossbar-array friendly, and its tunable, analog conductance makes it attractive for neuromorphic / in-memory computing, where the memory array itself performs multiply-accumulate operations.

PCM (Phase-Change Memory)

A chalcogenide alloy (e.g. germanium-antimony-tellurium) is switched between an amorphous (high-resistance) and crystalline (low-resistance) state with heat pulses; the resistance difference encodes the bit. Intel/Micron's 3D XPoint (Optane) used a related cross-point, transistor-free stackable architecture and was marketed as byte-addressable persistent memory positioned between DRAM and NAND — Intel discontinued the Optane business in 2022, but the cross-point array concept remains architecturally influential.

FeRAM

Uses a ferroelectric capacitor (e.g. lead zirconate titanate, PZT) in a DRAM-like 1T1C-style cell; the bit is the direction of the material's electric polarization, which persists without power. Switching that polarization takes far less energy than Flash's charge-pumping, giving very fast, low-power writes — but FeRAM hasn't scaled to high densities, so it stays in niche roles like smart cards and industrial/automotive controllers.

TechnologyStorage mechanismStandout propertyTypical role today
MRAM / STT-MRAMMagnetic state (MTJ)Fast + very high enduranceEmbedded automotive/industrial MCUs, replacing Flash/EEPROM at advanced nodes
ReRAMConductive filament (resistance)Simple, dense, analog-tunableEmerging embedded NVM & neuromorphic/in-memory compute research
PCM / 3D XPointAmorphous/crystalline phase (resistance)Byte-addressable, DRAM-NAND gap fillerLargely discontinued commercially (Optane, 2022); architecture lives on in cross-point research
FeRAMFerroelectric polarizationVery fast, very low write energySmart cards, metering, industrial control

HBM & the Memory Wall

Compute throughput on GPUs and AI accelerators has grown far faster than DRAM bandwidth and latency — a gap known as the memory wall. For large-model training and inference, performance is often bounded not by how many operations the compute die can do, but by how fast weights, activations, and KV-cache can move from memory, and moving that data can cost more energy than the arithmetic itself. High Bandwidth Memory (HBM) is the industry's primary answer.

⤢ Click to zoom HBM: TSV-Stacked DRAM on a Silicon Interposer DRAM dies (4–16-Hi) base logic die TSVs (through-silicon vias) GPU / AI accelerator die Silicon interposer (2.5D packaging — short, dense interconnect) A 1024-bit-wide data bus (vs. ~32-bit on GDDR) trades per-pin clock speed for much higher aggregate bandwidth at lower power. Generations HBM2E → HBM3 → HBM3E → HBM4 (2026) keep raising per-stack bandwidth & capacity as AI accelerators scale.

HBM stacks several DRAM dies vertically, connected by through-silicon vias (TSVs), atop a base logic die that handles I/O. The stack sits beside the compute die on a silicon interposer for 2.5D packaging — short, wide, dense connections instead of a narrow board-level bus.

Bandwidth from bus width BW = data rate × bus width ⁄ 8

A 1024-bit-wide HBM interface reaches very high aggregate bandwidth even at a modest per-pin data rate, which is also why it draws less power per bit moved than a narrow, high-clock bus like GDDR.

HBM vs. GDDR: GDDR (GDDR6/GDDR7) uses a narrower per-chip bus clocked much faster, and is cheaper and simpler to integrate — the default for mainstream consumer GPUs. HBM trades higher cost and 2.5D packaging complexity (interposer, TSVs) for far greater bandwidth per watt, which is why it's reserved for high-end AI/HPC accelerators. Because training and inference for large models are frequently bandwidth- and latency-bound rather than compute-bound, HBM supply has itself become a gating factor for AI accelerator production in 2025–2026.

Note on figuresExact HBM4-generation numbers (per-pin data rate, capacity per stack) vary across 2025–2026 sources and include roadmap projections alongside shipping specs — treat generation-to-generation bandwidth/capacity figures as directional rather than fixed until a part is in volume production.

Technology Comparison

TechnologyVolatile?Relative speedRelative densityEnduranceTypical use
SRAMYesFastestLowestUnlimited (no wear mechanism)CPU/GPU cache, registers, FPGA config
DRAMYesFastHighUnlimitedSystem main memory
NOR FlashNoFast read, slow writeLow–MedTens of thousands of P/E cycles (SLC-class)Embedded firmware / XIP code
NAND / 3D NANDNoMediumVery highLow thousands (QLC) to tens of thousands (SLC) P/E cyclesSSDs, USB, eMMC/UFS bulk storage
MRAM / STT-MRAMNoVery fastMediumVery high (10⁸⁽⁾+ cycles)Embedded automotive/industrial MCUs
ReRAMNoFastMed–High (crossbar-capable)High, improvingEmerging embedded NVM, neuromorphic compute
PCM / 3D XPointNoFast (DRAM–NAND gap)MediumHighMostly discontinued commercially
FeRAMNoVery fast writeLowVery highSmart cards, metering, industrial control
HBMYes (stacked DRAM)Very high bandwidthHigh, per-packageUnlimitedAI accelerators / GPUs

Subtopics

Memory Types & Operating Modes

1-port SRAM, true 2-port, pseudo 2-port, TCAM, and ROM compiler instances — clocks, bitcell ports, same-cycle read/write rules, and how to decode a compiler cell name.

Key Terms

TermMeaning
6T / 1T1CSix-transistor SRAM cell vs. one-transistor-one-capacitor DRAM cell
RefreshPeriodic read-and-rewrite of DRAM rows to replace leaked capacitor charge
RAS / CASRow / Column Address Strobe — DRAM's multiplexed addressing signals
Floating gateElectrically isolated gate in Flash/EPROM/EEPROM that stores charge to set VT
Fowler–Nordheim tunnelingQuantum tunneling of electrons through a thin oxide under a strong field — used to program/erase Flash
XIPExecute-In-Place — running code directly from NOR Flash without copying to RAM
P/E cycleOne program/erase cycle of a Flash cell; the finite count of these bounds device lifetime
Wear levelingController technique that spreads writes evenly across Flash blocks to extend lifetime
MTJMagnetic Tunnel Junction — the resistive storage element in MRAM
TSVThrough-Silicon Via — vertical interconnect used to stack DRAM dies in HBM
Memory wallThe growing gap between compute throughput and memory bandwidth/latency

Sources