#memory management

Tagged memory management · show all articles

A production server's memory riser tray drawn part-way out in a live rack row, one memory module held just above its slot mid-reseat, with a diagnostics cart alongside carrying a bandwidth-telemetry logger and a CXL card staged in an anti-static tray

AI Memory Systems and the Bandwidth Wall in Practice: An Advanced Technical Guide

A datasheet promises a bandwidth number no production node sustains. A field guide to the gap: paging and quantizing the KV cache, sizing batches against memory, staging CXL capacity on measured numbers, and profiling what a kernel is actually waiting on.

By Brecht Corbeel · October 4, 2026 · 16 min read

Browse by Category

Surprise me →

All AI Hardware & Semiconductors 115 AI Agents & Systems 43 Foundation Models 30 AI Industry 28 Model Evaluation 23 Technological Evolution 16 Edge AI & Electronics 15 Evolutionary Physics 14 Physics 13 Datacenters 13 Semiconductors 12 AI Infrastructure 12 AI Research 11 Human Systems 10 History of Technology 10 Evolutionary Psychology 10 Evolutionary Biology 10 Einstein & Evolution 10 Open Models 9 AI Safety 9 Robotics & Embodied AI 8 Biology 8 Human Evolution 6 AI for Science 6 AI Security 6 Materials Science 5 Institutions & Governance 5 Inference Economics 5 Governance & Policy 5 Future of Work 5 Future Studies 5 Evolutionary Nuclear Physics 5 Cultural Evolution 5 Neuroscience 4 Mathematics 4 Economic History 4 Crossed Fields 4 Chemistry 4 Space Science 3 Scientific Methods 3 Quantum Relativity 3 Invention & Engineering 3 Humanity & Technology 3 Human History 3 History of Science 3 Future Hardware 3 Ethics & Governance 3 Data & Training 3 Technology & Society 2 Semiconductor Markets 2 Security 2 Industry Analysis 2 Evolutionary AI 2 Earth Science 2 Datacenters & Infrastructure 2 Astronomy & Cosmology 2 Astrobiology 2 Writing & Information 1 Structural Biology 1 Small & Edge Models 1 Security & Technology 1 Scientific Revolution 1 Robotics 1 Quantum Information 1 Quantum Foundations 1 Quantum Biology 1 Policy & Regulation 1 Pokémon Formal Systems 1 Pokémon Formal Measurement 1 Pokémon Formal Machinery 1 Pokémon Economics 1 Planetary Science 1 Particle Physics 1 Origins & Astrobiology 1 Open-Weight Models 1 Metrology 1 Mathematics & Computation 1 Language Models & Evaluation 1 Labour & Technology 1 Interpretability 1 Institutions & State Capacity 1 Institutions & Economy 1 Industrial Revolutions 1 History of Computing 1 Game-Mechanic Economics 1 Evolutionary Economics 1 Evolution as Mechanism 1 Evolution & Ecology 1 Ethics of Technology 1 Energy & Civilization 1 Earth & Climate 1 Demography & Urbanization 1 Cumulative Culture 1 Cosmology 1 Condensed Matter 1 Chemistry & Catalysis 1 Alignment & Safety 1 Agriculture & Cities 1 Agent Evaluation 1 AI Ethics 1 AI Economics & Systems 1 AI & The Web 1