≤

The passage around this formula

A device can report gigabytes of free memory and still kill the process that asks for a much smaller allocation, because the operating system’s out-of-memory killer does not care about total free memory — it cares about whether a single contiguous block large enough for the request exists. Large weight tensors and growing key-value caches on a constrained device fragment memory into scattered free regions faster than they fill it, and an allocation can fail against a device that is, in aggregate, nowhere near full. is the condition that actually governs whether an allocation succeeds, and it can fail even when $S_{\mathrm{request}}$ $\le$ $\sum_i$ $\mathrm{free}_i$ holds comfortably — the requested…

Read this part in the article →