Symbol T
T is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →Published equation contexts
Why this formula appears here
Treat checkpointing as a classic renewal-reward tradeoff — this is my own worked derivation, using standard reasoning from fault-tolerant computing rather than any claim from the sources below. Let C be the wall-clock cost of writing one checkpoint and M be the job’s mean time between failures. Checkpointing on an interval T costs, per mean-time-between-failures period, roughly MC/T in write overhead, plus an expected T/2 of recomputation lost when a failure lands partway through an interval. The total wasted time per period is approximately . which is minimized at
Read the representative guide
Symbol C
the wall-clock cost of writing one checkpoint and M be the job’s mean time between failures.
Read this term in its guide →Denominator: 2
The complete quantity below the fraction bar; it must be nonzero for this division.
Read this term in its guide →How to interpret it
With a fixed numerator, increasing a nonzero denominator reduces the fraction. Its accuracy depends on the assumptions and range of use described in the article.
Research cited beside this formula
- [3] Check-N-Run: A Checkpointing System for Training Deep Learning Recommendation Models ↗
- [4] GEMINI: Fast Failure Recovery in Distributed Training with In-Memory Checkpoints ↗
- [5] Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures ↗
- [9] The Llama 3 Herd of Models ↗
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 10 · Datacenters
How AI Datacenter Systems Engineering Actually Works
This equation gives an approximation: it relates the quantities while allowing an approximation.
Treat checkpointing as a classic renewal-reward tradeoff — this is my own worked derivation, using standard reasoning from fault-tolerant computing rather than any claim from the sources below. Let C be the wall-clock cost of writing one checkpoint and M be the job’s mean time between failures. Checkpointing on an interval T costs, per mean-time-between-failures period, roughly MC/T in write overhead, plus an expected T/2 of recomputation lost when a failure lands partway through an interval. The total wasted time per period is approximately . which is minimized at
Meanings in this article
- : the because.
- : the wall-clock cost of writing one checkpoint and M be the job’s mean time between failures.