Equation 19 · Shrink It, Train It Small, or Search for It: The Main Strategies for Small Models, Compared

What does this equation mean?

a^* = \arg\min_{a \in \mathcal{A}} \ \mathcal{L}_{\text{val}}\big(w^*(a),\, a\big) \quad \text{subject to} \quad g(a) \le B, \qquad w^*(a) = \arg\min_{w} \ \mathcal{L}_{\text{train}}(w, a)

Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.

This equation states a bound: one expression must stay on the indicated side of the other under the article’s assumptions. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.

Read it piece by piece

a^*

Symbol a^*

$a^*$ is part of the quantity the equation computes from the expression on the right.

Understand this part →

a

Symbol a

a is part of the quantity the equation computes from the expression on the right.

Understand this part →

\mathcal{A}

Symbol A

a search space of candidate architectures designed in advance by the researchers, g(a) some measured deployment cost of architecture a.

Understand this part →

\mathcal{L}_{\text{val}}

Symbol L_val

$L_v$ al is one factor in the product that computes the quantity on the left.

Understand this part →

w^*

Symbol w^*

$w^*$ is one factor in the product that computes the quantity on the left.

Understand this part →

g

Symbol g

g is one factor in the product that computes the quantity on the left.

Understand this part →

B

Symbol B

the budget the target device imposes.

Understand this part →

w

Symbol w

w is one factor in the product that computes the quantity on the left.

Understand this part →

\mathcal{L}_{\text{train}}

Symbol L_train

$L_t$ rain is one factor in the product that computes the quantity on the left.

Understand this part →

=

The expressions on both sides represent the same quantity under the stated assumptions.

Understand this part →

See an illustrated explanation →

≤

Less than or equal to.

Understand this part →

See an illustrated explanation →

subscript

The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.

Understand this part →

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

What the article says around this equation

Zoph and Le established the modern form of the idea: a controller network, trained by reinforcement learning, proposes candidate child-network architectures, each of which is trained and evaluated, with the resulting performance used as a reward signal to improve the controller [ 7 ] . Formally, a NAS run of this kind is a constrained, nested optimization: $a^* = \arg\min_{a \in \mathcal{A}} \ \mathcal{L}_{\text{val}}\big(w^*(a),\, a\big) \quad \text{subject to} \quad g(a) \le B, \qquad w^*(a) = \arg\min_{w} \ \mathcal{L}_{\text{train}}(w, a)$ . where $\mathcal{A}$ is a search space of candidate architectures designed in advance by the researchers, g(a) some measured deployment cost of architecture a , and B a budget the target device imposes. Every term in that equation is a documented design choice, and the choice of g turns out to be where the edge-specific…

Read the full surrounding passage

Zoph and Le established the modern form of the idea: a controller network, trained by reinforcement learning, proposes candidate child-network architectures, each of which is trained and evaluated, with the resulting performance used as a reward signal to improve the controller [ 7 ] . Formally, a NAS run of this kind is a constrained, nested optimization: $a^* = \arg\min_{a \in \mathcal{A}} \ \mathcal{L}_{\text{val}}\big(w^*(a),\, a\big) \quad \text{subject to} \quad g(a) \le B, \qquad w^*(a) = \arg\min_{w} \ \mathcal{L}_{\text{train}}(w, a)$ . where $\mathcal{A}$ is a search space of candidate architectures designed in advance by the researchers, g(a) some measured deployment cost of architecture a , and B a budget the target device imposes. Every term in that equation is a documented design choice, and the choice of g turns out to be where the edge-specific literature does its most important work.

Read the equation in its article →

Sources cited in the surrounding passage

[7] Neural Architecture Search with Reinforcement Learning ↗

These citations give research context. Read each source to check which claims it supports.

Return to Shrink It, Train It Small, or Search for It: The Main Strategies for Small Models, Compared

See this formula across 1 published context →

Browse the mathematical compendium →