$$a^* = \arg\min_{a \in \mathcal{A}} \ \mathcal{L}_{\text{val}}\big(w^*(a),\, a\big) \quad \text{subject to} \quad g(a) \le B, \qquad w^*(a) = \arg\min_{w} \ \mathcal{L}_{\text{train}}(w, a)$$

Equation 19 · Edge AI & Electronics

Shrink It, Train It Small, or Search for It: The Main Strategies for Small Models, Compared

This equation states a bound: one expression must stay on the indicated side of the other under the article’s assumptions.

Zoph and Le established the modern form of the idea: a controller network, trained by reinforcement learning, proposes candidate child-network architectures, each of which is trained and evaluated, with the resulting performance used as a reward signal to improve the controller [ 7 ] . Formally, a NAS run of this kind is a constrained, nested optimization: $a^* = \arg\min_{a \in \mathcal{A}} \ \mathcal{L}_{\text{val}}\big(w^*(a),\, a\big) \quad \text{subject to} \quad g(a) \le B, \qquad w^*(a) = \arg\min_{w} \ \mathcal{L}_{\text{train}}(w, a)$. where $\mathcal{A}$ is a search space of candidate architectures designed in advance by the researchers, g(a) some measured deployment cost of architecture a , and B a budget the target device imposes. Every term in that equation is a documented design choice, and the choice of g turns out to be where the edge-specific…

Meanings in this article

• $\mathcal{A}$: a search space of candidate architectures designed in advance by the researchers, g(a) some measured deployment cost of architecture a.
• $B$: the budget the target device imposes.

Equation guide → · Article →