What does this equation mean?

Rejection sampling generates multiple candidate completions per prompt from a model and keeps only the ones a separate scoring step accepts, using the survivors as supervised fine-tuning data. Llama 2’s RLHF pipeline used exactly this for its first four rounds, generating candidates from the 70-billion-parameter model, scoring them against a trained reward model, and only introducing proximal policy optimisation as a second mechanism in later rounds once the returns from sampling alone began to taper [ 6 ] . A second, independent demonstration in mathematical reasoning makes the mechanism even more explicit: rejection sampling fine-tuning collects correct reasoning paths directly from a supervised model’s own sampled outputs, with no additional human annotation, and combining rejection samples pooled from multiple models raised a 7-billion-parameter LLaMA’s GSM8K accuracy from a 35.9 percent supervised baseline to 49.3 percent [ 7 ] . The method’s economics follow directly from its mechanism: if a checker accepts a fraction p of generated candidates, the expected cost of producing one accepted sample scales as