What does this equation mean?

normalizing each sampled response’s reward against the mean and standard deviation of a group of G responses to the same prompt rather than against a separately trained critic network [ 13 ] . Lambert and colleagues, building the fully open Tulu 3 post-training recipe, named the general approach explicitly, describing it as “a novel method we call Reinforcement Learning with Verifiable Rewards,” and used it alongside supervised fine-tuning and preference optimization rather than as a wholesale replacement for either [ 15 ] . The clearest large-scale demonstration is DeepSeek-R1: Guo and colleagues report training a model with reinforcement learning alone against a purely rule-based reward — combining an answer-correctness check with a format check, and deliberately avoiding a learned reward model because, in their account, a neural reward model “may suffer from reward hacking in large-scale reinforcement learning” — and observed pass@1 accuracy on the AIME 2024 competition-mathematics benchmark rise from 15.6 percent to 71.0 percent over training, reaching 86.7 percent with majority voting across 64 samples [ 14 ] . Formally, the reward itself is as simple as the reward model above was elaborate: