Rubrics, checklists of what a good answer should contain, are handy for training and grading language models, but writing a reliable one for a specific question is hard. You can lean on human-written rubrics or preference data, or you can have the model generate a rubric straight from the query, which is cheap but risky: nothing checks whether that rubric actually separates good answers from bad, rewards mere style, or unfairly punishes a valid alternative approach.

Rubrics on Trial starts from an empty set and grows one, generating synthetic pairs of responses conditioned on each candidate rubric and keeping only the rubrics that genuinely distinguish quality. No human labels and no model training are needed. Across five preference benchmark suites it reports the best average accuracy.

That summary is from the abstract, so read the paper for the validation criteria and benchmark breakdown.