Here's a clean sanity check for language models. Ask a model about a whole population and the answer should match what you get by splitting that population into groups, asking about each, and weighting the results back together. Basic probability says the two have to agree. The authors test whether LLMs actually respect this, using binary trees to carve a population into finer subgroups, then aggregating the model's answers back up.

The numbers routinely fail to line up. Looking at persona prompting, they find what they call the macro fallacy: answers rebuilt from many narrow subgroups often match real human data better than asking about the whole population directly. The model seems to know the pieces but doesn't add them up consistently. That gap, they argue, makes self-consistency a handy test that needs no answer key.

This rests on the abstract alone, so see the paper for which models they tested and how big the violations get.