Language models learn from enormous piles of scraped web text, and if someone slips poisoned content into that pile, the resulting bad behavior is hard to catch later. Earlier studies mostly poked at Wikipedia, which is neither as large nor as messy as a real training corpus, and they ignored the filtering pipelines that decide what gets kept. This paper looks at a more realistic route: the public comment and discussion widgets scattered across countless sites, which let an attacker inject text at web scale.

The other useful piece is a method called HalfLife that estimates how much injected content survives crawling and curation to reach the training data. That distinction matters, because injecting poison and having it actually make the cut are two different things. Their analysis points to third-party page content as a real attack surface for pretraining.

This draws on the abstract, which frames feasibility more than a finished attack, so read the paper for the actual evidence and its limits.