Running neural nets fast and cheaply is partly a hardware problem, and one promising trick is in-memory computing: doing the multiply-and-add right inside a ReRAM memory crossbar instead of shuttling data to a separate processor. The catch is that these blocks only handle fixed-weight multiplications well. Transformers, which lean on nonlinear steps and multiplying two changing matrices, get much less benefit, and the analog-to-digital converters at each block eat over 70 percent of its area and power.

This work redesigns the block to drop the converter entirely, replacing it with analog content-addressable memories that perform the nonlinear steps in place, and maps attention onto them. On CNN and Transformer benchmarks they report up to 40x and 1.9x better energy efficiency respectively, with area gains too.

I am reading these off the abstract, so check the paper for the design-space details and how broadly the gains apply.