Skip to content
Sign in
Artwork for Eye on AI Weekly Research Watch
Eye on AI Weekly Research Watch · September 16 · 2 min

Attention Quantization for Tabular Foundation Models

Tabular foundation models are served differently from chatbots, so the usual speedups don't carry over. Here the bottleneck is the attention calculation. Converting queries, keys and values to FP8 gave up to 1.7 times the speed with no meaningful accuracy loss on TabPFN-v3 and TabICLv2. The catch: quantization error on test rows has to match the training rows, or accuracy drops sharply. Authors: Jonas M. Kübler, Benjamin Jäger, Klemens Flöge, Noah Hollmann, Frank Hutter Paper: https://arxiv.org/abs/2609.13031v1

0:00-2:37
Play in

transcript

No transcript — this publisher did not publish one.

show notes

Tabular foundation models are served differently from chatbots, so the usual speedups don't carry over. Here the bottleneck is the attention calculation. Converting queries, keys and values to FP8 gave up to 1.7 times the speed with no meaningful accuracy loss on TabPFN-v3 and TabICLv2. The catch: quantization error on test rows has to match the training rows, or accuracy drops sharply.

Authors: Jonas M. Kübler, Benjamin Jäger, Klemens Flöge, Noah Hollmann, Frank Hutter

Paper: https://arxiv.org/abs/2609.13031v1

more episodes

All episodes