“Circuit priors in NN-bayes: more results” by Kaarel, Dmitry Vaintrob
We're posting lots of papers written with GPT-6 Astra on generalization properties of bayesian neural net learning. The following picture emerges: For various reasonable priors on MLP weights, the induced prior on functions is a circuit prior: for fixed depth, the ordering of functions by negative log prior is roughly their ordering by minimal circuit width. From a quantitative version of this statement, one gets the following sample complexity bound: if the function is implemented by a circuit of width , then conditioning on train data points is sufficient to get test accuracy. These results hold as long as is realizable; in particular, for arbitrarily overparametrized NN-bayes. Probably the main caveat is that these bounds have exponential dependence in the depth of the circuit implementing . One can get only polynomial dependence in depth by restricting the prior to only neural nets satisfying a robustness assumption. For various other reasonable priors on MLP weights,, one gets bad sample complexity when very overparametrized: when learning some functions , even after seeing train data points, the learner is still likely to have at least some constant test error rate. GPT-6 Astra [...] The original text contained 8 footnotes which were omitted from this narration. --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/oePzacZn6wNcJPvH3/circuit-priors-in-nn-bayes-more-results --- Narrated by TYPE III AUDIO.