Seminar in probability theory: Marilou Gabrié (ENS)

Theory of Deep Learning 5: Information theoretic approach to deep learning theory: a test using statistical physics methods

The complexity of deep neural networks remains an obstacle to the understanding of their great efficiency. Their generalisation ability, a priori counter intuitive, is not yet fully accounted for. Recently an information theoretic approach was proposed to investigate this question.
Relying on the heuristic replica method from statistical physics we present an estimator for entropies and mutual informations in models of deep model networks. Using this new tool, we test numerically the relation between generalisation and information.

Veranstaltung übernehmen als iCal