Entropy Numbers for Convex Combinations and MLPs

Improved bounds on the entropy numbers of convex combinations of parameterized functions (such as combinations of classifiers) and compositions of these combinations (such as multi-layer neural networks) are given. In the latter case especially, the new bounds presented are substantially smaller than the previous results. They show that even more substantial improvements are possible when the parameterized functions involve kernels with rapidly decreasing eigenvalues. This gives the best known bounds for the covering numbers of radial basis function networks, for instance.