University Libraries Promotion and Tenure Recognition

A fundamental task central to many problems in science and engineering is to specify a probability distribution for some variables of interest, given a finite number of measurements (effectively, given a training sample). Since there are numerous (in many cases, uncountably infinite) distributions consistent with given measurements, it may appear at first glance that there is no principled way to favor any particular solution. However, in his book Papers on Probability, Statistics, and Statistical Physics, E.T. Jaynes champions a statistical inference principle on which to choose probability distributions—the principle of maximum entropy—and provides compelling reasoning to favor this distribution over other distributions that are also consistent with the given data. This inference principle can help e.g. ascertain whether or not dice are loaded and provides a well-grounded framework for pattern recognition, decision making, and even for addressing combinatorial optimization problems. This principle and the argument favoring its use struck me forcibly as a graduate student, has inspired some of my past research work, and continues to inspire my research in statistical pattern recognition and machine learning. The maximum entropy distribution is unique, and so too, Jaynes' book is singular in its formative effective on my technical education and career.