Information entropy and the statistical physics of the classical ideal gas

The link between statistical mechanics and information theory is examined with reference to the classical ideal gas. In particular, Jaynes' assertion [E. T. Jaynes, Phys. Rev. 106(4), 620 (1957)] that a clear distinction between statistical and physical aspects, the latter consisting of the correct enumeration of the system's states and their properties, is questioned in relation to the information entropy. Jaynes suggested that statistical mechanics need not be considered a physical theory, as the techniques of statistical inference lead to physically sensible results. However, we show that a physical theory of the statistics of the ideal gas leads to expressions for the information entropy that differ from those found in statistical mechanics because the probability distributions underlying the entropy differ from the canonical distribution. In particular, we show that the distribution of energy states accessible to a classical ideal gas connected to a thermal reservoir is given by the Gamma distribution and in open systems, in which the number of particles fluctuates, by a weighted sum of Gamma distributions. Computer simulations using a hard-sphere model of a classical ideal gas demonstrate the ideas. We compare the Shannon information entropy with the thermodynamic entropy and argue on the basis of the work presented here that information theoretic entropy and thermodynamic entropy, whilst seemingly related, are not necessarily identical.