The dye diffuses in a complicated manner, which is difficult to precisely predict. However, this description is relatively simple only when the system is in a state of equilibrium.Įquilibrium may be illustrated with a simple example of a drop of food coloring falling into a glass of water. The (Shannon) entropy of a variable is defined as bits, where is the probability that is in the state, and is defined as 0 if. In mathematics, a more abstract definition is used. Therefore, the system can be described as a whole by only a few macroscopic parameters, called the thermodynamic variables: the total energy E, volume V, pressure P, temperature T, and so forth. Information Theory Entropy In physics, the word entropy has important physical implications as the amount of 'disorder' of a system.

During the process of attaining equilibrium, it is possible to tap into the system to perform work, as in a heat engine. The ensemble of microstates comprises a statistical distribution of probability for each microstate, and the group of most probable configurations accounts for the macroscopic state. The measurement of the extent of this evening-out process is called entropy. The large number of particles of the gas provides an infinite number of possible microstates for the sample, but collectively they exhibit a well-defined average of configuration, which is exhibited as the macrostate of the system, to which each individual microstate contribution is negligibly small. The collisions with the walls produce the macroscopic pressure of the gas, which illustrates the connection between microscopic and macroscopic phenomena.Ī microstate of the system is a description of the positions and momenta of all its particles. At a microscopic level, the gas consists of a vast number of freely moving atoms or molecules, which randomly collide with one another and with the walls of the container. The easily measurable parameters volume, pressure, and temperature of the gas describe its macroscopic condition ( state). A useful illustration is the example of a sample of gas contained in a container.

Ludwig Boltzmann defined entropy as a measure of the number of possible microscopic states ( microstates) of a system in thermodynamic equilibrium, consistent with its macroscopic thermodynamic properties, which constitute the macrostate of the system. Main article: Boltzmann's entropy formula

Entropy definition