The Mystery of Entropy

by Ben Best

CONTENTS: LINKS TO SECTIONS

  1. FORMS OF ENERGY
  2. ATTEMPT TO DESCRIBE ENTROPY USING UNITS
  3. REVERSIBLE AND IRREVERSIBLE PROCESSES
  4. REVERSIBLE AND IRREVERSIBLE CYCLES
  5. SIMPLE EXAMPLES OF ENTROPY

I. FORMS OF ENERGY

In order to attempt to understand the concept of entropy, it is best to begin by clarifying concepts associated with energy.

Energy can be designated by the symbol E, and has units of Joules (as well as units of calories, electron-Volts, etc.). Energy consists of macroscopic energy and microscopic energy. Notable forms of macroscopic energy are kinetic energy and potential energy. It is possible to measure the kinetic and gravitational potential energy of an object by defining these terms relative to the surface of the Earth. But there is no absolute (non-relative) kinetic or potential energy of any object having no frame of reference.

The microscopic energy of an object (the energy of an object's molecules and subatomic particles) is called the internal energy (designated by the symbol U). It is not possible to measure or quantify the total internal energy of an object even in relative terms. Only the change in internal energy (ΔU) is meaningful. Internal energy includes such things as the translational kinetic energy of molecules (corresponding to the temperature of an object), the rotational energy of molecules, the vibrational energy of molecules and the energy of chemical bonds of molecules as well as intermolecular forces and subatomic particle energy. Kinetic energy and potential energy can be quantified relative to a point on the Earth, but there is no reference point against which the many forms of internal energy can be quantified. (A glucose molecule at zero Kelvin will have internal energy.)

A change of energy is the difference between the initial and final states of an object or system, represented by the symbol ΔE.

      Mathematically: ΔE = Efinal − Einitial

According to the First Law of Thermodynamics, energy is neither created nor destroyed (i.e.,the net energy of the universe never changes), which means that the total ΔE must always be zero. But forms of energy can change, such as Kinetic Energy (KE) and gravitational Potential Energy (PE). If an apple falls from a tree, the gravitational potential energy of the apple decreases while the kinetic energy of the apple increases as the apple falls. When the apple hits the ground, kinetic energy of the apple becomes zero, and the internal energy of the apple and Earth (totalled for both) increases by an amount equal to the decrease in kinetic energy.

      Mathematically: ΔE = ΔPE  + ΔKE + ΔU = 0     (First Law of Thermodynamics)

As the apple falls there is a decrease in gravitational potenial energy (ΔPE is negative) that exactly equals the increase in kinetic energy (the positive value of ΔKE), which nets to zero. Until the apple hits the ground, ΔU is zero. After the apple hits the ground there is no more kinetic energy (KE = 0) and the decrease in potential energy (negative ΔPE) is exactly equal and opposite to the increase in internal energy (positive ΔU).

For a gas, there is a form of energy represented as the product of pressure (P) and volume (V), which is represented as PV. PV energy is independent of internal energy, but is dependent upon Temperature (T) and the number of molecules in the system for which PV energy is being quantified. Unlike other forms of energy, PV energy can be quantified in absolute (non-relative) terms.

      Mathematically: PV = nRT where n is the number of molecules (in moles) and R is the universal gas constant. (The equation is only exactly correct for an ideal gas, which represents molecules as forceless, infinitesimal particles.)

The sum of internal energy and PV energy defines a form of energy called the enthalpy (designated by the letter H). Enthalpy is thermal energy, i.e., the energy associated with changes in temperature. Enthalpy includes PV energy, which varies with temperature (nRT) and with the internal energy associated with chemical reactions. Although enthalpy is often equated with heat (the term enthalpy comes from the Greek word enthalpien, which means "heat"), in Thermodynamics enthalpy is distinct from heat (which is designated by the letter Q) and is distinct from internal energy (which is designated by the letter U).

      Mathematically, enthalpy is defined as: H = U + PV

Because enthalpy includes internal energy, total enthalpy of a system or object cannot be quantified or measured directly. But enthalpy change (ΔH) can be measured. For an exothermic (energy-releasing) chemical reaction ΔH is the energy generated by the reaction (including the PV term for ambient pressure resistance). ΔH is negative for an exothermic chemical reaction (the enthalpy of the system or object is reduced) whereas ΔH is positive for an endothermic (energy-absorbing) chemical reaction (the enthalpy of the system or object is increased by the amount of energy absorbed).

Although work (W) can be quantified in energy units (Joules), work is not a form of energy. Work is a means of transferring energy. Objects or systems have properties such as temperature, pressure, mass, volume, energy, etc., but work is not a property. An object can be said to contain energy, but an object cannot be said to contain work. If work is done on an object to increase the object's velocity, the kinetic energy of the object will increase, but the object is not said to contain more (or any) work. Work is simply a process by which the energy of an object can be changed. In thermodynamics, work is often the work of compression or expansion by a piston on a gas contained in a fixed volume chamber ("PV work").

Heat (Q), like work, can be quantified in Joules. Like work, in Thermodynamics it is incorrect to describe heat as a form of energy. Heat is a non-work transfer of energy from one body to another. Temperature is easily quantified, but heat is not. An object does not contain a defined amount of heat, just as an object does not contain a defined amount of work. Like work, heat is a means of energy transfer. Energy is transferred, heat is not transferred. Despite this distinction, the terminology is often abused and with such phrases as "heat content", "flow of heat" and "heat transfer". The change in enthalpy due to a process is not the same as the change in heat due to a process, because enthalpy is a form of energy, but heat is a means of energy transfer. A given change in the internal energy portion of enthalpy (thermal energy) may involve a mixture of heat and work energy transfers. Only by studying the process can the relative contribution of work and heat to final internal energy be determined.

      Mathematically: dU = δQ + δW     (First Law of Thermodynamics)

In words: the change in internal energy is the sum of the energy change due to heat plus the energy change due to work. dU differs from ΔU in that ΔU represents a finite change whereas dU represents an infinitesimal change. The infinitesimal change of internal energy is a sum of infinitesimal changes of energy due to heat and work. The symbols δQ & δW are used rather than the symbols dQ & dW to denote the fact that the exact contributions of either the heat or the work term cannot be determined. Only the sum of energy change due to both heat and work can be determined exactly.

The First Law of Thermodynamics states that energy is conserved, i.e., energy is neither created nor destroyed (E = mc2 is beyond the scope of Thermodynamics, but total of mass and energy would be conserved). The expression ΔE = ΔPE  + ΔKE + ΔU = 0 is thus a statement of the First Law of Thermodynamics in expressing the fact that although potential, kinetic and internal energy may change as a result of a process, total energy does not change. The equation dU = δQ + δW is also sometimes used as a statement of the First Law of Thermodynamics, despite the fact that it is restricted to internal energy rather than all forms of energy.

(return to contents)

II. ATTEMPT TO DESCRIBE ENTROPY USING UNITS

Entropy (designated by the letter S) is an elusive concept that is defined in terms of other elusive concepts. The units of entropy are Joules per Kelvin (J/K). Joules are a unit of energy, work or heat, but in the context of the units of entropy, Joules quantifies only heat. So the units of entropy can be described as heat divided by temperature. Like mass, energy or volume, entropy is a property of an object or system. Mysteriously, the property entropy is calculated by dividing the non-property heat by the property temperature.

      Mathematically: dS = (δQ)/T     or   S = (δQ)/T 

Entropy is always associated with the heat form of energy transfer. In general, the entropy of an object or system can increase or decrease, associated with thermal energy being transferred into or out of an object or system ("flow of heat" in or out). Although the entropy of an object or system can increase or decrease, the entropy of the universe as a whole can never decrease. As stated in the Second Law of Thermodynamics, for any process, the entropy of the universe will either stay the same or increase, but never decrease. Only for a reversible process can the entropy of the universe remain unchanged. For every irreversible process (i.e., every real process) the entropy of the universe increases. The direction of heat transfer is the same as the direction of entropy transfer. When heat is transferred out of a system, entropy is transferred out of the system. When heat is transferred into a system, entropy is transferred into the system.

Mathematically, the product of Temperature and Entropy is an energy term (a consequence of the fact that entropy is equal to heat, quantified as energy, divided by temperature). For this reason, Temperature multiplied by an infinitesimal amount of Entropy (TdS) can be equated to infinitesimals of internal energy, enthalpy, volume and pressure:

       TdS = dU + pdV

       TdS = dH − Vdp

Entropy can also be defined by the equation:

       S = kB ln Ω

where kB is the Boltzmann constant, ln is the natural logarithm, and Ω is a measure of the disorder of a system, quantified as the total number of microscopic states occupied. The larger the number of occupied microscopic states (i.e., the larger the number Ω), the greater the disorder of the system. The number Ω is a unitless natural number (0,1,2,3,...) that quantifies disorder. In terms of gas molecules, the number Ω corresponds to all the positions and momenta ("momentums") of all of the gas molecules. Geometrically, the possible microscopic states can be represented as quantized space and momentum, where each quantized position is represented as a tiny cube of three spatial dimensions and each quantized momentum is represented as a tiny "momentum cube" in three momentum dimensions. Combining spatial and momentum dimensions gives a six-dimensional "phase space" containing a finite number of six-dimensional cubes. The very least probable (and the most "ordered") arrangement corresponds to all of the gas molecules being in a single six-dimensional cube of phase space, i.e., Ω = 1 and S = 0. The most probably arrangment will maximize both Ω and S — which will be the equilibrium condition.

kB represents the universal gas constant R per molecule of gas. Thus, the units of kB (J/K) times the natural logarithm of Ω (a unitless natural number) gives the units of S, namely, J/K, corresponding to the disorder of a system. Processes that occur naturally in as system are those that increase the disorder of the system.

The Third Law of Thermodynamics is an empirical law that states that the entropy of a pure crystalline substance is zero at zero Kelvin. Substances that do not have a pure crystalline structure at zero Kelvin have nonzero entropy at zero Kelvin. For substances that do have a pure crystalline structure at zero Kelvin, the Third Law of Thermodynamics provides a reference standard for determining the absolute entropy of the substance at other temperatures.

(return to contents)

III. REVERSIBLE AND IRREVERSIBLE PROCESSES

The entropy change of any process will be the sum of the entropy transfer (into or out of a system) plus the entropy production of the process. The amount of entropy produced by a process is a measure of the irreversibility of the process.

      Mathematically: dS = (δQR)/T + σ

In words: the change in entropy is the sum of (reversible) entropy transfer [(δQR)/T] plus entropy production [σ].

Another way to attempt to understand entropy is to look at how the concept is used. The Second Law of Thermodynamics states that for any process, entropy in the universe either increases or does not change, but never decreases (although the entropy of an isolated system can decrease by contact with another system). Only if a process is reversible does the entropy of the universe not change.

What are examples of irreversible processes?

Entropy provides a means of quantifying irreversibility. In an irreversible process the amount of entropy of the universe increases. The amount of entropy increase in the universe associated with an irreversible process corresponds to the amount of energy that is no longer available to perform work.

The entropy of an isolated system increases as the state of equilibrium is approached. Equilibrium corresponds to maximum entropy (maximum disorder). Systems spontaneously proceed toward an equilibrium state (maximum entropy).

(return to contents)

IV. REVERSIBLE AND IRREVERSIBLE CYCLES

The Kelvin-Plank version of the Second Law of Thermodynamics states that it is impossible for an engine working in a cycle to covert heat entirely into work. For example, heat (QH) could flow into a chamber and cause pressure in the chamber to increase enough to move a piston. But in a real process, the energy corresponding to the amount of work (W) performed in moving the piston will always be less than the energy corresponding to the amount of heat (QH) that caused the gas to expand. Some heat (QC) will have been lost. The amount of entropy increase in the universe (ΔSu) corresponds to QC divided by the temperature (TC) of the lowest temperature heat sink to which the lost heat flowed. (This model of quantifying irreversibility is not so easily applied to the processes described in the previous section.)

The most efficient processes will be those that occur with a minimal increase in entropy. Thus, imaginary reversible processes (cycles) that involve no entropy production provide a quantifiable theoretical standard for optimal performance that can be compared to real processes (real cycles, real engines).

What is an example of a reversible process? There are no reversible processes in the real world, but physicists can fantasize a reversible process in the same way as they can fantasize a block sliding on a frictionless surface forever with an unchanging velocity.

A Carnot Cycle is an imaginary reversible process. A Carnot Cycle consists of two adiabatic work processes and two isothermal heat transfer processes.

An adiabatic work process is a work process that occurs without any heat transfer. There can be no entropy change in an adiabatic process because entropy change is always associated with a heat transfer. Therefore, adiabatic work processes are reversible and involve no entropy change. In a graph of Pressure versus Volume (PV graph), pressure drops most steeply as an adiabatic expansion begins, and drops increasingly less steeply as the expansion proceeds. Between two points on a PV graph, there can be many different possible adiabatic expansion curves, but all those curves will have the same entropy on each point of the curve.

An isothermal heat transfer process is a heat transfer process that occurs without any temperature change. In the Carnot Cycle, one isothermal heat transfer occurs with a hot reservoir and the other isothermal heat transfer occurs with a cold reservoir. The entropy increase (ΔSH) that occurs with the isothermal heat transfer (QH) at the hot reservoir (TH) is exactly equal to the entropy decrease (ΔSC) that occurs with the isothermal heat transfer (QC) at the cold reservoir (TC). Thus, the entropy increase from the hot reservoir and the entropy decrease to the cold reservoir results in no net entropy change for the Carnot Cycle.

Carnot Power Cycle
Carnot Cycle: T versus S graph

      Mathematically:     ΔSH = +|QH|/TH  (hot reservoir)

                          and     ΔSC = −|QC|/TC  (cold reservoir)

Note that negative change in entropy means that entropy is decreasing. Entropy is never negative.

In sum, the adiabatic work processes are reversible and involve no entropy change. The isothermal entropy increase and decrease associated with the two heat transfers are equal and opposite — canceling each other out to give no net entropy change. So the final entropy change after completion of one Carnot Cycle is zero, and the Carnot Cycle is reversible. Four processes in the Cycle occur which cause a change of state between four states (1,2,3,4), which can be represented as points on a PV (Pressure-Volume) graph. The four processes in the Cycle can be graphed as curves which depict the pressure and specific volume changes as each process proceeds between states. The Cycle can also be graphed in a similar manner on a TS (Temperature-Entropy) graph, which results in vertical or horizontal straight lines for each process connecting two states.

A Carnot Engine is an imaginary reversible engine that is based on the Carnot Cycle. Gas expands and is compressed in a chamber adjoining a frictionless piston. An equal amount of work is done by and on the piston. An equal amount of heat is transferred in as is transferred out of the chamber.

Carnot Engine
Process 1 → 2Process 2 → 3 Process 3 → 4Process 4 → 1 Carnot Power Cycle
[ Process 1 → 2 ] [ Process 2 → 3 ] [ Process 3 → 4 ] [ Process 4 → 1 ] [ Carnot Power Cycle ]

The final process returns the Carnot Engine to the original position of the piston and the original temperature, completing the cycle.

Process 2 → 3 and process 4 → 1 might seem especially mysterious insofar as these steps involve a heat flow without a temperature difference. How can heat flow if there is no temperature difference to drive the process? What is the motive force? Actually, heat can flow from one body to another at the same temperature if one of the bodies is undergoing a phase change (gas to liquid or liquid to solid) because heat is absorbed or released during a phase change, even though temperature does not change. Conceivably, the chamber at temperature TH could be in contact with a vast thermal reservoir undergoing a gas to liquid phase transition (releasing heat into the chamber at TH). The chamber at temperature TC could be in contact with a vast thermal reservoir undergoing a solid to liquid phase transition (absorbing heat from the chamber at TC). But the Carnot Engine is a fantasy in which phase transitions are not in the descriptions usually given.

In process 4 → 1, for example, an infinitesimal amount of compression would tend to cause an infinitesimal rise in temperature. But because the compression chamber is not insulated, but is imagined to be in thermal contact with a huge (infinite) reservoir at TC, the infinitesimal temperature rise in the compression chamber causes an infinitesimal flow of heat (energy) out of the compression chamber, thereby keeping the temperature in the compression chamber at TC. An infinite number of such infinitesimal steps result in a finite isothermal compression in which temperature remains at TC. A problem with this is that an infinitesimal heat flow at an infinitesimal temperature difference should correspond to an infinitesimal irreversibility. If an infinite number of infinitesimal compressions can sum to a finite compression, then an infinite number of infinitesimal irreversibilities should sum to a finite reversibility. But the Carnot Engine is a thought experiment, and that particular unhappy thought is excluded.

(return to contents)

V. SIMPLE EXAMPLES OF ENTROPY

Irreversibility of a process in a limited system need not mean absolute irreversibility of that process in the context of the universe (or wider system). If a hot branding iron is removed from a fire and placed on a rock at normal atmospheric temperature, the branding iron will cool as heat dissipatges into the environment from the branding iron. For the limited system of the branding iron, the rock and the air in the vicinity of the branding iron, an irreversible process has occurred. But the cooling of the branding iron is not irreversible in the sense that the branding iron can easily be put back in the fire. If the branding iron is heated to the same temperature it had when it was first removed from the fire, the entropy the branding iron lost during the cooling will be restored to the level of the entropy the iron had when it was first removed from the fire. An irreversible process is still occurring with the fire, but that is outside of the system that is defined to only include the branding iron, the rock and the air in the vicinity of the branding iron. Entropy of the smaller system decreases only as a result of entropy of the larger system increasing.

Similarly, an apple falling from an apple tree represents and irreversible process within the limited system of the apple and the tree. But if a person is added to the system, the person can lift the apple from the ground and place it on a branch of the apple tree. Fat and glucose in the muscle of the person is consumed in an irreversible process that nonetheless provides energy to reverse the process of the apple falling from the tree. Again, entropy of the smaller system decreases only as a result of entropy of the larger system increasing.

A vessel of water placed in a very cold environment will freeze. In forming ice, water molecules organize themselves into a crystal lattice. Although the disorder of the water molecules has decreased, the entropy of the universe as a whole has increased because of the dissipation of heat from the vessel to the cold environment.

(return to contents)

  [GO TO BEN BEST'S HOME PAGE] HOME PAGE