Entropies and Equilibria of Many-Particle Systems: An Essay on Recent Research

Abstract.This essay is intended to present a fruitful collaboration which has developed among a group of people whose names are listed above: entropy methods have proved over the last years to be an efficient tool for the understanding of the qualitative properties of physically sound models, for accurate numerics and for a more mathematical understanding of nonlinear PDEs. The goal of this essay is to sketch the historical development of the concept of entropy in connection with PDEs of continuum mechanics, to present recent results which have been obtained by the members of the group and to emphasize the most striking achievements of this research. The presentation is by no way an exhaustive review of the methods and results involving the entropy, not even in the field of PDEs. Many other researchers in and outside Europe have contributed to the development of this field, including – but not only – in collaboration with some of the people of the group. However, it can be claimed that this group had a leading role over the recent years and this essay is intended to explain how this occurred.


Motivation and Applications
In the fast moving development of new technologies, ranging from microelectronics to space crafts, applied mathematics plays a substantial role in two fundamental steps of the realization process: modeling and numerical simulation. These two issues are closely connected, and vital to continuously improve the physical description of the relevant phenomena. In many novel applications the modeling involves the knowledge of the behavior of systems composed of a large number of interacting particles: electrons in micro-devices, ions in the 1 plasma of fusion reactors, atoms in a Bose-Einstein condensate, gas flowing over the wings of aircrafts, etc. One of the main features of such systems is their tendency (if left alone) to converge to an equilibrium configuration as time becomes large (usually this time is rather small viewed at our macroscopic scale). This is even part of our daily experience at a macroscopic scale: whenever we create a breeze in a room by opening a window, after shutting it again the gas will come to rest in a very short time.
Very often, there is a thermodynamical principle underlying this property of trend to equilibrium: as time progresses, interactions between particles lead to the increase of a distinguished functional called entropy (second law of thermodynamics, formulated at the end of the nineteenth century). Gibbs' principle asserts that the equilibrium distribution is the one which achieves the maximum entropy under the constraints imposed by the conservation laws. On the other hand, when such a large particle system is subjected to a continuous exterior stimulus (e.g. a force field) it exhibits an interplay between non-equilibrium and equilibrium regimes, which are of particular interest in computational physics and scientific computing: E.g., the simulation of the air flow around a space capsule at re-entry into the atmosphere requires to combine fluid mechanical equations for the equilibrium regions with finer so-called kinetic models for the non-equilibrium domains. Such kinetic equations provide a very accurate description of reality by modeling the collisional particle interactions in the position and momentum phase space. However, since they are often prohibitively expensive for numerical simulations of real three-dimensional problems, one frequently resorts to fluid dynamical equations, which only use position (and of course time) as independent variables. But the finer kinetic description is still crucial to understand the range of validity of the simplified models.
Since a system in (local) equilibrium depends only on a limited number of unknownscertain macroscopic observables-both the identification of equilibrium configurations and the speed of convergence towards equilibrium are of paramount importance for applications. Knowing this convergence rate is the only way to understand the relevant time scale for the equilibration process, which in turn is important for modeling and hence for guessing the feasibility of numerical simulations. Driven by technological needs in semiconductor development, plasma physics, and aeronautics the mathematical research of equilibration processes in the kinetic theory of rarefied gases recently experienced an unprecedented thrust.
The Boltzmann equation is the most famous kinetic model, both due to its important current applications and for historical reasons: For this model L. Boltzmann proved his celebrated H-theorem about the increase of the entropy 1 , which was the first analytical proof ever of the second principle of thermodynamics. Boltzmann's analysis can be adapted for numerous variants of the Boltzmann equation which are used in the description of collective dynamics in physics, engineering and biology, and in the description of the temporal transition from non-equilibrium to equilibrium in thermodynamical processes.
This report describes a collaborative research program, carried out by the research groups of the authors, partly in cooperation with other applied mathematicians. In part already known methods and partly the development of new analytical techniques to calculate the (optimal) speed of convergence towards equilibrium for both nonlinear kinetic equations and diffusion equations (obtained by scaling limit techniques from kinetic models) were necessary.
As a result of this recent research work, several unexpected links between the study of entropy production in nonlinear many-particle systems and the field of functional inequalities were unreaveled. Thus, the findings have important implications for related problems, mainly in the modeling of diffusion processes. These evolution processes have a variety of applications in physics and in industry. Among other things, our results are important for the validation of diffusion processes modeling heat transport, porous media flows, composite materials manufacturing, fast diffusion (Okuda-Dawson diffusion of plasma, solid state physics), thin films, liquid films, and moving contact lines, which are of basic importance in video tape production, cooling devices and biomedical devices.
A good part of the results of this joint work, which are collected at the end of this report, can be found in the corresponding preprints in the Preprint Archive, http://hyke.org.
2 From Boltzmann's H-Theorem to entropy methods for partial differential equations For a better understanding of the main ideas which are at the basis of our studies, we briefly recall previous applications of entropy to sciences. Since its introduction by Boltzmann more than one hundred years ago, the notion of entropy has been widely used in the study of dissipating systems, both by physicists and by mathematicians. It has also gained its way in engineering applications via the important concept of information developed by Shannon 2 , and in statistics after the invention of Fisher's information 3 .
Different applications of the concept of entropy have been introduced by Lax 4 for hyperbolic systems of conservation laws, and by DiPerna 5 in the framework of compensated compactness. Also, the well-celebrated theorem of DiPerna and P.-L. Lions 6 on global existence of renormalized solutions of the Boltzmann equation, has a proof that is strongly based on the use of Boltzmann's entropy. Another, not so well-known example is the use of the entropy in the famous work of Nash 7 on the regularity of the solution of certain diffusion processes. In all these works, the entropy is introduced for a purpose which is not its original one, but appears surprisingly useful.
Twenty years ago, Cercignani 8 had stated a conjecture about Boltzmann's entropy production which would imply an exponentially fast trend to equilibrium, but there was no clue of the proof of this conjecture. We remark once again that the verification of this conjecture is not merely an academic exercise, but on the contrary it has e.g. extremely important consequences in numerical simulation of re-entry problems, since it provides a theoretical justification of the use of fluid dynamics in the modeling. In 1989 Desvillettes 9 found an interesting lower bound for the entropy production relative to the Boltzmann equation. However, this lower bound was not precise enough to settle the Cercignani conjecture.
At the beginning of the nineties, Carlen and Carvalho 10 managed to establish a more precise lower bound. More importantly, they pointed out some connections of Cercignani's conjecture with the field of information theory and some functional inequalities (so-called logarithmic Sobolev inequality) which had become famous over the last decades for their appearing in various branches of modern mathematics 11,12 . An equivalent inequality had been shown a long time ago by Stam, working in information theory 13 .
The Gross logarithmic Sobolev inequality was introduced by Toscani 14 into the theory of kinetic Fokker-Planck diffusion processes. His work suggested a more physical way of understanding the Bakry-Emery approach (originally developed in probabilistic diffusion theory), in terms of the physical entropy production and production of entropy production. This was at the beginning of the year 1997. Main problems were still mostly open then, but the kinetic community in Europe was beginning to be widely interested in these topics.

Entropy methods
The last years have seen the rapid development of so-called entropy methods for the study of convergence to equilibrium in many-particle systems, and have led to the solution of most of the problems stated above.
The main idea behind entropy methods is to establish quantitative variants of the mechanism of increase of the entropy. This approach has the merit to stand upon a clear physical basis, and experience has shown its robustness and flexibility. The by now standard strategy can be summed up in four steps: • Step 1 : Identify the equilibrium state and the entropy functional, or more generally Lyapunov functional, E, which is associated to the equation. In many cases, the derivation of this Lyapunov functional is quite standard 15 . • Step 2 : Given the entropy (Lyapunov) functional E, attaining its maximum at the equilibrium f ∞ , the discrepancy between a distribution function f and the equilibrium f ∞ is in this approach measured by E[f |f ∞ ] = E(f ∞ ) − E(f ), thereafter called relative entropy. Often it is not advisable to try directly to prove that f (t) converges to f ∞ , but rather to show that E(f (t)) converges to E(f ∞ ), which will be called "convergence in relative entropy". • Step 3 : One considers as a main object of study the "entropy production functional" P , which is just the time-derivative of the entropy, P (f (t)) = dE(f (t))/dt.

•
Step 4 : One tries to quantify the following idea: if, at some given time t, f (t) is far from f ∞ , then E(f (t)) will increase notably at later times.
When trying to implement the preceding general principles, one can succeed in proving an entropy-entropy production inequality: this is a functional type inequality of the type P (f ) ≥ Θ(E[f |f ∞ ]), where H → Θ(H) is some continuous function, strictly positive when H > 0. The main idea is that "entropy production controls relative entropy". Such an inequality implies an immediate solution to the problem of trend to equilibrium. Indeed, it implies a closed differential equation on the relative entropy, of the . Usual techniques for ordinary differential equations (inequalities) imply that E(f (t)|f ∞ ) converges to 0 as t → ∞. Moreover (and this is the crucial point for getting relevant results), if the function Θ is known with enough details, then one can compute an explicit rate of convergence. Thus, Θ(H) = λH ("linear bound") implies exponential convergence, with speed given by the constant λ, while Θ(H) = KH 1+α ("polynomial bound") implies polynomial rate of convergence, the relative entropy decaying at least like t −1/α .

Results
The entropy method described in the previous section has been developed by the research groups of the authors from 1997 on, starting with classical equations in kinetic theory. From 1998 on, the partners collaborated in the TMR European Network Project "Asymptotic Methods in Kinetic Theory"(http://www.math.tu-berlin.de/˜tmr). Their studies led to the following results: 1) the discovery of simple and sharp lower bounds for the entropy production of classical equations of kinetic theory such as Boltzmann's (settling the Cercignani conjecture, as we shall explain); 2) a very complete understanding of the entropy production method, which was generalized in an unexpected way to apply to a large family of equations, including nonlinear diffusion processes, and also mean-field diffusive equations. Moreover, the method was linked to the field of mass transportation; this field is well-known in probability theory 16 , with applications to economy (going back to the Nobel-prize winner Kantorovich). The Bakry-Emery method was re-interpretated, in a very surprising way, in terms of mass transportation, which enabled its generalization to complicated models such as granular flows; 3) a new way of studying time-intermediate asymptotic for equations which do not possess a steady state, but have certain self-similarity properties (such cases occur very frequently in applications); 4) theoretical results, in the form of new functional inequalities, which came as a byproduct of this study, even if this was not the main goal of the researchers.

Entropy production functionals of Boltzmann and Landau.
The results of this section have immediate applications to the development of numerical codes in rarefied gas dynamics and plasma physics. Moreover, they close a longstanding open problem in kinetic theory. When grazing collisions in the Boltzmann operator prevail, the evolution of the density in a dilute plasma is driven by the Landau operator 17 . An almost complete study of the entropy production functional of the Landau equation was performed 18 . By using the logarithmic Sobolev inequality Desvillettes and Villani succeeded in proving Cercignani's conjecture for an over-Maxwellian cross section. Also "sharp" entropy production bounds for the Boltzmann collisional operator were obtained 19 . Physically speaking, it was proved that Cercignani's conjecture 20 is "almost" true in the following sense:

Linear Fokker-Planck type diffusion equations.
A detailed analysis of entropy production inequalities for linear Fokker-Planck type equations was performed 24 . For this model, one can define a whole family of relative entropy functionals, of the form H Φ (f |f ∞ ) = Φ (f /f ∞ ) f ∞ , whose "extremals" are given by Φ(h) = h log h − h + 1 at one end, Φ(h) = (h − 1) 2 /2 at the other hand. For each of these entropies one can perform a Bakry-Emery type argument to prove logarithmic Sobolev type inequalities; and they are all the stronger as the nonlinearity in the relative entropy is weaker (the strongest one corresponding to the h log h nonlinearity). The implications of convergence in relative entropy have been established in great generality 25 . Moreover the interplay between the Fokker-Planck equation and the field of differential inequalities 26 was systematically analyzed. This is a surprising situation where a physical system is used to prove a result in PDE's analysis !

4.3
Non-linear diffusions of second order with potential confinement.
Maybe the most important application of entropy production methods is linked to the largetime behavior of diffusion problems. For the first time entropy production for porous medium and fast diffusion equations with potential confinement was studied 27 . Subsequently 28 , nonlinear Fokker-Planck type equations and strongly coupled systems of degenerate nonlinear parabolic equations were analyzed. These equations arise in semiconductor theory, plasma physics, and stellar dynamics. The entropy production analysis was also extended to systems of degenerate parabolic equations arising in non-equilibrium thermodynamics, semiconductor energy-transport theory, and alloy solidification processes 29 . Additional applications include explicit rates of convergence towards the self-similar solution of both slow (porous medium) and fast diffusion equations. This is a famous problem which goes back to the seventies 30 . Very recently, nonlinear fast diffusion equations with finite equilibrium mass and infinite equilibrium entropies 31 were atudied. The analysis of second order diffusion equation with potential confinement has been recently generalized to models of dissipating granular flows 32 . Further applications 33 were also investigated.

Non-linear diffusions of fourth order.
Recent applications of the entropy production technique led to new and interesting results for fourth order diffusion equations. Mainly, two different physical models were treated. The first one is a nonlinear fourth-order parabolic equation arising in quantum semiconductor modeling. Exponential convergence to equilibrium in relative entropy in the one-dimensional case with no-flux boundary conditions 34 has been shown. Secondly, a model arising in the field of the tension-dominated motion of a spreading droplet (thin-film equation) 35 has been investigated.

Mean field models.
Many physical multi-particle models include force terms, in particular self-consistent effects described by mean field interactions. A very popular model from plasma physics is the Vlasov-Fokker-Planck equation, in which the collision operator is the Fokker-Planck operator, and the forces include both confinement and self-consistent interactions. If the interaction is of Coulomb type, then one speaks of the Vlasov-Poisson-Fokker-Planck model. Some of the authors introduced 36 the entropy production method for drift-diffusion-Poisson equations and generalized the method to systems of two particle species 37 . Also more general nonlinear drift-diffusion systems with Poisson coupling 38 were treated. These results are reminiscent of the techniques developed for nonlinear second-order diffusions without self-consistent interactions.

Spatially inhomogeneous models
Entropy production techniques have been applied to spatially inhomogeneous models in two situations. The algebraic decay towards the global equilibrium for the linear spatially in-homogeneous Fokker-Planck equation in a confining potential 39 . This is the first explicit estimate of convergence towards the equilibrium in the context of the spatially inhomogeneous kinetic theory. Subsequently, a large-time-analysis of spatially inhomogeneous discrete velocity models for kinetic initial-boundary value problems 40 . Exponential convergence to the equilibrium is obtained here via elaborate multiplier techniques, since the mass transport across the boundary is the main equilibrating effect.
4.7 Steady-state mean field models.
The existence of steady states for diffusive (elliptic) equations for systems with a (nonlinear) mean field term (electrostatic conductor and, respectively, gravitational stellar mean field models) was analyzed by bifurcation, variational and comparison techniques involving the entropy 41 . The (singular) charge neutral (small Debye lenght) limit was also analyzed: for typical parameters ranges the problem is reduced to solve an obstacle problem, showing the occurrence of a depletion (vacuum) region 41,42 .

Expected impacts
Because of its clear physical basis, its robustness and flexibility, the entropy production method has generated much interest in the PDE community. Our studies of trend to equilibrium are mainly useful for problems in the modeling and numerical simulation of systems in which a thermodynamical principle prevails. This concerns in particular the technological development of microelectronics and nano-machines, models for systems in fluid dynamics, granular materials, plasma physics, stellar media and many others. The methods give information on the time scale for relaxation to equilibrium, and therefore on the qualitative behavior of the models as well as on their validity. The obtained results yield a new mathematical method which can be applied to a very large array of models. It is expected to become a standard tool of the analysis of evolution PDEs. In particular, the entropy production method helps in the rigorous derivation and validation of models and in computational physics, in particular for the justification of numerical schemes. Entropy production techniques are by now essential in computer simulation and modeling of diffusion processes, to justify the numerical passage from the mesoscopic level of description typical of kinetic equations towards the macroscopic equations of fluid dynamics. More than one century after its first introduction, entropy reveal itself to be a flexible and robust instrument both for theory and applications.