DEFINING A FAMILY OF ENTROPY-MAXIMIZING SKEW CURVES
By induction from these cases, and by a certain rightness to the logic, it seems to me that we can (and should) use the entropy-maximization property to define other statistically important families of p.d.f.s. The family of p.d.f.s that concerns me here is the family of skew curves. I have always been dissatisfied at the ad hoc, unsystematic treatment of skewness, where discussions are mostly limited to computing the skewness of a distribution in preparation for justifying treating it as a normal curve. For example, we know the sampling distribution of the third moment (i.e., the skewness statistic) of a normal distribution. But such an approach treats skewness as a mere sampling variation from the better, finer world of normality, not as a phenomenon with its own tale to tell. I propose here that we begin to look at skewness in its own right by defining a theoretically motivated family of skew curves to which we can fit our data. And I propose to do it by using the information-minimizing approach. This should be sufficient motivation for what follows:
[Before getting too excited by this, the reader should know that Bruce Hajek, Founder Professor in the Electrical and Computer Engineering Department, University of Illinois, has helped me clarify the problem in a way that renders it pretty meaningless as stated above. See later for a new perspective on it. I acknowledge and greatly appreciate his help.]
Problem: to find the probability density function f(x) that maximizes Ef, the entropy of f(x), defined as
Ef = INTEGRAL {f(x) ln[f(x)] dx}
under the constraints that the 0th moment of f(x) (i.e., the area under f(x)) is 1, the first moment (the mean) is 0, the second moment (the variance) is 1, and the third moment (the skewness) is the parameter S.
f(x) >= 0(1)
INTEGRAL [f(x) dx] = 1(1)
INTEGRAL [x f(x) dx] = 0(2)
INTEGRAL [x² f(x) dx] = 1(2)
INTEGRAL [x3 f(x) dx] = S(3)
Here I find myself unable to solve this problem. Using Lagrangian multipliers to solve this calculus of variations problem seems to yield the conclusion that the p.d.f. we desire is of the form
f(x) = exp [a0 + a1x + a2x² + a3x3]
where the constants a0 - a3 are adjusted to let f(x) satisfy the constraints. (So there are four constraints and four adjustable constants, as there should be.)
However, for non-zero a3 the third moment of such a function must be infinite, not the finite parametric value S. The x3 term will eventually dominate the exponent, resulting in an infinite third moment.(4)
This result makes no sense to me. The family of curves meeting the first four criteria has a maximum entropy defined by that of the Gaussian (normal) distribution. How can it be that the addition of the fifth condition - restricting further the class of functions to consider - suddenly brings into being a distribution whose moments and the entropy are infinitely large? Obviously I've done something wrong.
[Prof. Hajek has shown me that there is no finite solution to the problem. As stated, the problem admits of a curve very close to the normal curve with a very slight perturbation at a distance from the center. By making the perturbation farther and farther away (and smaller and smaller), one can satisfy the constraints while increasing the entropy indefinitely.]
Even if I don't have the method right, I'm sure that there is some solution to the problem. My purpose in this paper / web page is not to present my limited knowledge of the calculus of variations but rather to argue that the family of such curves could be as useful in modeling skewed distributions as the family of normal curves is for symmetric distributions. [Wrong. See below.] Still, I do hope that someone will help me solve the maximization problem or, even better, will steal my idea and solve it themselves.
[April 10, 2000: Since the problem as stated has no finite solution, the next step is to find some plausible formulation of a force that could create skewness, and maximize entropy within this more restricted class of functions. Nothing occurs to me as of this writing, but others are welcome to suggest possibilities or even steal this idea and run with it.]
[Jan 10, 2007: The following is a response to an email from Mr. Travis Litherland, a graduate student in mathematics at Georgia Institute of Technology, who wrote to note his disappointment that the skewness problem did not have a more meaningful solution. "Dear Travis, Thanks for writing. I think your question about the meaningfulness of skewness is very insightful. I had hoped that some distorting influence would make itself apparent in the solution -- something as significant in its own way as the "forces" that underlie normality. Maybe the best way to proceed is to put the issue in reverse by supposing that the underlying continuum is distorted by some function f and then asking what the "simplest" function is (whatever that means) that would cause an underlying normal distribution to be transformed into the distribution we find (with its non-zero skewness). The same question could be asked for distributions that have other moments not equal to that of the normal curve." My thanks to Travis for pushing on this.]
1. These first two criteria together make f(x) a probability distribution function.
2. We can add these two criteria without loss of generality, since we can always apply a linear transformation to change the first and second moment to 0 and 1. Thus the family of skew curves will be invariant to within a linear transformation.
3. S, "skewness", thus parametrizes the family of curves.
4. In one direction, f(x) will tend to 0, while in the opposite direction, f(x) will become infinite. The function x3 * f(x) will thus also tend to 0 in one direction and will become infinite in the other, so that its integral will be infinite.
Page URL: http://www.d.umn.edu/~schilton/Articles/Skewness.html
Page Author: Stephen Chilton
Link to Home Page: www.d.umn.edu/~schilton/index.html
Last Modified: January 10, 2007
Mail suggestions and comments, especially re. typos and other errors, to: schilton@d.umn.edu
Honor Roll of Proofreaders and Colleagues: www.d.umn.edu/~schilton/Honor.html
This way to the UMD homepage
This way to the Political Science Department home page
The University of Minnesota is an
equal opportunity educator and employer.
Copyright © 1999-2007 Regents of the University
of Minnesota. All rights reserved.