A Gaussian MIXTURE problem for everyone

[Reef Fish]

Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.

A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).

A MIXTURE of two Gaussian distributions with the same variance
and different means is NOT Gaussian. Whether the distribution of
the MIXTURE is bi-modal or unimodal depends on how far apart the
two means are!

If you MIX two Gaussian distributions with the same mean but
different variances, you'll get a unimodal non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of two Gaussian
distributions with same variance and different means will be
Platykurtic.

Leptokurtosis and Platykurtosis refer to the kurtosis of the
distribution compared to that of a Gaussian distribution.

The preceding three paragraphs are my recollections from the
textbook (by Chester I. Bliss) I had jettisoned into the trash
can 39 years ago. But what I recalled about the mixture of
two Gaussian distributions and how they relate to the kurtosis
of a Gaussian distribution seemed not to be found in other
textbooks of statistics. That is why I recalled that factoid
while the rest of the book was obsolete even 39 years ago.


Okay everyone. Make this a simple exercise (which was not
given in the book).

Let X and Y both be Gaussian distributions with variance 1,
and means at a and b respectively.

Characterize the unimodality and bi-modality of the EQUAL
mixture of those two distributions in terms of a and b.

-- Reef Fish Bob.

[Lou Thraki]

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).
A MIXTURE of two Gaussian distributions with the same variance
and different means is NOT Gaussian. Whether the distribution of
the MIXTURE is bi-modal or unimodal depends on how far apart the
two means are!
If you MIX two Gaussian distributions with the same mean but
different variances, you'll get a unimodal non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of two Gaussian
distributions with same variance and different means will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the kurtosis of the
distribution compared to that of a Gaussian distribution.
The preceding three paragraphs are my recollections from the
textbook (by Chester I. Bliss) I had jettisoned into the trash
can 39 years ago. But what I recalled about the mixture of
two Gaussian distributions and how they relate to the kurtosis
of a Gaussian distribution seemed not to be found in other
textbooks of statistics. That is why I recalled that factoid
while the rest of the book was obsolete even 39 years ago.
Okay everyone. Make this a simple exercise (which was not
given in the book).
Let X and Y both be Gaussian distributions with variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality of the EQUAL
mixture of those two distributions in terms of a and b.

Define c=b-a.
Find out when the derivative of the function

f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )

has a zero with a positive second derivative of f. Then it
is bi-modal, else unimodal.

f'(x) = -x*exp(-x^2/2) - (x-c)*exp(-(x-c)^2/2)

has a zero (the "symmetric" zero) at x=c/2.

f''(x) = ( x^2 - 1 )*exp(-x^2/2) + ( (x-c)^2 - 1 )*exp(-(x-c)^2/2)

and

f''(c/2) = ( c^2/2 - 2 )*exp(-c^2/8)

which is positive for c > 2.

[Lou Thraki]

Post by Lou Thraki

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).
A MIXTURE of two Gaussian distributions with the same variance
and different means is NOT Gaussian. Whether the distribution of
the MIXTURE is bi-modal or unimodal depends on how far apart the
two means are!
If you MIX two Gaussian distributions with the same mean but
different variances, you'll get a unimodal non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of two Gaussian
distributions with same variance and different means will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the kurtosis of the
distribution compared to that of a Gaussian distribution.
The preceding three paragraphs are my recollections from the
textbook (by Chester I. Bliss) I had jettisoned into the trash
can 39 years ago. But what I recalled about the mixture of
two Gaussian distributions and how they relate to the kurtosis
of a Gaussian distribution seemed not to be found in other
textbooks of statistics. That is why I recalled that factoid
while the rest of the book was obsolete even 39 years ago.
Okay everyone. Make this a simple exercise (which was not
given in the book).
Let X and Y both be Gaussian distributions with variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality of the EQUAL
mixture of those two distributions in terms of a and b.

Define c=b-a.
Find out when the derivative of the function
f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )
has a zero with a positive second derivative of f. Then it
is bi-modal, else unimodal.
f'(x) = -x*exp(-x^2/2) - (x-c)*exp(-(x-c)^2/2)
has a zero (the "symmetric" zero) at x=c/2.
f''(x) = ( x^2 - 1 )*exp(-x^2/2) + ( (x-c)^2 - 1 )*exp(-(x-c)^2/2)
and
f''(c/2) = ( c^2/2 - 2 )*exp(-c^2/8)
which is positive for c > 2.

For |c| > 2

[Reef Fish]

Post by Lou Thraki

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).
A MIXTURE of two Gaussian distributions with the same variance
and different means is NOT Gaussian. Whether the distribution of
the MIXTURE is bi-modal or unimodal depends on how far apart the
two means are!
If you MIX two Gaussian distributions with the same mean but
different variances, you'll get a unimodal non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of two Gaussian
distributions with same variance and different means will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the kurtosis of the
distribution compared to that of a Gaussian distribution.
The preceding three paragraphs are my recollections from the
textbook (by Chester I. Bliss) I had jettisoned into the trash
can 39 years ago. But what I recalled about the mixture of
two Gaussian distributions and how they relate to the kurtosis
of a Gaussian distribution seemed not to be found in other
textbooks of statistics. That is why I recalled that factoid
while the rest of the book was obsolete even 39 years ago.
Okay everyone. Make this a simple exercise (which was not
given in the book).
Let X and Y both be Gaussian distributions with variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality of the EQUAL
mixture of those two distributions in terms of a and b.

Define c=b-a.
Find out when the derivative of the function
f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )

I am using some terminology in my reply to illywacker's post
to explain what you did and why it worked.

Your f(x) is proportional to the pdf of the convex combination
of two Gaussian distributions N(a,1) and N(b,1), without
explicitly noting that this form works ONLY if p and (1-p)
is the same (.5) and f(x) is the pdf of the mixture, missing
the proportionality constand .5/sqrt(2*pi).

Since you're finding the zeros of the derivative of f(x), the
proportionality constant doesn't matter.

Post by Lou Thraki
has a zero with a positive second derivative of f. Then it
is bi-modal, else unimodal.

Not sure if the characterization is correct in general; it's not
necessary since one can easily decide on the modality by
looking (graphically) at the pdf of the mixture itself.

Post by Lou Thraki
f'(x) = -x*exp(-x^2/2) - (x-c)*exp(-(x-c)^2/2)
has a zero (the "symmetric" zero) at x=c/2.
f''(x) = ( x^2 - 1 )*exp(-x^2/2) + ( (x-c)^2 - 1 )*exp(-(x-c)^2/2)
and
f''(c/2) = ( c^2/2 - 2 )*exp(-c^2/8)
which is positive for c > 2.

You corrected the "c>2" to "!c!>2" yourself.

Well done! What that says is that if you take a 1:1 mix of
N(a,1) and N(b,1), it is NOT bimodal (as most people suspect)
at a and b, but UNIMODAL at (b-a)/2.

It is BIMODAL if and only if the means of the two Gaussian
distributions are GREATER than 2 units apart (the standard
deviation of each of the component is 1).

Based on this and other threads I've seen in sci.stat.math, I
am of the opinion that the readership is much stronger in
mathematical statistics matters than applied statistics.

For the applied folks, some descriptive comments might
help see what's going on.

If you plot the equal MIXTURE of N(0,1) and N(2,1)
<means exactly two 2 units apart), that distribution
still has only ONE MODE, at 1. I can't show any plots
here, but if you plot the mixture pdf, you'll see a
symmetric unimodal distribution whose .probability is
concentrated between -3 and 5, (about .999) and
the pdf has a very FLAT top between .5 and 1.5.

The mnemonic for the term platykurtic is the word
"plateau" of the mixture distribution.


X FX X FX X FX X FX
**** ****** **** ****** **** ****** **** ******
.5 .24079 .71 .24183 .91 .24197 1.11 .24197
.51 .24088 .72 .24185 .92 .24197 1.12 .24197
.52 .24096 .73 .24187 .93 .24197 1.13 .24196
.53 .24104 .74 .24188 .94 .24197 1.14 .24196
.54 .24112 .75 .24189 .95 .24197 1.15 .24196
.55 .24119 .76 .2419 .96 .24197 1.16 .24196
.56 .24125 .77 .24192 .97 .24197 1.17 .24195
.57 .24131 .78 .24192 .98 .24197 1.18 .24195
.58 .24137 .79 .24193 .99 .24197 1.19 .24194
.59 .24143 .8 .24194 1 .24197 1.2 .24194
.6 .24148 .81 .24194 1.01 .24197 1.21 .24193
.61 .24152 .82 .24195 1.02 .24197 1.22 .24192
.62 .24157 .83 .24195 1.03 .24197 1.23 .24192
.63 .24161 .84 .24196 1.04 .24197 1.24 .2419
.64 .24164 .85 .24196 1.05 .24197 1.25 .24189
.65 .24168 .86 .24196 1.06 .24197 1.26 .24188
.66 .24171 .87 .24196 1.07 .24197 1.27 .24187
.67 .24174 .88 .24197 1.08 .24197 1.28 .24185
.68 .24176 .89 .24197 1.09 .24197 1.29 .24183
.69 .24179 .9 .24197 1.1 .24197 1.3 .24181
.7 .24181


The max value of the N(0,1) pdf occurs at 0
1/SQRT(2*ACOS(-1)) = .39894

Notice that the maximum value of the mixture pdf is .24197
and is virtually flat at that value between .88 and 1.12.

Beteen .5 and 1.5, the pdf value drops only slightly, from
.242 to .240.

So there is the seldom talked about story of the convex
combination (or mixture) of two GAUSSIAN distributions,
illustrated on the simplest case of equal weight and same
variance.

I think an elementary paper on the GENERAL case of the
convex combination of two Gaussian distributions and the
characterization of its modality may be publishable in some
statistical journal, because I don't recall ever seen anything
that discussed this sort of thing except in the Bliss book
39 years ago, without any generality at all.

-- Reef Fish Bob.

[illywhacker]

Dear Reefer,

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).
A MIXTURE of two Gaussian distributions with the same variance
and different means is NOT Gaussian. Whether the distribution of
the MIXTURE is bi-modal or unimodal depends on how far apart the
two means are!

I do think you need to distinguish between linear combinations of
random variables, and linear (or rather convex) combinations of
distributions (i.e. measures). A linear combination of two random
variables, each of which is Gaussian-distributed, will be
Gaussian-distributed:

P(z) = \int dx dy \delta(z, ax + by) G(x; mu, sigma) G(y; mu', sigma')
.

On the other hand, a convex combination of two Gaussian measures is by
definition a mixture of Gaussians:

P(z) = p G(z; mu, sigma) + (1 - p) G(z; mu', sigma') .

illywhacker;

[Reef Fish]

Post by illywhacker
Dear Reefer,

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).
A MIXTURE of two Gaussian distributions with the same variance
and different means is NOT Gaussian. Whether the distribution of
the MIXTURE is bi-modal or unimodal depends on how far apart the
two means are!

I do think you need to distinguish between linear combinations of
random variables, and linear (or rather convex) combinations of
distributions (i.e. measures).

Point well taken, expecially about the usage of the term "convex
combination" (mathematically well-defined) instead of the
well-understood, but less explicitly defined "mixture" (statistics)
of two distributions (OR random variables for that matter).

For statisticians, if you say you have two random variables X
and Y and you want to take a MIXTURE of those two r.v.,
what could you possibly mean? That you sample 100p%
from the pdf (discrete or continuous) of X and 100(1-p)% from
the pdf of Y, and the resulting distribution will be the convex
combination of the pdfs.

Post by illywhacker
A linear combination of two random
variables, each of which is Gaussian-distributed, will be

which was my position in a different thread. You need go to
THAT thread which I had long abandoned and yielded to the
"surrealists" of pathological examples. <Which was the
background of my parenthetical remark about "linear
combinations".

Post by illywhacker
P(z) = \int dx dy \delta(z, ax + by) G(x; mu, sigma) G(y; mu', sigma')
.
On the other hand, a convex combination of two Gaussian measures is by
P(z) = p G(z; mu, sigma) + (1 - p) G(z; mu', sigma') .
illywhacker;

I agree that's a better way to put it, once one takes the definition of
a "convex combination" as combination with weights p and (1-p),
or more generally, positive weights of a1, a2, ... ,ak, which sum to
1.

Then for any r.v. X with pdf f(x) and r.v. Y with pdf g(y), the
"mixture" or "convex combination" will have pdf

p f(x) + (1-p)g(x) which is a pdf which itself is a linear combination
of two density functions.

Perhaps THAT's the real reason for the confusion among many
on the meaning of MIXTURE of two distributions (or r.v.).

Thanks for the GOOD POINT to prefer the mathematician's or
mathematical-statistician's usage of "convex combination"
over the usage of "mixture" of distributions that is more
commonly used by applied statisticians and computer
scientists, especially the latter, because one can easily
SIMULATE a sample from a mixture distribution without
ever finding the pdf of the mixture-distribution itself.

-- Reef Fish Bob.

[Kevin E. Thorpe]

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).

Okay, I'm going to risk hysterical laughter aimed in my general
direction and ask a potentially very stupid question about this.

When I was following that other thread, I pulled out one of my
math-stat books to take a look (I don't have it with me at the
moment, so my recollection may be not 100% accurate).

Unless I misread or misunderstood, it seemed that there
were some non-pathological examples considered where
X and Y were CORRELATED normal but X + Y was not
normal.

So, is the statement "A linear combination of two
Gaussian distributions is Gaussian" a tautology or are
there additional assumptions implied?

--
Kevin E. Thorpe
Assistant Professor, Department of Public Health Sciences
Faculty of Medicine, University of Toronto

[illywhacker]

Dear Kevin,

Post by Kevin E. Thorpe
Okay, I'm going to risk hysterical laughter aimed in my general
direction and ask a potentially very stupid question about this.

<snip>

Post by Kevin E. Thorpe
Unless I misread or misunderstood, it seemed that there
were some non-pathological examples considered where
X and Y were CORRELATED normal but X + Y was not
normal.
So, is the statement "A linear combination of two
Gaussian distributions is Gaussian" a tautology or are
there additional assumptions implied?

I hope the following avoids all semantic mishaps.

A linear (or rather convex) combination of Gaussian *distributions*
(i.e. measures), otherwise known as a mixture, is not Gaussian except
in the case that the means and variances are the same, when the
operation is trivial.

The distribution of a linear combination of *random variables*, each of
which has a Gaussian marginal distribution, may or may not be Gaussian
depending on the form of the joint distribution. If the joint
distribution is Gaussian, even if the variables are
dependent/correlated, then so is the distribution of the linear
combination. If the joint distribution is not Gaussian (for example,
one could construct such distributions from the Gaussian marginals
using copulas/copoulae), then the distribution of the linear
combination need not be Gaussian, although I suppose it might be.

If the marginals of *all* linear combinations are Gaussian, then I
believe the joint distribution must be Gaussian.

illywhacker;

[Reef Fish]

Post by Kevin E. Thorpe

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).

Okay, I'm going to risk hysterical laughter aimed in my general
direction and ask a potentially very stupid question about this.

Nah. Only Afonso and Ulrich are capable of generating
hysterical laughters in this group. ;)

Post by Kevin E. Thorpe
When I was following that other thread, I pulled out one of my
math-stat books to take a look (I don't have it with me at the
moment, so my recollection may be not 100% accurate).
Unless I misread or misunderstood, it seemed that there
were some non-pathological examples considered where
X and Y were CORRELATED normal but X + Y was not
normal.

If X and Y are correlated, all you need to know are their
covariances.

Post by Kevin E. Thorpe
So, is the statement "A linear combination of two
Gaussian distributions is Gaussian" a tautology or are
there additional assumptions implied?

Nothing tautological about it. STANDARD result in statistics!
Standard Mulrivariate Normal result too:

http://en.wikipedia.org/wiki/Multivariate_normal_distribution

Wiki> Linear transformation
Wiki> If Y = B X is a linear transformation of X where B
is an mxp matrix then Y has a multivariate normal
distribution with expected value B<mu> and variance
B<SIGMA>B'.

< X is multivariate normal MV( mu, SIGMA ) >
(Each row of B is a linear combination of the univariate normals)

The pothologists and surrealists are in the other thread, still
ongoing, I think. :-)

-- Reef Fish Bob.

[Kevin E. Thorpe]

Post by Reef Fish

Post by Kevin E. Thorpe

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).

<SNIP>

Post by Reef Fish

Post by Kevin E. Thorpe
When I was following that other thread, I pulled out one of my
math-stat books to take a look (I don't have it with me at the
moment, so my recollection may be not 100% accurate).
Unless I misread or misunderstood, it seemed that there
were some non-pathological examples considered where
X and Y were CORRELATED normal but X + Y was not
normal.

If X and Y are correlated, all you need to know are their
covariances.

As I feared, my memory was not 100% accurate. The example
given was actually the same example posted by Stephen J. Herschkorn
in his follow-up.

Post by Reef Fish

Post by Kevin E. Thorpe
So, is the statement "A linear combination of two
Gaussian distributions is Gaussian" a tautology or are
there additional assumptions implied?

Nothing tautological about it. STANDARD result in statistics!
http://en.wikipedia.org/wiki/Multivariate_normal_distribution
Wiki> Linear transformation
Wiki> If Y = B X is a linear transformation of X where B
is an mxp matrix then Y has a multivariate normal
distribution with expected value B<mu> and variance
B<SIGMA>B'.
< X is multivariate normal MV( mu, SIGMA ) >
(Each row of B is a linear combination of the univariate normals)

Looking further in my text I find a theorem.

Let (X1, ..., Xp) be a p-variate random variable.
(X1, ..., Xp) is p-variate normal iff every linear combination
of the Xi's is normally distributed.

illywhacker wrote:

illywhacker> If the marginals of *all* linear combinations are
Gaussian, then I
illywhacker> believe the joint distribution must be Gaussian.

This follows from the theorem.

Stephen J. Herschkorn wrote:

SJH> If X and Y are *jointly* normal, then any linear combination
of X
SJH> and Y is normally distributed.

Also an application of the theorem.

This looks like pretty good agreement to me.

The implication of the theorem is that X1, ..., Xp could
be marginally normal, but if (X1, ..., Xp) is NOT jointly
normal, then there exists some linear combination of
the Xi's that is NOT normally distributed. Ah, the power
of "if and only if."

Correct?

--
Kevin E. Thorpe
Assistant Professor, Department of Public Health Sciences
Faculty of Medicine, University of Toronto

[Reef Fish]

Post by Kevin E. Thorpe

Post by Reef Fish

Post by Kevin E. Thorpe

Post by Reef Fish
Several posters in the Gaussian mixture problem seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is Gaussian.
(Discounting the pathological examples discussed in another thread).

<SNIP>

Post by Reef Fish

Post by Kevin E. Thorpe
When I was following that other thread, I pulled out one of my
math-stat books to take a look (I don't have it with me at the
moment, so my recollection may be not 100% accurate).
Unless I misread or misunderstood, it seemed that there
were some non-pathological examples considered where
X and Y were CORRELATED normal but X + Y was not
normal.

If X and Y are correlated, all you need to know are their
covariances.

As I feared, my memory was not 100% accurate. The example
given was actually the same example posted by Stephen J. Herschkorn
in his follow-up.

Which I characterized as contrived, pathological, and surrealistic.

Post by Kevin E. Thorpe

Post by Reef Fish

Post by Kevin E. Thorpe
So, is the statement "A linear combination of two
Gaussian distributions is Gaussian" a tautology or are
there additional assumptions implied?

Nothing tautological about it. STANDARD result in statistics!
http://en.wikipedia.org/wiki/Multivariate_normal_distribution
Wiki> Linear transformation
Wiki> If Y = B X is a linear transformation of X where B
is an mxp matrix then Y has a multivariate normal
distribution with expected value B<mu> and variance
B<SIGMA>B'.
< X is multivariate normal MV( mu, SIGMA ) >
(Each row of B is a linear combination of the univariate normals)

Looking further in my text I find a theorem.
Let (X1, ..., Xp) be a p-variate random variable.
(X1, ..., Xp) is p-variate normal iff every linear combination
of the Xi's is normally distributed.

That one is well-known.

Feller's Volume 2. e.g. That it takes ALL (uncountably infinite)
marginals
to be normal before you even have a bivariate normal.

Post by Kevin E. Thorpe
illywhacker> If the marginals of *all* linear combinations are
Gaussian, then I
illywhacker> believe the joint distribution must be Gaussian.
This follows from the theorem.
SJH> If X and Y are *jointly* normal, then any linear combination
of X
SJH> and Y is normally distributed.
Also an application of the theorem.

Those are statements about MULTIVARIATE normal or JOINTLY
NORMAL distribtions.

A linear combination, in the pathological case, is about ONE single
linear combination. Has NOTHING to do with the multivariate normal.

Post by Kevin E. Thorpe
The implication of the theorem is that X1, ..., Xp could
be marginally normal, but if (X1, ..., Xp) is NOT jointly
normal, then there exists some linear combination of
the Xi's that is NOT normally distributed. Ah, the power
of "if and only if."

Sounds more like logical gibberish to me. :-)

An IFF statement would be something like a multivariate
distribution is Multivariate Normal If and only if all linear
combination of its marginals are normal.

Post by Kevin E. Thorpe
Correct?'

My if and only if statement about the multivariate normal.

I couldn't quite parse YOUR statement in LOGIC as
a valid categorical proposition or a categorical syllogism.

In any case, my concession to Stephen's pathological
example was on the basis it WAS pathological and it
pertained to only ONE single linear combination of two
normal distribtions. No relation to any of the standard
statistical results about normal distributions that are
marginals of higher dimensional normals.

-- Reef Fish Bob.

[illywhacker]

Dear Reefer,

Post by Reef Fish
In any case, my concession to Stephen's pathological
example was on the basis it WAS pathological and it
pertained to only ONE single linear combination of two
normal distribtions.

There are many examples though; see my July 20 post (#17) below.

illywhacker;

[illywhacker]

Post by illywhacker
There are many examples though; see my July 20 post (#17) below.

Let me try that again, since obviously 'number in list' is not a useful
identifier. See my post

http://groups.google.com/group/sci.stat.math/msg/3ed55dd6a147329b

below.

illywhacker;

[Kevin E. Thorpe]

Dear Reef Fish Bob,

In my reply below I will snip much and rearrange some of the
prowse for ease of reference with no further indication of
the edits.

I have read your reply a few times now to make sure I'm
clear on what you wrote. Without intending to be patronizing
(or suggesting you do otherwise), I would respectfully
request you do the same as you consider my comments.

KET> Looking further in my text I find a theorem.
KET>
KET> Let (X1, ..., Xp) be a p-variate random variable.
KET> (X1, ..., Xp) is p-variate normal iff every linear combination
KET> of the Xi's is normally distributed.

RF> An IFF statement would be something like a multivariate
RF> distribution is Multivariate Normal If and only if all linear
RF> combination of its marginals are normal.

I read those two statements as saying essentially the same thing.

KET> The implication of the theorem is that X1, ..., Xp could
KET> be marginally normal, but if (X1, ..., Xp) is NOT jointly
KET> normal, then there exists some linear combination of
KET> the Xi's that is NOT normally distributed. Ah, the power
KET> of "if and only if."

RF> Sounds more like logical gibberish to me. :-)

It made sense to me, but I'm biased since I wrote it. :-)

Since I don't understand where you feel the logic is faulty,
I will describe in pedantic detail how I arrived at the above
statement.

Let's use your statement: "... Multivariate Normal If and only
if all linear combination of its marginals are normal."

The two implications of the IFF are:

1. If all linear combinations of the marginals are normal,
then the joint distribution is multivariate normal.

2. If the joint distribution is multivariate normal, then all
linear combinations of the marginals are normal.

So, if the joint distribution is NOT multivariate normal
then NOT ALL linear combinations of the marginals are normal
since, if they were, by 1 the joint distribution MUST be
multivariate normal, contrary to the initial assumption in
this sentence.

That is essentially what I was saying above. Where is
the logical flaw?

Maybe my "if and only if" comment at the end of my earlier
statement threw you. It was a back-reference to the
theorem, not what I had written, since there was no IFF
in that at all.

--
Kevin E. Thorpe
Assistant Professor, Department of Public Health Sciences
Faculty of Medicine, University of Toronto

[Reef Fish]

Post by Kevin E. Thorpe
Dear Reef Fish Bob,
In my reply below I will snip much and rearrange some of the
prowse for ease of reference with no further indication of
the edits.
I have read your reply a few times now to make sure I'm
clear on what you wrote. Without intending to be patronizing
(or suggesting you do otherwise), I would respectfully
request you do the same as you consider my comments.

I always do, with or without any special request. :)

Post by Kevin E. Thorpe
KET> Looking further in my text I find a theorem.
KET>
KET> Let (X1, ..., Xp) be a p-variate random variable.
KET> (X1, ..., Xp) is p-variate normal iff every linear combination
KET> of the Xi's is normally distributed.
RF> An IFF statement would be something like a multivariate
RF> distribution is Multivariate Normal If and only if all linear
RF> combination of its marginals are normal.
I read those two statements as saying essentially the same thing.
KET> The implication of the theorem is that X1, ..., Xp could
KET> be marginally normal, but if (X1, ..., Xp) is NOT jointly
KET> normal, then there exists some linear combination of
KET> the Xi's that is NOT normally distributed. Ah, the power
KET> of "if and only if."
RF> Sounds more like logical gibberish to me. :-)
It made sense to me, but I'm biased since I wrote it. :-)
Since I don't understand where you feel the logic is faulty,
I will describe in pedantic detail how I arrived at the above
statement.

It had more to do with your statement while you were discussing
Stephen's argument about the linear combination of normal r.v.

Post by Kevin E. Thorpe
Let's use your statement: "... Multivariate Normal If and only
if all linear combination of its marginals are normal."
1. If all linear combinations of the marginals are normal,
then the joint distribution is multivariate normal.
2. If the joint distribution is multivariate normal, then all
linear combinations of the marginals are normal.

Then why not just say it that way? :-) And that's exactly
HOW those two facts about the Multivariate Normal are
stated (without exception)! I think even Wikipedia had
both of those statements, and

(2) was what I used to support the statement of linear combinations
in the "non-pathological case".

Post by Kevin E. Thorpe
So, if the joint distribution is NOT multivariate normal
then NOT ALL linear combinations of the marginals are normal

That's correct.

Post by Kevin E. Thorpe
That is essentially what I was saying above. Where is
the logical flaw?

None, given the way you clearify it now.

Post by Kevin E. Thorpe
Maybe my "if and only if" comment at the end of my earlier
statement threw you. It was a back-reference to the
theorem, not what I had written, since there was no IFF
in that at all.

What threw me was your apparent contrapositive statement,
and Stephen's pathological example, which had me looking for
some kind of "catagorical proposition" and "categorical syllogism"
(I DID mention those terms explicitly, in LOGIC) to try to tie
what you had written and its relevance to your discussion with
Stephen on HIS pathological example which was not one in
multivariate normal.

I am all clear now on YOUR point. Hope I clarified mine.

-- Reef Fish Bob.

[Kevin E. Thorpe]

Post by Reef Fish

Post by Kevin E. Thorpe
Dear Reef Fish Bob,
In my reply below I will snip much and rearrange some of the
prowse for ease of reference with no further indication of
the edits.
I have read your reply a few times now to make sure I'm
clear on what you wrote. Without intending to be patronizing
(or suggesting you do otherwise), I would respectfully
request you do the same as you consider my comments.

I always do, with or without any special request. :)

I'm glad, and as I indicated, I was not suggesting you do
otherwise.

Post by Reef Fish

Post by Kevin E. Thorpe
KET> Looking further in my text I find a theorem.
KET>
KET> Let (X1, ..., Xp) be a p-variate random variable.
KET> (X1, ..., Xp) is p-variate normal iff every linear combination
KET> of the Xi's is normally distributed.
RF> An IFF statement would be something like a multivariate
RF> distribution is Multivariate Normal If and only if all linear
RF> combination of its marginals are normal.
I read those two statements as saying essentially the same thing.
KET> The implication of the theorem is that X1, ..., Xp could
KET> be marginally normal, but if (X1, ..., Xp) is NOT jointly
KET> normal, then there exists some linear combination of
KET> the Xi's that is NOT normally distributed. Ah, the power
KET> of "if and only if."
RF> Sounds more like logical gibberish to me. :-)
It made sense to me, but I'm biased since I wrote it. :-)
Since I don't understand where you feel the logic is faulty,
I will describe in pedantic detail how I arrived at the above
statement.

It had more to do with your statement while you were discussing
Stephen's argument about the linear combination of normal r.v.

Post by Kevin E. Thorpe
Let's use your statement: "... Multivariate Normal If and only
if all linear combination of its marginals are normal."
1. If all linear combinations of the marginals are normal,
then the joint distribution is multivariate normal.
2. If the joint distribution is multivariate normal, then all
linear combinations of the marginals are normal.

Then why not just say it that way? :-) And that's exactly

It helps me to confirm my understanding to re-state
things in my own words.

Post by Reef Fish
HOW those two facts about the Multivariate Normal are
stated (without exception)! I think even Wikipedia had
both of those statements, and
(2) was what I used to support the statement of linear combinations
in the "non-pathological case".

Post by Kevin E. Thorpe
So, if the joint distribution is NOT multivariate normal
then NOT ALL linear combinations of the marginals are normal

That's correct.

Post by Kevin E. Thorpe
That is essentially what I was saying above. Where is
the logical flaw?

None, given the way you clearify it now.

Post by Kevin E. Thorpe
Maybe my "if and only if" comment at the end of my earlier
statement threw you. It was a back-reference to the
theorem, not what I had written, since there was no IFF
in that at all.

What threw me was your apparent contrapositive statement,
and Stephen's pathological example, which had me looking for
some kind of "catagorical proposition" and "categorical syllogism"
(I DID mention those terms explicitly, in LOGIC) to try to tie
what you had written and its relevance to your discussion with
Stephen on HIS pathological example which was not one in
multivariate normal.
I am all clear now on YOUR point. Hope I clarified mine.

I'm glad you are clear on my point. I am now clear on yours.

Thanks.

--
Kevin E. Thorpe
Assistant Professor, Department of Public Health Sciences
Faculty of Medicine, University of Toronto

[Stephen J. Herschkorn]

Post by Kevin E. Thorpe
The implication of the theorem is that X1, ..., Xp could
be marginally normal, but if (X1, ..., Xp) is NOT jointly
normal, then there exists some linear combination of
the Xi's that is NOT normally distributed. Ah, the power
of "if and only if."
Correct?

Correct. http://en.wikipedia.org/wiki/Multivariate_normal is a good
on-line reference.

[Reef Fish]

Post by Stephen J. Herschkorn

Post by Kevin E. Thorpe
The implication of the theorem is that X1, ..., Xp could
be marginally normal, but if (X1, ..., Xp) is NOT jointly
normal, then there exists some linear combination of
the Xi's that is NOT normally distributed. Ah, the power
of "if and only if."
Correct?

Correct. http://en.wikipedia.org/wiki/Multivariate_normal is a good
on-line reference.

No, Kevin's statement is NOT correct, as an iFF statement. I
explained that 2 MINUTES before your post. Understandable
that you didn't see it.

But I referenced the Wikipedia page to Kevin over 10 HOURS
before your post, together with an explanation. YOu should
have seen THAT!

Date: Thurs, Jul 20 2006 12:05 pm
Email: "Reef Fish" <***@yahoo.com>
Groups: sci.stat.math

http://en.wikipedia.org/wiki/Multivariate_normal_distribution

Wiki> Linear transformation
Wiki> If Y = B X is a linear transformation of X where B
is an mxp matrix then Y has a multivariate normal
distribution with expected value B<mu> and variance
B<SIGMA>B'.

< X is multivariate normal MV( mu, SIGMA ) >
(Each row of B is a linear combination of the univariate normals

===================


You should have stayed with your street-turoring on Manhattan. ;)
(see Herschkorn's sig)

-- Reef Fish Bob

Post by Stephen J. Herschkorn
--
Math Tutor on the Internet and in Central New Jersey and Manhattan

[Stephen J. Herschkorn]

Post by Kevin E. Thorpe
When I was following that other thread, I pulled out one of my
math-stat books to take a look (I don't have it with me at the
moment, so my recollection may be not 100% accurate).
Unless I misread or misunderstood, it seemed that there
were some non-pathological examples considered where
X and Y were CORRELATED normal but X + Y was not
normal.
So, is the statement "A linear combination of two
Gaussian distributions is Gaussian" a tautology or are
there additional assumptions implied?

This has been discussed much here recently.

If X and Y are *jointly* normal, then any linear combination of X
and Y is normally distributed. (You need to be a little careful
though: Do we consider 0 X + 0 Y, a random variable which is
identically zero, to be normally distributed? Also, do we allow
singular covariance matrices in our class of multivariate normal
distributions?)

However, it is possible for X and Y to each have marginal normal
distributions for X + Y to not be normally distributed. Here is the
standard example: Let X have standard normal distribution, P{I = 1}
= P{I = -1} = 1/2, I and X be independent, and Y = I X. Then
Y had standard normal distribution as well, but X + Y is not normally
distributed. (The distribution of X + Y is the mixture of a normal
and a point mass at 0.)

Or, to put in story form, Mr. and Mrs. Smith enter a casino and will
play the following game. The casino will generate a value which is
normally distributed with zero-mean; the value is Mrs. Smith's net
winnings. (A negative indicates a loss.) Mr. Smith will then flip a
fair coin. Heads, his net winnings are the same as Mrs. Smiths; tails,
his net winnings are the negative of Mrs. Smith's. Then Mr. and Mrs.
Smith's individual winnings are each normally distributed, but the
couple's total net winnings is not normally distributed.

Reef Fish will most likely start ranting now about how this response is
either incorrect or "SURREAL." You may safely ignore him. Indeed, you
will find no one else who supports his argument.

[illywhacker]

Post by Stephen J. Herschkorn
If X and Y are *jointly* normal, then any linear combination of X
and Y is normally distributed. (You need to be a little careful
though: Do we consider 0 X + 0 Y, a random variable which is
identically zero, to be normally distributed? Also, do we allow
singular covariance matrices in our class of multivariate normal
distributions?)

You do not need to be careful. You simply need to define what you mean
by the word 'Gaussian', a purely semantic point with no mathematical
substance. Perhaps we can call standard Gaussian distributions with
non-singular covariances 'Gaussian', but if we want to include singular
covariances in the definition, we will call the whole lot 'Bonobo
distributions'. Then we can state everything in terms of Bonobo
distributions and not have to worry.

illywhacker;

[Reef Fish]

Post by Stephen J. Herschkorn

Post by Kevin E. Thorpe
When I was following that other thread, I pulled out one of my
math-stat books to take a look (I don't have it with me at the
moment, so my recollection may be not 100% accurate).

This has been discussed much here recently.

< Stephen's rehash of what's in a DIFFERENT thread snipped >


Date: Thurs, Jul 20 2006 2:41 pm, Stephen wrote,

Post by Stephen J. Herschkorn
Reef Fish will most likely start ranting now about how this response is
either incorrect or "SURREAL." You may safely ignore him. Indeed, you
will find no one else who supports his argument.

LOL! Kevin know me (in this group) well enough not to need YOUR
advice about me.

No one else supports the "standard statistical" and "standard textbook"
cleim?


When you were sound asleep this morning:

Date: Thurs, Jul 20 2006 6:37 am
Email: "illywhacker" <***@free.fr>
Groups: sci.stat.math

A linear combination of two random variables, each of which
is Gaussian-distributed, will be Gaussian-distributed:

illywhacker;


2 Hours 31 minutes before your post, Stephen, I had already
given Kevin a NEW explanation based on the standard textbook
relation given in Wikipedia:

Date: Thurs, Jul 20 2006 12:05 pm
Email: "Reef Fish" <***@yahoo.com>
Groups: sci.stat.math

If X and Y are correlated, all you need to know are their
covariances.

STANDARD result in statistics!
Standard Mulrivariate Normal result too:

http://en.wikipedia.org/wiki/Multivariate_normal_distribution

Wiki> Linear transformation
Wiki> If Y = B X is a linear transformation of X where B
is an mxp matrix then Y has a multivariate normal
distribution with expected value B<mu> and variance
B<SIGMA>B'.

< X is multivariate normal MV( mu, SIGMA ) >
(Each row of B is a linear combination of the univariate normals)

End excerpt fro my 12:05 pm post.

Post by Stephen J. Herschkorn
--
Math Tutor on the Internet and in Central New Jersey and Manhattan

Why do you keep evading my question about your curious sig?
Do you consider Statistics "Math"? What subject do you tutor?
Where is "the internet".

Do you stand in the middle of the street in Central New Jersey
and Manhattan when you do your turoring?

-- Reef Fish Bob.

[illywhacker]

Post by Stephen J. Herschkorn
However, it is possible for X and Y to each have marginal normal
distributions for X + Y to not be normally distributed. Here is the
standard example: Let X have standard normal distribution, P{I = 1}
= P{I = -1} = 1/2, I and X be independent, and Y = I X. Then
Y had standard normal distribution as well, but X + Y is not normally
distributed. (The distribution of X + Y is the mixture of a normal
and a point mass at 0.)

Here is an infinite-dimensional family of examples. Let G be a 1d
Gaussian distribution with some mean and covariance. Let f: R -> R be a
function that integrates to zero, and suppose that

|f(x)| < G(x)

for all x. Clearly

|f(x)f(y)| < G(x)G(y) (*)

for all x and y.

Define

P(x, y) = G(x)G(y) + f(x)f(y) .

This is positive because of (*).

Integrating wrt x or y, gives P(x) = G(x) or P(y) = G(y), so both
marginals are Gaussian. Integrating again gives 1, so P(x, y) is a
probability distribution. It is clearly not Gaussian since f is
arbitrary apart from the constraints already imposed.

Now let z = x + y. Then

P(z) = \int dx dy \delta(z, x + y)(G(x)G(y) + f(x)f(y)

= \int dx G(x)G(z - x) + f(x)f(z - x) .

The first term is a convolution of Gaussians, hence Gaussian. The
second term need not be zero. A counter-example: let f be
antisymmetric, and evaluate it at z = 0. Then the second term is

\int dx f(x)f(-x) = -int dx f^{2}(x) < 0.

illywhacker;

[Jack Tomsky]

Post by Reef Fish
Several posters in the Gaussian mixture problem
seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is
Gaussian.
(Discounting the pathological examples discussed in
another thread).
A MIXTURE of two Gaussian distributions with the same
variance
and different means is NOT Gaussian. Whether the
distribution of
the MIXTURE is bi-modal or unimodal depends on how
far apart the
two means are!
If you MIX two Gaussian distributions with the same
mean but
different variances, you'll get a unimodal
non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of two
Gaussian
distributions with same variance and different means
will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the kurtosis
of the
distribution compared to that of a Gaussian
distribution.
The preceding three paragraphs are my recollections
from the
textbook (by Chester I. Bliss) I had jettisoned into
the trash
can 39 years ago. But what I recalled about the
mixture of
two Gaussian distributions and how they relate to the
kurtosis
of a Gaussian distribution seemed not to be found in
other
textbooks of statistics. That is why I recalled
that factoid
while the rest of the book was obsolete even 39 years
ago.
Okay everyone. Make this a simple exercise (which
was not
given in the book).
Let X and Y both be Gaussian distributions with
variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality of the
EQUAL
mixture of those two distributions in terms of a and
b.
-- Reef Fish Bob.

Without loss of generality, assume that a = 0 and b > 0. I used Excel's Solver and came up with the following results.

There is a constant m (around 2) such that if b <= m, then the mixture is unimodal with the mode being at b/2, that is, the midpoint of the means of the two Gaussians.

If b > m, then you have a bimodal distribution and as b gets larger, the two symmetrical modes diverge towards 0 and b.

All this could be subject to numerical errors to some extent. I used the Solver's default iteration settings.

Jack

[Reef Fish]

Post by Jack Tomsky

Post by Reef Fish
Several posters in the Gaussian mixture problem
seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian distributions is
Gaussian.
(Discounting the pathological examples discussed in
another thread).
A MIXTURE of two Gaussian distributions with the same
variance
and different means is NOT Gaussian. Whether the
distribution of
the MIXTURE is bi-modal or unimodal depends on how
far apart the
two means are!
If you MIX two Gaussian distributions with the same
mean but
different variances, you'll get a unimodal
non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of two
Gaussian
distributions with same variance and different means
will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the kurtosis
of the
distribution compared to that of a Gaussian
distribution.
The preceding three paragraphs are my recollections
from the
textbook (by Chester I. Bliss) I had jettisoned into
the trash
can 39 years ago. But what I recalled about the
mixture of
two Gaussian distributions and how they relate to the
kurtosis
of a Gaussian distribution seemed not to be found in
other
textbooks of statistics. That is why I recalled
that factoid
while the rest of the book was obsolete even 39 years
ago.
Okay everyone. Make this a simple exercise (which
was not
given in the book).
Let X and Y both be Gaussian distributions with
variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality of the
EQUAL
mixture of those two distributions in terms of a and
b.
-- Reef Fish Bob.

Without loss of generality, assume that a = 0 and b > 0.

Good start. That's essentially how Lou Thraki solved it
analytically, a couple of hours after I posed the problem, for the
simple special case:

Date: Thurs, Jul 20 2006 4:26 am
Email: "Lou Thraki" <***@yahoo.com>

LT> Define c=b-a.
LT> Find out when the derivative of the function
LT> f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )
LT> has a zero ... <snip>

Post by Jack Tomsky
I used Excel's Solver and came up with the following results.
There is a constant m (around 2) such that if b <= m, then the mixture is unimodal with the mode being at b/2, that is, the midpoint of the means of the two Gaussians.
If b > m, then you have a bimodal distribution and as b gets larger, the two symmetrical modes diverge towards 0 and b.
All this could be subject to numerical errors to some extent. I used the Solver's default iteration settings.
Jack

Jack, that's fairly impressive for the artificial intelligence of the
Excel solver. The solution for the special case is exactly correct!.

I gave an explanation of Thraki's analytic solution for the general
readership as well as some feel of the kurtosis (flatness) of the
unimodal mixture for m=2 (at your b=1).

I even suggested yesterday,

RF> I think an elementary paper on the GENERAL case of the
RF> convex combination of two Gaussian distributions and the
RF> characterization of its modality may be publishable in some
RF> statistical journal, because I don't recall ever seen anything
RF> that discussed this sort of thing except in the Bliss book
RF> 39 years ago, without any generality at all.

But the Excel story sounds interesting. Can you describe more
HOW you provide Excel the input and what does it actually did
to come up with the solution?

-- Reef Fish Bob.

[Jack Tomsky]

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
Several posters in the Gaussian mixture problem
seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian

distributions is

Post by Jack Tomsky

Post by Reef Fish
Gaussian.
(Discounting the pathological examples discussed

in

Post by Jack Tomsky

Post by Reef Fish
another thread).
A MIXTURE of two Gaussian distributions with the

same

Post by Jack Tomsky

Post by Reef Fish
variance
and different means is NOT Gaussian. Whether

the

Post by Jack Tomsky

Post by Reef Fish
distribution of
the MIXTURE is bi-modal or unimodal depends on

how

Post by Jack Tomsky

Post by Reef Fish
far apart the
two means are!
If you MIX two Gaussian distributions with the

same

Post by Jack Tomsky

Post by Reef Fish
mean but
different variances, you'll get a unimodal
non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of

two

Post by Jack Tomsky

Post by Reef Fish
Gaussian
distributions with same variance and different

means

Post by Jack Tomsky

Post by Reef Fish
will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the

kurtosis

Post by Jack Tomsky

Post by Reef Fish
of the
distribution compared to that of a Gaussian
distribution.
The preceding three paragraphs are my

recollections

Post by Jack Tomsky

Post by Reef Fish
from the
textbook (by Chester I. Bliss) I had jettisoned

into

Post by Jack Tomsky

Post by Reef Fish
the trash
can 39 years ago. But what I recalled about the
mixture of
two Gaussian distributions and how they relate to

the

Post by Jack Tomsky

Post by Reef Fish
kurtosis
of a Gaussian distribution seemed not to be found

in

Post by Jack Tomsky

Post by Reef Fish
other
textbooks of statistics. That is why I recalled
that factoid
while the rest of the book was obsolete even 39

years

Post by Jack Tomsky

Post by Reef Fish
ago.
Okay everyone. Make this a simple exercise

(which

Post by Jack Tomsky

Post by Reef Fish
was not
given in the book).
Let X and Y both be Gaussian distributions with
variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality of

the

Post by Jack Tomsky

Post by Reef Fish
EQUAL
mixture of those two distributions in terms of a

and

Post by Jack Tomsky

Post by Reef Fish
b.
-- Reef Fish Bob.

Without loss of generality, assume that a = 0 and b
0.

Good start. That's essentially how Lou Thraki
solved it
analytically, a couple of hours after I posed the
problem, for the
Date: Thurs, Jul 20 2006 4:26 am
LT> Define c=b-a.
LT> Find out when the derivative of the function
LT> f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )
LT> has a zero ... <snip>

Post by Jack Tomsky
I used Excel's Solver and came up with the

following results.

Post by Jack Tomsky
There is a constant m (around 2) such that if b <=

m, then the mixture is unimodal with the mode being
at b/2, that is, the midpoint of the means of the two
Gaussians.

Post by Jack Tomsky
If b > m, then you have a bimodal distribution and

as b gets larger, the two symmetrical modes diverge
towards 0 and b.

Post by Jack Tomsky
All this could be subject to numerical errors to

some extent. I used the Solver's default iteration
settings.

Post by Jack Tomsky
Jack

Jack, that's fairly impressive for the artificial
intelligence of the
Excel solver. The solution for the special case is
exactly correct!.
I gave an explanation of Thraki's analytic solution
for the general
readership as well as some feel of the kurtosis
(flatness) of the
unimodal mixture for m=2 (at your b=1).
I even suggested yesterday,
RF> I think an elementary paper on the GENERAL case
of the
RF> convex combination of two Gaussian distributions
and the
RF> characterization of its modality may be
publishable in some
RF> statistical journal, because I don't recall ever
seen anything
RF> that discussed this sort of thing except in the
Bliss book
RF> 39 years ago, without any generality at all.
But the Excel story sounds interesting. Can you
describe more
HOW you provide Excel the input and what does it
actually did
to come up with the solution?
-- Reef Fish Bob.

The density of the mixture with equal weights is proportional to

g(x;b) = exp(-x^2/2)+exp(-(x-b)^2/2)

The (1/2)/sqrt(2*pi) factor is irrelevant.

For each b, starting with b = 0.1 and in increments of 0.1 for b up to 3.0, I used Solver to maximize g(x;b). For b up to around 2.0, the x which Solver found to maximize g(x;b) was exactly b/2.

For b greater than 2.0, I restricted the search to x <= b/2. Solver's solution kept decreasing towards zero as b increased.

Are you saying that the 2.0 is exact? I had the mode for b = 2.0 as 1.000663, which might include some numerical or rounding errors.

Jack

[Jack Tomsky]

Post by Reef Fish

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
Several posters in the Gaussian mixture

problem

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian

distributions is

Post by Jack Tomsky

Post by Reef Fish
Gaussian.
(Discounting the pathological examples

discussed

Post by Reef Fish
in

Post by Jack Tomsky

Post by Reef Fish
another thread).
A MIXTURE of two Gaussian distributions with

the

Post by Reef Fish
same

Post by Jack Tomsky

Post by Reef Fish
variance
and different means is NOT Gaussian. Whether

the

Post by Jack Tomsky

Post by Reef Fish
distribution of
the MIXTURE is bi-modal or unimodal depends on

how

Post by Jack Tomsky

Post by Reef Fish
far apart the
two means are!
If you MIX two Gaussian distributions with the

same

Post by Jack Tomsky

Post by Reef Fish
mean but
different variances, you'll get a unimodal
non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture

of

Post by Reef Fish
two

Post by Jack Tomsky

Post by Reef Fish
Gaussian
distributions with same variance and different

means

Post by Jack Tomsky

Post by Reef Fish
will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the

kurtosis

Post by Jack Tomsky

Post by Reef Fish
of the
distribution compared to that of a Gaussian
distribution.
The preceding three paragraphs are my

recollections

Post by Jack Tomsky

Post by Reef Fish
from the
textbook (by Chester I. Bliss) I had

jettisoned

Post by Reef Fish
into

Post by Jack Tomsky

Post by Reef Fish
the trash
can 39 years ago. But what I recalled about

the

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
mixture of
two Gaussian distributions and how they relate

to

Post by Reef Fish
the

Post by Jack Tomsky

Post by Reef Fish
kurtosis
of a Gaussian distribution seemed not to be

found

Post by Reef Fish
in

Post by Jack Tomsky

Post by Reef Fish
other
textbooks of statistics. That is why I

recalled

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
that factoid
while the rest of the book was obsolete even

39

Post by Reef Fish
years

Post by Jack Tomsky

Post by Reef Fish
ago.
Okay everyone. Make this a simple exercise

(which

Post by Jack Tomsky

Post by Reef Fish
was not
given in the book).
Let X and Y both be Gaussian distributions

with

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality

of

Post by Reef Fish
the

Post by Jack Tomsky

Post by Reef Fish
EQUAL
mixture of those two distributions in terms of

a

Post by Reef Fish
and

Post by Jack Tomsky

Post by Reef Fish
b.
-- Reef Fish Bob.

Without loss of generality, assume that a = 0 and

b

Post by Reef Fish

Post by Jack Tomsky
0.

Good start. That's essentially how Lou Thraki
solved it
analytically, a couple of hours after I posed the
problem, for the
Date: Thurs, Jul 20 2006 4:26 am
LT> Define c=b-a.
LT> Find out when the derivative of the function
LT> f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )
LT> has a zero ... <snip>

Post by Jack Tomsky
I used Excel's Solver and came up with the

following results.

Post by Jack Tomsky
There is a constant m (around 2) such that if b

<=

Post by Reef Fish
m, then the mixture is unimodal with the mode

being

Post by Reef Fish
at b/2, that is, the midpoint of the means of the

two

Post by Reef Fish
Gaussians.

Post by Jack Tomsky
If b > m, then you have a bimodal distribution

and

Post by Reef Fish
as b gets larger, the two symmetrical modes

diverge

Post by Reef Fish
towards 0 and b.

Post by Jack Tomsky
All this could be subject to numerical errors to

some extent. I used the Solver's default

iteration

Post by Reef Fish
settings.

Post by Jack Tomsky
Jack

Jack, that's fairly impressive for the artificial
intelligence of the
Excel solver. The solution for the special case

is

Post by Reef Fish
exactly correct!.
I gave an explanation of Thraki's analytic

solution

Post by Reef Fish
for the general
readership as well as some feel of the kurtosis
(flatness) of the
unimodal mixture for m=2 (at your b=1).
I even suggested yesterday,
RF> I think an elementary paper on the GENERAL

case

Post by Reef Fish
of the
RF> convex combination of two Gaussian

distributions

Post by Reef Fish
and the
RF> characterization of its modality may be
publishable in some
RF> statistical journal, because I don't recall

ever

Post by Reef Fish
seen anything
RF> that discussed this sort of thing except in

the

Post by Reef Fish
Bliss book
RF> 39 years ago, without any generality at all.
But the Excel story sounds interesting. Can you
describe more
HOW you provide Excel the input and what does it
actually did
to come up with the solution?
-- Reef Fish Bob.

The density of the mixture with equal weights is
proportional to
g(x;b) = exp(-x^2/2)+exp(-(x-b)^2/2)
The (1/2)/sqrt(2*pi) factor is irrelevant.
For each b, starting with b = 0.1 and in increments
of 0.1 for b up to 3.0, I used Solver to maximize
g(x;b). For b up to around 2.0, the x which Solver
found to maximize g(x;b) was exactly b/2.
For b greater than 2.0, I restricted the search to x
<= b/2. Solver's solution kept decreasing towards
zero as b increased.
Are you saying that the 2.0 is exact? I had the mode
for b = 2.0 as 1.000663, which might include some
numerical or rounding errors.
Jack

On rereading Lou's derivation, I see that b = 2 is the exact cutoff between the unimodal and bimodal mixtures.

Jack

[Reef Fish]

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
Several posters in the Gaussian mixture problem
seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian

distributions is

Post by Jack Tomsky

Post by Reef Fish
Gaussian.
(Discounting the pathological examples discussed

in

Post by Jack Tomsky

Post by Reef Fish
another thread).
A MIXTURE of two Gaussian distributions with the

same

Post by Jack Tomsky

Post by Reef Fish
variance
and different means is NOT Gaussian. Whether

the

Post by Jack Tomsky

Post by Reef Fish
distribution of
the MIXTURE is bi-modal or unimodal depends on

how

Post by Jack Tomsky

Post by Reef Fish
far apart the
two means are!
If you MIX two Gaussian distributions with the

same

Post by Jack Tomsky

Post by Reef Fish
mean but
different variances, you'll get a unimodal
non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture of

two

Post by Jack Tomsky

Post by Reef Fish
Gaussian
distributions with same variance and different

means

Post by Jack Tomsky

Post by Reef Fish
will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the

kurtosis

Post by Jack Tomsky

Post by Reef Fish
of the
distribution compared to that of a Gaussian
distribution.
The preceding three paragraphs are my

recollections

Post by Jack Tomsky

Post by Reef Fish
from the
textbook (by Chester I. Bliss) I had jettisoned

into

Post by Jack Tomsky

Post by Reef Fish
the trash
can 39 years ago. But what I recalled about the
mixture of
two Gaussian distributions and how they relate to

the

Post by Jack Tomsky

Post by Reef Fish
kurtosis
of a Gaussian distribution seemed not to be found

in

Post by Jack Tomsky

Post by Reef Fish
other
textbooks of statistics. That is why I recalled
that factoid
while the rest of the book was obsolete even 39

years

Post by Jack Tomsky

Post by Reef Fish
ago.
Okay everyone. Make this a simple exercise

(which

Post by Jack Tomsky

Post by Reef Fish
was not
given in the book).
Let X and Y both be Gaussian distributions with
variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality of

the

Post by Jack Tomsky

Post by Reef Fish
EQUAL
mixture of those two distributions in terms of a

and

Post by Jack Tomsky

Post by Reef Fish
b.
-- Reef Fish Bob.

Without loss of generality, assume that a = 0 and b
0.

Good start. That's essentially how Lou Thraki
solved it
analytically, a couple of hours after I posed the
problem, for the
Date: Thurs, Jul 20 2006 4:26 am
LT> Define c=b-a.
LT> Find out when the derivative of the function
LT> f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )
LT> has a zero ... <snip>

Post by Jack Tomsky
I used Excel's Solver and came up with the

following results.

Post by Jack Tomsky
There is a constant m (around 2) such that if b <=

m, then the mixture is unimodal with the mode being
at b/2, that is, the midpoint of the means of the two
Gaussians.

Post by Jack Tomsky
If b > m, then you have a bimodal distribution and

as b gets larger, the two symmetrical modes diverge
towards 0 and b.

Post by Jack Tomsky
All this could be subject to numerical errors to

some extent. I used the Solver's default iteration
settings.

Post by Jack Tomsky
Jack

Jack, that's fairly impressive for the artificial
intelligence of the
Excel solver. The solution for the special case is
exactly correct!.
I gave an explanation of Thraki's analytic solution
for the general
readership as well as some feel of the kurtosis
(flatness) of the
unimodal mixture for m=2 (at your b=1).
I even suggested yesterday,
RF> I think an elementary paper on the GENERAL case
of the
RF> convex combination of two Gaussian distributions
and the
RF> characterization of its modality may be
publishable in some
RF> statistical journal, because I don't recall ever
seen anything
RF> that discussed this sort of thing except in the
Bliss book
RF> 39 years ago, without any generality at all.
But the Excel story sounds interesting. Can you
describe more
HOW you provide Excel the input and what does it
actually did
to come up with the solution?
-- Reef Fish Bob.

The density of the mixture with equal weights is proportional to
g(x;b) = exp(-x^2/2)+exp(-(x-b)^2/2)

That was what I had said in my explanation of Thraki's solution.

Post by Jack Tomsky
The (1/2)/sqrt(2*pi) factor is irrelevant.

I also said that to Thraki in the same post -- irrelevant as far as
finding the point at which the rel. max occurs. But it IS relevant
to the actual value of the DENSITY function of the mixture --
which I showed after explaining Thraki's analytic solution.

Jack, I thought you would have looked up that post of mine.
You obviously didn't. That was there:

RF> Your f(x) is proportional to the pdf of the convex combination
RF> of two Gaussian distributions N(a,1) and N(b,1), without
RF> explicitly noting that this form works ONLY if p and (1-p)
RF> is the same (.5) and f(x) is the pdf of the mixture, missing
RF> the proportionality constand .5/sqrt(2*pi).

RF> Since you're finding the zeros of the derivative of f(x), the
RF> proportionality constant doesn't matter.

Perhaps you did read my post and you're just trying to explain
what you did in Excel.

Post by Jack Tomsky
For each b, starting with b = 0.1 and in increments of 0.1 for b up to 3.0, I used Solver to maximize g(x;b). For b up to around 2.0, the x which Solver found to maximize g(x;b) was exactly b/2.
For b greater than 2.0, I restricted the search to x <= b/2. Solver's solution kept decreasing towards zero as b increased.
Are you saying that the 2.0 is exact?

Apparently, according to Thraki's analytic solution. I didn't
think it was either, when I did your kind of numerical
search manually, to check the density, the probability
coverage, etc. (that's why the (1/2)/sqrt(2*pi) factor IS
relevant for that calculation; otherwise the density function
would not integrate to 1!), I did get some numerical
rounding errors even around 2.

So, you just had Excel do a DUMB search. :-) I thought
Excel had some kind of artificial intelligence component in
it.

I had the mode for b = 2.0 as 1.000663, which might include some
numerical or rounding errors.

Post by Jack Tomsky
Jack

Take a look at Thraki's analytic solution and see if his c/2 has
an error. His c is your b. I'll take another look too.

The analytic solution for the GENERAL case may be easier than
the discrete search method such as yours.

In the general case, you'll have the convex weight w (which
wouldn't be 1/2 anymore), the two means and two variances
of the two normals. Then you have the modality being a function
of FIVE variables (which you may reparametrize to three), but
you're still left with a function of THREE variables to look for
the relative maxima rather than a function of only variable,
your m, and Thraki's c.

-- Reef Fish Bob.

[Jack Tomsky]

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
Several posters in the Gaussian mixture

problem

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
seemed to have
confused a MIXTURE with a LINEAR COMBINATION.
A linear combination of two Gaussian

distributions is

Post by Jack Tomsky

Post by Reef Fish
Gaussian.
(Discounting the pathological examples

discussed

Post by Jack Tomsky

Post by Reef Fish
in

Post by Jack Tomsky

Post by Reef Fish
another thread).
A MIXTURE of two Gaussian distributions with

the

Post by Jack Tomsky

Post by Reef Fish
same

Post by Jack Tomsky

Post by Reef Fish
variance
and different means is NOT Gaussian.

Whether

Post by Jack Tomsky

Post by Reef Fish
the

Post by Jack Tomsky

Post by Reef Fish
distribution of
the MIXTURE is bi-modal or unimodal depends

on

Post by Jack Tomsky

Post by Reef Fish
how

Post by Jack Tomsky

Post by Reef Fish
far apart the
two means are!
If you MIX two Gaussian distributions with

the

Post by Jack Tomsky

Post by Reef Fish
same

Post by Jack Tomsky

Post by Reef Fish
mean but
different variances, you'll get a unimodal
non-Gaussian distribution
that is Leptokurtic, while a unimodal mixture

of

Post by Jack Tomsky

Post by Reef Fish
two

Post by Jack Tomsky

Post by Reef Fish
Gaussian
distributions with same variance and

different

Post by Jack Tomsky

Post by Reef Fish
means

Post by Jack Tomsky

Post by Reef Fish
will be
Platykurtic.
Leptokurtosis and Platykurtosis refer to the

kurtosis

Post by Jack Tomsky

Post by Reef Fish
of the
distribution compared to that of a Gaussian
distribution.
The preceding three paragraphs are my

recollections

Post by Jack Tomsky

Post by Reef Fish
from the
textbook (by Chester I. Bliss) I had

jettisoned

Post by Jack Tomsky

Post by Reef Fish
into

Post by Jack Tomsky

Post by Reef Fish
the trash
can 39 years ago. But what I recalled about

the

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
mixture of
two Gaussian distributions and how they

relate to

Post by Jack Tomsky

Post by Reef Fish
the

Post by Jack Tomsky

Post by Reef Fish
kurtosis
of a Gaussian distribution seemed not to be

found

Post by Jack Tomsky

Post by Reef Fish
in

Post by Jack Tomsky

Post by Reef Fish
other
textbooks of statistics. That is why I

recalled

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
that factoid
while the rest of the book was obsolete even

39

Post by Jack Tomsky

Post by Reef Fish
years

Post by Jack Tomsky

Post by Reef Fish
ago.
Okay everyone. Make this a simple exercise

(which

Post by Jack Tomsky

Post by Reef Fish
was not
given in the book).
Let X and Y both be Gaussian distributions

with

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
variance 1,
and means at a and b respectively.
Characterize the unimodality and bi-modality

of

Post by Jack Tomsky

Post by Reef Fish
the

Post by Jack Tomsky

Post by Reef Fish
EQUAL
mixture of those two distributions in terms

of a

Post by Jack Tomsky

Post by Reef Fish
and

Post by Jack Tomsky

Post by Reef Fish
b.
-- Reef Fish Bob.

Without loss of generality, assume that a = 0

and b

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky
0.

Good start. That's essentially how Lou Thraki
solved it
analytically, a couple of hours after I posed the
problem, for the
Date: Thurs, Jul 20 2006 4:26 am
LT> Define c=b-a.
LT> Find out when the derivative of the function
LT> f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )
LT> has a zero ... <snip>

Post by Jack Tomsky
I used Excel's Solver and came up with the

following results.

Post by Jack Tomsky
There is a constant m (around 2) such that if b

<=

Post by Jack Tomsky

Post by Reef Fish
m, then the mixture is unimodal with the mode

being

Post by Jack Tomsky

Post by Reef Fish
at b/2, that is, the midpoint of the means of the

two

Post by Jack Tomsky

Post by Reef Fish
Gaussians.

Post by Jack Tomsky
If b > m, then you have a bimodal distribution

and

Post by Jack Tomsky

Post by Reef Fish
as b gets larger, the two symmetrical modes

diverge

Post by Jack Tomsky

Post by Reef Fish
towards 0 and b.

Post by Jack Tomsky
All this could be subject to numerical errors

to

Post by Jack Tomsky

Post by Reef Fish
some extent. I used the Solver's default

iteration

Post by Jack Tomsky

Post by Reef Fish
settings.

Post by Jack Tomsky
Jack

Jack, that's fairly impressive for the artificial
intelligence of the
Excel solver. The solution for the special case

is

Post by Jack Tomsky

Post by Reef Fish
exactly correct!.
I gave an explanation of Thraki's analytic

solution

Post by Jack Tomsky

Post by Reef Fish
for the general
readership as well as some feel of the kurtosis
(flatness) of the
unimodal mixture for m=2 (at your b=1).
I even suggested yesterday,
RF> I think an elementary paper on the GENERAL

case

Post by Jack Tomsky

Post by Reef Fish
of the
RF> convex combination of two Gaussian

distributions

Post by Jack Tomsky

Post by Reef Fish
and the
RF> characterization of its modality may be
publishable in some
RF> statistical journal, because I don't recall

ever

Post by Jack Tomsky

Post by Reef Fish
seen anything
RF> that discussed this sort of thing except in

the

Post by Jack Tomsky

Post by Reef Fish
Bliss book
RF> 39 years ago, without any generality at all.
But the Excel story sounds interesting. Can you
describe more
HOW you provide Excel the input and what does it
actually did
to come up with the solution?
-- Reef Fish Bob.

The density of the mixture with equal weights is

proportional to

Post by Jack Tomsky
g(x;b) = exp(-x^2/2)+exp(-(x-b)^2/2)

That was what I had said in my explanation of
Thraki's solution.

Post by Jack Tomsky
The (1/2)/sqrt(2*pi) factor is irrelevant.

I also said that to Thraki in the same post --
irrelevant as far as
finding the point at which the rel. max occurs. But
it IS relevant
to the actual value of the DENSITY function of the
mixture --
which I showed after explaining Thraki's analytic
solution.
Jack, I thought you would have looked up that post of
mine.
RF> Your f(x) is proportional to the pdf of the
convex combination
RF> of two Gaussian distributions N(a,1) and N(b,1),
without
RF> explicitly noting that this form works ONLY if p
and (1-p)
RF> is the same (.5) and f(x) is the pdf of the
mixture, missing
RF> the proportionality constand .5/sqrt(2*pi).
RF> Since you're finding the zeros of the derivative
of f(x), the
RF> proportionality constant doesn't matter.
Perhaps you did read my post and you're just trying
to explain
what you did in Excel.

Post by Jack Tomsky
For each b, starting with b = 0.1 and in increments

of 0.1 for b up to 3.0, I used Solver to maximize
g(x;b). For b up to around 2.0, the x which Solver
found to maximize g(x;b) was exactly b/2.

Post by Jack Tomsky
For b greater than 2.0, I restricted the search to

x <= b/2. Solver's solution kept decreasing towards
zero as b increased.

Post by Jack Tomsky
Are you saying that the 2.0 is exact?

Apparently, according to Thraki's analytic solution.
I didn't
think it was either, when I did your kind of
numerical
search manually, to check the density, the
probability
coverage, etc. (that's why the (1/2)/sqrt(2*pi)
factor IS
relevant for that calculation; otherwise the
density function
would not integrate to 1!), I did get some numerical
rounding errors even around 2.
So, you just had Excel do a DUMB search. :-) I
thought
Excel had some kind of artificial intelligence
component in
it.
I had the mode for b = 2.0 as 1.000663, which might
include some
numerical or rounding errors.

Post by Jack Tomsky
Jack

Take a look at Thraki's analytic solution and see if
his c/2 has
an error. His c is your b. I'll take another
look too.
The analytic solution for the GENERAL case may be
easier than
the discrete search method such as yours.
In the general case, you'll have the convex weight w
(which
wouldn't be 1/2 anymore), the two means and two
variances
of the two normals. Then you have the modality
being a function
of FIVE variables (which you may reparametrize to
three), but
you're still left with a function of THREE variables
to look for
the relative maxima rather than a function of only
variable,
your m, and Thraki's c.
-- Reef Fish Bob.

Lou's derivation is excellent. I should have read it more carefully originally.

Jack

[Reef Fish]

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky
Without loss of generality, assume that a = 0
and b > 0.

RF > Good start. That's essentially how Lou Thraki solved it
RF > analytically, a couple of hours after I posed the problem,

Post by Jack Tomsky

Post by Reef Fish

Post by Jack Tomsky

Post by Reef Fish
Date: Thurs, Jul 20 2006 4:26 am
LT> Define c=b-a.
LT> Find out when the derivative of the function
LT> f(x) = exp( -x^2/2 ) + exp( -(x-c)^2/2 )
LT> has a zero ... <snip>

Are you saying that the 2.0 is exact?

Apparently, according to Thraki's analytic solution.

Lou's derivation is excellent. I should have read it more carefully originally.
Jack

I agree that Lou's derivation is excellent.

Some problems are better done analytically than numerically.
Given your math. stat. training under Olkin, I would have expected
you to do the same, rather than resorting to Excel (which is, BTW,
the worst software to use for statistics) and I thought you had
found some unexpected "artificial intelligence" there. :-) But
you found the same old Excel, which excels in poor numerical
precision compared to other software.

It reminded me of a paper I acted as a referee, submitted
for publication to a journal. The authors similated some results
on the connectivity of random graphs. which seemed intractible
to derive analytically. It turned out I was aware of a very
obscure (and rather difficult) theoretical result involving
Stirling numbers of the Second Kind and showed the authors
(in my referee's report) that their simulation results came
remarkably close to some cases that I was able to check
analytically. I did not recommend to reject the paper, but
the authors withdrew the submission voluntarily and the
editor invited me to publish to analytically derivable results,
and I did.

That was 30 years ago when the attempt to use the computer
to solve everything wasn't quite as rampant as today. :-)
But that was the only case I came across in which the
simulation paper was submitted by two very competent
and well-known authors in the field, whom I knew well, and
they later jokingly said to me that I "scooped" their results
when they could tell the anonymous referee was me .

Just another little story of analytics versus brute force computing.
:-)

-- Reef Fish Bob.

[Jack Tomsky]

I often use the Excel Solver for such problems as maximum likelihood estimation and nonlinear curve-fitting. Since these spreadsheets are used by chemists, we want the spreadsheets to be as self-contained as possible. All they would have to do is to enter the input data and then click on print.

A challenge often is to build in equations for the starting values of the parameters so that they will be in the ballpark of the final optimal solutions. These are frequently suggested by such devices as graphically approximating the data by centroids and deriving closed-form expressions for the parameters.

A massive project here is to validate hundreds of Excel spreadsheets to satisfy the FDA. We've been doing them in batches of related spreadsheets. Each of the final validation reports end up being the size and weight of several large-city telephone books. To get the document over to the approvers for their approval signatures, I have to borrow a hand truck to wheel them over. (Don't even ask me how we paginate and develop the table of contents for these mixed types of printouts.)

The validations verify that the spreadsheets are in standardized formats with the formula cells locked and protected by a secret password. The heart of the validation is to verify that the answers that the spreadsheet obtains for the test cases match up with those from a gold standard. The gold standard we've been using are the JMP data tables from SAS.

When we began this project, JMP 5.1 had a peculiar rounding rule. For example, if Excel calculates a result of 1.35, for a one-decimal display, it uses the traditional rounding rules in displaying it as 1.4. If the additional decimal is five or more, it rounds up; otherwise, it rounds down.

JMP, on the other hand, would sometimes round up and sometimes round down when the next decimal is exactly five. I took 100 numbers in arithmetic sequence with the last decimal being five. I found that JMP rounded up 70% of the time and down 30% of the time. There did seem to be symmetrical patterns, though they were not simple to explain. The effect was that occasionally, there would be a discrepancy between the Excel spreadsheet and JMP. I had to count it as a deviation and explain that the "gold standard" JMP was incorrect.

I complained about this to SAS. Apparently I wasn't the only one who noticed this. They promised me that in the next version, JMP 6.0, they would reprogram their rounding rules to match up with the conventional rules that everyone else uses. When JMP 6.0 came out, I checked it against the same 100 cases and found that, indeed, all 100 cases were rounded up.

Jack