Particle Physics is Bad Science?

I suppose this is somewhat of a joke from the ISBA, prompted by Dennis Lindley–right?– but as I accord the actual degree of jokiness to be only ~33%, I’m raising it on my Msc Kvetching page.  Lindley (according to O’Hagan) wonders why scientists require so high a level of statistical significance before claiming to have evidence of a Higgs boson.  It is asked: “Are the particle physics community completely wedded to frequentist analysis?  If so, has anyone tried to explain what bad  science that is?”
Bad science?   I’d really like to understand what these representatives from the ISBA would recommend, if there is even a shred of seriousness here (or is Lindley just peeved that significance levels are getting so much press in connection with so important a discovery in particle physics?)
Well, read the letter and see what you think.

On Jul 10, 2012, at 9:46 PM, ISBA Webmaster wrote:

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We’ve heard a lot about the Higgs boson.  The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson.  Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level.  Five standard deviations, assuming normality,
means a p-value of around 0.0000005.  A number of questions spring to mind.

1.  Why such an extreme evidence requirement?  We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme.  Neither seems to be the case, so why
5-sigma?

2.  Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis.  Are the particle physics community completely
wedded to frequentist analysis?  If so, has anyone tried to explain what bad
science that is?

3.  We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data.  So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I’d be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

—-
Professor A O’Hagan            Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield        Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK           Fax:   +44 114 222 3759

So given that the Higgs boson does not have such an extremely small prior probability, a proper Bayesian analysis would have enabled evidence of the Higgs long before attaining such an “extreme evidence requirement”. Why has no one tried to explain to these scientists how with just a little Bayesian analysis, they might have been done in last year or years ago? I take it the Bayesian would also enjoy the simplicity and freedom of not having to adjust to take account of what physicists call “the Look Elsewhere Effect” (LEE).