.
How to interpret subjective Bayesians who want to be hard-nosed Bayesians is often like swimming round and round in a funnel of currents where there’s nothing to hold on to. Well, I think I’ve recently stopped the flow and pegged it. Christian Hennig and I have often discussed this (on my regular blog) and something Gelman posted today, linked me to an earlier exchange between he and Christian.
Christian: I came across an exchange between you and Andrew because it was linked to by Andrew on a current blog post
It really brings out the confusion I have had, we both have had, and which I am writing about right now (in my book), as to what people like Gelman mean when they talk about posterior probabilities. First:
a posterior of .9 to
H: “θ is positive”
is identified with giving 9 to 1 odds on H.
Gelman had said: “it seems absurd to assign a 90% belief to the conclusion. I am not prepared to offer 9 to 1 odds on the basis of a pattern someone happened to see that could plausibly have occurred by chance”
Then Christian says, this would be to suggest “I don’t believe it” means “it doesn’t agree with my subjective probability” and Christian doubts Andrew could mean that. But I say he does mean that. His posterior probability is his subjective (however evidence-based) probability. ‘
Next the question is, what’s the probability assigned to? I think it is assigned to H:θ > 0
As for the meaning of “this event would occur 90% of the time in the long run under repeated trials” I’m guessing that “this event” is also H. The repeated “trials” allude to a repeated θ generating mechanism, or over different systems each with a θ. The outputs would be claims of form H (or not-H or different assertions about the θ for the case at hand ), and he’s saying 90% of the time the outputs would be H, or H would be the case. The outputs are not ordinary test results, but states of affairs, namely θ > 0.
Bottom line: It seems to me that all Bayesians all who assign posteriors to parameters (aside from empirical Bayesians) really mean the kind of odds statement that you and I and most people associate with partial -belief or subjective probability. “Epistemic probability” would do as well, but equivocal. It doesn’t matter how terrifically objectively warranted that subjective probability assignment is, we’re talking meaning. And when one finally realizes this is what they meant all along, everything they say is less baffling. What do you think?
——————————————–
Background
Andrew Gelman:
First off, a claimed 90% probability that θ>0 seems too strong. Given that the p-value (adjusted for multiple comparisons) was only 0.2—that is, a result that strong would occur a full 20% of the time just by chance alone, even with no true difference—it seems absurd to assign a 90% belief to the conclusion. I am not prepared to offer 9 to 1 odds on the basis of a pattern someone happened to see that could plausibly have occurred by chance,
Christian Hennig says:
“Then the data under discussion (with a two-sided p-value of 0.2), combined with a uniform prior on θ, yields a 90% posterior probability that θ is positive. Do I believe this? No.”
What exactly would it mean to “believe” this? Are you referring to a “true unknown” posterior probability with which you compare the computed one? How would the “true” one be defined?
Later there’s this:
“I am not prepared to offer 9 to 1 odds on the basis of a pattern someone happened to see that could plausibly have occurred by chance, …”
…which kind of suggests that “I don’t believe it” means “it doesn’t agree with my subjective probability” – but knowing you a bit I’m pretty sure that’s not what you meant before. But what is it then?
Rejected Posts of D. Mayo 
I think Gelman wrote a bit more as a reply but I don’t think he left me convinced.
But I could try to interpret what Gelman may have meant. This may be wrong, though.
Gelman has at times interpreted the parameter prior in an almost frequentist/propensity manner, i.e., as a data generating process (albeit with strong idealisation, and some ambiguity about the reference set) that brings forth parameters with certain probabilities. Now he could say that the posterior probability under question is a consequence of the given model for the parameter generating process, and that by not believing it he means that he’d therefore believe that this model is wrong (the reference was wrongly chosen, the prior probabilities were wrong, something like this).
If he follows the “principal principle”, this then also would mean that he’d not bet according to this model, but this doesn’t mean he is a subjectivist.
You shouldn’t believe me when interpreting Gelman, you should ask himself, though.
Christian: When he construed him meaning in terms of the bets he’d take, you said that was subjectivist, did you not? I said it was evidence based. One compares the “degree of certainty” one holds in H to an imagined event whose occurrence one could be offered bets on. In his book he associates probability with partial knowledge and betting,, or analogies to that, so it’s no surprise. It also accords with the metaphor of a theta-generating mechanism.
I think I referred to Gelman’s personal “subjective probabilities” (assuming that there is a reasonable idea of what that is) but not to being “subjectivist” (which to me somehow seems to imply that that’s the best or even only way probabilities should be understood).
That’s not at all meant to be in contradiction to being “evidence-based”. You can have subjective probabilities that are evidence-based, why not?
Christian: Exactly, that’s my point. So when he assigns 9 to 1 odds, or whatever, to a hypotheses like theta exceeds 0 (even granting theta is fixed, but our assessments of it vary), he’s alluding to “his” betting odds, and thus “his partial belief” or uncertainty, and is thus referring to subjective or epistemic probability. Even if he’d demand quite a lot of good evidence before posting those odds, that is still what’s meant. Yes?
That’s my interpretation of his “not offering odds”. But I’d feel somewhat uncomfortable if you took my interpretation of him for granted. I still suspect that he may have intended something else but may have written it down in a misleading way (or at least I may have been misled). After all it’s the comment section of a blog discussion. His last word on the matter should not be taken from there.
Sure, I’ll ask him. But having just spent a week rereading Bayesians of every stripe, in the end, despite all the different nuances, what other interpretation is there for a probability assigned to a hypothesis, as a measure of uncertainty?
This isn’t mysterious. Not enough was known to determine theta. That leaves many possibilities for theta compatible with the evidence. Roughly 90% of those possibilities are greater than zero.
You think this could only be meaningful or useful if we add a requirement that theta be repeatable and that .9 approximate the fraction of times it’s greater than zero, but you’re wrong about this. It is meaningful (obviously) and useful (with a little thought) even without these gratuitous add-ons.
No, your 90% possibilities being greater than 0 fits my construal.
Laplace: I never said that it’s the *only* way of being meaningful. I only speculated about what Gelman may have had in mind. What you write explains one way of understanding the 90%, which is fair enough for me, but it doesn’t explain why and on what basis Gelman wrote that he doesn’t believe it.
Noah: I tried to comment on the comment you posted here:
http://cancelinfinity.blogspot.com/2015/06/what-is-conflict-between-uninformed.html
but it didn’t work.
My point was simply that posterior probabilities for Bayesians, even evidence-based sorts, are intended to represent degree of uncertainty, as in how one would bet. That’s a very different notion from how well-tested or extent of evidential warrant accorded a claim. As for conventional priors/posteriors, we’re told various different things: they are to reflect the beliefs of a group while letting the data be primary, or they reflect no or little information. So what was it in Gelman that you claim I have missed?
Click to access edwards-lindman-savage_1963.pdf
fraser_is-bayes-posterior-just-quick-and-dirty-confidence.pdf