5

In Bayesian literature, probabilities are considered subjective, and it is recommended to never assign a zero probability to anything. This is because a zero probability as a credence implies that you cannot change your mind with regards to a belief, since no evidence can “update” your belief. For this reason, even the most elaborately constructed or imagined scenarios that seem to have zero evidence (such as an invisible breathing dragon existing in your room) are postulated to have unknown, but non zero probabilities.

But when it comes to mundane events, we seem to assign exact, seemingly “known” probabilities all the time. For example, we assign a 1/6 credence to the outcome of a dice rolling on 6. This is based upon what we know of physics and the expected or historical frequency of how often a dice lands on 6.

But how is that observed frequency any different from the observed frequency of a miracle? The observed frequency of miracles (I.e. events violating natural law that we all agree upon that occurred) is 0.

If a skeptic says “but we don’t really know that frequency, perhaps there have been miracles that occurred that we just don’t know of yet, so we can’t assign a zero probability.”, then why can’t a skeptic say the same with regards to dice? For example, another skeptic could say “but we don’t really know how often a dice lands on 6. For all we know, it lands on 6 half the time, or almost all the time, and some invisible god is simply making you think that it’s landing on 6 1/6 of the time”.

Now of course, the second kind of skeptic scenario is rarely thought of, and no one ever uses that kind of scenario to deviate from assigning a 1/6 probability to a dice landing on 6. Why then should one use a similar kind of scenario to dispute, for example, the probability of a miracle being 0?

Improve this question
11
  • 1
    Re: arbitrary probabilities for dice, one place might be the Principal Principle (Lewis) / Also: what's wrong with assigning a very small initial probability to miracles? Commented 2 days ago
  • My rough-and-ready understanding of the PP is that one should adjust credences to any "empirically found" probabilities (frequencies) ~/~ Commented 2 days ago
  • 1
    @uncle_jo yes, but it seems as if events that have no occurrences are considered to have an “unknown” frequency rather than a zero frequency which seems inconsistent Commented 2 days ago
  • 2
    Credence is a measure of belief strength, & has overlap with but is distinct from probability en.wikipedia.org/wiki/Credence_(statistics) Commented 2 days ago
  • 2
    I assign zero credence to one die rolling a 7. Commented yesterday

7 Answers 7

Reset to default
8

Yes, but it is academic. Let's suppose you had never seen a dice rolled and had no idea about the physical characteristics of dice other than the fact that numbers could be presented by them. You assign a zero probability to the chance of a six being rolled, on the grounds you had no evidence of that having ever happened, and for all you know dice might only ever display numbers greater than a million. I could argue that you were wrong to do so, since having assigned a zero probability, no evidence could 'update' your belief, as you put it. However, I would be talking nonsense, because in practice you are free to update your belief at any time, and having seen what a dice is, and rolled one a few times, you would very rapidly conclude that the zero probability you assigned earlier was the wrong call. On the same basis, I would be quite content assigning a zero probability to dragons in my bedroom, and equally content to revisit that assumption if one appeared.

Put another way, Bayesian analysis is just a tool, one that can be useful when you have meaningful information to work with, but it is subject to the 'garbage in, garbage out' rule, and you are free to start again with fresh garbage whenever you like.

You might, for example, start with an assumption that the probability of any event being a miracle is 50%. You could then update that estimate, using Bayesian techniques, every time you experienced an event that was not miraculous, and after not very long you would find the probability dropping so close to zero as to make no difference.

Improve this answer
10
  • 1
    Very interesting point you make with regards to updating your beliefs as well, which at least to me seems to bolster the inconsistency even more. It’s considered a problem to assign something a 0 probability, but say the observed frequency of a dice rolling on 6 is 0.511. The next roll being 6 is technically updating that frequency as well. One may update their probability figure from 0.511 to a new slightly higher number which isn’t considered a problem in Bayesianism. Why is updating a 0 probability to a non zero probability then considered a problem? Commented 2 days ago
  • 1
    'Rules are for fools' about sums it up. Commented 2 days ago
  • 3
    If you assign a 0 probability to some event, and then revise that probability upward based on new evidence, Bayesian theory would hold that you didn't really assign a zero probability in the first place as shown by your subsequent behavior. You might have called the probability 0, but you really had a positive subjective probability in mind. Commented 2 days ago
  • 2
    It just means you had a distribution over the parameter rather than a single point mass at the estimated value of 20%. 20% may have been the maximum initially, but it was not the entirety of the probability mass over the parameter. I think you need to ask some basics at Cross Validated Commented yesterday
  • 3
    Yes, you can reject Bayesian statistics, Bayes' rule, and the update formulae, but this question is about and within a Bayesian framework. 0 is not disallowed. Priors can have a maximum at zero. They just can't have their full mass at zero, if you want to allow for Bayesian revision away from zero. Likewise, if the full mass were at 0.2, there is no Bayesian revision away from 0.2. This is not special about 0. Commented yesterday
4

A true Bayesian prior does not assign exact point probabilities. It is always a distribution over the parameter.

The prior of "1/6" that you mention cannot be a point prior for a true Bayesian prior. You are over-simplifying a Bayesian prior by representing it by a single number. It must be a distribution with mass at every value [0,1]. Typically, for a multinomial outcome like a dice roll, it will have the form of a Dirichlet distribution.

Therefore, with additional observations, that 1/6 (which is really just the maximum of the distribution for one of the sides) can slowly update.

But if there is no mass of the prior assigned to a particular outcome, it will always be zero in the posterior under the standard Bayesian update formula. Therefore, assigning no mass of the Bayesian prior to a given outcome is taking a strong, unrevisable epistemological position that the outcome is impossible.

Also note that not having observed an occurrence yet is consistent with it having a very low frequency of occurrence.

You ask:

then why can’t a skeptic say the same with regards to dice? For example, another skeptic could say “but we don’t really know how often a dice lands on 6.

They can. So that's why the prior is a Dirichlet prior for dice, and not an unrevisable commitment to 1/6.

Improve this answer
3
  • I thought one can technically assign any kind of prior to a hypothesis in bayesanism, and that it depends on what kind of bayesianism you adopt, such as objective or subjective bayesianism. Do you mind linking a resource that prohibits Bayesians from adopting a “point” probability as a prior, or that it must have a distribution with mass at every value from 0 to 1? Commented 2 days ago
  • 2
    That is best asked at Cross Validated, but as a primer see en.wikipedia.org/wiki/Prior_probability You can assign a point prior, but then you've committed to not ever revising your belief away from that, which defeats the point of Bayesian statistics. If you want a distribution to be susceptible to update under Bayes' rule, it cannot be a point. Commented 2 days ago
  • You can, of course, also update your priors for a dice if you have 1/6 for every integer 1..6; and the Wikipedia page also includes such a case for a ball under three cups. (It may still be an unbalanced dice where you update the probabilities). That is different from saying that it is fixed, and saying that it is 0. Commented 2 days ago
2

All predictions assume a stable pattern

All predictions are extrapolations.

Using probabilities derived from statistics, to predicts future events, hinges on the assumption that the pattern is stable, and not subject to sudden changes. Only then can the extrapolation be trusted.

In fact, this goes for all predictions, an assumption that the laws of nature do not go wild and crazy and decide to reinvent themselves willy-nilly. Again, only then ‒ with that assumption ‒ can predictions using the laws of nature to determine a future outcome be relied upon.

But this always comes with the awareness that we are, in fact, extrapolating from a past pattern. Nothing(!) says that the pattern will persist, nor does it say that this outcome will not be subject to a rare occurrence in the pattern.

What then is the difference?

The difference is that with dice, there is a pattern.

With miracles, there is not. Or rather, the pattern is that miracles do not happen. The pattern just went off the rails, not into the surrounding geography but straight up along the normal to the ground plane.

With dice, a skeptic can say "We have seen this pattern before, we expect it to continue", and then be pleasantly unsurprised when this happens.

With miracles, it is just a great big Whiskey Tango Foxtrot... we have no pattern for that ‒ the event of a miracle just broke the pattern, in a manner that is not redeemable.

So ‒ yes ‒ both dice rolls and miracles are technically in the "non-zero probability" category.

But they live at opposite ends of that scale. One has a stable, well-tested pattern behind it. The other would be like watching the pattern sprout wings and fly away while blowing you a raspberry.

...unless, of course, we misread the event and the pattern was intact all along ‒ which is usually what happens when someone claims a miracle.

Improve this answer
0

One way to look at it is that like anything else we only form beliefs about the material world through out experience of the material world. If we have no experience of dice, we cannot have beliefs about them. We form a belief as to the chances of getting a 6 after throwning a die, or even simply after looking at the makeup of the die. In either case, we will give at least one outcome some probability greater than zero.

Improve this answer
0

You can change your mind about the dice being fair

Let's say apriori you assign a 1/6 credence to the dice rolling 6 each time you roll it. But then, you roll it a hundred times and it shows 1 every time. A good Bayesianist would then deduce that the dice is probably rigged and update their credence to approximately 0.

So the 1/6 credence is not fixed, but can change in response to evidence (such as previous dice rolls).

However, due to how the math works, a credence of exactly 0 or 1 can never update. That's why they are so dangerous compared to numbers like 1/6.

If your credence is 1/6 you can change your mind, but if your credence is 0 or 1 you cannot change your mind without abandoning Bayesianism itself.

Improve this answer
0

Take a fair coin, a nickel, and assign a 50/50 probability that it's either heads or tails. This coin gets flipped 1000's of times and the recorded events match a 50/50 distribution. Then one day on the 6,000 flip, a miracle occurs. The nickel lands on edge. This event had 0 probability when the experiment started. Now it has a 1/6000 chance. The probabilities have now changed and should be updated.

In this instance the miracle doesn't matter because heads and tails still have equally likely outcomes so a choice of heads or tails is still fair. A "miracle" only warrants a fresh flip.

Improve this answer
0

To not assign 0/1 probability to real-world events is called Cromwell's rule.
"I beseech you, in the bowels of Christ, think it possible that you may be mistaken" ~ Cromwell

For a die roll, the probability of an outcome that is not 1, 2, 3, 4, 5, or 6 is not 0 (it could land perfectly balanced on one its edges/a corner) but it is almost surely 0. So P(1) + P(2) + P(3) + P(4) + P(5) + P(6) + P(not 1, 2, 3, 4, 5, 6) = 1/n + 1/n + 1/n + 1/n + 1/n + 1/n + 0.00000000000000000...1. We can safely ignore such outcomes and then n = 6. My math didn't come out right but you get the idea I suppose.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.