Testing D20

As promised, I finally got some Game Science dice today and immediately set out testing them to see if they're fair, then compared the result against the black D20 that came with my old red box Basic Set and I've had since god-knows-when.  Ready for the results?



Figure 1:  Histograming results initially suggests that the black D20 likes to produce a result of "18."



The inital results are shown in figure 1.  After rolling each die 100 times, I then applied the chi-squared test to estimate the likelyhood of such a result, given the differences between the observed and expected distributions. 

The net result!  The observed distribution of my Game Science D20 is about 80% likely given the expected distribution of a fair die, while the observed distribution of my black D20 is about 5% likely (see figure 2). 

Figure 2:  Chi-squared test probability of the observed distribution given the expected distribution of a fair D20.

If the observed distribution is 80% likely given the expected distribution of a fair die, then I'd bet that the die is fair (or at least as close to fair as is easily observable).  Therefore Game Science die passes the test of fairness fairly well.  The Black D20, however, is a lot more problematic.  I will be testing more of my dice to see if I observe similar results.  In the mean time, however, I think the black D20 will be the designated "hit the bad guys" die. 

Philosophizing About Probability

Over at Digital Orc he posted a video arguing about the distinction between "experimental" probability versus "theoretical" probability.  I disagree with his definitions.  Probability is always theoretical.  What he calls an "experimental" probability is better be described as an estimate of the probability subject to an experimental uncertainty.  From the rules of error propagation, it is actually possible to derive a formula for the uncertainty associated with one's estimation of a probability. 

Exactly how this distinction would be valuable to gamers is unclear to me, except maybe for assessing the fairness of one's dice.  Even then, though, to really do it properly you'd need to invoke Bayes theorem and calculate the probability that the dice are fair given that you rolled some number of successes.

What's valuable to gamers isn't a largely philosophical discussion of the nature of probability.  What's more useful is concrete tactical advice based on mathematical analysis.  That's more difficult, or at least more tedious.  Let's face it, multiplying all the little probabilities together to make the probability tree for a combat round is mind numbingly dull.  I still haven't filled out that darned transition matrix.  Maybe I should just give up and write the Monte Carlo.  Or maybe I should write a program for populating the matrix.  Huuuum...