我有一个方法，在给定的(M, N)次x创建2个不同的实例(math.random * x)该方法将创建对象M，其余时间对象N。
我编写了单元测试，模拟了随机数，因此我可以确保该方法的行为符合预期。但是我不确定如何（以及如果）测试概率是否准确，例如，如果x = 0.1我希望10个案例中有1个返回实例M.
我该如何测试此功能？
拆分测试。第一个测试应该允许你定义随机数生成器返回的内容（我假设你已经有了）。测试的这一部分只满足“如果随机数发生器会返回一些值，我会得到预期的结果”。
第二个测试应该使用一些统计分析函数运行随机数生成器（比如计算返回每个值的频率）。
我建议用一个返回“create M”和“create N”（或者可能只是0和1）的包装器包装真实的生成器。这样，您可以将实现与其使用的位置分开（创建两个不同实例的代码不需要知道如何初始化生成器或如何将实际结果转换为“create X”。
我将以Python的形式执行此操作。
首先描述您的功能：
def binomial_process(x):
'''
given a probability, x, return M with that probability,
else return N with probability 1-x
maybe: return random.random() > x
'''
然后测试此功能：
import random
def binom(x):
return random.random() > x
然后编写测试函数，首先是一个设置函数，用于汇总昂贵流程中的数据：
def setUp(x, n):
counter = dict()
for _ in range(n):
result = binom(x)
counter[result] = counter.get(result, 0) + 1
return counter
然后实际测试：
import scipy.stats
trials = 1000000
def test_binomial_process():
ps = (.01, .1, .33, .5, .66, .9, .99)
x_01 = setUp(.01, trials)
x_1 = setUp(.1, trials)
x_33 = setUp(.1, trials)
x_5 = setUp(.5, trials)
x_66 = setUp(.9, trials)
x_9 = setUp(.9, trials)
x_99 = setUp(.99, trials)
x_01_result = scipy.stats.binom_test(x_01.get(True, 0), trials, .01)
x_1_result = scipy.stats.binom_test(x_1.get(True, 0), trials, .1)
x_33_result = scipy.stats.binom_test(x_33.get(True, 0), trials, .33)
x_5_result = scipy.stats.binom_test(x_5.get(True, 0), trials)
x_66_result = scipy.stats.binom_test(x_66.get(True, 0), trials, .66)
x_9_result = scipy.stats.binom_test(x_9.get(True, 0), trials, .9)
x_99_result = scipy.stats.binom_test(x_99.get(True, 0), trials, .99)
setups = (x_01, x_1, x_33, x_5, x_66, x_9, x_99)
results = (x_01_result, x_1_result, x_33_result, x_5_result,
x_66_result, x_9_result, x_99_result)
print 'can reject the hypothesis that the following tests are NOT the'
print 'results of a binomial process (with their given respective'
print 'probabilities) with probability < .01, {0} trials each'.format(trials)
for p, setup, result in zip(ps, setups, results):
print 'p = {0}'.format(p), setup, result, 'reject null' if result < .01 else 'fail to reject'
然后编写你的函数（好吧，我们已经做过）：
def binom(x):
return random.random() > x
并运行您的测试：
test_binomial_process()
在最后的输出中给了我：
can reject the hypothesis that the following tests are NOT the
results of a binomial process (with their given respective
probabilities) with probability < .01, 1000000 trials each
p = 0.01 {False: 10084, True: 989916} 4.94065645841e-324 reject null
p = 0.1 {False: 100524, True: 899476} 1.48219693752e-323 reject null
p = 0.33 {False: 100633, True: 899367} 2.96439387505e-323 reject null
p = 0.5 {False: 500369, True: 499631} 0.461122365668 fail to reject
p = 0.66 {False: 900144, True: 99856} 2.96439387505e-323 reject null
p = 0.9 {False: 899988, True: 100012} 1.48219693752e-323 reject null
p = 0.99 {False: 989950, True: 10050} 4.94065645841e-324 reject null
为什么我们不能拒绝p = 0.5？让我们来看看scipy.stats.binom_test的帮助：
Help on function binom_test in module scipy.stats.morestats: binom_test(x, n=None, p=0.5, alternative='two-sided') Perform a test that the probability of success is p. This is an exact, two-sided test of the null hypothesis that the probability of success in a Bernoulli experiment is `p`. Parameters ---------- x : integer or array_like the number of successes, or if x has length 2, it is the number of successes and the number of failures. n : integer the number of trials. This is ignored if x gives both the number of successes and failures p : float, optional The hypothesized probability of success. 0 <= p <= 1. The default value is p = 0.5 alternative : {'two-sided', 'greater', 'less'}, optional Indicates the alternative hypothesis. The default value is 'two-sided'.
因此.5是测试的默认零假设，并且在这种情况下拒绝零假设是有道理的。