1.5: Bayesian Statistical Inference

\( \newcommand\bes{\begin{equation}\begin{split}}\)
\( \newcommand\ltwid{\propto}\)
\( \newcommand\ees{\end{split}\end{equation}}\)
\( \newcommand\mib{\mathbf}\)
\( \newcommand\Sa{\textsf a}\)
\( \newcommand\Sb{\textsf b}\)
\( \newcommand\Sc{\textsf c}\)
\( \newcommand\Sd{\textsf d}\)
\( \newcommand\Se{\textsf e}\)
\( \newcommand\Sf{\textsf f}\)
\( \newcommand\Sg{\textsf g}\)
\( \newcommand\Sh{\textsf h}\)
\( \newcommand\Si{\textsf i}\)
\( \newcommand\Sj{\textsf j}\)
\( \newcommand\Sk{\textsf k}\)
\( \newcommand\Sl{\textsf l}\)
\( \newcommand\Sm{\textsf m}\)
\( \newcommand\Sn{\textsf n}\)
\( \newcommand\So{\textsf o}\)
\( \newcommand\Sp{\textsf p}\)
\( \newcommand\Sq{\textsf q}\)
\( \newcommand\Sr{\textsf r}\)
\( \newcommand\Ss{\textsf s}\)
\( \newcommand\St{\textsf t}\)
\( \newcommand\Su{\textsf u}\)
\( \newcommand\Sv{\textsf v}\)
\( \newcommand\Sw{\textsf w}\)
\( \newcommand\Sx{\textsf x}\)
\( \newcommand\Sy{\textsf y}\)
\( \newcommand\Sz{\textsf z}\)
\( \newcommand\SA{\textsf A}\)
\( \newcommand\SB{\textsf B}\)
\( \newcommand\SC{\textsf C}\)
\( \newcommand\SD{\textsf D}\)
\( \newcommand\SE{\textsf E}\)
\( \newcommand\SF{\textsf F}\)
\( \newcommand\SG{\textsf G}\)
\( \newcommand\SH{\textsf H}\)
\( \newcommand\SI{\textsf I}\)
\( \newcommand\SJ{\textsf J}\)
\( \newcommand\SK{\textsf K}\)
\( \newcommand\SL{\textsf L}\)
\( \newcommand\SM{\textsf M}\)
\( \newcommand\SN{\textsf N}\)
\( \newcommand\SO{\textsf O}\)
\( \newcommand\SP{\textsf P}\)
\( \newcommand\SQ{\textsf Q}\)
\( \newcommand\SR{\textsf R}\)
\( \newcommand\SS{\textsf S}\)
\( \newcommand\ST{\textsf T}\)
\( \newcommand\SU{\textsf U}\)
\( \newcommand\SV{\textsf V}\)
\( \newcommand\SW{\textsf W}\)
\( \newcommand\SX{\textsf X}\)
\( \newcommand\SY{\textsf Y}\)
\( \newcommand\SZ{\textsf Z}\)
\( \newcommand\Ha{\hat a}\)
\( \newcommand\Hb{\hat b}\)
\( \newcommand\Hc{\hat c}\)
\( \newcommand\Hd{\hat d}\)
\( \newcommand\He{\hat e}\)
\( \newcommand\Hf{\hat f}\)
\( \newcommand\Hg{\hat g}\)
\( \newcommand\Hh{\hat h}\)
\( \newcommand\Hi{\hat \imath}\)
\( \newcommand\Hj{\hat \jmath}\)
\( \newcommand\Hk{\hat k}\)
\( \newcommand\Hl{\hat l}\)
\( \newcommand\Hm{\hat m}\)
\( \newcommand\Hn{\hat n}\)
\( \newcommand\Ho{\hat o}\)
\( \newcommand\Hp{\hat p}\)
\( \newcommand\Hq{\hat q}\)
\( \newcommand\Hr{\hat r}\)
\( \newcommand\Hs{\hat s}\)
\( \newcommand\Ht{\hat t}\)
\( \newcommand\Hu{\hat u}\)
\( \newcommand\Hv{\hat v}\)
\( \newcommand\Hw{\hat w}\)
\( \newcommand\Hx{\hat x}\)
\( \newcommand\Hy{\hat y}\)
\( \newcommand\Hz{\hat z}\)
\( \newcommand\HA{\hat A}\)
\( \newcommand\HB{\hat B}\)
\( \newcommand\HC{\hat C}\)
\( \newcommand\HD{\hat D}\)
\( \newcommand\HE{\hat E}\)
\( \newcommand\HF{\hat F}\)
\( \newcommand\HG{\hat G}\)
\( \newcommand\HH{\hat H}\)
\( \newcommand\HI{\hat I}\)
\( \newcommand\HJ{\hat J}\)
\( \newcommand\HK{\hat K}\)
\( \newcommand\HL{\hat L}\)
\( \newcommand\HM{\hat M}\)
\( \newcommand\HN{\hat N}\)
\( \newcommand\HO{\hat O}\)
\( \newcommand\HP{\hat P}\)
\( \newcommand\HQ{\hat Q}\)
\( \newcommand\HR{\hat R}\)
\( \newcommand\HS{\hat S}\)
\( \newcommand\HT{\hat T}\)
\( \newcommand\HU{\hat U}\)
\( \newcommand\HV{\hat V}\)
\( \newcommand\HW{\hat W}\)
\( \newcommand\HX{\hat X}\)
\( \newcommand\HY{\hat Y}\)
\( \newcommand\HZ{\hat Z}\)
\( \newcommand\Halpha{\hat\alpha}\)
\( \newcommand\Hbeta{\hat\beta}\)
\( \newcommand\Hgamma{\hat\gamma}\)
\( \newcommand\Hdelta{\hat\delta}\)
\( \newcommand\Hepsilon{\hat\epsilon}\)
\( \newcommand\Hvarepsilon{\hat\varepsilon}\)
\( \newcommand\Hzeta{\hat\zeta}\)
\( \newcommand\Heta{\hat\eta}\)
\( \newcommand\Htheta{\hat\theta}\)
\( \newcommand\Hvartheta{\hat\vartheta}\)
\( \newcommand\Hiota{\hat\iota}\)
\( \newcommand\Hkappa{\hat\kappa}\)
\( \newcommand\Hlambda{\hat\lambda}\)
\( \newcommand\Hmu{\hat\mu}\)
\( \newcommand\Hnu{\hat\nu}\)
\( \newcommand\Hxi{\hat\xi}\)
\( \newcommand\Hom{\hat\omicron}\)
\( \newcommand\Hpi{\hat\pi}\)
\( \newcommand\Hvarpi{\hat\varpi}\)
\( \newcommand\Hrho{\hat\rho}\)
\( \newcommand\Hvarrho{\hat\varrho}\)
\( \newcommand\Hsigma{\hat\sigma}\)
\( \newcommand\Hvarsigma{\hat\varsigma}\)
\( \newcommand\Htau{\var\tau}\)
\( \newcommand\Hupsilon{\hat\upsilon}\)
\( \newcommand\Hphi{\hat\phi}\)
\( \newcommand\Hvarphi{\hat\varphi}\)
\( \newcommand\Hchi{\hat\chi}\)
\( \newcommand\Hxhi{\hat\xhi}\)
\( \newcommand\Hpsi{\hat\psi}\)
\( \newcommand\Homega{\hat\omega}\)
\( \newcommand\HGamma{\hat\Gamma}\)
\( \newcommand\HDelta{\hat\Delta}\)
\( \newcommand\HTheta{\hat\Theta}\)
\( \newcommand\HLambda{\hat\Lambda}\)
\( \newcommand\HXi{\hat\Xi}\)
\( \newcommand\HPi{\hat\Pi}\)
\( \newcommand\HSigma{\hat\Sigma}\)
\( \newcommand\HUps{\hat\Upsilon}\)
\( \newcommand\HPhi{\hat\Phi}\)
\( \newcommand\HPsi{\hat\Psi}\)
\( \newcommand\HOmega{\hat\Omega}\)
\( \newcommand\xhat{\hat\Bx}\)
\( \newcommand\yhat{\hat\By}\)
\( \newcommand\zhat{\hat\Bz}\)
\( \newcommand\ehat{\hat\Be}\)
\( \newcommand\khat{\hat\Bk}\)
\( \newcommand\nhat{\hat\Bn}\)
\( \newcommand\rhat{\hat\Br}\)
\( \newcommand\phihat{\hat\Bphi}\)
\( \newcommand\thetahat{\hat\Btheta}\)
\( \newcommand\MA{\mathbb A}\)
\( \newcommand\MB{\mathbb B}\)
\( \newcommand\MC{\mathbb C}\)
\( \newcommand\MD{\mathbb D}\)
\( \newcommand\ME{\mathbb E}\)
\( \newcommand\MF{\mathbb F}\)
\( \newcommand\MG{\mathbb G}\)
\( \newcommand\MH{\mathbb H}\)
\( \newcommand\MI{\mathbb I}\)
\( \newcommand\MJ{\mathbb J}\)
\( \newcommand\MK{\mathbb K}\)
\( \newcommand\ML{\mathbb L}\)
\( \newcommand\MM{\mathbb M}\)
\( \newcommand\MN{\mathbb N}\)
\( \newcommand\MO{\mathbb O}\)
\( \newcommand\MP{\mathbb P}\)
\( \newcommand\MQ{\mathbb Q}\)
\( \newcommand\MR{\mathbb R}\)
\( \newcommand\MS{\mathbb S}\)
\( \newcommand\MT{\mathbb T}\)
\( \newcommand\MU{\mathbb U}\)
\( \newcommand\MV{\mathbb V}\)
\( \newcommand\MW{\mathbb W}\)
\( \newcommand\MX{\mathbb X}\)
\( \newcommand\MY{\mathbb Y}\)
\( \newcommand\MZ{\mathbb Z}\)
\( \newcommand\CA{\mathcal A}\)
\( \newcommand\CB{\mathcal B}\)
\( \newcommand\CC{\mathcal C}\)
\( \newcommand\CD{\mathcal D}\)
\( \newcommand\CE{\mathcal E}\)
\( \newcommand\CF{\mathcal F}\)
\( \newcommand\CG{\mathcal G}\)
\( \newcommand\CH{\mathcal H}\)
\( \newcommand\CI{\mathcal I}\)
\( \newcommand\CJ{\mathcal J}\)
\( \newcommand\CK{\mathcal K}\)
\( \newcommand\CL{\mathcal L}\)
\( \newcommand\CM{\mathcal M}\)
\( \newcommand\CN{\mathcal N}\)
\( \newcommand\CO{\mathcal O}\)
\( \newcommand\CP{\mathcal P}\)
\( \newcommand\CQ{\mathcal Q}\)
\( \newcommand\CR{\mathcal R}\)
\( \newcommand\CS{\mathcal S}\)
\( \newcommand\CT{\mathcal T}\)
\( \newcommand\CU{\mathcal U}\)
\( \newcommand\CV{\mathcal V}\)
\( \newcommand\CW{\mathcal W}\)
\( \newcommand\CX{\mathcal X}\)
\( \newcommand\CY{\mathcal Y}\)
\( \newcommand\CZ{\mathcal Z}\)
\( \newcommand\Fa{\mathfrak a}\)
\( \newcommand\Fb{\mathfrak b}\)
\( \newcommand\Fc{\mathfrak c}\)
\( \newcommand\Fd{\mathfrak d}\)
\( \newcommand\Fe{\mathfrak e}\)
\( \newcommand\Ff{\mathfrak f}\)
\( \newcommand\Fg{\mathfrak g}\)
\( \newcommand\Fh{\mathfrak h}\)
\( \newcommand\Fi{\mathfrak i}\)
\( \newcommand\Fj{\mathfrak j}\)
\( \newcommand\Fk{\mathfrak k}\)
\( \newcommand\Fl{\mathfrak l}\)
\( \newcommand\Fm{\mathfrak m}\)
\( \newcommand\Fn{\mathfrak n}\)
\( \newcommand\Fo{\mathfrak o}\)
\( \newcommand\Fp{\mathfrak p}\)
\( \newcommand\Fq{\mathfrak q}\)
\( \newcommand\Fr{\mathfrak r}\)
\( \newcommand\Fs{\mathfrak s}\)
\( \newcommand\Ft{\mathfrak t}\)
\( \newcommand\Fu{\mathfrak u}\)
\( \newcommand\Fv{\mathfrak v}\)
\( \newcommand\Fw{\mathfrak w}\)
\( \newcommand\Fx{\mathfrak x}\)
\( \newcommand\Fy{\mathfrak y}\)
\( \newcommand\Fz{\mathfrak z}\)
\( \newcommand\FA{\mathfrak A}\)
\( \newcommand\FB{\mathfrak B}\)
\( \newcommand\FC{\mathfrak C}\)
\( \newcommand\FD{\mathfrak D}\)
\( \newcommand\FE{\mathfrak E}\)
\( \newcommand\FF{\mathfrak F}\)
\( \newcommand\FG{\mathfrak G}\)
\( \newcommand\FH{\mathfrak H}\)
\( \newcommand\FI{\mathfrak I}\)
\( \newcommand\FJ{\mathfrak J}\)
\( \newcommand\FK{\mathfrak K}\)
\( \newcommand\FL{\mathfrak L}\)
\( \newcommand\FM{\mathfrak M}\)
\( \newcommand\FN{\mathfrak N}\)
\( \newcommand\FO{\mathfrak O}\)
\( \newcommand\FP{\mathfrak P}\)
\( \newcommand\FQ{\mathfrak Q}\)
\( \newcommand\FR{\mathfrak R}\)
\( \newcommand\FS{\mathfrak S}\)
\( \newcommand\FT{\mathfrak T}\)
\( \newcommand\FU{\mathfrak U}\)
\( \newcommand\FV{\mathfrak V}\)
\( \newcommand\FW{\mathfrak W}\)
\( \newcommand\FX{\mathfrak X}\)
\( \newcommand\FY{\mathfrak Y}\)
\( \newcommand\FZ{\mathfrak Z}\)
\( \newcommand\Da{\dot a}\)
\( \newcommand\Db{\dot b}\)
\( \newcommand\Dc{\dot c}\)
\( \newcommand\Dd{\dot d}\)
\( \newcommand\De{\dot e}\)
\( \newcommand\Df{\dot f}\)
\( \newcommand\Dg{\dot g}\)
\( \newcommand\Dh{\dot h}\)
\( \newcommand\Di{\dot \imath}\)
\( \newcommand\Dj{\dot \jmath}\)
\( \newcommand\Dk{\dot k}\)
\( \newcommand\Dl{\dot l}\)
\( \newcommand\Dm{\dot m}\)
\( \newcommand\Dn{\dot n}\)
\( \newcommand\Do{\dot o}\)
\( \newcommand\Dp{\dot p}\)
\( \newcommand\Dq{\dot q}\)
\( \newcommand\Dr{\dot r}\)
\( \newcommand\Ds{\dot s}\)
\( \newcommand\Dt{\dot t}\)
\( \newcommand\Du{\dot u}\)
\( \newcommand\Dv{\dot v}\)
\( \newcommand\Dw{\dot w}\)
\( \newcommand\Dx{\dot x}\)
\( \newcommand\Dy{\dot y}\)
\( \newcommand\Dz{\dot z}\)
\( \newcommand\DA{\dot A}\)
\( \newcommand\DB{\dot B}\)
\( \newcommand\DC{\dot C}\)
\( \newcommand\DD{\dot D}\)
\( \newcommand\DE{\dot E}\)
\( \newcommand\DF{\dot F}\)
\( \newcommand\DG{\dot G}\)
\( \newcommand\DH{\dot H}\)
\( \newcommand\DI{\dot I}\)
\( \newcommand\DJ{\dot J}\)
\( \newcommand\DK{\dot K}\)
\( \newcommand\DL{\dot L}\)
\( \newcommand\DM{\dot M}\)
\( \newcommand\DN{\dot N}\)
\( \newcommand\DO{\dot O}\)
\( \newcommand\DP{\dot P}\)
\( \newcommand\DQ{\dot Q}\)
\( \newcommand\DR{\dot R}\)
\( \newcommand\DS{\dot S}\)
\( \newcommand\DT{\dot T}\)
\( \newcommand\DU{\dot U}\)
\( \newcommand\DV{\dot V}\)
\( \newcommand\DW{\dot W}\)
\( \newcommand\DX{\dot X}\)
\( \newcommand\DY{\dot Y}\)
\( \newcommand\DZ{\dot Z}\)
\( \newcommand\Dalpha

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[1], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dbeta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[2], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dgamma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[3], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Ddelta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[4], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Depsilon

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[5], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dvarepsilon

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[6], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dzeta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[7], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Deta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[8], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dtheta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[9], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dvartheta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[10], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Diota

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[11], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dkappa

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[12], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dlambda

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[13], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dmu{\dot\mu}\)
\( \newcommand\Dnu{\dot\nu}\)
\( \newcommand\Dxi{\dot\xi}\)
\( \newcommand\Dom{\dot\omicron}\)
\( \newcommand\Dpi{\dot\pi}\)
\( \newcommand\Dvarpi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[14], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Drho{\dot\rho}\)
\( \newcommand\Dvarrho{\dot\varrho}\)
\( \newcommand\Dsigma{\dot\sigma}\)
\( \newcommand\Dvarsigma{\dot\varsigma}\)
\( \newcommand\Dtau{\var\tau}\)
\( \newcommand\Dupsilon{\dot\upsilon}\)
\( \newcommand\Dphi{\dot\phi}\)
\( \newcommand\Dvarphi{\dot\varphi}\)
\( \newcommand\Dchi{\dot\chi}\)
\( \newcommand\Dpsi{\dot\psi}\)
\( \newcommand\Domega{\dot\omega}\)
\( \newcommand\DGamma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[15], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\DDelta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[16], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\DTheta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[17], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\DLambda{\dot\Lambda}\)
\( \newcommand\DXi{\dot\Xi}\)
\( \newcommand\DPi{\dot\Pi}\)
\( \newcommand\DSigma{\dot\Sigma}\)
\( \newcommand\DUps{\dot\Upsilon}\)
\( \newcommand\DPhi{\dot\Phi}\)
\( \newcommand\DPsi{\dot\Psi}\)
\( \newcommand\DOmega{\dot\Omega}\)
\( \newcommand\Va{\vec a}\)
\( \newcommand\Vb{\vec b}\)
\( \newcommand\Vc{\vec c}\)
\( \newcommand\Vd{\vec d}\)
\( \newcommand\Ve{\vec e}\)
\( \newcommand\Vf{\vec f}\)
\( \newcommand\Vg{\vec g}\)
\( \newcommand\Vh{\vec h}\)
\( \newcommand\Vi{\vec \imath}\)
\( \newcommand\Vj{\vec \jmath}\)
\( \newcommand\Vk{\vec k}\)
\( \newcommand\Vl{\vec l}\)
\( \newcommand\Vm{\vec m}\)
\( \newcommand\Vn{\vec n}\)
\( \newcommand\Vo{\vec o}\)
\( \newcommand\Vp{\vec p}\)
\( \newcommand\Vq{\vec q}\)
\( \newcommand\Vr{\vec r}\)
\( \newcommand\Vs{\vec s}\)
\( \newcommand\Vt{\vec t}\)
\( \newcommand\Vu{\vec u}\)
\( \newcommand\Vv{\vec v}\)
\( \newcommand\Vw{\vec w}\)
\( \newcommand\Vx{\vec x}\)
\( \newcommand\Vy{\vec y}\)
\( \newcommand\Vz{\vec z}\)
\( \newcommand\VA{\vec A}\)
\( \newcommand\VB{\vec B}\)
\( \newcommand\VC{\vec C}\)
\( \newcommand\VD{\vec D}\)
\( \newcommand\VE{\vec E}\)
\( \newcommand\VF{\vec F}\)
\( \newcommand\VG{\vec G}\)
\( \newcommand\VH{\vec H}\)
\( \newcommand\VI{\vec I}\)
\( \newcommand\VJ{\vec J}\)
\( \newcommand\VK{\vec K}\)
\( \newcommand\VL{\vec L}\)
\( \newcommand\VM{\vec M}\)
\( \newcommand\VN{\vec N}\)
\( \newcommand\VO{\vec O}\)
\( \newcommand\VP{\vec P}\)
\( \newcommand\VQ{\vec Q}\)
\( \newcommand\VR{\vec R}\)
\( \newcommand\VS{\vec S}\)
\( \newcommand\VT{\vec T}\)
\( \newcommand\VU{\vec U}\)
\( \newcommand\VV{\vec V}\)
\( \newcommand\VW{\vec W}\)
\( \newcommand\VX{\vec X}\)
\( \newcommand\VY{\vec Y}\)
\( \newcommand\VZ{\vec Z}\)
\( \newcommand\Valpha{\vec\alpha}\)
\( \newcommand\Vbeta{\vec\beta}\)
\( \newcommand\Vgamma{\vec\gamma}\)
\( \newcommand\Vdelta{\vec\delta}\)
\( \newcommand\Vepsilon{\vec\epsilon}\)
\( \newcommand\Vvarepsilon{\vec\varepsilon}\)
\( \newcommand\Vzeta{\vec\zeta}\)
\( \newcommand\Veta{\vec\eta}\)
\( \newcommand\Vtheta{\vec\theta}\)
\( \newcommand\Vvartheta{\vec\vartheta}\)
\( \newcommand\Viota{\vec\iota}\)
\( \newcommand\Vkappa{\vec\kappa}\)
\( \newcommand\Vlambda{\vec\lambda}\)
\( \newcommand\Vmu

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[18], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vnu

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[19], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vxi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[20], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vom

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[21], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vpi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[22], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarpi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[23], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vrho

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[24], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarrho

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[25], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vsigma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[26], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarsigma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[27], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vtau

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[28], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vupsilon

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[29], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vphi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[30], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarphi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[31], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vchi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[32], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vpsi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[33], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vomega

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[34], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\VGamma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[35], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\VDelta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[36], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\VTheta{\vec\Theta}\)
\( \newcommand\VLambda{\vec\Lambda}\)
\( \newcommand\VXi{\vec\Xi}\)
\( \newcommand\VPi{\vec\Pi}\)
\( \newcommand\VSigma{\vec\Sigma}\)
\( \newcommand\VUps{\vec\Upsilon}\)
\( \newcommand\VPhi{\vec\Phi}\)
\( \newcommand\VPsi{\vec\Psi}\)
\( \newcommand\VOmega{\vec\Omega}\)
\( \newcommand\BA{\mib A}\)
\( \newcommand\BB{\mib B}\)
\( \newcommand\BC{\mib C}\)
\( \newcommand\BD{\mib D}\)
\( \newcommand\BE{\mib E}\)
\( \newcommand\BF{\mib F}\)
\( \newcommand\BG{\mib G}\)
\( \newcommand\BH{\mib H}\)
\( \newcommand\BI{\mib I}}\)
\( \newcommand\BJ{\mib J}\)
\( \newcommand\BK{\mib K}\)
\( \newcommand\BL{\mib L}\)
\( \newcommand\BM{\mib M}\)
\( \newcommand\BN{\mib N}\)
\( \newcommand\BO{\mib O}\)
\( \newcommand\BP{\mib P}\)
\( \newcommand\BQ{\mib Q}\)
\( \newcommand\BR{\mib R}\)
\( \newcommand\BS{\mib S}\)
\( \newcommand\BT{\mib T}\)
\( \newcommand\BU{\mib U}\)
\( \newcommand\BV{\mib V}\)
\( \newcommand\BW{\mib W}\)
\( \newcommand\BX{\mib X}\)
\( \newcommand\BY{\mib Y}\)
\( \newcommand\BZ{\mib Z}\)
\( \newcommand\Ba{\mib a}\)
\( \newcommand\Bb{\mib b}\)
\( \newcommand\Bc{\mib c}\)
\( \newcommand\Bd{\mib d}\)
\( \newcommand\Be{\mib e}\)
\( \newcommand\Bf{\mib f}\)
\( \newcommand\Bg{\mib g}\)
\( \newcommand\Bh{\mib h}\)
\( \newcommand\Bi{\mib i}\)
\( \newcommand\Bj{\mib j}\)
\( \newcommand\Bk{\mib k}\)
\( \newcommand\Bl{\mib l}\)
\( \newcommand\Bm{\mib m}\)
\( \newcommand\Bn{\mib n}\)
\( \newcommand\Bo{\mib o}\)
\( \newcommand\Bp{\mib p}\)
\( \newcommand\Bq{\mib q}\)
\( \newcommand\Br{\mib r}\)
\( \newcommand\Bs{\mib s}\)
\( \newcommand\Bt{\mib t}\)
\( \newcommand\Bu{\mib u}\)
\( \newcommand\Bv{\mib v}\)
\( \newcommand\Bw{\mib w}\)
\( \newcommand\Bx{\mib x}\)
\( \newcommand\By{\mib y}\)
\( \newcommand\Bz{\mib z}\)\)
\( \newcommand\vrh{\varrho}\)
\( \newcommand\vsig{\varsigma}\)
\( \newcommand\ups{\upsilon}\)
\( \newcommand\eps{\epsilon}\)
\( \newcommand\ve{\varepsilon}\)
\( \newcommand\vth{\vartheta}\)
\( \newcommand\vphi{\varphi}\)
\( \newcommand\xhi{\chi}\)
\( \newcommand\Ups{\Upsilon}\)
\( \newcommand\Balpha{\mib\alpha}\)
\( \newcommand\Bbeta{\mib\beta}\)
\( \newcommand\Bgamma{\mib\gamma}\)
\( \newcommand\Bdelta{\mib\delta}\)
\( \newcommand\Beps{\mib\epsilon}\)
\( \newcommand\Bve{\mib\varepsilon}\)
\( \newcommand\Bzeta{\mib\zeta}\)
\( \newcommand\Beta{\mib\eta}\)
\( \newcommand\Btheta{\mib\theta}\)
\( \newcommand\Bvth{\mib\vartheta}\)
\( \newcommand\Biota{\mib\iota}\)
\( \newcommand\Bkappa{\mib\kappa}\)
\( \newcommand\Blambda{\mib\lambda}\)
\( \newcommand\Bmu{\mib\mu}\)
\( \newcommand\Bnu{\mib\nu}\)
\( \newcommand\Bxi{\mib\xi}\)
\( \newcommand\Bom{\mib\omicron}\)
\( \newcommand\Bpi{\mib\pi}\)
\( \newcommand\Bvarpi{\mib\varpi}\)
\( \newcommand\Brho{\mib\rho}\)
\( \newcommand\Bvrh{\mib\varrho}\)
\( \newcommand\Bsigma{\mib\sigma}\)
\( \newcommand\Bvsig{\mib\varsigma}\)
\( \newcommand\Btau{\mib\tau}\)
\( \newcommand\Bups{\mib\upsilon}\)
\( \newcommand\Bphi{\mib\phi}\)
\( \newcommand\Bvphi{\mib\vphi}\)
\( \newcommand\Bchi{\mib\chi}\)
\( \newcommand\Bpsi{\mib\psi}\)
\( \newcommand\Bomega{\mib\omega}\)
\( \newcommand\BGamma{\mib\Gamma}\)
\( \newcommand\BDelta{\mib\Delta}\)
\( \newcommand\BTheta{\mib\Theta}\)
\( \newcommand\BLambda{\mib\Lambda}\)
\( \newcommand\BXi{\mib\Xi}\)
\( \newcommand\BPi{\mib\Pi}\)
\( \newcommand\BSigma{\mib\Sigma}\)
\( \newcommand\BUps{\mib\Upsilon}\)
\( \newcommand\BPhi{\mib\Phi}\)
\( \newcommand\BPsi{\mib\Psi}\)
\( \newcommand\BOmega{\mib\Omega}\)
\( \newcommand\Bxhi{\raise.35ex\hbox{$\Bchi$}}\)
\( \newcommand\RGamma{ \Gamma}\)
\( \newcommand\RDelta{ \Delta}\)
\( \newcommand\RTheta{ \Theta}\)
\( \newcommand\RLambda{ \Lambda}\)
\( \newcommand\RXi{ \Xi}\)
\( \newcommand\RPi{ \Pi}\)
\( \newcommand\RSigma{ \Sigma}\)
\( \newcommand\RUps{ \Upsilon}\)
\( \newcommand\RPhi{ \Phi}\)
\( \newcommand\RPsi{ \Psi}\)
\( \newcommand\ROmega{ \Omega}\)
\( \newcommand\RA{ A}\)
\( \newcommand\RB{ B}\)
\( \newcommand\RC{ C}\)
\( \newcommand\RD{ D}\)
\( \newcommand\RE{ E}\)
\( \newcommand\RF{ F}\)
\( \newcommand\RG{ G}\)
\( \newcommand\RH{ H}\)
\( \newcommand\RI{ I}\)
\( \newcommand\RJ{ J}\)
\( \newcommand\RK{ K}\)
\( \newcommand\RL{ L}\)
\( \newcommand { M}\)
\( \newcommand\RN{ N}\)
\( \newcommand\RO{ O}\)
\( \newcommand\RP{ P}\)
\( \newcommand\RQ{ Q}\)
\( \newcommand\RR{ R}\)
\( \newcommand\RS{ S}\)
\( \newcommand\RT{ T}\)
\( \newcommand\RU{ U}\)
\( \newcommand\RV{ V}\)
\( \newcommand\RW{ W}\)
\( \newcommand\RX{ X}\)
\( \newcommand\RY{ Y}\)
\( \newcommand\RZ{ Z}\)
\( \newcommand\Ra{ a}\)
\( \newcommand\Rb{ b}\)
\( \newcommand\Rc{ c}\)
\( \newcommand\Rd{ d}\)
\( \newcommand\Re{ e}\)
\( \newcommand\Rf{ f}\)
\( \newcommand\Rg{ g}\)
\( \newcommand\Rh{ h}\)
\( \newcommand\Ri{ i}\)
\( \newcommand\Rj{ j}\)
\( \newcommand\Rk{ k}\)
\( \newcommand\Rl{ l}\)
\( \newcommand { m}\)
\( \newcommand\Rn{ n}\)
\( \newcommand\Ro{ o}\)
\( \newcommand\Rp{ p}\)
\( \newcommand\Rq{ q}\)
\( \newcommand\Rr{ r}\)
\( \newcommand\Rs{ s}\)
\( \newcommand\Rt{ t}\)
\( \newcommand\Ru{ u}\)
\( \newcommand\Rv{ v}\)
\( \newcommand\Rw{ w}\)
\( \newcommand\Rx{ x}\)
\( \newcommand\Ry{ y}\)
\( \newcommand\Rz{ z}\)
\( \newcommand\BBA{\boldsymbol\RA}\)
\( \newcommand\BBB{\boldsymbol\RB}\)
\( \newcommand\BBC{\boldsymbol\RC}\)
\( \newcommand\BBD{\boldsymbol\RD}\)
\( \newcommand\BBE{\boldsymbol\RE}\)
\( \newcommand\BBF{\boldsymbol\RF}\)
\( \newcommand\BBG{\boldsymbol\RG}\)
\( \newcommand\BBH{\boldsymbol\RH}\)
\( \newcommand\BBI{\boldsymbol\RI}\)
\( \newcommand\BBJ{\boldsymbol\RJ}\)
\( \newcommand\BBK{\boldsymbol\RK}\)
\( \newcommand\BBL{\boldsymbol\RL}\)
\( \newcommand\BBM{\boldsymbol }\)
\( \newcommand\BBN{\boldsymbol\RN}\)
\( \newcommand\BBO{\boldsymbol\RO}\)
\( \newcommand\BBP{\boldsymbol\RP}\)
\( \newcommand\BBQ{\boldsymbol\RQ}\)
\( \newcommand\BBR{\boldsymbol\RR}\)
\( \newcommand\BBS{\boldsymbol\RS}\)
\( \newcommand\BBT{\boldsymbol\RT}\)
\( \newcommand\BBU{\boldsymbol\RU}\)
\( \newcommand\BBV{\boldsymbol\RV}\)
\( \newcommand\BBW{\boldsymbol\RW}\)
\( \newcommand\BBX{\boldsymbol\RX}\)
\( \newcommand\BBY{\boldsymbol\RY}\)
\( \newcommand\BBZ{\boldsymbol\RZ}\)
\( \newcommand\BBa{\boldsymbol\Ra}\)
\( \newcommand\BBb{\boldsymbol\Rb}\)
\( \newcommand\BBc{\boldsymbol\Rc}\)
\( \newcommand\BBd{\boldsymbol\Rd}\)
\( \newcommand\BBe{\boldsymbol\Re}\)
\( \newcommand\BBf{\boldsymbol\Rf}\)
\( \newcommand\BBg{\boldsymbol\Rg}\)
\( \newcommand\BBh{\boldsymbol\Rh}\}\)
\( \newcommand\BBi{\boldsymbol\Ri}\)
\( \newcommand\BBj{\boldsymbol\Rj}\)
\( \newcommand\BBk{\boldsymbol\Rk}\)
\( \newcommand\BBl{boldsymbol\Rl}\)
\( \newcommand\BBm{\boldsymbol }\)
\( \newcommand\BBn{\boldsymbol\Rn}\)
\( \newcommand\BBo{\boldsymbol\Ro}\)
\( \newcommand\BBp{\boldsymbol\Rp}\)
\( \newcommand\BBq{\boldsymbol\Rq}\)
\( \newcommand\BBr{\boldsymbol\Rr}\)
\( \newcommand\BBs{\boldsymbol\Rs}\)
\( \newcommand\BBt{\boldsymbol\Rt}\)
\( \newcommand\BBu{\boldsymbol\Ru}\)
\( \newcommand\BBv{\boldsymbol\Rv}\)
\( \newcommand\BBw{\boldsymbol\Rw}\)
\( \newcommand\BBx{\boldsymbol\Rx}\)
\( \newcommand\BBy{\boldsymbol\Ry}\)
\( \newcommand\BBz{\boldsymbol\Rz}\)
\( \newcommand\tcb{\textcolor{blue}\)
\( \newcommand\tcr{\textcolor{red}\)
\( \newcommand\bnabla{\boldsymbol{\nabla}}\)
\( \newcommand\Bell{\boldsymbol\ell}\)
\( \newcommand\dbar{\,{\mathchar'26\mkern-12mu d}} \)
\( \newcommand\ns{^\vphantom{*}}\)
\( \newcommand\uar{\uparrow}\)
\( \newcommand\dar{\downarrow}\)
\( \newcommand\impi{\int\limits_{-\infty}^{\infty}\!\!}\)
\( \newcommand\izpi{\int\limits_{0}^{\infty}\!\!}\)
\( \newcommand\etc{\it etc.\/}\)
\( \newcommand\etal{\it et al.\/}\)
\( \newcommand\opcit{\it op. cit.\/}\)
\( \newcommand\ie{\it i.e.\/}\)
\( \newcommand\Ie{\it I.e.\/}\)
\( \newcommand\viz{\it viz.\/}\)
\( \newcommand\eg{\it e.g.\/}\)
\( \newcommand\Eg{\it E.g.\/}\)
\( \newcommand\dbar{\,{\mathchar'26\mkern-12mu d}} \)
\( \def\sss#1{\scriptscriptstyle #1}\)
\( \def\ss#1{\scriptstyle #1}\)
\( \def\ssr#1{\scriptstyle #1}\)
\( \def\ssf#1{\scriptstyle #1}\)
\( \newcommand\NA{N_{\ssr{\!A}}}\)
\( \newcommand\lala{\langle\!\langle}\)
\( \newcommand\rara{\rangle\!\rangle}\)
\( \newcommand\blan{\big\langle}\)
\( \newcommand\bran{\big\rangle}\)
\( \newcommand\Blan{\Big\langle}\)
\( \newcommand\Bran{\Big\rangle}\)
\( \newcommand\intl{\int\limits}\)
\( \newcommand\half{\frac{1}{2}}\)
\( \newcommand\third{\frac{1}{3}}\)
\( \newcommand\fourth{\frac{1}{4}}\)
\( \newcommand\eighth{\frac{1}{8}}\)
\( \newcommand\uar{\uparrow}\)
\( \newcommand\dar{\downarrow}\)
\( \newcommand\undertext#1{$\underline{\hbox{#1}}$}\)
\( \newcommand\Tra{\mathop{\textsf{Tr}}\,}\)
\( \newcommand\det{\mathop{\textsf{det}}\,}\)
\( \def\tket#1{|  #1 \rangle}\)
\( \def\tbra#1{\langle #1|}\)
\( \def\tbraket#1#2{\langle #1  |   #2 \rangle}\)
\( \def\texpect#1#2#3{\langle #1 |   #2  |  #3 \rangle}\)
\( \def\sket#1{|  \, #1 \,  \rangle}\)
\( \def\sbra#1{\langle \,  #1 \, |}\)
\( \def\sbraket#1#2{\langle \, #1  \, |  \, #2 \,  \rangle}\)
\( \def\sexpect#1#2#3{\langle \, #1 \, | \,  #2  \, | \, #3 \, \rangle}\)
\(\def\ket#1{\big| \, #1\, \big\rangle}\)
\( \def\bra#1{\big\langle \, #1 \, \big|}\)
\( \def\braket#1#2{\big\langle \, #1\, \big| \,#2 \,\big\rangle}\)
\( \def\expect#1#2#3{\big\langle\, #1\, \big|\, #2\, \big| \,#3\, \big\rangle}\)
\( \newcommand\pz{\partial}\)
\( \newcommand\pzb{\bar{\partial}}\)
\( \newcommand\svph{\vphantom{\int}}\)
\( \newcommand\vph{\vphantom{\sum_i}}\)
\( \newcommand\bvph{\vphantom{\sum_N^N}}\)
\( \newcommand\nd{^{\vphantom{\dagger}}}\)
\( \newcommand\ns{^{\vphantom{*}}}\)
\( \newcommand\yd{^\dagger}\)
\( \newcommand\zb{\bar z}\)
\( \newcommand\zdot{\dot z}\)
\( \newcommand\zbdot{\dot{\bar z}}\)
\( \newcommand\kB{k_{\sss{B}}}\)
\( \newcommand\kT{k_{\sss{B}}T}\)
\( \newcommand\gtau{g_\tau}\)
\( \newcommand\Htil{\tilde H}\)
\( \newcommand\pairo{(\phi\nd_0,J\nd_0)}\)
\( \newcommand\pairm{(\phi\nd_0,J)}\)
\( \newcommand\pairob{(\Bphi\nd_0,\BJ\nd_0)}\)
\( \newcommand\pairmb{(\Bphi\nd_0,\BJ)}\)
\( \newcommand\pair{(\phi,J)}\)
\( \newcommand\Hz{H\nd_0}\)
\( \newcommand\Ho{H\nd_1}\)
\( \newcommand\Htz{\Htil\nd_0}\)
\( \newcommand\Hto{\Htil\nd_1}\)
\( \newcommand\oc{\omega_\Rc}\)

\(\newcommand \gtwid{\approx}\)

\( \newcommand\index{\textsf{ind}}\)
\( \newcommand\csch{\,{ csch\,}}\)
\( \newcommand\ctnh{\,{ ctnh\,}}\)
\( \newcommand\ctn{\,{ ctn\,}}\)
\( \newcommand\sgn{\,{ sgn\,}}\)
\( \def\tmapright#1{\xrightarrow \limits^{#1}}\)
\( \def\bmapright#1{\xrightarrow\limits_{#1}}\)
\( \newcommand\hfb{\hfill\break}\)
\( \newcommand\Rep{\textsf{Re}\,}\)
\( \newcommand\Imp{\textsf{Im}\,}\)
\( \newcommand\ncdot{\!\cdot\!}\)
\( \def\tmapright#1{ \smash{\mathop{\hbox to 35pt{\rightarrowfill}}\limits^{#1}}\ }\)
\( \def\bmapright#1{ \smash{\mathop{\hbox to 35pt{\rightarrowfill}}\limits_{#1}}\ }\)
\( \newcommand\bsqcap{\mbox{\boldmath{$\sqcap$}}}\)

\( \def\pabc#1#2#3{\left({\pz #1\over\pz #2}\right)\ns_{\!\!#3}}\)
\( \def\spabc#1#2#3{\big({\pz #1\over\pz #2}\big)\ns_{\!#3}}\)
\( \def\qabc#1#2#3{\pz^2\! #1\over\pz #2\,\pz #3}\)
\( \def\rabc#1#2#3#4{(\pz #1,\pz #2)\over (\pz #3,\pz #4)}\)
\( \newcommand\subA{\ns_\ssr{A}}\)
\( \newcommand\subB{\ns_\ssr{B}}\)
\( \newcommand\subC{\ns_\ssr{C}}\)
\( \newcommand\subD{\ns_\ssr{D}}\)
\( \newcommand\subAB{\ns_\ssr{AB}}\)
\( \newcommand\subBC{\ns_\ssr{BC}}\)
\( \newcommand\subCD{\ns_\ssr{CD}}\)
\( \newcommand\subDA{\ns_\ssr{DA}}\)
\( \def\lmapright#1{\ \ \smash{\mathop{\hbox to 55pt{\rightarrowfill}}\limits^{#1}}\ \ }\)
\( \def\enth#1{\RDelta {\textsf H}^0_\Rf[{ #1}]}\)
\( \newcommand\longrightleftharpoons{ \mathop{\vcenter{\hbox{\ooalign{\raise1pt\hbox{$\longrightharpoonup\joinrel$}\crcr  \lower1pt\hbox{$\longleftharpoondown\joinrel$}}}}}}\)
\( \newcommand\longrightharpoonup{\relbar\joinrel\rightharpoonup}\)
\( \newcommand\longleftharpoondown{\leftharpoondown\joinrel\relbar}\)
\( \newcommand\cds{\,\bullet\,}\)
\( \newcommand\ccs{\,\circ\,}\)
\( \newcommand\nsub{_{\vphantom{\dagger}}}\)
\( \newcommand\rhohat{\hat\rho}\)
\( \newcommand\vrhhat{\hat\vrh}\)
\( \newcommand\impi{\int\limits_{-\infty}^\infty\!\!\!}\)
\( \newcommand\brangle{\big\rangle}\)
\( \newcommand\blangle{\big\langle}\)
\( \newcommand\vet{\tilde\ve}\)
\( \newcommand\zbar{\bar z}\)
\( \newcommand\ftil{\tilde f}\)
\( \newcommand\XBE{\RXi\ns_\ssr{BE}}\)
\( \newcommand\XFD{\RXi\ns_\ssr{FD}}\)
\( \newcommand\OBE{\Omega\ns_\ssr{BE}}\)
\( \newcommand\OFD{\Omega\ns_\ssr{FD}}\)
\( \newcommand\veF{\ve\ns_\RF}\)
\( \newcommand\kF{k\ns_\RF}\)
\( \newcommand\kFu{k\ns_{\RF\uar}}\)
\( \newcommand\SZ{\textsf Z}}\) \( \newcommand\kFd{k\ns_{\RF\dar}\)
\( \newcommand\muB{\mu\ns_\ssr{B}}\)
\( \newcommand\mutB{\tilde\mu}\ns_\ssr{B}\)
\( \newcommand\xoN{\Bx\ns_1\,,\,\ldots\,,\,\Bx\ns_N}\)
\( \newcommand\rok{\Br\ns_1\,,\,\ldots\,,\,\Br\ns_k}\)
\( \newcommand\xhiOZ{\xhi^\ssr{OZ}}\)
\( \newcommand\xhihOZ

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/span[1], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\jhz{\HJ(0)}\)
\( \newcommand\nda{\nd_\alpha}\)
\( \newcommand\ndap{\nd_{\alpha'}}\)
\( \newcommand\labar

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/span[2], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.05:_Bayesian_Statistical_Inference), /content/body/p/span, line 1, column 23

\)
\( \newcommand\msa{m\ns_\ssr{A}}\)
\( \newcommand\msb{m\ns_\ssr{B}}\)
\( \newcommand\mss{m\ns_\Rs}\)
\( \newcommand\HBx{\hat\Bx}\)
\( \newcommand\HBy{\hat\By}\)
\( \newcommand\HBz{\hat\Bz}\)
\( \newcommand\thm{\theta\ns_m}\)
\( \newcommand\thp{\theta\ns_\phi}\)
\( \newcommand\mtil{\widetilde m}\)
\( \newcommand\phitil{\widetilde\phi}\)
\( \newcommand\delf{\delta\! f}\)
\( \newcommand\coll{\bigg({\pz f\over\pz t}\bigg)\nd_{\! coll}}\)
\( \newcommand\stre{\bigg({\pz f\over\pz t}\bigg)\nd_{\! str}}\)
\( \newcommand\idrp{\int\!\!{d^3\!r\,d^3\!p\over h^3}\>}\)
\( \newcommand\vbar{\bar v}\)
\( \newcommand\BCE{\mbox{\boldmath{$\CE$}}\!}\)
\( \newcommand\BCR{\mbox{\boldmath{$\CR$}}\!}\)
\( \newcommand\gla{g\nd_{\RLambda\nd}}\)
\( \newcommand\TA{T\ns_\ssr{A}}\)
\( \newcommand\TB{T\ns_\ssr{B}}\)
\( \newcommand\ncdot{\!\cdot\!}\)
\( \newcommand\NS{N\ns_{\textsf S}}\)

Frequentists and Bayesians

There field of statistical inference is roughly divided into two schools of practice: frequentism and Bayesianism. You can find several articles on the web discussing the differences in these two approaches. In both cases we would like to model observable data \(\Bx\) by a distribution. The distribution in general depends on one or more parameters \(\Btheta\). The basic worldviews of the two approaches are as follows:

Frequentism: Data \(\Bx\) are a random sample drawn from an infinite pool at some frequency. The underlying parameters \(\Btheta\), which are to be estimated, remain fixed during this process. There is no information prior to the model specification. The experimental conditions under which the data are collected are presumed to be controlled and repeatable. Results are generally expressed in terms of confidence intervals and confidence levels, obtained via statistical hypothesis testing. Probabilities have meaning only for data yet to be collected. Calculations generally are computationally straightforward.

Bayesianism: The only data \(\Bx\) which matter are those which have been observed. The parameters \(\Btheta\) are unknown and described probabilistically using a prior distribution, which is generally based on some available information but which also may be at least partially subjective. The priors are then to be updated based on observed data \(\Bx\). Results are expressed in terms of posterior distributions and credible intervals. Calculations can be computationally intensive.

In essence, frequentists say the data are random and the parameters are fixed. while Bayesians say the data are fixed and the parameters are random¹¹. Overall, frequentism has dominated over the past several hundred years, but Bayesianism has been coming on strong of late, and many physicists seem naturally drawn to the Bayesian perspective.

Updating Bayesian priors

Given data \(D\) and a hypothesis \(H\), Bayes’ theorem tells us \[P(H|D)={P(D|H)\,P(H)\over P(D)}\ .\] Typically the data is in the form of a set of values \(\Bx=\{x\ns_1,\ldots,x\ns_N\}\), and the hypothesis in the form of a set of parameters \(\Btheta=\{\theta\ns_1,\ldots,\theta\ns_K\}\). It is notationally helpful to express distributions of \(\Bx\) and distributions of \(\Bx\) conditioned on \(\Btheta\) using the symbol \(f\), and distributions of \(\Btheta\) and distributions of \(\Btheta\) conditioned on \(\Bx\) using the symbol \(\pi\), rather than using the symbol \(P\) everywhere. We then have \[\pi(\Btheta|\Bx)={f(\Bx|\Btheta)\,\pi(\Btheta)\over\int\limits_\Theta\!d\Btheta'\> f(\Bx|\Btheta')\,\pi(\Btheta')}\ , \label{BayesPost}\] where \(\Theta\ni\Btheta\) is the space of parameters. Note that \(\int_\Theta\!d\Btheta \>\pi(\Btheta|\Bx)=1\). The denominator of the RHS is simply \(f(\Bx)\), which is independent of \(\Btheta\), hence \(\pi(\Btheta|\Bx)\propto f(\Bx|\Btheta)\,\pi(\Btheta)\). We call \(\pi(\Btheta)\) the prior for \(\Btheta\), \(f(\Bx|\Btheta)\) the likelihood of \(\Bx\) given \(\Btheta\), and \(\pi(\Btheta|\Bx)\) the posterior for \(\Btheta\) given \(\Bx\). The idea here is that while our initial guess at the \(\Btheta\) distribution is given by the prior \(\pi(\Btheta)\), after taking data, we should update this distribution to the posterior \(\pi(\Btheta|\Bx)\). The likelihood \(f(\Bx|\Btheta)\) is entailed by our model for the phenomenon which produces the data. We can use the posterior to find the distribution of new data points \(\By\), called the posterior predictive distribution, \[f(\By|\Bx)=\int\limits_\Theta\!\!d\Btheta\,f(\By|\Btheta)\,\pi(\Btheta|\Bx)\ .\] This is the update of the prior predictive distribution, \[f(\Bx)=\int\limits_\Theta\!\!d\Btheta\,f(\Bx|\Btheta)\,\pi(\Btheta)\ .\]

Example \(\PageIndex{1}\): Coin Flipping

Consider a model of coin flipping based on a standard Bernoulli distribution, where \(\theta\in [0,1]\) is the probability for heads (\(x=1\)) and \(1-\theta\) the probability for tails (\(x=0\)). That is, \[\begin{split} f(x\ns_1,\ldots,x\ns_N|\theta)&=\prod_{j=1}^N \Big[(1-\theta)\,\delta\ns_{x_j,0} + \theta\,\delta\ns_{x_j,1}\Big]\\ &=\theta^X (1-\theta)^{N-X}\ , \end{split}\] where \(X=\sum_{j=1}^N x\ns_j\) is the observed total number of heads, and \(N-X\) the corresponding number of tails. We now need a prior \(\pi(\theta)\). We choose the Beta distribution, \[\pi(\theta)={\theta^{\alpha-1}(1-\theta)^{\beta-1}\over \SB(\alpha,\beta)}\ ,\] where \(\SB(\alpha,\beta)=\RGamma(\alpha)\,\RGamma(\beta)/\RGamma(\alpha+\beta)\) is the Beta function. One can check that \(\pi(\theta)\) is normalized on the unit interval: \(\int_0^1\!d\theta\,\pi(\theta)=1\) for all positive \(\alpha,\beta\). Even if we limit ourselves to this form of the prior, different Bayesians might bring different assumptions about the values of \(\alpha\) and \(\beta\). Note that if we choose \(\alpha=\beta=1\), the prior distribution for \(\theta\) is flat, with \(\pi(\theta)=1\).

We now compute the posterior distribution for \(\theta\): \[\pi(\theta|x\ns_1,\ldots,x\ns_N)={f(x\ns_1,\ldots,x\ns_N|\theta)\,\pi(\theta)\over\int_0^1\!d\theta'\, f(x\ns_1,\ldots,x\ns_N|\theta')\,\pi(\theta')} ={\theta^{X+\alpha-1} (1-\theta)^{N-X+\beta-1}\over\SB(X+\alpha,N-X+\beta)}\ .\] Thus, we retain the form of the Beta distribution, but with updated parameters, \[\begin{split} \alpha'&=X+\alpha\\ \beta'&=N-X+\beta\ . \end{split}\] The fact that the functional form of the prior is retained by the posterior is generally not the case in Bayesian updating. We can also compute the prior predictive, \[\begin{split} f(x\ns_1,\ldots,x\ns_N)&=\int\limits_0^1\!d\theta\>f(x\ns_1,\ldots,x\ns_N|\theta)\,\pi(\theta)\\ &={1\over\SB(\alpha,\beta)}\int\limits_0^1\!d\theta\>\theta^{X+\alpha-1}(1-\theta)^{N-X+\beta-1} ={\SB(X+\alpha,N-X+\beta)\over\SB(\alpha,\beta)}\ . \end{split}\] The posterior predictive is then \[\begin{split} f(y\ns_1,\ldots,y\ns_M | x\ns_1,\ldots,x\ns_N) &= \int\limits_0^1\!\!d\theta\>f(y\ns_1,\ldots,y\ns_M|\theta)\, \pi(\theta|x\ns_1,\ldots,x\ns_N) \\ &={1\over\SB(X+\alpha,N-X+\beta)}\int\limits_0^1\!\!d\theta\>\theta^{X+Y+\alpha-1}(1-\theta)^{N-X+M-Y+\beta-1}\\ &={\SB(X+Y+\alpha,N-X+M-Y+\beta)\over\SB(X+\alpha,N-X+\beta)}\ . \end{split}\]

Hyperparameters and conjugate priors

In Example \(\PageIndex{1}\), \(\theta\) is a parameter of the Bernoulli distribution, the likelihood, while quantities \(\alpha\) and \(\beta\) are hyperparameters which enter the prior \(\pi(\theta)\). Accordingly, we could have written \(\pi(\theta|\alpha,\beta)\) for the prior. We then have for the posterior \[\pi(\Btheta|\Bx,\Balpha)={f(\Bx|\Btheta)\,\pi(\Btheta|\Balpha)\over\int\limits_\Theta\!d\Btheta'\> f(\Bx|\Btheta')\,\pi(\Btheta'|\Balpha)}\ ,\] replacing Equation [BayesPost], , where \(\Balpha\in A\) is the vector of hyperparameters. The hyperparameters can also be distributed, according to a hyperprior \(\rho(\Balpha)\), and the hyperpriors can further be parameterized by hyperhyperparameters, which can have their own distributions, ad nauseum.

What use is all this? We’ve already seen a compelling example: when the posterior is of the same form as the prior, the Bayesian update can be viewed as an automorphism of the hyperparameter space \(A\), one set of hyperparameters \(\Balpha\) is mapped to a new set of hyperparameters \({\widetilde\Balpha}\).

Definition: A parametric family of distributions \(\CP=\big\{\pi(\Btheta|\Balpha) \,|\, \Btheta\in\Theta,\, \Balpha\in A\big\}\) is called a conjugate family for a family of distributions \(\big\{f(\Bx|\Btheta)\,|\, \Bx\in\CX,\,\Btheta\in\Theta\big\}\) if, for all \(\Bx\in\CX\) and \(\Balpha\in A\), \[\pi(\Btheta|\Bx,\Balpha)\equiv{f(\Bx|\Btheta)\,\pi(\Btheta|\Balpha)\over\int\limits_\Theta\!d\Btheta'\,f(\Bx|\Btheta')\, \pi(\Btheta'|\Balpha)}\in\CP\ .\] That is, \(\pi(\Btheta|\Bx,\Balpha)=\pi(\Btheta|{\widetilde\Balpha})\) for some \({\widetilde\Balpha}\in A\), with \({\widetilde\Balpha}={\widetilde\Balpha}(\Balpha,\Bx)\).

As an example, consider the conjugate Bayesian analysis of the Gaussian distribution. We assume a likelihood \[f(\Bx|u,s)=(2\pi s^2)^{-N/2}\exp\!\Bigg\{\!-{1\over 2s^2}\sum_{j=1}^N (x\ns_j-u)^2\Bigg\}\ .\] The parameters here are \(\Btheta=\{u,s\}\). Now consider the prior distribution \[\pi(u,s|\mu\ns_0,\sigma\ns_0)=(2\pi\sigma_0^2)^{-1/2}\,\exp\!\Bigg\{\!-{(u-\mu_0)^2\over 2\sigma_0^2}\Bigg\}\ .\] Note that the prior distribution is independent of the parameter \(s\) and only depends on \(u\) and the hyperparameters \(\Balpha=(\mu\ns_0,\sigma\ns_0)\). We now compute the posterior: \[\begin{split} \pi(u,s|\Bx,\mu\ns_0,\sigma\ns_0)&\propto f(\Bx|u,s)\,\pi(u,s|\mu\ns_0,\sigma\ns_0)\\ &=\exp\Bigg\{\!-\bigg({1\over2\sigma_0^2}+{N\over2 s^2}\bigg) u^2 + \bigg({\mu\ns_0\over\sigma_0^2} + {N\langle x\rangle\over s^2}\bigg) u - \bigg({\mu_0^2\over 2\sigma_0^2} + {N\langle x^2\rangle\over 2s^2}\bigg)\Bigg\}\ , \end{split}\] with \(\langle x\rangle={1\over N}\sum_{j=1}^N x\ns_j\) and \(\langle x^2\rangle={1\over N}\sum_{j=1}^N x^2_j\). This is also a Gaussian distribution for \(u\), and after supplying the appropriate normalization one finds \[\pi(u,s|\Bx,\mu\ns_0,\sigma\ns_0)=(2\pi\sigma_1^2)^{-1/2} \exp\Bigg\{\!-{(u-\mu\ns_1)^2\over2\sigma_1^2}\Bigg\} \quad ,\] with \[\begin{split} \mu\ns_1&=\mu\ns_0+{N\big(\langle x\rangle-\mu\ns_0\big)\sigma_0^2\over s^2 + N\sigma_0^2} \\ \sigma_1^2&={s^2\sigma_0^2\over s^2+N\sigma_0^2}\quad . \end{split}\] Thus, the posterior is among the same family as the prior, and we have derived the update rule for the hyperparameters \((\mu\ns_0,\sigma\ns_0)\to (\mu\ns_1,\sigma\ns_1)\). Note that \(\sigma\ns_1 < \sigma\ns_0\) , so the updated Gaussian prior is sharper than the original. The updated mean \(\mu\ns_1\) shifts in the direction of \(\langle x\rangle\) obtained from the data set.

The problem with priors

We might think that the for the coin flipping problem, the flat prior \(\pi(\theta)=1\) is an appropriate initial one, since it does not privilege any value of \(\theta\). This prior therefore seems ’objective’ or ’unbiased’, also called ’uninformative’. But suppose we make a change of variables, mapping the interval \(\theta\in [0,1]\) to the entire real line according to \(\zeta=\ln\big[\theta/(1-\theta)\big]\). In terms of the new parameter \(\zeta\), we write the prior as \({\tilde\pi}(\zeta)\). Clearly \(\pi(\theta)\,d\theta = {\tilde\pi}(\zeta)\,d\zeta\), so \({\tilde\pi}(\zeta)=\pi(\theta)\,d\theta/d\zeta\). For our example, find \({\tilde\pi}(\zeta)= \fourth{sech}^2\!(\zeta/2)\), which is not flat. Thus what was uninformative in terms of \(\theta\) has become very informative in terms of the new parameter \(\zeta\). Is there any truly unbiased way of selecting a Bayesian prior?

One approach, advocated by E. T. Jaynes, is to choose the prior distribution \(\pi(\Btheta)\) according to the principle of maximum entropy. For continuous parameter spaces, we must first define a parameter space metric so as to be able to ’count’ the number of different parameter states. The entropy of a distribution \(\pi(\Btheta)\) is then dependent on this metric: \(S=-\int\!d\mu(\Btheta)\, \pi(\Btheta)\,\ln\pi(\Btheta)\).

Another approach, due to Jeffreys, is to derive a parameterization-independent prior from the likelihood \(f(\Bx|\Btheta)\) using the so-called Fisher information matrix, \[\begin{split} I\ns_{ij}(\Btheta)&=-\ME\ns_\theta \bigg(\!{\pz^2 \ln\! f(\Bx|\Btheta)\over\pz\theta\ns_i\,\pz\theta\ns_j}\bigg)\\ &=-\!\int\!\! d\Bx\> f(\Bx|\Btheta)\>{\pz^2 \ln\! f(\Bx|\Btheta)\over\pz\theta\ns_i\,\pz\theta\ns_j}\ . \end{split}\] The Jeffreys prior \(\pi\ns_\ssr{J}(\Btheta)\) is defined as \[\pi\ns_\ssr{J}(\Btheta)\propto\sqrt{\det\! I(\Btheta)}\ .\] One can check that the Jeffries prior is invariant under reparameterization. As an example, consider the Bernoulli process, for which \(\ln f(\Bx|\theta)=X\ln\theta + (N-X)\ln (1-\theta)\), where \(X=\sum_{j=1}^N x\ns_j\). Then \[-{d^2 \ln p(\Bx|\theta)\over d\theta^2}={X\over\theta^2}+{N-X\over (1-\theta)^2}\ ,\] and since \(\ME\ns_\theta\, X=N\theta\), we have \[I(\theta)={N\over\theta(1-\theta)}\qquad\Rightarrow\qquad \pi\ns_\ssr{J}(\theta)={1\over\pi}\,{1\over\sqrt{\theta(1-\theta)}}\ ,\] which felicitously corresponds to a Beta distribution with \(\alpha=\beta=\half\). In this example the Jeffries prior turned out to be a conjugate prior, but in general this is not the case.

We can try to implement the Jeffreys procedure for a two-parameter family where each \(x\ns_j\) is normally distributed with mean \(\mu\) and standard deviation \(\sigma\). Let the parameters be \((\theta\ns_1,\theta\ns_2)=(\mu,\sigma)\). Then \[-\ln f(\Bx|\Btheta)=N\ln\sqrt{2\pi} + N\ln\sigma + {1\over 2\sigma^2}\sum_{j=1}^N(x\ns_j-\mu)^2\ ,\] and the Fisher information matrix is \[I(\Btheta)=-{\pz^2\ln\!f(\Bx|\Btheta)\over\pz\theta\ns_i\,\pz\theta\ns_j} = \begin{pmatrix} N\sigma^{-2} & \sigma^{-3}\sum_j (x\ns_j-\mu) \\ & \\ \sigma^{-3}\sum_j (x\ns_j-\mu) & -N\sigma^{-2} + 3\sigma^{-4}\sum_j (x\ns_j-\mu)^2\end{pmatrix}\ .\] Taking the expectation value, we have \(\ME\, (x\ns_j-\mu)=0\) and \(\ME\,(x\ns_j-\mu)^2=\sigma^2\), hence \[\ME\, I(\Btheta)=\begin{pmatrix} N\sigma^{-2} & 0 \\ 0 & 2N\sigma^{-2} \end{pmatrix}\] and the Jeffries prior is \(\pi\ns_\ssr{J}(\mu,\sigma)\propto \sigma^{-2}\). This is problematic because if we choose a flat metric on the \((\mu,\sigma)\) upper half plane, the Jeffries prior is not normalizable. Note also that the Jeffreys prior no longer resembles a Gaussian, and hence is not a conjugate prior.

---

1. The exception is the unbiased case \(p=q=\half\), where \(\langle X\rangle=0\).↩
2. The origin of \(\CC\) lies in the \(\CO(\ln N)\) and \(\CO(N^0)\) terms in the asymptotic expansion of \(\ln N!\). We have ignored these terms here. Accounting for them carefully reproduces the correct value of \(\CC\) in Equation [normC].↩
3. The function \(s(x)\) is the specific entropy.↩
4. A function \(g(x)\) is concave if \(-g(x)\) is convex.↩
5. See ‘An Introduction to Information Theory and Entropy’ by T. Carter, Santa Fe Complex Systems Summer School, June 2011. Available online at astarte.csustan.edu/$\sim$tom/SFI-CSSS/info-theory/info-lec.pdf.↩
6. My colleague John McGreevy refers to \(I(p)\) as the surprise of observing an event which occurs with probability \(p\). I like this very much.↩
7. See §10 of An Enquiry Concerning Human Understanding by David Hume (1748).↩
8. Such a measure is invariant with respect to canonical transformations, which are the broad class of transformations among coordinates and momenta which leave Hamilton’s equations of motion invariant, and which preserve phase space volumes under Hamiltonian evolution. For this reason \(d\mu\) is called an invariant phase space measure.↩
9. Memorize this!↩
10. Jean Baptiste Joseph Fourier (1768-1830) had an illustrious career. The son of a tailor, and orphaned at age eight, Fourier’s ignoble status rendered him ineligible to receive a commission in the scientific corps of the French army. A Benedictine minister at the École Royale Militaire of Auxerre remarked, "Fourier, not being noble, could not enter the artillery, although he were a second Newton." Fourier prepared for the priesthood but his affinity for mathematics proved overwhelming, and so he left the abbey and soon thereafter accepted a military lectureship position. Despite his initial support for revolution in France, in 1794 Fourier ran afoul of a rival sect while on a trip to Orleans and was arrested and very nearly guillotined. Fortunately the Reign of Terror ended soon after the death of Robespierre, and Fourier was released. He went on Napoleon Bonaparte’s 1798 expedition to Egypt, where he was appointed governor of Lower Egypt. His organizational skills impressed Napoleon, and upon return to France he was appointed to a position of prefect in Grenoble. It was in Grenoble that Fourier performed his landmark studies of heat, and his famous work on partial differential equations and Fourier series. It seems that Fourier’s fascination with heat began in Egypt, where he developed an appreciation of desert climate. His fascination developed into an obsession, and he became convinced that heat could promote a healthy body. He would cover himself in blankets, like a mummy, in his heated apartment, even during the middle of summer. On May 4, 1830, Fourier, so arrayed, tripped and fell down a flight of stairs. This aggravated a developing heart condition, which he refused to treat with anything other than more heat. Two weeks later, he died. Fourier’s is one of the 72 names of scientists, engineers and other luminaries which are engraved on the Eiffel Tower.↩
11. "A frequentist is a person whose long-run ambition is to be wrong 5% of the time. A Bayesian is one who, vaguely expecting a horse, and catching glimpse of a donkey, strongly believes he has seen a mule." – Charles Annis.↩