Skip to main content
\(\require{cancel}\)
Physics LibreTexts

1.2: Basic Concepts in Probability Theory

  • Page ID
    18542
  • \( \newcommand\bes{\begin{equation}\begin{split}}\)
    \( \newcommand\ltwid{\propto}\)
    \( \newcommand\ees{\end{split}\end{equation}}\)
    \( \newcommand\mib{\mathbf}\)
    \( \newcommand\Sa{\textsf a}\)
    \( \newcommand\Sb{\textsf b}\)
    \( \newcommand\Sc{\textsf c}\)
    \( \newcommand\Sd{\textsf d}\)
    \( \newcommand\Se{\textsf e}\)
    \( \newcommand\Sf{\textsf f}\)
    \( \newcommand\Sg{\textsf g}\)
    \( \newcommand\Sh{\textsf h}\)
    \( \newcommand\Si{\textsf i}\)
    \( \newcommand\Sj{\textsf j}\)
    \( \newcommand\Sk{\textsf k}\)
    \( \newcommand\Sl{\textsf l}\)
    \( \newcommand\Sm{\textsf m}\)
    \( \newcommand\Sn{\textsf n}\)
    \( \newcommand\So{\textsf o}\)
    \( \newcommand\Sp{\textsf p}\)
    \( \newcommand\Sq{\textsf q}\)
    \( \newcommand\Sr{\textsf r}\)
    \( \newcommand\Ss{\textsf s}\)
    \( \newcommand\St{\textsf t}\)
    \( \newcommand\Su{\textsf u}\)
    \( \newcommand\Sv{\textsf v}\)
    \( \newcommand\Sw{\textsf w}\)
    \( \newcommand\Sx{\textsf x}\)
    \( \newcommand\Sy{\textsf y}\)
    \( \newcommand\Sz{\textsf z}\)
    \( \newcommand\SA{\textsf A}\)
    \( \newcommand\SB{\textsf B}\)
    \( \newcommand\SC{\textsf C}\)
    \( \newcommand\SD{\textsf D}\)
    \( \newcommand\SE{\textsf E}\)
    \( \newcommand\SF{\textsf F}\)
    \( \newcommand\SG{\textsf G}\)
    \( \newcommand\SH{\textsf H}\)
    \( \newcommand\SI{\textsf I}\)
    \( \newcommand\SJ{\textsf J}\)
    \( \newcommand\SK{\textsf K}\)
    \( \newcommand\SL{\textsf L}\)
    \( \newcommand\SM{\textsf M}\)
    \( \newcommand\SN{\textsf N}\)
    \( \newcommand\SO{\textsf O}\)
    \( \newcommand\SP{\textsf P}\)
    \( \newcommand\SQ{\textsf Q}\)
    \( \newcommand\SR{\textsf R}\)
    \( \newcommand\SS{\textsf S}\)
    \( \newcommand\ST{\textsf T}\)
    \( \newcommand\SU{\textsf U}\)
    \( \newcommand\SV{\textsf V}\)
    \( \newcommand\SW{\textsf W}\)
    \( \newcommand\SX{\textsf X}\)
    \( \newcommand\SY{\textsf Y}\)
    \( \newcommand\SZ{\textsf Z}\)
    \( \newcommand\Ha{\hat a}\)
    \( \newcommand\Hb{\hat b}\)
    \( \newcommand\Hc{\hat c}\)
    \( \newcommand\Hd{\hat d}\)
    \( \newcommand\He{\hat e}\)
    \( \newcommand\Hf{\hat f}\)
    \( \newcommand\Hg{\hat g}\)
    \( \newcommand\Hh{\hat h}\)
    \( \newcommand\Hi{\hat \imath}\)
    \( \newcommand\Hj{\hat \jmath}\)
    \( \newcommand\Hk{\hat k}\)
    \( \newcommand\Hl{\hat l}\)
    \( \newcommand\Hm{\hat m}\)
    \( \newcommand\Hn{\hat n}\)
    \( \newcommand\Ho{\hat o}\)
    \( \newcommand\Hp{\hat p}\)
    \( \newcommand\Hq{\hat q}\)
    \( \newcommand\Hr{\hat r}\)
    \( \newcommand\Hs{\hat s}\)
    \( \newcommand\Ht{\hat t}\)
    \( \newcommand\Hu{\hat u}\)
    \( \newcommand\Hv{\hat v}\)
    \( \newcommand\Hw{\hat w}\)
    \( \newcommand\Hx{\hat x}\)
    \( \newcommand\Hy{\hat y}\)
    \( \newcommand\Hz{\hat z}\)
    \( \newcommand\HA{\hat A}\)
    \( \newcommand\HB{\hat B}\)
    \( \newcommand\HC{\hat C}\)
    \( \newcommand\HD{\hat D}\)
    \( \newcommand\HE{\hat E}\)
    \( \newcommand\HF{\hat F}\)
    \( \newcommand\HG{\hat G}\)
    \( \newcommand\HH{\hat H}\)
    \( \newcommand\HI{\hat I}\)
    \( \newcommand\HJ{\hat J}\)
    \( \newcommand\HK{\hat K}\)
    \( \newcommand\HL{\hat L}\)
    \( \newcommand\HM{\hat M}\)
    \( \newcommand\HN{\hat N}\)
    \( \newcommand\HO{\hat O}\)
    \( \newcommand\HP{\hat P}\)
    \( \newcommand\HQ{\hat Q}\)
    \( \newcommand\HR{\hat R}\)
    \( \newcommand\HS{\hat S}\)
    \( \newcommand\HT{\hat T}\)
    \( \newcommand\HU{\hat U}\)
    \( \newcommand\HV{\hat V}\)
    \( \newcommand\HW{\hat W}\)
    \( \newcommand\HX{\hat X}\)
    \( \newcommand\HY{\hat Y}\)
    \( \newcommand\HZ{\hat Z}\)
    \( \newcommand\Halpha{\hat\alpha}\)
    \( \newcommand\Hbeta{\hat\beta}\)
    \( \newcommand\Hgamma{\hat\gamma}\)
    \( \newcommand\Hdelta{\hat\delta}\)
    \( \newcommand\Hepsilon{\hat\epsilon}\)
    \( \newcommand\Hvarepsilon{\hat\varepsilon}\)
    \( \newcommand\Hzeta{\hat\zeta}\)
    \( \newcommand\Heta{\hat\eta}\)
    \( \newcommand\Htheta{\hat\theta}\)
    \( \newcommand\Hvartheta{\hat\vartheta}\)
    \( \newcommand\Hiota{\hat\iota}\)
    \( \newcommand\Hkappa{\hat\kappa}\)
    \( \newcommand\Hlambda{\hat\lambda}\)
    \( \newcommand\Hmu{\hat\mu}\)
    \( \newcommand\Hnu{\hat\nu}\)
    \( \newcommand\Hxi{\hat\xi}\)
    \( \newcommand\Hom{\hat\omicron}\)
    \( \newcommand\Hpi{\hat\pi}\)
    \( \newcommand\Hvarpi{\hat\varpi}\)
    \( \newcommand\Hrho{\hat\rho}\)
    \( \newcommand\Hvarrho{\hat\varrho}\)
    \( \newcommand\Hsigma{\hat\sigma}\)
    \( \newcommand\Hvarsigma{\hat\varsigma}\)
    \( \newcommand\Htau{\var\tau}\)
    \( \newcommand\Hupsilon{\hat\upsilon}\)
    \( \newcommand\Hphi{\hat\phi}\)
    \( \newcommand\Hvarphi{\hat\varphi}\)
    \( \newcommand\Hchi{\hat\chi}\)
    \( \newcommand\Hxhi{\hat\xhi}\)
    \( \newcommand\Hpsi{\hat\psi}\)
    \( \newcommand\Homega{\hat\omega}\)
    \( \newcommand\HGamma{\hat\Gamma}\)
    \( \newcommand\HDelta{\hat\Delta}\)
    \( \newcommand\HTheta{\hat\Theta}\)
    \( \newcommand\HLambda{\hat\Lambda}\)
    \( \newcommand\HXi{\hat\Xi}\)
    \( \newcommand\HPi{\hat\Pi}\)
    \( \newcommand\HSigma{\hat\Sigma}\)
    \( \newcommand\HUps{\hat\Upsilon}\)
    \( \newcommand\HPhi{\hat\Phi}\)
    \( \newcommand\HPsi{\hat\Psi}\)
    \( \newcommand\HOmega{\hat\Omega}\)
    \( \newcommand\xhat{\hat\Bx}\)
    \( \newcommand\yhat{\hat\By}\)
    \( \newcommand\zhat{\hat\Bz}\)
    \( \newcommand\ehat{\hat\Be}\)
    \( \newcommand\khat{\hat\Bk}\)
    \( \newcommand\nhat{\hat\Bn}\)
    \( \newcommand\rhat{\hat\Br}\)
    \( \newcommand\phihat{\hat\Bphi}\)
    \( \newcommand\thetahat{\hat\Btheta}\)
    \( \newcommand\MA{\mathbb A}\)
    \( \newcommand\MB{\mathbb B}\)
    \( \newcommand\MC{\mathbb C}\)
    \( \newcommand\MD{\mathbb D}\)
    \( \newcommand\ME{\mathbb E}\)
    \( \newcommand\MF{\mathbb F}\)
    \( \newcommand\MG{\mathbb G}\)
    \( \newcommand\MH{\mathbb H}\)
    \( \newcommand\MI{\mathbb I}\)
    \( \newcommand\MJ{\mathbb J}\)
    \( \newcommand\MK{\mathbb K}\)
    \( \newcommand\ML{\mathbb L}\)
    \( \newcommand\MM{\mathbb M}\)
    \( \newcommand\MN{\mathbb N}\)
    \( \newcommand\MO{\mathbb O}\)
    \( \newcommand\MP{\mathbb P}\)
    \( \newcommand\MQ{\mathbb Q}\)
    \( \newcommand\MR{\mathbb R}\)
    \( \newcommand\MS{\mathbb S}\)
    \( \newcommand\MT{\mathbb T}\)
    \( \newcommand\MU{\mathbb U}\)
    \( \newcommand\MV{\mathbb V}\)
    \( \newcommand\MW{\mathbb W}\)
    \( \newcommand\MX{\mathbb X}\)
    \( \newcommand\MY{\mathbb Y}\)
    \( \newcommand\MZ{\mathbb Z}\)
    \( \newcommand\CA{\mathcal A}\)
    \( \newcommand\CB{\mathcal B}\)
    \( \newcommand\CC{\mathcal C}\)
    \( \newcommand\CD{\mathcal D}\)
    \( \newcommand\CE{\mathcal E}\)
    \( \newcommand\CF{\mathcal F}\)
    \( \newcommand\CG{\mathcal G}\)
    \( \newcommand\CH{\mathcal H}\)
    \( \newcommand\CI{\mathcal I}\)
    \( \newcommand\CJ{\mathcal J}\)
    \( \newcommand\CK{\mathcal K}\)
    \( \newcommand\CL{\mathcal L}\)
    \( \newcommand\CM{\mathcal M}\)
    \( \newcommand\CN{\mathcal N}\)
    \( \newcommand\CO{\mathcal O}\)
    \( \newcommand\CP{\mathcal P}\)
    \( \newcommand\CQ{\mathcal Q}\)
    \( \newcommand\CR{\mathcal R}\)
    \( \newcommand\CS{\mathcal S}\)
    \( \newcommand\CT{\mathcal T}\)
    \( \newcommand\CU{\mathcal U}\)
    \( \newcommand\CV{\mathcal V}\)
    \( \newcommand\CW{\mathcal W}\)
    \( \newcommand\CX{\mathcal X}\)
    \( \newcommand\CY{\mathcal Y}\)
    \( \newcommand\CZ{\mathcal Z}\)
    \( \newcommand\Fa{\mathfrak a}\)
    \( \newcommand\Fb{\mathfrak b}\)
    \( \newcommand\Fc{\mathfrak c}\)
    \( \newcommand\Fd{\mathfrak d}\)
    \( \newcommand\Fe{\mathfrak e}\)
    \( \newcommand\Ff{\mathfrak f}\)
    \( \newcommand\Fg{\mathfrak g}\)
    \( \newcommand\Fh{\mathfrak h}\)
    \( \newcommand\Fi{\mathfrak i}\)
    \( \newcommand\Fj{\mathfrak j}\)
    \( \newcommand\Fk{\mathfrak k}\)
    \( \newcommand\Fl{\mathfrak l}\)
    \( \newcommand\Fm{\mathfrak m}\)
    \( \newcommand\Fn{\mathfrak n}\)
    \( \newcommand\Fo{\mathfrak o}\)
    \( \newcommand\Fp{\mathfrak p}\)
    \( \newcommand\Fq{\mathfrak q}\)
    \( \newcommand\Fr{\mathfrak r}\)
    \( \newcommand\Fs{\mathfrak s}\)
    \( \newcommand\Ft{\mathfrak t}\)
    \( \newcommand\Fu{\mathfrak u}\)
    \( \newcommand\Fv{\mathfrak v}\)
    \( \newcommand\Fw{\mathfrak w}\)
    \( \newcommand\Fx{\mathfrak x}\)
    \( \newcommand\Fy{\mathfrak y}\)
    \( \newcommand\Fz{\mathfrak z}\)
    \( \newcommand\FA{\mathfrak A}\)
    \( \newcommand\FB{\mathfrak B}\)
    \( \newcommand\FC{\mathfrak C}\)
    \( \newcommand\FD{\mathfrak D}\)
    \( \newcommand\FE{\mathfrak E}\)
    \( \newcommand\FF{\mathfrak F}\)
    \( \newcommand\FG{\mathfrak G}\)
    \( \newcommand\FH{\mathfrak H}\)
    \( \newcommand\FI{\mathfrak I}\)
    \( \newcommand\FJ{\mathfrak J}\)
    \( \newcommand\FK{\mathfrak K}\)
    \( \newcommand\FL{\mathfrak L}\)
    \( \newcommand\FM{\mathfrak M}\)
    \( \newcommand\FN{\mathfrak N}\)
    \( \newcommand\FO{\mathfrak O}\)
    \( \newcommand\FP{\mathfrak P}\)
    \( \newcommand\FQ{\mathfrak Q}\)
    \( \newcommand\FR{\mathfrak R}\)
    \( \newcommand\FS{\mathfrak S}\)
    \( \newcommand\FT{\mathfrak T}\)
    \( \newcommand\FU{\mathfrak U}\)
    \( \newcommand\FV{\mathfrak V}\)
    \( \newcommand\FW{\mathfrak W}\)
    \( \newcommand\FX{\mathfrak X}\)
    \( \newcommand\FY{\mathfrak Y}\)
    \( \newcommand\FZ{\mathfrak Z}\)
    \( \newcommand\Da{\dot a}\)
    \( \newcommand\Db{\dot b}\)
    \( \newcommand\Dc{\dot c}\)
    \( \newcommand\Dd{\dot d}\)
    \( \newcommand\De{\dot e}\)
    \( \newcommand\Df{\dot f}\)
    \( \newcommand\Dg{\dot g}\)
    \( \newcommand\Dh{\dot h}\)
    \( \newcommand\Di{\dot \imath}\)
    \( \newcommand\Dj{\dot \jmath}\)
    \( \newcommand\Dk{\dot k}\)
    \( \newcommand\Dl{\dot l}\)
    \( \newcommand\Dm{\dot m}\)
    \( \newcommand\Dn{\dot n}\)
    \( \newcommand\Do{\dot o}\)
    \( \newcommand\Dp{\dot p}\)
    \( \newcommand\Dq{\dot q}\)
    \( \newcommand\Dr{\dot r}\)
    \( \newcommand\Ds{\dot s}\)
    \( \newcommand\Dt{\dot t}\)
    \( \newcommand\Du{\dot u}\)
    \( \newcommand\Dv{\dot v}\)
    \( \newcommand\Dw{\dot w}\)
    \( \newcommand\Dx{\dot x}\)
    \( \newcommand\Dy{\dot y}\)
    \( \newcommand\Dz{\dot z}\)
    \( \newcommand\DA{\dot A}\)
    \( \newcommand\DB{\dot B}\)
    \( \newcommand\DC{\dot C}\)
    \( \newcommand\DD{\dot D}\)
    \( \newcommand\DE{\dot E}\)
    \( \newcommand\DF{\dot F}\)
    \( \newcommand\DG{\dot G}\)
    \( \newcommand\DH{\dot H}\)
    \( \newcommand\DI{\dot I}\)
    \( \newcommand\DJ{\dot J}\)
    \( \newcommand\DK{\dot K}\)
    \( \newcommand\DL{\dot L}\)
    \( \newcommand\DM{\dot M}\)
    \( \newcommand\DN{\dot N}\)
    \( \newcommand\DO{\dot O}\)
    \( \newcommand\DP{\dot P}\)
    \( \newcommand\DQ{\dot Q}\)
    \( \newcommand\DR{\dot R}\)
    \( \newcommand\DS{\dot S}\)
    \( \newcommand\DT{\dot T}\)
    \( \newcommand\DU{\dot U}\)
    \( \newcommand\DV{\dot V}\)
    \( \newcommand\DW{\dot W}\)
    \( \newcommand\DX{\dot X}\)
    \( \newcommand\DY{\dot Y}\)
    \( \newcommand\DZ{\dot Z}\)
    \( \newcommand\Dalpha

    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[1], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dbeta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[2], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dgamma
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[3], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Ddelta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[4], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Depsilon
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[5], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dvarepsilon
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[6], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dzeta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[7], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Deta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[8], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dtheta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[9], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dvartheta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[10], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Diota
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[11], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dkappa
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[12], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dlambda
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[13], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Dmu{\dot\mu}\)
    \( \newcommand\Dnu{\dot\nu}\)
    \( \newcommand\Dxi{\dot\xi}\)
    \( \newcommand\Dom{\dot\omicron}\)
    \( \newcommand\Dpi{\dot\pi}\)
    \( \newcommand\Dvarpi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[14], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Drho{\dot\rho}\)
    \( \newcommand\Dvarrho{\dot\varrho}\)
    \( \newcommand\Dsigma{\dot\sigma}\)
    \( \newcommand\Dvarsigma{\dot\varsigma}\)
    \( \newcommand\Dtau{\var\tau}\)
    \( \newcommand\Dupsilon{\dot\upsilon}\)
    \( \newcommand\Dphi{\dot\phi}\)
    \( \newcommand\Dvarphi{\dot\varphi}\)
    \( \newcommand\Dchi{\dot\chi}\)
    \( \newcommand\Dpsi{\dot\psi}\)
    \( \newcommand\Domega{\dot\omega}\)
    \( \newcommand\DGamma
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[15], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\DDelta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[16], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\DTheta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[17], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\DLambda{\dot\Lambda}\)
    \( \newcommand\DXi{\dot\Xi}\)
    \( \newcommand\DPi{\dot\Pi}\)
    \( \newcommand\DSigma{\dot\Sigma}\)
    \( \newcommand\DUps{\dot\Upsilon}\)
    \( \newcommand\DPhi{\dot\Phi}\)
    \( \newcommand\DPsi{\dot\Psi}\)
    \( \newcommand\DOmega{\dot\Omega}\)
    \( \newcommand\Va{\vec a}\)
    \( \newcommand\Vb{\vec b}\)
    \( \newcommand\Vc{\vec c}\)
    \( \newcommand\Vd{\vec d}\)
    \( \newcommand\Ve{\vec e}\)
    \( \newcommand\Vf{\vec f}\)
    \( \newcommand\Vg{\vec g}\)
    \( \newcommand\Vh{\vec h}\)
    \( \newcommand\Vi{\vec \imath}\)
    \( \newcommand\Vj{\vec \jmath}\)
    \( \newcommand\Vk{\vec k}\)
    \( \newcommand\Vl{\vec l}\)
    \( \newcommand\Vm{\vec m}\)
    \( \newcommand\Vn{\vec n}\)
    \( \newcommand\Vo{\vec o}\)
    \( \newcommand\Vp{\vec p}\)
    \( \newcommand\Vq{\vec q}\)
    \( \newcommand\Vr{\vec r}\)
    \( \newcommand\Vs{\vec s}\)
    \( \newcommand\Vt{\vec t}\)
    \( \newcommand\Vu{\vec u}\)
    \( \newcommand\Vv{\vec v}\)
    \( \newcommand\Vw{\vec w}\)
    \( \newcommand\Vx{\vec x}\)
    \( \newcommand\Vy{\vec y}\)
    \( \newcommand\Vz{\vec z}\)
    \( \newcommand\VA{\vec A}\)
    \( \newcommand\VB{\vec B}\)
    \( \newcommand\VC{\vec C}\)
    \( \newcommand\VD{\vec D}\)
    \( \newcommand\VE{\vec E}\)
    \( \newcommand\VF{\vec F}\)
    \( \newcommand\VG{\vec G}\)
    \( \newcommand\VH{\vec H}\)
    \( \newcommand\VI{\vec I}\)
    \( \newcommand\VJ{\vec J}\)
    \( \newcommand\VK{\vec K}\)
    \( \newcommand\VL{\vec L}\)
    \( \newcommand\VM{\vec M}\)
    \( \newcommand\VN{\vec N}\)
    \( \newcommand\VO{\vec O}\)
    \( \newcommand\VP{\vec P}\)
    \( \newcommand\VQ{\vec Q}\)
    \( \newcommand\VR{\vec R}\)
    \( \newcommand\VS{\vec S}\)
    \( \newcommand\VT{\vec T}\)
    \( \newcommand\VU{\vec U}\)
    \( \newcommand\VV{\vec V}\)
    \( \newcommand\VW{\vec W}\)
    \( \newcommand\VX{\vec X}\)
    \( \newcommand\VY{\vec Y}\)
    \( \newcommand\VZ{\vec Z}\)
    \( \newcommand\Valpha{\vec\alpha}\)
    \( \newcommand\Vbeta{\vec\beta}\)
    \( \newcommand\Vgamma{\vec\gamma}\)
    \( \newcommand\Vdelta{\vec\delta}\)
    \( \newcommand\Vepsilon{\vec\epsilon}\)
    \( \newcommand\Vvarepsilon{\vec\varepsilon}\)
    \( \newcommand\Vzeta{\vec\zeta}\)
    \( \newcommand\Veta{\vec\eta}\)
    \( \newcommand\Vtheta{\vec\theta}\)
    \( \newcommand\Vvartheta{\vec\vartheta}\)
    \( \newcommand\Viota{\vec\iota}\)
    \( \newcommand\Vkappa{\vec\kappa}\)
    \( \newcommand\Vlambda{\vec\lambda}\)
    \( \newcommand\Vmu
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[18], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vnu
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[19], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vxi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[20], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vom
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[21], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vpi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[22], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vvarpi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[23], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vrho
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[24], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vvarrho
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[25], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vsigma
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[26], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vvarsigma
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[27], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vtau
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[28], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vupsilon
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[29], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vphi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[30], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vvarphi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[31], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vchi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[32], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vpsi
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[33], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\Vomega
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[34], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\VGamma
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[35], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\VDelta
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/p[1]/span[36], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\VTheta{\vec\Theta}\)
    \( \newcommand\VLambda{\vec\Lambda}\)
    \( \newcommand\VXi{\vec\Xi}\)
    \( \newcommand\VPi{\vec\Pi}\)
    \( \newcommand\VSigma{\vec\Sigma}\)
    \( \newcommand\VUps{\vec\Upsilon}\)
    \( \newcommand\VPhi{\vec\Phi}\)
    \( \newcommand\VPsi{\vec\Psi}\)
    \( \newcommand\VOmega{\vec\Omega}\)
    \( \newcommand\BA{\mib A}\)
    \( \newcommand\BB{\mib B}\)
    \( \newcommand\BC{\mib C}\)
    \( \newcommand\BD{\mib D}\)
    \( \newcommand\BE{\mib E}\)
    \( \newcommand\BF{\mib F}\)
    \( \newcommand\BG{\mib G}\)
    \( \newcommand\BH{\mib H}\)
    \( \newcommand\BI{\mib I}}\)
    \( \newcommand\BJ{\mib J}\)
    \( \newcommand\BK{\mib K}\)
    \( \newcommand\BL{\mib L}\)
    \( \newcommand\BM{\mib M}\)
    \( \newcommand\BN{\mib N}\)
    \( \newcommand\BO{\mib O}\)
    \( \newcommand\BP{\mib P}\)
    \( \newcommand\BQ{\mib Q}\)
    \( \newcommand\BR{\mib R}\)
    \( \newcommand\BS{\mib S}\)
    \( \newcommand\BT{\mib T}\)
    \( \newcommand\BU{\mib U}\)
    \( \newcommand\BV{\mib V}\)
    \( \newcommand\BW{\mib W}\)
    \( \newcommand\BX{\mib X}\)
    \( \newcommand\BY{\mib Y}\)
    \( \newcommand\BZ{\mib Z}\)
    \( \newcommand\Ba{\mib a}\)
    \( \newcommand\Bb{\mib b}\)
    \( \newcommand\Bc{\mib c}\)
    \( \newcommand\Bd{\mib d}\)
    \( \newcommand\Be{\mib e}\)
    \( \newcommand\Bf{\mib f}\)
    \( \newcommand\Bg{\mib g}\)
    \( \newcommand\Bh{\mib h}\)
    \( \newcommand\Bi{\mib i}\)
    \( \newcommand\Bj{\mib j}\)
    \( \newcommand\Bk{\mib k}\)
    \( \newcommand\Bl{\mib l}\)
    \( \newcommand\Bm{\mib m}\)
    \( \newcommand\Bn{\mib n}\)
    \( \newcommand\Bo{\mib o}\)
    \( \newcommand\Bp{\mib p}\)
    \( \newcommand\Bq{\mib q}\)
    \( \newcommand\Br{\mib r}\)
    \( \newcommand\Bs{\mib s}\)
    \( \newcommand\Bt{\mib t}\)
    \( \newcommand\Bu{\mib u}\)
    \( \newcommand\Bv{\mib v}\)
    \( \newcommand\Bw{\mib w}\)
    \( \newcommand\Bx{\mib x}\)
    \( \newcommand\By{\mib y}\)
    \( \newcommand\Bz{\mib z}\)\)
    \( \newcommand\vrh{\varrho}\)
    \( \newcommand\vsig{\varsigma}\)
    \( \newcommand\ups{\upsilon}\)
    \( \newcommand\eps{\epsilon}\)
    \( \newcommand\ve{\varepsilon}\)
    \( \newcommand\vth{\vartheta}\)
    \( \newcommand\vphi{\varphi}\)
    \( \newcommand\xhi{\chi}\)
    \( \newcommand\Ups{\Upsilon}\)
    \( \newcommand\Balpha{\mib\alpha}\)
    \( \newcommand\Bbeta{\mib\beta}\)
    \( \newcommand\Bgamma{\mib\gamma}\)
    \( \newcommand\Bdelta{\mib\delta}\)
    \( \newcommand\Beps{\mib\epsilon}\)
    \( \newcommand\Bve{\mib\varepsilon}\)
    \( \newcommand\Bzeta{\mib\zeta}\)
    \( \newcommand\Beta{\mib\eta}\)
    \( \newcommand\Btheta{\mib\theta}\)
    \( \newcommand\Bvth{\mib\vartheta}\)
    \( \newcommand\Biota{\mib\iota}\)
    \( \newcommand\Bkappa{\mib\kappa}\)
    \( \newcommand\Blambda{\mib\lambda}\)
    \( \newcommand\Bmu{\mib\mu}\)
    \( \newcommand\Bnu{\mib\nu}\)
    \( \newcommand\Bxi{\mib\xi}\)
    \( \newcommand\Bom{\mib\omicron}\)
    \( \newcommand\Bpi{\mib\pi}\)
    \( \newcommand\Bvarpi{\mib\varpi}\)
    \( \newcommand\Brho{\mib\rho}\)
    \( \newcommand\Bvrh{\mib\varrho}\)
    \( \newcommand\Bsigma{\mib\sigma}\)
    \( \newcommand\Bvsig{\mib\varsigma}\)
    \( \newcommand\Btau{\mib\tau}\)
    \( \newcommand\Bups{\mib\upsilon}\)
    \( \newcommand\Bphi{\mib\phi}\)
    \( \newcommand\Bvphi{\mib\vphi}\)
    \( \newcommand\Bchi{\mib\chi}\)
    \( \newcommand\Bpsi{\mib\psi}\)
    \( \newcommand\Bomega{\mib\omega}\)
    \( \newcommand\BGamma{\mib\Gamma}\)
    \( \newcommand\BDelta{\mib\Delta}\)
    \( \newcommand\BTheta{\mib\Theta}\)
    \( \newcommand\BLambda{\mib\Lambda}\)
    \( \newcommand\BXi{\mib\Xi}\)
    \( \newcommand\BPi{\mib\Pi}\)
    \( \newcommand\BSigma{\mib\Sigma}\)
    \( \newcommand\BUps{\mib\Upsilon}\)
    \( \newcommand\BPhi{\mib\Phi}\)
    \( \newcommand\BPsi{\mib\Psi}\)
    \( \newcommand\BOmega{\mib\Omega}\)
    \( \newcommand\Bxhi{\raise.35ex\hbox{$\Bchi$}}\)
    \( \newcommand\RGamma{ \Gamma}\)
    \( \newcommand\RDelta{ \Delta}\)
    \( \newcommand\RTheta{ \Theta}\)
    \( \newcommand\RLambda{ \Lambda}\)
    \( \newcommand\RXi{ \Xi}\)
    \( \newcommand\RPi{ \Pi}\)
    \( \newcommand\RSigma{ \Sigma}\)
    \( \newcommand\RUps{ \Upsilon}\)
    \( \newcommand\RPhi{ \Phi}\)
    \( \newcommand\RPsi{ \Psi}\)
    \( \newcommand\ROmega{ \Omega}\)
    \( \newcommand\RA{ A}\)
    \( \newcommand\RB{ B}\)
    \( \newcommand\RC{ C}\)
    \( \newcommand\RD{ D}\)
    \( \newcommand\RE{ E}\)
    \( \newcommand\RF{ F}\)
    \( \newcommand\RG{ G}\)
    \( \newcommand\RH{ H}\)
    \( \newcommand\RI{ I}\)
    \( \newcommand\RJ{ J}\)
    \( \newcommand\RK{ K}\)
    \( \newcommand\RL{ L}\)
    \( \newcommand { M}\)
    \( \newcommand\RN{ N}\)
    \( \newcommand\RO{ O}\)
    \( \newcommand\RP{ P}\)
    \( \newcommand\RQ{ Q}\)
    \( \newcommand\RR{ R}\)
    \( \newcommand\RS{ S}\)
    \( \newcommand\RT{ T}\)
    \( \newcommand\RU{ U}\)
    \( \newcommand\RV{ V}\)
    \( \newcommand\RW{ W}\)
    \( \newcommand\RX{ X}\)
    \( \newcommand\RY{ Y}\)
    \( \newcommand\RZ{ Z}\)
    \( \newcommand\Ra{ a}\)
    \( \newcommand\Rb{ b}\)
    \( \newcommand\Rc{ c}\)
    \( \newcommand\Rd{ d}\)
    \( \newcommand\Re{ e}\)
    \( \newcommand\Rf{ f}\)
    \( \newcommand\Rg{ g}\)
    \( \newcommand\Rh{ h}\)
    \( \newcommand\Ri{ i}\)
    \( \newcommand\Rj{ j}\)
    \( \newcommand\Rk{ k}\)
    \( \newcommand\Rl{ l}\)
    \( \newcommand { m}\)
    \( \newcommand\Rn{ n}\)
    \( \newcommand\Ro{ o}\)
    \( \newcommand\Rp{ p}\)
    \( \newcommand\Rq{ q}\)
    \( \newcommand\Rr{ r}\)
    \( \newcommand\Rs{ s}\)
    \( \newcommand\Rt{ t}\)
    \( \newcommand\Ru{ u}\)
    \( \newcommand\Rv{ v}\)
    \( \newcommand\Rw{ w}\)
    \( \newcommand\Rx{ x}\)
    \( \newcommand\Ry{ y}\)
    \( \newcommand\Rz{ z}\)
    \( \newcommand\BBA{\boldsymbol\RA}\)
    \( \newcommand\BBB{\boldsymbol\RB}\)
    \( \newcommand\BBC{\boldsymbol\RC}\)
    \( \newcommand\BBD{\boldsymbol\RD}\)
    \( \newcommand\BBE{\boldsymbol\RE}\)
    \( \newcommand\BBF{\boldsymbol\RF}\)
    \( \newcommand\BBG{\boldsymbol\RG}\)
    \( \newcommand\BBH{\boldsymbol\RH}\)
    \( \newcommand\BBI{\boldsymbol\RI}\)
    \( \newcommand\BBJ{\boldsymbol\RJ}\)
    \( \newcommand\BBK{\boldsymbol\RK}\)
    \( \newcommand\BBL{\boldsymbol\RL}\)
    \( \newcommand\BBM{\boldsymbol }\)
    \( \newcommand\BBN{\boldsymbol\RN}\)
    \( \newcommand\BBO{\boldsymbol\RO}\)
    \( \newcommand\BBP{\boldsymbol\RP}\)
    \( \newcommand\BBQ{\boldsymbol\RQ}\)
    \( \newcommand\BBR{\boldsymbol\RR}\)
    \( \newcommand\BBS{\boldsymbol\RS}\)
    \( \newcommand\BBT{\boldsymbol\RT}\)
    \( \newcommand\BBU{\boldsymbol\RU}\)
    \( \newcommand\BBV{\boldsymbol\RV}\)
    \( \newcommand\BBW{\boldsymbol\RW}\)
    \( \newcommand\BBX{\boldsymbol\RX}\)
    \( \newcommand\BBY{\boldsymbol\RY}\)
    \( \newcommand\BBZ{\boldsymbol\RZ}\)
    \( \newcommand\BBa{\boldsymbol\Ra}\)
    \( \newcommand\BBb{\boldsymbol\Rb}\)
    \( \newcommand\BBc{\boldsymbol\Rc}\)
    \( \newcommand\BBd{\boldsymbol\Rd}\)
    \( \newcommand\BBe{\boldsymbol\Re}\)
    \( \newcommand\BBf{\boldsymbol\Rf}\)
    \( \newcommand\BBg{\boldsymbol\Rg}\)
    \( \newcommand\BBh{\boldsymbol\Rh}\}\)
    \( \newcommand\BBi{\boldsymbol\Ri}\)
    \( \newcommand\BBj{\boldsymbol\Rj}\)
    \( \newcommand\BBk{\boldsymbol\Rk}\)
    \( \newcommand\BBl{boldsymbol\Rl}\)
    \( \newcommand\BBm{\boldsymbol }\)
    \( \newcommand\BBn{\boldsymbol\Rn}\)
    \( \newcommand\BBo{\boldsymbol\Ro}\)
    \( \newcommand\BBp{\boldsymbol\Rp}\)
    \( \newcommand\BBq{\boldsymbol\Rq}\)
    \( \newcommand\BBr{\boldsymbol\Rr}\)
    \( \newcommand\BBs{\boldsymbol\Rs}\)
    \( \newcommand\BBt{\boldsymbol\Rt}\)
    \( \newcommand\BBu{\boldsymbol\Ru}\)
    \( \newcommand\BBv{\boldsymbol\Rv}\)
    \( \newcommand\BBw{\boldsymbol\Rw}\)
    \( \newcommand\BBx{\boldsymbol\Rx}\)
    \( \newcommand\BBy{\boldsymbol\Ry}\)
    \( \newcommand\BBz{\boldsymbol\Rz}\)
    \( \newcommand\tcb{\textcolor{blue}\)
    \( \newcommand\tcr{\textcolor{red}\)
    \( \newcommand\bnabla{\boldsymbol{\nabla}}\)
    \( \newcommand\Bell{\boldsymbol\ell}\)
    \( \newcommand\dbar{\,{\mathchar'26\mkern-12mu d}} \)
    \( \newcommand\ns{^\vphantom{*}}\)
    \( \newcommand\uar{\uparrow}\)
    \( \newcommand\dar{\downarrow}\)
    \( \newcommand\impi{\int\limits_{-\infty}^{\infty}\!\!}\)
    \( \newcommand\izpi{\int\limits_{0}^{\infty}\!\!}\)
    \( \newcommand\etc{\it etc.\/}\)
    \( \newcommand\etal{\it et al.\/}\)
    \( \newcommand\opcit{\it op. cit.\/}\)
    \( \newcommand\ie{\it i.e.\/}\)
    \( \newcommand\Ie{\it I.e.\/}\)
    \( \newcommand\viz{\it viz.\/}\)
    \( \newcommand\eg{\it e.g.\/}\)
    \( \newcommand\Eg{\it E.g.\/}\)
    \( \newcommand\dbar{\,{\mathchar'26\mkern-12mu d}} \)
    \( \def\sss#1{\scriptscriptstyle #1}\)
    \( \def\ss#1{\scriptstyle #1}\)
    \( \def\ssr#1{\scriptstyle #1}\)
    \( \def\ssf#1{\scriptstyle #1}\)
    \( \newcommand\NA{N_{\ssr{\!A}}}\)
    \( \newcommand\lala{\langle\!\langle}\)
    \( \newcommand\rara{\rangle\!\rangle}\)
    \( \newcommand\blan{\big\langle}\)
    \( \newcommand\bran{\big\rangle}\)
    \( \newcommand\Blan{\Big\langle}\)
    \( \newcommand\Bran{\Big\rangle}\)
    \( \newcommand\intl{\int\limits}\)
    \( \newcommand\half{\frac{1}{2}}\)
    \( \newcommand\third{\frac{1}{3}}\)
    \( \newcommand\fourth{\frac{1}{4}}\)
    \( \newcommand\eighth{\frac{1}{8}}\)
    \( \newcommand\uar{\uparrow}\)
    \( \newcommand\dar{\downarrow}\)
    \( \newcommand\undertext#1{$\underline{\hbox{#1}}$}\)
    \( \newcommand\Tra{\mathop{\textsf{Tr}}\,}\)
    \( \newcommand\det{\mathop{\textsf{det}}\,}\)
    \( \def\tket#1{|  #1 \rangle}\)
    \( \def\tbra#1{\langle #1|}\)
    \( \def\tbraket#1#2{\langle #1  |   #2 \rangle}\)
    \( \def\texpect#1#2#3{\langle #1 |   #2  |  #3 \rangle}\)
    \( \def\sket#1{|  \, #1 \,  \rangle}\)
    \( \def\sbra#1{\langle \,  #1 \, |}\)
    \( \def\sbraket#1#2{\langle \, #1  \, |  \, #2 \,  \rangle}\)
    \( \def\sexpect#1#2#3{\langle \, #1 \, | \,  #2  \, | \, #3 \, \rangle}\)
    \(\def\ket#1{\big| \, #1\, \big\rangle}\)
    \( \def\bra#1{\big\langle \, #1 \, \big|}\)
    \( \def\braket#1#2{\big\langle \, #1\, \big| \,#2 \,\big\rangle}\)
    \( \def\expect#1#2#3{\big\langle\, #1\, \big|\, #2\, \big| \,#3\, \big\rangle}\)
    \( \newcommand\pz{\partial}\)
    \( \newcommand\pzb{\bar{\partial}}\)
    \( \newcommand\svph{\vphantom{\int}}\)
    \( \newcommand\vph{\vphantom{\sum_i}}\)
    \( \newcommand\bvph{\vphantom{\sum_N^N}}\)
    \( \newcommand\nd{^{\vphantom{\dagger}}}\)
    \( \newcommand\ns{^{\vphantom{*}}}\)
    \( \newcommand\yd{^\dagger}\)
    \( \newcommand\zb{\bar z}\)
    \( \newcommand\zdot{\dot z}\)
    \( \newcommand\zbdot{\dot{\bar z}}\)
    \( \newcommand\kB{k_{\sss{B}}}\)
    \( \newcommand\kT{k_{\sss{B}}T}\)
    \( \newcommand\gtau{g_\tau}\)
    \( \newcommand\Htil{\tilde H}\)
    \( \newcommand\pairo{(\phi\nd_0,J\nd_0)}\)
    \( \newcommand\pairm{(\phi\nd_0,J)}\)
    \( \newcommand\pairob{(\Bphi\nd_0,\BJ\nd_0)}\)
    \( \newcommand\pairmb{(\Bphi\nd_0,\BJ)}\)
    \( \newcommand\pair{(\phi,J)}\)
    \( \newcommand\Hz{H\nd_0}\)
    \( \newcommand\Ho{H\nd_1}\)
    \( \newcommand\Htz{\Htil\nd_0}\)
    \( \newcommand\Hto{\Htil\nd_1}\)
    \( \newcommand\oc{\omega_\Rc}\)

    \(\newcommand \gtwid{\approx}\)

    \( \newcommand\index{\textsf{ind}}\)
    \( \newcommand\csch{\,{ csch\,}}\)
    \( \newcommand\ctnh{\,{ ctnh\,}}\)
    \( \newcommand\ctn{\,{ ctn\,}}\)
    \( \newcommand\sgn{\,{ sgn\,}}\)
    \( \def\tmapright#1{\xrightarrow \limits^{#1}}\)
    \( \def\bmapright#1{\xrightarrow\limits_{#1}}\)
    \( \newcommand\hfb{\hfill\break}\)
    \( \newcommand\Rep{\textsf{Re}\,}\)
    \( \newcommand\Imp{\textsf{Im}\,}\)
    \( \newcommand\ncdot{\!\cdot\!}\)
    \( \def\tmapright#1{ \smash{\mathop{\hbox to 35pt{\rightarrowfill}}\limits^{#1}}\ }\)
    \( \def\bmapright#1{ \smash{\mathop{\hbox to 35pt{\rightarrowfill}}\limits_{#1}}\ }\)
    \( \newcommand\bsqcap{\mbox{\boldmath{$\sqcap$}}}\)

    \( \def\pabc#1#2#3{\left({\pz #1\over\pz #2}\right)\ns_{\!\!#3}}\)
    \( \def\spabc#1#2#3{\big({\pz #1\over\pz #2}\big)\ns_{\!#3}}\)
    \( \def\qabc#1#2#3{\pz^2\! #1\over\pz #2\,\pz #3}\)
    \( \def\rabc#1#2#3#4{(\pz #1,\pz #2)\over (\pz #3,\pz #4)}\)
    \( \newcommand\subA{\ns_\ssr{A}}\)
    \( \newcommand\subB{\ns_\ssr{B}}\)
    \( \newcommand\subC{\ns_\ssr{C}}\)
    \( \newcommand\subD{\ns_\ssr{D}}\)
    \( \newcommand\subAB{\ns_\ssr{AB}}\)
    \( \newcommand\subBC{\ns_\ssr{BC}}\)
    \( \newcommand\subCD{\ns_\ssr{CD}}\)
    \( \newcommand\subDA{\ns_\ssr{DA}}\)
    \( \def\lmapright#1{\ \ \smash{\mathop{\hbox to 55pt{\rightarrowfill}}\limits^{#1}}\ \ }\)
    \( \def\enth#1{\RDelta {\textsf H}^0_\Rf[{ #1}]}\)
    \( \newcommand\longrightleftharpoons{ \mathop{\vcenter{\hbox{\ooalign{\raise1pt\hbox{$\longrightharpoonup\joinrel$}\crcr  \lower1pt\hbox{$\longleftharpoondown\joinrel$}}}}}}\)
    \( \newcommand\longrightharpoonup{\relbar\joinrel\rightharpoonup}\)
    \( \newcommand\longleftharpoondown{\leftharpoondown\joinrel\relbar}\)
    \( \newcommand\cds{\,\bullet\,}\)
    \( \newcommand\ccs{\,\circ\,}\)
    \( \newcommand\nsub{_{\vphantom{\dagger}}}\)
    \( \newcommand\rhohat{\hat\rho}\)
    \( \newcommand\vrhhat{\hat\vrh}\)
    \( \newcommand\impi{\int\limits_{-\infty}^\infty\!\!\!}\)
    \( \newcommand\brangle{\big\rangle}\)
    \( \newcommand\blangle{\big\langle}\)
    \( \newcommand\vet{\tilde\ve}\)
    \( \newcommand\zbar{\bar z}\)
    \( \newcommand\ftil{\tilde f}\)
    \( \newcommand\XBE{\RXi\ns_\ssr{BE}}\)
    \( \newcommand\XFD{\RXi\ns_\ssr{FD}}\)
    \( \newcommand\OBE{\Omega\ns_\ssr{BE}}\)
    \( \newcommand\OFD{\Omega\ns_\ssr{FD}}\)
    \( \newcommand\veF{\ve\ns_\RF}\)
    \( \newcommand\kF{k\ns_\RF}\)
    \( \newcommand\kFu{k\ns_{\RF\uar}}\)
    \( \newcommand\SZ{\textsf Z}}\) \( \newcommand\kFd{k\ns_{\RF\dar}\)
    \( \newcommand\muB{\mu\ns_\ssr{B}}\)
    \( \newcommand\mutB{\tilde\mu}\ns_\ssr{B}\)
    \( \newcommand\xoN{\Bx\ns_1\,,\,\ldots\,,\,\Bx\ns_N}\)
    \( \newcommand\rok{\Br\ns_1\,,\,\ldots\,,\,\Br\ns_k}\)
    \( \newcommand\xhiOZ{\xhi^\ssr{OZ}}\)
    \( \newcommand\xhihOZ
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/span[1], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\jhz{\HJ(0)}\)
    \( \newcommand\nda{\nd_\alpha}\)
    \( \newcommand\ndap{\nd_{\alpha'}}\)
    \( \newcommand\labar
    ParseError: invalid DekiScript (click for details)
    Callstack:
        at (Template:MathJaxArovas), /content/body/div/span[2], line 1, column 1
        at template()
        at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23
    
    \)
    \( \newcommand\msa{m\ns_\ssr{A}}\)
    \( \newcommand\msb{m\ns_\ssr{B}}\)
    \( \newcommand\mss{m\ns_\Rs}\)
    \( \newcommand\HBx{\hat\Bx}\)
    \( \newcommand\HBy{\hat\By}\)
    \( \newcommand\HBz{\hat\Bz}\)
    \( \newcommand\thm{\theta\ns_m}\)
    \( \newcommand\thp{\theta\ns_\phi}\)
    \( \newcommand\mtil{\widetilde m}\)
    \( \newcommand\phitil{\widetilde\phi}\)
    \( \newcommand\delf{\delta\! f}\)
    \( \newcommand\coll{\bigg({\pz f\over\pz t}\bigg)\nd_{\! coll}}\)
    \( \newcommand\stre{\bigg({\pz f\over\pz t}\bigg)\nd_{\! str}}\)
    \( \newcommand\idrp{\int\!\!{d^3\!r\,d^3\!p\over h^3}\>}\)
    \( \newcommand\vbar{\bar v}\)
    \( \newcommand\BCE{\mbox{\boldmath{$\CE$}}\!}\)
    \( \newcommand\BCR{\mbox{\boldmath{$\CR$}}\!}\)
    \( \newcommand\gla{g\nd_{\RLambda\nd}}\)
    \( \newcommand\TA{T\ns_\ssr{A}}\)
    \( \newcommand\TB{T\ns_\ssr{B}}\)
    \( \newcommand\ncdot{\!\cdot\!}\)
    \( \newcommand\NS{N\ns_{\textsf S}}\)

    Fundamental definitions

    The natural mathematical setting is set theory. Sets are generalized collections of objects. The basics: \(\omega\in A\) is a binary relation which says that the object \(\omega\) is an element of the set \(A\). Another binary relation is set inclusion. If all members of \(A\) are in \(B\), we write \(A\subseteq B\). The union of sets \(A\) and \(B\) is denoted \(A\cup B\) and the intersection of \(A\) and \(B\) is denoted \(A\cap B\). The Cartesian product of \(A\) and \(B\), denoted \(A\times B\), is the set of all ordered elements \((a,b)\) where \(a\in A\) and \(b\in B\).

    Some details: If \(\omega\) is not in \(A\), we write \(\omega\notin A\). Sets may also be objects, so we may speak of sets of sets, but typically the sets which will concern us are simple discrete collections of numbers, such as the possible rolls of a die {1,2,3,4,5,6}, or the real numbers \(\MR\), or Cartesian products such as \(\MR^N\). If \(A\subseteq B\) but \(A\ne B\), we say that \(A\) is a proper subset of \(B\) and write \(A\subset B\). Another binary operation is the set difference \(A\backslash B\), which contains all \(\omega\) such that \(\omega\in A\) and \(\omega\notin B\).

    In probability theory, each object \(\omega\) is identified as an event. We denote by \(\ROmega\) the set of all events, and \(\emptyset\) denotes the set of no events. There are three basic axioms of probability:

    • To each set \(A\) is associated a non-negative real number \(P(A)\), which is called the probability of \(A\).
    • \(P(\ROmega)=1\).
    • If \(\{A\ns_i\}\) is a collection of disjoint sets, if \(A\ns_i\cap A\ns_j=\emptyset\) for all \(i\ne j\), then \[P\Big(\bigcup_i A\ns_i\Big)=\sum_i P(A\ns_i)\ .\]

    From these axioms follow a number of conclusions. Among them, let \(\neg A=\ROmega\backslash A\) be the complement of \(A\), the set of all events not in \(A\). Then since \(A\cup\neg A=\ROmega\), we have \(P(\neg A)=1-P(A)\). Taking \(A=\ROmega\), we conclude \(P(\emptyset)=0\).

    The meaning of \(P(A)\) is that if events \(\omega\) are chosen from \(\ROmega\) at random, then the relative frequency for \(\omega\in A\) approaches \(P(A)\) as the number of trials tends to infinity. But what do we mean by ’at random’? One meaning we can impart to the notion of randomness is that a process is random if its outcomes can be accurately modeled using the axioms of probability. This entails the identification of a probability space \(\ROmega\) as well as a probability measure \(P\). For example, in the microcanonical ensemble of classical statistical physics, the space \(\ROmega\) is the collection of phase space points \(\Bvphi=\{q\ns_1,\ldots,q\ns_n,p\ns_1,\ldots,p\ns_n\}\) and the probability measure is \(d\mu=\Sigma^{-1}(E)\prod_{i=1}^n dq\ns_i\, dp\ns_i\,\delta\big(E-H(q,p)\big)\), so that for \(A\in\ROmega\) the probability of \(A\) is \(P(A)=\int\!d\mu\>\xhi\nd_A(\Bvphi)\), where \(\xhi\ns_A(\Bvphi)=1\) if \(\Bvphi\in A\) and \(\xhi\ns_A(\Bvphi)=0\) if \(\Bvphi\notin A\) is the characteristic function of \(A\). The quantity \(\Sigma(E)\) is determined by normalization: \(\int\!d\mu = 1\).

    Bayesian Statistics

    We now introduce two additional probabilities. The joint probability for sets \(A\) and \(B\) together is written \(P(A\cap B)\). That is, \(P(A\cap B)={Prob}[\omega\in A\ {and}\ \omega\in B]\). For example, \(A\) might denote the set of all politicians, \(B\) the set of all American citizens, and \(C\) the set of all living humans with an IQ greater than 60. Then \(A\cap B\) would be the set of all politicians who are also American citizens, Exercise: estimate \(P(A\cap B\cap C)\).

    The conditional probability of \(B\) given \(A\) is written \(P(B|A)\). We can compute the joint probability \(P(A\cap B)=P(B\cap A)\) in two ways: \[P(A\cap B)=P(A|B)\cdot P(B)=P(B|A)\cdot P(A)\ .\] Thus, \[P(A|B)={P(B|A)\, P(A)\over P(B)}\ ,\] a result known as Bayes’ theorem. Now suppose the ‘event space’ is partitioned as \(\{A\ns_i\}\). Then \[P(B)=\sum_i P(B|A\ns_i)\, P(A\ns_i)\ .\] We then have \[P(A\ns_i|B)={P(B|A\ns_i)\, P(A\ns_i)\over\sum_j P(B|A\ns_j)\, P(A\ns_j)}\ ,\] a result sometimes known as the extended form of Bayes’ theorem. When the event space is a ‘binary partition’ \(\{A,\neg A\}\), we have \[P(A|B)={P(B|A)\, P(A)\over P(B|A)\, P(A) + P(B|\neg A)\, P(\neg A)}\ . \label{Bayesbinary}\] Note that \(P(A|B)+P(\neg A|B)=1\) (which follows from \(\neg\neg A = A\)).

    As an example, consider the following problem in epidemiology. Suppose there is a rare but highly contagious disease \(A\) which occurs in \(0.01\%\) of the general population. Suppose further that there is a simple test for the disease which is accurate \(99.99\%\) of the time. That is, out of every 10,000 tests, the correct answer is returned 9,999 times, and the incorrect answer is returned only once. Now let us administer the test to a large group of people from the general population. Those who test positive are quarantined. Question: what is the probability that someone chosen at random from the quarantine group actually has the disease? We use Bayes’ theorem with the binary partition \(\{A,\neg A\}\). Let \(B\) denote the event that an individual tests positive. Anyone from the quarantine group has tested positive. Given this datum, we want to know the probability that that person has the disease. That is, we want \(P(A|B)\). Applying Equation [Bayesbinary] with \[P(A)=0.0001 \quad,\quad P(\neg A)=0.9999 \quad,\quad P(B|A)=0.9999 \quad,\quad P(B|\neg A)=0.0001\ ,\] we find \(P(A|B)=\half\). That is, there is only a \(50\%\) chance that someone who tested positive actually has the disease, despite the test being \(99.99\%\) accurate! The reason is that, given the rarity of the disease in the general population, the number of false positives is statistically equal to the number of true positives.

    In the above example, we had \(P(B|A)+P(B|\neg A)=1\), but this is not generally the case. What is true instead is \(P(B|A)+P(\neg B|A)=1\). Epidemiologists define the sensitivity of a binary classification test as the fraction of actual positives which are correctly identified, and the specificity as the fraction of actual negatives that are correctly identified. Thus, \(\textsf{se}=P(B|A)\) is the sensitivity and \(\textsf{sp}=P(\neg B|\neg A)\) is the specificity. We then have \(P(B|\neg A)=1-P(\neg B|\neg A)\). Therefore, \[P(B|A)+P(B|\neg A)=1+P(B|A)-P(\neg B|\neg A)=1+\textsf{se}-\textsf{sp}\ .\] In our previous example, \(\textsf{se}=\textsf{sp}=0.9999\), in which case the RHS above gives \(1\). In general, if \(P(A)\equiv f\) is the fraction of the population which is afflicted, then \[P(\textsf{infected}\,|\,\textsf{positive})={f\cdot\textsf{se}\over f\cdot\textsf{se} + (1-f)\cdot(1-\textsf{sp})}\ .\]

    For continuous distributions, we speak of a probability density. We then have \[P(y)=\int\!dx\>P(y|x)\, P(x)\] and \[P(x|y)={P(y|x)\, P(x)\over\int\!dx'\> P(y|x')\, P(x')}\ .\] The range of integration may depend on the specific application.

    The quantities \(P(A\ns_i)\) are called the prior distribution. Clearly in order to compute \(P(B)\) or \(P(A\ns_i|B)\) we must know the priors, and this is usually the weakest link in the Bayesian chain of reasoning. If our prior distribution is not accurate, Bayes’ theorem will generate incorrect results. One approach to approximating prior probabilities \(P(A\ns_i)\) is to derive them from a maximum entropy construction.

    Random variables and their averages

    Consider an abstract probability space \(\CX\) whose elements ( events) are labeled by \(x\). The average of any function \(f(x)\) is denoted as \(\ME f\) or \(\langle f\rangle\), and is defined for discrete sets as \[\ME f=\langle f\rangle=\sum_{x\in\CX} f(x)\, P(x)\ ,\] where \(P(x)\) is the probability of \(x\). For continuous sets, we have \[\ME f=\langle f\rangle=\int\limits_{\CX}\!\!dx\> f(x)\, P(x)\ .\] Typically for continuous sets we have \(\CX=\MR\) or \(\CX=\MR\ns_{\ge 0}\). Gardiner and other authors introduce an extra symbol, \(X\), to denote a random variable, with \(X(x)=x\) being its value. This is formally useful but notationally confusing, so we’ll avoid it here and speak loosely of \(x\) as a random variable.

    When there are two random variables \(x\in\CX\) and \(y\in\CY\), we have \(\ROmega=\CX\times\CY\) is the product space, and \[\ME f(x,y)=\langle f(x,y)\rangle =\sum_{x\in\CX}\sum_{y\in\CY} f(x,y)\,P(x,y)\ ,\] with the obvious generalization to continuous sets. This generalizes to higher rank products, \(x\ns_i\in\CX\ns_i\) with \(i\in\{1,\ldots,N\}\). The covariance of \(x\ns_i\) and \(x\ns_j\) is defined as \[C\ns_{ij}\equiv \blangle \big( x\ns_i - \langle x\ns_i\rangle \big) \big( x\ns_j - \langle x\ns_j\rangle \big) \brangle =\langle x\ns_i x\ns_j\rangle - \langle x\ns_i\rangle \langle x\ns_j\rangle\ .\]

    If \(f(x)\) is a convex function then one has \[\ME f(x) \ge f(\ME x)\ .\] For continuous functions, \(f(x)\) is convex if \(f''(x)\ge 0\) everywhere4. If \(f(x)\) is convex on some interval \([a,b]\) then for \(x\ns_{1,2}\in [a,b]\) we must have \[f\big(\lambda x\ns_1 + (1-\lambda) x\ns_2\big) \le \lambda f(x\ns_1) + (1-\lambda) f(x\ns_2)\ ,\] where \(\lambda\in [0,1]\). This is easily generalized to \[f\Big(\sum_n p\ns_n x\ns_n\Big) \le \sum_n p\ns_n f(x\ns_n)\ ,\] where \(p\ns_n=P(x\ns_n)\), a result known as Jensen’s theorem.

    • Was this article helpful?