1.2: Basic Concepts in Probability Theory

$\newcommand\bes{\begin{equation}\begin{split}}$
$\newcommand\ltwid{\propto}$
$\newcommand\ees{\end{split}\end{equation}}$
$\newcommand\mib{\mathbf}$
$\newcommand\Sa{\textsf a}$
$\newcommand\Sb{\textsf b}$
$\newcommand\Sc{\textsf c}$
$\newcommand\Sd{\textsf d}$
$\newcommand\Se{\textsf e}$
$\newcommand\Sf{\textsf f}$
$\newcommand\Sg{\textsf g}$
$\newcommand\Sh{\textsf h}$
$\newcommand\Si{\textsf i}$
$\newcommand\Sj{\textsf j}$
$\newcommand\Sk{\textsf k}$
$\newcommand\Sl{\textsf l}$
$\newcommand\Sm{\textsf m}$
$\newcommand\Sn{\textsf n}$
$\newcommand\So{\textsf o}$
$\newcommand\Sp{\textsf p}$
$\newcommand\Sq{\textsf q}$
$\newcommand\Sr{\textsf r}$
$\newcommand\Ss{\textsf s}$
$\newcommand\St{\textsf t}$
$\newcommand\Su{\textsf u}$
$\newcommand\Sv{\textsf v}$
$\newcommand\Sw{\textsf w}$
$\newcommand\Sx{\textsf x}$
$\newcommand\Sy{\textsf y}$
$\newcommand\Sz{\textsf z}$
$\newcommand\SA{\textsf A}$
$\newcommand\SB{\textsf B}$
$\newcommand\SC{\textsf C}$
$\newcommand\SD{\textsf D}$
$\newcommand\SE{\textsf E}$
$\newcommand\SF{\textsf F}$
$\newcommand\SG{\textsf G}$
$\newcommand\SH{\textsf H}$
$\newcommand\SI{\textsf I}$
$\newcommand\SJ{\textsf J}$
$\newcommand\SK{\textsf K}$
$\newcommand\SL{\textsf L}$
$\newcommand\SM{\textsf M}$
$\newcommand\SN{\textsf N}$
$\newcommand\SO{\textsf O}$
$\newcommand\SP{\textsf P}$
$\newcommand\SQ{\textsf Q}$
$\newcommand\SR{\textsf R}$
$\newcommand\SS{\textsf S}$
$\newcommand\ST{\textsf T}$
$\newcommand\SU{\textsf U}$
$\newcommand\SV{\textsf V}$
$\newcommand\SW{\textsf W}$
$\newcommand\SX{\textsf X}$
$\newcommand\SY{\textsf Y}$
$\newcommand\SZ{\textsf Z}$
$\newcommand\Ha{\hat a}$
$\newcommand\Hb{\hat b}$
$\newcommand\Hc{\hat c}$
$\newcommand\Hd{\hat d}$
$\newcommand\He{\hat e}$
$\newcommand\Hf{\hat f}$
$\newcommand\Hg{\hat g}$
$\newcommand\Hh{\hat h}$
$\newcommand\Hi{\hat \imath}$
$\newcommand\Hj{\hat \jmath}$
$\newcommand\Hk{\hat k}$
$\newcommand\Hl{\hat l}$
$\newcommand\Hm{\hat m}$
$\newcommand\Hn{\hat n}$
$\newcommand\Ho{\hat o}$
$\newcommand\Hp{\hat p}$
$\newcommand\Hq{\hat q}$
$\newcommand\Hr{\hat r}$
$\newcommand\Hs{\hat s}$
$\newcommand\Ht{\hat t}$
$\newcommand\Hu{\hat u}$
$\newcommand\Hv{\hat v}$
$\newcommand\Hw{\hat w}$
$\newcommand\Hx{\hat x}$
$\newcommand\Hy{\hat y}$
$\newcommand\Hz{\hat z}$
$\newcommand\HA{\hat A}$
$\newcommand\HB{\hat B}$
$\newcommand\HC{\hat C}$
$\newcommand\HD{\hat D}$
$\newcommand\HE{\hat E}$
$\newcommand\HF{\hat F}$
$\newcommand\HG{\hat G}$
$\newcommand\HH{\hat H}$
$\newcommand\HI{\hat I}$
$\newcommand\HJ{\hat J}$
$\newcommand\HK{\hat K}$
$\newcommand\HL{\hat L}$
$\newcommand\HM{\hat M}$
$\newcommand\HN{\hat N}$
$\newcommand\HO{\hat O}$
$\newcommand\HP{\hat P}$
$\newcommand\HQ{\hat Q}$
$\newcommand\HR{\hat R}$
$\newcommand\HS{\hat S}$
$\newcommand\HT{\hat T}$
$\newcommand\HU{\hat U}$
$\newcommand\HV{\hat V}$
$\newcommand\HW{\hat W}$
$\newcommand\HX{\hat X}$
$\newcommand\HY{\hat Y}$
$\newcommand\HZ{\hat Z}$
$\newcommand\Halpha{\hat\alpha}$
$\newcommand\Hbeta{\hat\beta}$
$\newcommand\Hgamma{\hat\gamma}$
$\newcommand\Hdelta{\hat\delta}$
$\newcommand\Hepsilon{\hat\epsilon}$
$\newcommand\Hvarepsilon{\hat\varepsilon}$
$\newcommand\Hzeta{\hat\zeta}$
$\newcommand\Heta{\hat\eta}$
$\newcommand\Htheta{\hat\theta}$
$\newcommand\Hvartheta{\hat\vartheta}$
$\newcommand\Hiota{\hat\iota}$
$\newcommand\Hkappa{\hat\kappa}$
$\newcommand\Hlambda{\hat\lambda}$
$\newcommand\Hmu{\hat\mu}$
$\newcommand\Hnu{\hat\nu}$
$\newcommand\Hxi{\hat\xi}$
$\newcommand\Hom{\hat\omicron}$
$\newcommand\Hpi{\hat\pi}$
$\newcommand\Hvarpi{\hat\varpi}$
$\newcommand\Hrho{\hat\rho}$
$\newcommand\Hvarrho{\hat\varrho}$
$\newcommand\Hsigma{\hat\sigma}$
$\newcommand\Hvarsigma{\hat\varsigma}$
$\newcommand\Htau{\var\tau}$
$\newcommand\Hupsilon{\hat\upsilon}$
$\newcommand\Hphi{\hat\phi}$
$\newcommand\Hvarphi{\hat\varphi}$
$\newcommand\Hchi{\hat\chi}$
$\newcommand\Hxhi{\hat\xhi}$
$\newcommand\Hpsi{\hat\psi}$
$\newcommand\Homega{\hat\omega}$
$\newcommand\HGamma{\hat\Gamma}$
$\newcommand\HDelta{\hat\Delta}$
$\newcommand\HTheta{\hat\Theta}$
$\newcommand\HLambda{\hat\Lambda}$
$\newcommand\HXi{\hat\Xi}$
$\newcommand\HPi{\hat\Pi}$
$\newcommand\HSigma{\hat\Sigma}$
$\newcommand\HUps{\hat\Upsilon}$
$\newcommand\HPhi{\hat\Phi}$
$\newcommand\HPsi{\hat\Psi}$
$\newcommand\HOmega{\hat\Omega}$
$\newcommand\xhat{\hat\Bx}$
$\newcommand\yhat{\hat\By}$
$\newcommand\zhat{\hat\Bz}$
$\newcommand\ehat{\hat\Be}$
$\newcommand\khat{\hat\Bk}$
$\newcommand\nhat{\hat\Bn}$
$\newcommand\rhat{\hat\Br}$
$\newcommand\phihat{\hat\Bphi}$
$\newcommand\thetahat{\hat\Btheta}$
$\newcommand\MA{\mathbb A}$
$\newcommand\MB{\mathbb B}$
$\newcommand\MC{\mathbb C}$
$\newcommand\MD{\mathbb D}$
$\newcommand\ME{\mathbb E}$
$\newcommand\MF{\mathbb F}$
$\newcommand\MG{\mathbb G}$
$\newcommand\MH{\mathbb H}$
$\newcommand\MI{\mathbb I}$
$\newcommand\MJ{\mathbb J}$
$\newcommand\MK{\mathbb K}$
$\newcommand\ML{\mathbb L}$
$\newcommand\MM{\mathbb M}$
$\newcommand\MN{\mathbb N}$
$\newcommand\MO{\mathbb O}$
$\newcommand\MP{\mathbb P}$
$\newcommand\MQ{\mathbb Q}$
$\newcommand\MR{\mathbb R}$
$\newcommand\MS{\mathbb S}$
$\newcommand\MT{\mathbb T}$
$\newcommand\MU{\mathbb U}$
$\newcommand\MV{\mathbb V}$
$\newcommand\MW{\mathbb W}$
$\newcommand\MX{\mathbb X}$
$\newcommand\MY{\mathbb Y}$
$\newcommand\MZ{\mathbb Z}$
$\newcommand\CA{\mathcal A}$
$\newcommand\CB{\mathcal B}$
$\newcommand\CC{\mathcal C}$
$\newcommand\CD{\mathcal D}$
$\newcommand\CE{\mathcal E}$
$\newcommand\CF{\mathcal F}$
$\newcommand\CG{\mathcal G}$
$\newcommand\CH{\mathcal H}$
$\newcommand\CI{\mathcal I}$
$\newcommand\CJ{\mathcal J}$
$\newcommand\CK{\mathcal K}$
$\newcommand\CL{\mathcal L}$
$\newcommand\CM{\mathcal M}$
$\newcommand\CN{\mathcal N}$
$\newcommand\CO{\mathcal O}$
$\newcommand\CP{\mathcal P}$
$\newcommand\CQ{\mathcal Q}$
$\newcommand\CR{\mathcal R}$
$\newcommand\CS{\mathcal S}$
$\newcommand\CT{\mathcal T}$
$\newcommand\CU{\mathcal U}$
$\newcommand\CV{\mathcal V}$
$\newcommand\CW{\mathcal W}$
$\newcommand\CX{\mathcal X}$
$\newcommand\CY{\mathcal Y}$
$\newcommand\CZ{\mathcal Z}$
$\newcommand\Fa{\mathfrak a}$
$\newcommand\Fb{\mathfrak b}$
$\newcommand\Fc{\mathfrak c}$
$\newcommand\Fd{\mathfrak d}$
$\newcommand\Fe{\mathfrak e}$
$\newcommand\Ff{\mathfrak f}$
$\newcommand\Fg{\mathfrak g}$
$\newcommand\Fh{\mathfrak h}$
$\newcommand\Fi{\mathfrak i}$
$\newcommand\Fj{\mathfrak j}$
$\newcommand\Fk{\mathfrak k}$
$\newcommand\Fl{\mathfrak l}$
$\newcommand\Fm{\mathfrak m}$
$\newcommand\Fn{\mathfrak n}$
$\newcommand\Fo{\mathfrak o}$
$\newcommand\Fp{\mathfrak p}$
$\newcommand\Fq{\mathfrak q}$
$\newcommand\Fr{\mathfrak r}$
$\newcommand\Fs{\mathfrak s}$
$\newcommand\Ft{\mathfrak t}$
$\newcommand\Fu{\mathfrak u}$
$\newcommand\Fv{\mathfrak v}$
$\newcommand\Fw{\mathfrak w}$
$\newcommand\Fx{\mathfrak x}$
$\newcommand\Fy{\mathfrak y}$
$\newcommand\Fz{\mathfrak z}$
$\newcommand\FA{\mathfrak A}$
$\newcommand\FB{\mathfrak B}$
$\newcommand\FC{\mathfrak C}$
$\newcommand\FD{\mathfrak D}$
$\newcommand\FE{\mathfrak E}$
$\newcommand\FF{\mathfrak F}$
$\newcommand\FG{\mathfrak G}$
$\newcommand\FH{\mathfrak H}$
$\newcommand\FI{\mathfrak I}$
$\newcommand\FJ{\mathfrak J}$
$\newcommand\FK{\mathfrak K}$
$\newcommand\FL{\mathfrak L}$
$\newcommand\FM{\mathfrak M}$
$\newcommand\FN{\mathfrak N}$
$\newcommand\FO{\mathfrak O}$
$\newcommand\FP{\mathfrak P}$
$\newcommand\FQ{\mathfrak Q}$
$\newcommand\FR{\mathfrak R}$
$\newcommand\FS{\mathfrak S}$
$\newcommand\FT{\mathfrak T}$
$\newcommand\FU{\mathfrak U}$
$\newcommand\FV{\mathfrak V}$
$\newcommand\FW{\mathfrak W}$
$\newcommand\FX{\mathfrak X}$
$\newcommand\FY{\mathfrak Y}$
$\newcommand\FZ{\mathfrak Z}$
$\newcommand\Da{\dot a}$
$\newcommand\Db{\dot b}$
$\newcommand\Dc{\dot c}$
$\newcommand\Dd{\dot d}$
$\newcommand\De{\dot e}$
$\newcommand\Df{\dot f}$
$\newcommand\Dg{\dot g}$
$\newcommand\Dh{\dot h}$
$\newcommand\Di{\dot \imath}$
$\newcommand\Dj{\dot \jmath}$
$\newcommand\Dk{\dot k}$
$\newcommand\Dl{\dot l}$
$\newcommand\Dm{\dot m}$
$\newcommand\Dn{\dot n}$
$\newcommand\Do{\dot o}$
$\newcommand\Dp{\dot p}$
$\newcommand\Dq{\dot q}$
$\newcommand\Dr{\dot r}$
$\newcommand\Ds{\dot s}$
$\newcommand\Dt{\dot t}$
$\newcommand\Du{\dot u}$
$\newcommand\Dv{\dot v}$
$\newcommand\Dw{\dot w}$
$\newcommand\Dx{\dot x}$
$\newcommand\Dy{\dot y}$
$\newcommand\Dz{\dot z}$
$\newcommand\DA{\dot A}$
$\newcommand\DB{\dot B}$
$\newcommand\DC{\dot C}$
$\newcommand\DD{\dot D}$
$\newcommand\DE{\dot E}$
$\newcommand\DF{\dot F}$
$\newcommand\DG{\dot G}$
$\newcommand\DH{\dot H}$
$\newcommand\DI{\dot I}$
$\newcommand\DJ{\dot J}$
$\newcommand\DK{\dot K}$
$\newcommand\DL{\dot L}$
$\newcommand\DM{\dot M}$
$\newcommand\DN{\dot N}$
$\newcommand\DO{\dot O}$
$\newcommand\DP{\dot P}$
$\newcommand\DQ{\dot Q}$
$\newcommand\DR{\dot R}$
$\newcommand\DS{\dot S}$
$\newcommand\DT{\dot T}$
$\newcommand\DU{\dot U}$
$\newcommand\DV{\dot V}$
$\newcommand\DW{\dot W}$
$\newcommand\DX{\dot X}$
$\newcommand\DY{\dot Y}$
$\newcommand\DZ{\dot Z}$
\( \newcommand\Dalpha

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[1], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dbeta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[2], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dgamma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[3], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Ddelta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[4], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Depsilon

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[5], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dvarepsilon

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[6], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dzeta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[7], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Deta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[8], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dtheta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[9], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dvartheta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[10], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Diota

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[11], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dkappa

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[12], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Dlambda

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[13], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
$\newcommand\Dmu{\dot\mu}$
$\newcommand\Dnu{\dot\nu}$
$\newcommand\Dxi{\dot\xi}$
$\newcommand\Dom{\dot\omicron}$
$\newcommand\Dpi{\dot\pi}$
\( \newcommand\Dvarpi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[14], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
$\newcommand\Drho{\dot\rho}$
$\newcommand\Dvarrho{\dot\varrho}$
$\newcommand\Dsigma{\dot\sigma}$
$\newcommand\Dvarsigma{\dot\varsigma}$
$\newcommand\Dtau{\var\tau}$
$\newcommand\Dupsilon{\dot\upsilon}$
$\newcommand\Dphi{\dot\phi}$
$\newcommand\Dvarphi{\dot\varphi}$
$\newcommand\Dchi{\dot\chi}$
$\newcommand\Dpsi{\dot\psi}$
$\newcommand\Domega{\dot\omega}$
\( \newcommand\DGamma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[15], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\DDelta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[16], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\DTheta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[17], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
$\newcommand\DLambda{\dot\Lambda}$
$\newcommand\DXi{\dot\Xi}$
$\newcommand\DPi{\dot\Pi}$
$\newcommand\DSigma{\dot\Sigma}$
$\newcommand\DUps{\dot\Upsilon}$
$\newcommand\DPhi{\dot\Phi}$
$\newcommand\DPsi{\dot\Psi}$
$\newcommand\DOmega{\dot\Omega}$
$\newcommand\Va{\vec a}$
$\newcommand\Vb{\vec b}$
$\newcommand\Vc{\vec c}$
$\newcommand\Vd{\vec d}$
$\newcommand\Ve{\vec e}$
$\newcommand\Vf{\vec f}$
$\newcommand\Vg{\vec g}$
$\newcommand\Vh{\vec h}$
$\newcommand\Vi{\vec \imath}$
$\newcommand\Vj{\vec \jmath}$
$\newcommand\Vk{\vec k}$
$\newcommand\Vl{\vec l}$
$\newcommand\Vm{\vec m}$
$\newcommand\Vn{\vec n}$
$\newcommand\Vo{\vec o}$
$\newcommand\Vp{\vec p}$
$\newcommand\Vq{\vec q}$
$\newcommand\Vr{\vec r}$
$\newcommand\Vs{\vec s}$
$\newcommand\Vt{\vec t}$
$\newcommand\Vu{\vec u}$
$\newcommand\Vv{\vec v}$
$\newcommand\Vw{\vec w}$
$\newcommand\Vx{\vec x}$
$\newcommand\Vy{\vec y}$
$\newcommand\Vz{\vec z}$
$\newcommand\VA{\vec A}$
$\newcommand\VB{\vec B}$
$\newcommand\VC{\vec C}$
$\newcommand\VD{\vec D}$
$\newcommand\VE{\vec E}$
$\newcommand\VF{\vec F}$
$\newcommand\VG{\vec G}$
$\newcommand\VH{\vec H}$
$\newcommand\VI{\vec I}$
$\newcommand\VJ{\vec J}$
$\newcommand\VK{\vec K}$
$\newcommand\VL{\vec L}$
$\newcommand\VM{\vec M}$
$\newcommand\VN{\vec N}$
$\newcommand\VO{\vec O}$
$\newcommand\VP{\vec P}$
$\newcommand\VQ{\vec Q}$
$\newcommand\VR{\vec R}$
$\newcommand\VS{\vec S}$
$\newcommand\VT{\vec T}$
$\newcommand\VU{\vec U}$
$\newcommand\VV{\vec V}$
$\newcommand\VW{\vec W}$
$\newcommand\VX{\vec X}$
$\newcommand\VY{\vec Y}$
$\newcommand\VZ{\vec Z}$
$\newcommand\Valpha{\vec\alpha}$
$\newcommand\Vbeta{\vec\beta}$
$\newcommand\Vgamma{\vec\gamma}$
$\newcommand\Vdelta{\vec\delta}$
$\newcommand\Vepsilon{\vec\epsilon}$
$\newcommand\Vvarepsilon{\vec\varepsilon}$
$\newcommand\Vzeta{\vec\zeta}$
$\newcommand\Veta{\vec\eta}$
$\newcommand\Vtheta{\vec\theta}$
$\newcommand\Vvartheta{\vec\vartheta}$
$\newcommand\Viota{\vec\iota}$
$\newcommand\Vkappa{\vec\kappa}$
$\newcommand\Vlambda{\vec\lambda}$
\( \newcommand\Vmu

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[18], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vnu

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[19], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vxi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[20], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vom

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[21], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vpi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[22], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarpi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[23], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vrho

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[24], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarrho

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[25], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vsigma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[26], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarsigma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[27], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vtau

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[28], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vupsilon

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[29], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vphi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[30], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vvarphi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[31], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vchi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[32], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vpsi

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[33], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\Vomega

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[34], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\VGamma

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[35], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
\( \newcommand\VDelta

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[36], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
$\newcommand\VTheta{\vec\Theta}$
$\newcommand\VLambda{\vec\Lambda}$
$\newcommand\VXi{\vec\Xi}$
$\newcommand\VPi{\vec\Pi}$
$\newcommand\VSigma{\vec\Sigma}$
$\newcommand\VUps{\vec\Upsilon}$
$\newcommand\VPhi{\vec\Phi}$
$\newcommand\VPsi{\vec\Psi}$
$\newcommand\VOmega{\vec\Omega}$
$\newcommand\BA{\mib A}$
$\newcommand\BB{\mib B}$
$\newcommand\BC{\mib C}$
$\newcommand\BD{\mib D}$
$\newcommand\BE{\mib E}$
$\newcommand\BF{\mib F}$
$\newcommand\BG{\mib G}$
$\newcommand\BH{\mib H}$
$\newcommand\BI{\mib I}}$
$\newcommand\BJ{\mib J}$
$\newcommand\BK{\mib K}$
$\newcommand\BL{\mib L}$
$\newcommand\BM{\mib M}$
$\newcommand\BN{\mib N}$
$\newcommand\BO{\mib O}$
$\newcommand\BP{\mib P}$
$\newcommand\BQ{\mib Q}$
$\newcommand\BR{\mib R}$
$\newcommand\BS{\mib S}$
$\newcommand\BT{\mib T}$
$\newcommand\BU{\mib U}$
$\newcommand\BV{\mib V}$
$\newcommand\BW{\mib W}$
$\newcommand\BX{\mib X}$
$\newcommand\BY{\mib Y}$
$\newcommand\BZ{\mib Z}$
$\newcommand\Ba{\mib a}$
$\newcommand\Bb{\mib b}$
$\newcommand\Bc{\mib c}$
$\newcommand\Bd{\mib d}$
$\newcommand\Be{\mib e}$
$\newcommand\Bf{\mib f}$
$\newcommand\Bg{\mib g}$
$\newcommand\Bh{\mib h}$
$\newcommand\Bi{\mib i}$
$\newcommand\Bj{\mib j}$
$\newcommand\Bk{\mib k}$
$\newcommand\Bl{\mib l}$
$\newcommand\Bm{\mib m}$
$\newcommand\Bn{\mib n}$
$\newcommand\Bo{\mib o}$
$\newcommand\Bp{\mib p}$
$\newcommand\Bq{\mib q}$
$\newcommand\Br{\mib r}$
$\newcommand\Bs{\mib s}$
$\newcommand\Bt{\mib t}$
$\newcommand\Bu{\mib u}$
$\newcommand\Bv{\mib v}$
$\newcommand\Bw{\mib w}$
$\newcommand\Bx{\mib x}$
$\newcommand\By{\mib y}$
$\newcommand\Bz{\mib z}$\)
$\newcommand\vrh{\varrho}$
$\newcommand\vsig{\varsigma}$
$\newcommand\ups{\upsilon}$
$\newcommand\eps{\epsilon}$
$\newcommand\ve{\varepsilon}$
$\newcommand\vth{\vartheta}$
$\newcommand\vphi{\varphi}$
$\newcommand\xhi{\chi}$
$\newcommand\Ups{\Upsilon}$
$\newcommand\Balpha{\mib\alpha}$
$\newcommand\Bbeta{\mib\beta}$
$\newcommand\Bgamma{\mib\gamma}$
$\newcommand\Bdelta{\mib\delta}$
$\newcommand\Beps{\mib\epsilon}$
$\newcommand\Bve{\mib\varepsilon}$
$\newcommand\Bzeta{\mib\zeta}$
$\newcommand\Beta{\mib\eta}$
$\newcommand\Btheta{\mib\theta}$
$\newcommand\Bvth{\mib\vartheta}$
$\newcommand\Biota{\mib\iota}$
$\newcommand\Bkappa{\mib\kappa}$
$\newcommand\Blambda{\mib\lambda}$
$\newcommand\Bmu{\mib\mu}$
$\newcommand\Bnu{\mib\nu}$
$\newcommand\Bxi{\mib\xi}$
$\newcommand\Bom{\mib\omicron}$
$\newcommand\Bpi{\mib\pi}$
$\newcommand\Bvarpi{\mib\varpi}$
$\newcommand\Brho{\mib\rho}$
$\newcommand\Bvrh{\mib\varrho}$
$\newcommand\Bsigma{\mib\sigma}$
$\newcommand\Bvsig{\mib\varsigma}$
$\newcommand\Btau{\mib\tau}$
$\newcommand\Bups{\mib\upsilon}$
$\newcommand\Bphi{\mib\phi}$
$\newcommand\Bvphi{\mib\vphi}$
$\newcommand\Bchi{\mib\chi}$
$\newcommand\Bpsi{\mib\psi}$
$\newcommand\Bomega{\mib\omega}$
$\newcommand\BGamma{\mib\Gamma}$
$\newcommand\BDelta{\mib\Delta}$
$\newcommand\BTheta{\mib\Theta}$
$\newcommand\BLambda{\mib\Lambda}$
$\newcommand\BXi{\mib\Xi}$
$\newcommand\BPi{\mib\Pi}$
$\newcommand\BSigma{\mib\Sigma}$
$\newcommand\BUps{\mib\Upsilon}$
$\newcommand\BPhi{\mib\Phi}$
$\newcommand\BPsi{\mib\Psi}$
$\newcommand\BOmega{\mib\Omega}$
$\newcommand\Bxhi{\raise.35ex\hbox{$\Bchi$}}$
$\newcommand\RGamma{ \Gamma}$
$\newcommand\RDelta{ \Delta}$
$\newcommand\RTheta{ \Theta}$
$\newcommand\RLambda{ \Lambda}$
$\newcommand\RXi{ \Xi}$
$\newcommand\RPi{ \Pi}$
$\newcommand\RSigma{ \Sigma}$
$\newcommand\RUps{ \Upsilon}$
$\newcommand\RPhi{ \Phi}$
$\newcommand\RPsi{ \Psi}$
$\newcommand\ROmega{ \Omega}$
$\newcommand\RA{ A}$
$\newcommand\RB{ B}$
$\newcommand\RC{ C}$
$\newcommand\RD{ D}$
$\newcommand\RE{ E}$
$\newcommand\RF{ F}$
$\newcommand\RG{ G}$
$\newcommand\RH{ H}$
$\newcommand\RI{ I}$
$\newcommand\RJ{ J}$
$\newcommand\RK{ K}$
$\newcommand\RL{ L}$
$\newcommand { M}$
$\newcommand\RN{ N}$
$\newcommand\RO{ O}$
$\newcommand\RP{ P}$
$\newcommand\RQ{ Q}$
$\newcommand\RR{ R}$
$\newcommand\RS{ S}$
$\newcommand\RT{ T}$
$\newcommand\RU{ U}$
$\newcommand\RV{ V}$
$\newcommand\RW{ W}$
$\newcommand\RX{ X}$
$\newcommand\RY{ Y}$
$\newcommand\RZ{ Z}$
$\newcommand\Ra{ a}$
$\newcommand\Rb{ b}$
$\newcommand\Rc{ c}$
$\newcommand\Rd{ d}$
$\newcommand\Re{ e}$
$\newcommand\Rf{ f}$
$\newcommand\Rg{ g}$
$\newcommand\Rh{ h}$
$\newcommand\Ri{ i}$
$\newcommand\Rj{ j}$
$\newcommand\Rk{ k}$
$\newcommand\Rl{ l}$
$\newcommand { m}$
$\newcommand\Rn{ n}$
$\newcommand\Ro{ o}$
$\newcommand\Rp{ p}$
$\newcommand\Rq{ q}$
$\newcommand\Rr{ r}$
$\newcommand\Rs{ s}$
$\newcommand\Rt{ t}$
$\newcommand\Ru{ u}$
$\newcommand\Rv{ v}$
$\newcommand\Rw{ w}$
$\newcommand\Rx{ x}$
$\newcommand\Ry{ y}$
$\newcommand\Rz{ z}$
$\newcommand\BBA{\boldsymbol\RA}$
$\newcommand\BBB{\boldsymbol\RB}$
$\newcommand\BBC{\boldsymbol\RC}$
$\newcommand\BBD{\boldsymbol\RD}$
$\newcommand\BBE{\boldsymbol\RE}$
$\newcommand\BBF{\boldsymbol\RF}$
$\newcommand\BBG{\boldsymbol\RG}$
$\newcommand\BBH{\boldsymbol\RH}$
$\newcommand\BBI{\boldsymbol\RI}$
$\newcommand\BBJ{\boldsymbol\RJ}$
$\newcommand\BBK{\boldsymbol\RK}$
$\newcommand\BBL{\boldsymbol\RL}$
$\newcommand\BBM{\boldsymbol }$
$\newcommand\BBN{\boldsymbol\RN}$
$\newcommand\BBO{\boldsymbol\RO}$
$\newcommand\BBP{\boldsymbol\RP}$
$\newcommand\BBQ{\boldsymbol\RQ}$
$\newcommand\BBR{\boldsymbol\RR}$
$\newcommand\BBS{\boldsymbol\RS}$
$\newcommand\BBT{\boldsymbol\RT}$
$\newcommand\BBU{\boldsymbol\RU}$
$\newcommand\BBV{\boldsymbol\RV}$
$\newcommand\BBW{\boldsymbol\RW}$
$\newcommand\BBX{\boldsymbol\RX}$
$\newcommand\BBY{\boldsymbol\RY}$
$\newcommand\BBZ{\boldsymbol\RZ}$
$\newcommand\BBa{\boldsymbol\Ra}$
$\newcommand\BBb{\boldsymbol\Rb}$
$\newcommand\BBc{\boldsymbol\Rc}$
$\newcommand\BBd{\boldsymbol\Rd}$
$\newcommand\BBe{\boldsymbol\Re}$
$\newcommand\BBf{\boldsymbol\Rf}$
$\newcommand\BBg{\boldsymbol\Rg}$
$\newcommand\BBh{\boldsymbol\Rh}\}$
$\newcommand\BBi{\boldsymbol\Ri}$
$\newcommand\BBj{\boldsymbol\Rj}$
$\newcommand\BBk{\boldsymbol\Rk}$
$\newcommand\BBl{boldsymbol\Rl}$
$\newcommand\BBm{\boldsymbol }$
$\newcommand\BBn{\boldsymbol\Rn}$
$\newcommand\BBo{\boldsymbol\Ro}$
$\newcommand\BBp{\boldsymbol\Rp}$
$\newcommand\BBq{\boldsymbol\Rq}$
$\newcommand\BBr{\boldsymbol\Rr}$
$\newcommand\BBs{\boldsymbol\Rs}$
$\newcommand\BBt{\boldsymbol\Rt}$
$\newcommand\BBu{\boldsymbol\Ru}$
$\newcommand\BBv{\boldsymbol\Rv}$
$\newcommand\BBw{\boldsymbol\Rw}$
$\newcommand\BBx{\boldsymbol\Rx}$
$\newcommand\BBy{\boldsymbol\Ry}$
$\newcommand\BBz{\boldsymbol\Rz}$
\( \newcommand\tcb{\textcolor{blue}\)
\( \newcommand\tcr{\textcolor{red}\)
$\newcommand\bnabla{\boldsymbol{\nabla}}$
$\newcommand\Bell{\boldsymbol\ell}$
$\newcommand\dbar{\,{\mathchar'26\mkern-12mu d}}$
$\newcommand\ns{^\vphantom{*}}$
$\newcommand\uar{\uparrow}$
$\newcommand\dar{\downarrow}$
$\newcommand\impi{\int\limits_{-\infty}^{\infty}\!\!}$
$\newcommand\izpi{\int\limits_{0}^{\infty}\!\!}$
$\newcommand\etc{\it etc.\/}$
$\newcommand\etal{\it et al.\/}$
$\newcommand\opcit{\it op. cit.\/}$
$\newcommand\ie{\it i.e.\/}$
$\newcommand\Ie{\it I.e.\/}$
$\newcommand\viz{\it viz.\/}$
$\newcommand\eg{\it e.g.\/}$
$\newcommand\Eg{\it E.g.\/}$
$\newcommand\dbar{\,{\mathchar'26\mkern-12mu d}}$
$\def\sss#1{\scriptscriptstyle #1}$
$\def\ss#1{\scriptstyle #1}$
$\def\ssr#1{\scriptstyle #1}$
$\def\ssf#1{\scriptstyle #1}$
$\newcommand\NA{N_{\ssr{\!A}}}$
$\newcommand\lala{\langle\!\langle}$
$\newcommand\rara{\rangle\!\rangle}$
$\newcommand\blan{\big\langle}$
$\newcommand\bran{\big\rangle}$
$\newcommand\Blan{\Big\langle}$
$\newcommand\Bran{\Big\rangle}$
$\newcommand\intl{\int\limits}$
$\newcommand\half{\frac{1}{2}}$
$\newcommand\third{\frac{1}{3}}$
$\newcommand\fourth{\frac{1}{4}}$
$\newcommand\eighth{\frac{1}{8}}$
$\newcommand\uar{\uparrow}$
$\newcommand\dar{\downarrow}$
$\newcommand\undertext#1{$\underline{\hbox{#1}}$}$
$\newcommand\Tra{\mathop{\textsf{Tr}}\,}$
$\newcommand\det{\mathop{\textsf{det}}\,}$
$\def\tket#1{|  #1 \rangle}$
$\def\tbra#1{\langle #1|}$
$\def\tbraket#1#2{\langle #1  |   #2 \rangle}$
$\def\texpect#1#2#3{\langle #1 |   #2  |  #3 \rangle}$
$\def\sket#1{|  \, #1 \,  \rangle}$
$\def\sbra#1{\langle \,  #1 \, |}$
$\def\sbraket#1#2{\langle \, #1  \, |  \, #2 \,  \rangle}$
$\def\sexpect#1#2#3{\langle \, #1 \, | \,  #2  \, | \, #3 \, \rangle}$
$\def\ket#1{\big| \, #1\, \big\rangle}$
$\def\bra#1{\big\langle \, #1 \, \big|}$
$\def\braket#1#2{\big\langle \, #1\, \big| \,#2 \,\big\rangle}$
$\def\expect#1#2#3{\big\langle\, #1\, \big|\, #2\, \big| \,#3\, \big\rangle}$
$\newcommand\pz{\partial}$
$\newcommand\pzb{\bar{\partial}}$
$\newcommand\svph{\vphantom{\int}}$
$\newcommand\vph{\vphantom{\sum_i}}$
$\newcommand\bvph{\vphantom{\sum_N^N}}$
$\newcommand\nd{^{\vphantom{\dagger}}}$
$\newcommand\ns{^{\vphantom{*}}}$
$\newcommand\yd{^\dagger}$
$\newcommand\zb{\bar z}$
$\newcommand\zdot{\dot z}$
$\newcommand\zbdot{\dot{\bar z}}$
$\newcommand\kB{k_{\sss{B}}}$
$\newcommand\kT{k_{\sss{B}}T}$
$\newcommand\gtau{g_\tau}$
$\newcommand\Htil{\tilde H}$
$\newcommand\pairo{(\phi\nd_0,J\nd_0)}$
$\newcommand\pairm{(\phi\nd_0,J)}$
$\newcommand\pairob{(\Bphi\nd_0,\BJ\nd_0)}$
$\newcommand\pairmb{(\Bphi\nd_0,\BJ)}$
$\newcommand\pair{(\phi,J)}$
$\newcommand\Hz{H\nd_0}$
$\newcommand\Ho{H\nd_1}$
$\newcommand\Htz{\Htil\nd_0}$
$\newcommand\Hto{\Htil\nd_1}$
$\newcommand\oc{\omega_\Rc}$

$\newcommand \gtwid{\approx}$

$\newcommand\index{\textsf{ind}}$
$\newcommand\csch{\,{ csch\,}}$
$\newcommand\ctnh{\,{ ctnh\,}}$
$\newcommand\ctn{\,{ ctn\,}}$
$\newcommand\sgn{\,{ sgn\,}}$
$\def\tmapright#1{\xrightarrow \limits^{#1}}$
$\def\bmapright#1{\xrightarrow\limits_{#1}}$
$\newcommand\hfb{\hfill\break}$
$\newcommand\Rep{\textsf{Re}\,}$
$\newcommand\Imp{\textsf{Im}\,}$
$\newcommand\ncdot{\!\cdot\!}$
$\def\tmapright#1{ \smash{\mathop{\hbox to 35pt{\rightarrowfill}}\limits^{#1}}\ }$
$\def\bmapright#1{ \smash{\mathop{\hbox to 35pt{\rightarrowfill}}\limits_{#1}}\ }$
$\newcommand\bsqcap{\mbox{\boldmath{$\sqcap$}}}$

$\def\pabc#1#2#3{\left({\pz #1\over\pz #2}\right)\ns_{\!\!#3}}$
$\def\spabc#1#2#3{\big({\pz #1\over\pz #2}\big)\ns_{\!#3}}$
$\def\qabc#1#2#3{\pz^2\! #1\over\pz #2\,\pz #3}$
$\def\rabc#1#2#3#4{(\pz #1,\pz #2)\over (\pz #3,\pz #4)}$
$\newcommand\subA{\ns_\ssr{A}}$
$\newcommand\subB{\ns_\ssr{B}}$
$\newcommand\subC{\ns_\ssr{C}}$
$\newcommand\subD{\ns_\ssr{D}}$
$\newcommand\subAB{\ns_\ssr{AB}}$
$\newcommand\subBC{\ns_\ssr{BC}}$
$\newcommand\subCD{\ns_\ssr{CD}}$
$\newcommand\subDA{\ns_\ssr{DA}}$
$\def\lmapright#1{\ \ \smash{\mathop{\hbox to 55pt{\rightarrowfill}}\limits^{#1}}\ \ }$
$\def\enth#1{\RDelta {\textsf H}^0_\Rf[{ #1}]}$
$\newcommand\longrightleftharpoons{ \mathop{\vcenter{\hbox{\ooalign{\raise1pt\hbox{$\longrightharpoonup\joinrel$}\crcr  \lower1pt\hbox{$\longleftharpoondown\joinrel$}}}}}}$
$\newcommand\longrightharpoonup{\relbar\joinrel\rightharpoonup}$
$\newcommand\longleftharpoondown{\leftharpoondown\joinrel\relbar}$
$\newcommand\cds{\,\bullet\,}$
$\newcommand\ccs{\,\circ\,}$
$\newcommand\nsub{_{\vphantom{\dagger}}}$
$\newcommand\rhohat{\hat\rho}$
$\newcommand\vrhhat{\hat\vrh}$
$\newcommand\impi{\int\limits_{-\infty}^\infty\!\!\!}$
$\newcommand\brangle{\big\rangle}$
$\newcommand\blangle{\big\langle}$
$\newcommand\vet{\tilde\ve}$
$\newcommand\zbar{\bar z}$
$\newcommand\ftil{\tilde f}$
$\newcommand\XBE{\RXi\ns_\ssr{BE}}$
$\newcommand\XFD{\RXi\ns_\ssr{FD}}$
$\newcommand\OBE{\Omega\ns_\ssr{BE}}$
$\newcommand\OFD{\Omega\ns_\ssr{FD}}$
$\newcommand\veF{\ve\ns_\RF}$
$\newcommand\kF{k\ns_\RF}$
$\newcommand\kFu{k\ns_{\RF\uar}}$
$\newcommand\SZ{\textsf Z}}$ \( \newcommand\kFd{k\ns_{\RF\dar}\)
$\newcommand\muB{\mu\ns_\ssr{B}}$
$\newcommand\mutB{\tilde\mu}\ns_\ssr{B}$
$\newcommand\xoN{\Bx\ns_1\,,\,\ldots\,,\,\Bx\ns_N}$
$\newcommand\rok{\Br\ns_1\,,\,\ldots\,,\,\Br\ns_k}$
$\newcommand\xhiOZ{\xhi^\ssr{OZ}}$
\( \newcommand\xhihOZ

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/span[1], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
$\newcommand\jhz{\HJ(0)}$
$\newcommand\nda{\nd_\alpha}$
$\newcommand\ndap{\nd_{\alpha'}}$
\( \newcommand\labar

ParseError: invalid DekiScript (click for details)

Callstack:
    at (Template:MathJaxArovas), /content/body/div/span[2], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.02:_Basic_Concepts_in_Probability_Theory), /content/body/p/span, line 1, column 23

\)
$\newcommand\msa{m\ns_\ssr{A}}$
$\newcommand\msb{m\ns_\ssr{B}}$
$\newcommand\mss{m\ns_\Rs}$
$\newcommand\HBx{\hat\Bx}$
$\newcommand\HBy{\hat\By}$
$\newcommand\HBz{\hat\Bz}$
$\newcommand\thm{\theta\ns_m}$
$\newcommand\thp{\theta\ns_\phi}$
$\newcommand\mtil{\widetilde m}$
$\newcommand\phitil{\widetilde\phi}$
$\newcommand\delf{\delta\! f}$
$\newcommand\coll{\bigg({\pz f\over\pz t}\bigg)\nd_{\! coll}}$
$\newcommand\stre{\bigg({\pz f\over\pz t}\bigg)\nd_{\! str}}$
$\newcommand\idrp{\int\!\!{d^3\!r\,d^3\!p\over h^3}\>}$
$\newcommand\vbar{\bar v}$
$\newcommand\BCE{\mbox{\boldmath{$\CE$}}\!}$
$\newcommand\BCR{\mbox{\boldmath{$\CR$}}\!}$
$\newcommand\gla{g\nd_{\RLambda\nd}}$
$\newcommand\TA{T\ns_\ssr{A}}$
$\newcommand\TB{T\ns_\ssr{B}}$
$\newcommand\ncdot{\!\cdot\!}$
$\newcommand\NS{N\ns_{\textsf S}}$

Fundamental definitions

The natural mathematical setting is set theory. Sets are generalized collections of objects. The basics: $\omega\in A$ is a binary relation which says that the object $\omega$ is an element of the set $A$. Another binary relation is set inclusion. If all members of $A$ are in $B$, we write $A\subseteq B$. The union of sets $A$ and $B$ is denoted $A\cup B$ and the intersection of $A$ and $B$ is denoted $A\cap B$. The Cartesian product of $A$ and $B$, denoted $A\times B$, is the set of all ordered elements $(a,b)$ where $a\in A$ and $b\in B$.

Some details: If $\omega$ is not in $A$, we write $\omega\notin A$. Sets may also be objects, so we may speak of sets of sets, but typically the sets which will concern us are simple discrete collections of numbers, such as the possible rolls of a die {1,2,3,4,5,6}, or the real numbers $\MR$, or Cartesian products such as $\MR^N$. If $A\subseteq B$ but $A\ne B$, we say that $A$ is a proper subset of $B$ and write $A\subset B$. Another binary operation is the set difference $A\backslash B$, which contains all $\omega$ such that $\omega\in A$ and $\omega\notin B$.

In probability theory, each object $\omega$ is identified as an event. We denote by $\ROmega$ the set of all events, and $\emptyset$ denotes the set of no events. There are three basic axioms of probability:

• To each set $A$ is associated a non-negative real number $P(A)$, which is called the probability of $A$.
• $P(\ROmega)=1$.
• If $\{A\ns_i\}$ is a collection of disjoint sets, if $A\ns_i\cap A\ns_j=\emptyset$ for all $i\ne j$, then $$P\Big(\bigcup_i A\ns_i\Big)=\sum_i P(A\ns_i)\ .$$

From these axioms follow a number of conclusions. Among them, let $\neg A=\ROmega\backslash A$ be the complement of $A$, the set of all events not in $A$. Then since $A\cup\neg A=\ROmega$, we have $P(\neg A)=1-P(A)$. Taking $A=\ROmega$, we conclude $P(\emptyset)=0$.

The meaning of $P(A)$ is that if events $\omega$ are chosen from $\ROmega$ at random, then the relative frequency for $\omega\in A$ approaches $P(A)$ as the number of trials tends to infinity. But what do we mean by ’at random’? One meaning we can impart to the notion of randomness is that a process is random if its outcomes can be accurately modeled using the axioms of probability. This entails the identification of a probability space $\ROmega$ as well as a probability measure $P$. For example, in the microcanonical ensemble of classical statistical physics, the space $\ROmega$ is the collection of phase space points $\Bvphi=\{q\ns_1,\ldots,q\ns_n,p\ns_1,\ldots,p\ns_n\}$ and the probability measure is $d\mu=\Sigma^{-1}(E)\prod_{i=1}^n dq\ns_i\, dp\ns_i\,\delta\big(E-H(q,p)\big)$, so that for $A\in\ROmega$ the probability of $A$ is $P(A)=\int\!d\mu\>\xhi\nd_A(\Bvphi)$, where $\xhi\ns_A(\Bvphi)=1$ if $\Bvphi\in A$ and $\xhi\ns_A(\Bvphi)=0$ if $\Bvphi\notin A$ is the characteristic function of $A$. The quantity $\Sigma(E)$ is determined by normalization: $\int\!d\mu = 1$.

Bayesian Statistics

We now introduce two additional probabilities. The joint probability for sets $A$ and $B$ together is written $P(A\cap B)$. That is, $P(A\cap B)={Prob}[\omega\in A\ {and}\ \omega\in B]$. For example, $A$ might denote the set of all politicians, $B$ the set of all American citizens, and $C$ the set of all living humans with an IQ greater than 60. Then $A\cap B$ would be the set of all politicians who are also American citizens, Exercise: estimate $P(A\cap B\cap C)$.

The conditional probability of $B$ given $A$ is written $P(B|A)$. We can compute the joint probability $P(A\cap B)=P(B\cap A)$ in two ways:

$$P(A\cap B)=P(A|B)\cdot P(B)=P(B|A)\cdot P(A)\ .$$ Thus, $$P(A|B)={P(B|A)\, P(A)\over P(B)}\ ,$$ a result known as Bayes’ theorem. Now suppose the ‘event space’ is partitioned as $\{A\ns_i\}$. Then $$P(B)=\sum_i P(B|A\ns_i)\, P(A\ns_i)\ .$$ We then have $$P(A\ns_i|B)={P(B|A\ns_i)\, P(A\ns_i)\over\sum_j P(B|A\ns_j)\, P(A\ns_j)}\ ,$$ a result sometimes known as the extended form of Bayes’ theorem. When the event space is a ‘binary partition’ $\{A,\neg A\}$, we have $$P(A|B)={P(B|A)\, P(A)\over P(B|A)\, P(A) + P(B|\neg A)\, P(\neg A)}\ . \label{Bayesbinary}$$ Note that $P(A|B)+P(\neg A|B)=1$ (which follows from $\neg\neg A = A$).

As an example, consider the following problem in epidemiology. Suppose there is a rare but highly contagious disease $A$ which occurs in $0.01\%$ of the general population. Suppose further that there is a simple test for the disease which is accurate $99.99\%$ of the time. That is, out of every 10,000 tests, the correct answer is returned 9,999 times, and the incorrect answer is returned only once. Now let us administer the test to a large group of people from the general population. Those who test positive are quarantined. Question: what is the probability that someone chosen at random from the quarantine group actually has the disease? We use Bayes’ theorem with the binary partition $\{A,\neg A\}$. Let $B$ denote the event that an individual tests positive. Anyone from the quarantine group has tested positive. Given this datum, we want to know the probability that that person has the disease. That is, we want $P(A|B)$. Applying Equation [Bayesbinary] with

$$P(A)=0.0001 \quad,\quad P(\neg A)=0.9999 \quad,\quad P(B|A)=0.9999 \quad,\quad P(B|\neg A)=0.0001\ ,$$ we find $P(A|B)=\half$. That is, there is only a $50\%$ chance that someone who tested positive actually has the disease, despite the test being $99.99\%$ accurate! The reason is that, given the rarity of the disease in the general population, the number of false positives is statistically equal to the number of true positives.

In the above example, we had $P(B|A)+P(B|\neg A)=1$, but this is not generally the case. What is true instead is $P(B|A)+P(\neg B|A)=1$. Epidemiologists define the sensitivity of a binary classification test as the fraction of actual positives which are correctly identified, and the specificity as the fraction of actual negatives that are correctly identified. Thus, $\textsf{se}=P(B|A)$ is the sensitivity and $\textsf{sp}=P(\neg B|\neg A)$ is the specificity. We then have $P(B|\neg A)=1-P(\neg B|\neg A)$. Therefore,

$$P(B|A)+P(B|\neg A)=1+P(B|A)-P(\neg B|\neg A)=1+\textsf{se}-\textsf{sp}\ .$$ In our previous example, $\textsf{se}=\textsf{sp}=0.9999$, in which case the RHS above gives $1$. In general, if $P(A)\equiv f$ is the fraction of the population which is afflicted, then $$P(\textsf{infected}\,|\,\textsf{positive})={f\cdot\textsf{se}\over f\cdot\textsf{se} + (1-f)\cdot(1-\textsf{sp})}\ .$$

For continuous distributions, we speak of a probability density. We then have

$$P(y)=\int\!dx\>P(y|x)\, P(x)$$ and $$P(x|y)={P(y|x)\, P(x)\over\int\!dx'\> P(y|x')\, P(x')}\ .$$ The range of integration may depend on the specific application.

The quantities $P(A\ns_i)$ are called the prior distribution. Clearly in order to compute $P(B)$ or $P(A\ns_i|B)$ we must know the priors, and this is usually the weakest link in the Bayesian chain of reasoning. If our prior distribution is not accurate, Bayes’ theorem will generate incorrect results. One approach to approximating prior probabilities $P(A\ns_i)$ is to derive them from a maximum entropy construction.

Random variables and their averages

Consider an abstract probability space $\CX$ whose elements ( events) are labeled by $x$. The average of any function $f(x)$ is denoted as $\ME f$ or $\langle f\rangle$, and is defined for discrete sets as

$$\ME f=\langle f\rangle=\sum_{x\in\CX} f(x)\, P(x)\ ,$$ where $P(x)$ is the probability of $x$. For continuous sets, we have $$\ME f=\langle f\rangle=\int\limits_{\CX}\!\!dx\> f(x)\, P(x)\ .$$ Typically for continuous sets we have $\CX=\MR$ or $\CX=\MR\ns_{\ge 0}$. Gardiner and other authors introduce an extra symbol, $X$, to denote a random variable, with $X(x)=x$ being its value. This is formally useful but notationally confusing, so we’ll avoid it here and speak loosely of $x$ as a random variable.

When there are two random variables $x\in\CX$ and $y\in\CY$, we have $\ROmega=\CX\times\CY$ is the product space, and

$$\ME f(x,y)=\langle f(x,y)\rangle =\sum_{x\in\CX}\sum_{y\in\CY} f(x,y)\,P(x,y)\ ,$$ with the obvious generalization to continuous sets. This generalizes to higher rank products, $x\ns_i\in\CX\ns_i$ with $i\in\{1,\ldots,N\}$. The covariance of $x\ns_i$ and $x\ns_j$ is defined as $$C\ns_{ij}\equiv \blangle \big( x\ns_i - \langle x\ns_i\rangle \big) \big( x\ns_j - \langle x\ns_j\rangle \big) \brangle =\langle x\ns_i x\ns_j\rangle - \langle x\ns_i\rangle \langle x\ns_j\rangle\ .$$

If $f(x)$ is a convex function then one has

$$\ME f(x) \ge f(\ME x)\ .$$ For continuous functions, $f(x)$ is convex if $f''(x)\ge 0$ everywhere⁴. If $f(x)$ is convex on some interval $[a,b]$ then for $x\ns_{1,2}\in [a,b]$ we must have $$f\big(\lambda x\ns_1 + (1-\lambda) x\ns_2\big) \le \lambda f(x\ns_1) + (1-\lambda) f(x\ns_2)\ ,$$ where $\lambda\in [0,1]$. This is easily generalized to $$f\Big(\sum_n p\ns_n x\ns_n\Big) \le \sum_n p\ns_n f(x\ns_n)\ ,$$ where $p\ns_n=P(x\ns_n)$, a result known as Jensen’s theorem.