Processing math: 99%
Skip to main content
Library homepage
 

Text Color

Text Size

 

Margin Size

 

Font Type

Enable Dyslexic Font
Physics LibreTexts

1.S: Summary

( \newcommand{\kernel}{\mathrm{null}\,}\)




























































































































































































































































































































\( \newcommand\Dalpha

ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[1], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dbeta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[2], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dgamma
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[3], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Ddelta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[4], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Depsilon
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[5], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dvarepsilon
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[6], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dzeta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[7], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Deta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[8], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dtheta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[9], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dvartheta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[10], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Diota
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[11], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dkappa
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[12], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Dlambda
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[13], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)





\( \newcommand\Dvarpi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[14], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)











\( \newcommand\DGamma
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[15], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\DDelta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[16], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\DTheta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[17], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)









































































\( \newcommand\Vmu
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[18], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vnu
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[19], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vxi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[20], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vom
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[21], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vpi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[22], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vvarpi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[23], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vrho
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[24], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vvarrho
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[25], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vsigma
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[26], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vvarsigma
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[27], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vtau
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[28], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vupsilon
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[29], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vphi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[30], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vvarphi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[31], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vchi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[32], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vpsi
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[33], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\Vomega
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[34], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\VGamma
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[35], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)
\( \newcommand\VDelta
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/p[1]/span[36], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)

















\newcommand\BI{\mib I}}










































\)










































































\newcommand { M}

























\newcommand { m}














































}


















\( \newcommand\tcb{\textcolor{blue}\)
\( \newcommand\tcr{\textcolor{red}\)



































1$#1_$






















































































\newcommand\SZ{\textsf Z}} \( \newcommand\kFd{k\ns_{\RF\dar}\)

\newcommand\mutB{\tilde\mu}\ns_\ssr{B}



\( \newcommand\xhihOZ
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/span[1], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)



\( \newcommand\labar
ParseError: invalid DekiScript (click for details)
Callstack:
    at (Template:MathJaxArovas), /content/body/div/span[2], line 1, column 1
    at template()
    at (Bookshelves/Thermodynamics_and_Statistical_Mechanics/Book:_Thermodynamics_and_Statistical_Mechanics_(Arovas)/01:_Fundamentals_of_Probability/1.S:_Summary), /content/body/p/span, line 1, column 23
\)





















References

  • C. Gardiner, Stochastic Methods (4th edition, Springer-Verlag, 2010) Very clear and complete text on stochastic methods with many applications.
  • J. M. Bernardo and A. F. M. Smith, Bayesian Theory (Wiley, 2000) A thorough textbook on Bayesian methods.
  • D. Williams, Weighing the Odds: A Course in Probability and Statistics (Cambridge, 2001) A good overall statistics textbook, according to a mathematician colleague.
  • E. T. Jaynes, Probability Theory (Cambridge, 2007) An extensive, descriptive, and highly opinionated presentation, with a strongly Bayesian approach.
  • A. N. Kolmogorov, Foundations of the Theory of Probability (Chelsea, 1956) The Urtext of mathematical probability theory.

Summary

Discrete distributions: Let n label the distinct possible outcomes of a discrete random process, and let pn be the probability for outcome n. Let A be a quantity which takes values which depend on n, with An being the value of A under the outcome n. Then the expected value of A is A=npnAn, where the sum is over all possible allowed values of n. We must have that the distribution is normalized, 1=npn=1.

Continuous distributions: When the random variable φ takes a continuum of values, we define the probability density P(φ) to be such that P(φ)dμ is the probability for the outcome to lie within a differential volume dμ of φ, where dμ=W(φ)ni=1dφi, were φ is an n-component vector in the configuration space Ω, and where the function W(φ) accounts for the possibility of different configuration space measures. Then if A(φ) is any function on Ω, the expected value of A is A=ΩdμP(φ)A(φ).

Central limit theorem: If {x1,,xN} are each independently distributed according to P(x), then the distribution of the sum X=Ni=1xi is PN(X)=dx1dxNP(x1)P(xN)δ(XNi=1xi)=dk2π[ˆP(k)]NeikX , where ˆP(k)=dxP(x)eikx is the Fourier transform of P(x). Assuming that the lowest moments of P(x) exist, ln[ˆP(k)]=iμk12σ2k2+O(k3), where μ=x and σ2=x2x2 are the mean and standard deviation. Then for N, PN(X)=(2πNσ2)1/2e(XNμ)2/2Nσ2 , which is a Gaussian with mean X=Nμ and standard deviation X2X2=Nσ. Thus, X is distributed as a Gaussian, even if P(x) is not a Gaussian itself.

Entropy: The entropy of a statistical distribution is {pn} is S=npnlnpn. (Sometimes the base 2 logarithm is used, in which case the entropy is measured in bits.) This has the interpretation of the information content per element of a random sequence.

Distributions from maximum entropy: Given a distribution {pn} subject to (K+1) constraints of the form Xa=nXanpn with a{0,,K}, where X0=X0n=1 (normalization), the distribution consistent with these constraints which maximizes the entropy function is obtained by extremizing the multivariable function S({pn},{λa})=npnlnpnKa=0λa(nXanpnXa) , with respect to the probabilities {pn} and the Lagrange multipliers {λa}. This results in a Gibbs distribution, pn=1Zexp{Ka=1λaXan} , where Z=e1+λ0 is determined by normalization, npn=1 ( the a=0 constraint) and the K remaining multipliers determined by the K additional constraints.

Multidimensional Gaussian integral: dx1dxnexp(12xiAijxj+bixi)=((2π)ndetA)1/2exp(12biA1ijbj) .

Bayes’ theorem: Let the conditional probability for B given A be P(B|A). Then Bayes’ theorem says P(A|B)=P(A)P(B|A)/P(B). If the ’event space’ is partitioned as {Ai}, then we have the extended form, P(Ai|B)=P(B|Ai)P(Ai)jP(B|Aj)P(Aj) . When the event space is a ‘binary partition’ {A,¬A}, as is often the case in fields like epidemiology ( test positive or test negative), we have P(A|B)=P(B|A)P(A)P(B|A)P(A)+P(B|¬A)P(¬A) . Note that P(A|B)+P(¬A|B)=1 (which follows from ¬¬A=A).

Updating Bayesian priors: Given data in the form of observed values x={x1,,xN}X and a hypothesis in the form of parameters θ={θ1,,θK}Θ, we write the conditional probability (density) for observing x given θ as f(x|θ). Bayes’ theorem says that the corresponding distribution π(θ|x) for θ conditioned on x is π(θ|x)=f(x|θ)π(θ)Θdθf(x|θ)π(θ) , We call π(θ) the prior for θ, f(x|θ) the likelihood of x given θ, and π(θ|x) the posterior for θ given x. We can use the posterior to find the distribution of new data points y, called the posterior predictive distribution, f(y|x)=Θdθf(y|θ)π(θ|x). This is the update of the prior predictive distribution, f(x)=Θdθf(x|θ)π(θ) . As an example, consider coin flipping with f(x|θ)=θX(1θ)NX, where N is the number of flips, and X=Nj=1xj with xj a discrete variable which is 0 for tails and 1 for heads. The parameter θ[0,1] is the probability to flip heads. We choose a prior π(θ)=θα1(1θ)β1/B(α,β) where \SB(\alpha,\beta)=\RGamma(\alpha)\,\RGamma(\beta)/\RGamma(\alpha+\beta) is the Beta distribution. This results in a normalized prior \int\limits_0^1 \!d\theta\,\pi(\theta)=1. The posterior distribution for \theta is then \pi(\theta|x\ns_1,\ldots,x\ns_N)={f(x\ns_1,\ldots,x\ns_N|\theta)\,\pi(\theta)\over\int_0^1\!d\theta'\, f(x\ns_1,\ldots,x\ns_N|\theta')\,\pi(\theta')} ={\theta^{X+\alpha-1} (1-\theta)^{N-X+\beta-1}\over\SB(X+\alpha,N-X+\beta)}\ . The prior predictive is f(\Bx)=\int\limits_0^1\!d\theta f(\Bx|\theta)\,\pi(\theta)=\SB(X+\alpha,N-X+\beta)/\SB(\alpha,\beta) , and the posterior predictive for the total number of heads Y in M flips is f(\By|\Bx)=\!\!\int\limits_0^1\!d\theta\,f(\By|\theta)\,\pi(\theta|\Bx)= {\SB(X+Y+\alpha,N-X+M-Y+\beta)\over\SB(X+\alpha,N-X+\beta)}\quad.


This page titled 1.S: Summary is shared under a CC BY-NC-SA license and was authored, remixed, and/or curated by Daniel Arovas.

Support Center

How can we help?