Computational Statistics [Martinez 2001]{Miller] [Mario F. Triola 2000]

2.2 : PROBABILITY

Random Experiment: process or action whose outcome can't be predicted w certainty and would likely change when d exp is repeated or function defined over the elements of sample space (Miller et al.)

Random variable, X : outcome from random experiment.
x is the observed value of a random variable X
discrete ran var - can take value from a finite or countably infinite
continuous ran var - can take values from an interval of real number

Sample space, S : set of all outcomes from exp. ex: 6-sided dice : sample space [ 1 2 3 4 5 6 ]
ANXIOM 2: P(S) = 1

Event, E : subset of outcomes in the sample space
AXIOM 1 : Probability event E mest be between 0 and 1 0 ≤ P(E) ≤ 1

Mutually exclusive events: two events that can't occur simaltaneously or jointly. can be extended to any num of events as long as all pairs of events is considered
AXIOM 3 : Mutually exclusive events E1, E2, ... EK
P(E 1 U E2 U...U EK)= ∑ i=1 to K P(Ei)

Probability : Measure of the likelihood that some event will occur

Probability distribution f(x): describes the probabilities associated w each possible value for the random variables

Cummulative distribution function, cdf, F(x): probabilty that the ran var X assumes a value
less than or equal to a given x. F(x) take value from zero to one

Probability density function: Probability distribution for continuous random variables
f(x) = P(a ≤ X ≤ b) = integrat'n from a to b: total area under the curve = 1
associated cdf : F(x) = P ( X ≤ x ) = (integration from -inf to x)

Probability mass function: Probabilty distribution of discrete random variables
f(xi) = P (X = xi) ; i = 1,2, ... ,
associated cdf: F(a) = ∑ xi ≤ a f(xi)

Equal likelihood model: Experiment where each of n outcomes is equally likely,
and assign a probability mass of 1/n

Relative frequency method: conduct the experiment n times and record d outcomes
probability is assigned by P(E) = f/n

2.3 : CONDITIONAL PROBABILITY AND INDEPENDENCE

Conditional Probability: event E given event F -->

P(E|F) = P(E ∩ F)/P(F) ; P(F) > 0
P(E ∩ F) = P(F) P(E|F) or P(E) P(F|E)

Independent Events:
P(E ∩ F) = P(E) P(F) or P(F) P(E)
P(E) = P(E ∩ F)/P(F)= P(E|F)
Therefore P(E) = P(E|F) or P(F) = P(F|E)

if extended to k events

P(E1 ∩ E2 ∩ ... ∩ EK) = ∏ i=1 to K P(Ei)

Bayes Theorem: initial probability is called prior probability.
new info is used to update prior probability to obtained posterior probability.

P(Er|F) = event Er given event F or 'effect' F was 'caused' by the event Er

P(Er|F) = P(Er ∩ F) / P(F) = P(Er) P(F|Er) /P(F)
P (F) = P(E1 ∩ F) + P(E2 ∩ F) +... +P(EK ∩F)
= P(E1) P(F|E1) + ... + P(EK) P(EK|F)
therefore

P(Er|F) = P(Er) P(F|Er) / P(E1) P(F|E1) + ... + P(EK) P(EK|F)

Example:

Tom services 60% of all cars and fail to wash the windshiled 1/10 time.
George services 15% of all cars and fail to wash the windshiled 1/10 time.
Jack services 20% of all cars and fail to wash the windshiled 1/ 20 time.
Peter services 5% of all cars and fail to wash the windshiled 1/20 time.
If customer complains later that her windshield ws not washed,
What is the probability that her car was serviced by jack?

P(Er|F)=P(E∩F)/P(F)
P(Er|F)=P(Er)P(F|Er)/[P(E1)P(F|E1)+P(E2)P(F|E2)+...+P(E4)P(F|E4)]
P(Er|F)=(0.2)(1/20)/[(0.6)(1/10)+(0.15)(1/10)+(0.2)(1/20)+(0.05)(1/20)]
P(Er|F)=0.114

therefore the probability that the windshield not washed (effect F) caused by Jack (event Er) is 0.114. This shows that even Jack only fail 1 windshield in 20 cars, 11% of windsheild failures are his responsibility.


2.4: EXPECTATION

Mathematical expectations have been playing an increasingly important role in scientific decision making, as it generally considered rational to select which ever alternative has the most promising mathematical expectation.

Mean provides a measure of central tendency of the distribution.

• Mean of n measurements : Arithmetic mean (data treatment)
• μ = ∑x / n

• Mean of group data ( frequency)
• μ = ∑(f x) / N

• Mean of probability distribution
• Mean or expected value of random variable defined using pdf or pmf.
• Expected value is sum of all possible values of the ran var where each one is weighted by the probability that X will take on a value. i.e: the probability of obtaining a1,a2,...,ai is p1,p2,...,pi
• μ = E[X] = a1p1 + a2p2 + ... + aipi = ∑(xi f(xi))

Variance is a measureb of dispersion in the distribution ( how much a single random variable varies). Large variance means that the observed value is most likely to be far from mean μ.

• Variance of n observation
  
• V(X) = ∑(x-μ)² /(n-1) = [n∑x² -(∑x)²]/n(n-1)

• Variance of group data (frequency)
• V(X) =[n∑x²f -(∑xf)²]/n(n-1)

• Variance of probability distribution
• Variance it the sum of the squared distances
• each one weighted by the probability that X = x.

• V(X) = E[(X-μ)²] = ∑(x-μ)² f(x)
  

• V(X) = E[X²]-μ² = E[X²]-(E[X])²
  

2.5:COMMON DISTRIBUTION

Discrete distribution: Binomial, Poisson

Continuous distribution: uniform, Normal, Exponential, Gamma. Chi-square, Weibull, beta, Multivariate Normal