Posted on

Probability for machine learning and data science – Basic Probability 3 of 6

In this post we will look at Counting. This post is organised as follows

  1. Basic rule of counting
  2. Permutations
  3. Combinations

Basic rule of counting

Take k tasks such that task I has n_i ways of doing it, then the total number of ways of doing k tasks is

n_1 \times n_2 \times n_3 \times .... \times n_4

We will see this by using an example

Lets take my favourite biryani restaurant which sells the following

  • 3 types of biryani
  • 15 types of gravies
  • 3 types of desserts

Now that I like biryani I need to choose 1 biryani, 1 gravy and 1 dessert. In how many ways can I do it?

let us break the problem in notation and do it. First we need to determine the k value

The k value for the problem is 3

k = 3

tasks with

n_1 = 3, n_2 = 15, n_3 = 3

So when using the formula above we have

3 \times 15 \times 3 = 135

So i can choose between 135 possible ways of my food.

Before we wrote

n_1 \times n_2 \times n_3 \times …. \times n_4

we can write this in product notation like this

\prod\limits_{i=1}^{k}n_i

Permutations

A permutation is defined as the number of ways of ordering n distant objects taken r at a time

^n p_r = n(n-1)(n-2)...(n-r+1)

Factorials

Lets us revise what a factorial is

the notation n! called n factorial is defined as

n! = n(n-1)(n-2)... 2 \times 1

Factorial in product notation

\prod\limits_{i=0}^{n-1}(n-i)

Lets write the permutation formula again

^n p_r = n(n-1)(n-2)...(n-r+1) = \frac{n!}{(n-r)!}

Combinations

The combination is dealing with when we take objects in which order does not matter.

Combinations can be calculated by using

\binom{n}{r} = \frac{n!}{r!(n-r)!}

You can read more about Counting, Permutations and Combinations from this book.

The topics covered in my posts are from this book.

Happy Learning

This post is made possible using LaTeX

Posted on

Probability for machine learning and data science – Basic Probability 2 of 6

Probability Axioms

In this post we will look into probability axioms there are 3 basic axioms of probability and it is mentioned below

Axiom 1 : For any set A

P(A) \geq 0

Axiom 2 :

P (\Omega) = 1

Axiom 3 : if A_1, A_2, ... is any set of disjoint events, then

P\left(\bigcup\limits_{i=1}^{\infty}A_i\right) = \sum\limits_{i=1}^{\infty}P(A_i)

Definition

if A = \bigcup\limits_{i=1}^{\infty} A_i,. and A_1, A_2, .... are disjoint, then A_1, A_2, .... is said to be a partition of A

Axiom 3 also holds for finite collection of events A_1,..., A_n which is trivially true if you set A_{n+1} = \emptyset for all i \in \N

By using the above axioms we can get more axioms. Below are the results from axioms

Compliments

P(A^c) = 1-P(A)

Differences

if A is contained in B ( A \subset B), then

P(B \cap A^c) = P(B) - P(A)

Inclusion – Exclusion

P(A \cup B) = P(A) + P(B) - P(A \cap B)

Equally likely outcomes

In some cases, we can safely assume the outcomes are equally likely like if we roll a fair dice or toss a fair coin twice or more.

Suppose

  • n_A is the number of sample points in event A
  • N is the number of sample points in a finite sample space \Omega

if all outcomes are equally likely in a sample space \Omega, then the probability that event A occurs is

P(A)=\frac{n_A}{N},
  • n_A is the number of sample points in A
  • N is the number of sample points in \Omega

So in this post we have seen the Axioms of probability and in the next post we will start with Counting

Happy Learning

This post was made possible by LaTeX

Posted on

Probability for Machine learning and Data Science – Basic Probability Part 1 of 6

In this post, we will explore the basics of probability. As you have learned in school the set notations. we will explore all the symbols that are used to represent the sets. The contents of this post are

  1. Sample Space
  2. An Event
  3. Set Notations

Sample Space

A sample space is defined as the set of all possible outcomes of a random experiment and it is denoted by \Omega.

Examples of sample spaces

  1. The annual rate for rainfall in TamilNadu could take any non-negative value.
  2. The number of cars passing at a given point on the national highway in one hour(This will take any non-negative integer).
  3. The outcome of tossing 2 different coins.

Let us see the notation of each of the following

The annual rate for rainfall in TamilNadu could take any non-negative value.

\Omega = \{x|x \geq 0, x \in \R\}

The number of cars passing at a given point on the national highway in one hour(This will take any non negative integer).

\Omega = \{x|x = 0,1,2,3,...\}

The outcome of tossing 2 different coins

\Omega = \{HH, HT, TH, TT\}

Events

Events are denoted by A or B and it is a combination of outcomes and it is a subset of sample space \Omega

Examples of Events include

  1. Rainfall less than 700mm in a year
  2. Three cars passing a given point
  3. Obtaining exactly 2 Heads

If we see these in notations

Rainfall less than 700mm in a year

A = \{x|0 \leq x \leq 700\}

Three cars passing a given point

B = \{3\}

Obtaining exactly 2 Heads

C = \{HH\}

Set Notations

Below are the commonly used set notations

Universial set : \Omega

Empty Set : \emptyset

Subset : A \subset B

Union : A \cup B

Intersection : A \cap B

Complement : A^c

Disjoint : A \cap B = \emptyset

We will see Probability Axioms in the next post

Happy Learning

This post is made possible by WordPress and LaTeX

Posted on

Probability for Machine learning and Data science

Since Data science and Machine learning rule the world we need to know what are the basic things that make these concepts works. For learning making the most out of machine learning and datascience it is crucial for us to learn probability and statistics inorder to get the deeper understanding of the concepts in machine learning and data science. In the following posts you are introduced and taken deeper into the probability concepts and how it closely related to statistics.

The prerequisite for learning this topic is

  1. Differential calculus \lim_{x\to\infty} f(x)
  2. Integral calculus \int_{-\infty}^{\infty} f(x)dx

This post is written in LaTeX and WordPress.

Let us see what are the topics covered

  1. Basic Probability
  2. Discrete Random Variables
  3. Continuous random variables

Basic Probability

The topics that are covered under basic probability are

  1. Set Notation eg. Universal Set : \Omega
  2. Probability Axioms eg . P(A) \geq 1
  3. Counting eg . n_1 \times n_2 \times n_3 \times n_4 \times ... \times n_k
  4. Conditional Probability P(A | B) = \frac{P(A \cap B)}{P(B)} Provided P(B) > 0
  5. Law of Total Probability P(A) = \sum_{i=1}^{n}P(A|B_i)P(B_i)
  6. Bayes rule P(B_j|A)=\frac{P(A\cap B_j)}{P(A)}=\frac{P(A|B_j)P(B_j)}{P(A)} for j=1,....,n

Discrete Random Variables

The topics that are covered under Discrete Random Variables are

  1. Probability Mass function \sum_{y_i \in \Omega_Y} f_Y(y_i) = 1
  2. Expected Values E[Y] = \sum_{y_i \in \Omega_y} y_i f(y_i)
  3. Variance Var(Y) = E[Y^2] - (E[Y])^2
  4. Standard Deviation \sqrt{Var(Y)}
  5. Geometric Distribution f(y) = (1-p)^{y-1} p, y=1,2,...
  6. Binomial Distribution f(y) = \binom ny p^y(1-p)^{n-y}, y=0,1,2,....,n
  7. Poisson Distribution \frac{e^{-\lambda}\lambda^y}{y!}

Continuous Random Variables

The topics that are covered under Continuous Random Variables are

  1. Probability Density function F(y)=\int_{-\infty}^{y} f(t)dt
  2. Expected Values E[Y] = \int_{-\infty}^{\infty} y f(y) dy
  3. Variance Var(Y) = E[(Y - \mu)^2]
  4. Uniform Distribution f(n) = \begin{cases} \frac{1}{b-a} , & \quad a \leq y \leq b, \\ 0, & \quad elsewhere. \end{cases}
  5. Normal Distribution f(y) = \frac{1}{\sigma \sqrt {2\pi}}e^{-\frac{(y-\mu)^2}{2\sigma^2}}, -\infty < y < \infty
  6. Exponential Distribution f(y) = \begin{cases}\lambda e^{-\lambda y} , & \quad 0 \leq y \leq \infty, \lambda > 0 \\ 0, & \quad otherwise \end{cases}

We will see all these topics in detail in the next posts.

Happy Learning

Thank You