Empirical Rule Vs Chebyshev’s Inequality
The Empirical rule and Chebyshev's inequality are the concepts used to find the proportion of the observations/values with some standard deviation away from the mean. You might ask? — If these topics are explaining the same concepts then why are we even comparing them?
These topics might be looking the same to you as of now but there is a big difference between them. In this blog post, we will be learning both of these important topics so at the end of this blog post you will understand the difference between them clearly. Now let’s start with understanding the Empirical Rule.
Empirical Rule
This concept is very related to the normal distribution. If you haven’t understood that concept, I would want you to read that first before you continue with this article.
Normal Distribution:- https://dugamakash.medium.com/probability-distributions-101-618511cb8f4
The empirical rule is also known as the ‘3-Sigma Rule’ is the rule in statistics which states that for a normal distribution, almost all observed values fall within the 3 standard deviations (denoted by σ) away from the mean value.
Let’s look at the table below to understand the definition more clearly.
Below observation can be concluded from above:-
- 68 % of data fall within 1-sigma or 1 standard deviation i.e. μ -σ of the mean
- 95% of data fall within 2-sigma or 2 standard deviations i.e. μ -2σ away from the mean.
- 99.7% of data fall within 3-sigma or 3 standard deviations i.e. μ -3σ away from the mean.
- Nearly all values lie within the 3 standard deviations of the mean.
- Data that falls away from 3 standard deviations considered as an outlier.
Thus due to % of data falls in each standard deviation this rule is also referred to as the ‘ 68–95–99.7 rule’ in the statistics.
If random variable X follows the gaussian distribution or normal distribution then it is denoted as- X ~ N(μ, σ)
If we have the information about the distribution of random variable as well as information about parameters μ, σ then we can use empirical rule to make conclusions about the observations.
Example — The mean height of the graduate college is 150 cm with a standard deviation of 10 cm. The population is normally distributed.
Since we know the natural phenomena like the height of the population takes the normal distribution. We can conclude the following,
Let X be the height of the students.
- P(μ -σ ≤ X ≤ μ + σ) = 68% … 68% of students height lies in the interval of [140 cm, 160 cm]
- P(μ -2σ ≤ X ≤ μ + 2σ) = 95% … 95% of students height lies in the interval of [130 cm, 170 cm]
- P(μ -3σ ≤ X ≤ μ + 3σ) = 99.7% … 99.7% of students height lies in the interval of [120 cm, 180 cm]
It works effectively but there is one problem with this rule. This empirical rule is only applicable when you are dealing with a normal distribution. But this is not the always case. We might not have normal distribution at all. Therefore in those cases, we can’t use the empirical rule. This is where Chebyshev’s Inequality theorem comes into the picture.
Chebyshev’s Inequality
We will define Chebyshev's Inequality in this section but before that let’s have a look at this example.
Let’s assume that you have got data of salaries of every individual person in the country. You’ve information about its mean (μ) which is 40000 Rs and the standard deviation (σ) is 10000 Rs. You’ve no information about the distribution of data.
With the help of this information, we need to answer the following questions.
A. What percentage of an individual has a salary in the range of [20000, 60000]? → 2σ
B. What percentage of an individual has a salary in the range of [10000, 70000]? → 3σ
Since we’ve no information about data distribution, can we answer questions like stated above? → Yes but not with the empirical rule as the rule 68–95–99.7 is only applicable when data is normally distributed. In this case, we do not have any information about the distribution and hence we will get these answers by applying Chebyshev’s Inequality theorem.
Chebyshev’s Inequality theorem defined as, given random variable X where we don’t know about the distribution or distribution is not normal and we have the information about the finite mean () and non-zero & finite standard deviation () then the Chebyshev’s Inequality theorem states that →
Thus above equation can be written as →
Let’s read the above equation in plain English →
Given a random variable X for which we know, the mean is finite and the standard deviation is non-zero. The probability that random variable X lies between ‘K-Standard Deviation’ / Kσ away from the mean is strictly greater than 1-1/K².
Let’s revisit our salary example and try to answer the asked question!
What percentage of an individual has a salary in the range of 2σ that is [20000, 60000]?
Greater than (or) At least 75% of individuals salary lies in the range of 2σ that is [20000, 60000]. Since we cannot have information about the distribution so the exact value cannot be determined. All we can say that is irrespective of any distribution, the % of an individual has a salary in the range of 2σ is greater than 75%. ( > 75%)
Now let’s cross-check this claim. Let’s consider the above distribution is gaussian distribution. Thus salary in the range of 2σ is 95% which is greater than 75%. Thus Chebyshev’s Inequality theorem is validated.
Summary →
Chebyshev’s theorem makes no assumptions about the distribution. It works with all types of data distribution whereas the Empirical theorem assumes data to be normally distributed.
Chebyshev’s inequality theorem provides a lower bound for a proportion of data inside an interval that is symmetric about the mean whereas the Empirical theorem provides the approximate amount of data within a given interval.
This is my attempt to put the difference between the two theorems. Let me know if you have difficulties in understanding any part of it.
Happy Learning :)
