The central limit theorem states that the distribution of independent samples approximates a normal distribution as the number of those samples increases. On the other hand, the Markov Chain states that even dependent samples will converge to a certain state as well.
There are 3 main features of Markov Chains.
A) State Space (For this example, it is sunny and cloudy)
B) Markov Assumption (All States depend ONLY on the previous state)
C) Transition Matrix (Matrix that describes how to transition between 1 state to the other)
The state where the dependent(dependent on the previous state) samples converge is called the STEADY STATE. This state means that the next iteration of the Markov chain would be the same as the previous like the equation below.
