Gibbs Sampling is used when the dimensions of the distribution you are trying to sample from are more than 2, AND when it is difficult to sample from that joint distribution.
For example, we want to sample from a joint distribution P(x,y). Both x and y are normally distributed and have a correlation like the image above. When it is hard to sample from the joint distribution, sample from the conditional distribution P(x|y)(A normal distribution of x when y is a fixed value) and P(y|x)(A normal distribution of y when x is a fixed value) instead.
To implement, you first get a sample from the joint distribution (Xo,Yo), then sample X1 from the conditional distribution P(x|Yo). Then sample Y1 from the conditional distribution P(y|X1). Repeat until the desired amount of samples.
Reference: Deep Learning By Ian Goodfellow, Yoshua Bengio and Aaron Courville
