In this paper, we develop an effective approach to simplify two-time-scale Markov chains with infinite state spaces by removal of states with fast leaving rates, which improves the simplification method of finite Markov chains. We introduce the concept of fast transition paths and show that the effective transitions of the reduced chain are the superposition of the direct transitions and the indirect transitions via all the fast transition paths. Furthermore, we apply our simplification approach to the standard Markov model of single-cell stochastic gene expression and provide a mathematical theory of random gene expression bursts. We also give the precise mathematical conditions for mRNAs and proteins to yield random bursts. It turns out the the random bursts exactly correspond to the fast transition paths of the Markov model. This helps us gain a better understanding of the physics behind random bursts as an emergent behavior from the complex biochemical reaction kinetics.