Consider the Markov process illustrated below. The transitions are labeled by the rate qij at which those transitions occur. The process can be viewed as a single-server queue where arrivals become increasingly discouraged as the queue lengthens. The word time average below refers to the limiting time average over each sample path of the process, except for a set of sample paths of probability 0.
(a) Find the time-average fraction of time pi spent in each state i > 0 in terms of p0 and then solve for p0. Hint: First find an equation relating pi to pi+1 for each i. It also may help to recall the power series expansion of ex.
(b) Find a closed form solution to J,i piνi, where νi is the rate at which transitions out of state i occur. Show that the embedded chain is positive recurrent for all choices of λ > 0 and μ > 0 and explain intuitively why this must be so.
(c) For the embedded Markov chain corresponding to this process, find the steady- state probabilities πi for each i≥ 0 and the transition probabilities Pij for each i, j.
(d) For each i, find both the time-average interval and the time-average number of overall state transitions between successive visits to i.
Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.