Suppose that students arrive at a lecturer's office according to a Poisson
process with rate ¡, and each conversation between a lecturer and a student is exponentially distributed with rate t. After each conversation is finished, there is a probability p that the student's problem is solved and he or she consequently leaves. With probability 1-p, however, the student suddenly remembers another question, and re-joins the queue immediately to talk to the lecturer again.
(a) Write down an appropriate state space S for a continuous-time Markov chain model of the number of students in the queue, and the transition matrix Q for the continuous-time
Markov chain.
(b) Let fj be the probability that the queue ever empties, given that it starts with j students. Write down a set of equations and boundary conditions satisfied by the fj .
(c) Give an explanation for your boundary conditions and explain what other conditions we need to distinguish which solution to these equations gives us the fj?
(d) Solve these equations and give the criterion for this continuous-time Markov chain to be recurrent.