Consider an inventory system in which the sequence of events during each period is as follows.
(1) We observe the inventory level (call it i) at the beginning of the period.
(2) If i 2, then 0 units are ordered. Delivery of all ordered units is immediate.
(3) With probability 1/3, 0 units are demanded during the period; with probability 1/3, 1 unit is demanded during the period; and with probability 1/3, 2 units are demanded during the period. (4) We observe the inventory level at the beginning of the next period.
Define a period's state to be the period's beginning inventory level. Determine the transition matrix that could be used to model this inventory system as a Markov chain.