Transition matrices (higher powers), Markov chains, steady state and long term probabilities: notes and practice questions

A market research company studies the customer loyalty between two competing streaming services, StreamSphere (S) and CineFlow (C), over a number of months.

Each month, customers either maintain their current subscription or switch to the other service.

In any given month, it is observed that the probability of a StreamSphere customer remaining with StreamSphere the following month is 0.85, and the probability of a CineFlow customer switching to StreamSphere the following month is 0.20.

This situation can be represented by the transition matrix

$\mathbf{T} = \begin{pmatrix} 0.85 & 0.20 \\ 0.15 & 0.80 \end{pmatrix}$

Interpret the value 0.15 in $\mathbf{T}$ in terms of the changes in customer loyalty between the streaming services.

Find the eigenvalues of matrix $\mathbf{T}$.

One of the eigenvectors of $\mathbf{T}$ is $\begin{pmatrix} 1 \\ -1 \end{pmatrix}$.

Find another, non-parallel, eigenvector and interpret it in context.

A new streaming service, CineStream (C), enters the market, competing with the established service, FilmFlick (F).

Market research shows that each month:

• 10% of CineStream subscribers switch to FilmFlick.

• 30% of FilmFlick subscribers switch to CineStream.

The transition matrix $T$ describing these changes, where the rows and columns are ordered (CineStream, FilmFlick), is given as $T = \begin{pmatrix} 0.90 & 0.30 \\ 0.10 & 0.70 \end{pmatrix}$.

The two eigenvalues for this matrix are $1$ and $0.60$. An eigenvector corresponding to the eigenvalue of $1$ is $\begin{pmatrix} 3 \\ 1 \end{pmatrix}$.

Find an eigenvector corresponding to the eigenvalue of $0.60$.

A diagonal matrix of eigenvalues is $D = \begin{pmatrix} 0.60 & 0 \\ 0 & 1 \end{pmatrix}$.

Write down an expression for $D^n$, giving your answer as a $2 \times 2$ matrix in terms of $n$.

When CineStream and FilmFlick first launched, there were a total of $8000$ users, all of whom initially subscribed to CineStream.

Assuming the total number of users remains constant, find an expression for the number of users who will favour FilmFlick after $n$ months.

(a) A study models student engagement with two online learning platforms, LearnFlow (L) and EduQuest (E), using a Markov chain. The transition matrix $P$ describes the probability of a student moving between platforms each week:

$P = \begin{pmatrix} a & 1-b \\ 1-a & b \end{pmatrix}$

where $a$ is the probability a student using LearnFlow stays on LearnFlow, and $b$ is the probability a student using EduQuest stays on EduQuest.

One of the eigenvalues for this matrix is equal to 1.

Find the eigenvector corresponding to the eigenvalue of 1.

(b) Hence, find an expression, in terms of $a$ and $b$, for the long-term proportion of students using LearnFlow.

(c) Hence or otherwise, find the long-term proportion of students using LearnFlow when $a = 0.7$ and $b = 0.4$.

(a) A student's study habits are modelled by a Markov chain. If the student studies Mathematics (M) on a given day, the probability they study Mathematics the following day is $0.75$. If the student studies Physics (P) on a given day, the probability they study Physics the following day is $0.85$.

Write down a transition matrix, $T$, that shows the movement of the student's study focus between Mathematics and Physics.

(b) On Monday, a student spent their study time on Mathematics. Find the probability that the student will be studying Physics on Friday.

(c) Write down the characteristic polynomial for the matrix $T$. Give your answer in the form $\lambda^2 + b\lambda + c = 0$.

(d) Calculate the eigenvectors for the matrix $T$.

(e) Write down matrices $P$ and $D$ such that $T= PDP^{-1}$, where $D$ is a diagonal matrix.

(f) Hence, find the long-term probability that the student is studying Physics.

A financial analyst models the daily price movement of a tech company's stock using a transition matrix. If the stock price increases on a particular day, the probability that it will increase the following day is $0.7$. If the stock price decreases on a particular day, the probability that it will increase the following day is $0.4$.

The transition matrix $T$ for this model is given by $T = \begin{pmatrix} 0.7 & 0.4 \\ 0.3 & 0.6 \end{pmatrix}$, where the rows/columns represent 'Increase' and 'Decrease' respectively.

(a) Given that the stock price increased today, calculate the probability that the stock price will increase in two days' time.

(b) Find the eigenvalues and corresponding eigenvectors of $T$.

(c) The matrix $T$ can be written in the form $PDP^{-1}$, where $D$ is a diagonal matrix.

(i) Write down a possible matrix $P$.

(ii) Write down the corresponding matrix $D$.

(d) Hence, determine the long-term percentage of days that the stock price will increase.

(a) A single data packet can be transferred between three servers, S1, S2, and S3, in a network. The possible direct transfers are:

• From S1, the packet can be sent to S2 or stay in S1.

• From S2, the packet can be sent to S1 or S3.

• From S3, the packet can be sent to S2 or stay in S3.

Write down the adjacency matrix for the directed graph representing these possible direct data transfers, ordering the servers S1, S2, S3.

(b) Find the total number of distinct data paths of length 5 from server S1 to server S2.

(c.i) Every possible sequence of 5 data transfers has the same probability of occurring. State this probability.

(c.ii) Use your answer to part (b) to find the probability that if a data packet was initially on server S1, it will be on server S2 after 5 transfers.

(d.i) A network administrator monitors the movement of two data packets. The possible combined states of the two packets (assuming they occupy distinct servers) are $S_{12}$ (packets on S1 and S2), $S_{13}$ (packets on S1 and S3), and $S_{23}$ (packets on S2 and S3).

The transitions between these states are modelled by the following transition matrix $T$, where rows represent the current state and columns represent the next state, in the order $S_{12}, S_{13}, S_{23}$:

$T = \begin{pmatrix} 0 & 0.25 & 0.4 \\ 0.6 & 0.5 & 0.6 \\ 0.4 & 0.25 & 0 \end{pmatrix}$

State the probability that if the packets are currently in state $S_{12}$, they will be in state $S_{13}$ after one transfer.

(d.ii) Using the transition matrix $T$ from part (d.i), state the probability that if the packets are currently in state $S_{13}$, they will be in state $S_{23}$ after one transfer.

(d.iii) Using the transition matrix $T$ from part (d.i), state the probability that if the packets are currently in state $S_{23}$, they will be in state $S_{12}$ after one transfer.

(e) Given that the two data packets are initially in state $S_{12}$ (on servers S1 and S2), find the probability that they will be in state $S_{23}$ (on servers S2 and S3) after 5 transfers.

(f) The data packets continue this pattern of transfers for a long period. Find the server that is occupied least and the proportion of the time it is free.

A market research firm models customer loyalty between two competing coffee shops, "The Daily Grind" (D) and "Bean There, Done That" (B), using a Markov chain. Each day, a customer may switch between the two shops.

The model is of the form

$\begin{pmatrix} D_{n+1} \\ B_{n+1} \end{pmatrix} = M \begin{pmatrix} D_n \\ B_n \end{pmatrix}$

where $D_n$ is the probability a customer chooses "The Daily Grind" on day $n$, and $B_n$ is the probability a customer chooses "Bean There, Done That" on day $n$, where $n \in \mathbb{N}$.

The transition matrix $M$ is found to be $\begin{pmatrix} 0.85 & k \\ 0.15 & 0.90 \end{pmatrix}$.

Write down the value of $k$.

State what $k$ represents in this context.

Find the eigenvalues of $M$.

Find the eigenvectors of $M$.

A new customer initially chooses 'The Daily Grind'. Calculate the probability that this customer chooses 'The Daily Grind' after 3 days.

Calculate the probability that this customer chooses 'The Daily Grind' in the long term.

On any given day, the probability that a customer buys a particular product depends only on which product they bought the previous day.

If a customer bought Product A on the previous day, the probability they buy Product A again today is $0.7$.

If a customer bought Product B on the previous day, the probability they buy Product A today is $q$.

On day $n$ this can be represented using the vector $v_n$ where

$v_n = \begin{pmatrix} \text{probability that a customer buys Product A on day } n \\ \text{probability that a customer buys Product B on day } n \end{pmatrix}$

A Markov chain model is formed where

$v_{n+1} = Mv_n$

Matrix $M$ is of the form $\begin{pmatrix} a & q \\ b & 1-q \end{pmatrix}$

Write down the value of

(i) $a$.

Write down the value of

(ii) $b$.

On day zero, a customer buys Product A. Find the probability

(i) that the customer buys Product A for all days from $n = 1$ to $n = 4$.

On day zero, a customer buys Product A. Find the probability

(ii) that the customer buys Product A on day 4, when $q = 0.2$.

Demonstrate that, for all values of $q$, one eigenvector of $M$ is $\begin{pmatrix} 1 \\ -1 \end{pmatrix}$ and hence state the associated eigenvalue.

Find, in terms of $q$, the steady state probability that a customer buys Product A on a given day.

In the long term, the company wants Product A to have at least 70% market share.

Find the minimum value of $q$ required for this to occur.

A large university offers three primary study modes for its students: Online (O), Hybrid (H), and In-person (I). Each academic year, students may choose to switch between these modes based on their preferences and circumstances.

A study tracked student movement between these modes over a single academic year, revealing the following transition probabilities:

• Of students initially studying Online, 85% remained Online, 10% switched to Hybrid, and 5% switched to In-person.

• Of students initially studying Hybrid, 8% switched to Online, 80% remained Hybrid, and 12% switched to In-person.

• Of students initially studying In-person, 3% switched to Online, 7% switched to Hybrid, and 90% remained In-person.

Assume that these transition probabilities remain constant each year and that the total number of students in the university remains constant.

Represent this information in a transition matrix $T$, ordering the states as Online, Hybrid, and In-person.

At the start of the study, the university had 5000 students studying Online, 8000 students studying Hybrid, and 7000 students studying In-person.

By using $T$, find the expected number of students studying in the Hybrid mode 5 academic years after the start of the study.

For matrix $T$ there exists a steady state vector

$u = \begin{pmatrix} u_1 \\ u_2 \\ u_3 \end{pmatrix}$,

where $u_1, u_2$ and $u_3$ are the proportions of the total student population in the Online, Hybrid, and In-person modes respectively, in the long term.

The steady state vector $u$ may be found by solving a system of equations.

Determine these equations that are to be solved.

By solving your system of equations, find $u$.

Use your answer to part (c)(ii) to determine the long-term expected population of the In-person mode.

Suggest two reasons why your answer to part (d) is not likely to be accurate. You may comment on both the model and the situation in context.

A competitive market has two dominant smartphone brands, Aura and Zenith. Each year, it is observed that $15\%$ of Aura customers switch to Zenith, while $5\%$ of Zenith customers switch to Aura. All other customer movements are negligible.

Write down a transition matrix $T$ representing the customer movements between the two brands in a particular year. Assume the order of brands is Aura, then Zenith.

Find the eigenvalues and corresponding eigenvectors of $T$.

Hence write down matrices $P$ and $D$ such that $T = PDP^{-1}$.

Initially, Brand Aura has $100\,000$ customers and Brand Zenith has $200\,000$ customers.

Find an expression for the number of customers Brand Aura has after $n$ years, where $n \in \mathbb{N}$.

Hence write down the number of customers that Brand Aura can expect to have in the long term.

A streaming service offers two subscription plans: Premium and Basic. The service has found that customers' choices follow a pattern. If a customer has the Premium plan in a given month, the probability they will keep the Premium plan the next month is 0.8. If a customer has the Basic plan in a given month, the probability they will upgrade to the Premium plan the next month is 0.1.

(a) Let the states be Premium (State 1) and Basic (State 2). Write down the transition matrix, T, for this Markov chain.

(b) A new customer subscribes to the Premium plan in their first month, January. Find the probability that this customer will have the Premium plan in March of the same year.

(c) Find the steady state probability vector for this Markov chain.

A marketing analyst models the monthly customer loyalty for two companies, Innovate Inc. and Legacy Corp., using the transition matrix $T = \begin{pmatrix} p & 1-q \\ 1-p & q \end{pmatrix}$, where $0 < p < 1$ and $0 < q < 1$. `p` is the probability a customer of Innovate Inc. remains with them the next month, and `q` is the probability a customer of Legacy Corp. remains with them the next month.

(a) Find the eigenvalues of the matrix T in terms of `p` and `q`.

(b) In the long term, the market share for the two companies reaches a steady state. Find the proportion of customers that are with Innovate Inc. and Legacy Corp. at this steady state, in terms of `p` and `q`.

Two coffee shops, Bean Buzz and Daily Grind, compete for customers in a small town. A market analysis shows that customer loyalty changes from month to month.

A customer who went to Bean Buzz in one month has a 60% probability of switching to Daily Grind the next month. A customer who went to Daily Grind in one month has a 30% probability of switching to Bean Buzz the next month.

This situation can be modelled by the transition matrix $T=\begin{pmatrix} 0.4 & 0.3 \\ 0.6 & 0.7 \end{pmatrix}$, where the first column represents the initial state of being a Bean Buzz customer and the second column represents the initial state of being a Daily Grind customer. The eigenvalues of $T$ are $1$ and $0.1$.

(a) Find an eigenvector corresponding to the eigenvalue of $1$. Give your answer in the form $\begin{pmatrix} x \\ y \end{pmatrix}$, where $x, y \in \mathbb{Z}^+$.

(b) Using your answer to part (a), or otherwise, find the long-term market share for Bean Buzz. Give your answer as a percentage to one decimal place.