OneStopGate.Com
OnestopGate   OnestopGate
   Thursday, March 28, 2024 Login  
OnestopGate
Home | Overview | Syllabus | Tutorials | FAQs | Downloads | Recommended Websites | Advertise | Payments | Contact Us | Forum
OneStopGate

GATE Resources
Gate Articles
Gate Books
Gate Colleges 
Gate Downloads 
Gate Faqs
Gate Jobs
Gate News 
Gate Sample Papers
Training Institutes

GATE Overview
Overview
GATE Eligibility
Structure Of GATE
GATE Coaching Centers
Colleges Providing M.Tech/M.E.
GATE Score
GATE Results
PG with Scholarships
Article On GATE
Admission Process For M.Tech/ MCP-PhD
GATE Topper 2012-13
GATE Forum




GATE 2025 Exclusive
Organizing Institute
Important Dates
How to Apply
Discipline Codes
GATE 2025 Exam Structure

GATE 2025 Syllabus
Aerospace Engg..
Agricultural Engg..
Architecture and Planning
Chemical Engg..
Chemistry
Civil Engg..
Computer Science / IT
Electronics & Communication Engg..
Electrical Engg..
Engineering Sciences
Geology and Geophysics
Instrumentation Engineering
Life Sciences
Mathematics
Mechanical Engg..
Metallurgical Engg..
Mining Engg..
Physics
Production & Industrial Engg..
Pharmaceutical Sciences
Textile Engineering and Fibre Science

GATE Study Material
Aerospace Engg..
Agricultural Engg..
Chemical Engg..
Chemistry
Civil Engg..
Computer Science / IT
Electronics & Communication Engg..
Electrical Engg..
Engineering Sciences
Instrumentation Engg..
Life Sciences
Mathematics
Mechanical Engg..
Physics
Pharmaceutical Sciences
Textile Engineering  and Fibre Science

GATE Preparation
GATE Pattern
GATE Tips N Tricks
Compare Evaluation
Sample Papers 
Gate Downloads 
Experts View

CEED 2013
CEED Exams
Eligibility
Application Forms
Important Dates
Contact Address
Examination Centres
CEED Sample Papers

Discuss GATE
GATE Forum
Exam Cities
Contact Details
Bank Details

Miscellaneous
Advertisment
Contact Us


Home » GATE Study Material » Mathematics » Linear Algebra » Markov Chains

Markov Chains

Looking for GATE Preparation Material? Join & Get here now!

** Gate 2013 Question Papers.. ** CEED 2013 Results.. ** Gate 2013 Question Papers With Solutions.. ** GATE 2013 CUT-OFFs.. ** GATE 2013 Results.. **

Markov Chains

Markov Chains

In a previous page, we studied the movement between the city and suburbs. Indeed, if I are S are the initial population of the inner city and the suburban area, and if we assume that every year 40% of the inner city population moves to the suburbs, while 30% of the suburb population moves to the inner part of the city, then after one year the populations are given by

\begin{displaymath}\left(\begin{array}{c}
0.6 I + 0.3 S\\
0.4 I + 0.7 S\\
\end...
...ay}\right) \left(\begin{array}{c}
I\\
S\\
\end{array}\right).\end{displaymath}

The matrix

\begin{displaymath}P = \left(\begin{array}{cc}
0.6&0.3\\
0.4&0.7\\
\end{array}\right) \end{displaymath}

is very special. Indeed, the entries of each column vectors are positive and their sum is 1. Such vectors are called probability vectors. A matrix for which all the column vectors are probability vectors is called transition or stochastic matrix. Andrei Markov, a russian mathematician, was the first one to study these matrices. At the beginning of this century he developed the fundamentals of the Markov Chain theory.
A Markov chain is a process that consists of a finite number of states and some known probabilities pij, where pij is the probability of moving from state j to state i. In the example above, we have two states: living in the city and living in the suburbs. The number pij represents the probability of moving from state i to state j in one year. We may have more than two states. For example, political affiliation: Democrat, Republican, and Independent. For example, pij represents the probability of a son belonging to party i if his father belonged to party j.
Of particular interest is a probability vector p such that $A \mbox{\bf p} = \mbox{\bf p}$, that is, an eigenvector of A associated to the eigenvalue 1. Such vector is called a steady state vector. In the example above, the steady state vectors are given by the system

\begin{displaymath}\left(\begin{array}{cc}
0.6&0.3\\
0.4&0.7\\
\end{array}\rig...
...}
-0.4&0.3\\
0.4&-0.3\\
\end{array}\right)\cdot X = {\cal O}.\end{displaymath}

This system reduces to the equation -0.4 x + 0.3 y = 0. It is easy to see that, if we set $x = 0.3 \alpha$, then

\begin{displaymath}X = \left(\begin{array}{c}
x\\
y\\
\end{array}\right) = \alpha \left(\begin{array}{c}
0.3\\
0.4\\
\end{array}\right).\end{displaymath}

So the vector $\mbox{\bf p}_1 = \displaystyle \left(\begin{array}{c}
0.3\\
0.4\\
\end{array}\right)$ is a steady state vector of the matrix above. So if the populations of the city and the suburbs are given by the vector $\mbox{\bf p}_1$, after one year the proportions remain the same (though the people may move between the city and the suburbs).

Let us discuss another example on population dynamics.

Example: Age Distribution of Trees in a Forest
Trees in a forest are assumed in this simple model to fall into four age groups: b(k) denotes the number of baby trees in the forest (age group 0-15 years) at a given time period k; similarly y(k),m(k) and o(k) denote the number of young trees (16-30 years of age), middle-aged trees (age 31-45), and old trees (older than 45 years of age), respectively. The length of one time period is 15 years.
How does the age distribution change from one time period to the next? The model makes the following three assumptions:

  • A certain percentage of trees in each age group dies.
  • Surviving trees enter into the next age group; old trees remain old.
  • Lost trees are replaced by baby trees.

Note that the total tree population does not change over time.

We obtain the following difference equations:

b(k+1) = $\displaystyle d_b\cdot b(k)+d_y \cdot y(k) +d_m\cdot m(k) + d_o\cdot o(k)$ (1)
y(k+1) = (1-db) b(k) (2)
m(k+1) = (1-dy) y(k) (3)
o(k+1) = (1-dm) m(k) + (1-do) o(k) (4)

Here 0 < db,dy,dm,do <1 denote the loss rates in each age group in percent.

Let

\begin{displaymath}{\bf x}(k)=\left(\begin{array}{c}b(k)\\ y(k)\\ m(k)\\ o(k)\end{array}\right)\end{displaymath}

be the ``age distribution vector". Consider the matrix

\begin{displaymath}A = \left(\begin{array}{cccc}
d_b&d_y&d_m&d_o\\
1-d_b&0&0&0\\
0&1-d_y&0&0\\
0&0&1-d_m&1-d_o\\
\end{array}\right).\end{displaymath}

Then we have

\begin{displaymath}{\bf x}(k+1)=A\cdot {\bf x}(k).\end{displaymath}

Note that the matrix A is a stochastic matrix!
If, db=.1,dy=.2,dm=.3 and do=.4, then

\begin{displaymath}A = \left(\begin{array}{cccc}
0.1&0.2&0.3&0.4\\
0.9&0&0&0\\
0&0.8&0&0\\
0&0&0.7&0.6\\
\end{array}\right).\end{displaymath}

After easy calculations, we find the steady state vector for the age distribution in the forest:

\begin{displaymath}\mbox{\bf p} = \left(\begin{array}{cccc}
1/3.88\\
0.9/3.88\\...
...n{array}{cccc}
1\\
0.9\\
0.72\\
1.26\\
\end{array}\right) .\end{displaymath}

Assume a total tree population of 50,000 trees. Suppose the forest is newly planted, i.e.

\begin{displaymath}{\bf x}(0)=\left(\begin{array}{c}50,000\\ 0\\ 0\\ 0\end{array}\right)\end{displaymath}

After 15 years, the age distribution in the forest is given by

\begin{displaymath}{\bf x}(1) = \left(\begin{array}{cccc}
0.1&0.2&0.3&0.4\\
0.9...
...\begin{array}{cccc}
0.1\\
0.9\\
0\\
0\\
\end{array}\right).\end{displaymath}

After 30 years, we have

\begin{displaymath}{\bf x}(2) = \left(\begin{array}{cccc}
0.1&0.2&0.3&0.4\\
0.9...
...in{array}{cccc}
0.19\\
0.09\\
0.72\\
0\\
\end{array}\right)\end{displaymath}

and after 45 years

\begin{displaymath}{\bf x}(3) = 50,000\left(\begin{array}{cccc}
0.19\\
0.09\\
0.72\\
0\\
\end{array}\right)\end{displaymath}

After 15n years, where $n=1,2,\cdots$, the age distribution in the forest is given by

\begin{displaymath}{\bf x}(n) = \left(\begin{array}{cccc}
0.1&0.2&0.3&0.4\\
0.9...
...eft(\begin{array}{cccc}
1\\
0\\
0\\
0\\
\end{array}\right).\end{displaymath}

So the problem is to find the nth-power of the matrix A. We have seen that diagonalization technique may be helpful to solve this problem. Another problem deals with the long term behavior of the sequence x(n) when n gets large.

The calculations on the example above becomes tedious. Let us illustrate the problem on a small matrix.

Example. Consider the stochastic matrix

\begin{displaymath}A = \left(\begin{array}{cccc}
0.8&0.2\\
0.2&0.8\\
\end{array}\right).\end{displaymath}

Note this is a symmetric matrix. The characteristic polynomial of A is

\begin{displaymath}p(\lambda) = (0.8 - \lambda)^2-0.2^2 = (1-\lambda)(0.6 - \lambda)\end{displaymath}

An eigenvector associated to 1 is

\begin{displaymath}\left(\begin{array}{cccc}
1\\
1\\
\end{array}\right)\end{displaymath}

and an eigenvector associated to 0.6 is

\begin{displaymath}\left(\begin{array}{cccc}
1\\
-1\\
\end{array}\right).\end{displaymath}

If we set

\begin{displaymath}P = \left(\begin{array}{rr}
1&1\\
1&-1\\
\end{array}\right),\end{displaymath}

then we have

\begin{displaymath}P^{-1}AP = D = \left(\begin{array}{cc}
1&0\\
0&0.6\\
\end{array}\right).\end{displaymath}

So, we have

\begin{displaymath}A^n = P D^nP^{-1} = P\left(\begin{array}{cc}
1&0\\
0&(0.6)^n\\
\end{array}\right)P^{-1}.\end{displaymath}

When n gets large, the matrices An get closer to the matrix

\begin{displaymath}P\left(\begin{array}{cc}
1&0\\
0&0\\
\end{array}\right)P^{-1}.\end{displaymath}

So the sequence of vectors defined by

\begin{displaymath}X(n+1) = A X(n),\;\;\mbox{given $X(0)$}\end{displaymath}

will get closer to

\begin{displaymath}X(\infty) = P\left(\begin{array}{cc}
1&0\\
0&0\\
\end{array}\right)P^{-1}X(0)\end{displaymath}

when n gets large. If $X(0) = \displaystyle \left(\begin{array}{cc}
a\\
b\\
\end{array}\right)$, then we have

\begin{displaymath}X(\infty) = \left(\begin{array}{rr}
1&1\\
1&-1\\
\end{array...
...ac{a+b}{2}\left(\begin{array}{cc}
1\\
1\\
\end{array}\right).\end{displaymath}

Note that the vector $X(\infty)$ is proportional to the unique steady state vector of A

\begin{displaymath}{\bf p} = \frac{1}{2}\left(\begin{array}{cc}
1\\
1\\
\end{array}\right).\end{displaymath}

This is not surprising. In fact there is a general result similar to the one above for any stochastic matrix.



Discussion Center

Discuss/
Query

Papers/
Syllabus

Feedback/
Suggestion

Yahoo
Groups

Sirfdosti
Groups

Contact
Us

MEMBERS LOGIN
  
Email ID:
Password:

  Forgot Password?
 New User? Register!

INTERVIEW EBOOK
Get 9,000+ Interview Questions & Answers in an eBook. Interview Question & Answer Guide
  • 9,000+ Interview Questions
  • All Questions Answered
  • 5 FREE Bonuses
  • Free Upgrades
GATE RESOURCES
 
  • Gate Books
  • Training Institutes
  • Gate FAQs
  • GATE BOOKS
     
  • Mechanical Engineeering Books
  • Robotics Automations Engineering Books
  • Civil Engineering Books
  • Chemical Engineering Books
  • Environmental Engineering Books
  • Electrical Engineering Books
  • Electronics Engineering Books
  • Information Technology Books
  • Software Engineering Books
  • GATE Preparation Books
  • Exciting Offers



    GATE Exam, Gate 2009, Gate Papers, Gate Preparation & Related Pages


    GATE Overview | GATE Eligibility | Structure Of GATE | GATE Training Institutes | Colleges Providing M.Tech/M.E. | GATE Score | GATE Results | PG with Scholarships | Article On GATE | GATE Forum | GATE 2009 Exclusive | GATE 2009 Syllabus | GATE Organizing Institute | Important Dates for GATE Exam | How to Apply for GATE | Discipline / Branch Codes | GATE Syllabus for Aerospace Engineering | GATE Syllabus for Agricultural Engineering | GATE Syllabus for Architecture and Planning | GATE Syllabus for Chemical Engineering | GATE Syllabus for Chemistry | GATE Syllabus for Civil Engineering | GATE Syllabus for Computer Science / IT | GATE Syllabus for Electronics and Communication Engineering | GATE Syllabus for Engineering Sciences | GATE Syllabus for Geology and Geophysics | GATE Syllabus for Instrumentation Engineering | GATE Syllabus for Life Sciences | GATE Syllabus for Mathematics | GATE Syllabus for Mechanical Engineering | GATE Syllabus for Metallurgical Engineering | GATE Syllabus for Mining Engineering | GATE Syllabus for Physics | GATE Syllabus for Production and Industrial Engineering | GATE Syllabus for Pharmaceutical Sciences | GATE Syllabus for Textile Engineering and Fibre Science | GATE Preparation | GATE Pattern | GATE Tips & Tricks | GATE Compare Evaluation | GATE Sample Papers | GATE Downloads | Experts View on GATE | CEED 2009 | CEED 2009 Exam | Eligibility for CEED Exam | Application forms of CEED Exam | Important Dates of CEED Exam | Contact Address for CEED Exam | CEED Examination Centres | CEED Sample Papers | Discuss GATE | GATE Forum of OneStopGATE.com | GATE Exam Cities | Contact Details for GATE | Bank Details for GATE | GATE Miscellaneous Info | GATE FAQs | Advertisement on GATE | Contact Us on OneStopGATE |
    Copyright © 2024. One Stop Gate.com. All rights reserved Testimonials |Link To Us |Sitemap |Privacy Policy | Terms and Conditions|About Us
    Our Portals : Academic Tutorials | Best eBooksworld | Beyond Stats | City Details | Interview Questions | India Job Forum | Excellent Mobiles | Free Bangalore | Give Me The Code | Gog Logo | Free Classifieds | Jobs Assist | Interview Questions | One Stop FAQs | One Stop GATE | One Stop GRE | One Stop IAS | One Stop MBA | One Stop SAP | One Stop Testing | Web Hosting | Quick Site Kit | Sirf Dosti | Source Codes World | Tasty Food | Tech Archive | Software Testing Interview Questions | Free Online Exams | The Galz | Top Masala | Vyom | Vyom eBooks | Vyom International | Vyom Links | Vyoms | Vyom World
    C Interview Questions | C++ Interview Questions | Send Free SMS | Placement Papers | SMS Jokes | Cool Forwards | Romantic Shayari