# Lecture 35: Computation of combined process parameters: Markov Analysis

Hello, we are dealing with Computation of
combined process parameters. Today we will see the Markov Analysis .
The topic of today’s presentation is Computation of combined process parameters Markov Analysis.
We will see Markov Analysis with respect to a two state component. And then we will calculate
the unavailability, the unconditional, failure and repair intensity, expected number of failures
and repairs. Then we will go for three state Markov Analysis; primarily our material chosen
from this book taken from this book . So, let us now understand what is Markov Analysis
here, and how we are going to use it. So, Markovian process and exponential distribution
they are very much related . So, ah it is basically the method which deals with the
state transitions and then finally, the steady state ah probability values. And we will consider
this diagram what you have already seen . So, we are thinking that the component has 2 states,
one is normal another one is failed . So, if the component is working; that means,
in normal state it can remain in that state or a transition to the failed state may take
place . Now, the transition from normal state to failed state that will happen based on
certain probability values, that will be known as transition probability. Like probability
ah that the system will be or the component will be at failed state given that it was
at normal. So, we will basically create a small time interval delta t and then we say
suppose at t equal to t. If the component is at normal and then at
x t equal to t plus delta t the component will be at failed state then the probability
is probability 1 given 0, 1 for your failed state and 0 for working on normal state ok
. Now, if the component is at failed state; then it will be repaired . So, there will
is also a transition probability that the failed component will ultimately repaired
to as good as new and that probability is probability 0 given 1.
Now then we will assign ah some the parameter values that is lambda and mu with those probabilities.
And and we identify all possible ah states and their probabilities. And then we will
try to formulate the problem as unavailability analysis problem. To find out the what how
to calculate the unavailability of the component ok . So, we essentially you know what is lambda
t, and also you know what is mu t . So, for is very small interval of time delta t lambda
t mu t is nothing, but the probability that the component falls during a small interval
t and delta t . So, that is why you see that if the component
is at working state and then at t equal to t at what is the probability that it will
it will fail at c equal to x t plus delta t. Then this probability is nothing, but lambda
t d t, so that is what is written here. At the same time suppose the component is at
failed state then you also know that this mu t and d t that is nothing, but the that
component will be repaired during a small interval of time delta t. And it was as good
as new at time t is equal to 0 and failed at time t .
So, then if the the the transition probability will be mu t d t ok. Now then what will be
the probability that the component is working at time t equal to 0 and will also be working
at time t plus delta t . So, that is basically probability 0 given 0 that mean; normal given
normal, normal at time t plus delta t given normal at time t . So, that probability is
nothing, but 1 minus lambda t d t. The reason you see when the component is at normal state
it can either remain at normal or it may go to the failed state so that is basically two
condition . So, that mean if the if the component transition
from normal to failed is lambda t d t and then remaining at normal will be 1 minus lambda
d t because the probability of the two will be 1 and that is also true for failed state
case . Now, the same thing this concept with with the use of indicator variable that mean
x t is 0 means that component is at normal that time t equal to t .
And x t plus delta t equal to 0 that mean component is normal at time x time t plus
delta t so, we can use indicator variable to denote the probabilities . So, then what
is lambda t d t lambda t d t is this component at time t it is not working at time t plus
delta t it failed . So, then using indicator variable you can write like this; the probability
that x t plus delta t equal to 1 given that x t equal to 0. In the same manner you can
write down this, this and this . So, essentially then what are the things we
have considered here you please understand. We have considered that component has two
states; one is normal state or failed state. Component change it is states with with with
with a probability well, that is known as transition probability. And we have already
seen that that the lambda probably lambda t d t gives a probability transition probability
from normal to failure and mu t d t gives the transition probability from failure to
normal. And a component can be either in failed state
or normal state so, as a result given that the component at time t is at normal and it
will remain normal at time t plus delta t. That will be 1 minus lambda t delta t that
is you are written here. And in the same way for failure case you can write like this ok
. So, now with this background what you want to do you want to find out that what is the
unavailability of of a system. Here it is a is a component we are basically
dealing with the component, but same analogy can be written can be talked ah when we discuss
the system failure also . So for, for the unavailability of the component we can write
down that component that probability that x x t plus delta t equal to 1, that is what
we ah we will be interested to find out. Probability x t plus delta t equal to 1. That will be
the unavailability within this delta period ok . So, now let us see that how we can compute
this one ok. So, you have seen in fact, you know that;
what is your unavailability ok . So, unavailability the probability that the component is in failed
state at time t equal to t, given that as it was as good as new when time t equal to
0 ok. So, Q t means what you can write Qt probability x t equal to 1. Now then we can
write Q plus delta t equal to probability that x t plus delta t equal to 1 ok .
So, if you if you see here that at t plus delta t it is unavailable; then what are the
how what are the ways it can happen? One is that the component is normal means at x t
equal to 0 at time t equal to t and it has it has been the transition has taken place
to failed state. Or it is basically at failed state and time t equal to t and t equal to
t plus delta t it remain in the failed state. So, that these are the two things so that
is what is written here . So, what we have written here you will see, probability x t
plus delta t equal to 1, given that x t equal to 0 t. It was here, then it transition take
place; so it was here that is the probability x t equal to 0 and it transition at takes
place x t plus delta t 1 so this from this to this.
Second one is this at time t equal to t or x t equal to and x t equal to 1 it is in the
failed state and it remain failed state. So, that is why probability x t plus delta t equal
to 1 given x t equal to 1 into this . So, these are the basically two probabilities
are multiplied and then summed up. You understood I hope ok. So, what we have written then;
in order to write should I should I repeat this, let me repeat.
What is Q t plus delta t you will say that probability that x t plus delta t equal to
1 . So, what are the different way it can take place; first one is it is as 0 that is
P x t equal to 0 that is. And at x t plus 1 t plus delta t this is 1 given that at x
t at equal to 0 so this is the first part. Similarly you can write the second part, so
these two this probability time this probability plus this probability to this probability
ok this is what is the unavailability at time t plus delta t .
So, now what happened you just put these values, like this value, as well as this value, this
value, as well as this value. The question is what is the probability the transition
probability from normal to fails lambda t d t ok. Then what is the probability the probability
x t equal to 0 . It is basically probability that it is available component is available
at t equal at ah basically t equal to t. So that means, it is 1 minus unavailability
so you written 1 minus unavailability 1 minus Q t . Then so this one is lambda d t, this
one is 1 minus Q t . Then what will happen to this, x t 1 given that it was 1 is 1 minus
this is 1 minus mu d t . What will be the multiplication? It is failed Q t so that is
what is written here 1 minus mu d t . So, we can do some algebraic manipulation
here . So, Q t plus delta t you write from this side you bring Q t, because this into
this is Q t bring this. And this side you write this d t is there as well as d t is
there d t will be here. Now d t then here d t into 1 and that a lambda and here basically
that d t into mu. So, lambda minus lambda minus mu into Q t and plus lambda d t you
will see. If you are not understanding I am writing
this one this will be lambda d t minus lambda d t Q t plus Q t minus mu d t Q t so what
we have written there this minus this written there .
Then you see d t lambda is there and ok d t lambda t Q t is there mu t lambda t Q t
is there so minus and minus so minus mu lambda minus mu d t d t and Q t Q t then 1 lambda
d t this here ok . So, now the Q these minus these if you divide this one by d t you will
get this equation. What is this in between what is there?
So this can be written this can be written as Q t plus delta t minus Q t by d t equal
to this much. So, lambda plus minus lambda minus mu Q t . So, now, this quantity is nothing,
but del Q t by d t that is what is written here.
So, del Q t by d t equal to this plus this this is the first order differential equation
. So, you have to solve this equation and if once you solve this we will see the one
only formula . That once you solve this given this condition t equal to 0 unavailability
is 0 the the the solution to this to this differential equation will be this; Q t equal
to lambda by lambda plus mu 1 minus e to the power minus lambda plus mu into t ok .
So, this is our unavailability and for two state system using Markov Analysis ok . So,
the failure rate all those things failure rate is constant and ultimately ah that failure
density exponential failure quality distribution ok. Now we will we will show you that how
do arrive at the solution from given this differential equation ok.
This is the equation Bernoulli form of differential equation. First order del y by del t R t y,
S t y to the power n. If your differential equation of is this form, then the solution
to this equation will be of this form. Where I t is this e to the power integration 1 minus
n R t d t ok . So, now you have to see that our equation
this equation whether it is this form or not . So, first you do one thing that if I say
that y is Q t y is Q t then del y by d t is del Q by . So, del y by del t equal to del
Q t by d t ok . Now if we say R t equal to lambda plus mu
if you say R t equal to lambda plus mu and y equal to Q t. Already y equal to Q t then
this into this is nothing, but this quantity. So, in quantity is R t into y . And this quantity
is d y by d t plus equal to lambda what is we are getting here S t y y to the power n.
Now if we put n equal to 0 then what will be this side S t .
So, S t equal to lambda now if we say S t equal to lambda so; that means, for n equal
to 0 ; the our these equation is actually this equation, satisfying this. Once this
equation satisfying this we can use the solution this is the solution that is what you are
writing here . So, our I t our I t is e to the power 1 minus n .
So, I t I can write e to the power integration 1 minus n R t d t n equal to 0 . So, that
mean e to the power integration 1 minus 0 is 1 what is our R t R t is lambda plus mu
lambda plus mu into d t so, I t you got it . So, you know I t then our solution n equal
to 0 so, y y equal to Q t equal to 1 by this I t into 1 minus n is 1.
So, I t you write I t as it is S t is what? S t is lambda so S t lambda d t plus c . So,
now, you take the integration first integration of this and then also integration of this
and when you put this to ultimately will be getting suppose integration of this ok same
thing. Now you will be getting this equation ok .
So, lambda by lambda plus mu into, into your lambda by lambda this c into e power minus
lambda mu t now at initial condition if you put equal to 0 Q t Q t equal to 0 so, even
this becomes 0 . So, then here t is 0 e to the power 0 is want to c will be minus this
c will be minus this how this at 0 equal to lambda by lambda plus mu plus c into e to
the power 0 means 1. So, c will be minus lambda by lambda plus
mu . So, you got c equal to lambda plus minus lambda lambda plus mu and if you put in this
equation the c part you will be getting this ok . So, lambda by 1 minus lambda into 1 minus
e to the power minus lambda mu into t. Now this is the equation we have obtained using
Markov analysis and this is our unavailability equation .
So, if you know unavailability equation and we have seen in the last class that the how
this unavailability is related with unconditional failure intensity and unconditional repair
intensity last class you recall . So, we have written there the w t is unconditional failure
intensity which is lambda into availability. Now availability 1 minus Q t so already we
have identified the value of Q t. So, you put here the value of Q t and finally, you
are getting this is your unavailability equation, ah unconditional failure intensity equation
ok . So, Q t is what? Q t we got lambda by lambda plus mu e to the power into 1 minus
e to the power minus lambda plus mu into t. Then w t is this again the repair 1 the unconditional
repair ah intensity you will be writing like this ok . So, earlier I said that these are
all intensity so you write intensity . So, unconditional repair intensity is mu into
this now mu is known Q t is known so this is your equation.
So, you can calculate expected number of failures once you have ah unavailability you are getting
w t. So, once you have small w t, then that expected number of failure ok. This is 0 to
t time you just integrate w t d t 0 to t this integration will give you this equation. And
expected number of repair will give you this equation ok .
Please try, and I am sure that ah you will be able to do it. This is very straightforward
case so on only the that Markov state transition you have to understand. And in case of two
state you will be having 0 and 1 you use the indicator variables . And also you know that
what are the probability once the lambda the parameter for failure and repair processes
are given . So, using Markov Analysis so you will be able
to find out the Q t and from Q t other parameters of the component will be estimated including
w t capital w like this means. Your unconditional failure, ah intensity expected number of failures,
unconditional failure, repair intensity, and expected number of repairs. And also if you
want that mean time and other things also you will be able to calculate ok .
Now, let us see another example, where we are basically thinking that the component
is not is 2 state one, component it is a 3 state component. What are what do you mean
by 3 state component? We are saying so I am writing here component, here also component,
here also component ok. Just I will go back little ok, so here we have written in terms
of component no problem . So, here also I am writing in terms of component ok.
Now that mean how many states are there 0? So, 0 state 1 is 0 normal, then 1 it failed,
but failed safely, and two mean failed unsafely ok . So, you have heard the concept called
failed safe system component maybe failed safe . So, it fail, but it will not create
any problem to the neighboring component all the system as a whole
But if it fail and ultimately it leads to loss apart from the component loss itself
. So, that we know or it will create problem numbering component of the system as a whole
. So, then what will happen it is basically we are saying that it is unsafely . So, here
also you you can find out the transition probabilities and you can create the equations for availability
or as well as unavailability ok . Now, suppose the component is at state 1 and
that mean and it is repaired and the repair; repair intensity that is the parameter mu
s . So, then ah repair rate basically mu s is basically that that the parameter value
and per unit time the probability failure per unit time .
So, then if you multiplied by d t then what will happen? That mean if the system is at
failed state, what is the probability that it will it will be in the working state is
mu s d t . Suppose it is working, but what is the probability during transit to your
failed state then lambda s d t. Similarly suppose the it is working, but it can go to
the unsafe totally lambda u d t and it will be repaired back to normal it is mu u d t
mu for repair probability and lambda for that is usually your probability; obviously, per
unit time power into per unit time . Then if you are interested to know that what
is the availability of the system at time t plus delta t; x t plus delta t then there
is indicator value will be 0. This is basically availability; that means, the probability
at t will at the system will be at ah component will be at 0 state, at time t plus delta t
. So, how many ways it can happen? It can happen fast if it is at the state 1 and it
is a a transition takes place to state 0. It is at state 0 remain at state 0 it was
at state 2 and transition has taken to state 0. This is what is written here first it is
at state 1; that mean probability x t equal to 1 . So, it the transition takes place within
delta t time, so that is basically this . So, these two multiplication talks about that
the component at state 1 at time t equal to t and it is state 0 at time t equal to t plus
delta t then this is the probability. Similarly it is at state 0 this and it remains
in this, this. Similarity it is state two this remain at state 2 this. So, this 3 three
terms will be added. Probability term related to staying at state 1 and transition to state
0, staying at state 0 and remaining to a state 0, staying at state 2 and transition to state
0 so this. Now if you basically now algebraic manipulation if you do you will be getting.
So, what you will do you put the values basically. So, P 0 t plus delta t; now what is if it
is at state 1 coming back to state 0 that mean mu S into delta t. So, they sorry, we
have written written mu S delta t . Then it was in state 1 that is P 1 t, now
in the and the second one is will be what is it it it is at state it was at state 0
that is P 0 t . And it remains at this then there are 3 states so this probability plus
this probability and plus this probability will be equal to 1. So, as a result the it
will remain at state 0 probability is 1 minus lambda u d t and lambda s d t. So, that is
written here ok; now you understand . So, domain algebraic manipulation here and finally,
you will come to this equation ok. So, del P 0 by del t plus lambda s d t delta
u d t, P 0 t equal to mu s P 1 t, mu u, P 2 t . Now, what is mu s? Mu s is this means
it failed safely and it it it it is repaired. What is the probability of repair?
Similarly it it it it remain and failed unsafely, then this is probability. It is working going
to state 1, going to state 2 ok. Now you you have to solve this, but in order to solve
this you required to know P 1 and P 2 also . So, what you have to do you have to find
out this two also. So, in the same analogy the way we have found
out P 0, t plus delta t you have, you will be able to find out P 1 t plus delta t. So,
p 1 t plus delta t means it is in state 1, then this is nothing, but that it is in the
state 0 transition takes place to state 1 into that it was in state 0 plus it was in
state 1 remain in state 1. There is nothing related to we have not considered anything
related to the transition from fail safe to unfail unsafe or fail unsafe to fail safe.
This is not possible under the given condition this is not possible. So, there is no no other
transition, apart from transition related from 0 to 1 or remain at 1. Similarly, here
0 to 2 or remain at 2; there is no state transition from 1 to 2 or 2 to 1. So, as a result when
we are computing the system will ah that ah unavailability at our system is at state 1.
Then then what we are doing , then we are writing that the possible out possibilities
outcomes are like this ok. So, first one is this, second one is this
then algebraic manipulation, you got this equation. And for the other case P t plus
delta ah d t this one here also two out, two possible scenarios. So, there probabilities
are ah every scenario probability is computed. These two are summed up to get this and then
through algebraic manipulation again we got this equation .
So, you have 3 differential equations this P, P 0, P 1, and P 2. Here we have straight
way using P 0, P 1 and P 2 . So 0 for normal state, 1 for failed safe state, and another
2 for ah fail unsafe state . Now when we put boundary condition and use the ah that proper
solution approach. What happened we will get P 0 is this P 1
is this P 2 is this . Let me know what happened, P 0 means it is the system availability, P
1 unavailability because it failed safely, P 2 unavailability failed unsafely ok . So,
the situation maybe 4 state, maybe in state situation. And you can use that Markov chain
or Markov Analysis to find out the ah that different ah scenarios, different probability
parameters, for components ok under consideration. We hope that you have understood it, and you
we have taken the 3 state case example from this book. And 2 state one from this book.
And over all the Markov chain and analysis or Markov Analysis are such. This some portion
is given in this book , but for the solution approach you may refer to some differential
equation book ok. Thank you, very much.