Markov-chains-马尔科夫链.docx

MarkovChains4.1 INTRODUCTIONANDEXAMP1.ESConsiderastochasticprocessXn,n=0,1,2,.thattakesonafiniteorcountablenumberofpossiblevalues.Unlessotherwisementioned,thissetofpossiblewillbedenotedbythesetofnonnegativeintegers0,1,2,.IfX,=i,thentheprocessissaidtobeinstateiattimen.Wesupposethatwhenevertheprocessisinstatei,thereisafixedprobabilityP11thatitwillnextbeinstatej.Thatis,wesupposethat0PX"1=jXnl=i|iXl=iJ.X=i=Pijforallstatesi0,i,.in-,i,jandalln20.SuchastochasticprocessisknownasaMarkovchain.Equation()maybeinterpretedasstatingthat,foraMarkovchain,theconditionaldistributionofanyfuturestateX11.1giventhepaststatesXo,X.,Xll-andthepresentstateX11zisindependentofthepaststatesanddependsonlyonthepresentstate.ThisiscalledtheMarkovianproperty.Theva1uePiJrepresentstheprobabilitythattheprocesswill,wheninstatei,nextmakeatransitionintostatej.Sinceprobabilitiesarenonnegativeandsincetheprocessmustmakeatransitionintosomestate,WehavethataPi声O,i,j2O：Z4=l，i=o,1/?./.:1.etPdenotethematrixofonc-steptransitionprobabilitiesPij,sothat%0-%,1*EXAMP1.E4.1（八）TheM/G/lQueue.SupposethatcustomersarriveataservicecenterinaccordancewithaPoissonprocesswithrate.Thereisasingleserverandthosearrivalsfindingtheserverfreegoimmediatelyintoservice;allotherswaitinlineuntiltheirserviceturn.TheservicetimesofsuccessivecustomersareassumedtobeindependentrandomvariableshavingacommondistributionG：andtheyarealsoassumedtobeindependentofthearrivalprocess.TheabovesystemiscalledtheM/G/lqueueingsystem.TheletterMstandsforthefactthattheinterarrivaldistributionofcustomersisexponential,Gfortheservicedistribution；thenumber1indicatesthatthereisasingleserver.IfweletX(t)denotethenumberofcustomersinthesystematt,then；.X(t)11>0wouldnotpossesstheMarkovianpropertythattheconditionaldistributionofthefuturedependsonyonthepresentandnotonthepast.Forifweknowthenumberinthesystemattimet,then,topredictfuturebehavior,whereaswewouldnotcarehowmuchtimehadelapsedsincethelastarrival(sincethearrivalprocessismemory1ess),wewouldcarehowlongthepersoninservicehadalreadybeenthere(sincetheservicedistributionGisarbitraryandthereforenotmemoryless).Asameansofgettingaroundtheabovedi1emmaletusonlylookatthesystematmomentswhencustomersdepart.Thatis,letXndenotethenumberofcustomersleftbehindbythenthdeparture,n1.Also,letYndenotethenumberofcustomersarrivingduringtheserviceperiodofthe(n+l)stcustomer.WhenXn>0,thenthdepartureleavesbehindXncustomers-ofwhichoneentersserviceandtheotherXn-Iwaitinline.Hence,atthenextdeparturethesystemwillcontaintheXn-Icustomersthatwereinlineinadditiontoanyarrivalsduringtheservicetimeofthe(n+l)stcustomer.SinceasimiIarargumentholdswhenX,=0,WeseethatO)U=X.T+ZM'>°1匕if×,=0SinceYh,nN1,representthenumberofarrivalsinnonoverlappingserviceintervals,itfollows,thearrivalprocessbeingBoissonprocess,thattheyareindependentandOPYn=j=,e'it-dG(x).j=0,1Form(4,1.2)and()tfollowsthatX.n=i,2,.)isaMarkovchainwithtransitionprobabiHtiesgivenbyPll=e山邛dG(x).j0JP,*P11=OotherwiseSupposethatcustomersEXAMP1.E4.1(B)TheM/G/lQueue.arriveatasingle-servercenterinaccordancewithanarbitraryrenewalprocesshavinginterarrivaldistributionG.SupposefurtherthattheservicedistributionisexponentialwithrateKIfweletX11denotethenumberofcustomersinthesystemasseenbythentharrival,itiseasytoseethattheprocessX11,n21isaMarkovchain.TocomputethetransitionprobabilitiesPijforthisMarkovchain,letusfirstnotethat,aslongastherearecustomerstobeserved,thenumberofservicesinanylengthoftimetisaPoissonrandomvariablewithmeant.Thisistruesincethetimebetweensuccessiveservicesisexponentialand,asweknow,thisimpliesthatnumberofservicesthusconstitutesaPoissonprocess.Therefore,P1.i.券dG(r),i0Whichfollowssinceifanarrivalfindsiinthesystem,thenthenextarrivalwi11findi+1minusthenumbersserved,andtheprobabilitythatjwillbeservediseasilyseen(byconditioningonthetimebetweenthesuccessivearrivals)toequaltheright-handsideoftheabove.TheformulaforP10islittledifferent(itistheprobabilitythatatleasti+1PoissoneventsoccurinarandomlengthoftimehavingdistributionG)andthusisgivenbyP-fe-G(O,i0RemarkThereadershouldnotethatintheprevioustwoexamplesWewereabletodiscoveranembeddedMarkowchainbylookingattheprocessonlyatcertaintimepoints,andbychoosingthesetimepointssoastoexploitthelackofmemoryoftheexponentialdistribution.Thisisoftenafruitfulapproachforprocessesinwhichtheexponentialdistributionispresent.EXMP1.E4.1(C)SuasofIndependent,IdenticallyDistributedRandoavariables.TheGeneralRandomWalk.1.etXi,i1,beindependentandidenticallydistributedwithP(Xi=j)=aj,j=0,±1,.IfweletSo=OandS,二£x,r-lThenS11,n0isaMarkovchainforwhichFlj=aj-Sn,n0：iscalledthegeneralrandomwalkandwi11bestudiedinchapter7.EXAMP1.E4.1(D)TheAbsolutevalueoftheSimpleRandcmWallk.TherandomwalkSn,n>l),whereS11=,Xi.issaidtobeasimplerandomWaIkifforsomep,0<p<l,P(X1=I)=P,P(Xi=-l)=q三l-p.Thusinthesimplerandomwalktheprocessalwayseithergoesuponestep(withprobabiIityp)ordownonestep(withprobabi