Energy and mean-payoff parity Markov Decision Processes Conference Paper


Author(s): Chatterjee, Krishnendu; Doyen, Laurent
Title: Energy and mean-payoff parity Markov Decision Processes
Title Series: LNCS
Affiliation IST Austria
Abstract: We consider Markov Decision Processes (MDPs) with mean-payoff parity and energy parity objectives. In system design, the parity objective is used to encode ω-regular specifications, and the mean-payoff and energy objectives can be used to model quantitative resource constraints. The energy condition re- quires that the resource level never drops below 0, and the mean-payoff condi- tion requires that the limit-average value of the resource consumption is within a threshold. While these two (energy and mean-payoff) classical conditions are equivalent for two-player games, we show that they differ for MDPs. We show that the problem of deciding whether a state is almost-sure winning (i.e., winning with probability 1) in energy parity MDPs is in NP ∩ coNP, while for mean- payoff parity MDPs, the problem is solvable in polynomial time, improving a recent PSPACE bound.
Conference Title: MFCS: Mathematical Foundations of Computer Science
Volume: 6907
Conference Dates: August 22-26, 2011
Conference Location: Warsaw, Poland
Publisher: Springer  
Location: Berlin, Heidelberg
Date Published: 2011-09-28
Start Page: 206
End Page: 218
Sponsor: This work was partially supported by FWF NFN Grant S11407-N23 (RiSE) and a Microsoft faculty fellowship.
URL:
DOI: 10.1007/978-3-642-22993-0_21
Open access: yes (repository)
IST Austria Authors
Related IST Austria Work