ABSTRACT

This paper presents a general model for the formulation and solution of the risk-sensitive dynamic decision problem that maximizes the certain equivalent of the discounted rewards of a time-varying Markov decision process. The problem is solved by applying the principle of optimality and stochastic dynamic programming to the immediate rewards and the certain equivalent associated with the remaining transitions of a time-varying Markov process over a finite or infinite time horizon, under the assumptions of constant risk aversion and discounting of future cash flows. The solution provides transient and stationary optimal decision policies which depend on the presence or absence of discounting. The construction equipment replacement problem serves as an example application of the model to illustrate the solution methodology, and the sensitivity of the optimal policy to the discount factor and the degree of risk aversion.

AUTHORS

Photios G. Ioannou

Civil & Environmental Engineering Department

University of Michigan

Ann Arbor, Michigan 48109-2125, U.S.A.

e-mail: photios@umich.edu

KEYWORDS

Dynamic programming, optimal policies, Markov process, decision analysis, uncertainty, discounting, risk aversion, engineering economy, equipment replacement.

AVAILABILITY

Ioannou, P.G., "Dynamic Probabilistic Decision Processes," Journal of Construction Engineering and Management, ASCE, Vol. 115, No. 2, 237-257, June 1989.