Polish空间上的折扣马氏过程量子化策略的渐近优化

Abstract

Abstract:

In this paper, we study the asymptotic optimality of the quantized stationary policies for continuous-time Markov decision processes (CTMDPs) with Polish space and state-dependent discount factors. Firstly, the existence and uniqueness of the discounted optimal equation (DOE) and its solution are established. Secondly, the existence of the optimal deterministic stationary policies is proved under appropriate conditions. In addition, in order to discretize the action space, a series of quantization policies are constructed to approximate the optimal stationary policies of the discounted CTMDPs in general state (Polish) space by using the policies in finite action space. Finally, an example is given to illustrate the asymptotic approximation results of this paper.

Key words: Continuous-time Markov decision processes, State-dependent discount factors, Discounted criterion, Quantized stationary policies, Asymptotic optimality

CLC Number:

O211.6

Xiao Wu,Yinying Kong,Zhenbin Guo. Asymptotic Optimality of Quantized Stationary Policies in Continuous-Time Markov Decision Processes with Polish Spaces[J].Acta mathematica scientia,Series A, 2022, 42(2): 594-604.

Trendmd

References 23

1	Journals Doshi B T . Continuous-time control of Markov processes on an arbitrary state space: discounted rewards. Ann Stat, 1976, 4: 1219- 1235
2	Dynkin E B , Yushkevich A A . Controlled Markov Processes. New York: Springer, 1979
3	Feinberg E A . Continuous-time jump Markov decision processes: A discrete-event approach. Math Oper Res, 2004, 29: 492- 524 doi: 10.1287/moor.1040.0089
4	Guo X P . Continuous-time Markov decision processes with discounted rewards: The case of Polish spaces. Math Oper Res, 2007, 32 (1): 73- 87 doi: 10.1287/moor.1060.0210
5	Guo X P . Constrained optimization for average cost continuous-time Markov decision processes. IEEE Trans Autom Control, 2007, 52 (6): 1139- 1143 doi: 10.1109/TAC.2007.899040
6	Guo X P , Hernández-Lerma O . Continuous-time Markov Decision Processes: Theory and Applications. Berlin: Springer, 2009
7	Guo X P , Song X Y . Discounted continuous-time constrained Markov decision processes in Polish spaces. Ann Appl Probab, 2011, 21 (5): 2016- 2049
8	Hernández-Lerma O , Lasserre J B . Discrete-Time Markov Control Processes. New York: Springer, 1996
9	Hernández-Lerma O , Lasserre J B . Further Topics on Discrete-Time Markov Control Processes. New York: Springer, 1999
10	Puterman M L . Markov Decision Processes. New York: Wiley, 1994
11	Feinberg E A , Shwartz A . Markov decision models with weighted discounted criteria. Math Oper Res, 1994, 19 (1): 152- 168 doi: 10.1287/moor.19.1.152
12	González-Hernández J , López-Martínez R R , Pérez-Hernández J R . Markov control processes with randomized discounted cost. Math Methods Oper Res, 2007, 65: 27- 44 doi: 10.1007/s00186-006-0092-2
13	Wu X , Guo X P . First passage optimality and variance minimization of Markov decision processes with varying discount factors. J Appl Probab, 2015, 52 (2): 441- 456 doi: 10.1239/jap/1437658608
14	Wu X , Zhang J Y . Finite approximation of the first passage models for discrete-time Markov decision processes with varying discount factors. Discrete Event Dyn Syst, 2016, 26 (4): 669- 683 doi: 10.1007/s10626-014-0209-3
15	Hinderer K . Foundations of Non Stationary Dynamic Programming with Discrete Time Parameter. New York: Springer, 1970
16	Ye L E , Guo X P . Continuous-time Markov decision processes with state-dependent discount factors. Acta Appl Math, 2012, 121: 5- 27 doi: 10.1007/s10440-012-9669-3
17	Bertsekas D , Tsitsiklis J . Neuro-Dynammic Programming. Boston: Athena Scientific, 1996
18	Cavazos-Cadena R . Finite-state approximations for denumerable state discounted Markov decision processes. Appl Math Optim, 1986, 14: 1- 26 doi: 10.1007/BF01442225
19	Dufour F , Prieto-Rumeau T . Finite linear programming approximations of constrained discounted Markov decision processes. SIAM J Control Optim, 2013, 51 (2): 1298- 1324 doi: 10.1137/120867925
20	Wu X , Guo X P . Convergence of Markov decision processes with constraints and state-action dependent discount factors. Sci China Math, 2020, 63 (1): 167- 182 doi: 10.1007/s11425-017-9292-1
21	Saldi N , Linder T , Yuksel S . Asymptotic optimality and rates of convergence of quantized stationary policies in stochastic control. IEEE Transactions on Automatic Control, 2014, 60 (2): 553- 558
22	Saldi N , Linder T , Yuksel S . Near optimality of quantized policies in stochastic control under weak continuity conditions. J Math Anal Appl, 2016, 435: 321- 337 doi: 10.1016/j.jmaa.2015.10.008
23	Lund R B , Meyn S P , Tweedie R L . Computable exponential convergence rates for stochastically ordered Markov processes. Ann Appl Probab, 1996, 6: 218- 237

[1]	Wen Yuzhen, Wang Shaolin. Stackelberg Stochastic Differential Game of Insurer and Reinsurer Under Mean-Variance Framework [J]. Acta mathematica scientia,Series A, 2025, 45(4): 1327-1353.
[2]	Liu Yu, Chen Guanggan, Li Shuyong. Nonlinear Stability of Traveling Waves for Stochastic Kuramoto-Sivashinsky Equation [J]. Acta mathematica scientia,Series A, 2025, 45(3): 790-806.
[3]	Chen Yating, Liu Haiyan, Chen Mi. $\alpha$-Robust Optimal Investment Strategy Under Inflation [J]. Acta mathematica scientia,Series A, 2024, 44(6): 1617-1629.
[4]	Yang Hong, Zhang Xiaoguang. A Stochastic SIS Epidemic Model on Simplex Complexes [J]. Acta mathematica scientia,Series A, 2024, 44(5): 1392-1399.
[5]	Wang Qiufen, Zhang Shuwen. Stochastic Predator-Prey System with Fear Effect and Holling-Type III Functional Response [J]. Acta mathematica scientia,Series A, 2024, 44(5): 1380-1391.
[6]	Wang Yankai, Peng Xingchun. Time-Consistent Risk Control and Investment Strategies With Transaction Costs [J]. Acta mathematica scientia,Series A, 2024, 44(5): 1400-1414.
[7]	Fan Xiequan,Hu Haijuan,Wu Hao,Ye Yinna. Comparison on the Criticality Parameters for Two Supercritical Branching Processes in Random Environments [J]. Acta mathematica scientia,Series A, 2023, 43(5): 1440-1470.
[8]	Chen Yong,Li Ying,Sheng Ying,Gu Xiangmeng. Parameter Estimation for an Ornstein-Uhlenbeck Process Driven by a Type of Gaussian Noise with Hurst Parameter $H\in (0,\frac{1}{2})$ [J]. Acta mathematica scientia,Series A, 2023, 43(5): 1483-1518.
[9]	Yao Luogen,Liu Huan,Chen Qiqiong. Application of VG Distortion Operator in Option Pricing [J]. Acta mathematica scientia,Series A, 2023, 43(5): 1575-1584.
[10]	Lu Jiaxin,Dong Hua. Equilibrium Investment Strategy of DC Pension Plan with Mispricing and Return of Premiums Clauses Under the 4/2 Stochastic Volatility Model [J]. Acta mathematica scientia,Series A, 2023, 43(3): 939-956.
[11]	Huang ,Liu Haiyan,Chen Mi. Proportional Reinsurance and Investment Based on the Ornstein-Uhlenbeck Process in the Presence of Two Reinsurers [J]. Acta mathematica scientia,Series A, 2023, 43(3): 957-969.
[12]	Chen Yong,Gu Xiangmeng. An Improved Berry-Esséen Bound of Least Squares Estimation for Fractional Ornstein-Uhlenbeck Processes [J]. Acta mathematica scientia,Series A, 2023, 43(3): 855-882.
[13]	Chen Yejun, Ding Huisheng. Asymptotically Almost Periodic Solutions for Stochastic Differential Equations in Infinite Dimensions [J]. Acta mathematica scientia,Series A, 2023, 43(2): 341-354.
[14]	Yanjun Zhao,Xiaohui Sun,Li Su,Wenxuan Li. Qualitative Analysis of Stochastic SIRS Epidemic Model with Logistic Growth and Beddington-DeAngelis Incidence Rate [J]. Acta mathematica scientia,Series A, 2022, 42(6): 1861-1872.
[15]	Junfeng Liu,Lei Mao,Zhi Wang. Moment Bounds for the Fractional Stochastic Heat Equation with Spatially Inhomogeneous White Noise [J]. Acta mathematica scientia,Series A, 2022, 42(4): 1186-1208.

Asymptotic Optimality of Quantized Stationary Policies in Continuous-Time Markov Decision Processes with Polish Spaces

RichHTML

PDF (PC)

Abstract

Cite this article

share this article

References 23

Related Articles 15

Metrics

Comments

Recommended 10