两个带重启方向的改进 HS 型共轭梯度法

Abstract

Abstract:

The conjugate gradient method is one of the effective methods to solve large-scale unconstrained optimization. In this paper, the Hestenes-Stiefel (HS) conjugate parameter is improved, and then two extended HS-type conjugate gradient methods with restart directions are established by introducing restart conditions and restart directions. The first method produces descent direction under the weak Wolfe line search, and the second one obtains sufficient descent independent of any line search. Under conventional assumptions, the global convergence results of the two proposed methods are analyzed and obtained. Finally, the numerical comparison results and performance graphs show the effectiveness of the new methods.

Key words: Unconstrained optimization, Conjugate gradient method, Restart direction, Weak Wolfe line search, Global convergence

CLC Number:

O221.2

Liu Pengjie, Wu Yanqiang, Shao Feng, Zhang Yan, Shao Hu. Two Extended HS-type Conjugate Gradient Methods with Restart Directions[J].Acta mathematica scientia,Series A, 2023, 43(2): 570-580.

Trendmd

References

[1]	Hestenes M R, Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 1952, 49(6): 409-436
[2]	Fletcher R, Reeves C M. Function minimization by conjugate gradients. Computer Journal, 1964, 7(2): 149-154
[3]	Polak E, Ribière G. Note sur la convergence de méthodes de directions conjuées. Revue Fran?aise Informatique Et De Recherche Opérationnelle, 1969, 3(16): 35-43
[4]	Polyak B T. The conjugate gradient method in extremal problems. USSR Computational Mathematics and Mathematical Physics, 1969, 9(4): 94-112
[5]	Dai Y H, Yuan Y X. A nonlinear conjugate gradient method with a strong global convergence property. SIAM Journal on Optimization, 1999, 10(1): 177-182
[6]	Yao S W, Wei Z X, Huang H. A note about WYL's conjugate gradient method and its application. Applied Mathematics and Computation, 2007, 191(2): 381-388
[7]	Zhang L. An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation. Applied Mathematics and Computation, 2009, 215(6): 2269-2274
[8]	江羡珍, 马国栋, 简金宝. Wolfe 线搜索下一个新的全局收敛共轭梯度法. 工程数学学报, 2011, 28(6): 779-786
[8]	Jiang X Z, Ma G D, Jian J B. A new globally convergent conjugate gradient method with Wolfe line search. Chinese Journal of Engineering Mathematics, 2011, 28(6): 779-786
[9]	Zhu Z B, Zhang D D, Wang S. Two modified DY conjugate gradient methods for unconstrained optimization problems. Applied Mathematics and Computation, 2020, 373( 15): 125004
[10]	Jiang X Z, Jian J B, Song D, Liu P J. An improved Polak-Ribière-Polyak conjugate gradient method with an efficient restart direction. Computational and Applied Mathematics, 2021, 40(174): 1-24
[11]	Kou C X, Dai Y H. A modified self-scaling memoryless Broyden-Fletcher-Goldfarb-Shanno method for unconstrained optimization. Journal of Optimization Theory and Applications, 2015, 165: 209-224
[12]	Jian J B, Han L, Jiang X Z. A hybrid conjugate gradient method with descent property for unconstrained optimization. Applied Mathematical Modelling, 2015, 39(3/4): 1281-1290
[13]	Dai Z F, Wen F H. Another improved Wei-Yao-Liu nonlinear conjugate gradient method with sufficient descent property. Applied Mathematics and Computation, 2012, 218(14): 7421-7430
[14]	江羡珍, 韩麟, 简金宝. Wolfe 线搜索下一个全局收敛的混合共轭梯度法. 计算数学, 2012, 34(1): 103-112
[14]	Jiang X Z, Han L, Jian J B. A new globally convergent mixed conjugate gradient method with Wolfe line search. Mathematica Numerica Sinica, 2012, 34(1): 103-112
[15]	简金宝, 刘鹏杰, 江羡珍. 一个充分下降的谱三项共轭梯度法. 应用数学学报, 2020, 43(6): 1000-1012
[15]	Jian J B, Liu P J, Jiang X Z. A spectral three-term conjugate gradient method with sufficient descent property. Acta Mathematicae Applicatae Sinica, 2012, 43(6): 1000-1012
[16]	Jiang X Z, Jian J B. Improved Fletcher-Reeves and Dai-Yuan conjugate gradient methods with the strong Wolfe line search. Journal of Computational and Applied Mathematics, 2019, 348: 525-534
[17]	Liu Y F, Zhu Z B, Zhang B X. Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing. Journal of Applied Mathematics and Computing, 2022, 68: 1787-1816
[18]	Jiang X Z, Liao W, Yin J H, Jian J B. A new family of hybrid three-term conjugate gradient methods with applications in image restoration. Numerical Algorithms, 2022, 91: 161-191
[19]	朱志斌, 耿远航. 一个改进的 WYL 型三项共轭梯度法. 数学物理学报, 2021, 41A(6): 1871-1879
[19]	Zhu Z B, Geng Y H. An modified three-term WYL conjugate gradient method. Acta Mathematica Scientia, 2021, 41A(6): 1871-1879
[20]	马国栋. 强 Wolfe 线搜索下的修正 PRP 和 HS 共轭梯度法. 数学物理学报, 2021, 41A(3): 837-847
[20]	Ma G D. Improved PRP and HS conjugate gradient methods with strong Wolfe line search. Acta Mathematica Scientia, 2021, 41A(3): 837-847
[21]	江羡珍, 廖伟, 简金宝, 毋晓迪. 一个带重启步的改进 PRP 型谱共轭梯度法. 数学物理学报, 2022, 42A(1): 216-227
[21]	Jiang X Z, Liao W, Jian J B, Wu X D. An improved PRP type spectral conjugate gradient method with restart steps. Acta Mathematica Scientia, 2022, 42A(1): 216-227
[22]	袁功林, 吴宇伦, Pham H. 基于非单调线搜索的 HS-DY 形共轭梯度方法及在图像恢复中的应用. 数学物理学报, 2022, 42A(2): 605-620
[22]	Yuan G L, Wu Y L, Pham H. A modified HS-DY-type method with nonmonotone line search for image restoration and unconstrained optimization problems. Acta Mathematica Scientia, 2022, 42A(2): 605-620
[23]	Yuan G L, Zhou Y J, Zhang M X. A hybrid conjugate gradient algorithm for nonconvex functions and its applications in image restoration problems. Journal of the Operations Research Society of China, 2022, https://doi.org/10.1007/s40305-022-00424-6
[24]	Zoutendijk G. Nonlinear programming computational methods//Abadie J. ed Integer and Nonlinear Programming. Amsterdam: North-Holland, 1970: 37-86
[25]	Gould N I M, Orban D, Toint P L. CUTEr and SifDec: A constrained and unconstrained testing environment, revisited. ACM Transactions on Mathematical Software, 2003, 29(4): 373-394
[26]	Moré J J, Garbow B S, Hillstrome K E. Testing unconstrained optimization software. ACM Transactions on Mathematical Software, 1981, 7: 17-41
[27]	Andrei N. An unconstrained optimization test functions collection. Advanced Modeling and Optimization, 2008, 10(1): 147-161
[28]	Dolan E D, Moré J J. Benchmarking optimization software with performance profiles. Mathematical Programming, 2002, 91(2): 201-213

Two Extended HS-type Conjugate Gradient Methods with Restart Directions

RichHTML

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	Pengjie Liu, Hu Shao, Jiayi Li, Duanduan Dong. An Inertial Three-Term Conjugate Gradient Projection Method Based on BFGS Update [J]. Acta mathematica scientia,Series A, 2025, 45(5): 1698-1710.
[2]	Li Jiaofen, Kong Lvyuan, Song Jiashuo, Wen Yaqiong. A Riemannian Optimization Approach for a Class of Matrix Trace Function Extremum Problem in Feature Extraction [J]. Acta mathematica scientia,Series A, 2024, 44(4): 1012-1036.
[3]	Jian Jinbao, Dai Yu, Yin Jianghua. An Inertial Conjugate Gradient Projection Method for the Split Feasibility Problem [J]. Acta mathematica scientia,Series A, 2024, 44(4): 1066-1079.
[4]	Cai Yu, Zhou Guanghui. Global Convergence of a WYL Type Spectral Conjugate Gradient Method [J]. Acta mathematica scientia,Series A, 2024, 44(1): 173-184.
[5]	Jian Jinbao,Lin Hui,Ma Guodong. A Splitting Sequence Quadratic Programming Algorithm for the Large-Scale Nonconvex Nonseparable Optimization Problems [J]. Acta mathematica scientia,Series A, 2023, 43(4): 1284-1296.
[6]	Ning Zhang,Jinkui Liu. Spectral LS-type Projection Algorithm for Solving Nonlinear Pseudo-Monotone Equations [J]. Acta mathematica scientia,Series A, 2022, 42(6): 1886-1897.
[7]	Tingwei Pan,Suxiang He. The Greedy Simplex Algorithm for Double Sparsity Constrained Optimization Problems [J]. Acta mathematica scientia,Series A, 2022, 42(3): 920-933.
[8]	Gonglin Yuan,Yulun Wu,Hongtruong Pham. A Modified HS-DY-Type Method with Nonmonotone Line Search for Image Restoration and Unconstrained Optimization Problems [J]. Acta mathematica scientia,Series A, 2022, 42(2): 605-620.
[9]	Xianzhen Jiang,Wei Liao,Jinbao Jian,Xiaodi Wu. An Improved PRP Type Spectral Conjugate Gradient Method with Restart Steps [J]. Acta mathematica scientia,Series A, 2022, 42(1): 216-227.
[10]	Zhibin Zhu,Yuanhang Geng. A Modified Three-Term WYL Conjugate Gradient Method [J]. Acta mathematica scientia,Series A, 2021, 41(6): 1871-1879.
[11]	Guodong Ma. Improved PRP and HS Conjugate Gradient Methods with the Strong Wolfe Line Search [J]. Acta mathematica scientia,Series A, 2021, 41(3): 837-847.
[12]	Xiaoni Chi,Rong Zeng,Sanyang Liu,Zhibin Zhu. A Regularized Nonmonotone Inexact Smoothing Newton Algorithm for Weighted Symmetric Cone Complementarity Problems [J]. Acta mathematica scientia,Series A, 2021, 41(2): 507-522.
[13]	Yueyuan Liu,Kai Wang,Shujuan Qin,Jiaofen Li. An Effective Algorithm for Generalized Sylvester Equation Minimization Problem Under Columnwise Orthogonal Constraints [J]. Acta mathematica scientia,Series A, 2021, 41(2): 479-495.
[14]	Liu Yang,Zuicha Deng. An Inverse Initial Value Problem for Degenerate Parabolic Equations [J]. Acta mathematica scientia,Series A, 2020, 40(4): 891-903.
[15]	Guodong Ma. A Strongly Convergent Generalized Gradient Projection Method for Minimax Optimization with General Constraints [J]. Acta mathematica scientia,Series A, 2020, 40(3): 641-649.