| Peer-Reviewed

Stability of a Regularized Newton Method with Two Potentials

Received: 14 February 2020     Accepted: 30 November 2020     Published: 22 January 2021
Views:       Downloads:
Abstract

In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.

Published in International Journal of Theoretical and Applied Mathematics (Volume 7, Issue 1)
DOI 10.11648/j.ijtam.20210701.11
Page(s) 1-11
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2021. Published by Science Publishing Group

Keywords

Monotone Inclusions, Newton Method, Levenberg-Marquardt Regularization, Dissipative Dynamical Systems, Lyapunov Analysis, Weak Asymptotic Convergence, Forward-Backward Algorithms, Gradient-Projection Methods

References
[1] B. Abbas, H. Attouch, Dynamical systems and forward-backward algorithms associated with the sum of a convex subdifferential and a monotone cocoercive operator, Optimization, (2014) http://dx.doi.org/10.1080/02331934.2014.971412.
[2] B. Abbas, H. Attouch, B. F. Svaiter, Newton-like dynamics and forward-backward methods for structured monotone inclusions in Hilbert spaces, J. Optim. Theory Appl., 161 (2014), No. 2, pp. 331-360.
[3] Alvarez, F., Attouch, H., Bolte, J., Redont, P.: A second- order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics. J. Math. Pures Appl. 81, 747–779 (2002).
[4] Antipin, A. S.: Minimization of convex functions on convex sets by means of differential equations. Differential Equations. 30, 1365–1375 (1994).
[5] Attouch, H., Briceno, L., Combettes, P.L.: A parallel splitting method for coupled monotone inclusions. SIAM J. Control Optim. 48, 3246–3270 (2010).
[6] H. Attouch, P. Redont, B. F. Svaiter, Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces, J. Optim. Theory Appl., 157 (2013), No. 3, pp. 624–650. International Journal of Theoretical and Applied Mathematics 2020; 7(1): 1-11 11
[7] Attouch, H., Svaiter, B.F.: A continuous dynamical Newton-like approach to solving monotone inclusions. SIAM J. Control Optim. 49, 574–598 (2011).
[8] Bauschke, H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory. CMS books in Mathematics, Springer (2011).
[9] Beck, A., Teboulle, M.: Gradient-based algorithms with applications in signal recovery problems. In: Palomar, D., Eldar, Y. (eds.): Convex Optimization in Signal Processing and Communications, pp. 33–88. Cambridge University Press, (2010).
[10] Bolte, J.: Continuous gradient projection method in Hilbert spaces. J. Optim. Theory Appl. 119, 235–259 (2003).
[11] Br´ ezis, H.: Op´ erateurs Maximaux Monotones et Semi- Groupes de Contractions dans les Espaces de Hilbert. North-Holland/Elsevier, New-York (1973).
[12] Br´ ezis, H.: Analyse Fonctionnelle. Masson, Paris (1983).
[13] Bruck, R. E.: Asymptotic convergence of nonlinear contraction semigroups in Hilbert spaces. J. Funct. Anal. 18, 15-26 (1975).
[14] Dennis, J. E., Schnabel, R. B.: Numerical Methods for Unconstrained Minimization. Prentice-Hall, Englewood Cliffs, NJ,1983. ReprintedbySIAMpublications(1993).
[15] Ferreira, O. P., Svaiter, B. F.: Kantorovich’s theorem on Newton’s method on Riemannian manifolds. Journal of Complexity 18, 304–329 (2002).
[16] G¨uler, O.: On the convergence of the proximal point algorithm for convex minimization, SIAM J. Control Optim. 29, 403–419 (1991).
[17] Haraux, A.: Syst` emes dynamiques dissipatifs et applications. RMA 17, Masson, Paris, (1991).
[18] Nocedal, J., Wright, S.: Numerical Optimization. Springer Series in Operations Research, Springer-Verlag, New-York (1999).
[19] Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Amer. Math. Soc. 73, 591–597 (1967).
[20] Peypouquet, J., Sorin, S.: Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time, J. of Convex Analysis 17, 1113-1163 (2010).
[21] Rockafellar, R. T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976).
[22] Rockafellar, R. T.: Maximal monotone relations and the second derivatives of nonsmooth functions. Ann. Inst. Henri Poincar? 2, 167–184 (1985).
[23] Solodov, M. V., Svaiter, B. F.: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M., Qi, L., (eds): Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, pp. 355-369. Kluwer Academic Publishers, (1999).
Cite This Article
  • APA Style

    Boushra Abbas, Ramez Koudsieh. (2021). Stability of a Regularized Newton Method with Two Potentials. International Journal of Theoretical and Applied Mathematics, 7(1), 1-11. https://doi.org/10.11648/j.ijtam.20210701.11

    Copy | Download

    ACS Style

    Boushra Abbas; Ramez Koudsieh. Stability of a Regularized Newton Method with Two Potentials. Int. J. Theor. Appl. Math. 2021, 7(1), 1-11. doi: 10.11648/j.ijtam.20210701.11

    Copy | Download

    AMA Style

    Boushra Abbas, Ramez Koudsieh. Stability of a Regularized Newton Method with Two Potentials. Int J Theor Appl Math. 2021;7(1):1-11. doi: 10.11648/j.ijtam.20210701.11

    Copy | Download

  • @article{10.11648/j.ijtam.20210701.11,
      author = {Boushra Abbas and Ramez Koudsieh},
      title = {Stability of a Regularized Newton Method with Two Potentials},
      journal = {International Journal of Theoretical and Applied Mathematics},
      volume = {7},
      number = {1},
      pages = {1-11},
      doi = {10.11648/j.ijtam.20210701.11},
      url = {https://doi.org/10.11648/j.ijtam.20210701.11},
      eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ijtam.20210701.11},
      abstract = {In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.},
     year = {2021}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - Stability of a Regularized Newton Method with Two Potentials
    AU  - Boushra Abbas
    AU  - Ramez Koudsieh
    Y1  - 2021/01/22
    PY  - 2021
    N1  - https://doi.org/10.11648/j.ijtam.20210701.11
    DO  - 10.11648/j.ijtam.20210701.11
    T2  - International Journal of Theoretical and Applied Mathematics
    JF  - International Journal of Theoretical and Applied Mathematics
    JO  - International Journal of Theoretical and Applied Mathematics
    SP  - 1
    EP  - 11
    PB  - Science Publishing Group
    SN  - 2575-5080
    UR  - https://doi.org/10.11648/j.ijtam.20210701.11
    AB  - In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.
    VL  - 7
    IS  - 1
    ER  - 

    Copy | Download

Author Information
  • Mathematics Department, Tichreen University, Lattakia, Syria

  • Mechatroncis Department, Manara University, Lattakia, Syria

  • Sections