VAGA: a novel viscosity-based accelerated gradient algorithm: Convergence analysis and applications

Abstract

Proximal Algorithms are known to be very popular in the area of signal processing, image reconstruction, variational inequality and convex optimization due to their small iteration costs and applicability to the non-smooth optimization problems. Various real-world machine learning problems have been solved utilizing the non-smooth convex loss minimization framework, and a recent trend is to design new accelerated algorithms to solve such frameworks efficiently. In this paper, we propose a novel viscosity-based accelerated gradient algorithm (VAGA), that utilizes the concept of viscosity approximation method of fixed point theory for solving the learning problems. We discuss the boundedness of the sequence generated by this iterative algorithm and prove the strong convergence of the algorithm under the few specific conditions. To test the practical performance of the algorithm on real-world problems, we applied it to solve the regularized multitask regression problem with sparsity-inducing regularizers. We present the detailed comparative analysis of our algorithm with few traditional proximal algorithms on three real benchmark multitask regression datasets. We also apply the proposed algorithm to the task of joint splice-site recognition problem of bio-informatics. The improved results demonstrate the efficacy of our algorithm over state-of-the-art proximal gradient descent algorithms. To the best of our knowledge, it is the first time that a viscosity-based iterative algorithm is applied to solve the real world problem of regression and recognition. © 2017, Springer Science+Business Media, LLC, part of Springer Nature.