节点文献

基于梯度跟踪和分布式重球加速的分布式随机优化算法(英文)

A distributed stochastic optimization algorithm with gradient-tracking and distributed heavy-ball acceleration

  • 推荐 CAJ下载
  • PDF下载
  • 不支持迅雷等下载工具,请取消加速工具后下载。

【作者】 孙碧皓胡锦辉夏大文李华青

【Author】 Bihao SUN;Jinhui HU;Dawen XIA;Huaqing LI;Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing,College of Electronic and Information Engineering, Southwest University;College of Data Science and Information Engineering, Guizhou Minzu University;

【通讯作者】 李华青;

【机构】 西南大学电子信息工程学院非线性电路与智能信息处理重庆市重点实验室贵州民族大学数据科学与信息工程学院

【摘要】 由于在机器学习和信号处理中的广泛应用,近年来分布式优化得到良好发展。本文致力于研究分布式优化以求解目标函数全局最小值。该目标是分布在n个节点的无向网络上的平滑且强凸的局部成本函数总和。与已有工作不同的是,我们使用分布式重球项以提高算法的收敛性能。为使现有分布式随机一阶梯度算法的收敛加速,将动量项与梯度跟踪技术结合。仿真结果表明,在不增加复杂度的情况下,所提算法具有比GT-SAGA更高收敛速率。在真实数据集上的数值实验证明了该算法的有效性和正确性。

【Abstract】 Distributed optimization has been well developed in recent years due to its wide applications in machine learning and signal processing. In this paper, we focus on investigating distributed optimization to minimize a global objective. The objective is a sum of smooth and strongly convex local cost functions which are distributed over an undirected network of n nodes. In contrast to existing works, we apply a distributed heavy-ball term to improve the convergence performance of the proposed algorithm. To accelerate the convergence of existing distributed stochastic first-order gradient methods, a momentum term is combined with a gradient-tracking technique. It is shown that the proposed algorithm has better acceleration ability than GT-SAGA without increasing the complexity. Extensive experiments on real-world datasets verify the effectiveness and correctness of the proposed algorithm.

【基金】 Project supported by the Open Research Fund Program of Data Recovery Key Laboratory of Sichuan Province,China (No.DRN2001);the National Natural Science Foundation of China(Nos. 61773321 and 61762020);the Science and Technology Top-Notch Talents Support Project of Colleges and Universities in Guizhou Province,China (No. QJHKY2016065);the Science and Technology Foundation of Guizhou Province,China(No. QKHJC20181083);the Science and Technology Talents Fund for Excellent Young of Guizhou Province,China (No.QKHPTRC20195669)
  • 【文献出处】 Frontiers of Information Technology & Electronic Engineering ,信息与电子工程前沿(英文) , 编辑部邮箱 ,2021年11期
  • 【分类号】TP18
  • 【下载频次】66
节点文献中: 

本文链接的文献网络图示:

本文的引文网络