A Flexible ADMM Algorithm for Big Data Applications

作者：Robinson Daniel P; Tappenden Rachael^*

来源：Journal of Scientific Computing, 2017, 71(1): 435-467.

DOI：10.1007/s10915-016-0306-6

• ADMM
• Convex optimization
• Gauss-Seidel
• Jacobi
• Regularization
• Separable function

We present a Flexible Alternating Direction Method of Multipliers (F-ADMM) algorithm for solving optimization problems involving a strongly convex objective function that is separable into blocks, subject to (non-separable) linear equality constraints. The F-ADMM algorithm uses a Gauss-Seidel scheme to update blocks of variables, and a regularization term is added to each of the subproblems. We prove, under common assumptions, that F-ADMM is globally convergent and that the iterates converge linearly. We also present a special case of F-ADMM that is partially parallelizable, which makes it attractive in a big data setting. In particular, we partition the data into groups, so that each group consists of multiple blocks of variables. By applying F-ADMM to this partitioning of the data, and using a specific regularization matrix, we obtain a hybrid ADMM (H-ADMM) algorithm: the grouped data is updated in a Gauss-Seidel fashion, and the blocks within each group are updated in a Jacobi (parallel) manner. Convergence of H-ADMM follows directly from the convergence properties of F-ADMM. Also, we describe a special case of H-ADMM that may be applied to functions that are convex, rather than strongly convex. Numerical experiments demonstrate the practical advantages of these new algorithms.

• 出版日期2017-4