Suboptimality bounds for trace-bounded SDPs enable a faster and scalable low-rank SDP solver SDPLR+ (arxiv.org)
arXiv:2406.10407v3 Announce Type: replace-cross
Abstract: Semidefinite programs (SDPs) and their solvers are powerful tools with many applications in machine learning and data science. Designing scalable SDP solvers is challenging because by standard the positive semidefinite decision variable is an $n \times n$ dense matrix, even though the input is often an $n \times n$ sparse matrix. However, the solution may not require a full-rank matrix, as shown by Barvinok and Pataki. Two decades ago, Burer and Monteiro developed an SDP solver \texttt{SDPLR} that optimizes over a low-rank factorization instead of the full matrix. This greatly decreases the storage cost and works well for many problems. The original solver \texttt{SDPLR} tracks only the primal infeasibility of the solution, preventing early termination at moderate accuracy. We use a suboptimality bound for trace-bounded SDP problems that enables us to track the progress better and perform early termination. We then develop \texttt{SDPLR+}, which starts the optimization with an extremely low-rank factorization and dynamically updates the rank based on the primal infeasibility and suboptimality. This further speeds up the computation and saves storage. Numerical comparisons on Max Cut, Minimum Bisection, Cut Norm, and Lov\'{a}sz Theta problems with many recent memory-efficient scalable SDP solvers demonstrate the scalability of \texttt{SDPLR+} up to problems with million-by-million decision variables. It is often the fastest solver to a moderate accuracy of $10^{-2}$. Further experiments on $\mu$-conductance, matrix completion, and $k$-means clustering show the potential of \texttt{SDPLR+} on a broader range of data science applications.
Abstract: Semidefinite programs (SDPs) and their solvers are powerful tools with many applications in machine learning and data science. Designing scalable SDP solvers is challenging because by standard the positive semidefinite decision variable is an $n \times n$ dense matrix, even though the input is often an $n \times n$ sparse matrix. However, the solution may not require a full-rank matrix, as shown by Barvinok and Pataki. Two decades ago, Burer and Monteiro developed an SDP solver \texttt{SDPLR} that optimizes over a low-rank factorization instead of the full matrix. This greatly decreases the storage cost and works well for many problems. The original solver \texttt{SDPLR} tracks only the primal infeasibility of the solution, preventing early termination at moderate accuracy. We use a suboptimality bound for trace-bounded SDP problems that enables us to track the progress better and perform early termination. We then develop \texttt{SDPLR+}, which starts the optimization with an extremely low-rank factorization and dynamically updates the rank based on the primal infeasibility and suboptimality. This further speeds up the computation and saves storage. Numerical comparisons on Max Cut, Minimum Bisection, Cut Norm, and Lov\'{a}sz Theta problems with many recent memory-efficient scalable SDP solvers demonstrate the scalability of \texttt{SDPLR+} up to problems with million-by-million decision variables. It is often the fastest solver to a moderate accuracy of $10^{-2}$. Further experiments on $\mu$-conductance, matrix completion, and $k$-means clustering show the potential of \texttt{SDPLR+} on a broader range of data science applications.
Comments