SCALING: Sparse Matrix Scalings

Major version history

2014-12-17 Version 1.0.0

Initial public release

4.1 Installation

Please see the SPRAL install documentation.

4.2 Usage overview

4.2.1 Calling sequences

Access to the package requires inclusion of either spral.h (for the entire SPRAL library) or spral_scaling.h (for just the relevant routines). i.e.

The following functions are available to the user:

• spral_scaling_default_auction_options() initializes the options structure for the auction algorithm.
• spral_scaling_default_equilib_options() initializes the options structure for the norm equilibriation algorithm.
• spral_scaling_default_hungarian_options() initializes the options structure for the Hungarian algorithm.
• spral_scaling_auction_sym() and spral_scaling_auction_unsym() generate approximate matching-based scalings for symmetric and unsymmetric/rectangular matrices respectively using an auction algorithm.
• spral_scaling_equilib_sym() and spral_scaling_equilib_unsym() generate norm-equilibration scalings for symmetric and unsymmetric/rectangular matrices respectively.
• spral_scaling_hungarian_sym() and spral_scaling_hungarian_unsym() generate matching-based scalings for a symmetric and unsymmetric/rectangular matrices respectively using the Hungarian algorithm.

4.2.2 Data formats

Compressed Sparse Column (CSC) Format

---

---

This standard data format consists of the following data:

Non-zero matrix entries are ordered by increasing column index and stored in the arrays row[] and val[] such that row[k] holds the row number and val[k] holds the value of the k-th entry. The ptr[] array stores column pointers such that ptr[i] is the position in row[] and val[] of the first entry in the i-th column, and ptr[n] is one more than the total number of entries. There must be no duplicate or out of range entries. Entries that are zero, including those on the diagonal, need not be specified.

For symmetric matrices, only the lower triangular entries of $A$ should be supplied. For unsymmetric matrices, all entries in the matrix should be supplied.

Note that these routines offer no checking of user data, and the behaviour of these routines with misformatted data is undefined.

To illustrate the CSC format, the following arrays describe the symmetric matrix shown in Figure 4.1.

4.3 Auction Algorithm

4.3.1 spral_scaling_auction_default_options()

To initialize a variable of type struct spral_scaling_auction_options the following routine is provided.

void spral_scaling_auction_default_options(struct spral_scaling_auction_options *options);

*options

is the instance to be initialized.

4.3.2 spral_scaling_auction_sym()

To generate a scaling for a symmetric matrix using an auction algorithm such that the entry of maximum absolute value in each row and column is approximately 1.0,

n, ptr[n+1], row[ptr[n]], val[ptr[n]]

must hold the lower triangular part of $A$ in compressed sparse column format as described in Section 4.2.2.

scaling[n]

holds, on exit, the diagonal of $D$. scaling[i] holds $d_i$, the scaling corresponding to the i-th row and column.

match[m]

may be NULL. If it is non-NULL, then on exit it specifies the matching of rows to columns. Row i is matched to column match[i], or is unmatched if match[i]$=$-1.

*options

specifies the algorithmic options used by the subroutine, as explained in Section 4.3.4.

*inform

is used to return information about the execution of the subroutine, as explained in Section 4.3.5.

4.3.3 spral_scaling_auction_unsym()

To generate a scaling for an unsymmetric or rectangular matrix using an auction algorithm such that the entry of maximum absolute value in each row and column is approximately 1.0,

m, n, ptr[n+1], row[ptr[n]], val[ptr[n]]

must hold the lower triangular part of $A$ in compressed sparse column format as described in Section 4.2.2.

rscaling[m]

holds, on exit, the diagonal of $D_r$. rscaling[i] holds $d_i^r$, the scaling corresponding to the i-th row.

cscaling[n]

holds, on exit, the diagonal of $D_c$. cscaling[j] holds $d_j^c$, the scaling corresponding to the j-th column.

match[m]

may be NULL. If it is non-NULL, then on exit it specifies the matching of rows to columns. Row i is matched to column match[i], or is unmatched if match[i]$=$-1.

*options

specifies the algorithmic options used by the subroutine, as explained in Section 4.3.4.

*inform

is used to return information about the execution of the subroutine, as explained in Section 4.3.5.

4.3.4 struct spral_scaling_auction_options

The structure spral_scaling_auction_options is used to specify the options used by the routines
spral_scaling_auction_sym() and spral_scaling_auction_unsym(). The components, that must be given default values through a call to spral_scaling_default_auction_options(), are:

int array_base

specifies the array indexing base. It must have the value either 0 (C indexing) or 1 (Fortran indexing). If array_base$=$1, the entries of arrays ptr[],row[] and match[] start at 1, not 0. Further, entries of match[] that are unmatched are indicated by a value of 0, not -1. The default value is array_base$=$0.

float eps_initial

specifies the initial value of the minimum improvement parameter $𝜖$ as described in Section 4.3.7.

int max_iterations

specifies the maximum number of iterations the algorithm may perform. The default is max_iterations$=$30000.

int max_unchanged[3]

specifies, together with min_proportion[], the termination conditions for the algorithm, as described in Section 4.3.7. The default is max_unchanged[]$=$ { 10, 100, 100 }.

float min_proportion[3]

specifies, together with max_unchanged(:), the termination conditions for the algorithm, as described in Section 4.3.7. The default is min_proportion[]$=$ { 0.90, 0.0, 0.0 }.

4.3.5 struct spral_scaling_auction_inform

The structure spral_scaling_auction_inform is used to hold parameters that give information about the progress of the routines spral_scaling_auction_sym() and spral_scaling_auction_unsym(). The components are:

int flag

gives the exit status of the algorithm (details in Section 4.3.6).

int iterations

holds the number of iterations performed.

int matched

holds the number of rows and columns that have been matched. As the algorithm may terminate before a full matching is obtained, this only provides a lower bound on the structural rank.

int stat

holds, in the event of an allocation error, the Fortran stat parameter if it is available (and is set to 0 otherwise).

int unmatchable

holds the number of columns designated as unmatchable. A column is designated as unmatchable if there is no way to match it that improves the quality of the matching. It provides an approximate lower bound on the structural rank deficiency.

4.3.6 Error Flags

A successful return from a routine is indicated by inform.flag having the value zero. A negative value is associated with an error message.

Possible negative (error) values are:

-1

Allocation error. If available, the Fortran stat parameter is returned in inform.stat.

4.3.7 Algorithm description

This algorithm finds a fast approximation to the matching and scaling produced by the HSL package MC64. If an optimal matching is required, use the Hungarian algorithm instead. The algorithm works by solving the following maximum product optimization problem using an auction algorithm. The scaling is derived from the dual variables associated with the solution.

$$\begin{array}{rcll}\underset{\sigma }{max}& \prod _{i=1}^m\prod _{j=1}^n\left|a_{ij}\right|{\sigma }_{ij}& & \\ s.t.& \sum _{i=1}^m{\sigma }_{ij}=1,& \forall j=1,n& \\ & \sum _{j=1}^n{\sigma }_{ij}=1,& \forall i=1,m& \\ & {\sigma }_{ij}\in \left\{0,1\right\}.& & \end{array}$$

The array $\sigma$ gives a matching of rows to columns.

By using the transformation

$$w_{ij}=log{c}_j-log\left|a_{ij}\right|,$$

where $c_j=\underset{i}{max}\left|a_{ij}\right|$, the maximum product problem in $a_{ij}$ is replaced by a minimum sum problem in $w_{ij}$ where all entries are positive. By standard optimization theory, there exist dual variables $u$ and $v$ corresponding to the constraints that satisfy the first order optimality conditions

$$\begin{array}{rcll}{w}_{ij}-u_i-v_j=0,& & \text{if }{\sigma }_{ij}=1,& \\ w_{ij}-u_i-v_j\ge 0,& & \text{if }{\sigma }_{ij}=0.& \end{array}$$

To obtain a scaling we define scaling matrices $D_r$ and $D_c$ as

$$\begin{array}{rcll}& d_i^r=e^{u_i},& & \\ & d_i^c=e^{v_i}.& & \end{array}$$

If a symmetric scaling is required, we average these as

$$d_i=\sqrt{d_i^r{d}_i^c}.$$

By the first order optimality conditions, these scaling matrices guarantee that

$$\begin{array}{rcll}{d}_i^r\left|a_{ij}\right|d_j^c=1,& & \text{if }{\sigma }_{ij}=1,& \\ d_i^r\left|a_{ij}\right|d_j^c\le 1,& & \text{if }{\sigma }_{ij}=0.& \end{array}$$

To solve the minimum sum problem an auction algorithm is used. The algorithm is not guaranteed to find an optimal matching. However it can find an approximate matching very quickly. A matching is maintained along with the row pricing vector $u$. In each major iteration, we loop over each column in turn. If the column $j$ is unmatched, we calculate the value $p_i=w_{ij}-u_i$ for each entry and find the maximum across the column. If this maximum is positive, the current matching can be improved by matching column $j$ with row $i$. This may mean that the previous match of row $i$ now becomes unmatched. We update the price of row $i$, that is $u_i$, to reflect this new benefit and continue to the next column.

To prevent incremental shuffling, we insist that the value of adding a new column is at least a threshold value $𝜖$ above zero, where $𝜖$ is based on the last iteration in which row $i$ changed its match. This is done by adding $𝜖$ to the price $u_i$, where $𝜖=\mathtt{\text{options.eps\_initial}}+\mathtt{\text{itr}}∕\left(n+1\right)$, where itr is the current iteration number.

The algorithm terminates if any of the following are satisfied:

• All entries are matched.
• The number of major iterations exceeds options.max_iterations.
• At least options.max_unchanged[0] iterations have passed without the cardinality of the matching increasing, and the proportion of matched columns is options.min_proportion[0].
• At least options.max_unchanged[1] iterations have passed without the cardinality of the matching increasing, and the proportion of matched columns is options.min_proportion[1].
• At least options.max_unchanged[2] iterations have passed without the cardinality of the matching increasing, and the proportion of matched columns is options.min_proportion[2].

The different combinations given by options.max_unchanged[0:2] and options.min_proportion[0:2] allow a wide range of termination heuristics to be specified by the user depending on their particular needs. Note that the matching and scaling produced will always be approximate as $𝜖$ is non-zero.

Further details are given in the following paper:

• J.D. Hogg and J.A. Scott. (2014). On the efficient scaling of sparse symmetric matrices using an auction algorithm. RAL Technical Report RAL-P-2014-002.

4.3.8 Example of spral_scaling_auction_sym()

The following code shows an example usage of spral_scaling_auction_sym().

The above code produces the following output.

4.4 Norm-equilibration algorithm

4.4.1 spral_scaling_equilib_default_options()

To initialize a variable of type struct spral_scaling_equilib_options the following routine is provided.

void spral_scaling_equilib_default_options(struct spral_scaling_equilib_options *options);

*options

is the instance to be initialized.

4.4.2 spral_scaling_equilib_sym()

To generate a scaling for a symmetric matrix using a norm equilibration algorithm such that the infinity norm of each row and column is equal to $1.0\pm \tau$,

n, ptr[n+1], row[ptr[n]], val[ptr[n]]

must hold the lower triangular part of $A$ in compressed sparse column format as described in Section 4.2.2.

scaling[n]

holds, on exit, the diagonal of $D$. scaling[i] holds $d_i$, the scaling corresponding to the i-th row and column.

*options

specifies the algorithmic options used by the subroutine, as explained in Section 4.4.4.

*inform

is used to return information about the execution of the subroutine, as explained in Section 4.4.5.

4.4.3 spral_scaling_equilib_unsym()

To generate a scaling for an unsymmetric or rectangular matrix using a norm equilibration algorithm such that the infinity norm of each row and column is equal to $1.0\pm \tau$,

m, n, ptr[n+1], row[ptr[n]], val[ptr[n]]

must hold the lower triangular part of $A$ in compressed sparse column format as described in Section 4.2.2.

rscaling[m]

holds, on exit, the diagonal of $D_r$. scaling[i] holds $d_i^r$, the scaling corresponding to the i-th row.

cscaling[n]

holds, on exit, the diagonal of $D_c$. scaling[j] holds $d_j^c$, the scaling corresponding to the j-th column.

*options

specifies the algorithmic options used by the subroutine, as explained in Section 4.4.4.

*inform

is used to return information about the execution of the subroutine, as explained in Section 4.4.5.

4.4.4 struct spral_scaling_equilib_options

The structure spral_scaling_equilib_options is used to specify the options used by the routines
spral_scaling_equilib_sym() and spral_scaling_equilib_unsym(). The components, that must be given default values through a call to spral_scaling_default_equilib_options(), are:

int array_base

specifies the array indexing base. It must have the value either 0 (C indexing) or 1 (Fortran indexing). If array_base$=$1, the entries of arrays ptr[] and row[] start at 1, not 0. The default value is array_base$=$0.

int max_iterations

specifies the maximum number of iterations the algorithm may perform. The default is max_iterations=10.

float tol

specifies the convergence tolerance for the algorithm (though often termination is based on max_iterations). The default is tol = 1e-8.

4.4.5 struct spral_scaling_equilib_inform

The structure spral_scaling_equilib_inform is used to hold parameters that give information about the progress of the routines spral_scaling_equilib_sym() and spral_scaling_equilib_unsym(). The components are:

int flag

gives the exit status of the algorithm (details in Section 4.4.6).

int iterations

holds, on exit, the number of iterations performed.

int stat

holds, in the event of an allocation error or deallocation error, the Fortran stat parameter if it is available (and is set to 0 otherwise).

4.4.6 Error Flags

A successful return from a routine is indicated by inform.flag having the value zero. A negative value is associated with an error message.

Possible negative (error) values are:

-1

Allocation error. If available, the Fortran stat parameter is returned in inform.stat.

4.4.7 Algorithm description

This algorithm is very similar to that used by the HSL routine MC77. An iterative method is used to scale the infinity norm of both rows and columns to 1 with an asymptotic linear rate of convergence of $\frac{1}{2}$, preserving symmetry if the matrix is symmetric.

For unsymmetric matrices, the algorithm outline is as follows:

For symmetric matrices, $A^{\left(k-1\right)}$ is symmetric, so $D_r^{\left(k\right)}=D_c^{\left(k\right)}$, and some operations can be skipped.

Further details are given in the following paper:

• P. Knight, D. Ruiz and B. Ucar. (2012). A symmetry preserving algorithm for matrix scaling. INRIA Research Report 7552.

4.4.8 Example of spral_scaling_equilib_sym()

The following code shows an example usage of spral_scaling_equilib_sym().

The above code produces the following output.

4.5 Hungarian algorithm

4.5.1 spral_scaling_hungarian_default_options()

To initialize a variable of type struct spral_scaling_hungarian_options the following routine is provided.

void spral_scaling_hungarian_default_options(struct spral_scaling_hungarian_options *options);

*options

is the instance to be initialized.

4.5.2 spral_scaling_hungarian_sym()

To generate a scaling for a symmetric matrix using the Hungarian algorithm such that the entry of maximum absolute value in each row and column is 1.0,

n, ptr[n+1], row[ptr[n]], val[ptr[n]]

must hold the lower triangular part of $A$ in compressed sparse column format as described in Section 4.2.2.

scaling[n]

holds, on exit, the diagonal of $D$. scaling[i] holds $D_{ii}$, the scaling corresponding to the i-th row and column.

match[m]

may be NULL. If it is non-NULL, then on exit it specifies the matching of rows to columns. Row i is matched to column match[i], or is unmatched if match[i]$=$-1.

*options

specifies the algorithmic options used by the subroutine, as explained in Section 4.5.4.

*inform

is used to return information about the execution of the subroutine, as explained in Section 4.5.5.

4.5.3 spral_scaling_hungarian_unsym()

To generate a scaling for an unsymmetric or rectangular matrix using the Hungarian algorithm such that the entry of maximum absolute value in each row and column is 1.0,

m, n, ptr[n+1], row[ptr[n]], val[ptr[n]]

must hold the lower triangular part of $A$ in compressed sparse column format as described in Section 4.2.2.

rscaling[m]

holds, on exit, the diagonal of $D_r$. rscaling[i] holds $d_i^r$, the scaling corresponding to the i-th row.

cscaling[n]

holds, on exit, the diagonal of $D_c$. cscaling[j] holds $d_j^c$, the scaling corresponding to the j-th column.

match[m]

may be NULL. If it is non-NULL, then on exit it specifies the matching of rows to columns. Row i is matched to column match[i], or is unmatched if match[i]$=$-1.

*options

specifies the algorithmic options used by the subroutine, as explained in Section 4.5.4.

*inform

is used to return information about the execution of the subroutine, as explained in Section 4.5.5.

4.5.4 struct spral_scaling_hungarian_options

The structure spral_scaling_hungarian_options is used to specify the options used by the routines
spral_scaling_hungarian_sym() and spral_scaling_hungarian_unsym(). The components, that must be given default values through a call to spral_scaling_default_hungarian_options(), are:

int array_base

specifies the array indexing base. It must have the value either 0 (C indexing) or 1 (Fortran indexing). If array_base$=$1, the entries of arrays ptr[],row[] and match[] start at 1, not 0. Further, entries of match[] that are unmatched are indicated by a value of 0, not -1. The default value is array_base$=$0.

bool scale_if_singular

specifies whether scaling shuold continue if the matrix $A$ is found to be structurally singular. If scale_if_singular$=$true, and the $A$ is structurally singular, a partial scaling corresponding to a maximum cardinality matching will be returned and a warning issued. Otherwise an identity scaling will be returned and an error issued.

4.5.5 struct spral_scaling_hungarian_inform

The structure spral_scaling_hungarian_inform is used to hold parameters that give information about the progress of the routines spral_scaling_hungarian_sym() and spral_scaling_hungarian_unsym(). The components are:

int flag

gives the exit status of the algorithm (details in Section 4.5.6).

int matched

holds the number of rows and columns that have been matched (i.e. the structural rank).

int stat

holds, in the event of an allocation error or deallocation error, the Fortran stat parameter if it is available (and is set to 0 otherwise).

4.5.6 Error Flags

A successful return from a routine is indicated by inform.flag having the value zero. A negative value is associated with an error message and a positive value with a warning.

Possible negative (error) values are:

-1

Allocation error. If available, the Fortran stat parameter is returned in inform.stat.

-2

Matrix $A$ is structurally rank-deficient. This error is returned only if options.scale_if_singular$=$false. The scaling vector is set to 1.0 and a matching of maximum cardinality returned in the optional argument match[], if present.

Possible positive (warning) values are:

+1

Matrix $A$ is structurally rank-deficient. This warning is returned only if options.scale_if_singular$=$true.

4.5.7 Algorithm description

This algorithm is the same as used by the HSL package MC64. A scaling is derived from dual variables found during the solution of the below maximum product optimization problem using the Hungarian algorithm.

$$\begin{array}{rcll}\underset{\sigma }{max}& \prod _{i=1}^m\prod _{j=1}^n\left|a_{ij}\right|{\sigma }_{ij}& & \\ s.t.& \sum _{i=1}^m{\sigma }_{ij}=1,& \forall j=1,n& \\ & \sum _{j=1}^n{\sigma }_{ij}=1,& \forall i=1,m& \\ & {\sigma }_{ij}\in \left\{0,1\right\}.& & \end{array}$$

The array $\sigma$ gives a matching of rows to columns.

By using the transformation

$$w_{ij}=log{c}_j-log\left|a_{ij}\right|,$$

where $c_j=\underset{i}{max}\left|a_{ij}\right|$, the maximum product problem in $a_{ij}$ is replaced by a minimum sum problem in $w_{ij}$ where all entries are positive. By standard optimization theory, there exist dual variables $u$ and $v$ corresponding to the constraints that satisfy the first order optimality conditions

$$\begin{array}{rcll}{w}_{ij}-u_i-v_j=0,& & \text{if }{\sigma }_{ij}=1,& \\ w_{ij}-u_i-v_j\ge 0,& & \text{if }{\sigma }_{ij}=0.& \end{array}$$

To obtain a scaling we define scaling matrices $D_r$ and $D_c$ as

$$\begin{array}{rcll}& d_i^r=e^{u_i},& & \\ & d_i^c=e^{v_i}.& & \end{array}$$

If a symmetric scaling is required, we average these as

$$d_i=\sqrt{d_i^r{d}_i^c}.$$

By the first order optimality conditions, these scaling matrices guarantee that

$$\begin{array}{rcll}{d}_i^r\left|a_{ij}\right|d_j^c=1,& & \text{if }{\sigma }_{ij}=1,& \\ d_i^r\left|a_{ij}\right|d_j^c\le 1,& & \text{if }{\sigma }_{ij}=0.& \end{array}$$

To solve the minimum sum problem, the Hungarian algorithm maintains an optimal matching on a subset of the rows and columns. It proceeds to grow this set by finding augmenting paths from an unmatched row to an unmatched column. The algorithm is guaranteed to find the optimal solution in a fixed number of steps, but can be very slow as it may need to explore the full matrix a number of times equal to the dimension of the matrix. To minimize the solution time, a warmstarting heuristic is used to construct an initial optimal subset matching.

Further details are given in the following paper:

• I.S. Duff and J. Koster. (1997). The design and use of algorithms for permuting large entries to the diagonal of sparse matrices. SIAM J. Matrix Anal. Applics. 20(4), pp 889–901.

4.5.8 Example usage of spral_scaling_hungarian_unsym()

The following code shows an example usage of hungarian_scale_unsym().

The above code produces the following output.