Overview of performance values

The following statistics were calculated from the performance values of each algorithm:

obs (number of observations within performance values)
nas (number of NAs, i.e., missing values, within performance values)
min (minimum), mean (arithmetic mean), max (maximum), sd (standard deviation)
qu_1st (1st quartile = lower quartile = 25%-quantile)
med (median = 50%-quantile)
qu_3rd (3rd quartile = upper quartile = 75%-quantile)
coeff_var (coefficient of variation = standard deviation / arithmetic mean)

	obs	min	qu_1st	med	mean	qu_3rd	max	sd	coeff_var
clasp.2.1.3.h1.n1	1294	0.00758305	0.332115	4.67268	116.869	84.1796	600	212.621	1.81931
clasp.2.1.3.h10.n1	1294	0.00770103	0.262997	4.02998	144.648	162.111	600	240.413	1.66206
clasp.2.1.3.h11.n1	1294	0.00740399	0.301595	11.6867	204.771	600	600	265.671	1.2974
clasp.2.1.3.h2.n1	1294	0.00925794	0.28428	5.07186	146.476	167.974	600	239.663	1.63619
clasp.2.1.3.h3.n1	1294	0.00816606	1.82161	43.5837	213.154	600	600	256.555	1.20362
clasp.2.1.3.h4.n1	1294	0.00769212	0.496408	8.68188	143.092	171.907	600	226.561	1.58333
clasp.2.1.3.h5.n1	1294	0.00760604	0.349059	6.05358	136.994	119.925	600	231.776	1.69187
clasp.2.1.3.h6.n1	1294	0.00735496	0.251958	4.57641	134.845	139.78	600	229.828	1.70439
clasp.2.1.3.h7.n1	1294	0.0114029	0.497469	7.96087	172.087	444.938	600	252.539	1.46751
clasp.2.1.3.h8.n1	1294	0.00752191	0.573702	7.95723	139.512	174.695	600	223.784	1.60405
clasp.2.1.3.h9.n1	1294	0.00817802	0.471468	13.9671	166.92	295.037	600	243.42	1.45831

Summary of the runstatus per algorithm

The following table summarizes the runstatus of each algorithm over all instances (in %).

	ok	timeout
clasp.2.1.3.h1.n1	85.858	14.142
clasp.2.1.3.h10.n1	80.139	19.861
clasp.2.1.3.h11.n1	71.638	28.362
clasp.2.1.3.h2.n1	79.907	20.093
clasp.2.1.3.h3.n1	74.034	25.966
clasp.2.1.3.h4.n1	83.076	16.924
clasp.2.1.3.h5.n1	81.530	18.470
clasp.2.1.3.h6.n1	82.689	17.311
clasp.2.1.3.h7.n1	76.430	23.570
clasp.2.1.3.h8.n1	84.621	15.379
clasp.2.1.3.h9.n1	78.671	21.329

Dominated Algorithms

Here, you'll find an overview of dominating/dominated algorithms:
None of the algorithms was superior to any of the other.

An algorithm (A) is considered to be superior to an other algorithm (B), if it has at least an equal performance on all instances (compared to B) and if it is better on at least one of them. A missing value is automatically a worse performance. However, instances which could not be solved by either one of the algorithms, were not considered for the dominance relation.

Visualisations

Important note w.r.t. some of the following plots:
If appropriate, we imputed performance values for failed or censored runs. We used max + 0.3 * (max - min), in case of minimization problems, or min - 0.3 * (max - min), in case of maximization problems.
In addition, a small noise is added to the imputed values (except for the cluster matrix, based on correlations, which is shown at the end of this page).

Boxplots of performance values

Imputing the performance values of failed or censored runs (as described in the red note at the beginning of this section):

Discarding the performance values of failed or censored runs:

## Warning: Removed 2865 rows containing non-finite values (stat_boxplot).

Estimated densitities of performance values

Imputing the performance values of failed or censored runs (as described in the red note at the beginning of this section):

Discarding the performance values of failed or censored runs:

Estimated cumulative distribution functions of performance values

Imputing the performance values of failed runs (as described in the red note at the beginning of this section):

Discarding the performance values of failed or censored runs:

Scatterplot matrix of the performance values

The figure underneath shows pairwise scatterplots of the performance values.

Imputing the performance values of failed and censored runs (as described in the red note at the beginning of this section):

Clustering algorithms based on their correlations

The following figure shows the correlations of the ranks of the performance values. Per default it will show the correlation coefficient of spearman. Missing values were imputed prior to computing the correlation coefficients. The algorithms are ordered in a way that similar (highly correlated) algorithms are close to each other. Per default the clustering is based on hierarchical clustering, using Ward's method.