Overview of performance values

The following statistics were calculated from the performance values of each algorithm:

obs (number of observations within performance values)
nas (number of NAs, i.e., missing values, within performance values)
min (minimum), mean (arithmetic mean), max (maximum), sd (standard deviation)
qu_1st (1st quartile = lower quartile = 25%-quantile)
med (median = 50%-quantile)
qu_3rd (3rd quartile = upper quartile = 75%-quantile)
coeff_var (coefficient of variation = standard deviation / arithmetic mean)

	obs	min	qu_1st	med	mean	qu_3rd	max	sd	coeff_var
X2361_weka.OneR	105	0.036898	0.414	0.704942	0.647421	0.899552	1	0.274286	0.423659
X2362_weka.J48	105	0.188206	0.760131	0.871396	0.819622	0.953968	1	0.190278	0.232153
X2364_weka.IBk	105	0.138	0.7322	0.88838	0.793618	0.963	1	0.233432	0.294137
X2367_weka.REPTree	105	0.138889	0.730469	0.855072	0.802228	0.952601	1	0.201269	0.250887
X2368_weka.RandomTree	105	0.126667	0.6745	0.832844	0.762269	0.916505	0.999825	0.20923	0.274484
X2369_weka.RandomForest	105	0.195734	0.796321	0.930219	0.853415	0.97351	1	0.177608	0.208114
X2370_weka.LMT	105	0.212045	0.818	0.91009	0.855673	0.971638	1	0.176554	0.206333
X2373_weka.JRip	105	0.183864	0.746094	0.86095	0.813124	0.948801	1	0.190539	0.23433
X2381_weka.NaiveBayes	105	0.173913	0.689655	0.8008	0.760041	0.908738	1	0.198041	0.260567
X2647_weka.Logistic	105	0.166	0.749	0.859547	0.81341	0.946325	1	0.186974	0.229865
X2869_weka.SMO_PolyKernel	105	0.2	0.758717	0.867483	0.82192	0.965049	1	0.194051	0.236095
X2891_weka.HyperPipes	105	0.045008	0.446377	0.683461	0.63796	0.877915	0.998449	0.266015	0.416978
X2893_weka.OLM	105	0.009381	0.128155	0.4054	0.423398	0.658662	0.953704	0.287687	0.679473
X2897_weka.ConjunctiveRule	105	0.018136	0.298999	0.700231	0.598107	0.855072	0.97667	0.294418	0.49225
X2898_weka.SimpleCart	105	0.138889	0.757941	0.87225	0.818857	0.957033	1	0.193076	0.235787
X2900_weka.LADTree	105	0.186951	0.68236	0.831415	0.783457	0.948589	1	0.198358	0.253183
X2903_weka.AdaBoostM1_DecisionStump	105	0.011875	0.300787	0.736	0.620457	0.899804	0.992	0.309975	0.499591
X2906_weka.Bagging_REPTree	105	0.138889	0.789123	0.893468	0.828509	0.959656	1	0.192743	0.232639
X6250_weka.DecisionTable	105	0.200753	0.65625	0.833884	0.769882	0.932067	1	0.206808	0.268624
X6352_weka.BayesNet	105	0.204517	0.711695	0.840936	0.79034	0.930963	1	0.184037	0.232858
X6378_weka.LogitBoost_DecisionStump	105	0.212045	0.746403	0.870389	0.820406	0.953968	1	0.184494	0.224882

Summary of the runstatus per algorithm

The following table summarizes the runstatus of each algorithm over all instances (in %).

	ok	timeout	memout	not_applicable	crash	other
X2361_weka.OneR	100.000	0.000	0.000	0.000	0.000	0.000
X2362_weka.J48	100.000	0.000	0.000	0.000	0.000	0.000
X2364_weka.IBk	100.000	0.000	0.000	0.000	0.000	0.000
X2367_weka.REPTree	100.000	0.000	0.000	0.000	0.000	0.000
X2368_weka.RandomTree	100.000	0.000	0.000	0.000	0.000	0.000
X2369_weka.RandomForest	100.000	0.000	0.000	0.000	0.000	0.000
X2370_weka.LMT	100.000	0.000	0.000	0.000	0.000	0.000
X2371_weka.HoeffdingTree	100.000	0.000	0.000	0.000	0.000	0.000
X2373_weka.JRip	100.000	0.000	0.000	0.000	0.000	0.000
X2381_weka.NaiveBayes	100.000	0.000	0.000	0.000	0.000	0.000
X2647_weka.Logistic	100.000	0.000	0.000	0.000	0.000	0.000
X2869_weka.SMO_PolyKernel	100.000	0.000	0.000	0.000	0.000	0.000
X2882_weka.SMO_RBFKernel	100.000	0.000	0.000	0.000	0.000	0.000
X2889_weka.IBk	100.000	0.000	0.000	0.000	0.000	0.000
X2891_weka.HyperPipes	100.000	0.000	0.000	0.000	0.000	0.000
X2893_weka.OLM	100.000	0.000	0.000	0.000	0.000	0.000
X2894_weka.FURIA	100.000	0.000	0.000	0.000	0.000	0.000
X2897_weka.ConjunctiveRule	100.000	0.000	0.000	0.000	0.000	0.000
X2898_weka.SimpleCart	100.000	0.000	0.000	0.000	0.000	0.000
X2900_weka.LADTree	100.000	0.000	0.000	0.000	0.000	0.000
X2903_weka.AdaBoostM1_DecisionStump	100.000	0.000	0.000	0.000	0.000	0.000
X2904_weka.AdaBoostM1_J48	100.000	0.000	0.000	0.000	0.000	0.000
X2906_weka.Bagging_REPTree	100.000	0.000	0.000	0.000	0.000	0.000
X6250_weka.DecisionTable	100.000	0.000	0.000	0.000	0.000	0.000
X6352_weka.BayesNet	100.000	0.000	0.000	0.000	0.000	0.000
X6355_weka.AdaBoostM1_NaiveBayes	100.000	0.000	0.000	0.000	0.000	0.000
X6378_weka.LogitBoost_DecisionStump	100.000	0.000	0.000	0.000	0.000	0.000
X8990_weka.MultilayerPerceptron	100.000	0.000	0.000	0.000	0.000	0.000
X8994_weka.MultilayerPerceptron	100.000	0.000	0.000	0.000	0.000	0.000
X8995_weka.MultilayerPerceptron	100.000	0.000	0.000	0.000	0.000	0.000

Dominated Algorithms

Here, you'll find an overview of dominating/dominated algorithms:
None of the algorithms was superior to any of the other.

An algorithm (A) is considered to be superior to an other algorithm (B), if it has at least an equal performance on all instances (compared to B) and if it is better on at least one of them. A missing value is automatically a worse performance. However, instances which could not be solved by either one of the algorithms, were not considered for the dominance relation.

Visualisations

Important note w.r.t. some of the following plots:
If appropriate, we imputed performance values for failed or censored runs. We used max + 0.3 * (max - min), in case of minimization problems, or min - 0.3 * (max - min), in case of maximization problems.
In addition, a small noise is added to the imputed values (except for the cluster matrix, based on correlations, which is shown at the end of this page).

Boxplots of performance values

Imputing the performance values of failed or censored runs (as described in the red note at the beginning of this section):

Discarding the performance values of failed or censored runs:

Estimated densitities of performance values

Imputing the performance values of failed or censored runs (as described in the red note at the beginning of this section):

## Error in seq.default(min, max, by = by): 'from' must be a finite number

Discarding the performance values of failed or censored runs:

## Error in seq.default(min, max, by = by): 'from' must be a finite number

Estimated cumulative distribution functions of performance values

Imputing the performance values of failed runs (as described in the red note at the beginning of this section):

## Error in seq.default(min, max, by = by): 'from' must be a finite number

Discarding the performance values of failed or censored runs:

## Error in seq.default(min, max, by = by): 'from' must be a finite number

Scatterplot matrix of the performance values

The figure underneath shows pairwise scatterplots of the performance values.

Imputing the performance values of failed and censored runs (as described in the red note at the beginning of this section):

Clustering algorithms based on their correlations

The following figure shows the correlations of the ranks of the performance values. Per default it will show the correlation coefficient of spearman. Missing values were imputed prior to computing the correlation coefficients. The algorithms are ordered in a way that similar (highly correlated) algorithms are close to each other. Per default the clustering is based on hierarchical clustering, using Ward's method.