Synopsis
Generates a ROC chart for the models created by each of the inner learners and plot all charts in the same plotter.
Description
This operator uses its inner operators (each of those must produce a model) and calculates the ROC curve for each of them. All ROC curves together are plotted in the same plotter. The comparison is based on the average values of a k-fold cross validation. Alternatively, this operator can use an internal split into a test and a training set from the given data set. Please note that a former predicted label of the given example set will be removed during the application of this operator.
Input
- example set: expects: ExampleSet
Output
- exampleSet:
- rocComparison:
Parameters
- number of folds: The number of folds used for a cross validation evaluation (-1: use simple split ratio).
- split ratio: Relative size of the training set
- sampling type: Defines the sampling type of the cross validation (linear = consecutive subsets, shuffled = random subsets, stratified = random subsets with class distribution kept constant)
- use local random seed: Indicates if a local random seed should be used.
- local random seed: Specifies the local random seed
- use example weights: Indicates if example weights should be regarded (use weight 1 for each example otherwise).
- roc bias: Determines how the ROC (and AUC) are evaluated: Count correct predictions first, last, or alternatingly