Synopsis
This operator realizes feature selection by forward selection and backward elimination, respectively.
Description
This operator realizes the two deterministic greedy feature selection algorithms forward selection and backward elimination. However, we added some enhancements to the standard algorithms which are described below:
Forward Selection
- * Create an initial population with n individuals where n is the input example set's number of attributes. Each individual will use exactly one of the features.
- Evaluate the attribute sets and select only the best k.
- For each of the k attribute sets do: If there are j unused attributes, make j copies of the attribute set and add exactly one of the previously unused attributes to the attribute set.
- As long as the performance improved in the last p iterations go to 2
- * Start with an attribute set which uses all features.
- Evaluate all attribute sets and select the best k.
- For each of the k attribute sets do: If there are j attributes used, make j copies of the attribute set and remove exactly one of the previously used attributes from the attribute set.
- As long as the performance improved in the last p iterations go to 2
The parameter k can be specified by the parameter keep_best
, the parameter p can be specified by the parameter generations_without_improval
. These parameters have default values 1 which means that the standard selection algorithms are used. Using other values increase the runtime but might help to avoid local extrema in the search for the global optimum.
Another unusual parameter is maximum_number_of_generations
. This parameter bounds the number of iterations to this maximum of feature selections / deselections. In combination with generations_without_improval
this allows several different selection schemes (which are described for forward selection, backward elimination works analogous):
-
maximum_number_of_generations
= m andgenerations_without_improval
= p: Selects maximal m features. The selection stops if not performance improvement was measured in the last p generations. -
maximum_number_of_generations
= -1 andgenerations_without_improval
= p: Tries to selects new features until no performance improvement was measured in the last p generations. -
maximum_number_of_generations
= m andgenerations_without_improval
= -1: Selects maximal m features. The selection stops is not stopped until all combinations with maximal m were tried. However, the result might contain less features than these. -
maximum_number_of_generations
= -1 andgenerations_without_improval
= -1: Test all combinations of attributes (brute force, this might take a very long time and should only be applied to small attribute sets).
Input
- example set in: expects: ExampleSetMetaData: #examples: = 0; #attributes: 0
- through 1:
Output
- example set out:
- weights:
- performance:
Parameters
- selection direction: Forward selection or backward elimination.
- limit generations without improval: Indicates if the optimization should be aborted if this number of generations showed no improvement. If unchecked, always the maximal number of generations will be used.
- generations without improval: Stop after n generations without improval of the performance.
- limit number of generations: Defines if the number of generations should be limited on a specific number.
- keep best: Keep the best n individuals in each generation.
- maximum number of generations: Defines the maximum amount of generations.
- normalize weights: Indicates if the final weights should be normalized.
- use local random seed: Indicates if a local random seed should be used.
- local random seed: Specifies the local random seed
- show stop dialog: Determines if a dialog with a button should be displayed which stops the run: the best individual is returned.
- user result individual selection: Determines if the user wants to select the final result individual from the last population.
- show population plotter: Determines if the current population should be displayed in performance space.
- plot generations: Update the population plotter in these generations.
- constraint draw range: Determines if the draw range of the population plotter should be constrained between 0 and 1.
- draw dominated points: Determines if only points which are not Pareto dominated should be painted.
- population criteria data file: The path to the file in which the criteria data of the final population should be saved.
- maximal fitness: The optimization will stop if the fitness reaches the defined maximum.