Optimize Selection


Synopsis

This operator realizes feature selection by forward selection and backward elimination, respectively.


Description

This operator realizes the two deterministic greedy feature selection algorithms forward selection and backward elimination. However, we added some enhancements to the standard algorithms which are described below:


Forward Selection

    * Create an initial population with n individuals where n is the input example set's number of attributes. Each individual will use exactly one of the features.
    • Evaluate the attribute sets and select only the best k.
    • For each of the k attribute sets do: If there are j unused attributes, make j copies of the attribute set and add exactly one of the previously unused attributes to the attribute set.
    • As long as the performance improved in the last p iterations go to 2
====Backward Elimination====
    * Start with an attribute set which uses all features.
    • Evaluate all attribute sets and select the best k.
    • For each of the k attribute sets do: If there are j attributes used, make j copies of the attribute set and remove exactly one of the previously used attributes from the attribute set.
    • As long as the performance improved in the last p iterations go to 2

The parameter k can be specified by the parameter keep_best, the parameter p can be specified by the parameter generations_without_improval. These parameters have default values 1 which means that the standard selection algorithms are used. Using other values increase the runtime but might help to avoid local extrema in the search for the global optimum.

Another unusual parameter is maximum_number_of_generations. This parameter bounds the number of iterations to this maximum of feature selections / deselections. In combination with generations_without_improval this allows several different selection schemes (which are described for forward selection, backward elimination works analogous):


Input


Output


Parameters


ExampleProcess