Synopsis
Creates batches from the input examples and performs its inner operators on each of these batches which might be useful for applying methods on very large data sets directly in databases.
Description
This operator groups the input examples into batches of the specified size and performs the inner operators on all batches subsequently. This might be useful for very large data sets which cannot be load into memory but must be handled in a database. In these cases, preprocessing methods or model applications and other tasks can be performed on each batch and the result might be again written into a database table (by using the DatabaseExampleSetWriter in its append mode). Note that the output of this operator is not composed of the results of the nested subprocess. In fact the subprocess does not need to deliver any output since it operates on a subset view of the input example set.
Input
- example set: expects: ExampleSetMetaData: #examples: = 0; #attributes: 0
Output
- example set:
Parameters
- batch size: This number of examples is processed batch-wise by the inner operators of this operator.