`merge-batch-stream `: Cannot configure option `batchSize` in flux

[TobiasNx]

merge-batch-stream should have an option batchsize

metafacture-core/metafacture-plumbing/src/main/java/org/metafacture/plumbing/StreamBatchMerger.java

Lines 27 to 29 in 3281f1a

	* Merges a sequence of {@link #setBatchSize(long)} records. On a
	* <i>close-stream</i> event, a record containing fewer source records may be
	* created.

metafacture-core/metafacture-plumbing/src/main/java/org/metafacture/plumbing/StreamBatchMerger.java

Lines 54 to 65 in 3281f1a

	/**
	* Sets the number of records that should be merged into a batch.
	* <p>
	* The default batch size is 1, wich means that no records are merged.
	* <p>
	* This parameter must not be changed during processing.
	*
	* @param batchSize the number of records that should be merged.
	*/
	public void setBatchSize(final long batchSize) {
	this.batchSize = batchSize;
	}

When trying to use it in a Flux, an exception is thrown:

"https://raw.githubusercontent.com/metafacture/metafacture-core/master/metafacture-runner/src/main/dist/examples/read/pica/10.pica"
| open-http
| as-lines
| decode-pica
| fix("retain('003@.a')")
| merge-batch-stream(batchSize="10")
| encode-formeta(style="multiline")
| print
;

org.metafacture.commons.reflection.ReflectionException: Method StreamBatchMerger.batchsize does not exist

Playground-Link

[blackwinter]

Maybe we should finally add support for more/all primitive data types in ConfigurableClass.convertValue(). Currently, only boolean and int (as well as enum and, of course, String) are supported. Which leaves:

• byte
• short
• long <--- This issue!
• float
• double
• char

Not sure what the repercussions would be. At least one obstacle might be overloaded methods, because as it is implemented now, only one method is associated with any setter name. So there could be inefficient or even wrong conversions (e.g., in order to convert the input value to a char, typically the first character in the string is extracted; but there may have just as well been a String setter accepting the whole value). We would probably have to change the setter logic to select the "smallest" type that fits the whole value, or something like that. And even then we're not completely safe from different setters applying different behaviour to their respective values.

[TobiasNx]

Maybe a simpler solution would be change batchSize to int as it is used in all other scenarios like:

metafacture-core/metafacture-monitoring/src/main/java/org/metafacture/monitoring/ObjectBatchLogger.java

Line 97 in 682da89

public void setBatchSize(final int batchSize) {

[blackwinter]

Absolutely. I was just thinking that it might be time for a generic solution since this issue has come up before (e.g. in ObjectSleeper, see 18b6ae0).