This repository was archived by the owner on Nov 11, 2022. It is now read-only.
Version 0.4.20150414
Pre-release
Pre-release
·
1881 commits
to master
since this release
- Initial Beta release of the Dataflow SDK for Java.
- Improved execution performance in many areas of the system.
- Added support for progress estimation and dynamic work rebalancing for user-defined sources.
- Added support for user-defined sources to provide the timestamp of the values read via
Reader.getCurrentTimestamp(). - Added support for user-defined sinks.
- Added support for custom types in
PubsubIO. - Added support for reading and writing XML files. See
XmlSourceandXmlSink. - Renamed
DatastoreIO.Write.totoDatastoreIO.writeTo. In addition, entities written to Cloud Datastore must have complete keys. - Renamed
ReadSourcetransform intoRead. - Replaced
Source.createBasicReaderwithSource.createReader. - Added support for triggers, which allows getting early or partial results for a window, and specifying when to process late data. See
Window.into.triggering. - Reduced visibility of
PTransform'sgetInput(),getOutput(),getPipeline(), andgetCoderRegistry(). These methods will soon be deleted. - Renamed
DoFn.ProcessContext#windowstoDoFn.ProcessContext#window. In order for aDoFnto callDoFn.ProcessContext#window, it must implementRequiresWindowAccess. - Added
DoFn.ProcessContext#windowingInternalsto enable windowing on third-party runners. - Added support for side inputs when running streaming pipelines on the
[Blocking]DataflowPipelineRunner. - Changed
[Keyed]CombineFn.addInput()to return the new accumulator value. RenamedCombine.perElement().withHotKeys()toCombine.perElement().withHotKeyFanout(). - Renamed
First.oftoSample.anyandRateLimitingtoIntraBundleParallelizationto better represent its functionality.