This repository was archived by the owner on Nov 11, 2022. It is now read-only.
Version 0.4.20150710
Pre-release
Pre-release
·
1637 commits
to master
since this release
- Added support for per-window tables to
BigQueryIO. - Added support for a custom source implementation for Avro. See
AvroSourcefor more details. - Removed 250GiB Google Cloud Storage file size upload restriction.
- Fixed
BigQueryIO.Writetable creation bug in streaming mode. - Changed
Source.createReader()andBoundedSource.createReader()to be abstract. - Moved
Source.splitIntoBundles()toBoundedSource.splitIntoBundles(). - Added support for reading bounded views of a PubSub stream in
PubsubIOfor non-streaming Dataflow pipeline runners andDirectPipelineRunner. - Added support for getting a
Coderusing aClassto theCoderRegistry. - Changed
CoderRegistry.registerCoder(Class<T>, Coder<T>)to enforce that the provided coder actually encodes values of the given class, and its use with raw types of generic classes is forbidden as it will rarely work correctly. - Migrate to
Create.withCoder()andCreateTimestamped.withCoder()instead of callingsetCoder()on the outcomingPCollectionwhen theCreatePTransformis being applied. - Added three successively more detailed
WordCountexamples. - Removed
PTransform.getDefaultName()which was redundant withPTransform.getKindString(). - Added support a unique name check for PTransform's during job creation.
- Removed
PTransform.withName()andPTransform.setName(). The name of a transform is now immutable after construction. Library transforms (likeCombine) can provide builder-like methods to change the name. Names can always be overridden at the location where the transform is applied usingapply("name", transform). - Added the ability to select the network for worker VMs using
DataflowPipelineWorkerPoolOptions.setNetwork(String).