HERE Data SDK for Java & Scala 2.19.0

2.19.0.4

Examples

Added

• A new compaction library has been added to the Index storage toolset.
  Continually indexing data from small stream messages leads to the creation of metadata references to many tiny files.
  This grows the size of Index storage and adversely impacts query performance.
  With the release of this library, you can compact your Index layer metadata and data to greatly reduce your index storage size, improve query performance, and optimize big data processing by working with larger files.
  Furthermore, you can use RSQL queries to submit criteria to compact your data within certain time windows, or by other attributes you've defined to optimize the compaction process, all while you continue to index incoming data.

There are two new batch examples that show how to use the Index Compaction Library to quickly develop a compaction application that compacts parquet and protobuf format data.

Changed

• The Data Archiving Library now enables you to define Flink metrics to be reported in your Flink processing Grafana dashboard
  so that you can monitor your data archive stream processing health and activity.
  See more here
  about which Data Archiving Library metrics are available to review via custom dashboards in Grafana.

The data-archive-avro-example is changed to show how you can add user defined metrics which will be available to analyse in grafana.

Deprecated

• The merge API of the Data Archiving Library used in the data-archive examples is no longer recommended to compact files for optimization.
  Use the Index Compaction Library instead.