Skip to content

Fix date and ip sources in the composite aggregation #29370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 9, 2018

Conversation

jimczi
Copy link
Contributor

@jimczi jimczi commented Apr 4, 2018

This commit fixes the formatting of the values in the composite aggregation response.
date fields should return timestamp as longs when used in a terms source and ip fields should always be formatted as strings.
This commit also fixes the parsing of the after key for these field types.
Finally, this change disables the index optimization for the ip field and any source that provides a missing value.

This commit fixes the formatting of the values in the composite
aggregation response. `date` fields should return timestamp as longs
when used in a `terms` source and `ip` fields should always
be formatted as strings.
This commit also fixes the parsing of the `after` key for these field
types.
Finally, this commit disables the index optimization for the
`ip` field and any source that provides a `missing` value.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

Copy link
Contributor

@jpountz jpountz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general, I'm just curious why we still use the RAW format with dates in some cases.

final DocValueFormat format;
if (format() == null && fieldType instanceof DateFieldMapper.DateFieldType) {
// defaults to the raw format on date fields (preserve timestamp as longs).
format = DocValueFormat.RAW;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why shouldn't we use the format from the mapping?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. In 6.2 we always return the raw value and there is no distinction in this aggregation to return getKey or getKeyAsString since the returned object is a map. So I think it's preferable to return the raw value unless a format has been specified.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. for the record we'll probably need to change this anyway one day for #18518

@jimczi jimczi merged commit d755fcf into elastic:master Apr 9, 2018
@jimczi jimczi deleted the composite_agg_source_type branch April 9, 2018 08:49
jimczi added a commit that referenced this pull request Apr 9, 2018
This commit fixes the formatting of the values in the composite
aggregation response. `date` fields should return timestamp as longs
when used in a `terms` source and `ip` fields should always be formatted as strings.
This commit also fixes the parsing of the `after` key for these field types.
Finally, this commit disables the index optimization for the `ip` field and any source that provides a `missing` value.
@JunZhou2016
Copy link

This is good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants