Skip to content

Commit 41afe85

Browse files
authored
Xcontent filter: Add map and builder filter benchmark (#81575)
I add the map and builder in the `FilterContentBenchmark`. The result show that parser performance is much better than map and builder. Later my colleague and I will commit some PR, that It will replace from map and builder to parser. In wildcard case, map filter is so slow: ``` Benchmark (fieldCount) (inclusive) (type) Mode Cnt Score Error Units FilterContentBenchmark.filterWithBuilder 10_wildcard_field true cluster_stats avgt 2 229729.345 ns/op FilterContentBenchmark.filterWithMap 10_wildcard_field true cluster_stats avgt 2 29668878.146 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_wildcard_field true cluster_stats avgt 2 237100.911 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_wildcard_field true cluster_stats avgt 2 221254.827 ns/op ``` and parse filter has a little improvement with builder filter: ``` Benchmark (fieldCount) (inclusive) (type) Mode Cnt Score Error Units FilterContentBenchmark.filterWithBuilder 10_field true cluster_stats avgt 2 170686.503 ns/op FilterContentBenchmark.filterWithMap 10_field true cluster_stats avgt 2 492973.006 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_field true cluster_stats avgt 2 142003.739 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_field true cluster_stats avgt 2 127459.484 ns/op ``` Here is the full benchmark result: ``` Benchmark (fieldCount) (inclusive) (type) Mode Cnt Score Error Units FilterContentBenchmark.filterWithBuilder 10_field true cluster_stats avgt 2 170686.503 ns/op FilterContentBenchmark.filterWithBuilder 10_field true index_stats avgt 2 19893.310 ns/op FilterContentBenchmark.filterWithBuilder 10_field true node_stats avgt 2 26955.237 ns/op FilterContentBenchmark.filterWithBuilder half_field true cluster_stats avgt 2 399922.076 ns/op FilterContentBenchmark.filterWithBuilder half_field true index_stats avgt 2 34104.100 ns/op FilterContentBenchmark.filterWithBuilder half_field true node_stats avgt 2 45339.887 ns/op FilterContentBenchmark.filterWithBuilder all_field true cluster_stats avgt 2 541790.127 ns/op FilterContentBenchmark.filterWithBuilder all_field true index_stats avgt 2 50051.251 ns/op FilterContentBenchmark.filterWithBuilder all_field true node_stats avgt 2 66082.753 ns/op FilterContentBenchmark.filterWithBuilder wildcard_field true cluster_stats avgt 2 207682.012 ns/op FilterContentBenchmark.filterWithBuilder wildcard_field true index_stats avgt 2 20100.143 ns/op FilterContentBenchmark.filterWithBuilder wildcard_field true node_stats avgt 2 26076.856 ns/op FilterContentBenchmark.filterWithBuilder 10_wildcard_field true cluster_stats avgt 2 229729.345 ns/op FilterContentBenchmark.filterWithBuilder 10_wildcard_field true index_stats avgt 2 28191.823 ns/op FilterContentBenchmark.filterWithBuilder 10_wildcard_field true node_stats avgt 2 34450.794 ns/op FilterContentBenchmark.filterWithMap 10_field true cluster_stats avgt 2 492973.006 ns/op FilterContentBenchmark.filterWithMap 10_field true index_stats avgt 2 154447.596 ns/op FilterContentBenchmark.filterWithMap 10_field true node_stats avgt 2 181779.656 ns/op FilterContentBenchmark.filterWithMap half_field true cluster_stats avgt 2 2370660.246 ns/op FilterContentBenchmark.filterWithMap half_field true index_stats avgt 2 401988.258 ns/op FilterContentBenchmark.filterWithMap half_field true node_stats avgt 2 483548.818 ns/op FilterContentBenchmark.filterWithMap all_field true cluster_stats avgt 2 3113373.152 ns/op FilterContentBenchmark.filterWithMap all_field true index_stats avgt 2 476683.420 ns/op FilterContentBenchmark.filterWithMap all_field true node_stats avgt 2 652082.500 ns/op FilterContentBenchmark.filterWithMap wildcard_field true cluster_stats avgt 2 459415.604 ns/op FilterContentBenchmark.filterWithMap wildcard_field true index_stats avgt 2 70230.935 ns/op FilterContentBenchmark.filterWithMap wildcard_field true node_stats avgt 2 89341.348 ns/op FilterContentBenchmark.filterWithMap 10_wildcard_field true cluster_stats avgt 2 29668878.146 ns/op FilterContentBenchmark.filterWithMap 10_wildcard_field true index_stats avgt 2 28205368.190 ns/op FilterContentBenchmark.filterWithMap 10_wildcard_field true node_stats avgt 2 29332307.103 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_field true cluster_stats avgt 2 142003.739 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_field true index_stats avgt 2 18086.331 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_field true node_stats avgt 2 23383.744 ns/op FilterContentBenchmark.filterWithNewParserConfig half_field true cluster_stats avgt 2 389990.443 ns/op FilterContentBenchmark.filterWithNewParserConfig half_field true index_stats avgt 2 34894.318 ns/op FilterContentBenchmark.filterWithNewParserConfig half_field true node_stats avgt 2 45604.779 ns/op FilterContentBenchmark.filterWithNewParserConfig all_field true cluster_stats avgt 2 549971.331 ns/op FilterContentBenchmark.filterWithNewParserConfig all_field true index_stats avgt 2 51233.495 ns/op FilterContentBenchmark.filterWithNewParserConfig all_field true node_stats avgt 2 67761.011 ns/op FilterContentBenchmark.filterWithNewParserConfig wildcard_field true cluster_stats avgt 2 213109.821 ns/op FilterContentBenchmark.filterWithNewParserConfig wildcard_field true index_stats avgt 2 20191.982 ns/op FilterContentBenchmark.filterWithNewParserConfig wildcard_field true node_stats avgt 2 25866.681 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_wildcard_field true cluster_stats avgt 2 237100.911 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_wildcard_field true index_stats avgt 2 27744.183 ns/op FilterContentBenchmark.filterWithNewParserConfig 10_wildcard_field true node_stats avgt 2 34841.488 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_field true cluster_stats avgt 2 127459.484 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_field true index_stats avgt 2 15194.965 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_field true node_stats avgt 2 18798.854 ns/op FilterContentBenchmark.filterWithParserConfigCreated half_field true cluster_stats avgt 2 238616.088 ns/op FilterContentBenchmark.filterWithParserConfigCreated half_field true index_stats avgt 2 20972.354 ns/op FilterContentBenchmark.filterWithParserConfigCreated half_field true node_stats avgt 2 28047.925 ns/op FilterContentBenchmark.filterWithParserConfigCreated all_field true cluster_stats avgt 2 274096.558 ns/op FilterContentBenchmark.filterWithParserConfigCreated all_field true index_stats avgt 2 24948.247 ns/op FilterContentBenchmark.filterWithParserConfigCreated all_field true node_stats avgt 2 33163.017 ns/op FilterContentBenchmark.filterWithParserConfigCreated wildcard_field true cluster_stats avgt 2 209233.272 ns/op FilterContentBenchmark.filterWithParserConfigCreated wildcard_field true index_stats avgt 2 20056.664 ns/op FilterContentBenchmark.filterWithParserConfigCreated wildcard_field true node_stats avgt 2 25519.848 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_wildcard_field true cluster_stats avgt 2 221254.827 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_wildcard_field true index_stats avgt 2 24113.686 ns/op FilterContentBenchmark.filterWithParserConfigCreated 10_wildcard_field true node_stats avgt 2 31885.001 ns/op ```
1 parent 6819d57 commit 41afe85

File tree

1 file changed

+46
-2
lines changed

1 file changed

+46
-2
lines changed

benchmarks/src/main/java/org/elasticsearch/benchmark/xcontent/FilterContentBenchmark.java

+46-2
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,14 @@
88

99
package org.elasticsearch.benchmark.xcontent;
1010

11+
import org.elasticsearch.common.Strings;
1112
import org.elasticsearch.common.bytes.BytesReference;
1213
import org.elasticsearch.common.io.Streams;
1314
import org.elasticsearch.common.io.stream.BytesStreamOutput;
1415
import org.elasticsearch.common.util.Maps;
1516
import org.elasticsearch.common.xcontent.XContentHelper;
17+
import org.elasticsearch.common.xcontent.support.XContentMapValues;
18+
import org.elasticsearch.search.fetch.subphase.FetchSourcePhase;
1619
import org.elasticsearch.xcontent.XContentBuilder;
1720
import org.elasticsearch.xcontent.XContentParser;
1821
import org.elasticsearch.xcontent.XContentParserConfiguration;
@@ -39,8 +42,8 @@
3942
import java.util.stream.Collectors;
4043

4144
@Fork(1)
42-
@Warmup(iterations = 2)
43-
@Measurement(iterations = 3)
45+
@Warmup(iterations = 1)
46+
@Measurement(iterations = 2)
4447
@BenchmarkMode(Mode.AverageTime)
4548
@OutputTimeUnit(TimeUnit.NANOSECONDS)
4649
@State(Scope.Benchmark)
@@ -122,6 +125,47 @@ public BytesReference filterWithNewParserConfig() throws IOException {
122125
return filter(contentParserConfiguration);
123126
}
124127

128+
@Benchmark
129+
public BytesReference filterWithMap() throws IOException {
130+
Map<String, Object> sourceMap = XContentHelper.convertToMap(source, false).v2();
131+
String[] includes;
132+
String[] excludes;
133+
if (inclusive) {
134+
includes = filters.toArray(Strings.EMPTY_ARRAY);
135+
excludes = null;
136+
} else {
137+
includes = null;
138+
excludes = filters.toArray(Strings.EMPTY_ARRAY);
139+
}
140+
Map<String, Object> filterMap = XContentMapValues.filter(sourceMap, includes, excludes);
141+
return FetchSourcePhase.objectToBytes(filterMap, XContentType.JSON, Math.min(1024, source.length()));
142+
}
143+
144+
@Benchmark
145+
public BytesReference filterWithBuilder() throws IOException {
146+
BytesStreamOutput streamOutput = new BytesStreamOutput(Math.min(1024, source.length()));
147+
Set<String> includes;
148+
Set<String> excludes;
149+
if (inclusive) {
150+
includes = filters;
151+
excludes = Set.of();
152+
} else {
153+
includes = Set.of();
154+
excludes = filters;
155+
}
156+
XContentBuilder builder = new XContentBuilder(
157+
XContentType.JSON.xContent(),
158+
streamOutput,
159+
includes,
160+
excludes,
161+
XContentType.JSON.toParsedMediaType()
162+
);
163+
try (XContentParser parser = XContentType.JSON.xContent().createParser(XContentParserConfiguration.EMPTY, source.streamInput())) {
164+
builder.copyCurrentStructure(parser);
165+
return BytesReference.bytes(builder);
166+
}
167+
}
168+
125169
private XContentParserConfiguration buildParseConfig() {
126170
Set<String> includes;
127171
Set<String> excludes;

0 commit comments

Comments
 (0)