Skip to content

[BUG] Unable to deserialize adjacency matrix aggregation buckets #1478

Closed
@CodeSlothTrent

Description

@CodeSlothTrent

What is the bug?

A clear and concise description of the bug.

The OpenSearchClient is unable to deserialize Adjacency Matrix aggregation results.

How can one reproduce the bug?

Steps to reproduce the behavior.

A unit test can be found in the public repository at the following link:
https://github.com/CodeSlothTrent/codesloth-search-samples-java/blob/cab26928e8ccf367f9e8e10b01e1c4bc77ab2602/OpenSearchSamples/src/test/java/KeywordDemo/KeywordAggregationTests.java#L271

Code copied here for reference

@Test
    public void keywordMapping_CanBeUsedForAdjacencyMatrixAggregation() throws Exception {
        // Create a test index with keyword mapping for the ProductNames field
        try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
                mapping.properties("productNames", Property.of(p -> p.keyword(k -> k))))) {

            // Create and index user favourite products documents
            UserFavouriteProducts[] userFavouriteProducts = new UserFavouriteProducts[]{
                    new UserFavouriteProducts(1, new String[]{"mouse", "mouse pad"}),
                    new UserFavouriteProducts(2, new String[]{"mouse"}),
                    new UserFavouriteProducts(3, new String[]{"keyboard"}),
                    new UserFavouriteProducts(4, new String[]{"mouse pad", "keyboard"}),
                    new UserFavouriteProducts(5, new String[]{"mouse", "keyboard"}),
                    new UserFavouriteProducts(6, new String[]{"mouse", "mouse pad"})
            };
            testIndex.indexDocuments(userFavouriteProducts);

            // Create a map of the different filters
            var filterMap = Map.of(
                    "mouse", Query.of(q -> q.term(t -> t.field("productNames").value(FieldValue.of("mouse")))),
                    "mouse pad", Query.of(q -> q.term(t -> t.field("productNames").value(FieldValue.of("mouse pad")))),
                    "keyboard", Query.of(q -> q.term(t -> t.field("productNames").value(FieldValue.of("keyboard")))));

            // Create a search request with adjacency matrix aggregation
            SearchRequest searchRequest = new SearchRequest.Builder()
                    .index(testIndex.getName())
                    .query(builder -> builder.matchAll(new MatchAllQuery.Builder().build()))
                    .aggregations("product_matrix", a -> a
                            .adjacencyMatrix(am -> am
                                    .filters(filterMap)
                            )
                    )
                    .q("?typed_keys=true")
                    .build();

            OpenSearchRequestLogger.LogRequestJson(searchRequest);

            // Execute the search request
            SearchResponse<UserFavouriteProducts> response = openSearchClient.search(searchRequest, UserFavouriteProducts.class);

            // Verify the results
            assertThat(response.aggregations()).isNotNull();

            AdjacencyMatrixAggregate matrixAgg = response.aggregations().get("product_matrix").adjacencyMatrix();

            // Check the bucket counts
            Map<String, Long> bucketCounts = matrixAgg.buckets().array().stream()
                    .collect(Collectors.toMap(
                            AdjacencyMatrixBucket::toString,
                            AdjacencyMatrixBucket::docCount
                    ));

            // Format the results for verification
            String formattedResults = bucketCounts.entrySet().stream()
                    .map(entry -> entry.getKey() + ":" + entry.getValue())
                    .collect(Collectors.joining(", "));

            // Verify the expected results
            assertThat(bucketCounts).containsEntry("keyboard", 3L);
            assertThat(bucketCounts).containsEntry("keyboard&mouse", 1L);
            assertThat(bucketCounts).containsEntry("keyboard&mouse pad", 1L);
            assertThat(bucketCounts).containsEntry("mouse", 4L);
            assertThat(bucketCounts).containsEntry("mouse pad", 3L);
            assertThat(bucketCounts).containsEntry("mouse&mouse pad", 2L);
        }

Example query produced

{
  "aggregations" : {
    "product_matrix" : {
      "adjacency_matrix" : {
        "filters" : {
          "keyboard" : {
            "term" : {
              "productNames" : {
                "value" : "keyboard"
              }
            }
          },
          "mouse pad" : {
            "term" : {
              "productNames" : {
                "value" : "mouse pad"
              }
            }
          },
          "mouse" : {
            "term" : {
              "productNames" : {
                "value" : "mouse"
              }
            }
          }
        }
      }
    }
  },
  "query" : {
    "match_all" : { }
  }
}

What is the expected behavior?

A clear and concise description of what you expected to happen.

The above query when run in opensearch dashboards developer tools returns results

Image

However, looking at the results in code, the buckets are not populated

Image

What is your host/environment?

Operating system, version.

Windows 11 Home. opensearch-java 2.15.0.

Do you have any screenshots?

If applicable, add screenshots to help explain your problem.

Do you have any additional context?

Add any other context about the problem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions