ARAX failing NeverShow tests because of data from automat-robokop. What to do?

[edeutsch]

There are over a dozen automated NeverShow tests that ARAX is failing but Aragorn is passing based on data from automat-robokop. For example:

Aragorn passes all these. ARAX fails all them.
The first one is NeverShow MMP3

https://arax.ci.transltr.io/?r=873719be-f82d-4648-a3a8-7328322aafa0

automat-robokop is telling us this relationship.
ARAX ranks it quite low, but there it is
But we fail the test because we show it.

There are many others in this same category.

My main question is: How is Aragorn passing this test when its main KG (robokop?) is feeding this information?

The second question is: how should ARAX pass this test? Should we just stop using automat-robokop? Or what would be the best approach?

[maximusunc]

Hmmm, this is odd. If we query Robokop directly through cypher, we don't see this node coming back:
match (g:biolink:Gene)-[x:biolink:affects]->(a:biolink:ChemicalEntity {id:"CHEBI:29103"}) return *;

[edeutsch]

According to ARAX, sending this query:

{
  "message": {
    "query_graph": {
      "edges": {
        "t_edge": {
          "knowledge_type": "lookup",
          "object": "ON",
          "predicates": [
            "biolink:affects"
          ],
          "qualifier_constraints": [
            {
              "qualifier_set": [
                {
                  "qualifier_type_id": "biolink:object_aspect_qualifier",
                  "qualifier_value": "activity_or_abundance"
                },
                {
                  "qualifier_type_id": "biolink:object_direction_qualifier",
                  "qualifier_value": "increased"
                }
              ]
            }
          ],
          "subject": "SN"
        }
      },
      "nodes": {
        "ON": {
          "categories": [
            "biolink:Gene",
            "biolink:Protein"
          ],
          "is_set": true
        },
        "SN": {
          "categories": [
            "biolink:ChemicalEntity"
          ],
          "ids": [
            "CHEBI:29103"
          ],
          "is_set": false
        }
      }
    }
  },
  "submitter": "infores:arax"
}

to automat-robokop in CI, returns 454 edges:

including the NeverShow ones.

I have not set up a repro to independently show this, but have no reason to doubt ARAX on this.

[maximusunc]

Yep. I will do more digging!

[edeutsch]

Any progress on finding out why automat-robokop is providing this NeverShow information?

[maximusunc]

We've confirmed that Robokopkg has this answer, so Aragorn must have a cached response it gives each week that missed this particular answer? We do have other edges from Robokop. So we should assume that Aragorn should be failing all of these tests as well.

As far as the solution goes, I'm not sure what to do. Is it a bad edge in Robokop and we should go in and remove it from the kg?

[edeutsch]

It does seem strange that Aragorn has a very long term cache that has better data than the robokop kp.

Note that it is not just one edge, it is many edges.

1. One solution is to make a list of all these bad edges in the test results and remove them, but this seems like defeating the purpose of the test since it is presumably just a small subset of bad connections, so just removing those makes the test seems better than the system really is

2. A better solution would be to understand where all these edges are coming from and make changes to the robokop kp ingest process to not bring in the bad information that is probably pervasive than just the specific edges that we test. This would seem important to understand as it might affect the ongoing ingesting process by DINGO.

3. If those action can't be taken soon, we could just block ARAX from querying robokop kg. This could be a pretty quick fix.