Issue with filters and projections

[springcomp]

See jmespath-community/jmespath.test#35

Credits: Reported by Dmitriy on Gitter.

[springcomp]

The root cause for the issue is that a filter [?…] bracket-specifier ends a projection, whereas a flatten [] bracket-specifier does not. I’m not sure where does this requirement come from.

Here are the two expressions:

• Expression 1: [?stack==''].branch[?starts_with(@, 'one')] (this incorrectly raises an invalid-type error). Edit this turns out to be the correct result by design. But this is very counter-intuitive and should probably be fixed by a proposal in the spec.
• Expression 2: [].branch[?starts_with(@, 'one')] this returns [ [ "one/" ], [] ]

Expression 1 AST

• Expression 1: [?stack==''].branch[?starts_with(@, 'one')]

graph TD
  filter --> where_stack
  filter --> identity_
  filter --> function_starts_with

  where_stack --> identity
  where_stack --> field_branch
  where_stack --> comparator

  comparator --> field_stack
  comparator --> string_empty
  
  function_starts_with --> current
  function_starts_with --> string_one

Loading

JSON representation of the AST

{
	"type": "filter_projection",
	"children": [
		{
			"type": "filter_projection",
			"children": [
				{
					"type": "identity",
					"children": []
				},
				{
					"type": "field",
					"children": [],
					"value": "branch"
				},
				{
					"type": "comparator",
					"children": [
						{
							"type": "field",
							"children": [],
							"value": "stack"
						},
						{
							"type": "literal",
							"value": "",
							"children": []
						}
					],
					"value": "eq"
				}
			]
		},
		{
			"type": "identity",
			"children": []
		},
		{
			"type": "function_expression",
			"children": [
				{
					"type": "current",
					"children": []
				},
				{
					"type": "literal",
					"value": "one",
					"children": []
				}
			],
			"value": "starts_with"
		}
	]
}

Expression 2 AST

• Expression 2: [].branch[?starts_with(@, 'one')] this returns [ [ "one/" ], [] ]

graph TD
  projection --> flatten

  flatten --> identity_
  flatten --> filter_projection

  filter_projection --> field_branch
  filter_projection --> identity
  filter_projection --> function_starts_with
  
  function_starts_with --> current
  function_starts_with --> string_one

Loading

JSON representation of the AST

{
	"type": "projection",
	"children": [
		{
			"type": "flatten",
			"children": [
				{
					"type": "identity",
					"children": []
				}
			]
		},
		{
			"type": "filter_projection",
			"children": [
				{
					"type": "field",
					"children": [],
					"value": "branch"
				},
				{
					"type": "identity",
					"children": []
				},
				{
					"type": "function_expression",
					"children": [
						{
							"type": "current",
							"children": []
						},
						{
							"type": "literal",
							"value": "one",
							"children": []
						}
					],
					"value": "starts_with"
				}
			]
		}
	]
}

[springcomp]

By contrast, JmesPath.Net does not have the bug returns a result. The reason is that JmesPath.NET uses a Lex-Yacc based parser syntax that is different than the Pratt-based parser used in most other implementations. The behaviour (intuitive, but incorrect) is due to the initial lack of deep knowledge for how bracket-specifier were designed.

There, the ASTs looks like this:

Expression 1 AST

• Expression 1 [?stack==''].branch[?starts_with(@, 'one')]

graph TD
  index_expression --> sub_expression
  index_expression --> filter_projection

  sub_expression --> where_stack
  sub_expression --> identifier_branch

  where_stack --> equal_operator

  equal_operator --> identifier_stack
  equal_operator --> string_empty 

  filter_projection --> function_starts_with
  
 function_starts_with --> current
 function_starts_with --> identifier_one

Loading

Expression 2 AST

• Expression 2 [].branch[?starts_with(@, 'one')]

graph TD
  index_expression --> sub_expression
  index_expression --> filter_projection

  sub_expression --> flatten
  sub_expression --> identifier_branch

  filter_projection --> function_starts_with
  
 function_starts_with --> current
 function_starts_with --> identifier_one

Loading