Factorize dig logic ; add a few options ; more tests ; fix README (#137)

* Add a few tests to improve coverage and highlight pending behaviors.

* Factorize digging logic to a Dig helper module.

* Add support for a :use_symbols option.

* Add support for objects responding to dig.

* Fix digging behavior in presence of explicit null/nil.

* Fix & test README claims, document available options.

Also includes a library fix when default_path_leaf_to_null is
used.

* Introduce :allow_send and document it (defaults to true).

Author: blambeau

README.md

@@ -4,8 +4,8 @@ This is an implementation of http://goessner.net/articles/JsonPath/.
 
 ## What is JsonPath?
 
-JsonPath is a way of addressing elements within a JSON object. Similar to xpath of yore, JsonPath lets you
-traverse a json object and manipulate or access it.
+JsonPath is a way of addressing elements within a JSON object. Similar to xpath
+of yore, JsonPath lets you traverse a json object and manipulate or access it.
 
 ## Usage
 
@@ -15,8 +15,8 @@ There is stand-alone usage through the binary `jsonpath`
 
     jsonpath [expression] (file|string)
 
-    If you omit the second argument, it will read stdin, assuming one valid JSON object
-    per line. Expression must be a valid jsonpath expression.
+    If you omit the second argument, it will read stdin, assuming one valid JSON
+    object per line. Expression must be a valid jsonpath expression.
 
 ### Library
 
@@ -40,8 +40,8 @@ json = <<-HERE_DOC
 HERE_DOC
 ```
 
-Now that we have a JSON object, let's get all the prices present in the object. We create an object for the path
-in the following way.
+Now that we have a JSON object, let's get all the prices present in the object.
+We create an object for the path in the following way.
 
 ```ruby
 path = JsonPath.new('$..price')
@@ -54,14 +54,15 @@ path.on(json)
 # => [19.95, 8.95, 12.99, 8.99, 22.99]
 ```
 
-Or on some other object ...
+Or reuse it later on some other object (thread safe) ...
 
 ```ruby
 path.on('{"books":[{"title":"A Tale of Two Somethings","price":18.88}]}')
 # => [18.88]
 ```
 
-You can also just combine this into one mega-call with the convenient `JsonPath.on` method.
+You can also just combine this into one mega-call with the convenient
+`JsonPath.on` method.
 
 ```ruby
 JsonPath.on(json, '$..author')
@@ -73,59 +74,114 @@ Of course the full JsonPath syntax is supported, such as array slices
 ```ruby
 JsonPath.new('$..book[::2]').on(json)
 # => [
-#      {"price"=>8.95, "category"=>"reference", "author"=>"Nigel Rees", "title"=>"Sayings of the Century"},
-#      {"price"=>8.99, "category"=>"fiction", "author"=>"Herman Melville", "title"=>"Moby Dick", "isbn"=>"0-553-21311-3"}
+#      {"price" => 8.95, "category" => "reference", "title" => "Sayings of the Century", "author" => "Nigel Rees"},
+#      {"price" => 8.99, "category" => "fiction", "isbn" => "0-553-21311-3", "title" => "Moby Dick", "author" => "Herman Melville","color" => "blue"},
 #    ]
 ```
 
-...and evals.
+...and evals, including those with conditional operators
 
 ```ruby
-JsonPath.new('$..price[?(@ < 10)]').on(json)
+JsonPath.new("$..price[?(@ < 10)]").on(json)
 # => [8.95, 8.99]
+
+JsonPath.new("$..book[?(@['price'] == 8.95 || @['price'] == 8.99)].title").on(json)
+# => ["Sayings of the Century", "Moby Dick"]
+
+JsonPath.new("$..book[?(@['price'] == 8.95 && @['price'] == 8.99)].title").on(json)
+# => []
 ```
 
-There is a convenience method, `#first` that gives you the first element for a JSON object and path.
+There is a convenience method, `#first` that gives you the first element for a
+JSON object and path.
 
 ```ruby
-JsonPath.new('$..color').first(object)
+JsonPath.new('$..color').first(json)
 # => "red"
 ```
 
 As well, we can directly create an `Enumerable` at any time using `#[]`. 
 
 ```ruby
-enum = JsonPath.new('$..color')[object]
+enum = JsonPath.new('$..color')[json]
 # => #<JsonPath::Enumerable:...>
 enum.first
 # => "red"
 enum.any?{ |c| c == 'red' }
 # => true
 ```
 
-### More examples
+For more usage examples and variations on paths, please visit the tests. There
+are some more complex ones as well.
 
-For more usage examples and variations on paths, please visit the tests. There are some more complex ones as well.
+### Querying ruby data structures
 
-### Conditional Operators Are Also Supported
+If you have ruby hashes with symbolized keys as input, you
+can use `:use_symbols` to make JsonPath work fine on them too:
 
 ```ruby
-  def test_or_operator
-    assert_equal [@object['store']['book'][1], @object['store']['book'][3]], JsonPath.new("$..book[?(@['price'] == 13 || @['price'] == 23)]").on(@object)
-  end
+book = { title: "Sayings of the Century" }
 
-  def test_and_operator
-    assert_equal [], JsonPath.new("$..book[?(@['price'] == 13 && @['price'] == 23)]").on(@object)
-  end
+JsonPath.new('$.title').on(book)
+# => []
 
-  def test_and_operator_with_more_results
-    assert_equal [@object['store']['book'][1]], JsonPath.new("$..book[?(@['price'] < 23 && @['price'] > 9)]").on(@object)
-  end
+JsonPath.new('$.title', use_symbols: true).on(book)
+# => ["Sayings of the Century"]
+```
+
+JsonPath also recognizes objects responding to `dig` (introduced
+in ruby 2.3), and therefore works out of the box with Struct,
+OpenStruct, and other Hash-like structures:
+
+```ruby
+book_class = Struct.new(:title)
+book = book_class.new("Sayings of the Century")
+
+JsonPath.new('$.title').on(book)
+# => ["Sayings of the Century"]
+```
+
+JsonPath is able to query pure ruby objects and uses `__send__`
+on them. The option is enabled by default in JsonPath 1.x, but
+we encourage to enable it explicitly:
+
+```ruby
+book_class = Class.new{ attr_accessor :title }
+book = book_class.new
+book.title = "Sayings of the Century"
+
+JsonPath.new('$.title', allow_send: true).on(book)
+# => ["Sayings of the Century"]
+```
+
+### Other available options
+
+By default, JsonPath does not return null values on unexisting paths.
+This can be changed using the `:default_path_leaf_to_null` option
+
+```ruby
+JsonPath.new('$..book[*].isbn').on(json)
+# => ["0-553-21311-3", "0-395-19395-8"]
+
+JsonPath.new('$..book[*].isbn', default_path_leaf_to_null: true).on(json)
+# => [nil, nil, "0-553-21311-3", "0-395-19395-8"]
+```
+
+When JsonPath returns a Hash, you can ask to symbolize its keys
+using the `:symbolize_keys` option
+
+```ruby
+JsonPath.new('$..book[0]').on(json)
+# => [{"category" => "reference", ...}]
+
+JsonPath.new('$..book[0]', symbolize_keys: true).on(json)
+# => [{category: "reference", ...}]
 ```
 
 ### Selecting Values
 
-It's possible to select results once a query has been defined after the query. For example given this JSON data:
+It's possible to select results once a query has been defined after the query. For
+example given this JSON data:
 
 ```bash
 {
@@ -168,15 +224,10 @@ It's possible to select results once a query has been defined after the query. F
 ]
 ```
 
-### Running an individual test
-
-```ruby
-ruby -Ilib:../lib test/test_jsonpath.rb --name test_wildcard_on_intermediary_element_v6
-```
-
 ### Manipulation
 
-If you'd like to do substitution in a json object, you can use `#gsub` or `#gsub!` to modify the object in place.
+If you'd like to do substitution in a json object, you can use `#gsub`
+or `#gsub!` to modify the object in place.
 
 ```ruby
 JsonPath.for('{"candy":"lollipop"}').gsub('$..candy') {|v| "big turks" }.to_hash
@@ -188,7 +239,9 @@ The result will be
 {'candy' => 'big turks'}
 ```
 
-If you'd like to remove all nil keys, you can use `#compact` and `#compact!`. To remove all keys under a certain path, use `#delete` or `#delete!`. You can even chain these methods together as follows:
+If you'd like to remove all nil keys, you can use `#compact` and `#compact!`.
+To remove all keys under a certain path, use `#delete` or `#delete!`. You can
+even chain these methods together as follows:
 
 ```ruby
 json = '{"candy":"lollipop","noncandy":null,"other":"things"}'
@@ -202,4 +255,11 @@ o = JsonPath.for(json).
 
 # Contributions
 
-Please feel free to submit an Issue or a Pull Request any time you feel like you would like to contribute. Thank you!
+Please feel free to submit an Issue or a Pull Request any time you feel like
+you would like to contribute. Thank you!
+
+## Running an individual test
+
+```ruby
+ruby -Ilib:../lib test/test_jsonpath.rb --name test_wildcard_on_intermediary_element_v6
+```

lib/jsonpath.rb

@@ -3,6 +3,7 @@
 require 'strscan'
 require 'multi_json'
 require 'jsonpath/proxy'
+require 'jsonpath/dig'
 require 'jsonpath/enumerable'
 require 'jsonpath/version'
 require 'jsonpath/parser'
@@ -12,10 +13,17 @@
 class JsonPath
   PATH_ALL = '$..*'
 
+  DEFAULT_OPTIONS = {
+    :default_path_leaf_to_null => false,
+    :symbolize_keys => false,
+    :use_symbols => false,
+    :allow_send => true
+  }
+
   attr_accessor :path
 
   def initialize(path, opts = {})
-    @opts = opts
+    @opts = DEFAULT_OPTIONS.merge(opts)
     scanner = StringScanner.new(path.strip)
     @path = []
     until scanner.eos?

lib/jsonpath/dig.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+class JsonPath
+  module Dig
+
+    # Similar to what Hash#dig or Array#dig
+    def dig(context, *keys)
+      keys.inject(context){|memo,k|
+        dig_one(memo, k)
+      }
+    end
+
+    # Returns a hash mapping each key from keys
+    # to its dig value on context.
+    def dig_as_hash(context, keys)
+      keys.each_with_object({}) do |k, memo|
+        memo[k] = dig_one(context, k)
+      end
+    end
+
+    # Dig the value of k on context.
+    def dig_one(context, k)
+      case context
+      when Hash
+        context[@options[:use_symbols] ? k.to_sym : k]
+      when Array
+        context[k.to_i]
+      else
+        if context.respond_to?(:dig)
+          context.dig(k)
+        elsif @options[:allow_send]
+          context.__send__(k)
+        end
+      end
+    end
+
+    # Yields the block if context has a diggable
+    # value for k
+    def yield_if_diggable(context, k, &blk)
+      case context
+      when Array
+        nil
+      when Hash
+        k = @options[:use_symbols] ? k.to_sym : k
+        return yield if context.key?(k) || @options[:default_path_leaf_to_null]
+      else
+        if context.respond_to?(:dig)
+          digged = dig_one(context, k)
+          yield if !digged.nil? || @options[:default_path_leaf_to_null]
+        elsif @options[:allow_send] && context.respond_to?(k.to_s) && !Object.respond_to?(k.to_s)
+          yield
+        end
+      end
+    end
+
+  end
+end

lib/jsonpath/enumerable.rb

@@ -3,6 +3,7 @@
 class JsonPath
   class Enumerable
     include ::Enumerable
+    include Dig
 
     def initialize(path, object, mode, options = {})
       @path = path.path
@@ -12,12 +13,7 @@ def initialize(path, object, mode, options = {})
     end
 
     def each(context = @object, key = nil, pos = 0, &blk)
-      node =
-        if key
-          context.is_a?(Hash) || context.is_a?(Array) ? context[key] : context.__send__(key)
-        else
-          context
-        end
+      node = key ? dig_one(context, key) : context
       @_current_node = node
       return yield_value(blk, context, key) if pos == @path.size
 
@@ -47,11 +43,10 @@ def each(context = @object, key = nil, pos = 0, &blk)
     def filter_context(context, keys)
       case context
       when Hash
-        # TODO: Change this to `slice(*keys)` when ruby version support is > 2.4
-        context.select { |k| keys.include?(k) }
+        dig_as_hash(context, keys)
       when Array
         context.each_with_object([]) do |c, memo|
-          memo << c.select { |k| keys.include?(k) }
+          memo << dig_as_hash(c, keys)
         end
       end
     end
@@ -61,16 +56,14 @@ def handle_wildecard(node, expr, _context, _key, pos, &blk)
         case sub_path[0]
         when '\'', '"'
           k = sub_path[1, sub_path.size - 2]
-          if node.is_a?(Hash)
-            node[k] ||= nil if @options[:default_path_leaf_to_null]
-            each(node, k, pos + 1, &blk) if node.key?(k)
-          elsif node.respond_to?(k.to_s) && !Object.respond_to?(k.to_s)
+          yield_if_diggable(node, k) do
             each(node, k, pos + 1, &blk)
           end
         when '?'
           handle_question_mark(sub_path, node, pos, &blk)
         else
           next if node.is_a?(Array) && node.empty?
+          next if node.nil? # when default_path_leaf_to_null is true
 
           array_args = sub_path.split(':')
           if array_args[0] == '*'
@@ -130,7 +123,7 @@ def handle_question_mark(sub_path, node, pos, &blk)
     def yield_value(blk, context, key)
       case @mode
       when nil
-        blk.call(key ? context[key] : context)
+        blk.call(key ? dig_one(context, key) : context)
       when :compact
         if key && context[key].nil?
           key.is_a?(Integer) ? context.delete_at(key) : context.delete(key)
@@ -162,12 +155,12 @@ def process_function_or_literal(exp, default = nil)
           el == '@' ? '@' : "['#{el}']"
         end.join
         begin
-          return JsonPath::Parser.new(@_current_node).parse(exp_to_eval)
+          return JsonPath::Parser.new(@_current_node, @options).parse(exp_to_eval)
         rescue StandardError
           return default
         end
       end
-      JsonPath::Parser.new(@_current_node).parse(exp)
+      JsonPath::Parser.new(@_current_node, @options).parse(exp)
     end
   end
 end