Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Diff reports confusing output when used with "fuzzy" matchers like anything #551

Open
Epigene opened this issue Oct 25, 2022 · 1 comment

Comments

@Epigene
Copy link

Epigene commented Oct 25, 2022

Subject of the issue

Using .to match with a complex expectation that mixes pure values with "fuzzy" matchers like anything produces false positives in Diff, obfuscating which key actually differs from the expectation.
I don't know whether this is a problem with match or anything/*_including.

Your environment

  • Ruby version: 2.7.5 (also on 2.7.6)
  • rspec-expectations version: 3.11.0

Steps to reproduce

# frozen_string_literal: true

begin
  require "bundler/inline"
rescue LoadError => e
  $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler"
  raise e
end

gemfile(true) do
  source "https://rubygems.org"

  gem "rspec", "3.11.0"
  gem "rspec-expectations", "3.11.0"
  gem "rspec-mocks", "3.11.1"
end

puts "Ruby version is: #{RUBY_VERSION}"
require 'rspec/autorun'

RSpec.describe 'false positive in #match diff' do
  it 'reports excess rows as mismatching' do
    expect(
      a: :a,
      b: [:b, :bb],
      c: :c,
      d: :d,
    ).to match(
      a: anything,
      b: array_including(:b),
      c: :c,     
      d: :actual_mismatch,
    )
  end
end

Output:

$ ruby rspec_demo.rb
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using bundler 2.3.20
Using diff-lcs 1.5.0
Using rspec-support 3.11.1
Using rspec-core 3.11.0
Using rspec-expectations 3.11.0
Using rspec-mocks 3.11.1
Using rspec 3.11.0
Ruby version is: 2.7.5
F

Expected behavior

I'm expecting that Diff part will highlight only the key(s) actually failing their asserts, which is only :d.

Failures:

  1) false positive in #match diff reports excess rows as mismatching
     Failure/Error:
       expect(
         a: :a,
         b: [:b, :bb],
         c: :c,
         d: :d,
       ).to match(
         a: anything,
         b: array_including(:b),
         c: :c,
         d: :actual_mismatch,

       expected {:a=>:a, :b=>[:b, :bb], :c=>:c, :d=>:d} to match {:a=>#<RSpec::Mocks::ArgumentMatchers::AnyArgMatcher:0x00007ff34d667f18>, :b=>#<RSpec::Mocks::ArgumentMatchers::ArrayIncludingMatcher:0x00007ff34d58cc88 @expected=[:b]>, :c=>:c, :d=>:actual_mismatch}
       Diff:

       @@ -1,5 +1,5 @@
       -:d => :actual_mismatch,
       +:d => :d,

     # rspec_demo.rb:23:in `block (2 levels) in <main>'

Actual behavior

Unfortunately, keys :a and :b are also highlighted, likely due to them using "fuzzy" matchers.

Failures:

  1) false positive in #match diff reports excess rows as mismatching
     Failure/Error:
       expect(
         a: :a,
         b: [:b, :bb],
         c: :c,
         d: :d,
       ).to match(
         a: anything,
         b: array_including(:b),
         c: :c,
         d: :actual_mismatch,

       expected {:a=>:a, :b=>[:b, :bb], :c=>:c, :d=>:d} to match {:a=>#<RSpec::Mocks::ArgumentMatchers::AnyArgMatcher:0x00007ff34d667f18>, :b=>#<RSpec::Mocks::ArgumentMatchers::ArrayIncludingMatcher:0x00007ff34d58cc88 @expected=[:b]>, :c=>:c, :d=>:actual_mismatch}
       Diff:

       @@ -1,5 +1,5 @@
       -:a => anything,
       -:b => array_including(b),
       +:a => :a,
       +:b => [:b, :bb],
        :c => :c,
       -:d => :actual_mismatch,
       +:d => :d,

     # rspec_demo.rb:23:in `block (2 levels) in <main>'
@JonRowe JonRowe transferred this issue from rspec/rspec-expectations Oct 25, 2022
@JonRowe JonRowe changed the title .to match Diff reports false positives when used with "fuzzy" matchers like anything Diff reports confusing output when used with "fuzzy" matchers like anything Oct 25, 2022
@JonRowe
Copy link
Member

JonRowe commented Oct 25, 2022

👋 Hi, this is a known limitation of our differ which essentially only uses strings to produce diffs, so its unaware that the two sides did match it just sees the difference in the output used, theres not much we can do about it without a complete rewrite of the differ which is something we are interested in but has yet to find someone with enough time to finish it, I've transferred the issue to rspec-support which provides this functionality and renamed it slightly to better keep track of it, it related to #365 and #518.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants