`qunit/require-expect` with `error:except-simple` flags tests that contain functions or loops unrelated to any assertions

[BillyRayPreachersSon]

We've set up our linting for qunit/require-expect as follows:

'qunit/require-expect': ['error', 'except-simple'],

Given the following two tests - where the only difference is the syntax used to reject after a call to checkThing, why should the first one be flagged as erroneous?

// Fails linting with ESLint: Should use `expect()` when using assertions outside of the top-level test callback.(qunit/require-expect)

test('checks things', function() {
  const service = this.owner.lookup('service:my-service');
  service.checkService = EmberObject.create({
    checkThing: sinon.stub(() => reject()),
  });

  service.run();
  sinon.assert.calledOnce(service.checkService.checkThing);
});

test('checks things again', function() {
  const service = this.owner.lookup('service:my-service');
  service.checkService = EmberObject.create({
    checkThing: sinon.stub().rejects(),
  });

  service.run();
  sinon.assert.calledOnce(service.checkService.checkThing);
});

As an aside, we've set up our test-helper.js file to allow Sinon.JS's assertions to count as QUnit assertions, so there's no issue with having no assertions in these tests.

[BillyRayPreachersSon]

Sorry - I forgot to mention, we're using "eslint-plugin-qunit": "8.2.5" 🙂 .

[BillyRayPreachersSon]

A couple more examples - here, the first test fails linting for the same reason:

// Fails linting with ESLint: Should use `expect()` when using assertions outside of the top-level test callback.(qunit/require-expect)
test('some thing', function() {
  const someStub = sinon.stub();

  run(() => {
    console.log('o hai!');    // eslint-disable-line no-console
  });

  sinon.assert.notCalled(someStub);
});

test('some other thing', function() {
  const someStub = sinon.stub();
  sinon.assert.notCalled(someStub);
});

Why should the addition of a run block containing a console.log cause the linting outcome to change?

[BillyRayPreachersSon]

Another example, this time with a standard for loop causing the issue:

// Fails linting with ESLint: Should use `expect()` when using assertions outside of the top-level test callback.(qunit/require-expect)
test('does something', async function() {
  const someStub = sinon.stub();
  this.set('someStub', someStub);
  await render(hbs`<SomeComponent @someStub={{this.someStub}} @someProp={{this.someProp}} />`);

  for (let i = 0; i < 5; i++) {
    this.set('someProp', i);
  }

  sinon.assert.calledOnce(someStub);
});

test('does something else', async function() {
  const someStub = sinon.stub();
  this.set('someStub', someStub);
  await render(hbs`<SomeComponent @someStub={{this.someStub}} @someProp={{this.someProp}} />`);

  sinon.assert.calledOnce(someStub);
});

[Krinkle]

@BillyRayPreachersSon Thanks for the report. The except-simple threshold asserts there are no nested function that call assert. I suspect the nested function in your example is misclassified by that logic.

By the way, we no longer recommend except-simple in the ESLint preset and in QUnit docs. It was the default for many years, and it'll remain forever a stable part of the QUnit API, so no worries if that's your choice. You can read about this in The value and benefit of assert.expect() and assert.expect() - QUnit API where we recommend error:never-except-zero.