Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand the 'Known Issues' documentation article #823

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

stmontgomery
Copy link
Contributor

This expands the Known issues DocC article to give examples and describe several different ways it can be used.

Checklist:

  • Code and documentation should follow the style of the Style Guide.
  • If public symbols are renamed or modified, DocC references should be updated.

Resolves rdar://137961874

@stmontgomery stmontgomery added documentation Improvements or additions to documentation enhancement New feature or request issue-handling Related to Issue handling within the testing library labels Nov 13, 2024
@stmontgomery stmontgomery self-assigned this Nov 13, 2024
@stmontgomery
Copy link
Contributor Author

@swift-ci please test

Copy link

@plemarquand plemarquand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff!

Sources/Testing/Testing.docc/known-issues.md Outdated Show resolved Hide resolved
@@ -10,13 +10,135 @@ See https://swift.org/LICENSE.txt for license information
See https://swift.org/CONTRIBUTORS.txt for Swift project authors
-->

Highlight known issues when running tests.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this specific phrasing came from our DevPubs friends? So you might want to ask them to look too.


### Mark a thrown error as known

If an `Error` is caught by the closure passed to `withKnownIssue()`, the issue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to explain the difference from #expect(throws:)?

}
```

If the underlying problem is truly nondeterministic, you may acknowledge this
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

If the underlying problem is unpredictable and causes your test to fail at random, you can pass isIntermittent: true to let the testing library know that your test will not always record an issue:


If `withKnownIssue()` sometimes succeeds but other times records an issue
indicating there were no known issues, this may indicate a nondeterministic
failure or a "flaky" test.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want a paragraph here emphasizing "fix your tests first". We get a fair amount of feedback asking "why even have XFAIL semantics?"

#expect(foodTruck.grill.isHeating)
}
}
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to add one more example showing all of the knobs and doodads in a single call?

Copy link
Contributor

@grynspan grynspan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicks only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request issue-handling Related to Issue handling within the testing library
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants