Skip to content

Commit e50458e

Browse files
committed
Add guidance for fixing spec test failures in dependabot PRs
1 parent c6a2b52 commit e50458e

File tree

1 file changed

+29
-0
lines changed

1 file changed

+29
-0
lines changed

CONTRIBUTING.md

+29
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,35 @@ The following environment variables are used for [CSFLE testing](https://github.
109109
* `KMS_TLS_CA_FILE`
110110
* `KMS_TLS_CERTIFICATE_KEY_FILE`
111111

112+
### Updating spec tests
113+
114+
Tests from the MongoDB Specifications repository are included through a
115+
submodule and updated automatically through Dependabot. To update tests
116+
manually, switch to the `tests/specifications` directory and update the
117+
repository to the appropriate commit. Remember to commit this change to the
118+
library repository.
119+
120+
#### Handling test failures on updates
121+
122+
Failures on updates can occur for multiple reasons, and the remedy to this will
123+
depend on the type of failure. Note that only tests for implemented
124+
specifications are run in the test runner.
125+
126+
* If a specification is not fully implemented (e.g. a recent change to the spec
127+
has not been applied yet), skip the test in question with a reference to the
128+
ticket that covers the change
129+
* If a test fails because it uses features not yet implemented in the unified
130+
test runner, skip the corresponding test with a reference to the ticket that
131+
covers implementing the new features
132+
* If the test failure points to a bug in the spec, consider the effort required
133+
to fix the failure. If it's a small change, commit and push the fix directly
134+
to the pull request. Otherwise, skip the test with a reference to a ticket to
135+
fix the failing test.
136+
137+
The goal is that the library passes tests with the latest spec version at all
138+
times, either by implementing small changes quickly, or by skipping tests as
139+
necessary.
140+
112141
## Code quality
113142

114143
Before submitting a pull request, please ensure that your code adheres to the

0 commit comments

Comments
 (0)