-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathlychee.toml
More file actions
218 lines (212 loc) · 12.2 KB
/
Copy pathlychee.toml
File metadata and controls
218 lines (212 loc) · 12.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
# Dead-link checking for the documentation and the built site.
#
# Two link dialects live in this repo and neither is a plain filesystem path, so
# both are excluded here and checked by the pass that can actually resolve them:
#
# @/docs/config.md Zola's internal-link syntax. Resolved by Zola at build
# time against content/, and Zola FAILS THE BUILD on a
# broken one -- so these are already gated, by the tool that
# defines them. lychee reads them as relative paths and
# looks for `website/content/docs/@/docs/config.md`, which
# produced 107 phantom errors on the first run.
#
# /docs/install/ Site-absolute URLs, in website/content/ only. They resolve
# against the built output rather than the Markdown tree, so
# the second CI pass checks them over website/public with
# --root-dir set. They CANNOT be handled with an `exclude`
# pattern: lychee reports "Cannot resolve root-relative
# link" during resolution, before there is a URI for a
# pattern to match, so an exclude entry for them is inert.
# That is why pass one takes docs/ and the repo-root
# Markdown and leaves website/content/ to pass two.
#
# Working documents are skipped for the same reason Vale skips them: internal
# audience, and they cite scratch paths and future work that was never meant to
# resolve.
max_concurrency = 8
# 30s, raised from 20 after kamailio.org timed out a release commit's Quality
# job with ZERO dead links -- one timeout was enough to fail the run.
#
# SUPERSEDED for that host, and kept because the value still serves every other
# slow page. kamailio.org later moved from slow to REFUSING -- "Connection
# reset by peer" -- which no timeout fixes, and it now sits in `exclude` with
# that evidence. Read the two together: this note explains the number, the
# exclusion explains why the number stopped being enough.
#
# Not tuned away: the page is alive and merely slow. Measured three times in a
# row from here, back to back: 18.5s, 9.2s, 9.0s, all HTTP 200. A first hit
# landing at 18.5s against a 20s ceiling is not a transient blip, it is a
# margin of 1.5s on a link the docs legitimately cite, and a retry does not
# help when the SLOW path is the one being retried.
timeout = 30
# Raised from 2/2. This widens the window for the TIMEOUT class only -- the
# kamailio.org case above -- and deliberately does not pretend to fix the
# rejected-status case that motivated the look (see bestpractices.dev below).
# lychee doubles the wait each time, so 4 retries at 3s rides out about 45s
# (3 + 6 + 12 + 24) where 2 at 2s gave 6s. Retries cost nothing on a healthy
# run: they happen only after a failure.
max_retries = 4
retry_wait_time = 3
# 429 is rate limiting, not a dead link -- GitHub and crates.io both do it under
# CI concurrency. 206 is partial content, which some CDNs answer HEAD with.
#
# 502, 503 and 504 join them, and the reason is the same one 429 is here for.
# On 2026-08-11 a run checked 5,914 links, 5,785 passed, and the single failure
# was github.com answering 503 for one of ITS OWN blob URLs. Later the SAME DAY,
# on the 0.5.93 release commit, github.com answered 504 Gateway Timeout for two
# more of its own blob URLs (src/error.rs, with and without a #L fragment) and
# turned main red again — after 503 had been handled. Every one of those links
# was valid before, during and after. lychee's retry does not rescue them:
# `max_retries` governs transport-level failures, while a status in neither
# `accept` nor the retryable set is a verdict, and only 429 is retried among
# rejected codes.
#
# 502 is listed with them though it has not been observed here. All three are
# one condition — an intermediary could not get an answer from the origin —
# and admitting them one funeral at a time means each new variant costs another
# red main and another release held up. The class is what is being accepted,
# not three separate incidents.
#
# The risk is small and worth naming: a permanently 5xx-ing host would pass.
# That is a service being down, not a link being wrong, and this gate exists to
# catch the second. A dead link answers 404, which is still rejected, as is 500
# — an origin that answered and failed is not a gateway that never reached it.
accept = [200, 206, 429, 502, 503, 504]
exclude = [
# Zola internal links -- see above. Matched on the resolved path because
# lychee turns them into file:// URIs before exclusions apply, so a pattern
# anchored with ^@/ never matches.
'/@/',
# Patreon answers GitHub-hosted runners with 403 Forbidden while serving the
# page normally elsewhere: it blocks datacenter address ranges, and a browser
# user-agent does not change it. Verified in both directions -- 308 (a live
# redirect) from a residential connection, 403 from the Actions runner on the
# same URL in the same minute. Excluded rather than adding 403 to `accept`,
# which would blind the whole run to real authorization failures for the sake
# of one sponsor link. This is the failure mode that gets link checkers
# deleted, so it is written down instead of tuned away.
'patreon\.com',
# LinkedIn answers non-browser user agents with HTTP 999 "Request Denied",
# their long-standing anti-scraping response, and rejects HEAD with 405 even
# from a browser UA. Verified three ways against the footer profile URL inside
# one minute: default UA HEAD -> 999, browser UA HEAD -> 405, browser UA GET
# -> 200. The page is live; only the automated request shapes are refused.
#
# This link is in the site FOOTER, so one denial fails every built page at
# once -- the run that caught it reported the same URL dead in 30-odd files.
# Neither status belongs in `accept`: 999 is not a real HTTP status and would
# be meaningless anywhere else, and accepting 405 site-wide would hide
# genuinely misrouted links to rescue a single footer credit.
#
# Worth knowing: this passed for weeks before failing, and the failing log
# line reads "Error (cached)". That looks like a stale cache and is the
# opposite -- a fresh failure being written to it. lychee-action caches
# results between runs and the old entry aged out, which is why this surfaced
# now rather than when the link landed in 2092d3a.
'linkedin\.com',
# The IEEE Registration Authority's EtherType public listing, cited by
# docs/encapsulations.md as the authoritative source for every EtherType value
# in it. RFC 9542 hands that question to the IEEE rather than IANA, so this is
# the primary source and not a convenience link.
#
# It timed out the Quality job with zero dead links. NOT a slow-page case, so
# raising `timeout` the way kamailio.org warranted above would not have helped:
# measured three times back to back from a residential connection, HTTP 200 in
# 0.24s, 0.24s and 0.31s for 1,863,318 bytes, TTFB 0.08s, and HEAD answers 200
# in 0.06s. Nothing here is slow.
#
# The runner is refused, not delayed -- the same datacenter-range treatment as
# the Patreon entry above. Some client-shape filtering also exists: an earlier
# fetch from a different HTTP client on this same machine got HTTP 418, while
# curl gets 200 with both a default and a browser user-agent, so it is not a
# simple user-agent block either.
#
# Excluded rather than delinked. The value of citing it is that a reader can
# check the numbers against the body that assigns them; dropping the URL to
# satisfy a link checker would remove the provenance the page exists to carry.
'standards-oui\.ieee\.org',
# Kamailio's module documentation, cited by docs/mcp-estate.md for the
# siptrace modparams an operator has to set. The `timeout` note above raised
# 20 -> 30 for this host and recorded the reason: the page is alive and
# merely slow. That reasoning no longer covers what it does now.
#
# The failure CLASS moved. Two consecutive Quality runs on main reported
# different ones -- 1 TIMEOUT / 0 errors, then 0 timeouts / 1 ERROR
# "Connection reset by peer (os error 104)". Measured from two machines
# inside minutes: HTTP 200 in 28.7s (1.3s inside the 30s ceiling), a timeout
# at 27.9s, a connection failure in 0.1s, and a curl that never returned.
#
# No timeout value fixes a reset, which is why this is an exclusion and not
# another bump. Three straight Quality failures on main with ZERO dead links
# in the repository, and `pre-push` refuses a `v*` tag whose commit has a
# failed run -- so a live host's bad week would have blocked the next
# release.
#
# Excluded rather than delinked, for the same reason as the IEEE entry: the
# page is the canonical documentation for the modparams the surrounding
# prose tells the reader to set, and dropping the URL to satisfy a link
# checker would take that away from the reader to buy a green run.
'kamailio\.org',
# The OpenSSF Best Practices badge, cited in README.md and in
# docs/design/openssf-badge-answers.md. It answered a Quality job with 503
# Service Unavailable on four links at once and failed the run with ZERO dead
# links in the repository -- 1842 of 1851 successful, every error that one
# host. It was healthy from here minutes later: five consecutive fetches of
# the failing URL, HTTP 200 every time, 0.12-0.20s, two redirects each.
#
# This is the one case above where raising `max_retries` looks like the
# obvious fix and is inert. lychee only retries a rejected status code when it
# is 429 (lychee-lib/src/retry.rs):
#
# impl RetryExt for ErrorKind {
# fn should_retry(&self) -> bool {
# ...
# } else {
# matches!(self, Self::RejectedStatusCode(StatusCode::TOO_MANY_REQUESTS))
# }
#
# A 503 that arrives as a completed HTTP response becomes
# `RejectedStatusCode(503)` and falls into that arm. The `is_server_error()`
# branch in the same file belongs to the `reqwest::StatusCode` impl, which is
# reached for transport-level failures, not for a status lychee rejected
# against `accept`. The CI log wording -- "Rejected status code: 503" -- names
# the variant. Retrying more would have changed nothing.
#
# Adding 503 to `accept` was the other option and is worse: it would blind
# every link in the repository to server errors, which is most of how a page
# dies, to rescue one badge.
#
# What this costs, stated plainly because it is a real loss and not a free
# win: nothing now checks that this URL resolves, so a changed project ID or a
# retired badge page would go unnoticed here. Unlike the three entries above,
# the host is not refusing the runner -- it was simply down -- so this is an
# exclusion for availability, not for access. Reverse it if lychee grows
# per-host accepts or retries rejected 5xx.
'bestpractices\.dev',
]
exclude_path = [
"docs/superpowers",
"docs/research",
"target",
"website/public/mermaid.min.js",
]
# Mail addresses in the security policy and config docs are illustrative or
# already correct; there is nothing to reach over the network to confirm.
# (The key is `include_mail`, defaulting off -- `exclude_mail` is not a field,
# and lychee rejects the whole config rather than ignoring an unknown one.)
include_mail = false
# Fragment checking (`file.md#anchor`) is intentionally OFF, and not because it
# would be noisy. Anchors are already gated by link_integrity_test --
# generated_site_anchors_resolve_under_zola, website_intra_docs_links_resolve and
# wiki_intra_docs_links_resolve -- which slugify headings the way Zola and the
# GitHub wiki do, i.e. the way the places these documents are published do.
#
# lychee's slugger disagrees on headings containing a slash: for
# `### GET /v1/dialogs` it wants `get-v1dialogs`, where Zola resolves
# `get-v1-dialogs`. Enabled here it reported the repo's correct anchor as dead,
# and changing the anchor to satisfy it broke all three of those gates at once.
# Two sluggers, one of which is not used to publish anything.
#
# (Also worth knowing if you try: lychee 0.24.2 rejects `include_fragments` as a
# config key while accepting --include-fragments as a flag, and a rejected key
# fails the whole config rather than being ignored.)