Repository navigation
feat(url): read JavaScript pages and refuse challenges honestly - #67
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--urlno longer reports success on JavaScript shells and captcha pages. It refuses them with a plain reason, the HTTP status and the challenge vendor.--render-js autoruns JavaScript only when the static page is a shell. JS capture falls back to the fetched HTML on heap, DOM-guard and timer-quota limits, and the soft window now also bounds script and module loading. Legacy and mislabelled charsets decode with the source recorded in the receipt. Link records become standard Markdown, and--no-linksdrops destinations. The receipt staysbrewdoc.receipt/2with additive fields. Version 0.7.0. Closes #66.Live bench, 54 URLs (static / auto / always): 39 / 40 / 38 successes and 0 false successes (0.6.0: 6). Median latency 0.65 / 0.75 / 2.74 s. JS maximum 13.7 s, down from 28-33 s. hn.algolia recovers in auto mode. acloserlisten and therecoup convert (TLS ALPN
http/1.1). The heap and DOM fallbacks recover the code.claude.com changelog and yakutia-daily.Validation: full suite on macOS and in a linux/arm64 container (all tests, including Linux-only native suites), self-check, npm launcher test, and wheel/sdist installs on CPython 3.12-3.14 all pass locally.
Still open: the macOS Python trust store lacks GlobalSign Root R46 (3 bench pages), and pages that need more than the bounded engine (crates.io, dvnovosti) refuse at
javascript_required.