Skip to content

fix(route/cw): Use browser to fetch full text.#22292

Open
dzx-dzx wants to merge 4 commits into
DIYgod:masterfrom
dzx-dzx:cw-0607
Open

fix(route/cw): Use browser to fetch full text.#22292
dzx-dzx wants to merge 4 commits into
DIYgod:masterfrom
dzx-dzx:cw-0607

Conversation

@dzx-dzx

@dzx-dzx dzx-dzx commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Involved Issue / 该 PR 相关 Issue

Close #

Example for the Proposed Route(s) / 路由地址示例

/cw/today

New RSS Route Checklist / 新 RSS 路由检查表

  • New Route / 新的路由
  • Anti-bot or rate limit / 反爬/频率限制
    • If yes, do your code reflect this sign? / 如果有, 是否有对应的措施?
  • Date and time / 日期和时间
    • Parsed / 可以解析
    • Correct time zone / 时区正确
  • New package added / 添加了新的包
  • Puppeteer

Note / 说明

Copilot AI review requested due to automatic review settings June 16, 2026 09:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added the route label Jun 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Auto Review

No clear rule violations found in the current diff.

@github-actions github-actions Bot added the auto: not ready to review Users can't get the RSS feed output according to automated testing results label Jun 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Successfully generated as following:

http://localhost:1200/cw/today - Failed ❌
HTTPError: Response code 503 (Service Unavailable)

Error Message:<br/>TypeError: e.author is not iterable
Route: /cw/today
Full Route: /cw/today
Node Version: v24.16.0
Git Hash: 946c3b3e

@github-actions github-actions Bot added auto: not ready to review Users can't get the RSS feed output according to automated testing results and removed auto: not ready to review Users can't get the RSS feed output according to automated testing results labels Jun 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Successfully generated as following:

http://localhost:1200/cw/today - Failed ❌
HTTPError: Response code 503 (Service Unavailable)

Error Message:<br/>TypeError: r is not a function
Route: /cw/today
Full Route: /cw/today
Node Version: v24.17.0
Git Hash: f9b32231

@github-actions

Copy link
Copy Markdown
Contributor

Auto Review

  • Rule 43 (lib/routes/cw/utils.ts): parseItems creates a new browser page per item inside Promise.all(list.map(...)), opening multiple pages concurrently. This wastes resources and can cause instability. Fix: Process items sequentially with a for...of loop, or add a concurrency limit (e.g., process in batches).

@github-actions github-actions Bot added auto: not ready to review Users can't get the RSS feed output according to automated testing results and removed auto: not ready to review Users can't get the RSS feed output according to automated testing results labels Jun 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Successfully generated as following:

http://localhost:1200/cw/today - Failed ❌
HTTPError: Response code 503 (Service Unavailable)

Error Message:<br/>TypeError: r is not a function
Route: /cw/today
Full Route: /cw/today
Node Version: v24.17.0
Git Hash: d7b8c61d

@github-actions github-actions Bot added auto: not ready to review Users can't get the RSS feed output according to automated testing results and removed auto: not ready to review Users can't get the RSS feed output according to automated testing results labels Jun 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Successfully generated as following:

http://localhost:1200/cw/today - Failed ❌
HTTPError: Response code 503 (Service Unavailable)

Error Message:<br/>Error: page.evaluate: Target page, context or browser has been closed
Route: /cw/today
Full Route: /cw/today
Node Version: v24.17.0
Git Hash: ff7dfbc0

@github-actions

Copy link
Copy Markdown
Contributor

Auto Review

  • Rule 41 (lib/routes/cw/utils.ts:97-101): New browser pages in parseItems have no request type filtering. Other Puppeteer usage in this same file (getCookie, parsePage) correctly adds page.route('**/*', ...) to only allow document and script resources, but the pages created for fetching article details will load all resource types (images, stylesheets, fonts, etc.), wasting resources. Fix: Add request type filtering before page.goto():
const page = await context.newPage();
await page.route('**/*', (route) => {
    const request = route.request();
    request.resourceType() === 'document' || request.resourceType() === 'script' ? route.continue() : route.abort();
});

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto: not ready to review Users can't get the RSS feed output according to automated testing results route

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants