[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-05-30 #35862
Replies: 3 comments
-
|
💥 WHOOSH! 🦸 The Smoke-Test Agent BURSTS through the page! KA-POW! 💢 All systems nominal — the Claude engine is fighting fit! 🛡️ Run 26675766599 reporting for duty. THWIP! 🕸️ Back to the shadows... until next time! ⚡ Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
Beta Was this translation helpful? Give feedback.
-
|
Smoke test sprite was here - beep boop, confetti in the logs. Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
Beta Was this translation helpful? Give feedback.
-
|
This discussion has been marked as outdated by Documentation Noob Tester. A newer discussion is available at Discussion #36059. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
🔴 Critical Issues
1. Prerequisites section is dense and uses unexplained acronyms
The prerequisites list mentions "GH CLI", "GitHub Actions", and version requirements but doesn't explain what these are or link to their installation guides. A true beginner who only knows they want AI-powered workflows would be lost immediately.
Affected page: Quick Start → Prerequisites
2. "Frontmatter" term used without definition
The Quick Start uses "frontmatter" (e.g., "Keep editing the
.mdfile") without defining it anywhere visible. A user who has never worked with static site generators or Jekyll will have no idea what this means.3. Lock file concept introduced without enough context
The guide mentions
.lock.ymland says "Do not edit directly — regenerate it with [command]", but never clearly explains why there are two files (.mdsource +.lock.ymlcompiled) or what a "compiled" workflow means in this context. The mental model gap is significant.4.
GITHUB_COPILOT_TOKENsetup is buried in NotesThe guide mentions a separate Copilot token is required but buries the setup instructions in collapsible note callouts. New users running the interactive setup will hit an auth failure before reaching those notes.
🟡 Confusing Areas
1. Home page hero text assumes AI workflow literacy
The hero text reads: "augment your existing, deterministic CI/CD with Continuous AI capabilities." For a beginner, terms like "deterministic CI/CD" and "Continuous AI" are jargon-heavy. A simpler hook like "Add AI to your GitHub repo in 5 minutes" would land better.
2. Navigation has too many top-level items for beginners
The nav bar shows: Quick Start, Create, Examples, Docs, FAQ, Blog, Peli's Agent Factory. A new user doesn't know what order to visit these in. There's no clear "Start here →" affordance.
3. Step 2 of Quick Start mixes too many concepts at once
Step 2 ("Add the sample workflow and trigger a run") covers: the
gh aw addcommand, theowner/repo@refformat, engine selection (Copilot/Claude/Codex/Gemini), token configuration, and file structure — all in one step. This should be split into substeps.4. CLI Commands page: "Most Common Commands" section is great but placed below the fold
The most useful section for beginners (Most Common Commands) appears after a wall of installation instructions. New users see the installation block first and may stop reading before reaching the useful command reference.
5. "Peli's Agent Factory" in the nav bar has no explanation
This link appears in the main nav with no tooltip or description. A new user has no idea what this is — it looks like a personal link that accidentally ended up in the main navigation.
🟢 What Worked Well
Recommendations
Quick Wins 🚀
frontmatter,lock file,GH CLI extension,Copilot token— even a single sentence definition inline would help..md+.lock.ymlrelationship upfront with a simple diagram:workflow.md(you edit) →gh aw compile→workflow.lock.yml(GitHub Actions runs this).Longer-Term 🗓️
Screenshots
📎 [home.png] — Home page overview
📎 [quick-start.png] — Quick Start guide
📎 [cli-commands.png] — CLI Commands page
References: §26675130908
Warning
Firewall blocked 5 domains
The following domains were blocked by the firewall during workflow execution:
accounts.google.comandroid.clients.google.comclients2.google.comsafebrowsingohttpgateway.googleapis.comwww.google.comSee Network Configuration for more information.
Beta Was this translation helpful? Give feedback.
All reactions