[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-06-27

[github-actions[bot]]

Summary

• Date: 2026-06-27 | Pages visited: [Home]((localhost/redacted) [Quick Start]((localhost/redacted) [CLI Commands]((localhost/redacted)
• Overall: Docs look professional with clear structure, but several Quick Start steps assume technical knowledge (PAT creation, GitHub Actions scopes, frontmatter) that blocks true beginners well before the 10-minute estimate.

---

🔴 Critical Issues

1. COPILOT_GITHUB_TOKEN setup is confusing and buried

Page: Quick Start → Step 2

The Note callout explains creating a fine-grained PAT with Copilot Requests → Read permission, but the GitHub UI path is non-obvious. Worse, gh secret set COPILOT_GITHUB_TOKEN < /path/to/token.txt assumes the token was saved to a file — a beginner will have it on their clipboard and be stuck.

📎 [copilot-token-note.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/90d9554f4e468d9261b54c242c60dd2520bee91e3b3f4e8e17145d239784d063.png?raw=true

Fix: Use gh secret set COPILOT_GITHUB_TOKEN without the file redirect — the CLI prompts interactively. Add a dedicated Authentication page with screenshots.

2. Prerequisites don't link to AI provider signups

Page: Quick Start → Prerequisites

Lists Copilot, Claude, Codex, Gemini but only Copilot gets a helpful note. No links for Claude/Codex/Gemini account setup.

📎 [prerequisites.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/6942dbda8406ba529dc6cbe38ee8126e49b7e63ba28c2205eee9c97f4f0aec00.png?raw=true

Fix: Add signup links for each provider, or add a "Which AI engine should I pick?" callout recommending Copilot for existing GitHub users.

3. gh aw init vs gh aw add-wizard — confusing entry point

CLI Commands page lists gh aw init first ("Set up repository") but Quick Start uses gh aw add-wizard. A beginner will wonder if they missed a step.

📎 [cli-commands.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/2dcd23d5531ba37472f81f2d4e537d6d97cbcf8b58dee4fd416bf81c2376854a.png?raw=true

---

🟡 Confusing Areas

4. "Frontmatter" undefined in Step 4

"If you changed the frontmatter (the configuration block between the --- markers...)" — first use of the term without a proper definition or link.

📎 [step4-frontmatter.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/db8313064eae081ad4b9894be21c1efc482071409502e19aa02a2cb90b9acf53.png?raw=true

5. add-wizard format explained after the command

gh aw add-wizard githubnext/agentics/daily-repo-status is shown before explaining what the <owner>/<repo>/<workflow-name> parts mean. Swap explanation before the command, or annotate with comments.

📎 [step2-add-wizard.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/f4721cdf201555ee90aa45b48e9cb85fdbce62e8018e8d76fa9f009177dce27d.png?raw=true

6. Lock file concept raises more questions than it answers

"The .lock.yml is the compiled GitHub Actions workflow..." — do I commit both files? What if they get out of sync? Add TL;DR: "Think of .lock.yml like a compiled binary — the system manages it automatically, just commit it alongside the .md file."

7. Home hero assumes familiarity with "coding agents"

"Run the coding agents you know and love" assumes the reader knows what that means. Add a one-sentence definition for newcomers.

📎 [home-viewport.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/eb8a79e9cc9168cc2f117fcbbf1d7c4267d76024d0093be26249853e9109b18c.png?raw=true

---

🟢 What Worked Well

• Clear 4-step structure with estimated time (10 min)
• "Most Common Commands" table at top of CLI page — excellent
• Example output image showing what a completed report looks like
• Alt installer (curl | bash) as a safety net
• Quick Start reachable from multiple nav paths

---

Recommendations

Quick wins: (1) Use interactive gh secret set TOKEN without file redirect. (2) Define frontmatter inline on first use. (3) Annotate add-wizard command parts with comments. (4) Add provider signup links in prerequisites.

Medium-term: (5) Dedicated "Setting Up Your AI Engine" page with screenshots for all 4 providers. (6) Beginner-friendly hero tagline defining "coding agent". (7) Collapse advanced CLI topics (version pinning, GHES) into <details> blocks.

Longer-term: (8) Inline troubleshooting in Quick Start for top 3 errors. (9) Short video walkthrough of the Quick Start flow.

---

Automated noob test · §28279301367

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · 87.7 AIC · ⌖ 19.2 AIC · ⊞ 6.5K · ◷

• expires on Jun 27, 2026, 9:12 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-06-28T05:12:20.194Z.

Closed by Workflow