Skip to content

Add shopify doc fetch command to download MD doc from shopify.dev#7766

Merged
isaacroldan merged 11 commits into
mainfrom
add-fetch-doc-command
Jun 23, 2026
Merged

Add shopify doc fetch command to download MD doc from shopify.dev#7766
isaacroldan merged 11 commits into
mainfrom
add-fetch-doc-command

Conversation

@nelsonwittwer

@nelsonwittwer nelsonwittwer commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

What

Adds shopify doc fetch --url <url>, which downloads a complete document from shopify.dev and prints it to stdout (or writes it to a file with --output).

This is part of the new doc namespace for documentation tooling — the "full document" half. The "discovery" half is doc search (#7770).

Why

Every page on shopify.dev has a Markdown representation. doc fetch requests that representation, giving agents (and scripts) a clean, verbatim way to pull instructional content from the centralized docs — for example, a set of instructions an agent follows like a centrally-served skill.

Details

  • shopify doc fetch --url <url> — requests the Markdown version of the page.
  • --output <path> (-o) — write to a file instead of stdout.
  • Only shopify.dev (and subdomains) URLs are allowed; malformed/disallowed URLs and non-ok responses throw before any output.
shopify doc fetch --url https://shopify.dev/docs/api/shopify-cli
shopify doc fetch --url https://shopify.dev/docs/api/shopify-cli --output docs/shopify-cli.md

🎩 Testing instructions

doc fetch hits a public endpoint on shopify.dev — no auth, no app, no login required. It just needs network access.

# 1. Check out this branch and build (dev.js runs the prebuilt bundle, so a build is required)
cd <your cli checkout>
git checkout add-fetch-doc-command && git pull
pnpm nx build cli

Run commands from the repo root via node packages/cli/bin/dev.js …:

1. Fetch a real page → prints Markdown to stdout

node packages/cli/bin/dev.js doc fetch --url https://shopify.dev/docs/api/shopify-cli | head -20
#   → Markdown content of the page

2. --output writes to a file (creating missing parent dirs)

node packages/cli/bin/dev.js doc fetch --url https://shopify.dev/docs/api/shopify-cli --output /tmp/doc-fetch/cli.md
head -5 /tmp/doc-fetch/cli.md
#   → "Saved …" message, and the file exists with the Markdown content

3. Disallowed host → rejected before any network call

node packages/cli/bin/dev.js doc fetch --url https://example.com/whatever
#   → "Only documents from the following hosts can be fetched: shopify.dev."

4. Malformed URL → rejected

node packages/cli/bin/dev.js doc fetch --url not-a-url
#   → "Invalid URL: not-a-url"

5. Non-ok response surfaces status

node packages/cli/bin/dev.js doc fetch --url https://shopify.dev/this-page-does-not-exist-xyz
#   → "Failed to fetch …: 404 Not Found"  (exit non-zero)

6. Required arg + help

node packages/cli/bin/dev.js doc fetch          # → "Missing required flag url"
node packages/cli/bin/dev.js doc fetch --help   # → usage, description, examples, --output flag

Inspecting the Monorail analytics payload

This branch is rebased on the app-context telemetry PR (#7771), so every command emits a command-analytics event. To see exactly what would be sent to Monorail — without sending it — set two env vars:

  • SHOPIFY_CLI_NO_ANALYTICS=1 — dump the payload via debug logging instead of POSTing it.
  • SHOPIFY_FLAG_VERBOSE=1 — raise the log level so the debug line prints.
SHOPIFY_CLI_NO_ANALYTICS=1 SHOPIFY_FLAG_VERBOSE=1 \
  node packages/cli/bin/dev.js doc fetch --url https://shopify.dev/docs/api/shopify-cli 2>&1 \
  | grep "Skipping command analytics"
#   → a "Skipping command analytics, payload: {…}" line with "command": "doc fetch"

To confirm the inherited app-context hook, run the same command from inside an app directory while logged in — the payload then also carries app-level identifiers (api_key, project_type, app_*). Run it from a non-app directory and those fields are absent.

Notes

  • @shopify/cli: minor.
  • Lives under the doc topic (commands/doc/fetch.ts, registered as doc:fetch); adds a doc topic description.
  • Generated artifacts regenerated (oclif manifest, README, dev-docs interface, e2e command-tree snapshot).
  • Rebased on telemetry-app-context-everywhere (Capture app context in analytics for every command run inside an app #7771), so it inherits the app-context telemetry hook automatically.

Related

🤖 Generated with Claude Code

@nelsonwittwer nelsonwittwer requested review from a team as code owners June 9, 2026 14:15
@github-actions github-actions Bot added the Area: @shopify/cli @shopify/cli package issues label Jun 9, 2026
Comment thread packages/cli/src/cli/commands/fetch.ts Outdated
Comment thread packages/cli/src/cli/services/commands/fetch.ts Outdated
@nelsonwittwer nelsonwittwer marked this pull request as draft June 9, 2026 14:59
Comment thread docs-shopify.dev/commands/interfaces/doc-fetch.interface.ts
Comment thread packages/cli/src/cli/services/commands/fetch-doc.ts Outdated
@nelsonwittwer nelsonwittwer force-pushed the add-fetch-doc-command branch from 60b956c to 32e22ff Compare June 9, 2026 20:06
@nelsonwittwer nelsonwittwer changed the base branch from main to telemetry-app-context-everywhere June 9, 2026 20:12
@nelsonwittwer nelsonwittwer changed the title [Feature] Add shopify fetch command to download docs from shopify.dev Add doc fetch command (download docs from shopify.dev) Jun 10, 2026
@nelsonwittwer nelsonwittwer changed the title Add doc fetch command (download docs from shopify.dev) Add shopify doc fetch command to download MD doc from shopify.dev Jun 15, 2026
@nelsonwittwer nelsonwittwer force-pushed the add-fetch-doc-command branch from 85aa853 to 1a95946 Compare June 22, 2026 17:08
@nelsonwittwer nelsonwittwer marked this pull request as ready for review June 22, 2026 18:41
@nelsonwittwer nelsonwittwer force-pushed the add-fetch-doc-command branch from d8e0bc6 to 45f9b37 Compare June 22, 2026 20:24
@nelsonwittwer nelsonwittwer changed the base branch from telemetry-app-context-everywhere to main June 22, 2026 20:25
@github-actions

Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/session.d.ts
@@ -47,6 +47,22 @@ export declare function isUserAccount(account: AccountInfo): account is UserAcco
  * @returns True if the account is a ServiceAccount.
  */
 export declare function isServiceAccount(account: AccountInfo): account is ServiceAccountInfo;
+/**
+ * Reports whether the CLI already has stored credentials, without prompting the
+ * user, opening a browser, or making any network request.
+ *
+ * This is a passive, side-effect-free check: it reads the local session store and
+ * returns  when at least one valid session is present. Unlike the
+ *  functions, it never triggers a login flow and never logs
+ * the user out. Because it does not contact the network, it cannot tell whether the
+ * stored token is currently valid/unexpired — only that credentials exist locally.
+ *
+ * Intended for best-effort, opportunistic behaviour (for example, enriching
+ * telemetry only for users who are already logged in).
+ *
+ * @returns True if local credentials exist, false otherwise.
+ */
+export declare function sessionExists(): Promise<boolean>;
 /**
  * Ensure that we have a valid session with no particular scopes.
  *

nelsonwittwer and others added 10 commits June 23, 2026 13:02
Adds a top-level `fetch` command that downloads a shopify.dev page and
prints it to stdout. It requests the Markdown representation by default
(via the `Accept` header) and accepts a `--content-type` flag to override
it. URLs are restricted to shopify.dev. This gives agents a way to pull
instructional content from the centralized docs repo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Rename command/class from `fetch`/`Fetch` to `fetch-doc`/`FetchDoc`
  (and the service to `fetchDocService`) to avoid an overly generic name.
- Replace the hardcoded shopify.dev host check with an extensible
  `ALLOWED_HOSTS` array constant.
- Regenerate oclif manifest, README, and shopify.dev docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Distinguish the two docs commands by intent:
- search: discovery — surface the most relevant pieces of content.
- fetch-doc: retrieve a complete document verbatim, like a
  centrally-served skill an agent follows in its entirety.

Each description cross-references the other so agents pick the right tool.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ery page

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These commands previously rejected the CLI's standard global flags. Adding
`globalFlags` makes them consistent with the rest of the CLI and lets
`--verbose` surface the Monorail analytics payload for local verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- `--output <path>` (`-o`) writes the document to a file (creating any
  missing parent directories) instead of printing to stdout.
- Remove the `--content-type` flag: fetch-doc now always requests the
  Markdown representation. Returning HTML is noisy and expensive and not
  a behavior we want to encourage.
- Regenerate oclif manifest, README, and shopify.dev docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the doc-namespace plan, documentation commands live under a `doc`
topic. Rename `fetch-doc` → `doc fetch`:

- commands/doc/fetch.ts (class DocFetch), services/commands/doc/fetch.ts
  (docFetchService), and the colocated test.
- Register as `doc:fetch`; add a `doc` topic description.
- Update cross-references (`search` now points to `doc fetch`; `doc fetch`
  points to `doc search`).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Identifies the CLI as the calling surface to shopify.dev so traffic
originating from the CLI can be attributed as such.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the CLI design guidelines: flags make intent explicit, don't rely
on order, and shouldn't be shortened — so the URL is now --url and the
-o alias for --output is dropped. Regenerated manifest + README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@isaacroldan isaacroldan force-pushed the add-fetch-doc-command branch from fc084c2 to 99313e9 Compare June 23, 2026 11:05
Assisted-By: devx/214c76e8-6ac0-48f1-bff5-0ef68c95d938
@isaacroldan isaacroldan force-pushed the add-fetch-doc-command branch from 99313e9 to fbeae30 Compare June 23, 2026 11:10
@isaacroldan isaacroldan added this pull request to the merge queue Jun 23, 2026
Merged via the queue into main with commit b9665e9 Jun 23, 2026
28 checks passed
@isaacroldan isaacroldan deleted the add-fetch-doc-command branch June 23, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: @shopify/cli @shopify/cli package issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants