mirror of https://git.openapi.site/https://github.com/desirecore/market.git synced 2026-09-05 22:04:07 +08:00

Files

Yige 6538f01555 docs(web-access): sync site-pattern baselines off the retired Playwright fallback (#86 )

## What / 变更内容

S31（#85）把 `web-access` 技能升到 v3.0.0，删掉了 `SKILL.md` 与
`references/browser-tools.md`
里「用户手工启动带 `--remote-debugging-port` 的 Chrome + Python Playwright」的回落路径，但
**5 个 per-site 基线文件未同步**，仍然引导 Agent 在正文抽取（以及部分站点的登录态复用）上
回落到已废弃的 Playwright / 外部 CDP：

- `references/site-patterns/zhihu.com.md`
- `references/site-patterns/weibo.com.md`
- `references/site-patterns/bilibili.com.md`
- `references/site-patterns/xiaohongshu.com.md`
- `references/site-patterns/feishu.cn.md`

本 PR 把这 5 处旧表述替换为已落地的内置浏览器能力：

- 正文抽取 → `BrowserAct(page.extract-text)`（format/maxBytes/nextCursor 分页）
- 知乎/B站的「批量回答/评论」抽取 → `BrowserAct(page.element, op: all-inner-texts,
selector: loc=css:...)`
  （九 op 判别式命令的只读档，一次拿全部匹配元素文本）
- 登录态未授予 `browser.import.*` 时的处置，由「直接回落 / 回落 CDP」改为「如实告知用户无法复用
登录态，按未登录继续或放弃」，与 `references/browser-tools.md` 里 `BrowserImport` 一节已修好的
  措辞对齐
- 各文件 `历史更新` 追加一条同步记录；`SKILL.md` 版本号 `3.0.0` → `3.0.1`
（只改 `references/*`，未改 `SKILL.md`/`SKILL.zh-CN.md` 正文，`source_hash` 不受影响）

`references/browser-tools.md` 现存的 2 处 "Playwright" 字样经核实均为**已移除**的历史性陈述
（"…并移除 Python Playwright 回落"、"无需…Python / Playwright"），不构成误导性指令，本 PR
未改动该文件。

Refs: desirecore/desirecore#2074, #85

## Test plan / 测试计划

- [x] 通读改动后的每个 site-pattern 文件：新写法对应的能力（`page.extract-text` /
`page.element`
      op: `all-inner-texts`）均已在 v3.0 落地，未引用不存在的参数
- [x] `compute_source_hash` 只 hash `SKILL.md` frontmatter 之后的 body +
i18n strings，本次只改
      `references/*.md`，不触发 source_hash drift
- [ ] CI（i18n validate）

- [x] CLA

2026-08-19 20:22:27 +08:00

25 KiB

Raw Blame History

name, description, license, version, type, risk_level, status, disable-model-invocation, tags, provides, metadata, market

name

description

license

version

type

risk_level

status

disable-model-invocation

web-access skill

L0: One-line Summary

A three-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, and reach, interact with, and read login-gated sites through the governed built-in browser (v3.0 ships bulk text extraction, waits, and code mode in-browser; the Python Playwright fallback is gone).

L1: Overview & Use Cases

Capability

web-access is a procedural skill that provides three complementary layers of web access:

L1 (WebSearch + WebFetch): public, static pages
L2 (Jina Reader): JS-rendered heavy pages, saving tokens by default
L3 (governed built-in browser, capability surface completed in v3.0): reach, interact with, and read logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. Bulk text extraction (page.extract-text), discriminated waits (page.wait), and code mode (BrowserScript) all close the loop inside this layer

The v2.x fourth layer — "user manually launches a debug Chrome + Python Playwright CDP" — was removed in v3.0: every reason it existed for (no bulk text channel, evaluate unusable, screenshots must activate-serialize) is now covered by the built-in browser, see the cheatsheet below.

v3.0: governed built-in browser (default-hidden, exposed only after Skill activation)

When you call Skill('web-access'), the following 9 tools are injected into the current session so the LLM can drive the built-in browser directly:

Tool	Purpose
BrowserManage	Create/destroy isolated BrowserSpace, start sessions, manage tabs
BrowserSnapshot	`semantic` / `text` / `accessibility` / `visual` snapshots — the primary way to read a page
BrowserAct	One governed action per call: navigate, input, extract text, wait, element ops, screenshot, …
BrowserScript	Code mode: one async JS script issues browser commands back-to-back, eliminating per-action round trips (trust level equals Bash)
BrowserImport	Import Cookies from the user's Chrome/Edge/Firefox/Safari profile (human-approved; *needs `browser.import.` granted by the Host — not available to a plain `create_space` session**)
BrowserShare	Delegate a Space/Session to another Agent (isolated / snapshot / copy-on-write / live)
SitePatternRead / SitePatternWrite	Per-domain "site experience" (AgentFS three-layer)
LocalBookmarks	Search local Chrome bookmarks / history

Important

: before Skill('web-access') is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See references/browser-tools.md.

Removed in v2.1: BrowserListTabs / BrowserNavigate / BrowserEval / BrowserClick / BrowserScreenshot / BrowserScroll / BrowserSetFiles / BrowserCloseTab and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+; page.extract-text / page.element / page.wait / inline wait blocks / cross-origin iframe snapshots need v10.0.112+, and BrowserScript needs a build containing S17/S18.

Use Cases

The user needs to search for current information or research a specific topic
The user needs to fetch public web content or technical documentation
The user needs to access logged-in sites (Xiaohongshu, Bilibili, Weibo, Feishu, Twitter, etc.) and read the body text
The user needs to pull data from a site's own API in a logged-in context (lists, comments, orders, …)
The user needs to compare products, aggregate news, or investigate API/library versions

Core Value

Three-layer progression: from lightweight search to heavy JS rendering to logged-in access — pick on demand
Token optimization: Jina Reader cuts token usage by 50–80% by default; page.extract-text's maxBytes/cursor paging keeps even long logged-in articles under control
Logged-in session reuse: where the Host has granted browser.import.*, BrowserImport brings the user's Cookies into an isolated Space — no re-login required
Zero external dependencies: no Python/Playwright install, no manually launched debug Chrome

L2: Detailed Specification

Output Rule

When you complete a research task, you MUST cite all source URLs in your response. Distinguish between:

Quoted facts: directly from a fetched page → cite the URL
Inferences: your synthesis or analysis → mark as "(analysis/inference)"

If any fetch fails, explicitly tell the user which URL failed and which fallback you used.

Tool Selection Decision Tree

User intent
  │
  ├─ "Search for information about X" (no specific URL)
  │     └─→ WebSearch → pick top 3-5 results → fetch each (see next branches)
  │
  ├─ "Read this public page" (static HTML, docs, news)
  │     └─→ WebFetch(url) directly
  │
  ├─ "Read this heavy-JS page" (SPA, React/Vue sites, Medium, etc.)
  │     └─→ Bash: curl -sL "https://r.jina.ai/<original-url>"
  │          (Jina Reader = default for JS-rendered content, saves tokens)
  │
  ├─ "Read this login-gated page" (Xiaohongshu/Bilibili/Weibo/Feishu/Twitter/Zhihu/WeChat)
  │     └─→ BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
  │         → BrowserAct(page.extract-text)          ← body text read out directly, maxBytes/cursor paging
  │         Logged in? BrowserImport only if browser.import.* was granted
  │
  ├─ "Pull data from the site's API in a logged-in context"
  │     └─→ fetch.browser recipe: run fetch inside BrowserAct(page.evaluate) with that origin's cookies
  │
  ├─ "API documentation / GitHub / npm package info"
  │     └─→ Prefer official API endpoints over scraping HTML:
  │          - GitHub: gh api repos/owner/name
  │          - npm:    curl https://registry.npmjs.org/<pkg>
  │          - PyPI:   curl https://pypi.org/pypi/<pkg>/json
  │
  └─ "Real-time interactive task" (click, fill form, scroll, screenshot)
        └─→ built-in browser (BrowserManage → BrowserAct → BrowserSnapshot —
             see references/browser-tools.md, no Python needed)

Three-layer strategy summary

Layer	Use case	Primary tool	Token cost
L1	Public, static	`WebFetch`	Low
L2	JS-heavy, long articles, token savings	`Bash curl r.jina.ai`	Lowest (Markdown pre-cleaned)
L3	Login-gated navigation, interaction & extraction (PRIMARY)	built-in browser (BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript)	Medium

Default priority: L1 for simple public pages → L2 for heavy → L3 for login-gated (body text and in-site API data included).

Supported Sites Matrix

Site	Recommended Layer	Notes
Wikipedia, MDN, official docs	L1 WebFetch	Static, clean HTML
GitHub README, issues, PRs	`gh api` (best) → L1 WebFetch	Prefer API
Hacker News, Reddit	L1 WebFetch	Public content
Medium, Dev.to	L2 Jina Reader	JS-rendered, member gates
Twitter/X	L3 (or L2 Jina with `x.com`)	Login required for full thread
Xiaohongshu (xiaohongshu.com)	L3 built-in browser + BrowserImport	Login required; body text via page.extract-text
Bilibili (bilibili.com)	L3 built-in browser + BrowserImport	Login needed for video desc/comments
Weibo (weibo.com)	L3 built-in browser + BrowserImport	Long posts require login
Zhihu (zhihu.com)	L3 built-in browser + BrowserImport	Long articles + comments require login
Feishu Docs (feishu.cn)	L3 built-in browser + BrowserImport	Login required
WeChat Official Accounts (mp.weixin.qq.com)	L2 Jina Reader	Usually public, Jina cleans better
LinkedIn	L3 built-in browser + BrowserImport	Login wall

Tool Reference

Layer 1: WebSearch + WebFetch

WebSearch — discover URLs for an unknown topic:

WebSearch(query="latest typescript 5.5 features 2026", max_results=5)

Tips:

Include the year for time-sensitive topics
Use allowed_domains / blocked_domains to constrain

WebFetch — extract clean Markdown from a known URL:

WebFetch(url="https://example.com/article")

Tips:

Results cached for 15 min
Returns cleaned Markdown with title + URL + body
If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (built-in browser)

Layer 2: Jina Reader (default for heavy pages)

Jina Reader (r.jina.ai) is a free public proxy that renders pages server-side and returns clean Markdown. Use it as the default for any page where WebFetch produces garbled or truncated output, and as the preferred extractor for JS-heavy SPAs.

curl -sL "https://r.jina.ai/https://example.com/article"

Why Jina is the default token-saver:

Strips nav/footer/ads automatically
Handles JS-rendered SPAs
Returns 50-80% fewer tokens than raw HTML
No API key needed for basic use (~20 req/min)

See references/jina-reader.md for advanced endpoints and rate limits.

The full command surface, capability tiers, and boundaries are in references/browser-tools.md. The loop:

BrowserManage(create_space) → BrowserManage(start_session) for a sessionId + first tab
Need login state → BrowserImport (only if the Host granted browser.import.*; without that grant the user's Cookies cannot be reused — tell the user and continue without login or abort)
BrowserAct(tab.navigate) to reach the page
BrowserSnapshot(semantic) for interactive-element ref handles (cross-origin iframe elements are in the same tree with globally sequential refs)
Interact via input.* (humanized trajectories) or page.element (bulk form writes); wait for results via page.wait or inline wait blocks
Read body text via BrowserSnapshot(text) or BrowserAct(page.extract-text); pull API data via the fetch.browser recipe
BrowserManage(close_session) when done

Multi-step sequences (navigate→snapshot→click→wait→extract) can be done in one BrowserScript run, skipping the per-action IPC round trips.

L3 Cheatsheet (v3.0)

Reading a page: pick the channel by need

You need	Use	Note
Interactive elements + `ref` / `loc=` handles	`BrowserSnapshot({ mode: 'semantic' })`	Buttons/inputs/links + per-line `[ref=eN]` and (when producible) `[loc=...]` stable selectors; cross-origin iframes in the same tree
Article body text	`BrowserSnapshot({ mode: 'text' })` or `BrowserAct({ action: 'page.extract-text' })`	markdown/text formats; beyond maxBytes it truncates and hands back a nextCursor for paging — no error
Accessibility tree	`BrowserSnapshot({ mode: 'accessibility' })`	Respects `depth` (default 50, max 100) and the maxBytes budget — truncates + pages instead of failing wholesale
What the page looks like	`BrowserSnapshot({ mode: 'visual' })` or `BrowserAct({ action: 'page.screenshot' })`	Pixels land directly in the result (vision models read them in place); `clip={x,y,width,height,scale}` for element-level crops (scale up to 4) and `captureBeyondViewport` for full-page capture

Command surface at a glance

BrowserAct actions grouped by purpose (full enum in the tool schema):

tab.*: navigate / back / forward / reload / activate / close
input.*: move / click / double-click / drag / wheel / touch / pinch / key / text — humanized input (#1808: consistent UA/UA-CH identity + real trajectories); the preferred interaction channel on anti-bot sites
page.element (discriminated op × selector, nine ops): write ops fill / select-option / check / uncheck / scroll-into-view; read ops get-attribute / bounding-box / count / all-inner-texts. Selectors speak the loc= dialect or a snapshot ref (with snapshotId). fill refuses input[type=password]
page.wait (discriminated until, nine values): poll-type load / domcontentloaded / networkidle / selector / url / timeout return waited:false on timeout; event-type request / response / download throw on timeout. Default 10s, max 60s
Inline wait blocks: params.wait (isomorphic to page.wait params) on tab.navigate / input.click / input.key / page.element{op:"fill"} — one receipt completes "act→wait for result", the waiter registers before the action, no cross-IPC race
page.evaluate: { expression, awaitPromise }, return values cross as-is (over-budget results truncate with a truncated flag, never throw); capability browser.page.evaluate sits behind the human gate (allow-all mode skips the card)
page.extract-text / page.screenshot / page.wait: see the table above and the fetch.browser recipe

loc= selector dialect (S7/S9): e<N> (must carry the snapshotId of the snapshot that issued the ref), loc=css: / loc=role: / loc=text: / loc=testid:, bare CSS, composable with internal:nth/last/scope/filter. Unknown prefixes fail explicitly — never silently degrade to CSS.

Choosing the interaction channel: input.* vs page.element

On anti-bot sites (Xiaohongshu/Weibo/Bilibili etc.) always prefer input.*: it rides the #1808 humanized input pipeline (coordinate dispatch, humanized trajectories, auditable visualization) plus the identity layer that keeps UA/UA-CH free of Electron/Headless tells
page.element write ops fit bulk form filling on sites that don't detect automation: one call fills/selects/checks, far faster than per-element input.click + input.text
Red line: page.element deliberately has no click — pointer actions must go through input.*; invoking el.click() via JS bypasses the entire humanization investment and is an explicitly forbidden fallback

The right way to pull a site's own API (lists, comments, orders, any JSON) in a logged-in context: run fetch in the page context — it carries that origin's cookies automatically, stays same-origin, is bounded by Grant origins, and rides the existing page.evaluate gate. It is a wrapper usage of BrowserAct({ action: 'page.evaluate' }):

BrowserAct:
  action: page.evaluate
  params:
    expression: |
      fetch('/api/v1/comments?page=1&size=20', {
        headers: { accept: 'application/json' }
      }).then(r => r.text())
    awaitPromise: true        # default true; waits for the returned Promise to settle

Notes:

tab.navigate to any page on the site first (establishes the origin and cookies), then fire the fetch; use a relative path so it is same-origin by construction
Return values cross as-is; for large JSON take .text() and slice it yourself, or page through multiple calls
Only the current tab's origin is reachable (Grant origins constraint); for another site's API, navigate there first
page.evaluate is a human-gated capability: outside allow-all mode an approval card appears — explain the purpose to the user

Recommended flow (Xiaohongshu example)

1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
   ← an explicit list *narrows* the lease; include browser.input.pointer.wheel to scroll
     and browser.observe.snapshot to extract text
3. Reuse the login: only when the Host granted browser.import.*,
   BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }).
   Without that grant the user's cookies cannot be reused — say so, then continue
   as logged-out or abort.
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
5. BrowserSnapshot({ mode: 'semantic' })        ← interaction refs (cross-origin iframes in the same tree)
6. BrowserAct({ action: 'page.extract-text', params: { format: 'markdown' } })
   ← body text read out directly; if too long, pass back the returned nextCursor to continue
7. Need to confirm rendering → BrowserSnapshot({ mode: 'visual' }) (pixels readable in place)
8. SitePatternRead({ domain: 'xiaohongshu.com' })  ← read accumulated experience
9. At task end → BrowserManage({ action: 'close_session', sessionId })
10. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })

Site Experience Accumulation

When the task ends and you've discovered new anti-bot pitfalls, effective selectors, or platform quirks, call:

SitePatternWrite({
  domain: "xiaohongshu.com",
  scope: "agent",     // agent=shared (Git-tracked, can be published); user=private
  mode: "merge",      // merge appends; replace overwrites
  content: "## Known pitfalls\n- 2026-08: ...",
  confidence: "medium"
})

Reads use a three-layer priority order:

SitePatternRead({ domain: "xiaohongshu.com" })
  → users/<userId>/agents/<agentId>/memory/site-patterns/   (user-private)
  → agents/<agentId>/memory/site-patterns/                  (agent-shared, Git)
  → defaults/global-skills/web-access/references/site-patterns/  (global baseline, read-only)

Content containing cookies / tokens / phone numbers / emails will automatically downgrade scope='user' and notify you.

Common Workflows

Read references/workflows.md for detailed templates:

Tech docs lookup
Competitor research
News aggregation & timelines
API/library version investigation

Read references/jina-reader.md for Jina Reader positioning, rate limits, and advanced endpoints.

Read references/browser-tools.md for the full built-in browser command surface, capability tiers, and known boundaries.

Quick Workflow: Multi-Source Research

1. WebSearch(query) → 5 candidate URLs
2. Skim titles + snippets → pick 3 most relevant
3. Classify each URL by layer (L1 / L2 / L3)
4. Fetch all in parallel (single message, multiple tool calls)
5. If any fetch returns < 200 chars or garbled → retry via next layer
6. Synthesize: contradictions? consensus? outliers?
7. Report with inline [source](url) citations + a Sources list at the end

Anti-Patterns (Avoid)

❌ Using WebFetch on obviously heavy sites — Medium, Twitter, Xiaohongshu will waste tokens or fail. Jump straight to L2/L3.
❌ Fetching one URL at a time when you need 5 — batch in a single message.
❌ Trusting a single source — cross-check ≥ 2 sources for non-trivial claims.
❌ Fetching the search result page itself — WebSearch already returns snippets; fetch the actual articles.
❌ Ignoring the cache — WebFetch caches 15 min, reuse freely.
❌ Scraping when an API exists — GitHub, npm, PyPI, Wikipedia all have JSON APIs; a logged-in site's own API goes through the fetch.browser recipe.
❌ Forgetting the year in time-sensitive queries — "best AI models" returns 2023 results; "best AI models 2026" returns current.
❌ Hardcoding login credentials in scripts — login state can only come from Cookies imported via BrowserImport.
❌ Citing only after the fact — collect URLs as you fetch, not from memory afterwards.
❌ (v3.0) Using page.element for bulk interaction on anti-bot sites — it is a direct JS call that bypasses humanized trajectories; on anti-bot sites always use input.*; keep page.element for bulk form filling where the site doesn't detect automation.
❌ (v3.0) Reading body text from screenshots — page.extract-text / BrowserSnapshot(text) hand you markdown/text with paged budgets; save screenshots for layout confirmation and CAPTCHAs where you truly must look.
❌ (v3.0) Tolerating per-action round trips when BrowserScript would do — navigate→snapshot→click→wait→extract runs as one script; remember BrowserScript's trust level equals Bash and the script source passes one human approval.
❌ (v3.0) Discovering new pitfalls and not writing a site-pattern — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth SitePatternWrite(scope='agent', mode='merge').
❌ (v3.0) Writing cookies / phone numbers to scope='agent' — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.

Example Interaction

User: "Grab the contents of this Xiaohongshu note for me: https://www.xiaohongshu.com/explore/abc123"

Agent workflow:

1. Recognize → Xiaohongshu is an L3 logged-in site
2. BrowserManage(create_space + start_session)
   Need login state → if browser.import.* was granted, run the BrowserImport four steps;
   otherwise tell the user the login cannot be reused and continue with the public part
3. BrowserAct(tab.navigate → the note URL)
4. BrowserAct(page.extract-text, format: markdown)
   ← body text read out directly; page with nextCursor if over budget
5. When returning to the user:
   - Cite the original URL
   - Quote facts from the extracted text with source links
6. Tell the user: "Fetched via the built-in browser, original link: [xhs](url)"
7. BrowserManage(close_session)

25 KiB Raw Blame History Unescape Escape