From af4176bbd73a24d3162942aea10bca329c373920 Mon Sep 17 00:00:00 2001 From: Yige Date: Thu, 6 Aug 2026 00:49:35 +0800 Subject: [PATCH] =?UTF-8?q?feat(web-access):=20=E8=BF=81=E7=A7=BB=E5=88=B0?= =?UTF-8?q?=E5=86=85=E7=BD=AE=E5=8F=97=E7=AE=A1=E6=B5=8F=E8=A7=88=E5=99=A8?= =?UTF-8?q?=E5=B7=A5=E5=85=B7=EF=BC=88v2.1.0=EF=BC=89=20(#72)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## 变更说明 / Description ### 中文 客户端 v10.0.98（desirecore/desirecore#1596）停用了旧的 `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其 cdp-proxy 后端，调用会直接返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。而 web-access v2.0.2 的 `provides.tools` 仍声明这批工具——技能激活后注入的是一组必然失败的工具。 本次把 `provides.tools` 换成统一浏览器工具，并同步正文与参考文档： - `provides.tools`：`BrowserManage` / `BrowserSnapshot` / `BrowserAct` / `BrowserImport` / `BrowserShare` + 保留 `SitePatternRead` / `SitePatternWrite` / `LocalBookmarks` - 中英文 SKILL 正文同步改写（L0 / 能力描述 / 决策树 / 四层策略表 / L3-fast 速查 / 反模式） - `references/browser-tools.md` 整篇重写为新 API + 实测边界 - 5 份站点经验（小红书 / B站 / 微博 / 知乎 / 飞书）的流程改用新工具 - 版本 2.0.2 → 2.1.0，`updated_at` 更新，i18n `source_hash` 重算 **L3-fast 的定位相应收窄**：内置浏览器负责「到达 + 交互 + 截图 + 隔离」，抽取长正文仍回落 Jina Reader（公开页）或 Playwright（登录态）——理由见下方实测。 ### English Client v10.0.98 retired the legacy `BrowserXxx` tools and the cdp-proxy behind them, so web-access v2.0.2 was injecting a set of tools that always fail. This PR migrates `provides.tools` to the unified browser tools and rewrites the body, the browser-tools reference, and the five site-pattern playbooks accordingly. L3-fast is re-scoped to navigation/interaction/screenshots; bulk text extraction still falls back to Jina Reader or Playwright. ## 测试方式 / Test Plan 在客户端 v10.0.98 + `electron-embedded` Provider 上实测： - [x] `provides.tools` 里 8 个工具 ID 全部在 builtin registry 中存在 - [x] 把本 PR 的技能装进 dev 实例，带 `skillIds:['web-access']` 驱动智能体：真实调用 `BrowserManage(create_space)` → `BrowserManage(start_session)` → `BrowserAct(tab.navigate)` → `BrowserManage(close_session)`，全部 success - [x] `scripts/i18n/validate-i18n.py` 全仓库通过（中英文标题数一致、source_hash 一致） 文档中记录的边界均来自实测，而非推测： | 边界 | 实测现象 | |------|---------| | 截图前必须 `tab.activate` | 标签页默认停在 `(-10000,-10000,1x1)`，直接截图卡满 30s deadline 并触发 `browser.host.gone`，之后全部 `BROWSER_TAB_HOST_NOT_FOUND` | | `page.evaluate` 不是取文通道 | 每次调用需人工审批；字符串/对象返回值被替换为 `[REDACTED:browser-runtime-value]`，仅 number/boolean/null 穿透 | | `accessibility` 快照真实页面不可用 | example.com 正常返回 StaticText；维基百科条目一律 `BROWSER_RESULT_TOO_LARGE`（2 MB 上限，且 `depth` 参数被宿主忽略） | | `semantic` 快照不含正文 | 只列 button / input / a 等可交互元素 | ## 风险与回滚 / Risk and rollback - 纯技能内容变更，无脚本或清单结构改动 - 需要客户端 v10.0.98+；旧客户端装到本版会拿到一组不存在的工具名（旧客户端上原本那批工具也已失效，不构成回退） - 回滚即 revert 本 PR --- skills/web-access/SKILL.md | 136 +++++++----- skills/web-access/SKILL.zh-CN.md | 97 +++++---- skills/web-access/references/browser-tools.md | 200 +++++++++++------- .../references/site-patterns/bilibili.com.md | 4 +- .../references/site-patterns/feishu.cn.md | 4 +- .../references/site-patterns/weibo.com.md | 6 +- .../site-patterns/xiaohongshu.com.md | 14 +- .../references/site-patterns/zhihu.com.md | 6 +- 8 files changed, 281 insertions(+), 186 deletions(-) diff --git a/skills/web-access/SKILL.md b/skills/web-access/SKILL.md index 5a59e74..5dcaf45 100644 --- a/skills/web-access/SKILL.md +++ b/skills/web-access/SKILL.md @@ -5,16 +5,18 @@ description: >- — searching for current information, fetching public web pages, browsing login-gated sites (微博/小红书/B站/飞书/Twitter), comparing products, researching topics, gathering documentation, or summarizing news. - This skill orchestrates three complementary layers: (1) WebSearch + WebFetch + This skill orchestrates four complementary layers: (1) WebSearch + WebFetch for public pages, (2) Jina Reader as the default token-optimization layer for - heavy/JS-rendered pages, and (3) Chrome DevTools Protocol (CDP) via Python - Playwright for login-gated sites that require the user's existing browser - session. Always cite source URLs. Use when 用户提到 联网搜索、上网查、 + heavy/JS-rendered pages, (3) the governed built-in browser (isolated + BrowserSpace + Cookie import) to reach and interact with login-gated sites, + and (4) Chrome DevTools Protocol via Python Playwright as the fallback for + bulk text extraction and complex automation. Always cite source URLs. + Use when 用户提到 联网搜索、上网查、 查资料、抓取网页、研究、调研、最新资讯、文档查询、对比、竞品、技术文档、 新闻、网址、URL、找一下、搜一下、查一下、小红书、B站、微博、飞书、Twitter、 推特、X、知乎、公众号、已登录、登录状态。 license: Complete terms in LICENSE.txt -version: 2.0.2 +version: 2.1.0 type: procedural risk_level: low status: enabled @@ -29,20 +31,17 @@ tags: - playwright provides: tools: - - BrowserListTabs - - BrowserNavigate - - BrowserEval - - BrowserClick - - BrowserScreenshot - - BrowserScroll - - BrowserSetFiles - - BrowserCloseTab + - BrowserManage + - BrowserSnapshot + - BrowserAct + - BrowserImport + - BrowserShare - SitePatternRead - SitePatternWrite - LocalBookmarks metadata: author: desirecore - updated_at: '2026-05-05' + updated_at: '2026-08-06' i18n: default_locale: en-US source_locale: zh-CN @@ -51,17 +50,17 @@ metadata: - en-US zh-CN: name: 联网访问 - short_desc: 联网搜索、网页抓取、登录态浏览器访问（CDP）、研究调研工作流 - description: 三层联网访问工具包——搜索公开页面、Jina 优化抓取、CDP 登录态浏览器访问。 + short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问、研究调研工作流 + description: 四层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问、Python Playwright CDP 兜底。 body: ./SKILL.zh-CN.md - source_hash: sha256:0ba170b3126a0823 + source_hash: sha256:2e6753fc05142e9a translated_by: human en-US: name: Web Access - short_desc: Web search, page fetching, logged-in browser access via CDP, research workflows - description: A three-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, and reach logged-in sites via Chrome CDP. + short_desc: Web search, page fetching, logged-in access via the governed built-in browser, research workflows + description: A four-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, reach logged-in sites through the governed built-in browser, and fall back to Chrome CDP. body: ./SKILL.md - source_hash: sha256:4cda9ca594fbd974 + source_hash: sha256:2e6753fc05142e9a translated_by: human market: icon: >- @@ -90,7 +89,7 @@ market: ## L0: One-line Summary -A three-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, and reach login-gated sites via Chrome CDP. +A four-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, reach login-gated sites through the governed built-in browser, and fall back to Chrome CDP. ## L1: Overview & Use Cases @@ -100,24 +99,26 @@ web-access is a **procedural skill** that provides four complementary layers of - **L1** (WebSearch + WebFetch): public, static pages - **L2** (Jina Reader): JS-rendered heavy pages, saving tokens by default -- **L3-fast** (BrowserXxx builtin tool family — **new in v2.0**): preferred for logged-in sites — zero Python dependency, in-process cdp-proxy, supports CDP real-mouse events +- **L3-fast** (governed built-in browser — **rewritten in v2.1**): reach and *interact with* logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. **Reading long article text is still L2/L3-fallback's job** — see the extraction note below - **L3-fallback** (Chrome CDP + Python Playwright): backup for complex automation (long waits, race conditions, custom in-browser scripts) -### v2.0 — BrowserXxx tool family (default-hidden, exposed only after Skill activation) +### v2.1 — governed built-in browser (default-hidden, exposed only after Skill activation) -When you call `Skill('web-access')`, the following 11 tools are injected into the current session so the LLM can drive Chrome directly: +When you call `Skill('web-access')`, the following 8 tools are injected into the current session so the LLM can drive the built-in browser directly: | Tool | Purpose | |------|---------| -| BrowserListTabs / BrowserNavigate / BrowserCloseTab | Tab management | -| BrowserEval | Run JS to extract data | -| BrowserClick (`mode: js \| real-mouse`) | Click elements; real-mouse defeats anti-bot | -| BrowserScreenshot / BrowserScroll | Screenshots, scroll to trigger lazy loading | -| BrowserSetFiles | Upload local files (requires user confirmation) | +| BrowserManage | Create/destroy isolated BrowserSpace, start sessions, manage tabs | +| BrowserSnapshot | `semantic` / `accessibility` / `visual` page snapshots — the primary way to read a page | +| BrowserAct | One governed action per call: navigate, click, type, scroll, screenshot, … | +| BrowserImport | Import Cookies from the user's Chrome/Edge/Firefox/Safari profile (human-approved) | +| BrowserShare | Delegate a Space/Session to another Agent (isolated / snapshot / copy-on-write / live) | | SitePatternRead / SitePatternWrite | Per-domain "site experience" (AgentFS three-layer) | | LocalBookmarks | Search local Chrome bookmarks / history | > **Important**: before `Skill('web-access')` is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See [references/browser-tools.md](references/browser-tools.md). +> +> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+. ### Use Cases @@ -130,7 +131,7 @@ When you call `Skill('web-access')`, the following 11 tools are injected into th - **Four-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand - **Token optimization**: Jina Reader cuts token usage by 50–80% by default -- **Logged-in session reuse**: connect to the user's already-logged-in Chrome via CDP — no re-login required +- **Logged-in session reuse**: import the user's existing Cookies into an isolated Space via BrowserImport (or attach to their Chrome via CDP in the fallback layer) — no re-login required ## L2: Detailed Specification @@ -146,7 +147,7 @@ If any fetch fails, explicitly tell the user which URL failed and which fallback ## Prerequisites: Chrome CDP Setup (for login-gated sites) -**Only required when accessing sites that need the user's login session** (Xiaohongshu / Bilibili / Weibo / Feishu / Twitter / Zhihu / WeChat Official Accounts). +**Only required for the L3-fallback layer** (Python Playwright). The L3-fast built-in browser does not need this — use `BrowserImport` to reuse the user's login instead. ### One-time setup @@ -205,10 +206,11 @@ User intent │ (Jina Reader = default for JS-rendered content, saves tokens) │ ├─ "Read this login-gated page" (Xiaohongshu/Bilibili/Weibo/Feishu/Twitter/Zhihu/WeChat) - │ └─→ 1. Verify CDP ready (curl http://localhost:9222/json/version) - │ 2. Bash: python3 script with playwright.connect_over_cdp() - │ 3. Extract content → feed to Jina Reader for clean Markdown - │ (or use BeautifulSoup directly on the raw HTML) + │ ├─→ Reach it: BrowserManage(create_space/start_session) → BrowserImport(cookies for + │ │ that domain) → BrowserAct(tab.navigate) → tab.activate + page.screenshot + │ │ to confirm you landed on the content (semantic snapshot has no body text) + │ └─→ Extract the text: verify CDP ready, then python3 playwright.connect_over_cdp() + │ → page.content() → Jina Reader / BeautifulSoup │ ├─ "API documentation / GitHub / npm package info" │ └─→ Prefer official API endpoints over scraping HTML: @@ -217,9 +219,9 @@ User intent │ - PyPI: curl https://pypi.org/pypi//json │ └─ "Real-time interactive task" (click, fill form, scroll, screenshot) - ├─→ **Default: BrowserXxx tools** (BrowserNavigate / BrowserEval / BrowserClick / BrowserScreenshot — + ├─→ **Default: built-in browser** (BrowserManage → BrowserAct → BrowserSnapshot — │ see references/browser-tools.md, no Python needed) - └─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when BrowserXxx is insufficient + └─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when the built-in browser is insufficient (e.g., complex race conditions, multi-event waits, long-running in-browser scripts) ``` @@ -229,10 +231,10 @@ User intent |-------|----------|--------------|------------| | L1 | Public, static | `WebFetch` | Low | | L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) | -| **L3-fast** | **Login-gated, interactive (PRIMARY)** | **BrowserXxx tool family** | Medium | +| **L3-fast** | **Login-gated navigation & interaction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot)** | Medium | | L3-fallback | Complex automation (race / long-wait / custom scripts) | `Bash + Python Playwright CDP` | Medium | -**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when BrowserXxx is insufficient. +**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when the built-in browser is insufficient. --- @@ -348,33 +350,49 @@ See [references/cdp-browser.md](references/cdp-browser.md) for: --- -## L3-fast: BrowserXxx Tool Cheatsheet (v2.0 recommended) +## L3-fast: Built-in Browser Cheatsheet (v2.1) **Only after you call `Skill('web-access')` will the following tools appear in `tools[]`.** | Tool | One-line example | |------|-----------------| -| `BrowserListTabs()` | List all open tabs | -| `BrowserNavigate({ url })` | Open URL in a new tab | -| `BrowserNavigate({ target, url })` | Navigate an existing tab | -| `BrowserEval({ target, expression })` | Run JS in the tab to extract structured data | -| `BrowserClick({ target, selector, mode: 'real-mouse' })` | Real-mouse mode for anti-bot-strict sites | -| `BrowserScreenshot({ target })` | Saved under ${DESIRECORE_ROOT}/screenshots/ | -| `BrowserScroll({ target, direction: 'bottom' })` | Trigger lazy loading | -| `BrowserSetFiles({ target, selector, files })` | Upload local files (**user confirmation required**) | -| `BrowserCloseTab({ target })` | Clean up temporary tabs at task end | +| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | One isolated Space per task | +| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | Start a session, returns sessionId + first tab | +| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | Tab / lifecycle management | +| `BrowserAct({ action: 'tab.navigate', params: { url } })` | Navigate the current tab | +| `BrowserSnapshot({ mode: 'semantic' })` | Read the page: interactive elements + `ref` handles | +| `BrowserAct({ action: 'input.click', params: { ref } })` | Click by snapshot `ref`, never by raw x/y | +| `BrowserAct({ action: 'input.text', params: { text } })` | Type into the focused element | +| `BrowserAct({ action: 'input.wheel', params: { deltaX: 0, deltaY: 720 } })` | Scroll to trigger lazy loading | +| `BrowserAct({ action: 'tab.activate', params: { bounds } })` → `page.screenshot` | **activate first, then screenshot** | +| `BrowserImport({ action: 'discover' \| 'plan' \| 'apply' })` | Reuse the user's login cookies (human-approved) | Full API and edge cases: see [references/browser-tools.md](references/browser-tools.md). +**How to read a page** — pick by what you need: + +| You need | Use | Note | +|----------|-----|------| +| Interactive elements + `ref` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons / inputs / links only — **no article text** | +| Article text on a small/simple page | `BrowserSnapshot({ mode: 'accessibility' })` | Returns StaticText nodes; fails with `BROWSER_RESULT_TOO_LARGE` on real content pages (2 MB result cap, and the `depth` argument is currently ignored by the host) | +| Article text on a real page | L2 Jina Reader (public) or L3-fallback Playwright (login-gated) | The built-in browser has no working bulk text-extraction channel yet | +| What the page looks like | `tab.activate` → `BrowserAct({ action: 'page.screenshot' })` | Full-page only; read it visually | + +`page.evaluate` is **not** an extraction channel: it needs human approval per call and its string/object return values come back as `[REDACTED:browser-runtime-value]`. + ### Recommended flow (Xiaohongshu example) ``` -1. BrowserListTabs() → check whether there's an already-logged-in xhs tab -2. If not → BrowserNavigate({ url: "https://www.xiaohongshu.com/explore/abc123" }) -3. BrowserEval({ target, expression: "...JSON.stringify({title, content})" }) -4. SitePatternRead({ domain: "xiaohongshu.com" }) ← read accumulated experience -5. At task end → BrowserCloseTab({ target }) -6. If you find a new pitfall → SitePatternWrite({ domain, scope: "agent", mode: "merge", content }) +1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' }) +2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] }) +3. BrowserImport({ action: 'discover' }) → plan → apply for xiaohongshu.com ← reuse login +4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } }) +5. BrowserSnapshot({ mode: 'semantic' }) ← element refs for interaction (no body text) + tab.activate + page.screenshot ← confirm the note actually rendered + → extract the note text via L3-fallback Playwright +6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience +7. At task end → BrowserManage({ action: 'close_session', sessionId }) +8. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) ``` --- @@ -446,9 +464,11 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi - ❌ **Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current. - ❌ **Hardcoding login credentials in scripts** — always rely on the user's pre-logged CDP session. - ❌ **Citing only after the fact** — collect URLs as you fetch, not from memory afterwards. -- ❌ **(v2.0) Writing Python heredoc when BrowserXxx would do** — slow, requires Python+Playwright install, and bloats context. Prefer L3-fast; fall back to Python only when BrowserXxx is insufficient (race / long-wait / custom scripts). -- ❌ **(v2.0) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`. -- ❌ **(v2.0) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer. +- ❌ **(v2.1) Writing Python heredoc when the built-in browser would do** — slow, requires Python+Playwright install, and bloats context. Prefer L3-fast; fall back to Python only when the built-in browser is insufficient (race / long-wait / custom scripts). +- ❌ **(v2.1) Calling `page.screenshot` before `tab.activate`** — tabs are parked off-screen at `(-10000,-10000,1x1)` and have no compositing surface, so the capture stalls until the 30s deadline **and the tab host is destroyed**; every later call then fails with `BROWSER_TAB_HOST_NOT_FOUND`. Always `tab.activate` with real bounds first. +- ❌ **(v2.1) Reaching for `page.evaluate` to read a page** — it needs human approval on every call, its string/object return values come back as `[REDACTED:browser-runtime-value]` (only number/boolean/null survive), and anti-debug sites stall it for tens of seconds. Read pages with `BrowserSnapshot` instead. +- ❌ **(v2.1) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`. +- ❌ **(v2.1) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer. --- diff --git a/skills/web-access/SKILL.zh-CN.md b/skills/web-access/SKILL.zh-CN.md index 8f56e6a..38ebb20 100644 --- a/skills/web-access/SKILL.zh-CN.md +++ b/skills/web-access/SKILL.zh-CN.md @@ -4,7 +4,7 @@ ## L0：一句话摘要 -四层联网访问工具包——搜索公开页面、Jina 优化抓取、BrowserXxx 内置工具家族（v2.0）、Python Playwright CDP 兜底。 +四层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问（v2.1）、Python Playwright CDP 兜底。 ## L1：概述与使用场景 @@ -14,24 +14,26 @@ web-access 是一个**流程型技能（Procedural Skill）**，提供四层互 - **L1**（WebSearch + WebFetch）：公开页面，轻量 - **L2**（Jina Reader）：JS 渲染的重页面，默认节省 Token -- **L3-fast**（BrowserXxx 内置工具家族，**v2.0 新增**）：登录态站点首选——零 Python 依赖、内置 cdp-proxy 子进程、支持 CDP 真实鼠标事件 +- **L3-fast**（内置受管浏览器，**v2.1 重写**）：到达并*操作*登录态/交互站点——每个任务独立 BrowserSpace 隔离、零 Python 依赖、每次动作都有可审计回执。**抽取长正文仍归 L2 / L3-fallback**，见下方取文说明 - **L3-fallback**（Chrome CDP + Python Playwright）：复杂自动化场景兜底（长等待、特殊 race condition 等） -### v2.0 新增：BrowserXxx 工具家族（默认隐藏，激活后才暴露） +### v2.1 重写：内置受管浏览器（默认隐藏，激活后才暴露） -调用 `Skill('web-access')` 加载本技能时，以下 11 个工具被注入到当前会话，让 LLM 直接驱动浏览器： +调用 `Skill('web-access')` 加载本技能时，以下 8 个工具被注入到当前会话，让 LLM 直接驱动内置浏览器： | 工具 | 用途 | |------|------| -| BrowserListTabs / BrowserNavigate / BrowserCloseTab | tab 管理 | -| BrowserEval | 执行 JS 提取数据 | -| BrowserClick (`mode: js \| real-mouse`) | 点击元素，real-mouse 反爬更强 | -| BrowserScreenshot / BrowserScroll | 截图、滚动触发懒加载 | -| BrowserSetFiles | 上传本地文件（需用户确认） | +| BrowserManage | 建/销隔离 BrowserSpace、启动会话、管理标签页 | +| BrowserSnapshot | `semantic` / `accessibility` / `visual` 快照——读页面的主通道 | +| BrowserAct | 一次调用一个受管动作：导航、点击、输入、滚动、截图…… | +| BrowserImport | 从用户 Chrome/Edge/Firefox/Safari 配置导入 Cookie（需人工审批） | +| BrowserShare | 把 Space/Session 委派给其他 Agent（隔离 / 快照 / 写时复制 / 实时共享） | | SitePatternRead / SitePatternWrite | 按域名累积"站点经验"（AgentFS 三层） | | LocalBookmarks | 检索本地 Chrome 书签 / 历史 | > **重要**：未调用 Skill('web-access') 之前，这些工具**不会**出现在 LLM 的 tools 列表里——默认对话不消耗其 token。详见 [references/browser-tools.md](references/browser-tools.md)。 +> +> **v2.1 已移除**：`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用，调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本版要求客户端 v10.0.98+。 ### 使用场景 @@ -44,7 +46,7 @@ web-access 是一个**流程型技能（Procedural Skill）**，提供四层互 - **四层递进**：从轻量搜索到重度 JS 渲染到登录态访问，按需选择 - **Token 优化**：Jina Reader 默认减少 50-80% Token 消耗 -- **登录态复用**：通过 CDP 连接用户已登录的 Chrome，无需重复登录 +- **登录态复用**：用 BrowserImport 把用户已有 Cookie 导入隔离 Space（兜底层仍可 CDP 连用户 Chrome），无需重复登录 ## L2：详细规范 @@ -60,7 +62,7 @@ If any fetch fails, explicitly tell the user which URL failed and which fallback ## Prerequisites: Chrome CDP Setup (for login-gated sites) -**Only required when accessing sites that need the user's login session** (小红书/B站/微博/飞书/Twitter/知乎/公众号). +**Only required for the L3-fallback layer**（Python Playwright）。L3-fast 内置浏览器不需要——用 `BrowserImport` 复用登录态即可。 ### One-time setup @@ -119,10 +121,11 @@ User intent │ (Jina Reader = default for JS-rendered content, saves tokens) │ ├─ "Read this login-gated page" (小红书/B站/微博/飞书/Twitter/知乎/公众号) - │ └─→ 1. Verify CDP ready (curl http://localhost:9222/json/version) - │ 2. Bash: python3 script with playwright.connect_over_cdp() - │ 3. Extract content → feed to Jina Reader for clean Markdown - │ (or use BeautifulSoup directly on the raw HTML) + │ ├─→ 到达：BrowserManage(create_space/start_session) → BrowserImport(按域导入 Cookie) + │ │ → BrowserAct(tab.navigate) → tab.activate + page.screenshot 确认落到正文页 + │ │ （semantic 快照不含正文，不能用来读内容） + │ └─→ 取正文：确认 CDP 就绪后 python3 playwright.connect_over_cdp() + │ → page.content() → Jina Reader / BeautifulSoup │ ├─ "API documentation / GitHub / npm package info" │ └─→ Prefer official API endpoints over scraping HTML: @@ -131,9 +134,9 @@ User intent │ - PyPI: curl https://pypi.org/pypi//json │ └─ "Real-time interactive task" (click, fill form, scroll, screenshot) - ├─→ **Default: BrowserXxx tools** (BrowserNavigate / BrowserEval / BrowserClick / BrowserScreenshot — + ├─→ **Default: 内置受管浏览器** (BrowserManage → BrowserAct → BrowserSnapshot — │ see references/browser-tools.md, no Python needed) - └─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when BrowserXxx is insufficient + └─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when 内置浏览器 is insufficient (e.g., complex race conditions, multi-event waits, long-running in-browser scripts) ``` @@ -143,10 +146,10 @@ User intent |-------|----------|--------------|------------| | L1 | Public, static | `WebFetch` | Low | | L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) | -| **L3-fast** | **Login-gated, interactive (PRIMARY)** | **BrowserXxx 工具家族** | Medium | +| **L3-fast** | **登录态导航与交互 (PRIMARY)** | **内置受管浏览器（BrowserManage / BrowserAct / BrowserSnapshot）** | Medium | | L3-fallback | 复杂自动化（race / long-wait / 自定义脚本） | `Bash + Python Playwright CDP` | Medium | -**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when BrowserXxx 不够用。 +**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when 内置浏览器不够用。 --- @@ -262,33 +265,49 @@ See [references/cdp-browser.md](references/cdp-browser.md) for: --- -## L3-fast: BrowserXxx 工具速查（v2.0 推荐） +## L3-fast: 内置受管浏览器速查（v2.1） **只在你调用 `Skill('web-access')` 加载本技能后，下面这组工具才会出现在 tools[] 里。** | 工具 | 一行示例 | |------|---------| -| `BrowserListTabs()` | 列出所有打开 tab | -| `BrowserNavigate({ url })` | 在新 tab 打开 URL | -| `BrowserNavigate({ target, url })` | 在指定 tab 跳转 | -| `BrowserEval({ target, expression })` | 在 tab 内跑 JS，提取结构化数据 | -| `BrowserClick({ target, selector, mode: 'real-mouse' })` | 反爬严格站点用真实鼠标事件 | -| `BrowserScreenshot({ target })` | 写入 ${DESIRECORE_ROOT}/screenshots/ | -| `BrowserScroll({ target, direction: 'bottom' })` | 触发懒加载 | -| `BrowserSetFiles({ target, selector, files })` | 上传本地文件（**需用户确认**） | -| `BrowserCloseTab({ target })` | 任务收尾清理临时 tab | +| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | 每个任务一个隔离 Space | +| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | 启动会话，返回 sessionId 与首个标签页 | +| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | 标签页与生命周期管理 | +| `BrowserAct({ action: 'tab.navigate', params: { url } })` | 当前标签页导航 | +| `BrowserSnapshot({ mode: 'semantic' })` | 读页面：可交互元素 + `ref` 句柄 | +| `BrowserAct({ action: 'input.click', params: { ref } })` | 按快照 `ref` 点击，不要用裸 x/y | +| `BrowserAct({ action: 'input.text', params: { text } })` | 向聚焦元素输入文本 | +| `BrowserAct({ action: 'input.wheel', params: { deltaX: 0, deltaY: 720 } })` | 滚动触发懒加载 | +| `BrowserAct({ action: 'tab.activate', params: { bounds } })` → `page.screenshot` | **先 activate 再截图** | +| `BrowserImport({ action: 'discover' \| 'plan' \| 'apply' })` | 复用用户登录态 Cookie（需人工审批） | 完整 API 与边界条件见 [references/browser-tools.md](references/browser-tools.md)。 +**怎么读页面** —— 按需求选通道： + +| 你要什么 | 用什么 | 说明 | +|----------|--------|------| +| 可交互元素 + `ref` 句柄 | `BrowserSnapshot({ mode: 'semantic' })` | 只有按钮/输入框/链接，**不含正文文本** | +| 小页面的正文 | `BrowserSnapshot({ mode: 'accessibility' })` | 返回 StaticText 节点；真实内容页会报 `BROWSER_RESULT_TOO_LARGE`（结果上限 2 MB，且 `depth` 参数当前被宿主忽略） | +| 真实页面的正文 | L2 Jina Reader（公开页）或 L3-fallback Playwright（登录态） | 内置浏览器目前没有可用的批量取文通道 | +| 页面长什么样 | `tab.activate` → `BrowserAct({ action: 'page.screenshot' })` | 只有整页截图，靠看图读 | + +`page.evaluate` **不是**取文通道：每次调用需人工审批，且字符串/对象返回值会被替换成 `[REDACTED:browser-runtime-value]`。 + ### 推荐流程（小红书示例） ``` -1. BrowserListTabs() → 看是否已有登录态 tab -2. 没有 → BrowserNavigate({ url: "https://www.xiaohongshu.com/explore/abc123" }) -3. BrowserEval({ target, expression: "(...)JSON.stringify({title, content})" }) -4. SitePatternRead({ domain: "xiaohongshu.com" }) ← 读累积经验 -5. 任务结束 → BrowserCloseTab({ target }) -6. 如发现新陷阱 → SitePatternWrite({ domain, scope: "agent", mode: "merge", content }) +1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' }) +2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] }) +3. BrowserImport({ action: 'discover' }) → plan → apply 授权 xiaohongshu.com ← 复用登录态 +4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } }) +5. BrowserSnapshot({ mode: 'semantic' }) ← 只拿交互用的元素 ref（不含正文） + tab.activate + page.screenshot ← 确认确实渲染出了笔记 + → 正文抽取走 L3-fallback Playwright +6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← 读累积经验 +7. 任务结束 → BrowserManage({ action: 'close_session', sessionId }) +8. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) ``` --- @@ -360,9 +379,11 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi - ❌ **Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current. - ❌ **Hardcoding login credentials in scripts** — always rely on the user's pre-logged CDP session. - ❌ **Citing only after the fact** — collect URLs as you fetch, not from memory afterwards. -- ❌ **(v2.0) 在能用 BrowserXxx 时仍写 Python heredoc** — 慢、依赖 Python+Playwright 安装、上下文体积大。优先 L3-fast；只在 BrowserXxx 不够（race / 长等待 / 自定义脚本）时才回退。 -- ❌ **(v2.0) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')`。 -- ❌ **(v2.0) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级，但你不该故意往 agent 层写敏感信息。 +- ❌ **(v2.1) 在能用内置浏览器时仍写 Python heredoc** — 慢、依赖 Python+Playwright 安装、上下文体积大。优先 L3-fast；只在内置浏览器不够（race / 长等待 / 自定义脚本）时才回退。 +- ❌ **(v2.1) 没 `tab.activate` 就直接 `page.screenshot`** — 标签页默认停在窗口外 `(-10000,-10000,1x1)`，没有合成表面，截图会卡满 30 秒 deadline **并把标签页宿主打掉**，之后所有调用都报 `BROWSER_TAB_HOST_NOT_FOUND`。务必先用真实 bounds `tab.activate`。 +- ❌ **(v2.1) 用 `page.evaluate` 读页面** — 每次调用都需人工审批，返回的字符串/对象会被治理策略替换为 `[REDACTED:browser-runtime-value]`（只有 number/boolean/null 能穿透），且反调试站点会把它挂起几十秒。读页面请用 `BrowserSnapshot`。 +- ❌ **(v2.1) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')`。 +- ❌ **(v2.1) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级，但你不该故意往 agent 层写敏感信息。 --- diff --git a/skills/web-access/references/browser-tools.md b/skills/web-access/references/browser-tools.md index 1a5fba4..522dc48 100644 --- a/skills/web-access/references/browser-tools.md +++ b/skills/web-access/references/browser-tools.md @@ -1,113 +1,163 @@ -# BrowserXxx 工具速查（L3-fast） +# 内置受管浏览器工具速查（L3-fast） -> 适配自 [eze-is/web-access](https://github.com/eze-is/web-access) 的 `references/cdp-api.md`（MIT，作者 一泽 Eze）。 -> DesireCore 把 cdp-proxy 内嵌为子进程，并通过这组 BuiltinTool 调用——比直接 curl 更安全（不暴露端口给 prompt 注入）、比 Python heredoc 更轻（无 Python/Playwright 依赖）。 +> v2.1 起，本层从「cdp-proxy 驱动用户自己的 Chrome」改为「DesireCore 内置受管浏览器」。 +> 每个任务跑在独立 BrowserSpace 里（Cookie / Storage / 缓存互不串扰），每个动作都经过 +> Capability → Grant → Lease → Origin → Host fencing 校验并留下可审计回执。 +> +> 要求客户端 **v10.0.98+**。旧的 `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / +> `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / +> `BrowserCloseTab` 与其背后的 cdp-proxy 已停用，调用会直接返回 +> 「该旧 BrowserXxx/cdp-proxy 入口已停用；请改用 BrowserManage、BrowserSnapshot 或 BrowserAct」。 -## 何时用 BrowserXxx vs Python Playwright +## 何时用内置浏览器 vs Python Playwright | 场景 | 推荐 | |------|------| -| 抓取登录态站点（小红书 / B站 / 微博 / 飞书 / 知乎） | **BrowserXxx**（fast，零 Python） | -| 简单点击 / 滚动 / 截图 / 上传文件 | **BrowserXxx** | -| 需要复杂等待逻辑（page.wait_for_selector + race condition） | Python Playwright（cdp-browser.md） | -| 需要在浏览器内运行长时间脚本（>30 s） | Python Playwright | +| 到达并操作登录态站点（小红书 / B站 / 微博 / 飞书 / 知乎） | **内置浏览器**（配合 BrowserImport 导入 Cookie） | +| 抽取登录态站点的长正文 | Python Playwright（内置浏览器没有批量取文通道，见下） | +| 简单点击 / 滚动 / 截图 | **内置浏览器** | +| 多个任务要互不串扰地并行 | **内置浏览器**（一任务一 Space） | +| 需要复杂等待逻辑（wait_for_selector + race condition） | Python Playwright（cdp-browser.md） | +| 需要在浏览器内运行长时间脚本（>30 s） | Python Playwright（内置浏览器单条命令 30 s deadline） | +| 需要对元素做放大裁剪截图 | Python Playwright（内置浏览器只有整页截图） | ## 前置条件 -1. 用户已启动调试模式 Chrome（端口 9222；见 SKILL.md 的 Prerequisites 部分） -2. cdp-proxy 子进程会在首次工具调用时由 agent-service lazy spawn +1. 客户端 v10.0.98+，`~/.desirecore/config/browser.json` 里 `electron-embedded` 或 + `standalone-managed` Provider 处于 `enabled`（默认即开启） +2. 无需用户手工启动调试 Chrome，也无需 Python / Playwright ## 工具一览 每个工具默认 `hidden: true`，**只有 web-access 技能被激活后才暴露给 LLM**。 -### BrowserListTabs +### BrowserManage -列出已打开 tab。返回每行 `[targetId] title — url`。 +Space / Session / Tab 的生命周期。 ```yaml -BrowserListTabs: - # 无参数 +BrowserManage: + action: create_space # list_spaces | create_space | start_session | join_session + # | leave_session | list_sessions | list_tabs | create_tab + # | freeze_session | resume_session | close_session + name: xhs-note # create_space 用 + persistence: ephemeral # ephemeral=查完即弃，不落 Profile；persistent=保留登录态 + providerPreference: [electron-embedded] ``` -### BrowserNavigate - -打开 URL；省略 `target` 时新开 tab。 - ```yaml -BrowserNavigate: - url: https://www.xiaohongshu.com/explore/... - target: <可选 targetId> +BrowserManage: + action: start_session + spaceId: bsp_xxx + capabilities: # 只申请你真正要用的，最小权限 + - browser.observe.tabs + - browser.observe.snapshot + - browser.observe.screenshot + - browser.navigate.create-tab + - browser.navigate.url + - browser.navigate.activate-tab + - browser.input.pointer.click + - browser.input.keyboard ``` -### BrowserEval +`create_space` 会触发一次用户确认。任务收尾用 `close_session` 释放。 -在指定 tab 内执行 JS（`returnByValue: true` + `awaitPromise: true`）。 +### BrowserSnapshot + +读页面的**主通道**。返回可交互元素及其 `ref` 句柄、`rect`、`disabled` 状态。 ```yaml -BrowserEval: - target: - expression: | - (() => { - const t = document.querySelector('h1.video-title')?.textContent || '' - const d = document.querySelector('.video-desc')?.textContent || '' - return JSON.stringify({ title: t.trim(), desc: d.trim() }) - })() +BrowserSnapshot: + mode: semantic # semantic（默认）| accessibility | visual + sessionId: bss_xxx # 当前 Agent 有多个会话时用于消歧 + tabId: btab_xxx ``` 提示： -- DOM 节点不能直接返回，要提取属性 -- 大量数据用 `JSON.stringify()` 包裹返回字符串 -- 风险等级 medium：是任意 JS 入口，不要执行不可信代码 +- `semantic` 只列 button / input / a 等可交互元素，**不含图片和正文文本节点** +- 元素 `name` 往往是 placeholder（如「请输入」），无标签时只能靠 `rect.y` 排序定位 +- 页面重排后旧 `ref` 会失效——**每次交互前重新取快照** +- `accessibility` 能拿到 StaticText 正文，但**只在很简单的页面上可用**：实测 + example.com 正常返回，维基百科条目一律 `BROWSER_RESULT_TOO_LARGE`（回执上限 2 MB）， + 且 schema 里的 `depth` 参数目前被宿主忽略（硬编码 depth=50），调小也没用 +- 因此**取真实页面正文仍要靠 L2 Jina Reader（公开页）或 L3-fallback Playwright（登录态）** -### BrowserClick +### BrowserAct + +一次调用一个受管动作。完整 `action` 见工具 schema，常用的： ```yaml -BrowserClick: - target: - selector: button.submit - mode: real-mouse # 默认 js；登录态站点反爬严格时建议 real-mouse +BrowserAct: + action: tab.navigate + params: { url: https://www.xiaohongshu.com/explore/... } ``` -`mode: real-mouse` 走 CDP `Input.dispatchMouseEvent` 派发真实鼠标事件——能触发文件对话框、绕过部分反自动化检测。 - -### BrowserScreenshot - ```yaml -BrowserScreenshot: - target: - filename: 自定义文件名.png # 可选；写入 ${DESIRECORE_ROOT}/screenshots/ +BrowserAct: + action: input.click + params: { ref: bref_xxx } # 用快照 ref；坐标会因页面重排失效 ``` -### BrowserScroll - ```yaml -BrowserScroll: - target: - direction: bottom # 或 top；与 y 二选一 - # y: 3000 # 按像素滚动 +BrowserAct: + action: input.text + params: { text: 搜索关键词 } ``` -### BrowserSetFiles（**需用户确认**） - -为 input[type=file] 设置本地文件，绕过文件对话框。涉及上传，必须经 user confirmation。 - ```yaml -BrowserSetFiles: - target: - selector: input[type=file] - files: - - /Users/me/Pictures/photo.png +BrowserAct: + action: input.wheel + params: { deltaX: 0, deltaY: 720, x: 640, y: 400 } ``` -### BrowserCloseTab - ```yaml -BrowserCloseTab: - target: +BrowserAct: + action: tab.activate # 截图前必须先做这一步 + params: { bounds: { x: 0, y: 0, width: 1280, height: 900 } } ``` -任务收尾建议清理你创建的临时 tab，避免 cdp-proxy 的 tab 池堆积（15 min 后会自动 GC，但显式关更整洁）。 +```yaml +BrowserAct: + action: page.screenshot + params: { format: png } # 结果落 artifact store，回执给 artifact.id / sha256 / bytes +``` + +### BrowserImport（**需人工审批**） + +把用户浏览器里的登录态 Cookie 导入当前 Space，替代旧版「attach 用户已登录的 Chrome」。 + +```yaml +BrowserImport: + action: discover # discover | plan | apply | ... +``` + +```yaml +BrowserImport: + action: apply + planId: bip_xxx + domains: [xiaohongshu.com] # 必须逐域显式授权 + conflictStrategy: newer-wins +``` + +解密与过滤全在 Host 侧完成，**Cookie 值不会进入 Agent 上下文或审计日志**。 + +### BrowserShare + +把 Space / Session 委派给另一个 Agent，`shareMode` 可选 `snapshot`（只读副本）、 +`copy-on-write`（写时复制）、`live-shared`（实时共享）、`handoff`（移交控制权）。 + +## 已知边界（照做，别试探） + +| 边界 | 说明 | +|------|------| +| **截图前必须 `tab.activate`** | 标签页默认停在 `(-10000,-10000,1x1)`，没有合成表面。直接截图会卡满 30 s deadline，**并把标签页宿主打掉**，之后全部报 `BROWSER_TAB_HOST_NOT_FOUND` | +| **同时只有一个可见标签页** | `tab.activate` 绑定主窗口、全局互斥。多 Space 可以并发导航/快照/输入，但截图必须逐个 activate 串行 | +| **`page.evaluate` 基本不可用** | 每次调用需人工审批；返回的字符串/对象被替换为 `[REDACTED:browser-runtime-value]`（只有 number/boolean/null 穿透）；反调试站点会把它挂起几十秒。读页面用 `BrowserSnapshot` | +| **`cdp.raw` 需人工审批** | `browser.raw_cdp.*` 属于永远人工闸门的能力，无人值守流程用不了（元素级裁剪截图因此不可用） | +| **没有批量取文通道** | `semantic` 不含正文，`accessibility` 在真实页面上超限，`page.evaluate` 被审批+脱敏。要正文请回落 Jina / Playwright | +| **单条命令 30 s deadline** | 超时即判 `browser.host.gone`，会话作废 | +| **用户真实鼠标会抢控制权** | 鼠标划过可见标签页会触发 `trusted-user-input` 并递增 control epoch，可能打断 Agent 的租约 | +| **artifact 不要走 `/save`** | 该接口会弹系统「另存为」对话框等人点。artifact 文件在 `${DESIRECORE_ROOT}/browser/artifacts//`，直接读即可，默认保留 24 小时 | ## SitePatternRead / SitePatternWrite @@ -120,7 +170,7 @@ SitePatternWrite: mode: merge # 默认 merge 追加；replace 覆盖 content: | ## 已知陷阱 - - 2026-05: ... + - 2026-08: ... ``` 含 cookie/token/手机号/邮箱时会自动降级 scope='user'。 @@ -129,14 +179,18 @@ SitePatternWrite: | 错误 | 原因 | 解决 | |------|------|------| -| `Chrome 未开启远程调试端口` | 用户没启动调试 Chrome | 引导用户跑 SKILL.md 中的启动命令 | -| `attach 失败` | targetId 无效或 tab 已关闭 | 重新 `BrowserListTabs` | -| `CDP 命令超时` | 页面长时间未响应 | 检查页面状态，必要时 `BrowserCloseTab` 后重开 | -| `CDP proxy 请求被中止或超时` | 代理子进程异常或网络故障 | 等待 ProxyController 自动重启（最多 3 次） | +| `该旧 BrowserXxx/cdp-proxy 入口已停用` | 还在调 v2.0 的旧工具 | 改用 BrowserManage / BrowserSnapshot / BrowserAct | +| `BROWSER_TAB_HOST_NOT_FOUND` | 标签页宿主已销毁（多因上一条命令超时） | 重建 Session；检查是否漏了 `tab.activate` | +| `BROWSER_COMMAND_DEADLINE_EXCEEDED` | 单条命令超 30 s | 截图先 activate；避免 `page.evaluate` | +| `BROWSER_TOOL_SESSION_FORBIDDEN` | 会话已关闭/崩溃，或不属于当前 Agent | 重新 `list_sessions`，必要时重建 | +| `BROWSER_TOOL_SESSION_AMBIGUOUS` | 当前 Agent 有多个会话且未传 sessionId | 显式传 `sessionId` | +| `BROWSER_HUMAN_APPROVAL_REQUIRED` | 触到人工闸门能力（evaluate / raw_cdp / 上传下载 / Cookie 导入等） | 向用户说明用途并等待审批，或换用无需审批的路径 | +| `BROWSER_RESULT_TOO_LARGE` | 回执超过 2 MB（`accessibility` 快照最常见） | 改用 `semantic` 快照 + 截图；取正文回落 Jina / Playwright | ## 调用链路 ``` -Agent → BrowserXxx 工具 → proxy-client → cdp-proxy 子进程 → Chrome (DevTools Protocol) - ↑ 首次调用 lazy spawn 子进程 +Agent → BrowserManage/Snapshot/Act → browser-use service（Capability/Grant/Lease/Policy 校验） + → BrowserHost（electron-embedded 或 standalone-managed）→ Chromium + ↑ 每步产出带 digest 的回执，写入审计事件流 ``` diff --git a/skills/web-access/references/site-patterns/bilibili.com.md b/skills/web-access/references/site-patterns/bilibili.com.md index ced7636..d76f3d1 100644 --- a/skills/web-access/references/site-patterns/bilibili.com.md +++ b/skills/web-access/references/site-patterns/bilibili.com.md @@ -5,7 +5,7 @@ type: site-pattern pinned: true confidence: medium learned_at: '2026-05-05' -updated_at: '2026-05-05' +updated_at: '2026-08-06' --- ## L0 @@ -18,7 +18,7 @@ updated_at: '2026-05-05' ## 推荐流程 - 仅取标题/UP 主/播放数 → WebFetch 即可 -- 取完整简介 / 评论 → BrowserNavigate + BrowserEval +- 取完整简介 / 评论 → 内置浏览器 BrowserAct(tab.navigate) 到达页面，正文抽取回落 L3-fallback Playwright - 批量数据 → 优先尝试 `api.bilibili.com` 公开接口（有专门的 wbi 签名机制） ## 推荐选择器 diff --git a/skills/web-access/references/site-patterns/feishu.cn.md b/skills/web-access/references/site-patterns/feishu.cn.md index 9862b22..d9a7945 100644 --- a/skills/web-access/references/site-patterns/feishu.cn.md +++ b/skills/web-access/references/site-patterns/feishu.cn.md @@ -5,7 +5,7 @@ type: site-pattern pinned: true confidence: high learned_at: '2026-05-05' -updated_at: '2026-05-05' +updated_at: '2026-08-06' --- ## L0 @@ -19,7 +19,7 @@ updated_at: '2026-05-05' ## 推荐流程 1. 公开文档（companion 页面/官方文档站点 `feishu.cn/hc/...`）→ WebFetch / Jina -2. 用户文档 → BrowserNavigate（已登录 Chrome）+ BrowserEval 抓正文 +2. 用户文档 → 内置浏览器 BrowserImport 导入 Cookie + BrowserAct(tab.navigate) 到达，正文抽取回落 Playwright 3. 长期程序化访问 → 申请 OpenAPI access_token，走 HTTP，避免 CDP ## 推荐选择器 diff --git a/skills/web-access/references/site-patterns/weibo.com.md b/skills/web-access/references/site-patterns/weibo.com.md index af99701..9f64671 100644 --- a/skills/web-access/references/site-patterns/weibo.com.md +++ b/skills/web-access/references/site-patterns/weibo.com.md @@ -5,7 +5,7 @@ type: site-pattern pinned: true confidence: medium learned_at: '2026-05-05' -updated_at: '2026-05-05' +updated_at: '2026-08-06' --- ## L0 @@ -18,8 +18,8 @@ updated_at: '2026-05-05' ## 推荐流程 1. 单条公开微博 → 优先尝试 `https://m.weibo.cn/status/`，WebFetch 看是否拿到正文 -2. 拿不到 → BrowserNavigate（已登录态 Chrome）+ BrowserClick 点"展开" + BrowserEval 取正文 -3. 用户主页时间线：必须登录，BrowserScroll 触发懒加载 +2. 拿不到 → 内置浏览器 BrowserImport 导入 Cookie + BrowserAct(tab.navigate) + BrowserAct(input.click ref) 点"展开"，正文抽取回落 Playwright +3. 用户主页时间线：必须登录，BrowserAct(input.wheel) 触发懒加载 ## 推荐选择器 - 单条详情：`.weibo-text` / `[class*='WB_text']` diff --git a/skills/web-access/references/site-patterns/xiaohongshu.com.md b/skills/web-access/references/site-patterns/xiaohongshu.com.md index 45fa2b4..122542d 100644 --- a/skills/web-access/references/site-patterns/xiaohongshu.com.md +++ b/skills/web-access/references/site-patterns/xiaohongshu.com.md @@ -5,7 +5,7 @@ type: site-pattern pinned: true confidence: high learned_at: '2026-05-05' -updated_at: '2026-05-05' +updated_at: '2026-08-06' --- ## L0 @@ -18,10 +18,10 @@ updated_at: '2026-05-05' - xsec_token：详情页 URL 包含 `xsec_token=...`，**通过站内交互生成**——直接拼 URL 容易失败 ## 推荐流程 -1. 通过 BrowserListTabs 找到已登录的 xhs tab；如无，BrowserNavigate 打开 -2. BrowserEval 注入选择器抓取标题与正文 -3. 列表页用 `[...document.querySelectorAll('.note-card a')].map(a => a.href)` 收集详情页链接 -4. 单条笔记结尾调用 `BrowserCloseTab` 释放 tab，避免 tab 池溢出 +1. BrowserManage(create_space/start_session) → BrowserImport 按域导入 xiaohongshu.com 的 Cookie +2. BrowserAct(tab.navigate) 打开笔记；列表页用 BrowserSnapshot(semantic) 收 `.note-card` 链接的 ref +3. 正文抽取回落 L3-fallback Playwright（内置浏览器的 semantic 快照不含正文，accessibility 在本站会超限） +4. 收尾 BrowserManage(close_session)；ephemeral Space 不留 Profile ## 推荐选择器 - 笔记标题：`#detail-title` 或 `h1[class*="title"]` @@ -36,8 +36,8 @@ updated_at: '2026-05-05' ## 反爬细节 - xsec_token 由前端生成，与会话/路由绑定，不能跨 tab 复用太久 -- User-Agent 检查较弱，但 navigator.webdriver 必须为 false（cdp-proxy 已默认绕过） -- click 操作建议优先用 `mode='real-mouse'`（CDP 真实鼠标事件），避免被 .click() 检测 +- User-Agent 检查较弱，但 navigator.webdriver 必须为 false +- 点击走 `BrowserAct(input.click)`（CDP 派发的真实指针事件），不要用页面内 `.click()` ## 速率限制 - 自评：单 IP 每分钟 ≤ 30 个详情页请求；超过会出验证码 diff --git a/skills/web-access/references/site-patterns/zhihu.com.md b/skills/web-access/references/site-patterns/zhihu.com.md index e24e2cd..f08daef 100644 --- a/skills/web-access/references/site-patterns/zhihu.com.md +++ b/skills/web-access/references/site-patterns/zhihu.com.md @@ -5,7 +5,7 @@ type: site-pattern pinned: true confidence: medium learned_at: '2026-05-05' -updated_at: '2026-05-05' +updated_at: '2026-08-06' --- ## L0 @@ -18,8 +18,8 @@ updated_at: '2026-05-05' ## 推荐流程 1. 专栏文章 → WebFetch / Jina Reader 优先 -2. 问答详情 → BrowserNavigate + BrowserEval；批量回答用 `[...document.querySelectorAll('.AnswerItem')].map(...)` -3. 评论 → 必须登录，BrowserClick 展开，BrowserScroll 加载更多 +2. 问答详情 → 内置浏览器 BrowserAct(tab.navigate) 到达；批量回答的正文抽取回落 Playwright（`.AnswerItem`） +3. 评论 → 必须登录，BrowserAct(input.click ref) 展开，BrowserAct(input.wheel) 加载更多 ## 推荐选择器 - 文章标题：`h1.Post-Title` / `h1[class*='Title']`