mirror of
https://git.openapi.site/https://github.com/desirecore/market.git
synced 2026-09-05 23:43:58 +08:00
feat(web-access): 迁移到内置受管浏览器工具（v2.1.0） (#72)
## 变更说明 / Description ### 中文 客户端 v10.0.98（desirecore/desirecore#1596）停用了旧的 `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其 cdp-proxy 后端，调用会直接返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。而 web-access v2.0.2 的 `provides.tools` 仍声明这批工具——技能激活后注入的是一组必然失败的工具。 本次把 `provides.tools` 换成统一浏览器工具，并同步正文与参考文档： - `provides.tools`：`BrowserManage` / `BrowserSnapshot` / `BrowserAct` / `BrowserImport` / `BrowserShare` + 保留 `SitePatternRead` / `SitePatternWrite` / `LocalBookmarks` - 中英文 SKILL 正文同步改写（L0 / 能力描述 / 决策树 / 四层策略表 / L3-fast 速查 / 反模式） - `references/browser-tools.md` 整篇重写为新 API + 实测边界 - 5 份站点经验（小红书 / B站 / 微博 / 知乎 / 飞书）的流程改用新工具 - 版本 2.0.2 → 2.1.0，`updated_at` 更新，i18n `source_hash` 重算 **L3-fast 的定位相应收窄**：内置浏览器负责「到达 + 交互 + 截图 + 隔离」，抽取长正文仍回落 Jina Reader（公开页）或 Playwright（登录态）——理由见下方实测。 ### English Client v10.0.98 retired the legacy `BrowserXxx` tools and the cdp-proxy behind them, so web-access v2.0.2 was injecting a set of tools that always fail. This PR migrates `provides.tools` to the unified browser tools and rewrites the body, the browser-tools reference, and the five site-pattern playbooks accordingly. L3-fast is re-scoped to navigation/interaction/screenshots; bulk text extraction still falls back to Jina Reader or Playwright. ## 测试方式 / Test Plan 在客户端 v10.0.98 + `electron-embedded` Provider 上实测： - [x] `provides.tools` 里 8 个工具 ID 全部在 builtin registry 中存在 - [x] 把本 PR 的技能装进 dev 实例，带 `skillIds:['web-access']` 驱动智能体：真实调用 `BrowserManage(create_space)` → `BrowserManage(start_session)` → `BrowserAct(tab.navigate)` → `BrowserManage(close_session)`，全部 success - [x] `scripts/i18n/validate-i18n.py` 全仓库通过（中英文标题数一致、source_hash 一致） 文档中记录的边界均来自实测，而非推测： | 边界 | 实测现象 | |------|---------| | 截图前必须 `tab.activate` | 标签页默认停在 `(-10000,-10000,1x1)`，直接截图卡满 30s deadline 并触发 `browser.host.gone`，之后全部 `BROWSER_TAB_HOST_NOT_FOUND` | | `page.evaluate` 不是取文通道 | 每次调用需人工审批；字符串/对象返回值被替换为 `[REDACTED:browser-runtime-value]`，仅 number/boolean/null 穿透 | | `accessibility` 快照真实页面不可用 | example.com 正常返回 StaticText；维基百科条目一律 `BROWSER_RESULT_TOO_LARGE`（2 MB 上限，且 `depth` 参数被宿主忽略） | | `semantic` 快照不含正文 | 只列 button / input / a 等可交互元素 | ## 风险与回滚 / Risk and rollback - 纯技能内容变更，无脚本或清单结构改动 - 需要客户端 v10.0.98+；旧客户端装到本版会拿到一组不存在的工具名（旧客户端上原本那批工具也已失效，不构成回退） - 回滚即 revert 本 PR
This commit is contained in:
@@ -5,16 +5,18 @@ description: >-
|
||||
— searching for current information, fetching public web pages, browsing
|
||||
login-gated sites (微博/小红书/B站/飞书/Twitter), comparing products,
|
||||
researching topics, gathering documentation, or summarizing news.
|
||||
This skill orchestrates three complementary layers: (1) WebSearch + WebFetch
|
||||
This skill orchestrates four complementary layers: (1) WebSearch + WebFetch
|
||||
for public pages, (2) Jina Reader as the default token-optimization layer for
|
||||
heavy/JS-rendered pages, and (3) Chrome DevTools Protocol (CDP) via Python
|
||||
Playwright for login-gated sites that require the user's existing browser
|
||||
session. Always cite source URLs. Use when 用户提到 联网搜索、上网查、
|
||||
heavy/JS-rendered pages, (3) the governed built-in browser (isolated
|
||||
BrowserSpace + Cookie import) to reach and interact with login-gated sites,
|
||||
and (4) Chrome DevTools Protocol via Python Playwright as the fallback for
|
||||
bulk text extraction and complex automation. Always cite source URLs.
|
||||
Use when 用户提到 联网搜索、上网查、
|
||||
查资料、抓取网页、研究、调研、最新资讯、文档查询、对比、竞品、技术文档、
|
||||
新闻、网址、URL、找一下、搜一下、查一下、小红书、B站、微博、飞书、Twitter、
|
||||
推特、X、知乎、公众号、已登录、登录状态。
|
||||
license: Complete terms in LICENSE.txt
|
||||
version: 2.0.2
|
||||
version: 2.1.0
|
||||
type: procedural
|
||||
risk_level: low
|
||||
status: enabled
|
||||
@@ -29,20 +31,17 @@ tags:
|
||||
- playwright
|
||||
provides:
|
||||
tools:
|
||||
- BrowserListTabs
|
||||
- BrowserNavigate
|
||||
- BrowserEval
|
||||
- BrowserClick
|
||||
- BrowserScreenshot
|
||||
- BrowserScroll
|
||||
- BrowserSetFiles
|
||||
- BrowserCloseTab
|
||||
- BrowserManage
|
||||
- BrowserSnapshot
|
||||
- BrowserAct
|
||||
- BrowserImport
|
||||
- BrowserShare
|
||||
- SitePatternRead
|
||||
- SitePatternWrite
|
||||
- LocalBookmarks
|
||||
metadata:
|
||||
author: desirecore
|
||||
updated_at: '2026-05-05'
|
||||
updated_at: '2026-08-06'
|
||||
i18n:
|
||||
default_locale: en-US
|
||||
source_locale: zh-CN
|
||||
@@ -51,17 +50,17 @@ metadata:
|
||||
- en-US
|
||||
zh-CN:
|
||||
name: 联网访问
|
||||
short_desc: 联网搜索、网页抓取、登录态浏览器访问（CDP）、研究调研工作流
|
||||
description: 三层联网访问工具包——搜索公开页面、Jina 优化抓取、CDP 登录态浏览器访问。
|
||||
short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问、研究调研工作流
|
||||
description: 四层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问、Python Playwright CDP 兜底。
|
||||
body: ./SKILL.zh-CN.md
|
||||
source_hash: sha256:0ba170b3126a0823
|
||||
source_hash: sha256:2e6753fc05142e9a
|
||||
translated_by: human
|
||||
en-US:
|
||||
name: Web Access
|
||||
short_desc: Web search, page fetching, logged-in browser access via CDP, research workflows
|
||||
description: A three-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, and reach logged-in sites via Chrome CDP.
|
||||
short_desc: Web search, page fetching, logged-in access via the governed built-in browser, research workflows
|
||||
description: A four-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, reach logged-in sites through the governed built-in browser, and fall back to Chrome CDP.
|
||||
body: ./SKILL.md
|
||||
source_hash: sha256:4cda9ca594fbd974
|
||||
source_hash: sha256:2e6753fc05142e9a
|
||||
translated_by: human
|
||||
market:
|
||||
icon: >-
|
||||
@@ -90,7 +89,7 @@ market:
|
||||
|
||||
## L0: One-line Summary
|
||||
|
||||
A three-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, and reach login-gated sites via Chrome CDP.
|
||||
A four-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, reach login-gated sites through the governed built-in browser, and fall back to Chrome CDP.
|
||||
|
||||
## L1: Overview & Use Cases
|
||||
|
||||
@@ -100,24 +99,26 @@ web-access is a **procedural skill** that provides four complementary layers of
|
||||
|
||||
- **L1** (WebSearch + WebFetch): public, static pages
|
||||
- **L2** (Jina Reader): JS-rendered heavy pages, saving tokens by default
|
||||
- **L3-fast** (BrowserXxx builtin tool family — **new in v2.0**): preferred for logged-in sites — zero Python dependency, in-process cdp-proxy, supports CDP real-mouse events
|
||||
- **L3-fast** (governed built-in browser — **rewritten in v2.1**): reach and *interact with* logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. **Reading long article text is still L2/L3-fallback's job** — see the extraction note below
|
||||
- **L3-fallback** (Chrome CDP + Python Playwright): backup for complex automation (long waits, race conditions, custom in-browser scripts)
|
||||
|
||||
### v2.0 — BrowserXxx tool family (default-hidden, exposed only after Skill activation)
|
||||
### v2.1 — governed built-in browser (default-hidden, exposed only after Skill activation)
|
||||
|
||||
When you call `Skill('web-access')`, the following 11 tools are injected into the current session so the LLM can drive Chrome directly:
|
||||
When you call `Skill('web-access')`, the following 8 tools are injected into the current session so the LLM can drive the built-in browser directly:
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| BrowserListTabs / BrowserNavigate / BrowserCloseTab | Tab management |
|
||||
| BrowserEval | Run JS to extract data |
|
||||
| BrowserClick (`mode: js \| real-mouse`) | Click elements; real-mouse defeats anti-bot |
|
||||
| BrowserScreenshot / BrowserScroll | Screenshots, scroll to trigger lazy loading |
|
||||
| BrowserSetFiles | Upload local files (requires user confirmation) |
|
||||
| BrowserManage | Create/destroy isolated BrowserSpace, start sessions, manage tabs |
|
||||
| BrowserSnapshot | `semantic` / `accessibility` / `visual` page snapshots — the primary way to read a page |
|
||||
| BrowserAct | One governed action per call: navigate, click, type, scroll, screenshot, … |
|
||||
| BrowserImport | Import Cookies from the user's Chrome/Edge/Firefox/Safari profile (human-approved) |
|
||||
| BrowserShare | Delegate a Space/Session to another Agent (isolated / snapshot / copy-on-write / live) |
|
||||
| SitePatternRead / SitePatternWrite | Per-domain "site experience" (AgentFS three-layer) |
|
||||
| LocalBookmarks | Search local Chrome bookmarks / history |
|
||||
|
||||
> **Important**: before `Skill('web-access')` is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See [references/browser-tools.md](references/browser-tools.md).
|
||||
>
|
||||
> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+.
|
||||
|
||||
### Use Cases
|
||||
|
||||
@@ -130,7 +131,7 @@ When you call `Skill('web-access')`, the following 11 tools are injected into th
|
||||
|
||||
- **Four-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand
|
||||
- **Token optimization**: Jina Reader cuts token usage by 50–80% by default
|
||||
- **Logged-in session reuse**: connect to the user's already-logged-in Chrome via CDP — no re-login required
|
||||
- **Logged-in session reuse**: import the user's existing Cookies into an isolated Space via BrowserImport (or attach to their Chrome via CDP in the fallback layer) — no re-login required
|
||||
|
||||
## L2: Detailed Specification
|
||||
|
||||
@@ -146,7 +147,7 @@ If any fetch fails, explicitly tell the user which URL failed and which fallback
|
||||
|
||||
## Prerequisites: Chrome CDP Setup (for login-gated sites)
|
||||
|
||||
**Only required when accessing sites that need the user's login session** (Xiaohongshu / Bilibili / Weibo / Feishu / Twitter / Zhihu / WeChat Official Accounts).
|
||||
**Only required for the L3-fallback layer** (Python Playwright). The L3-fast built-in browser does not need this — use `BrowserImport` to reuse the user's login instead.
|
||||
|
||||
### One-time setup
|
||||
|
||||
@@ -205,10 +206,11 @@ User intent
|
||||
│ (Jina Reader = default for JS-rendered content, saves tokens)
|
||||
│
|
||||
├─ "Read this login-gated page" (Xiaohongshu/Bilibili/Weibo/Feishu/Twitter/Zhihu/WeChat)
|
||||
│ └─→ 1. Verify CDP ready (curl http://localhost:9222/json/version)
|
||||
│ 2. Bash: python3 script with playwright.connect_over_cdp()
|
||||
│ 3. Extract content → feed to Jina Reader for clean Markdown
|
||||
│ (or use BeautifulSoup directly on the raw HTML)
|
||||
│ ├─→ Reach it: BrowserManage(create_space/start_session) → BrowserImport(cookies for
|
||||
│ │ that domain) → BrowserAct(tab.navigate) → tab.activate + page.screenshot
|
||||
│ │ to confirm you landed on the content (semantic snapshot has no body text)
|
||||
│ └─→ Extract the text: verify CDP ready, then python3 playwright.connect_over_cdp()
|
||||
│ → page.content() → Jina Reader / BeautifulSoup
|
||||
│
|
||||
├─ "API documentation / GitHub / npm package info"
|
||||
│ └─→ Prefer official API endpoints over scraping HTML:
|
||||
@@ -217,9 +219,9 @@ User intent
|
||||
│ - PyPI: curl https://pypi.org/pypi/<pkg>/json
|
||||
│
|
||||
└─ "Real-time interactive task" (click, fill form, scroll, screenshot)
|
||||
├─→ **Default: BrowserXxx tools** (BrowserNavigate / BrowserEval / BrowserClick / BrowserScreenshot —
|
||||
├─→ **Default: built-in browser** (BrowserManage → BrowserAct → BrowserSnapshot —
|
||||
│ see references/browser-tools.md, no Python needed)
|
||||
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when BrowserXxx is insufficient
|
||||
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when the built-in browser is insufficient
|
||||
(e.g., complex race conditions, multi-event waits, long-running in-browser scripts)
|
||||
```
|
||||
|
||||
@@ -229,10 +231,10 @@ User intent
|
||||
|-------|----------|--------------|------------|
|
||||
| L1 | Public, static | `WebFetch` | Low |
|
||||
| L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) |
|
||||
| **L3-fast** | **Login-gated, interactive (PRIMARY)** | **BrowserXxx tool family** | Medium |
|
||||
| **L3-fast** | **Login-gated navigation & interaction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot)** | Medium |
|
||||
| L3-fallback | Complex automation (race / long-wait / custom scripts) | `Bash + Python Playwright CDP` | Medium |
|
||||
|
||||
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when BrowserXxx is insufficient.
|
||||
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when the built-in browser is insufficient.
|
||||
|
||||
---
|
||||
|
||||
@@ -348,33 +350,49 @@ See [references/cdp-browser.md](references/cdp-browser.md) for:
|
||||
|
||||
---
|
||||
|
||||
## L3-fast: BrowserXxx Tool Cheatsheet (v2.0 recommended)
|
||||
## L3-fast: Built-in Browser Cheatsheet (v2.1)
|
||||
|
||||
**Only after you call `Skill('web-access')` will the following tools appear in `tools[]`.**
|
||||
|
||||
| Tool | One-line example |
|
||||
|------|-----------------|
|
||||
| `BrowserListTabs()` | List all open tabs |
|
||||
| `BrowserNavigate({ url })` | Open URL in a new tab |
|
||||
| `BrowserNavigate({ target, url })` | Navigate an existing tab |
|
||||
| `BrowserEval({ target, expression })` | Run JS in the tab to extract structured data |
|
||||
| `BrowserClick({ target, selector, mode: 'real-mouse' })` | Real-mouse mode for anti-bot-strict sites |
|
||||
| `BrowserScreenshot({ target })` | Saved under ${DESIRECORE_ROOT}/screenshots/ |
|
||||
| `BrowserScroll({ target, direction: 'bottom' })` | Trigger lazy loading |
|
||||
| `BrowserSetFiles({ target, selector, files })` | Upload local files (**user confirmation required**) |
|
||||
| `BrowserCloseTab({ target })` | Clean up temporary tabs at task end |
|
||||
| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | One isolated Space per task |
|
||||
| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | Start a session, returns sessionId + first tab |
|
||||
| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | Tab / lifecycle management |
|
||||
| `BrowserAct({ action: 'tab.navigate', params: { url } })` | Navigate the current tab |
|
||||
| `BrowserSnapshot({ mode: 'semantic' })` | Read the page: interactive elements + `ref` handles |
|
||||
| `BrowserAct({ action: 'input.click', params: { ref } })` | Click by snapshot `ref`, never by raw x/y |
|
||||
| `BrowserAct({ action: 'input.text', params: { text } })` | Type into the focused element |
|
||||
| `BrowserAct({ action: 'input.wheel', params: { deltaX: 0, deltaY: 720 } })` | Scroll to trigger lazy loading |
|
||||
| `BrowserAct({ action: 'tab.activate', params: { bounds } })` → `page.screenshot` | **activate first, then screenshot** |
|
||||
| `BrowserImport({ action: 'discover' \| 'plan' \| 'apply' })` | Reuse the user's login cookies (human-approved) |
|
||||
|
||||
Full API and edge cases: see [references/browser-tools.md](references/browser-tools.md).
|
||||
|
||||
**How to read a page** — pick by what you need:
|
||||
|
||||
| You need | Use | Note |
|
||||
|----------|-----|------|
|
||||
| Interactive elements + `ref` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons / inputs / links only — **no article text** |
|
||||
| Article text on a small/simple page | `BrowserSnapshot({ mode: 'accessibility' })` | Returns StaticText nodes; fails with `BROWSER_RESULT_TOO_LARGE` on real content pages (2 MB result cap, and the `depth` argument is currently ignored by the host) |
|
||||
| Article text on a real page | L2 Jina Reader (public) or L3-fallback Playwright (login-gated) | The built-in browser has no working bulk text-extraction channel yet |
|
||||
| What the page looks like | `tab.activate` → `BrowserAct({ action: 'page.screenshot' })` | Full-page only; read it visually |
|
||||
|
||||
`page.evaluate` is **not** an extraction channel: it needs human approval per call and its string/object return values come back as `[REDACTED:browser-runtime-value]`.
|
||||
|
||||
### Recommended flow (Xiaohongshu example)
|
||||
|
||||
```
|
||||
1. BrowserListTabs() → check whether there's an already-logged-in xhs tab
|
||||
2. If not → BrowserNavigate({ url: "https://www.xiaohongshu.com/explore/abc123" })
|
||||
3. BrowserEval({ target, expression: "...JSON.stringify({title, content})" })
|
||||
4. SitePatternRead({ domain: "xiaohongshu.com" }) ← read accumulated experience
|
||||
5. At task end → BrowserCloseTab({ target })
|
||||
6. If you find a new pitfall → SitePatternWrite({ domain, scope: "agent", mode: "merge", content })
|
||||
1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
|
||||
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
|
||||
3. BrowserImport({ action: 'discover' }) → plan → apply for xiaohongshu.com ← reuse login
|
||||
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
|
||||
5. BrowserSnapshot({ mode: 'semantic' }) ← element refs for interaction (no body text)
|
||||
tab.activate + page.screenshot ← confirm the note actually rendered
|
||||
→ extract the note text via L3-fallback Playwright
|
||||
6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience
|
||||
7. At task end → BrowserManage({ action: 'close_session', sessionId })
|
||||
8. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
|
||||
```
|
||||
|
||||
---
|
||||
@@ -446,9 +464,11 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
|
||||
- ❌ **Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current.
|
||||
- ❌ **Hardcoding login credentials in scripts** — always rely on the user's pre-logged CDP session.
|
||||
- ❌ **Citing only after the fact** — collect URLs as you fetch, not from memory afterwards.
|
||||
- ❌ **(v2.0) Writing Python heredoc when BrowserXxx would do** — slow, requires Python+Playwright install, and bloats context. Prefer L3-fast; fall back to Python only when BrowserXxx is insufficient (race / long-wait / custom scripts).
|
||||
- ❌ **(v2.0) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`.
|
||||
- ❌ **(v2.0) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.
|
||||
- ❌ **(v2.1) Writing Python heredoc when the built-in browser would do** — slow, requires Python+Playwright install, and bloats context. Prefer L3-fast; fall back to Python only when the built-in browser is insufficient (race / long-wait / custom scripts).
|
||||
- ❌ **(v2.1) Calling `page.screenshot` before `tab.activate`** — tabs are parked off-screen at `(-10000,-10000,1x1)` and have no compositing surface, so the capture stalls until the 30s deadline **and the tab host is destroyed**; every later call then fails with `BROWSER_TAB_HOST_NOT_FOUND`. Always `tab.activate` with real bounds first.
|
||||
- ❌ **(v2.1) Reaching for `page.evaluate` to read a page** — it needs human approval on every call, its string/object return values come back as `[REDACTED:browser-runtime-value]` (only number/boolean/null survive), and anti-debug sites stall it for tens of seconds. Read pages with `BrowserSnapshot` instead.
|
||||
- ❌ **(v2.1) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`.
|
||||
- ❌ **(v2.1) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
## L0：一句话摘要
|
||||
|
||||
四层联网访问工具包——搜索公开页面、Jina 优化抓取、BrowserXxx 内置工具家族（v2.0）、Python Playwright CDP 兜底。
|
||||
四层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问（v2.1）、Python Playwright CDP 兜底。
|
||||
|
||||
## L1：概述与使用场景
|
||||
|
||||
@@ -14,24 +14,26 @@ web-access 是一个**流程型技能（Procedural Skill）**，提供四层互
|
||||
|
||||
- **L1**（WebSearch + WebFetch）：公开页面，轻量
|
||||
- **L2**（Jina Reader）：JS 渲染的重页面，默认节省 Token
|
||||
- **L3-fast**（BrowserXxx 内置工具家族，**v2.0 新增**）：登录态站点首选——零 Python 依赖、内置 cdp-proxy 子进程、支持 CDP 真实鼠标事件
|
||||
- **L3-fast**（内置受管浏览器，**v2.1 重写**）：到达并*操作*登录态/交互站点——每个任务独立 BrowserSpace 隔离、零 Python 依赖、每次动作都有可审计回执。**抽取长正文仍归 L2 / L3-fallback**，见下方取文说明
|
||||
- **L3-fallback**（Chrome CDP + Python Playwright）：复杂自动化场景兜底（长等待、特殊 race condition 等）
|
||||
|
||||
### v2.0 新增：BrowserXxx 工具家族（默认隐藏，激活后才暴露）
|
||||
### v2.1 重写：内置受管浏览器（默认隐藏，激活后才暴露）
|
||||
|
||||
调用 `Skill('web-access')` 加载本技能时，以下 11 个工具被注入到当前会话，让 LLM 直接驱动浏览器：
|
||||
调用 `Skill('web-access')` 加载本技能时，以下 8 个工具被注入到当前会话，让 LLM 直接驱动内置浏览器：
|
||||
|
||||
| 工具 | 用途 |
|
||||
|------|------|
|
||||
| BrowserListTabs / BrowserNavigate / BrowserCloseTab | tab 管理 |
|
||||
| BrowserEval | 执行 JS 提取数据 |
|
||||
| BrowserClick (`mode: js \| real-mouse`) | 点击元素，real-mouse 反爬更强 |
|
||||
| BrowserScreenshot / BrowserScroll | 截图、滚动触发懒加载 |
|
||||
| BrowserSetFiles | 上传本地文件（需用户确认） |
|
||||
| BrowserManage | 建/销隔离 BrowserSpace、启动会话、管理标签页 |
|
||||
| BrowserSnapshot | `semantic` / `accessibility` / `visual` 快照——读页面的主通道 |
|
||||
| BrowserAct | 一次调用一个受管动作：导航、点击、输入、滚动、截图…… |
|
||||
| BrowserImport | 从用户 Chrome/Edge/Firefox/Safari 配置导入 Cookie（需人工审批） |
|
||||
| BrowserShare | 把 Space/Session 委派给其他 Agent（隔离 / 快照 / 写时复制 / 实时共享） |
|
||||
| SitePatternRead / SitePatternWrite | 按域名累积"站点经验"（AgentFS 三层） |
|
||||
| LocalBookmarks | 检索本地 Chrome 书签 / 历史 |
|
||||
|
||||
> **重要**：未调用 Skill('web-access') 之前，这些工具**不会**出现在 LLM 的 tools 列表里——默认对话不消耗其 token。详见 [references/browser-tools.md](references/browser-tools.md)。
|
||||
>
|
||||
> **v2.1 已移除**：`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用，调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本版要求客户端 v10.0.98+。
|
||||
|
||||
### 使用场景
|
||||
|
||||
@@ -44,7 +46,7 @@ web-access 是一个**流程型技能（Procedural Skill）**，提供四层互
|
||||
|
||||
- **四层递进**：从轻量搜索到重度 JS 渲染到登录态访问，按需选择
|
||||
- **Token 优化**：Jina Reader 默认减少 50-80% Token 消耗
|
||||
- **登录态复用**：通过 CDP 连接用户已登录的 Chrome，无需重复登录
|
||||
- **登录态复用**：用 BrowserImport 把用户已有 Cookie 导入隔离 Space（兜底层仍可 CDP 连用户 Chrome），无需重复登录
|
||||
|
||||
## L2：详细规范
|
||||
|
||||
@@ -60,7 +62,7 @@ If any fetch fails, explicitly tell the user which URL failed and which fallback
|
||||
|
||||
## Prerequisites: Chrome CDP Setup (for login-gated sites)
|
||||
|
||||
**Only required when accessing sites that need the user's login session** (小红书/B站/微博/飞书/Twitter/知乎/公众号).
|
||||
**Only required for the L3-fallback layer**（Python Playwright）。L3-fast 内置浏览器不需要——用 `BrowserImport` 复用登录态即可。
|
||||
|
||||
### One-time setup
|
||||
|
||||
@@ -119,10 +121,11 @@ User intent
|
||||
│ (Jina Reader = default for JS-rendered content, saves tokens)
|
||||
│
|
||||
├─ "Read this login-gated page" (小红书/B站/微博/飞书/Twitter/知乎/公众号)
|
||||
│ └─→ 1. Verify CDP ready (curl http://localhost:9222/json/version)
|
||||
│ 2. Bash: python3 script with playwright.connect_over_cdp()
|
||||
│ 3. Extract content → feed to Jina Reader for clean Markdown
|
||||
│ (or use BeautifulSoup directly on the raw HTML)
|
||||
│ ├─→ 到达：BrowserManage(create_space/start_session) → BrowserImport(按域导入 Cookie)
|
||||
│ │ → BrowserAct(tab.navigate) → tab.activate + page.screenshot 确认落到正文页
|
||||
│ │ （semantic 快照不含正文，不能用来读内容）
|
||||
│ └─→ 取正文：确认 CDP 就绪后 python3 playwright.connect_over_cdp()
|
||||
│ → page.content() → Jina Reader / BeautifulSoup
|
||||
│
|
||||
├─ "API documentation / GitHub / npm package info"
|
||||
│ └─→ Prefer official API endpoints over scraping HTML:
|
||||
@@ -131,9 +134,9 @@ User intent
|
||||
│ - PyPI: curl https://pypi.org/pypi/<pkg>/json
|
||||
│
|
||||
└─ "Real-time interactive task" (click, fill form, scroll, screenshot)
|
||||
├─→ **Default: BrowserXxx tools** (BrowserNavigate / BrowserEval / BrowserClick / BrowserScreenshot —
|
||||
├─→ **Default: 内置受管浏览器** (BrowserManage → BrowserAct → BrowserSnapshot —
|
||||
│ see references/browser-tools.md, no Python needed)
|
||||
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when BrowserXxx is insufficient
|
||||
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when 内置浏览器 is insufficient
|
||||
(e.g., complex race conditions, multi-event waits, long-running in-browser scripts)
|
||||
```
|
||||
|
||||
@@ -143,10 +146,10 @@ User intent
|
||||
|-------|----------|--------------|------------|
|
||||
| L1 | Public, static | `WebFetch` | Low |
|
||||
| L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) |
|
||||
| **L3-fast** | **Login-gated, interactive (PRIMARY)** | **BrowserXxx 工具家族** | Medium |
|
||||
| **L3-fast** | **登录态导航与交互 (PRIMARY)** | **内置受管浏览器（BrowserManage / BrowserAct / BrowserSnapshot）** | Medium |
|
||||
| L3-fallback | 复杂自动化（race / long-wait / 自定义脚本） | `Bash + Python Playwright CDP` | Medium |
|
||||
|
||||
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when BrowserXxx 不够用。
|
||||
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when 内置浏览器不够用。
|
||||
|
||||
---
|
||||
|
||||
@@ -262,33 +265,49 @@ See [references/cdp-browser.md](references/cdp-browser.md) for:
|
||||
|
||||
---
|
||||
|
||||
## L3-fast: BrowserXxx 工具速查（v2.0 推荐）
|
||||
## L3-fast: 内置受管浏览器速查（v2.1）
|
||||
|
||||
**只在你调用 `Skill('web-access')` 加载本技能后，下面这组工具才会出现在 tools[] 里。**
|
||||
|
||||
| 工具 | 一行示例 |
|
||||
|------|---------|
|
||||
| `BrowserListTabs()` | 列出所有打开 tab |
|
||||
| `BrowserNavigate({ url })` | 在新 tab 打开 URL |
|
||||
| `BrowserNavigate({ target, url })` | 在指定 tab 跳转 |
|
||||
| `BrowserEval({ target, expression })` | 在 tab 内跑 JS，提取结构化数据 |
|
||||
| `BrowserClick({ target, selector, mode: 'real-mouse' })` | 反爬严格站点用真实鼠标事件 |
|
||||
| `BrowserScreenshot({ target })` | 写入 ${DESIRECORE_ROOT}/screenshots/ |
|
||||
| `BrowserScroll({ target, direction: 'bottom' })` | 触发懒加载 |
|
||||
| `BrowserSetFiles({ target, selector, files })` | 上传本地文件（**需用户确认**） |
|
||||
| `BrowserCloseTab({ target })` | 任务收尾清理临时 tab |
|
||||
| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | 每个任务一个隔离 Space |
|
||||
| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | 启动会话，返回 sessionId 与首个标签页 |
|
||||
| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | 标签页与生命周期管理 |
|
||||
| `BrowserAct({ action: 'tab.navigate', params: { url } })` | 当前标签页导航 |
|
||||
| `BrowserSnapshot({ mode: 'semantic' })` | 读页面：可交互元素 + `ref` 句柄 |
|
||||
| `BrowserAct({ action: 'input.click', params: { ref } })` | 按快照 `ref` 点击，不要用裸 x/y |
|
||||
| `BrowserAct({ action: 'input.text', params: { text } })` | 向聚焦元素输入文本 |
|
||||
| `BrowserAct({ action: 'input.wheel', params: { deltaX: 0, deltaY: 720 } })` | 滚动触发懒加载 |
|
||||
| `BrowserAct({ action: 'tab.activate', params: { bounds } })` → `page.screenshot` | **先 activate 再截图** |
|
||||
| `BrowserImport({ action: 'discover' \| 'plan' \| 'apply' })` | 复用用户登录态 Cookie（需人工审批） |
|
||||
|
||||
完整 API 与边界条件见 [references/browser-tools.md](references/browser-tools.md)。
|
||||
|
||||
**怎么读页面** —— 按需求选通道：
|
||||
|
||||
| 你要什么 | 用什么 | 说明 |
|
||||
|----------|--------|------|
|
||||
| 可交互元素 + `ref` 句柄 | `BrowserSnapshot({ mode: 'semantic' })` | 只有按钮/输入框/链接，**不含正文文本** |
|
||||
| 小页面的正文 | `BrowserSnapshot({ mode: 'accessibility' })` | 返回 StaticText 节点；真实内容页会报 `BROWSER_RESULT_TOO_LARGE`（结果上限 2 MB，且 `depth` 参数当前被宿主忽略） |
|
||||
| 真实页面的正文 | L2 Jina Reader（公开页）或 L3-fallback Playwright（登录态） | 内置浏览器目前没有可用的批量取文通道 |
|
||||
| 页面长什么样 | `tab.activate` → `BrowserAct({ action: 'page.screenshot' })` | 只有整页截图，靠看图读 |
|
||||
|
||||
`page.evaluate` **不是**取文通道：每次调用需人工审批，且字符串/对象返回值会被替换成 `[REDACTED:browser-runtime-value]`。
|
||||
|
||||
### 推荐流程（小红书示例）
|
||||
|
||||
```
|
||||
1. BrowserListTabs() → 看是否已有登录态 tab
|
||||
2. 没有 → BrowserNavigate({ url: "https://www.xiaohongshu.com/explore/abc123" })
|
||||
3. BrowserEval({ target, expression: "(...)JSON.stringify({title, content})" })
|
||||
4. SitePatternRead({ domain: "xiaohongshu.com" }) ← 读累积经验
|
||||
5. 任务结束 → BrowserCloseTab({ target })
|
||||
6. 如发现新陷阱 → SitePatternWrite({ domain, scope: "agent", mode: "merge", content })
|
||||
1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
|
||||
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
|
||||
3. BrowserImport({ action: 'discover' }) → plan → apply 授权 xiaohongshu.com ← 复用登录态
|
||||
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
|
||||
5. BrowserSnapshot({ mode: 'semantic' }) ← 只拿交互用的元素 ref（不含正文）
|
||||
tab.activate + page.screenshot ← 确认确实渲染出了笔记
|
||||
→ 正文抽取走 L3-fallback Playwright
|
||||
6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← 读累积经验
|
||||
7. 任务结束 → BrowserManage({ action: 'close_session', sessionId })
|
||||
8. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
|
||||
```
|
||||
|
||||
---
|
||||
@@ -360,9 +379,11 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
|
||||
- ❌ **Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current.
|
||||
- ❌ **Hardcoding login credentials in scripts** — always rely on the user's pre-logged CDP session.
|
||||
- ❌ **Citing only after the fact** — collect URLs as you fetch, not from memory afterwards.
|
||||
- ❌ **(v2.0) 在能用 BrowserXxx 时仍写 Python heredoc** — 慢、依赖 Python+Playwright 安装、上下文体积大。优先 L3-fast；只在 BrowserXxx 不够（race / 长等待 / 自定义脚本）时才回退。
|
||||
- ❌ **(v2.0) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')`。
|
||||
- ❌ **(v2.0) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级，但你不该故意往 agent 层写敏感信息。
|
||||
- ❌ **(v2.1) 在能用内置浏览器时仍写 Python heredoc** — 慢、依赖 Python+Playwright 安装、上下文体积大。优先 L3-fast；只在内置浏览器不够（race / 长等待 / 自定义脚本）时才回退。
|
||||
- ❌ **(v2.1) 没 `tab.activate` 就直接 `page.screenshot`** — 标签页默认停在窗口外 `(-10000,-10000,1x1)`，没有合成表面，截图会卡满 30 秒 deadline **并把标签页宿主打掉**，之后所有调用都报 `BROWSER_TAB_HOST_NOT_FOUND`。务必先用真实 bounds `tab.activate`。
|
||||
- ❌ **(v2.1) 用 `page.evaluate` 读页面** — 每次调用都需人工审批，返回的字符串/对象会被治理策略替换为 `[REDACTED:browser-runtime-value]`（只有 number/boolean/null 能穿透），且反调试站点会把它挂起几十秒。读页面请用 `BrowserSnapshot`。
|
||||
- ❌ **(v2.1) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')`。
|
||||
- ❌ **(v2.1) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级，但你不该故意往 agent 层写敏感信息。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,113 +1,163 @@
|
||||
# BrowserXxx 工具速查（L3-fast）
|
||||
# 内置受管浏览器工具速查（L3-fast）
|
||||
|
||||
> 适配自 [eze-is/web-access](https://github.com/eze-is/web-access) 的 `references/cdp-api.md`（MIT，作者 一泽 Eze）。
|
||||
> DesireCore 把 cdp-proxy 内嵌为子进程，并通过这组 BuiltinTool 调用——比直接 curl 更安全（不暴露端口给 prompt 注入）、比 Python heredoc 更轻（无 Python/Playwright 依赖）。
|
||||
> v2.1 起，本层从「cdp-proxy 驱动用户自己的 Chrome」改为「DesireCore 内置受管浏览器」。
|
||||
> 每个任务跑在独立 BrowserSpace 里（Cookie / Storage / 缓存互不串扰），每个动作都经过
|
||||
> Capability → Grant → Lease → Origin → Host fencing 校验并留下可审计回执。
|
||||
>
|
||||
> 要求客户端 **v10.0.98+**。旧的 `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` /
|
||||
> `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` /
|
||||
> `BrowserCloseTab` 与其背后的 cdp-proxy 已停用，调用会直接返回
|
||||
> 「该旧 BrowserXxx/cdp-proxy 入口已停用；请改用 BrowserManage、BrowserSnapshot 或 BrowserAct」。
|
||||
|
||||
## 何时用 BrowserXxx vs Python Playwright
|
||||
## 何时用内置浏览器 vs Python Playwright
|
||||
|
||||
| 场景 | 推荐 |
|
||||
|------|------|
|
||||
| 抓取登录态站点（小红书 / B站 / 微博 / 飞书 / 知乎） | **BrowserXxx**（fast，零 Python） |
|
||||
| 简单点击 / 滚动 / 截图 / 上传文件 | **BrowserXxx** |
|
||||
| 需要复杂等待逻辑（page.wait_for_selector + race condition） | Python Playwright（cdp-browser.md） |
|
||||
| 需要在浏览器内运行长时间脚本（>30 s） | Python Playwright |
|
||||
| 到达并操作登录态站点（小红书 / B站 / 微博 / 飞书 / 知乎） | **内置浏览器**（配合 BrowserImport 导入 Cookie） |
|
||||
| 抽取登录态站点的长正文 | Python Playwright（内置浏览器没有批量取文通道，见下） |
|
||||
| 简单点击 / 滚动 / 截图 | **内置浏览器** |
|
||||
| 多个任务要互不串扰地并行 | **内置浏览器**（一任务一 Space） |
|
||||
| 需要复杂等待逻辑（wait_for_selector + race condition） | Python Playwright（cdp-browser.md） |
|
||||
| 需要在浏览器内运行长时间脚本（>30 s） | Python Playwright（内置浏览器单条命令 30 s deadline） |
|
||||
| 需要对元素做放大裁剪截图 | Python Playwright（内置浏览器只有整页截图） |
|
||||
|
||||
## 前置条件
|
||||
|
||||
1. 用户已启动调试模式 Chrome（端口 9222；见 SKILL.md 的 Prerequisites 部分）
|
||||
2. cdp-proxy 子进程会在首次工具调用时由 agent-service lazy spawn
|
||||
1. 客户端 v10.0.98+，`~/.desirecore/config/browser.json` 里 `electron-embedded` 或
|
||||
`standalone-managed` Provider 处于 `enabled`（默认即开启）
|
||||
2. 无需用户手工启动调试 Chrome，也无需 Python / Playwright
|
||||
|
||||
## 工具一览
|
||||
|
||||
每个工具默认 `hidden: true`，**只有 web-access 技能被激活后才暴露给 LLM**。
|
||||
|
||||
### BrowserListTabs
|
||||
### BrowserManage
|
||||
|
||||
列出已打开 tab。返回每行 `[targetId] title — url`。
|
||||
Space / Session / Tab 的生命周期。
|
||||
|
||||
```yaml
|
||||
BrowserListTabs:
|
||||
# 无参数
|
||||
BrowserManage:
|
||||
action: create_space # list_spaces | create_space | start_session | join_session
|
||||
# | leave_session | list_sessions | list_tabs | create_tab
|
||||
# | freeze_session | resume_session | close_session
|
||||
name: xhs-note # create_space 用
|
||||
persistence: ephemeral # ephemeral=查完即弃，不落 Profile；persistent=保留登录态
|
||||
providerPreference: [electron-embedded]
|
||||
```
|
||||
|
||||
### BrowserNavigate
|
||||
|
||||
打开 URL；省略 `target` 时新开 tab。
|
||||
|
||||
```yaml
|
||||
BrowserNavigate:
|
||||
url: https://www.xiaohongshu.com/explore/...
|
||||
target: <可选 targetId>
|
||||
BrowserManage:
|
||||
action: start_session
|
||||
spaceId: bsp_xxx
|
||||
capabilities: # 只申请你真正要用的，最小权限
|
||||
- browser.observe.tabs
|
||||
- browser.observe.snapshot
|
||||
- browser.observe.screenshot
|
||||
- browser.navigate.create-tab
|
||||
- browser.navigate.url
|
||||
- browser.navigate.activate-tab
|
||||
- browser.input.pointer.click
|
||||
- browser.input.keyboard
|
||||
```
|
||||
|
||||
### BrowserEval
|
||||
`create_space` 会触发一次用户确认。任务收尾用 `close_session` 释放。
|
||||
|
||||
在指定 tab 内执行 JS（`returnByValue: true` + `awaitPromise: true`）。
|
||||
### BrowserSnapshot
|
||||
|
||||
读页面的**主通道**。返回可交互元素及其 `ref` 句柄、`rect`、`disabled` 状态。
|
||||
|
||||
```yaml
|
||||
BrowserEval:
|
||||
target: <targetId>
|
||||
expression: |
|
||||
(() => {
|
||||
const t = document.querySelector('h1.video-title')?.textContent || ''
|
||||
const d = document.querySelector('.video-desc')?.textContent || ''
|
||||
return JSON.stringify({ title: t.trim(), desc: d.trim() })
|
||||
})()
|
||||
BrowserSnapshot:
|
||||
mode: semantic # semantic（默认）| accessibility | visual
|
||||
sessionId: bss_xxx # 当前 Agent 有多个会话时用于消歧
|
||||
tabId: btab_xxx
|
||||
```
|
||||
|
||||
提示：
|
||||
- DOM 节点不能直接返回，要提取属性
|
||||
- 大量数据用 `JSON.stringify()` 包裹返回字符串
|
||||
- 风险等级 medium：是任意 JS 入口，不要执行不可信代码
|
||||
- `semantic` 只列 button / input / a 等可交互元素，**不含图片和正文文本节点**
|
||||
- 元素 `name` 往往是 placeholder（如「请输入」），无标签时只能靠 `rect.y` 排序定位
|
||||
- 页面重排后旧 `ref` 会失效——**每次交互前重新取快照**
|
||||
- `accessibility` 能拿到 StaticText 正文，但**只在很简单的页面上可用**：实测
|
||||
example.com 正常返回，维基百科条目一律 `BROWSER_RESULT_TOO_LARGE`（回执上限 2 MB），
|
||||
且 schema 里的 `depth` 参数目前被宿主忽略（硬编码 depth=50），调小也没用
|
||||
- 因此**取真实页面正文仍要靠 L2 Jina Reader（公开页）或 L3-fallback Playwright（登录态）**
|
||||
|
||||
### BrowserClick
|
||||
### BrowserAct
|
||||
|
||||
一次调用一个受管动作。完整 `action` 见工具 schema，常用的：
|
||||
|
||||
```yaml
|
||||
BrowserClick:
|
||||
target: <targetId>
|
||||
selector: button.submit
|
||||
mode: real-mouse # 默认 js；登录态站点反爬严格时建议 real-mouse
|
||||
BrowserAct:
|
||||
action: tab.navigate
|
||||
params: { url: https://www.xiaohongshu.com/explore/... }
|
||||
```
|
||||
|
||||
`mode: real-mouse` 走 CDP `Input.dispatchMouseEvent` 派发真实鼠标事件——能触发文件对话框、绕过部分反自动化检测。
|
||||
|
||||
### BrowserScreenshot
|
||||
|
||||
```yaml
|
||||
BrowserScreenshot:
|
||||
target: <targetId>
|
||||
filename: 自定义文件名.png # 可选；写入 ${DESIRECORE_ROOT}/screenshots/
|
||||
BrowserAct:
|
||||
action: input.click
|
||||
params: { ref: bref_xxx } # 用快照 ref；坐标会因页面重排失效
|
||||
```
|
||||
|
||||
### BrowserScroll
|
||||
|
||||
```yaml
|
||||
BrowserScroll:
|
||||
target: <targetId>
|
||||
direction: bottom # 或 top；与 y 二选一
|
||||
# y: 3000 # 按像素滚动
|
||||
BrowserAct:
|
||||
action: input.text
|
||||
params: { text: 搜索关键词 }
|
||||
```
|
||||
|
||||
### BrowserSetFiles（**需用户确认**）
|
||||
|
||||
为 input[type=file] 设置本地文件，绕过文件对话框。涉及上传，必须经 user confirmation。
|
||||
|
||||
```yaml
|
||||
BrowserSetFiles:
|
||||
target: <targetId>
|
||||
selector: input[type=file]
|
||||
files:
|
||||
- /Users/me/Pictures/photo.png
|
||||
BrowserAct:
|
||||
action: input.wheel
|
||||
params: { deltaX: 0, deltaY: 720, x: 640, y: 400 }
|
||||
```
|
||||
|
||||
### BrowserCloseTab
|
||||
|
||||
```yaml
|
||||
BrowserCloseTab:
|
||||
target: <targetId>
|
||||
BrowserAct:
|
||||
action: tab.activate # 截图前必须先做这一步
|
||||
params: { bounds: { x: 0, y: 0, width: 1280, height: 900 } }
|
||||
```
|
||||
|
||||
任务收尾建议清理你创建的临时 tab，避免 cdp-proxy 的 tab 池堆积（15 min 后会自动 GC，但显式关更整洁）。
|
||||
```yaml
|
||||
BrowserAct:
|
||||
action: page.screenshot
|
||||
params: { format: png } # 结果落 artifact store，回执给 artifact.id / sha256 / bytes
|
||||
```
|
||||
|
||||
### BrowserImport（**需人工审批**）
|
||||
|
||||
把用户浏览器里的登录态 Cookie 导入当前 Space，替代旧版「attach 用户已登录的 Chrome」。
|
||||
|
||||
```yaml
|
||||
BrowserImport:
|
||||
action: discover # discover | plan | apply | ...
|
||||
```
|
||||
|
||||
```yaml
|
||||
BrowserImport:
|
||||
action: apply
|
||||
planId: bip_xxx
|
||||
domains: [xiaohongshu.com] # 必须逐域显式授权
|
||||
conflictStrategy: newer-wins
|
||||
```
|
||||
|
||||
解密与过滤全在 Host 侧完成，**Cookie 值不会进入 Agent 上下文或审计日志**。
|
||||
|
||||
### BrowserShare
|
||||
|
||||
把 Space / Session 委派给另一个 Agent，`shareMode` 可选 `snapshot`（只读副本）、
|
||||
`copy-on-write`（写时复制）、`live-shared`（实时共享）、`handoff`（移交控制权）。
|
||||
|
||||
## 已知边界（照做，别试探）
|
||||
|
||||
| 边界 | 说明 |
|
||||
|------|------|
|
||||
| **截图前必须 `tab.activate`** | 标签页默认停在 `(-10000,-10000,1x1)`，没有合成表面。直接截图会卡满 30 s deadline，**并把标签页宿主打掉**，之后全部报 `BROWSER_TAB_HOST_NOT_FOUND` |
|
||||
| **同时只有一个可见标签页** | `tab.activate` 绑定主窗口、全局互斥。多 Space 可以并发导航/快照/输入，但截图必须逐个 activate 串行 |
|
||||
| **`page.evaluate` 基本不可用** | 每次调用需人工审批；返回的字符串/对象被替换为 `[REDACTED:browser-runtime-value]`（只有 number/boolean/null 穿透）；反调试站点会把它挂起几十秒。读页面用 `BrowserSnapshot` |
|
||||
| **`cdp.raw` 需人工审批** | `browser.raw_cdp.*` 属于永远人工闸门的能力，无人值守流程用不了（元素级裁剪截图因此不可用） |
|
||||
| **没有批量取文通道** | `semantic` 不含正文，`accessibility` 在真实页面上超限，`page.evaluate` 被审批+脱敏。要正文请回落 Jina / Playwright |
|
||||
| **单条命令 30 s deadline** | 超时即判 `browser.host.gone`，会话作废 |
|
||||
| **用户真实鼠标会抢控制权** | 鼠标划过可见标签页会触发 `trusted-user-input` 并递增 control epoch，可能打断 Agent 的租约 |
|
||||
| **artifact 不要走 `/save`** | 该接口会弹系统「另存为」对话框等人点。artifact 文件在 `${DESIRECORE_ROOT}/browser/artifacts/<bart_id>/`，直接读即可，默认保留 24 小时 |
|
||||
|
||||
## SitePatternRead / SitePatternWrite
|
||||
|
||||
@@ -120,7 +170,7 @@ SitePatternWrite:
|
||||
mode: merge # 默认 merge 追加；replace 覆盖
|
||||
content: |
|
||||
## 已知陷阱
|
||||
- 2026-05: ...
|
||||
- 2026-08: ...
|
||||
```
|
||||
|
||||
含 cookie/token/手机号/邮箱时会自动降级 scope='user'。
|
||||
@@ -129,14 +179,18 @@ SitePatternWrite:
|
||||
|
||||
| 错误 | 原因 | 解决 |
|
||||
|------|------|------|
|
||||
| `Chrome 未开启远程调试端口` | 用户没启动调试 Chrome | 引导用户跑 SKILL.md 中的启动命令 |
|
||||
| `attach 失败` | targetId 无效或 tab 已关闭 | 重新 `BrowserListTabs` |
|
||||
| `CDP 命令超时` | 页面长时间未响应 | 检查页面状态，必要时 `BrowserCloseTab` 后重开 |
|
||||
| `CDP proxy 请求被中止或超时` | 代理子进程异常或网络故障 | 等待 ProxyController 自动重启（最多 3 次） |
|
||||
| `该旧 BrowserXxx/cdp-proxy 入口已停用` | 还在调 v2.0 的旧工具 | 改用 BrowserManage / BrowserSnapshot / BrowserAct |
|
||||
| `BROWSER_TAB_HOST_NOT_FOUND` | 标签页宿主已销毁（多因上一条命令超时） | 重建 Session；检查是否漏了 `tab.activate` |
|
||||
| `BROWSER_COMMAND_DEADLINE_EXCEEDED` | 单条命令超 30 s | 截图先 activate；避免 `page.evaluate` |
|
||||
| `BROWSER_TOOL_SESSION_FORBIDDEN` | 会话已关闭/崩溃，或不属于当前 Agent | 重新 `list_sessions`，必要时重建 |
|
||||
| `BROWSER_TOOL_SESSION_AMBIGUOUS` | 当前 Agent 有多个会话且未传 sessionId | 显式传 `sessionId` |
|
||||
| `BROWSER_HUMAN_APPROVAL_REQUIRED` | 触到人工闸门能力（evaluate / raw_cdp / 上传下载 / Cookie 导入等） | 向用户说明用途并等待审批，或换用无需审批的路径 |
|
||||
| `BROWSER_RESULT_TOO_LARGE` | 回执超过 2 MB（`accessibility` 快照最常见） | 改用 `semantic` 快照 + 截图；取正文回落 Jina / Playwright |
|
||||
|
||||
## 调用链路
|
||||
|
||||
```
|
||||
Agent → BrowserXxx 工具 → proxy-client → cdp-proxy 子进程 → Chrome (DevTools Protocol)
|
||||
↑ 首次调用 lazy spawn 子进程
|
||||
Agent → BrowserManage/Snapshot/Act → browser-use service（Capability/Grant/Lease/Policy 校验）
|
||||
→ BrowserHost（electron-embedded 或 standalone-managed）→ Chromium
|
||||
↑ 每步产出带 digest 的回执，写入审计事件流
|
||||
```
|
||||
|
||||
@@ -5,7 +5,7 @@ type: site-pattern
|
||||
pinned: true
|
||||
confidence: medium
|
||||
learned_at: '2026-05-05'
|
||||
updated_at: '2026-05-05'
|
||||
updated_at: '2026-08-06'
|
||||
---
|
||||
|
||||
## L0
|
||||
@@ -18,7 +18,7 @@ updated_at: '2026-05-05'
|
||||
|
||||
## 推荐流程
|
||||
- 仅取标题/UP 主/播放数 → WebFetch 即可
|
||||
- 取完整简介 / 评论 → BrowserNavigate + BrowserEval
|
||||
- 取完整简介 / 评论 → 内置浏览器 BrowserAct(tab.navigate) 到达页面，正文抽取回落 L3-fallback Playwright
|
||||
- 批量数据 → 优先尝试 `api.bilibili.com` 公开接口（有专门的 wbi 签名机制）
|
||||
|
||||
## 推荐选择器
|
||||
|
||||
@@ -5,7 +5,7 @@ type: site-pattern
|
||||
pinned: true
|
||||
confidence: high
|
||||
learned_at: '2026-05-05'
|
||||
updated_at: '2026-05-05'
|
||||
updated_at: '2026-08-06'
|
||||
---
|
||||
|
||||
## L0
|
||||
@@ -19,7 +19,7 @@ updated_at: '2026-05-05'
|
||||
|
||||
## 推荐流程
|
||||
1. 公开文档（companion 页面/官方文档站点 `feishu.cn/hc/...`）→ WebFetch / Jina
|
||||
2. 用户文档 → BrowserNavigate（已登录 Chrome）+ BrowserEval 抓正文
|
||||
2. 用户文档 → 内置浏览器 BrowserImport 导入 Cookie + BrowserAct(tab.navigate) 到达，正文抽取回落 Playwright
|
||||
3. 长期程序化访问 → 申请 OpenAPI access_token，走 HTTP，避免 CDP
|
||||
|
||||
## 推荐选择器
|
||||
|
||||
@@ -5,7 +5,7 @@ type: site-pattern
|
||||
pinned: true
|
||||
confidence: medium
|
||||
learned_at: '2026-05-05'
|
||||
updated_at: '2026-05-05'
|
||||
updated_at: '2026-08-06'
|
||||
---
|
||||
|
||||
## L0
|
||||
@@ -18,8 +18,8 @@ updated_at: '2026-05-05'
|
||||
|
||||
## 推荐流程
|
||||
1. 单条公开微博 → 优先尝试 `https://m.weibo.cn/status/<id>`，WebFetch 看是否拿到正文
|
||||
2. 拿不到 → BrowserNavigate（已登录态 Chrome）+ BrowserClick 点"展开" + BrowserEval 取正文
|
||||
3. 用户主页时间线：必须登录，BrowserScroll 触发懒加载
|
||||
2. 拿不到 → 内置浏览器 BrowserImport 导入 Cookie + BrowserAct(tab.navigate) + BrowserAct(input.click ref) 点"展开"，正文抽取回落 Playwright
|
||||
3. 用户主页时间线：必须登录，BrowserAct(input.wheel) 触发懒加载
|
||||
|
||||
## 推荐选择器
|
||||
- 单条详情：`.weibo-text` / `[class*='WB_text']`
|
||||
|
||||
@@ -5,7 +5,7 @@ type: site-pattern
|
||||
pinned: true
|
||||
confidence: high
|
||||
learned_at: '2026-05-05'
|
||||
updated_at: '2026-05-05'
|
||||
updated_at: '2026-08-06'
|
||||
---
|
||||
|
||||
## L0
|
||||
@@ -18,10 +18,10 @@ updated_at: '2026-05-05'
|
||||
- xsec_token：详情页 URL 包含 `xsec_token=...`，**通过站内交互生成**——直接拼 URL 容易失败
|
||||
|
||||
## 推荐流程
|
||||
1. 通过 BrowserListTabs 找到已登录的 xhs tab；如无，BrowserNavigate 打开
|
||||
2. BrowserEval 注入选择器抓取标题与正文
|
||||
3. 列表页用 `[...document.querySelectorAll('.note-card a')].map(a => a.href)` 收集详情页链接
|
||||
4. 单条笔记结尾调用 `BrowserCloseTab` 释放 tab，避免 tab 池溢出
|
||||
1. BrowserManage(create_space/start_session) → BrowserImport 按域导入 xiaohongshu.com 的 Cookie
|
||||
2. BrowserAct(tab.navigate) 打开笔记；列表页用 BrowserSnapshot(semantic) 收 `.note-card` 链接的 ref
|
||||
3. 正文抽取回落 L3-fallback Playwright（内置浏览器的 semantic 快照不含正文，accessibility 在本站会超限）
|
||||
4. 收尾 BrowserManage(close_session)；ephemeral Space 不留 Profile
|
||||
|
||||
## 推荐选择器
|
||||
- 笔记标题：`#detail-title` 或 `h1[class*="title"]`
|
||||
@@ -36,8 +36,8 @@ updated_at: '2026-05-05'
|
||||
|
||||
## 反爬细节
|
||||
- xsec_token 由前端生成，与会话/路由绑定，不能跨 tab 复用太久
|
||||
- User-Agent 检查较弱，但 navigator.webdriver 必须为 false（cdp-proxy 已默认绕过）
|
||||
- click 操作建议优先用 `mode='real-mouse'`（CDP 真实鼠标事件），避免被 .click() 检测
|
||||
- User-Agent 检查较弱，但 navigator.webdriver 必须为 false
|
||||
- 点击走 `BrowserAct(input.click)`（CDP 派发的真实指针事件），不要用页面内 `.click()`
|
||||
|
||||
## 速率限制
|
||||
- 自评：单 IP 每分钟 ≤ 30 个详情页请求；超过会出验证码
|
||||
|
||||
@@ -5,7 +5,7 @@ type: site-pattern
|
||||
pinned: true
|
||||
confidence: medium
|
||||
learned_at: '2026-05-05'
|
||||
updated_at: '2026-05-05'
|
||||
updated_at: '2026-08-06'
|
||||
---
|
||||
|
||||
## L0
|
||||
@@ -18,8 +18,8 @@ updated_at: '2026-05-05'
|
||||
|
||||
## 推荐流程
|
||||
1. 专栏文章 → WebFetch / Jina Reader 优先
|
||||
2. 问答详情 → BrowserNavigate + BrowserEval；批量回答用 `[...document.querySelectorAll('.AnswerItem')].map(...)`
|
||||
3. 评论 → 必须登录，BrowserClick 展开，BrowserScroll 加载更多
|
||||
2. 问答详情 → 内置浏览器 BrowserAct(tab.navigate) 到达；批量回答的正文抽取回落 Playwright（`.AnswerItem`）
|
||||
3. 评论 → 必须登录，BrowserAct(input.click ref) 展开，BrowserAct(input.wheel) 加载更多
|
||||
|
||||
## 推荐选择器
|
||||
- 文章标题：`h1.Post-Title` / `h1[class*='Title']`
|
||||
|
||||
Reference in New Issue
Block a user