feat(web-access): v3.0 内置浏览器能力面同步——删除 Python Playwright 回落(S31) (#85)

## 变更内容 / What

浏览器升级计划 S31 最终验收信号:web-access 技能与内置受管浏览器新能力面对齐,**删除「用户手工启动调试 Chrome +
Python Playwright」回落路径**。

- **删除回落路径**:Prerequisites: Chrome CDP Setup、Layer 3 CDP Browser(Python
Playwright 模板)、Installation Note(pip install
playwright)整段删除;references/cdp-browser.md 文件删除;jina-reader.md 的 CDP 引用改为
page.extract-text
- **订正陈旧断言**:
- 「没有批量取文通道」→ BrowserSnapshot mode:text /
page.extract-text(maxBytes/cursor 分页,超出截断给 nextCursor)
- 「page.evaluate 基本不可用」→ 返回真实值(expression/awaitPromise,超预算截断标
truncated);仍走人工闸门
- 「截图前必须 tab.activate / 串行截图 / BROWSER_TAB_HOST_NOT_FOUND」→ S36 订正:Agent
单标签会话免 activate;多标签后台 tab 秒级报 BROWSER_VIEWPORT_UNAVAILABLE;命令超时只 stop 不
close,标签页可重试
  - 「accessibility 超限即失败」→ 尊重 depth + maxBytes 截断翻页(S8)
  - 「只有整页截图」→ clip{x,y,width,height,scale≤4} + captureBeyondViewport
- **provides.tools 加 BrowserScript**(code-mode;信任级别等同 Bash)
- **新增选用规则(D4 唯一约束机制)**:反检测站点一律优先 input.*(#1808 输入拟真 +
身份一致性);page.element 写类仅用于表单批量填充等站点不检测场景;JS 直调 el.click() 为禁止回退
- **新增 fetch.browser 配方**:page.evaluate 页面上下文跑 fetch(带 origin Cookie、同
origin、受 Grant origins 约束)——登录态取站内接口的正解
- **能力速查**:page.element 九 op / page.wait 九 until / inline wait 块 / loc=
方言 / BrowserScript / 跨源 iframe 快照(S35)
- **版本** 2.2.1 → 3.0.0(删除回落层为 breaking);source_hash
重算;required_client_version 维持 10.0.98(新能力在正文标注 10.0.112+)

## Why

v2.x 时代回落路径存在的每一条理由(无批量取文、evaluate 不可用、截图必须串行 activate)均已被
S2–S14/S35/S36 覆盖;文档继续引导用户手工起调试 Chrome 会误导新 Agent 走已废弃路径。

双语同步修改(SKILL.md / SKILL.zh-CN.md heading 数一致,i18n-validate 通过)。

- [x] CLA
This commit is contained in:
2026-08-17 01:25:46 -04:00
committed by GitHub
parent 43239669e5
commit 0d4d7b208b
5 changed files with 482 additions and 868 deletions

View File

@@ -5,18 +5,17 @@ description: >-
— searching for current information, fetching public web pages, browsing — searching for current information, fetching public web pages, browsing
login-gated sites (微博/小红书/B站/飞书/Twitter), comparing products, login-gated sites (微博/小红书/B站/飞书/Twitter), comparing products,
researching topics, gathering documentation, or summarizing news. researching topics, gathering documentation, or summarizing news.
This skill orchestrates four complementary layers: (1) WebSearch + WebFetch This skill orchestrates three complementary layers: (1) WebSearch + WebFetch
for public pages, (2) Jina Reader as the default token-optimization layer for for public pages, (2) Jina Reader as the default token-optimization layer for
heavy/JS-rendered pages, (3) the governed built-in browser (isolated heavy/JS-rendered pages, and (3) the governed built-in browser (isolated
BrowserSpace + Cookie import) to reach and interact with login-gated sites, BrowserSpace + Cookie import + bulk text extraction + waits + code mode) to
and (4) Chrome DevTools Protocol via Python Playwright as the fallback for reach, interact with, and read login-gated sites. Always cite source URLs.
bulk text extraction and complex automation. Always cite source URLs.
Use when 用户提到 联网搜索、上网查、 Use when 用户提到 联网搜索、上网查、
查资料、抓取网页、研究、调研、最新资讯、文档查询、对比、竞品、技术文档、 查资料、抓取网页、研究、调研、最新资讯、文档查询、对比、竞品、技术文档、
新闻、网址、URL、找一下、搜一下、查一下、小红书、B站、微博、飞书、Twitter、 新闻、网址、URL、找一下、搜一下、查一下、小红书、B站、微博、飞书、Twitter、
推特、X、知乎、公众号、已登录、登录状态。 推特、X、知乎、公众号、已登录、登录状态。
license: Complete terms in LICENSE.txt license: Complete terms in LICENSE.txt
version: 2.2.1 version: 3.0.0
type: procedural type: procedural
risk_level: low risk_level: low
status: enabled status: enabled
@@ -27,13 +26,14 @@ tags:
- fetch - fetch
- research - research
- browsing - browsing
- browser
- cdp - cdp
- playwright
provides: provides:
tools: tools:
- BrowserManage - BrowserManage
- BrowserSnapshot - BrowserSnapshot
- BrowserAct - BrowserAct
- BrowserScript
- BrowserImport - BrowserImport
- BrowserShare - BrowserShare
- SitePatternRead - SitePatternRead
@@ -41,7 +41,7 @@ provides:
- LocalBookmarks - LocalBookmarks
metadata: metadata:
author: desirecore author: desirecore
updated_at: '2026-08-06' updated_at: '2026-08-17'
i18n: i18n:
default_locale: en-US default_locale: en-US
source_locale: zh-CN source_locale: zh-CN
@@ -50,17 +50,17 @@ metadata:
- en-US - en-US
zh-CN: zh-CN:
name: 联网访问 name: 联网访问
short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问、研究调研工作流 short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问与取文、研究调研工作流
description: 层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问、Python Playwright CDP 兜底 description: 层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问、交互与取文
body: ./SKILL.zh-CN.md body: ./SKILL.zh-CN.md
source_hash: sha256:ce46e1982de8cd65 source_hash: sha256:20c98f047378220a
translated_by: human translated_by: human
en-US: en-US:
name: Web Access name: Web Access
short_desc: Web search, page fetching, logged-in access via the governed built-in browser, research workflows short_desc: Web search, page fetching, logged-in access via the governed built-in browser, research workflows
description: A four-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, reach logged-in sites through the governed built-in browser, and fall back to Chrome CDP. description: A three-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, and reach, interact with, and read logged-in sites through the governed built-in browser.
body: ./SKILL.md body: ./SKILL.md
source_hash: sha256:ce46e1982de8cd65 source_hash: sha256:20c98f047378220a
translated_by: human translated_by: human
market: market:
icon: >- icon: >-
@@ -73,9 +73,8 @@ market:
stroke="url(#wa-a)" stroke-width="1" stroke="url(#wa-a)" stroke-width="1"
stroke-opacity="0.35"/><path d="M2 10h16" stroke="url(#wa-a)" stroke-opacity="0.35"/><path d="M2 10h16" stroke="url(#wa-a)"
stroke-width="1" stroke-opacity="0.35"/><path d="M10 2v16" stroke-width="1" stroke-opacity="0.35"/><path d="M10 2v16"
stroke="url(#wa-a)" stroke-width="1" stroke="url(#wa-a)" stroke-width="1" stroke-opacity="0.35"/><circle cx="18.5"
stroke-opacity="0.35"/><circle cx="18.5" cy="18.5" r="2.5" cy="18.5" r="2.5" stroke="#34C759" stroke-width="1.5" fill="#34C759"
stroke="#34C759" stroke-width="1.5" fill="#34C759"
fill-opacity="0.12"/><path d="M20.5 20.5l2 2" stroke="#34C759" fill-opacity="0.12"/><path d="M20.5 20.5l2 2" stroke="#34C759"
stroke-width="1.8" stroke-linecap="round"/></svg> stroke-width="1.8" stroke-linecap="round"/></svg>
category: research category: research
@@ -90,28 +89,30 @@ market:
## L0: One-line Summary ## L0: One-line Summary
A four-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, reach login-gated sites through the governed built-in browser, and fall back to Chrome CDP. A three-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, and reach, interact with, and read login-gated sites through the governed built-in browser (v3.0 ships bulk text extraction, waits, and code mode in-browser; the Python Playwright fallback is gone).
## L1: Overview & Use Cases ## L1: Overview & Use Cases
### Capability ### Capability
web-access is a **procedural skill** that provides four complementary layers of web access: web-access is a **procedural skill** that provides three complementary layers of web access:
- **L1** (WebSearch + WebFetch): public, static pages - **L1** (WebSearch + WebFetch): public, static pages
- **L2** (Jina Reader): JS-rendered heavy pages, saving tokens by default - **L2** (Jina Reader): JS-rendered heavy pages, saving tokens by default
- **L3-fast** (governed built-in browser**rewritten in v2.1**): reach and *interact with* logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. **Reading long article text is still L2/L3-fallback's job**see the extraction note below - **L3** (governed built-in browser, capability surface completed in v3.0): reach, *interact with*, and **read** logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. Bulk text extraction (`page.extract-text`), discriminated waits (`page.wait`), and code mode (`BrowserScript`) all close the loop inside this layer
- **L3-fallback** (Chrome CDP + Python Playwright): backup for complex automation (long waits, race conditions, custom in-browser scripts)
### v2.1 — governed built-in browser (default-hidden, exposed only after Skill activation) The v2.x fourth layer — "user manually launches a debug Chrome + Python Playwright CDP" — was removed in v3.0: every reason it existed for (no bulk text channel, evaluate unusable, screenshots must activate-serialize) is now covered by the built-in browser, see the cheatsheet below.
When you call `Skill('web-access')`, the following 8 tools are injected into the current session so the LLM can drive the built-in browser directly: ### v3.0: governed built-in browser (default-hidden, exposed only after Skill activation)
When you call `Skill('web-access')`, the following 9 tools are injected into the current session so the LLM can drive the built-in browser directly:
| Tool | Purpose | | Tool | Purpose |
|------|---------| |------|---------|
| BrowserManage | Create/destroy isolated BrowserSpace, start sessions, manage tabs | | BrowserManage | Create/destroy isolated BrowserSpace, start sessions, manage tabs |
| BrowserSnapshot | `semantic` / `accessibility` / `visual` page snapshots — the primary way to read a page | | BrowserSnapshot | `semantic` / `text` / `accessibility` / `visual` snapshots — the primary way to read a page |
| BrowserAct | One governed action per call: navigate, click, type, scroll, screenshot, … | | BrowserAct | One governed action per call: navigate, input, extract text, wait, element ops, screenshot, … |
| BrowserScript | **Code mode**: one async JS script issues browser commands back-to-back, eliminating per-action round trips (trust level equals Bash) |
| BrowserImport | Import Cookies from the user's Chrome/Edge/Firefox/Safari profile (human-approved; **needs `browser.import.*` granted by the Host — not available to a plain `create_space` session**) | | BrowserImport | Import Cookies from the user's Chrome/Edge/Firefox/Safari profile (human-approved; **needs `browser.import.*` granted by the Host — not available to a plain `create_space` session**) |
| BrowserShare | Delegate a Space/Session to another Agent (isolated / snapshot / copy-on-write / live) | | BrowserShare | Delegate a Space/Session to another Agent (isolated / snapshot / copy-on-write / live) |
| SitePatternRead / SitePatternWrite | Per-domain "site experience" (AgentFS three-layer) | | SitePatternRead / SitePatternWrite | Per-domain "site experience" (AgentFS three-layer) |
@@ -119,20 +120,22 @@ When you call `Skill('web-access')`, the following 8 tools are injected into the
> **Important**: before `Skill('web-access')` is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See [references/browser-tools.md](references/browser-tools.md). > **Important**: before `Skill('web-access')` is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See [references/browser-tools.md](references/browser-tools.md).
> >
> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+. > **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+; `page.extract-text` / `page.element` / `page.wait` / inline wait blocks / cross-origin iframe snapshots need v10.0.112+, and `BrowserScript` needs a build containing S17/S18.
### Use Cases ### Use Cases
- The user needs to search for current information or research a specific topic - The user needs to search for current information or research a specific topic
- The user needs to fetch public web content or technical documentation - The user needs to fetch public web content or technical documentation
- The user needs to access logged-in sites (Xiaohongshu, Bilibili, Weibo, Feishu, Twitter, etc.) - The user needs to access logged-in sites (Xiaohongshu, Bilibili, Weibo, Feishu, Twitter, etc.) and **read the body text**
- The user needs to pull data from a site's own API in a logged-in context (lists, comments, orders, …)
- The user needs to compare products, aggregate news, or investigate API/library versions - The user needs to compare products, aggregate news, or investigate API/library versions
### Core Value ### Core Value
- **Four-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand - **Three-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand
- **Token optimization**: Jina Reader cuts token usage by 5080% by default - **Token optimization**: Jina Reader cuts token usage by 5080% by default; `page.extract-text`'s maxBytes/cursor paging keeps even long logged-in articles under control
- **Logged-in session reuse**: attach to the user's Chrome via CDP in the fallback layer, or — where the Host has granted `browser.import.*` — import their Cookies into an isolated Space via BrowserImport; either way, no re-login required - **Logged-in session reuse**: where the Host has granted `browser.import.*`, BrowserImport brings the user's Cookies into an isolated Space no re-login required
- **Zero external dependencies**: no Python/Playwright install, no manually launched debug Chrome
## L2: Detailed Specification ## L2: Detailed Specification
@@ -144,53 +147,6 @@ When you complete a research task, you **MUST** cite all source URLs in your res
If any fetch fails, explicitly tell the user which URL failed and which fallback you used. If any fetch fails, explicitly tell the user which URL failed and which fallback you used.
---
## Prerequisites: Chrome CDP Setup (for login-gated sites)
**Required for the L3-fallback layer** (Python Playwright) — and, in practice, still the reliable way to reuse a login. The L3-fast built-in browser can skip it only when the Host has granted `browser.import.*` so that `BrowserImport` actually works; otherwise set this up.
### One-time setup
Launch a dedicated Chrome instance with remote debugging enabled:
**macOS**:
```bash
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Linux**:
```bash
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Windows (PowerShell)**:
```powershell
& "C:\Program Files\Google\Chrome\Application\chrome.exe" `
--remote-debugging-port=9222 `
--user-data-dir="$env:USERPROFILE\.desirecore\chrome-profile"
```
After launch:
1. Manually log in to the sites you need (Xiaohongshu, Bilibili, Weibo, Feishu, …)
2. Leave this Chrome window open in the background
3. Verify the debug endpoint: `curl -s http://localhost:9222/json/version` should return JSON
### Verify CDP is ready
Before any CDP operation, always run:
```bash
curl -s http://localhost:9222/json/version | python3 -c "import sys,json; d=json.load(sys.stdin); print('CDP ready:', d.get('Browser'))"
```
If the command fails, tell the user: "Please launch Chrome with the remote debugging port enabled (see the Prerequisites section of the web-access skill)."
---
## Tool Selection Decision Tree ## Tool Selection Decision Tree
``` ```
@@ -207,12 +163,12 @@ User intent
│ (Jina Reader = default for JS-rendered content, saves tokens) │ (Jina Reader = default for JS-rendered content, saves tokens)
├─ "Read this login-gated page" (Xiaohongshu/Bilibili/Weibo/Feishu/Twitter/Zhihu/WeChat) ├─ "Read this login-gated page" (Xiaohongshu/Bilibili/Weibo/Feishu/Twitter/Zhihu/WeChat)
─→ Reach it: BrowserManage(create_space/start_session) → BrowserAct(tab.navigate) ─→ BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
→ tab.activate + page.screenshot to confirm you landed on the content → BrowserAct(page.extract-text) ← body text read out directly, maxBytes/cursor paging
(semantic snapshot has no body text). Logged-in? BrowserImport only if │ Logged in? BrowserImport only if browser.import.* was granted
│ browser.import.* was granted — otherwise reuse the login via CDP below.
│ └─→ Extract the text: verify CDP ready, then python3 playwright.connect_over_cdp() ├─ "Pull data from the site's API in a logged-in context"
→ page.content() → Jina Reader / BeautifulSoup └─→ fetch.browser recipe: run fetch inside BrowserAct(page.evaluate) with that origin's cookies
├─ "API documentation / GitHub / npm package info" ├─ "API documentation / GitHub / npm package info"
│ └─→ Prefer official API endpoints over scraping HTML: │ └─→ Prefer official API endpoints over scraping HTML:
@@ -221,24 +177,19 @@ User intent
│ - PyPI: curl https://pypi.org/pypi/<pkg>/json │ - PyPI: curl https://pypi.org/pypi/<pkg>/json
└─ "Real-time interactive task" (click, fill form, scroll, screenshot) └─ "Real-time interactive task" (click, fill form, scroll, screenshot)
─→ **Default: built-in browser** (BrowserManage → BrowserAct → BrowserSnapshot — ─→ built-in browser (BrowserManage → BrowserAct → BrowserSnapshot —
see references/browser-tools.md, no Python needed) see references/browser-tools.md, no Python needed)
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when the built-in browser is insufficient
(e.g., complex race conditions, multi-event waits, long-running in-browser scripts)
``` ```
### Four-layer strategy summary ### Three-layer strategy summary
| Layer | Use case | Primary tool | Token cost | | Layer | Use case | Primary tool | Token cost |
|-------|----------|--------------|------------| |-------|----------|--------------|------------|
| L1 | Public, static | `WebFetch` | Low | | L1 | Public, static | `WebFetch` | Low |
| L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) | | L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) |
| **L3-fast** | **Login-gated navigation & interaction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot)** | Medium | | **L3** | **Login-gated navigation, interaction & extraction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript)** | Medium |
| L3-fallback | Complex automation (race / long-wait / custom scripts) | `Bash + Python Playwright CDP` | Medium |
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when the built-in browser is insufficient. **Default priority**: L1 for simple public pages → L2 for heavy → **L3 for login-gated (body text and in-site API data included)**.
---
## Supported Sites Matrix ## Supported Sites Matrix
@@ -248,16 +199,14 @@ User intent
| GitHub README, issues, PRs | `gh api` (best) → L1 WebFetch | Prefer API | | GitHub README, issues, PRs | `gh api` (best) → L1 WebFetch | Prefer API |
| Hacker News, Reddit | L1 WebFetch | Public content | | Hacker News, Reddit | L1 WebFetch | Public content |
| Medium, Dev.to | L2 Jina Reader | JS-rendered, member gates | | Medium, Dev.to | L2 Jina Reader | JS-rendered, member gates |
| Twitter/X | L3 CDP (or L2 Jina with `x.com`) | Login required for full thread | | Twitter/X | L3 (or L2 Jina with `x.com`) | Login required for full thread |
| Xiaohongshu (xiaohongshu.com) | L3 CDP | Login required | | Xiaohongshu (xiaohongshu.com) | L3 built-in browser + BrowserImport | Login required; body text via page.extract-text |
| Bilibili (bilibili.com) | L3 CDP | Login needed for video desc/comments | | Bilibili (bilibili.com) | L3 built-in browser + BrowserImport | Login needed for video desc/comments |
| Weibo (weibo.com) | L3 CDP | Long posts require login | | Weibo (weibo.com) | L3 built-in browser + BrowserImport | Long posts require login |
| Zhihu (zhihu.com) | L3 CDP | Long articles + comments require login | | Zhihu (zhihu.com) | L3 built-in browser + BrowserImport | Long articles + comments require login |
| Feishu Docs (feishu.cn) | L3 CDP | Login required | | Feishu Docs (feishu.cn) | L3 built-in browser + BrowserImport | Login required |
| WeChat Official Accounts (mp.weixin.qq.com) | L2 Jina Reader | Usually public, Jina cleans better | | WeChat Official Accounts (mp.weixin.qq.com) | L2 Jina Reader | Usually public, Jina cleans better |
| LinkedIn | L3 CDP | Login wall | | LinkedIn | L3 built-in browser + BrowserImport | Login wall |
---
## Tool Reference ## Tool Reference
@@ -280,7 +229,7 @@ WebFetch(url="https://example.com/article")
Tips: Tips:
- Results cached for 15 min - Results cached for 15 min
- Returns cleaned Markdown with title + URL + body - Returns cleaned Markdown with title + URL + body
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (CDP) - If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (built-in browser)
### Layer 2: Jina Reader (default for heavy pages) ### Layer 2: Jina Reader (default for heavy pages)
@@ -298,111 +247,94 @@ Why Jina is the default token-saver:
See [references/jina-reader.md](references/jina-reader.md) for advanced endpoints and rate limits. See [references/jina-reader.md](references/jina-reader.md) for advanced endpoints and rate limits.
### Layer 3: CDP Browser (login-gated access) ### Layer 3: built-in browser (login-gated access)
Use Python Playwright's `connect_over_cdp()` to attach to the user's running Chrome (which already has login cookies). **No re-login needed.** The full command surface, capability tiers, and boundaries are in [references/browser-tools.md](references/browser-tools.md). The loop:
**Minimal template**: 1. `BrowserManage(create_space)``BrowserManage(start_session)` for a sessionId + first tab
```bash 2. Need login state → `BrowserImport` (only if the Host granted `browser.import.*`; without that grant the user's Cookies cannot be reused — tell the user and continue without login or abort)
python3 << 'PY' 3. `BrowserAct(tab.navigate)` to reach the page
from playwright.sync_api import sync_playwright 4. `BrowserSnapshot(semantic)` for interactive-element `ref` handles (cross-origin iframe elements are in the same tree with globally sequential refs)
5. Interact via `input.*` (humanized trajectories) or `page.element` (bulk form writes); wait for results via `page.wait` or inline wait blocks
6. Read body text via `BrowserSnapshot(text)` or `BrowserAct(page.extract-text)`; pull API data via the fetch.browser recipe
7. `BrowserManage(close_session)` when done
TARGET_URL = "https://www.xiaohongshu.com/explore/..." Multi-step sequences (navigate→snapshot→click→wait→extract) can be done in one `BrowserScript` run, skipping the per-action IPC round trips.
with sync_playwright() as p: ## L3 Cheatsheet (v3.0)
browser = p.chromium.connect_over_cdp("http://localhost:9222")
context = browser.contexts[0] # reuse user's default context (has cookies)
page = context.new_page()
page.goto(TARGET_URL, wait_until="domcontentloaded")
page.wait_for_timeout(2000) # let lazy content load
html = page.content()
page.close()
# Print first 500 chars to verify ### Reading a page: pick the channel by need
print(html[:500])
PY
```
**Extract text via BeautifulSoup** (no Jina round-trip):
```bash
python3 << 'PY'
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto("https://www.bilibili.com/video/BV...", wait_until="networkidle")
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
title = soup.select_one("h1.video-title")
desc = soup.select_one(".video-desc")
print("Title:", title.get_text(strip=True) if title else "N/A")
print("Desc:", desc.get_text(strip=True) if desc else "N/A")
PY
```
See [references/cdp-browser.md](references/cdp-browser.md) for:
- Per-site selectors (Xiaohongshu / Bilibili / Weibo / Zhihu / Feishu)
- Scrolling & lazy-load patterns
- Screenshot & form-fill recipes
- Troubleshooting connection issues
---
## L3-fast: Built-in Browser Cheatsheet (v2.1)
**Only after you call `Skill('web-access')` will the following tools appear in `tools[]`.**
| Tool | One-line example |
|------|-----------------|
| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | One isolated Space per task |
| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | Start a session, returns sessionId + first tab |
| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | Tab / lifecycle management |
| `BrowserAct({ action: 'tab.navigate', params: { url } })` | Navigate the current tab |
| `BrowserSnapshot({ mode: 'semantic' })` | Read the page: interactive elements + `ref` handles |
| `BrowserAct({ action: 'input.click', params: { ref } })` | Click by snapshot `ref`, never by raw x/y |
| `BrowserAct({ action: 'input.text', params: { text } })` | Type into the focused element |
| `BrowserAct({ action: 'input.wheel', params: { x: 640, y: 400, deltaX: 0, deltaY: 720 } })` | Scroll to trigger lazy loading — `x`/`y` (or `ref`) is **required**, and the session needs `browser.input.pointer.wheel` |
| `BrowserAct({ action: 'tab.activate', params: { bounds } })``page.screenshot` | **activate first, then screenshot** |
| `BrowserImport({ action: 'discover' \| 'create_plan' \| 'dry_run' \| 'apply' \| 'rollback' \| 'list_plans' })` | Reuse the user's login cookies — those 6 are the **complete** action set; happy path is `discover → create_plan → dry_run → apply`. **Needs `browser.import.*`, which `create_space` does not grant** (see below) |
Full API and edge cases: see [references/browser-tools.md](references/browser-tools.md).
**How to read a page** — pick by what you need:
| You need | Use | Note | | You need | Use | Note |
|----------|-----|------| |----------|-----|------|
| Interactive elements + `ref` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons / inputs / links only — **no article text** | | Interactive elements + `ref` / `loc=` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons/inputs/links + per-line `[ref=eN]` and (when producible) `[loc=...]` stable selectors; cross-origin iframes in the same tree |
| Article text on a small/simple page | `BrowserSnapshot({ mode: 'accessibility' })` | Returns StaticText nodes; fails with `BROWSER_RESULT_TOO_LARGE` on real content pages (2 MB result cap, and the `depth` argument is currently ignored by the host) | | Article body text | `BrowserSnapshot({ mode: 'text' })` or `BrowserAct({ action: 'page.extract-text' })` | markdown/text formats; beyond maxBytes it truncates and hands back a nextCursor for paging — no error |
| Article text on a real page | L2 Jina Reader (public) or L3-fallback Playwright (login-gated) | The built-in browser has no working bulk text-extraction channel yet | | Accessibility tree | `BrowserSnapshot({ mode: 'accessibility' })` | Respects `depth` (default 50, max 100) and the maxBytes budget — truncates + pages instead of failing wholesale |
| What the page looks like | `tab.activate` `BrowserAct({ action: 'page.screenshot' })` | Full-page only; read it visually | | What the page looks like | `BrowserSnapshot({ mode: 'visual' })` or `BrowserAct({ action: 'page.screenshot' })` | Pixels land directly in the result (vision models read them in place); `clip={x,y,width,height,scale}` for element-level crops (scale up to 4) and `captureBeyondViewport` for full-page capture |
`page.evaluate` is **not** an extraction channel: it needs human approval per call and its string/object return values come back as `[REDACTED:browser-runtime-value]`. ### Command surface at a glance
`BrowserAct` actions grouped by purpose (full enum in the tool schema):
- **tab.***: `navigate` / `back` / `forward` / `reload` / `activate` / `close`
- **input.***: `move` / `click` / `double-click` / `drag` / `wheel` / `touch` / `pinch` / `key` / `text` — humanized input (#1808: consistent UA/UA-CH identity + real trajectories); the preferred interaction channel on anti-bot sites
- **page.element** (discriminated op × selector, nine ops): write ops `fill` / `select-option` / `check` / `uncheck` / `scroll-into-view`; read ops `get-attribute` / `bounding-box` / `count` / `all-inner-texts`. Selectors speak the `loc=` dialect or a snapshot `ref` (with snapshotId). `fill` refuses `input[type=password]`
- **page.wait** (discriminated until, nine values): poll-type `load` / `domcontentloaded` / `networkidle` / `selector` / `url` / `timeout` return `waited:false` on timeout; event-type `request` / `response` / `download` throw on timeout. Default 10s, max 60s
- **Inline wait blocks**: `params.wait` (isomorphic to page.wait params) on `tab.navigate` / `input.click` / `input.key` / `page.element{op:"fill"}` — one receipt completes "act→wait for result", the waiter registers before the action, no cross-IPC race
- **page.evaluate**: `{ expression, awaitPromise }`, return values cross as-is (over-budget results truncate with a `truncated` flag, never throw); capability `browser.page.evaluate` sits behind the human gate (allow-all mode skips the card)
- **page.extract-text / page.screenshot / page.wait**: see the table above and the fetch.browser recipe
`loc=` selector dialect (S7/S9): `e<N>` (must carry the snapshotId of the snapshot that issued the ref), `loc=css:` / `loc=role:` / `loc=text:` / `loc=testid:`, bare CSS, composable with `internal:nth/last/scope/filter`. Unknown prefixes fail explicitly — never silently degrade to CSS.
### Choosing the interaction channel: input.* vs page.element
- **On anti-bot sites (Xiaohongshu/Weibo/Bilibili etc.) always prefer `input.*`**: it rides the #1808 humanized input pipeline (coordinate dispatch, humanized trajectories, auditable visualization) plus the identity layer that keeps UA/UA-CH free of Electron/Headless tells
- **`page.element` write ops fit bulk form filling on sites that don't detect automation**: one call fills/selects/checks, far faster than per-element input.click + input.text
- **Red line**: `page.element` deliberately has no click — pointer actions must go through `input.*`; invoking `el.click()` via JS bypasses the entire humanization investment and is an explicitly forbidden fallback
### The fetch.browser recipe: pull API data with login state
The right way to pull a site's own API (lists, comments, orders, any JSON) in a logged-in context: run `fetch` **in the page context** — it carries that origin's cookies automatically, stays same-origin, is bounded by Grant origins, and rides the existing `page.evaluate` gate. It is a wrapper usage of `BrowserAct({ action: 'page.evaluate' })`:
```yaml
BrowserAct:
action: page.evaluate
params:
expression: |
fetch('/api/v1/comments?page=1&size=20', {
headers: { accept: 'application/json' }
}).then(r => r.text())
awaitPromise: true # default true; waits for the returned Promise to settle
```
Notes:
- `tab.navigate` to any page on the site first (establishes the origin and cookies), then fire the fetch; use a relative path so it is same-origin by construction
- Return values cross as-is; for large JSON take `.text()` and slice it yourself, or page through multiple calls
- Only the current tab's origin is reachable (Grant origins constraint); for another site's API, navigate there first
- `page.evaluate` is a human-gated capability: outside allow-all mode an approval card appears — explain the purpose to the user
### Recommended flow (Xiaohongshu example) ### Recommended flow (Xiaohongshu example)
``` ```
1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' }) 1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] }) 2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
← an explicit list *narrows* the lease; include browser.input.pointer.wheel if you will scroll ← an explicit list *narrows* the lease; include browser.input.pointer.wheel to scroll
3. Reuse the login: L3-fallback Playwright against the user's own Chrome is the default path. and browser.observe.snapshot to extract text
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }) only works if the 3. Reuse the login: only when the Host granted browser.import.*,
Host granted browser.import.* separately — create_space alone does not. If it is denied, fall back. BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }).
Without that grant the user's cookies cannot be reused — say so, then continue
as logged-out or abort.
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } }) 4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
5. BrowserSnapshot({ mode: 'semantic' }) ← element refs for interaction (no body text) 5. BrowserSnapshot({ mode: 'semantic' }) ← interaction refs (cross-origin iframes in the same tree)
tab.activate + page.screenshot ← confirm the note actually rendered 6. BrowserAct({ action: 'page.extract-text', params: { format: 'markdown' } })
→ extract the note text via L3-fallback Playwright ← body text read out directly; if too long, pass back the returned nextCursor to continue
6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience 7. Need to confirm rendering → BrowserSnapshot({ mode: 'visual' }) (pixels readable in place)
7. At task end → BrowserManage({ action: 'close_session', sessionId }) 8. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience
8. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) 9. At task end → BrowserManage({ action: 'close_session', sessionId })
10. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
``` ```
--- ## Site Experience Accumulation
## Site Experience Accumulation (v2.0)
When the task ends and you've discovered new anti-bot pitfalls, effective selectors, or platform quirks, call: When the task ends and you've discovered new anti-bot pitfalls, effective selectors, or platform quirks, call:
@@ -411,7 +343,7 @@ SitePatternWrite({
domain: "xiaohongshu.com", domain: "xiaohongshu.com",
scope: "agent", // agent=shared (Git-tracked, can be published); user=private scope: "agent", // agent=shared (Git-tracked, can be published); user=private
mode: "merge", // merge appends; replace overwrites mode: "merge", // merge appends; replace overwrites
content: "## Known pitfalls\n- 2026-05: ...", content: "## Known pitfalls\n- 2026-08: ...",
confidence: "medium" confidence: "medium"
}) })
``` ```
@@ -427,8 +359,6 @@ SitePatternRead({ domain: "xiaohongshu.com" })
Content containing cookies / tokens / phone numbers / emails will **automatically downgrade scope='user'** and notify you. Content containing cookies / tokens / phone numbers / emails will **automatically downgrade scope='user'** and notify you.
---
## Common Workflows ## Common Workflows
Read [references/workflows.md](references/workflows.md) for detailed templates: Read [references/workflows.md](references/workflows.md) for detailed templates:
@@ -437,11 +367,9 @@ Read [references/workflows.md](references/workflows.md) for detailed templates:
- News aggregation & timelines - News aggregation & timelines
- API/library version investigation - API/library version investigation
Read [references/cdp-browser.md](references/cdp-browser.md) for login-gated site recipes (Xiaohongshu / Bilibili / Weibo / Zhihu / Feishu).
Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader positioning, rate limits, and advanced endpoints. Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader positioning, rate limits, and advanced endpoints.
--- Read [references/browser-tools.md](references/browser-tools.md) for the full built-in browser command surface, capability tiers, and known boundaries.
## Quick Workflow: Multi-Source Research ## Quick Workflow: Multi-Source Research
@@ -455,27 +383,22 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
7. Report with inline [source](url) citations + a Sources list at the end 7. Report with inline [source](url) citations + a Sources list at the end
``` ```
---
## Anti-Patterns (Avoid) ## Anti-Patterns (Avoid)
-**Using WebFetch on obviously heavy sites** — Medium, Twitter, Xiaohongshu will waste tokens or fail. Jump straight to L2/L3. -**Using WebFetch on obviously heavy sites** — Medium, Twitter, Xiaohongshu will waste tokens or fail. Jump straight to L2/L3.
-**Launching headless Chrome instead of CDP attach** — loses user's login state, triggers anti-bot, slow cold start. Always use `connect_over_cdp()` to attach to the user's existing session.
-**Fetching one URL at a time when you need 5** — batch in a single message. -**Fetching one URL at a time when you need 5** — batch in a single message.
-**Trusting a single source** — cross-check ≥ 2 sources for non-trivial claims. -**Trusting a single source** — cross-check ≥ 2 sources for non-trivial claims.
-**Fetching the search result page itself** — WebSearch already returns snippets; fetch the actual articles. -**Fetching the search result page itself** — WebSearch already returns snippets; fetch the actual articles.
-**Ignoring the cache** — WebFetch caches 15 min, reuse freely. -**Ignoring the cache** — WebFetch caches 15 min, reuse freely.
-**Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs. -**Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs; a logged-in site's own API goes through the fetch.browser recipe.
-**Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current. -**Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current.
-**Hardcoding login credentials in scripts**always rely on the user's pre-logged CDP session. -**Hardcoding login credentials in scripts**login state can only come from Cookies imported via BrowserImport.
-**Citing only after the fact** — collect URLs as you fetch, not from memory afterwards. -**Citing only after the fact** — collect URLs as you fetch, not from memory afterwards.
-**(v2.1) Writing Python heredoc when the built-in browser would do** — slow, requires Python+Playwright install, and bloats context. Prefer L3-fast; fall back to Python only when the built-in browser is insufficient (race / long-wait / custom scripts). -**(v3.0) Using page.element for bulk interaction on anti-bot sites** — it is a direct JS call that bypasses humanized trajectories; on anti-bot sites always use `input.*`; keep `page.element` for bulk form filling where the site doesn't detect automation.
-**(v2.1) Calling `page.screenshot` before `tab.activate`** — tabs are parked off-screen at `(-10000,-10000,1x1)` and have no compositing surface, so the capture stalls until the 30s deadline **and the tab host is destroyed**; every later call then fails with `BROWSER_TAB_HOST_NOT_FOUND`. Always `tab.activate` with real bounds first. -**(v3.0) Reading body text from screenshots** — `page.extract-text` / `BrowserSnapshot(text)` hand you markdown/text with paged budgets; save screenshots for layout confirmation and CAPTCHAs where you truly must look.
-**(v2.1) Reaching for `page.evaluate` to read a page** — it needs human approval on every call, its string/object return values come back as `[REDACTED:browser-runtime-value]` (only number/boolean/null survive), and anti-debug sites stall it for tens of seconds. Read pages with `BrowserSnapshot` instead. -**(v3.0) Tolerating per-action round trips when BrowserScript would do** — navigate→snapshot→click→wait→extract runs as one script; remember BrowserScript's trust level equals Bash and the script source passes one human approval.
-**(v2.1) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`. -**(v3.0) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`.
-**(v2.1) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer. -**(v3.0) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.
---
## Example Interaction ## Example Interaction
@@ -484,26 +407,15 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
**Agent workflow**: **Agent workflow**:
``` ```
1. Recognize → Xiaohongshu is an L3 logged-in site 1. Recognize → Xiaohongshu is an L3 logged-in site
2. Check CDP: curl -s http://localhost:9222/json/version 2. BrowserManage(create_space + start_session)
├─ Failure → prompt the user to launch Chrome in debug mode, abort Need login state → if browser.import.* was granted, run the BrowserImport four steps;
└─ Success → continue otherwise tell the user the login cannot be reused and continue with the public part
3. Bash: python3 connect_over_cdp script → page.goto(url) → page.content() 3. BrowserAct(tab.navigate → the note URL)
4. BeautifulSoup extract h1 title, .note-content, .comments 4. BrowserAct(page.extract-text, format: markdown)
← body text read out directly; page with nextCursor if over budget
5. When returning to the user: 5. When returning to the user:
- Cite the original URL - Cite the original URL
- If content is long, run it through Jina to save tokens - Quote facts from the extracted text with source links
6. Tell the user: "Fetched via your logged-in session, original link: [xhs](url)" 6. Tell the user: "Fetched via the built-in browser, original link: [xhs](url)"
7. BrowserManage(close_session)
``` ```
---
## Installation Note
CDP features require Python + Playwright installed:
```bash
pip3 install playwright beautifulsoup4
python3 -m playwright install chromium # only needed if user hasn't installed Chrome
```
If `playwright` is not installed when the user requests a login-gated site, run the install commands in Bash and explain you're setting up the browser automation dependency.

View File

@@ -4,28 +4,30 @@
## L0一句话摘要 ## L0一句话摘要
层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问v2.1)、Python Playwright CDP 兜底。 层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问与交互v3.0 起取文/等待/脚本全部内置,不再需要 Python Playwright 兜底
## L1概述与使用场景 ## L1概述与使用场景
### 能力描述 ### 能力描述
web-access 是一个**流程型技能Procedural Skill**,提供层互补的联网访问能力: web-access 是一个**流程型技能Procedural Skill**,提供层互补的联网访问能力:
- **L1**WebSearch + WebFetch公开页面轻量 - **L1**WebSearch + WebFetch公开页面轻量
- **L2**Jina ReaderJS 渲染的重页面,默认节省 Token - **L2**Jina ReaderJS 渲染的重页面,默认节省 Token
- **L3-fast**(内置受管浏览器,**v2.1 重写**):到达并*操作*登录态/交互站点——每个任务独立 BrowserSpace 隔离、零 Python 依赖、每次动作都有可审计回执。**抽取长正文仍归 L2 / L3-fallback**,见下方取文说明 - **L3**(内置受管浏览器,v3.0 能力面补全):到达、操作并**读取**登录态/交互站点——每个任务独立 BrowserSpace 隔离、零 Python 依赖、每次动作都有可审计回执。批量取文(`page.extract-text`)、判别式等待(`page.wait`)、代码模式(`BrowserScript`)都在本层内闭环
- **L3-fallback**Chrome CDP + Python Playwright复杂自动化场景兜底长等待、特殊 race condition 等)
### v2.1 重写:内置受管浏览器(默认隐藏,激活后才暴露) v2.x 时代的第四层「用户手工启动调试 Chrome + Python Playwright CDP」已在 v3.0 移除它依赖的每一条理由无批量取文通道、evaluate 不可用、截图必须串行 activate都被内置浏览器的新能力覆盖见下方速查。
调用 `Skill('web-access')` 加载本技能时,以下 8 个工具被注入到当前会话,让 LLM 直接驱动内置浏览器: ### v3.0:内置受管浏览器(默认隐藏,激活后才暴露)
调用 `Skill('web-access')` 加载本技能时,以下 9 个工具被注入到当前会话,让 LLM 直接驱动内置浏览器:
| 工具 | 用途 | | 工具 | 用途 |
|------|------| |------|------|
| BrowserManage | 建/销隔离 BrowserSpace、启动会话、管理标签页 | | BrowserManage | 建/销隔离 BrowserSpace、启动会话、管理标签页 |
| BrowserSnapshot | `semantic` / `accessibility` / `visual` 快照——读页面的主通道 | | BrowserSnapshot | `semantic` / `text` / `accessibility` / `visual` 四种快照——读页面的主通道 |
| BrowserAct | 一次调用一个受管动作:导航、点击、输入、滚动、截图…… | | BrowserAct | 一次调用一个受管动作:导航、输入、取文、等待、元素操作、截图…… |
| BrowserScript | **代码模式**:一段异步 JS 连续下发浏览器命令,消除逐动作往返(信任级别等同 Bash |
| BrowserImport | 从用户 Chrome/Edge/Firefox/Safari 配置导入 Cookie需人工审批**还需 Host 授予 `browser.import.*`,普通 `create_space` 会话拿不到** | | BrowserImport | 从用户 Chrome/Edge/Firefox/Safari 配置导入 Cookie需人工审批**还需 Host 授予 `browser.import.*`,普通 `create_space` 会话拿不到** |
| BrowserShare | 把 Space/Session 委派给其他 Agent隔离 / 快照 / 写时复制 / 实时共享) | | BrowserShare | 把 Space/Session 委派给其他 Agent隔离 / 快照 / 写时复制 / 实时共享) |
| SitePatternRead / SitePatternWrite | 按域名累积"站点经验"AgentFS 三层) | | SitePatternRead / SitePatternWrite | 按域名累积"站点经验"AgentFS 三层) |
@@ -33,20 +35,22 @@ web-access 是一个**流程型技能Procedural Skill**,提供四层互
> **重要**:未调用 Skill('web-access') 之前,这些工具**不会**出现在 LLM 的 tools 列表里——默认对话不消耗其 token。详见 [references/browser-tools.md](references/browser-tools.md)。 > **重要**:未调用 Skill('web-access') 之前,这些工具**不会**出现在 LLM 的 tools 列表里——默认对话不消耗其 token。详见 [references/browser-tools.md](references/browser-tools.md)。
> >
> **v2.1 已移除**`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用,调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本版要求客户端 v10.0.98+。 > **v2.1 已移除**`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用,调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本版要求客户端 v10.0.98+`page.extract-text` / `page.element` / `page.wait` / 内联 wait 块 / 跨源 iframe 快照需 v10.0.112+`BrowserScript` 需包含 S17/S18 的更新版本
### 使用场景 ### 使用场景
- 用户需要搜索当前信息或研究特定主题 - 用户需要搜索当前信息或研究特定主题
- 用户需要抓取公开网页内容或技术文档 - 用户需要抓取公开网页内容或技术文档
- 用户需要访问登录态站点小红书、B站、微博、飞书、Twitter 等) - 用户需要访问登录态站点小红书、B站、微博、飞书、Twitter 等)并**读出正文**
- 用户需要在登录态下调站内接口取数(列表、评论、订单等)
- 用户需要对比产品、聚合新闻或调查 API/库版本 - 用户需要对比产品、聚合新闻或调查 API/库版本
### 核心价值 ### 核心价值
- **层递进**:从轻量搜索到重度 JS 渲染到登录态访问,按需选择 - **层递进**:从轻量搜索到重度 JS 渲染到登录态访问,按需选择
- **Token 优化**Jina Reader 默认减少 50-80% Token 消耗 - **Token 优化**Jina Reader 默认减少 50-80% Token 消耗`page.extract-text` 的 maxBytes/cursor 分页让登录态长文也可控
- **登录态复用**兜底层 CDP 连用户 Chrome或在 Host 授予 `browser.import.*` 时用 BrowserImport 把 Cookie 导入隔离 Space;两条路都不必重新登录 - **登录态复用**Host 授予 `browser.import.*` 时用 BrowserImport 把 Cookie 导入隔离 Space不必重新登录
- **零外部依赖**:不再要求 Python/Playwright 安装,也不再要求用户手工启动调试 Chrome
## L2详细规范 ## L2详细规范
@@ -58,53 +62,6 @@ When you complete a research task, you **MUST** cite all source URLs in your res
If any fetch fails, explicitly tell the user which URL failed and which fallback you used. If any fetch fails, explicitly tell the user which URL failed and which fallback you used.
---
## Prerequisites: Chrome CDP Setup (for login-gated sites)
**L3-fallback 层Python Playwright必需**实际上也仍是复用登录态最稳的一条路。L3-fast 内置浏览器只有在 Host 已授予 `browser.import.*``BrowserImport` 真能用的前提下才可以跳过这一步;否则照样要配。
### One-time setup
Launch a dedicated Chrome instance with remote debugging enabled:
**macOS**:
```bash
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Linux**:
```bash
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Windows (PowerShell)**:
```powershell
& "C:\Program Files\Google\Chrome\Application\chrome.exe" `
--remote-debugging-port=9222 `
--user-data-dir="$env:USERPROFILE\.desirecore\chrome-profile"
```
After launch:
1. Manually log in to the sites you need (小红书、B站、微博、飞书 …)
2. Leave this Chrome window open in the background
3. Verify the debug endpoint: `curl -s http://localhost:9222/json/version` should return JSON
### Verify CDP is ready
Before any CDP operation, always run:
```bash
curl -s http://localhost:9222/json/version | python3 -c "import sys,json; d=json.load(sys.stdin); print('CDP ready:', d.get('Browser'))"
```
If the command fails, tell the user: "请先启动 Chrome 并开启远程调试端口(见 web-access 技能的 Prerequisites 部分)。"
---
## Tool Selection Decision Tree ## Tool Selection Decision Tree
``` ```
@@ -121,12 +78,12 @@ User intent
│ (Jina Reader = default for JS-rendered content, saves tokens) │ (Jina Reader = default for JS-rendered content, saves tokens)
├─ "Read this login-gated page" (小红书/B站/微博/飞书/Twitter/知乎/公众号) ├─ "Read this login-gated page" (小红书/B站/微博/飞书/Twitter/知乎/公众号)
─→ 到达:BrowserManage(create_space/start_session) → BrowserAct(tab.navigate) ─→ BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
tab.activate + page.screenshot 确认落到正文 │ → BrowserAct(page.extract-text) ← 正文直接读出,支持 maxBytes/cursor 分
semantic 快照不含正文,不能用来读内容) 要登录态:仅在已授予 browser.import.* 时用 BrowserImport
│ 要登录态:仅在已授予 browser.import.* 时用 BrowserImport否则走下面的 CDP
│ └─→ 取正文:确认 CDP 就绪后 python3 playwright.connect_over_cdp() ├─ "Pull data from the site's API in a logged-in context"
→ page.content() → Jina Reader / BeautifulSoup └─→ fetch.browser 配方BrowserAct(page.evaluate) 里跑 fetch带该 origin 的 Cookie
├─ "API documentation / GitHub / npm package info" ├─ "API documentation / GitHub / npm package info"
│ └─→ Prefer official API endpoints over scraping HTML: │ └─→ Prefer official API endpoints over scraping HTML:
@@ -135,24 +92,19 @@ User intent
│ - PyPI: curl https://pypi.org/pypi/<pkg>/json │ - PyPI: curl https://pypi.org/pypi/<pkg>/json
└─ "Real-time interactive task" (click, fill form, scroll, screenshot) └─ "Real-time interactive task" (click, fill form, scroll, screenshot)
─→ **Default: 内置受管浏览器** (BrowserManage → BrowserAct → BrowserSnapshot — ─→ 内置受管浏览器 (BrowserManage → BrowserAct → BrowserSnapshot —
see references/browser-tools.md, no Python needed) see references/browser-tools.md, no Python needed)
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when 内置浏览器 is insufficient
(e.g., complex race conditions, multi-event waits, long-running in-browser scripts)
``` ```
### 层策略总结 ### 层策略总结
| Layer | Use case | Primary tool | Token cost | | Layer | Use case | Primary tool | Token cost |
|-------|----------|--------------|------------| |-------|----------|--------------|------------|
| L1 | Public, static | `WebFetch` | Low | | L1 | Public, static | `WebFetch` | Low |
| L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) | | L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) |
| **L3-fast** | **登录态导航交互 (PRIMARY)** | **内置受管浏览器BrowserManage / BrowserAct / BrowserSnapshot** | Medium | | **L3** | **登录态导航交互与取文 (PRIMARY)** | **内置受管浏览器BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript** | Medium |
| L3-fallback | 复杂自动化race / long-wait / 自定义脚本) | `Bash + Python Playwright CDP` | Medium |
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when 内置浏览器不够用 **Default priority**: L1 for simple public pages → L2 for heavy → **L3 for login-gated(含正文与站内接口取数)**
---
## Supported Sites Matrix ## Supported Sites Matrix
@@ -162,16 +114,14 @@ User intent
| GitHub README, issues, PRs | `gh api` (best) → L1 WebFetch | Prefer API | | GitHub README, issues, PRs | `gh api` (best) → L1 WebFetch | Prefer API |
| Hacker News, Reddit | L1 WebFetch | Public content | | Hacker News, Reddit | L1 WebFetch | Public content |
| Medium, Dev.to | L2 Jina Reader | JS-rendered, member gates | | Medium, Dev.to | L2 Jina Reader | JS-rendered, member gates |
| Twitter/X | L3 CDP (or L2 Jina with `x.com`) | Login required for full thread | | Twitter/X | L3(或 L2 Jina with `x.com` | Login required for full thread |
| 小红书 (xiaohongshu.com) | L3 CDP | 强制登录 | | 小红书 (xiaohongshu.com) | L3 内置浏览器 + BrowserImport | 强制登录;正文走 page.extract-text |
| B站 (bilibili.com) | L3 CDP | 视频描述/评论需登录 | | B站 (bilibili.com) | L3 内置浏览器 + BrowserImport | 视频描述/评论需登录 |
| 微博 (weibo.com) | L3 CDP | 长微博需登录 | | 微博 (weibo.com) | L3 内置浏览器 + BrowserImport | 长微博需登录 |
| 知乎 (zhihu.com) | L3 CDP | 长文+评论需登录 | | 知乎 (zhihu.com) | L3 内置浏览器 + BrowserImport | 长文+评论需登录 |
| 飞书文档 (feishu.cn) | L3 CDP | 必须登录 | | 飞书文档 (feishu.cn) | L3 内置浏览器 + BrowserImport | 必须登录 |
| 公众号 (mp.weixin.qq.com) | L2 Jina Reader | 通常公开Jina 处理更干净 | | 公众号 (mp.weixin.qq.com) | L2 Jina Reader | 通常公开Jina 处理更干净 |
| LinkedIn | L3 CDP | 登录墙 | | LinkedIn | L3 内置浏览器 + BrowserImport | 登录墙 |
---
## Tool Reference ## Tool Reference
@@ -194,11 +144,11 @@ WebFetch(url="https://example.com/article")
Tips: Tips:
- Results cached for 15 min - Results cached for 15 min
- Returns cleaned Markdown with title + URL + body - Returns cleaned Markdown with title + URL + body
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (CDP) - If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (built-in browser)
### Layer 2: Jina Reader (default for heavy pages) ### Layer 2: Jina Reader(重页默认)
Jina Reader (`r.jina.ai`) is a free public proxy that renders pages server-side and returns clean Markdown. Use it as the **default** for any page where WebFetch produces garbled or truncated output, and as the **preferred** extractor for JS-heavy SPAs. Jina Reader (`r.jina.ai`) 免费公共代理,服务端渲染页面并返回干净 Markdown。WebFetch 输出乱码/截断时的默认升级路径JS 重页面的首选抓取器。
```bash ```bash
curl -sL "https://r.jina.ai/https://example.com/article" curl -sL "https://r.jina.ai/https://example.com/article"
@@ -212,111 +162,93 @@ Why Jina is the default token-saver:
See [references/jina-reader.md](references/jina-reader.md) for advanced endpoints and rate limits. See [references/jina-reader.md](references/jina-reader.md) for advanced endpoints and rate limits.
### Layer 3: CDP Browser (login-gated access) ### Layer 3: 内置受管浏览器(登录态与交互)
Use Python Playwright's `connect_over_cdp()` to attach to the user's running Chrome (which already has login cookies). **No re-login needed.** 完整命令面、能力档位与边界条件见 [references/browser-tools.md](references/browser-tools.md)。这里的循环是:
**Minimal template**: 1. `BrowserManage(create_space)``BrowserManage(start_session)` 拿 sessionId + 首个 tab
```bash 2. 需要登录态 → `BrowserImport`(仅当 Host 已授予 `browser.import.*`;没有这层授权就无法复用用户 Cookie告诉用户并按无登录态继续或放弃
python3 << 'PY' 3. `BrowserAct(tab.navigate)` 到达页面
from playwright.sync_api import sync_playwright 4. `BrowserSnapshot(semantic)` 拿可交互元素 `ref`(跨源 iframe 的元素也在同一棵树里ref 全局连续编号)
5. 交互用 `input.*`(拟真轨迹)或 `page.element`(表单批量写);等结果用 `page.wait` 或内联 wait 块
6. 取正文用 `BrowserSnapshot(text)``BrowserAct(page.extract-text)`;取接口数据用 fetch.browser 配方
7. 任务收尾 `BrowserManage(close_session)`
TARGET_URL = "https://www.xiaohongshu.com/explore/..." 多动作连续编排(导航→快照→点击→等待→取文)可用 `BrowserScript` 一段脚本完成,省去逐动作 IPC 往返。
with sync_playwright() as p: ## L3 速查v3.0
browser = p.chromium.connect_over_cdp("http://localhost:9222")
context = browser.contexts[0] # reuse user's default context (has cookies)
page = context.new_page()
page.goto(TARGET_URL, wait_until="domcontentloaded")
page.wait_for_timeout(2000) # let lazy content load
html = page.content()
page.close()
# Print first 500 chars to verify ### 读页面:按需求选通道
print(html[:500])
PY
```
**Extract text via BeautifulSoup** (no Jina round-trip):
```bash
python3 << 'PY'
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto("https://www.bilibili.com/video/BV...", wait_until="networkidle")
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
title = soup.select_one("h1.video-title")
desc = soup.select_one(".video-desc")
print("Title:", title.get_text(strip=True) if title else "N/A")
print("Desc:", desc.get_text(strip=True) if desc else "N/A")
PY
```
See [references/cdp-browser.md](references/cdp-browser.md) for:
- Per-site selectors (小红书/B站/微博/知乎/飞书)
- Scrolling & lazy-load patterns
- Screenshot & form-fill recipes
- Troubleshooting connection issues
---
## L3-fast: 内置受管浏览器速查v2.1
**只在你调用 `Skill('web-access')` 加载本技能后,下面这组工具才会出现在 tools[] 里。**
| 工具 | 一行示例 |
|------|---------|
| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | 每个任务一个隔离 Space |
| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | 启动会话,返回 sessionId 与首个标签页 |
| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | 标签页与生命周期管理 |
| `BrowserAct({ action: 'tab.navigate', params: { url } })` | 当前标签页导航 |
| `BrowserSnapshot({ mode: 'semantic' })` | 读页面:可交互元素 + `ref` 句柄 |
| `BrowserAct({ action: 'input.click', params: { ref } })` | 按快照 `ref` 点击,不要用裸 x/y |
| `BrowserAct({ action: 'input.text', params: { text } })` | 向聚焦元素输入文本 |
| `BrowserAct({ action: 'input.wheel', params: { x: 640, y: 400, deltaX: 0, deltaY: 720 } })` | 滚动触发懒加载 —— `x`/`y`(或 `ref`**必填**,且会话要带 `browser.input.pointer.wheel` |
| `BrowserAct({ action: 'tab.activate', params: { bounds } })``page.screenshot` | **先 activate 再截图** |
| `BrowserImport({ action: 'discover' \| 'create_plan' \| 'dry_run' \| 'apply' \| 'rollback' \| 'list_plans' })` | 复用用户登录态 Cookie —— 这 6 个就是**全部**动作,主流程走 `discover → create_plan → dry_run → apply`。**需要 `browser.import.*`,而 `create_space` 不会授予**(见下) |
完整 API 与边界条件见 [references/browser-tools.md](references/browser-tools.md)。
**怎么读页面** —— 按需求选通道:
| 你要什么 | 用什么 | 说明 | | 你要什么 | 用什么 | 说明 |
|----------|--------|------| |----------|--------|------|
| 可交互元素 + `ref` 句柄 | `BrowserSnapshot({ mode: 'semantic' })` | 只有按钮/输入框/链接**不含正文文本** | | 可交互元素 + `ref` / `loc=` 句柄 | `BrowserSnapshot({ mode: 'semantic' })` | 按钮/输入框/链接 + 每行 `[ref=eN]` 与(能产出时)`[loc=...]` 稳定选择器;跨源 iframe 元素同树 |
| 页面正文 | `BrowserSnapshot({ mode: 'accessibility' })` | 返回 StaticText 节点;真实内容页会报 `BROWSER_RESULT_TOO_LARGE`(结果上限 2 MB`depth` 参数当前被宿主忽略) | | 页面正文 | `BrowserSnapshot({ mode: 'text' })``BrowserAct({ action: 'page.extract-text' })` | markdown/text 两种格式;超 maxBytes 截断并给 nextCursor 续读cursor 分页),不报错 |
| 真实页面的正文 | L2 Jina Reader公开页或 L3-fallback Playwright登录态 | 内置浏览器目前没有可用的批量取文通道 | | 辅助功能树 | `BrowserSnapshot({ mode: 'accessibility' })` | 尊重 `depth`(默认 50 上限 100与 maxBytes 预算,超出截断+翻页,不再整体报错 |
| 页面长什么样 | `tab.activate` `BrowserAct({ action: 'page.screenshot' })` | 只有整页截图,靠看图读 | | 页面长什么样 | `BrowserSnapshot({ mode: 'visual' })` `BrowserAct({ action: 'page.screenshot' })` | 像素直接进结果(视觉模型当场看);支持 `clip={x,y,width,height,scale}` 元素级裁剪放大scale 最大 4`captureBeyondViewport` 整页截图 |
`page.evaluate` **不是**取文通道:每次调用需人工审批,且字符串/对象返回值会被替换成 `[REDACTED:browser-runtime-value]` ### 命令面速览
`BrowserAct``action` 按用途分组(完整枚举见工具 schema
- **tab.***`navigate` / `back` / `forward` / `reload` / `activate` / `close`
- **input.***`move` / `click` / `double-click` / `drag` / `wheel` / `touch` / `pinch` / `key` / `text` —— 走输入拟真(#1808UA/UA-CH 身份一致 + 真实轨迹),反检测站点的交互首选
- **page.element**(判别式 op × selector九 op写类 `fill` / `select-option` / `check` / `uncheck` / `scroll-into-view`;读类 `get-attribute` / `bounding-box` / `count` / `all-inner-texts`。selector 说 `loc=` 方言或快照 `ref`(配 snapshotId`fill` 拒绝 `input[type=password]`
- **page.wait**(判别式 until九种轮询型 `load` / `domcontentloaded` / `networkidle` / `selector` / `url` / `timeout` 超时返回 `waited:false`;事件型 `request` / `response` / `download` 超时抛错。默认 10s、上限 60s
- **内联 wait 块**`tab.navigate` / `input.click` / `input.key` / `page.element{op:"fill"}``params.wait`(形态与 page.wait 同构)——一条回执完成「动作→等结果」,等待器先于动作注册,无跨 IPC 竞态
- **page.evaluate**`{ expression, awaitPromise }`,返回值原样过界(超预算截断并标 `truncated`,不抛错);能力档 `browser.page.evaluate` 走人工闸门allow-all 模式免卡片)
- **page.extract-text / page.screenshot / page.wait**:见上表与 fetch.browser 配方
`loc=` 选择器方言S7/S9`e<序号>`(须配签发该 ref 的快照 snapshotId`loc=css:` / `loc=role:` / `loc=text:` / `loc=testid:`、裸 CSS可叠 `internal:nth/last/scope/filter`。未知前缀显式报错,绝不静默降级。
### 交互通道选用input.* vs page.element
- **反检测站点(小红书/微博/B站等一律优先 `input.*`**:走 #1808 的输入行为拟真(坐标派发、拟真轨迹、可视化可审计),配套身份一致性层让 UA/UA-CH 不带 Electron/Headless 痕迹
- **`page.element` 写类适用于表单批量填充等站点不检测的场景**:一次调用完成 fill/select-option/check比逐元素 input.click+input.text 快得多
- **红线**`page.element` 刻意不含 click——指针动作必须走 `input.*`,用 JS 直调 `el.click()` 会绕开全部拟真投入,属于明确禁止的回退
### fetch.browser 配方:带登录态取接口数据
登录态下取站内接口(列表、评论、订单等 JSON的正解在**页面上下文**里跑 `fetch`——自动带该 origin 的 Cookie同 origin、受 Grant origins 约束,走 `page.evaluate` 已有闸门。这是 `BrowserAct({ action: 'page.evaluate' })` 的封装用法:
```yaml
BrowserAct:
action: page.evaluate
params:
expression: |
fetch('/api/v1/comments?page=1&size=20', {
headers: { accept: 'application/json' }
}).then(r => r.text())
awaitPromise: true # 默认 true表达式返回 Promise 时等它 settle
```
要点:
-`tab.navigate` 到该站任意页面(建立 origin 与 Cookie再发 fetch路径写相对路径天然同 origin
- 返回值原样过界;大 JSON 用 `.text()` 拿原文自己截取,或分页多次取
- 只能访问当前 tab originGrant origins 约束);跨站接口请先导航过去
- `page.evaluate` 属人工闸门能力:非 allow-all 模式会弹审批卡,向用户说明用途即可
### 推荐流程(小红书示例) ### 推荐流程(小红书示例)
``` ```
1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' }) 1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] }) 2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
← 显式列表是做减法;要滚动就必须把 browser.input.pointer.wheel 列进去 ← 显式列表是做减法;要滚动就把 browser.input.pointer.wheel 列进去
3. 复用登录态:默认走 L3-fallback Playwright 连用户自己的 Chrome。 要取文就带 browser.observe.snapshot
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }) 只有在 Host 另行 3. 复用登录态:仅在 Host 已授予 browser.import.* 时走
授予 browser.import.* 时才走得通 —— 光靠 create_space 拿不到;被拒就直接回落 BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' })
没有这层授权就无法复用用户 Cookie——如实告诉用户按无登录态继续或放弃。
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } }) 4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
5. BrowserSnapshot({ mode: 'semantic' }) ← 拿交互用的元素 ref不含正文 5. BrowserSnapshot({ mode: 'semantic' }) ← 拿交互元素 ref跨源 iframe 同树
tab.activate + page.screenshot ← 确认确实渲染出了笔记 6. BrowserAct({ action: 'page.extract-text', params: { format: 'markdown' } })
正文抽取走 L3-fallback Playwright 正文直接读出;太长就传上一页返回的 nextCursor 续读
6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← 读累积经验 7. 需要确认渲染效果 → BrowserSnapshot({ mode: 'visual' })(像素直接可看)
7. 任务结束 → BrowserManage({ action: 'close_session', sessionId }) 8. SitePatternRead({ domain: 'xiaohongshu.com' }) ← 读累积经验
8. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) 9. 任务结束 → BrowserManage({ action: 'close_session', sessionId })
10. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
``` ```
--- ## 站点经验积累
## 站点经验积累v2.0 新增)
任务结束如果发现新的反爬陷阱、有效选择器、平台特征,调用: 任务结束如果发现新的反爬陷阱、有效选择器、平台特征,调用:
@@ -325,7 +257,7 @@ SitePatternWrite({
domain: "xiaohongshu.com", domain: "xiaohongshu.com",
scope: "agent", // agent=共享(受 Git 管理发布给其他用户user=私有 scope: "agent", // agent=共享(受 Git 管理发布给其他用户user=私有
mode: "merge", // merge 追加replace 覆盖 mode: "merge", // merge 追加replace 覆盖
content: "## 已知陷阱\n- 2026-05: ...", content: "## 已知陷阱\n- 2026-08: ...",
confidence: "medium" confidence: "medium"
}) })
``` ```
@@ -341,8 +273,6 @@ SitePatternRead({ domain: "xiaohongshu.com" })
含 cookie / token / 手机号 / 邮箱时 SitePatternWrite **自动降级 scope='user'** 并提示。 含 cookie / token / 手机号 / 邮箱时 SitePatternWrite **自动降级 scope='user'** 并提示。
---
## Common Workflows ## Common Workflows
Read [references/workflows.md](references/workflows.md) for detailed templates: Read [references/workflows.md](references/workflows.md) for detailed templates:
@@ -351,11 +281,9 @@ Read [references/workflows.md](references/workflows.md) for detailed templates:
- 新闻聚合与时间线 (News aggregation) - 新闻聚合与时间线 (News aggregation)
- API/库版本调查 (Library version investigation) - API/库版本调查 (Library version investigation)
Read [references/cdp-browser.md](references/cdp-browser.md) for login-gated site recipes (小红书/B站/微博/知乎/飞书).
Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader positioning, rate limits, and advanced endpoints. Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader positioning, rate limits, and advanced endpoints.
--- Read [references/browser-tools.md](references/browser-tools.md) for the full built-in browser command surface, capability tiers, and known boundaries.
## Quick Workflow: Multi-Source Research ## Quick Workflow: Multi-Source Research
@@ -369,27 +297,22 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
7. Report with inline [source](url) citations + a Sources list at the end 7. Report with inline [source](url) citations + a Sources list at the end
``` ```
---
## Anti-Patterns (Avoid) ## Anti-Patterns (Avoid)
-**Using WebFetch on obviously heavy sites** — Medium, Twitter, 小红书 will waste tokens or fail. Jump straight to L2/L3. -**Using WebFetch on obviously heavy sites** — Medium, Twitter, 小红书 will waste tokens or fail. Jump straight to L2/L3.
-**Launching headless Chrome instead of CDP attach** — loses user's login state, triggers anti-bot, slow cold start. Always use `connect_over_cdp()` to attach to the user's existing session.
-**Fetching one URL at a time when you need 5** — batch in a single message. -**Fetching one URL at a time when you need 5** — batch in a single message.
-**Trusting a single source** — cross-check ≥ 2 sources for non-trivial claims. -**Trusting a single source** — cross-check ≥ 2 sources for non-trivial claims.
-**Fetching the search result page itself** — WebSearch already returns snippets; fetch the actual articles. -**Fetching the search result page itself** — WebSearch already returns snippets; fetch the actual articles.
-**Ignoring the cache** — WebFetch caches 15 min, reuse freely. -**Ignoring the cache** — WebFetch caches 15 min, reuse freely.
-**Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs. -**Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs; 登录态站内接口走 fetch.browser 配方.
-**Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current. -**Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current.
-**Hardcoding login credentials in scripts**always rely on the user's pre-logged CDP session. -**Hardcoding login credentials in scripts**登录态只能来自 BrowserImport 导入的 Cookie.
-**Citing only after the fact** — collect URLs as you fetch, not from memory afterwards. -**Citing only after the fact** — collect URLs as you fetch, not from memory afterwards.
-**(v2.1) 在能用内置浏览器时仍写 Python heredoc** — 慢、依赖 Python+Playwright 安装、上下文体积大。优先 L3-fast只在内置浏览器不够race / 长等待 / 自定义脚本)时才回退 -**(v3.0) 在反检测站点用 page.element 批量交互** — 它是 JS 直调,绕开拟真轨迹;反检测站点一律 `input.*``page.element` 只用于不检测场景的表单批量填充
-**(v2.1) 没 `tab.activate` 就直接 `page.screenshot`** — 标签页默认停在窗口外 `(-10000,-10000,1x1)`,没有合成表面,截图会卡满 30 秒 deadline **并把标签页宿主打掉**,之后所有调用都报 `BROWSER_TAB_HOST_NOT_FOUND`。务必先用真实 bounds `tab.activate` -**(v3.0) 用截图靠"看图读字"取正文** — `page.extract-text` / `BrowserSnapshot(text)` 直接给 markdown/text带分页预算截图留给版面确认与验证码这类必须看图的场合
-**(v2.1) 用 `page.evaluate` 读页面** — 每次调用都需人工审批,返回的字符串/对象会被治理策略替换为 `[REDACTED:browser-runtime-value]`(只有 number/boolean/null 能穿透),且反调试站点会把它挂起几十秒。读页面请用 `BrowserSnapshot` -**(v3.0) 逐动作往返还能忍时不换 BrowserScript** — 导航→快照→点击→等待→取文五连用一段脚本完成;但记住 BrowserScript 信任级别等同 Bash脚本源码要过一次人工审批
-**(v2.1) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')` -**(v3.0) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')`
-**(v2.1) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级,但你不该故意往 agent 层写敏感信息。 -**(v3.0) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级,但你不该故意往 agent 层写敏感信息。
---
## Example Interaction ## Example Interaction
@@ -398,26 +321,15 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
**Agent workflow**: **Agent workflow**:
``` ```
1. 识别 → 小红书是 L3 登录态站点 1. 识别 → 小红书是 L3 登录态站点
2. 检查 CDPcurl -s http://localhost:9222/json/version 2. BrowserManage(create_space + start_session)
├─ 失败 → 提示用户启动 Chrome 调试模式,终止 需要登录态 → 已授予 browser.import.* 就走 BrowserImport 四步;
└─ 成功 → 继续 没授予 → 告诉用户无法复用登录态,按公开可见部分继续
3. Bash: python3 connect_over_cdp 脚本 → page.goto(url) → page.content() 3. BrowserAct(tab.navigate → 笔记 URL)
4. BeautifulSoup 提取 h1 title、.note-content、.comments 4. BrowserAct(page.extract-text, format: markdown)
← 正文直接读出;超预算就按 nextCursor 续读
5. 返回给用户时: 5. 返回给用户时:
- 引用原 URL - 引用原 URL
- 若内容很长,用 Jina 清洗一遍节省 token - 引用正文事实,标注来源链接
6. 告知用户:「已通过你的登录态抓取,原链接:[xhs](url)」 6. 告知用户:「已通过内置浏览器抓取,原链接:[xhs](url)」
7. BrowserManage(close_session)
``` ```
---
## Installation Note
CDP features require Python + Playwright installed:
```bash
pip3 install playwright beautifulsoup4
python3 -m playwright install chromium # only needed if user hasn't installed Chrome
```
If `playwright` is not installed when the user requests a login-gated site, run the install commands in Bash and explain you're setting up the browser automation dependency.

View File

@@ -1,26 +1,15 @@
# 内置受管浏览器工具速查L3-fast # 内置受管浏览器工具速查L3
> v2.1 起本层从「cdp-proxy 驱动用户自己的 Chrome」改为「DesireCore 内置受管浏览器」。 > v2.1 起本层从「cdp-proxy 驱动用户自己的 Chrome」改为「DesireCore 内置受管浏览器」v3.0 补全能力面(批量取文 / 元素操作 / 等待 / 代码模式)并移除 Python Playwright 回落
> 每个任务跑在独立 BrowserSpace 里Cookie / Storage / 缓存互不串扰),每个动作都经过 > 每个任务跑在独立 BrowserSpace 里Cookie / Storage / 缓存互不串扰),每个动作都经过
> Capability → Grant → Lease → Origin → Host fencing 校验并留下可审计回执。 > Capability → Grant → Lease → Origin → Host fencing 校验并留下可审计回执。
> >
> 要求客户端 **v10.0.98+**。旧的 `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / > 要求客户端 **v10.0.98+**。`page.extract-text` / `page.element` / `page.wait` / 内联 wait 块 /
> `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / > 跨源 iframe 快照需 **v10.0.112+**`BrowserScript` 需包含 S17/S18 的版本。旧的
> `BrowserCloseTab` 与其背后的 cdp-proxy 已停用,调用会直接返回 > `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` /
> `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 与其背后的 cdp-proxy 已停用,调用会直接返回
> 「该旧 BrowserXxx/cdp-proxy 入口已停用;请改用 BrowserManage、BrowserSnapshot 或 BrowserAct」。 > 「该旧 BrowserXxx/cdp-proxy 入口已停用;请改用 BrowserManage、BrowserSnapshot 或 BrowserAct」。
## 何时用内置浏览器 vs Python Playwright
| 场景 | 推荐 |
|------|------|
| 到达并操作登录态站点(小红书 / B站 / 微博 / 飞书 / 知乎) | **内置浏览器**(登录态优先走 L3-fallback CDP仅在已授予 `browser.import.*` 时才用 BrowserImport 导入 Cookie |
| 抽取登录态站点的长正文 | Python Playwright内置浏览器没有批量取文通道见下 |
| 简单点击 / 滚动 / 截图 | **内置浏览器** |
| 多个任务要互不串扰地并行 | **内置浏览器**(一任务一 Space |
| 需要复杂等待逻辑wait_for_selector + race condition | Python Playwrightcdp-browser.md |
| 需要在浏览器内运行长时间脚本(>30 s | Python Playwright内置浏览器单条命令 30 s deadline |
| 需要对元素做放大裁剪截图 | Python Playwright内置浏览器只有整页截图 |
## 前置条件 ## 前置条件
1. 客户端 v10.0.98+`~/.desirecore/config/browser.json``electron-embedded` 1. 客户端 v10.0.98+`~/.desirecore/config/browser.json``electron-embedded`
@@ -68,27 +57,52 @@ BrowserManage:
### BrowserSnapshot ### BrowserSnapshot
读页面的**主通道**。返回可交互元素及其 `ref` 句柄、`rect``disabled` 状态。 读页面的**主通道**。四种 mode
```yaml ```yaml
BrowserSnapshot: BrowserSnapshot:
mode: semantic # semantic默认| accessibility | visual mode: semantic # semantic默认| text | accessibility | visual
sessionId: bss_xxx # 当前 Agent 有多个会话时用于消歧 sessionId: bss_xxx # 当前 Agent 有多个会话时用于消歧
tabId: btab_xxx tabId: btab_xxx # 非活动标签页时用于消歧
options: {...} # 各 mode 专属参数,见下
``` ```
提示: **semantic** —— 可交互元素清单button / input / a 等)+ `ref` 句柄。每行形如
- `semantic` 只列 button / input / a 等可交互元素,**不含图片和正文文本节点** `[ref=e12] [loc=...] 按钮 提交`
- 元素 `name` 往往是 placeholder如「请输入」无标签时只能靠 `rect.y` 排序定位 - `loc=` 是该元素的稳定选择器S7仅当能产出时才给出五级优先级 data-testid →
- 页面重排后旧 `ref` 会失效——**每次交互前重新取快照** 唯一 CSS 安全 id → name 属性 → 链接 href剥 query 前缀匹配)→ role+name
- `accessibility` 能拿到 StaticText 正文,但**只在很简单的页面上可用**:实测 - 跨源 iframeOOPIF的元素并入同一棵树ref 全局连续编号S35
example.com 正常返回,维基百科条目一律 `BROWSER_RESULT_TOO_LARGE`(回执上限 2 MB - 页面重排后旧 `ref` 会失效——**每次交互前重新取快照**;带 ref 的命令要同时给签发该
且 schema 里的 `depth` 参数目前被宿主忽略(硬编码 depth=50调小也没用 ref 的快照 `snapshotId`
- 因此**取真实页面正文仍要靠 L2 Jina Reader公开页或 L3-fallback Playwright登录态** - `options``scope`viewport=视口内元素优先截断)、`maxElements`(默认 400 上限 2000
`maxBytes`(默认 256KB 上限 1MB`cursor` 翻页、`limit`
**text** —— 正文批量取文通道之一S3/S4
```yaml
BrowserSnapshot:
mode: text
options:
format: markdown # markdown默认保留标题/列表/表格)| text纯文本
maxBytes: 65536 # 默认 65536最大 524288超出不报错截断并返回 nextCursor
cursor: t1:...:... # 上一页返回的 nextCursor同参数续读
scope: full_page # full_page默认| viewport | ref配 ref + snapshotId 取子树)
includeLinks: true # 链接目标写进正文(只写 http/https
includeTables: true # 表格保留逐行结构
```
**accessibility** —— AX 树S8 起**尊重 `depth`、预算截断不炸**
`options.depth`(默认 50 上限 100`maxElements``maxBytes``cursor`。真实内容页不再
`BROWSER_RESULT_TOO_LARGE` 整体失败——超预算截断给 nextCursor 续读。
**visual** —— 截图(像素直接进结果):
`options.format`png/jpeg`quality``captureBeyondViewport`(整页)、
`clip={x,y,width,height,scale}`(元素级裁剪放大,`scale` 最大 4
### BrowserAct ### BrowserAct
一次调用一个受管动作。完整 `action` 见工具 schema常用的 一次调用一个受管动作。`action` 分组
**tab.*** —— `navigate` / `back` / `forward` / `reload` / `activate` / `close`
```yaml ```yaml
BrowserAct: BrowserAct:
@@ -96,10 +110,15 @@ BrowserAct:
params: { url: https://www.xiaohongshu.com/explore/... } params: { url: https://www.xiaohongshu.com/explore/... }
``` ```
**input.*** —— `move` / `click` / `double-click` / `drag` / `wheel` / `touch` / `pinch` /
`key` / `text`。走 #1808 输入拟真(坐标派发、拟真轨迹、身份一致性),**反检测站点的交互首选**
```yaml ```yaml
BrowserAct: BrowserAct:
action: input.click action: input.click
params: { ref: bref_xxx } # 用快照 ref坐标会因页面重排失效 params:
ref: e12 # 语义快照签发的短号 ref不要裸 x/y
snapshotId: snap-xxx # 签发该 ref 的快照回执里的 snapshotId必带
``` ```
```yaml ```yaml
@@ -114,37 +133,120 @@ BrowserAct:
params: { deltaX: 0, deltaY: 720, x: 640, y: 400 } params: { deltaX: 0, deltaY: 720, x: 640, y: 400 }
``` ```
```yaml 带元素 ref 的 `input.*` 必须在 params 顶层同时带同一条快照回执的 `snapshotId`——ref 序号跨
BrowserAct: 快照会重复,缺 snapshotId 一律拒绝执行。
action: tab.activate # 截图前必须先做这一步
params: { bounds: { x: 0, y: 0, width: 1280, height: 900 } } **page.element** —— 判别式元素命令op × selector九 opS10。**表单批量填充等站点
``` 不检测场景用**;反检测站点交互走 input.*
```yaml ```yaml
BrowserAct: BrowserAct:
action: page.screenshot action: page.element
params: { format: png } # 结果落 artifact store回执给 artifact.id / sha256 / bytes params:
op: fill # 写类fill | select-option | check | uncheck | scroll-into-view
# 读类get-attribute | bounding-box | count | all-inner-texts
selector: loc=css:input#email # loc= 方言或 e12 / ref=e12配 snapshotId
value: user@example.com # fill / select-option 用
snapshotId: bsnp_xxx # selector 用 ref 时必带
``` ```
- 写类走 `browser.input.keyboard` 能力档(与 input.text 同档);读类走 `browser.observe.snapshot` 只读档
- `fill``input[type=password]` **一律拒绝**
- `selector` 方言:`e<序号>` / `ref=e<序号>``loc=css:` / `loc=role:` / `loc=text:` /
`loc=testid:`、裸 CSS可叠 `internal:nth/last/scope/filter`。未知前缀(如 `loc=xpath:`)显式报错,绝不静默降级
**page.wait** —— 判别式等待until九种S11。等待不改页面状态走只读档
```yaml
BrowserAct:
action: page.wait
params:
until: networkidle # load | domcontentloaded | networkidle | selector | url | timeout
# | request | response | download
timeoutMs: 10000 # 默认 10000上限 60000
idleMs: 500 # networkidle 静默窗,默认 500
```
- 轮询型load/domcontentloaded/networkidle/selector/url/timeout超时返回 `waited: false`,不抛错
- 事件型request/response/download超时**抛错**
- `waitForFunction` 刻意不在枚举里——任意 JS 走 page.evaluate 的能力档与审批
**内联 wait 块**S14—— `tab.navigate` / `input.click` / `input.key` / `page.element{op:"fill"}`
支持 `params.wait`(形态与 page.wait 参数同构),一条回执完成「动作→等结果」,等待器先于动作注册,
不跨两次 IPC 出竞态:
```yaml
BrowserAct:
action: tab.navigate
params:
url: https://example.com/login
wait: { until: networkidle, timeoutMs: 15000 }
```
**page.evaluate** —— 页面上下文求值S2 起返回真实值):
```yaml
BrowserAct:
action: page.evaluate
params:
expression: document.title
awaitPromise: true # 默认 true
```
- 返回值**原样过界**(字符串/对象/数组都真实返回);超预算截断并标 `truncated`,不抛错
- 能力档 `browser.page.evaluate` 属人工闸门:非 allow-all 模式每次调用弹审批卡
- 登录态取站内接口走 [fetch.browser 配方](见下节)
**page.extract-text** —— 与 `BrowserSnapshot mode:text` 同款取文通道,作为动作下发:
scopeviewport/full_page/ref+ ref + snapshotId、formatmarkdown/text、maxBytes默认
65536 最大 524288、cursor`^t1:` 续读、includeLinks、includeTables。超出预算截断 + nextCursor。
**page.screenshot** —— 与 `BrowserSnapshot mode:visual` 同款:`format``quality`
`captureBeyondViewport``clip={x,y,width,height,scale}`。结果落 artifact store回执给
artifact.id / sha256 / bytes截图像素直接进结果见「截图」节
### BrowserScript代码模式S17+S18
一段异步 JS 在 Worker 里跑,通过注入的 `page` / `tab` / `input` / `snapshot` / `console` /
`performance` 门面连续下发命令——**导航→快照→点击→等待→取文一气呵成**,消除逐动作往返:
```yaml
BrowserScript:
code: |
const snap = await snapshot.semantic();
receipt.log(snap.content);
await input.click({ ref: 'e12', snapshotId: snap.snapshotId });
await page.wait({ until: 'networkidle' });
receipt.log(await page['extract-text']({ format: 'markdown' }));
sessionId: bss_xxx # 可选,多会话消歧
tabId: btab_xxx # 可选,缺省用会话活动 tab
totalBudgetMs: 180000 # 默认 180000上限 600000审批等待不计入
maxCommands: 500 # 每次运行命令条数上限
```
- ❗**信任级别等同 Bash**Worker 不是沙箱,脚本可访问 Node fs/net/child_process。两层审批
脚本源码先过一次与 Bash 同档的人类审批allow-all / ask-external 豁免;脚本源码不可记忆),
其内每个受管浏览器能力再按「能力 × 本次运行」各问一次
- helper 白名单 24 个:`snapshot.*` / `page.extract-text` / `tab.*` / `input.*` /
`page.evaluate` / `page.wait` / `page.element` / `console.read` / `performance.metrics`
`console.log(page)` 可输出各 helper 的 signature/params/example 文档
- 会话被用户接管、Lease 轮换 ⇒ **硬停**:后续命令一律不再下发
- 需要 `browser.*` 相应能力;`receipt.log(...)``console.log` 输出文本
### BrowserImport**需人工审批 + 额外授权** ### BrowserImport**需人工审批 + 额外授权**
把用户浏览器里的登录态 Cookie 导入当前 Space。 把用户浏览器里的登录态 Cookie 导入当前 Space——**唯一的登录态复用通道**
> **前置条件(先看这里,别直接试)**`browser.import.discover` / > **前置条件(先看这里,别直接试)**`browser.import.discover` /
> `browser.import.cookies.inspect` / `browser.import.cookies` 三个能力**不在 `agentDefault` 里** > `browser.import.cookies.inspect` / `browser.import.cookies` 三个能力**不在 `agentDefault` 里**
> 而 `BrowserManage(create_space)` 建的 Agent grant 就是按 `agentDefault` 签的。也就是说 > 而 `BrowserManage(create_space)` 建的 Agent grant 就是按 `agentDefault` 签的。也就是说
> **仅靠 create_space / start_session 走不通 BrowserImport**,必须由 Host/用户侧另行授予 import > **仅靠 create_space / start_session 走不通 BrowserImport**,必须由 Host/用户侧另行授予 import
> 能力(`agentElevated` 或 Workbench 路径)。没有这层授权就别在这条路上耗——**直接回落 > 能力(`agentElevated` 或 Workbench 路径)。没有这层授权就别在这条路上耗——如实告诉用户
> L3-fallbackPython Playwright 连用户已登录的 Chrome**,那是当前更稳的登录态复用方式 > 「当前无法复用你的登录态」,按无登录态继续或放弃
动作枚举只有这 6 个:`discover` | `create_plan` | `dry_run` | `apply` | `rollback` | `list_plans` 动作枚举只有这 6 个:`discover` | `create_plan` | `dry_run` | `apply` | `rollback` | `list_plans`
**没有 `plan`**)。完整流程是 `discover → create_plan → dry_run → apply` **没有 `plan`**)。完整流程是 `discover → create_plan → dry_run → apply`
```yaml
BrowserImport:
action: discover # 列出可导入的来源,返回不透明 sourceProfileId
```
```yaml ```yaml
BrowserImport: BrowserImport:
action: create_plan # 域名授权、来源、冲突策略都在这一步定死 action: create_plan # 域名授权、来源、冲突策略都在这一步定死
@@ -155,57 +257,85 @@ BrowserImport:
conflictStrategy: newer-wins # keep-target | replace-target | newer-wins | fail-on-conflict conflictStrategy: newer-wins # keep-target | replace-target | newer-wins | fail-on-conflict
``` ```
```yaml `dry_run` 先看命中多少条再 `apply`(只认 `create_plan` 返回的 `planId`,前缀 `bimp_`);写坏了
BrowserImport: `rollback` + 同一个 planId 回退;`list_plans` 查历史。解密与过滤全在 Host 侧完成,
action: dry_run # 先看命中多少条,再决定要不要真的写入 **Cookie 值不会进入 Agent 上下文或审计日志**
planId: bimp_xxx # create_plan 返回的 ID前缀是 bimp_
```
```yaml
BrowserImport:
action: apply # 只认 planId域名/策略在 create_plan 时已固化
planId: bimp_xxx
```
写坏了用 `action: rollback` + 同一个 `planId` 回退;`list_plans` 查当前 Space 的历史 plan。
解密与过滤全在 Host 侧完成,**Cookie 值不会进入 Agent 上下文或审计日志**。
### BrowserShare ### BrowserShare
把 Space / Session 委派给另一个 Agent`shareMode` 可选 `snapshot`(只读副本)、 把 Space / Session 委派给另一个 Agent`shareMode` 可选 `snapshot`(只读副本)、
`copy-on-write`(写时复制)、`live-shared`(实时共享)、`handoff`(移交控制权)。 `copy-on-write`(写时复制)、`live-shared`(实时共享)、`handoff`(移交控制权)。
## fetch.browser 配方:带登录态取接口数据
站内接口(列表、评论、订单等 JSON在登录态下的正解——**页面上下文跑 `fetch`**,自动带该
origin 的 Cookie同 origin、受 Grant origins 约束,走 `page.evaluate` 已有闸门:
```yaml
BrowserAct:
action: page.evaluate
params:
expression: |
fetch('/api/v1/comments?page=1&size=20', {
headers: { accept: 'application/json' }
}).then(r => r.text())
awaitPromise: true
```
-`tab.navigate` 到该站任意页面建立 origin 与 Cookie再发 fetch路径写**相对路径**
- 只能访问当前 tab origin跨站接口先导航过去
- 大 JSON 用 `.text()` 拿原文自己截取,或分页多次取
- 非 allow-all 模式会弹审批卡page.evaluate 人工闸门),向用户说明用途即可
## 交互通道选用input.* vs page.element反检测决策
| 场景 | 用什么 | 原因 |
|------|--------|------|
| 反检测站点(小红书/微博/B站等的一切点击/输入 | **`input.*`** | 坐标派发 + #1808 拟真轨迹 + 身份一致性UA/UA-CH 无 Electron/Headless 痕迹) |
| 表单批量填充等**站点不检测**的场景 | **`page.element` 写类** | 一条命令完成 fill/select-option/check比逐元素 input.* 快得多 |
| 需要元素属性/坐标/计数/批量文本 | `page.element` 读类 | 只读档browser.observe.snapshot无写审批 |
| 任何「用 JS 直调 el.click()」的想法 | ❌ 禁止 | 绕开全部拟真投入S10 红线);指针动作只走 input.* |
## 截图:像素直接给你,通常不需要再 Read ## 截图:像素直接给你,通常不需要再 Read
> **需要客户端 10.0.100+。** 更老的版本上截图只回元数据; `BrowserSnapshot mode:visual``BrowserAct page.screenshot` 会把截图像素作为 image 块
> 若需要画面信息请改用 `semantic` / `accessibility` 快照,或回落 Jina / Playwright **直接放进工具结果**——视觉模型当场就能看,不必再调 `Read`
`BrowserSnapshot` `mode: visual``BrowserAct` `page.screenshot` 会把截图像素作为 - **元素级裁剪放大**`options.clip={x,y,width,height,scale}``scale` 最大 4。先用
image 块**直接放进工具结果**——视觉模型当场就能看,不必再调 `Read` `semantic` 快照拿到元素坐标,再截那一块并放大——验证码、小按钮在整页截图里只有几十像素,
看不清时用它。**不必走 `cdp.raw`**
只有这几种情况才需要走 `result.artifact.absolutePath``Read` 一次: - **整页截图**`captureBeyondViewport: true`(超出视口部分也截)
- 只有结果里明确写了「截图已保存,但…未附带像素」(超预算/体积过大)或需要原始分辨率时,
- 结果里明确写了「截图已保存,但…未附带像素」(超出单图预算、或体积过大) 才走 `result.artifact.absolutePath``Read` 一次
- 你需要的是**原始分辨率**,而附带的像素被压缩过
对同一张图既看 image 块又 `Read` 一遍,只会让它在上下文里占两份。
当前模型不支持视觉输入时,结果会明说「你看不到它的内容」——**此时不要凭空描述画面** 当前模型不支持视觉输入时,结果会明说「你看不到它的内容」——**此时不要凭空描述画面**
改用 `semantic` / `accessibility` 快照拿页面信息。 改用 `semantic` / `text` 快照拿页面信息。
## 视口与 tab.activateS36 起)
Agent 会话actor ≠ user的标签页**常驻离屏原位,保住合成表面**,不再停放成 1×1
| 场景 | 行为 |
|------|------|
| embedded单标签会话 | 无需 `tab.activate` 即可截图/取视口presentation 即呈现) |
| embedded多标签会话的后台 tab | 仍需先 `tab.activate`;未激活的视口快判**秒级**报 `BROWSER_VIEWPORT_UNAVAILABLE`,不再挂满 30s也不会毁掉标签页 |
| standalone | 后台 tab 可直接截图 |
多 Space 依然可以并发导航/快照/输入,互不串扰。
## 已知边界(照做,别试探) ## 已知边界(照做,别试探)
| 边界 | 说明 | | 边界 | 说明 |
|------|------| |------|------|
| **截图前必须 `tab.activate`** | 标签页默认停在 `(-10000,-10000,1x1)`,没有合成表面。直接截图会卡满 30 s deadline**并把标签页宿主打掉**,之后全部报 `BROWSER_TAB_HOST_NOT_FOUND` | | **`page.evaluate` 走人工闸门** | 能力 `browser.page.evaluate` 属 always-human-gate非 allow-all 模式每次弹审批卡。返回值已原样过界(超预算截断标 `truncated`),登录态取接口走 fetch.browser 配方 |
| **同时只有一个可见标签页** | `tab.activate` 绑定主窗口、全局互斥。多 Space 可以并发导航/快照/输入,但截图必须逐个 activate 串行 | | **`cdp.raw` 需人工审批** | `browser.raw_cdp.*` 属于永远人工闸门的能力,无人值守流程用不了。元素级裁剪 / 整页截图用 `clip` / `captureBeyondViewport`,别走它 |
| **`page.evaluate` 基本不可用** | 每次调用需人工审批;返回的字符串/对象被替换为 `[REDACTED:browser-runtime-value]`(只有 number/boolean/null 穿透);反调试站点会把它挂起几十秒。读页面用 `BrowserSnapshot` | | **`page.element` 没有 click** | 指针动作必须走 `input.*`拟真轨迹、可审计JS 直调 `el.click()` 是明确禁止的回退 |
| **`cdp.raw` 需人工审批** | `browser.raw_cdp.*` 属于永远人工闸门的能力,无人值守流程用不了。**元素级裁剪不必走它**(需 10.0.100+)——`BrowserSnapshot``options.clip={x,y,width,height,scale}` 直接支持(`scale` 最大 4先用 `semantic` 快照拿到元素坐标,再截那一块并放大。验证码、小按钮在整页截图里只有几十像素,看不清时用它 | | **`fill` 拒绝密码框** | `input[type=password]` 一律拒绝——密码输入只走 `input.text`(拟真键入) |
| **没有批量取文通道** | `semantic` 不含正文,`accessibility` 在真实页面上超限,`page.evaluate` 被审批+脱敏。要正文请回落 Jina / Playwright | | **单条命令 30 s deadline** | 超时只掐掉**这一条命令**stop 加载),标签页仍可用,重试即可;不再打掉宿主。等待类命令超时连 stop 都跳过 |
| **单条命令 30 s deadline** | 超时即判 `browser.host.gone`,会话作废 | | **多标签后台 tab 先 activate** | embedded 多标签会话里后台 tab 截图前先 `tab.activate`,否则秒级 `BROWSER_VIEWPORT_UNAVAILABLE` |
| **用户真实操作会抢控制权** | 在可见标签页上**点击、按键、滚轮、触摸**会触发 `trusted-user-input` 并递增 control epoch打断 Agent 的租约。**鼠标只是划过不会**(默认策略 `intentional-input`,需 10.0.100+;更老版本上划过也会打断),所以用户看着页面移动光标是安全的 | | **用户真实操作会抢控制权** | 在可见标签页上**点击、按键、滚轮、触摸**会触发 `trusted-user-input` 并递增 control epoch打断 Agent 的租约。**鼠标只是划过不会**(默认策略 `intentional-input`),所以用户看着页面移动光标是安全的 |
| **artifact 不要走 `/save`** | 该接口会弹系统「另存为」对话框等人点。**用回执里的 `result.artifact.absolutePath`**(需 10.0.100+;更老版本没有这个字段,只能放弃直接读,改用页面内提取)(工具会把它登记进本次会话的可读白名单),不要自己拼路径——`${DESIRECORE_ROOT}` 这类变量在路径展开里不认(只认 `~` / `$HOME` / `$USERPROFILE`),拼出来是相对路径、`Read` 会报「文件不存在」。默认保留 24 小时 | | **BrowserScript 信任级别等同 Bash** | 脚本可访问 Node fs/net/child_process源码过一次与 Bash 同档审批 + 每能力一问;接管/Lease 轮换即硬停 |
| **artifact 不要走 `/save`** | 该接口会弹系统「另存为」对话框等人点。**用回执里的 `result.artifact.absolutePath`**(工具会把它登记进本次会话的可读白名单),不要自己拼路径——`${DESIRECORE_ROOT}` 这类变量在路径展开里不认(只认 `~` / `$HOME` / `$USERPROFILE`),拼出来是相对路径、`Read` 会报「文件不存在」。默认保留 24 小时 |
## SitePatternRead / SitePatternWrite ## SitePatternRead / SitePatternWrite
@@ -228,17 +358,18 @@ SitePatternWrite:
| 错误 | 原因 | 解决 | | 错误 | 原因 | 解决 |
|------|------|------| |------|------|------|
| `该旧 BrowserXxx/cdp-proxy 入口已停用` | 还在调 v2.0 的旧工具 | 改用 BrowserManage / BrowserSnapshot / BrowserAct | | `该旧 BrowserXxx/cdp-proxy 入口已停用` | 还在调 v2.0 的旧工具 | 改用 BrowserManage / BrowserSnapshot / BrowserAct |
| `BROWSER_TAB_HOST_NOT_FOUND` | 标签页宿主已销毁(多因上一条命令超时) | 重建 Session检查是否漏了 `tab.activate` | | `BROWSER_TAB_HOST_NOT_FOUND` | 标签页宿主真的没了(渲染进程崩溃/销毁) | 重建 Session。命令超时**不再**引发此错误——超时只 stop 这一条命令,标签页保留 |
| `BROWSER_COMMAND_DEADLINE_EXCEEDED` | 单条命令超 30 s | 截图先 activate避免 `page.evaluate` | | `BROWSER_COMMAND_DEADLINE_EXCEEDED` | 单条命令超 30 s | 标签页仍停在上一页/空白页,直接重试;慢加载常态不算故障 |
| `BROWSER_VIEWPORT_UNAVAILABLE` | embedded 多标签会话的后台 tab 没有可用视口 | 先 `tab.activate` 再截图(秒级快判,不是挂死) |
| `BROWSER_TOOL_SESSION_FORBIDDEN` | 会话已关闭/崩溃,或不属于当前 Agent | 重新 `list_sessions`,必要时重建 | | `BROWSER_TOOL_SESSION_FORBIDDEN` | 会话已关闭/崩溃,或不属于当前 Agent | 重新 `list_sessions`,必要时重建 |
| `BROWSER_TOOL_SESSION_AMBIGUOUS` | 当前 Agent 有多个会话且未传 sessionId | 显式传 `sessionId` | | `BROWSER_TOOL_SESSION_AMBIGUOUS` | 当前 Agent 有多个会话且未传 sessionId | 显式传 `sessionId` |
| `BROWSER_HUMAN_APPROVAL_REQUIRED` | 触到人工闸门能力evaluate / raw_cdp / 上传下载 / Cookie 导入等) | 向用户说明用途并等待审批,或换用无需审批的路径 | | `BROWSER_HUMAN_APPROVAL_REQUIRED` | 触到人工闸门能力evaluate / raw_cdp / 上传下载 / Cookie 导入等) | 向用户说明用途并等待审批,或换用无需审批的路径(取文用 extract-text裁剪用 clip |
| `BROWSER_RESULT_TOO_LARGE` | 回执超过 2 MB`accessibility` 快照最常见) | 改用 `semantic` 快照 + 截图;取正文回落 Jina / Playwright | | `BROWSER_RESULT_TOO_LARGE` | 回执超过结果上限 | text/semantic/accessibility 都有 maxBytes + cursor 分页,调小 maxBytes 续读即可,不再整体失败 |
## 调用链路 ## 调用链路
``` ```
Agent → BrowserManage/Snapshot/Act → browser-use serviceCapability/Grant/Lease/Policy 校验) Agent → BrowserManage/Snapshot/Act/Script → browser-use serviceCapability/Grant/Lease/Policy 校验)
→ BrowserHostelectron-embedded 或 standalone-managed→ Chromium → BrowserHostelectron-embedded 或 standalone-managed→ Chromium
↑ 每步产出带 digest 的回执,写入审计事件流 ↑ 每步产出带 digest 的回执,写入审计事件流
``` ```

View File

@@ -1,330 +0,0 @@
# CDP Browser Access — Login-Gated Sites Manual
Detailed recipes for accessing sites that require the user's login session, via Chrome DevTools Protocol (CDP) + Python Playwright.
**Precondition**: Chrome is already running with `--remote-debugging-port=9222` and the user has manually logged in to the target sites. See the main SKILL.md `Prerequisites` section for the launch command.
---
## Why CDP attach, not headless
| Approach | Login state | Anti-bot | Speed | Cost |
|----------|-------------|----------|-------|------|
| Headless Playwright (new context) | ❌ Empty cookies | ❌ Flagged as bot | Slow cold start | Re-login pain |
| `playwright.chromium.launch(headless=False)` | ❌ Fresh profile | ⚠ Sometimes flagged | Slow | Same |
| **CDP attach (`connect_over_cdp`)** | ✅ User's real cookies | ✅ Looks human | Instant | Zero friction |
**Rule**: For any login-gated site, always attach to the user's running Chrome.
---
## Core Template
Every CDP script follows this shape:
```python
from playwright.sync_api import sync_playwright
def fetch_with_cdp(url: str, wait_selector: str | None = None) -> str:
"""Attach to user's Chrome via CDP, fetch URL, return HTML."""
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
# browser.contexts[0] is the user's default context (with cookies)
context = browser.contexts[0]
page = context.new_page()
try:
page.goto(url, wait_until="domcontentloaded", timeout=30000)
if wait_selector:
page.wait_for_selector(wait_selector, timeout=10000)
else:
page.wait_for_timeout(2000) # generic settle
return page.content()
finally:
page.close()
# DO NOT call browser.close() — that would close the user's Chrome!
if __name__ == "__main__":
html = fetch_with_cdp("https://example.com")
print(html[:1000])
```
**Critical**: Never call `browser.close()` when using CDP attach — you'd kill the user's Chrome. Only close the page you opened.
---
## Site Recipes
### 小红书 (xiaohongshu.com)
```python
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
NOTE_URL = "https://www.xiaohongshu.com/explore/XXXXXXXX"
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto(NOTE_URL, wait_until="domcontentloaded")
page.wait_for_selector("#detail-title", timeout=10000)
page.wait_for_timeout(1500) # let images/comments load
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
title = (soup.select_one("#detail-title") or {}).get_text(strip=True) if soup.select_one("#detail-title") else None
desc = (soup.select_one("#detail-desc") or {}).get_text(" ", strip=True) if soup.select_one("#detail-desc") else None
author = soup.select_one(".author-wrapper .username")
print("Title:", title)
print("Author:", author.get_text(strip=True) if author else None)
print("Desc:", desc)
```
**Selectors** (may drift over time — update if they fail):
- Title: `#detail-title`
- Description: `#detail-desc`
- Author: `.author-wrapper .username`
- Images: `.swiper-slide img`
- Comments: `.parent-comment .content`
### B站 (bilibili.com)
```python
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
VIDEO_URL = "https://www.bilibili.com/video/BVxxxxxxxxx"
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto(VIDEO_URL, wait_until="networkidle")
page.wait_for_timeout(2000)
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
print("Title:", soup.select_one("h1.video-title").get_text(strip=True) if soup.select_one("h1.video-title") else None)
print("UP:", soup.select_one(".up-name").get_text(strip=True) if soup.select_one(".up-name") else None)
print("Desc:", soup.select_one(".desc-info-text").get_text(" ", strip=True) if soup.select_one(".desc-info-text") else None)
```
**Tip**: For B站 evaluations, the [公开 API](https://api.bilibili.com/x/web-interface/view?bvid=XXXX) often returns JSON without needing CDP. Try it first:
```bash
curl -s "https://api.bilibili.com/x/web-interface/view?bvid=BVxxxxxxxxx" | python3 -m json.tool
```
### 微博 (weibo.com)
```python
WEIBO_URL = "https://weibo.com/u/1234567890" # or /detail/xxx
# Same CDP template
# Selectors:
# .Feed_body_3R0rO .detail_wbtext_4CRf9 — post text
# .ALink_default_2ibt1 — user link
# article[aria-label="微博"] — each feed item
```
**Note**: Weibo uses React + heavy obfuscation. Selectors change frequently. If selectors fail, pipe the HTML through Jina for clean Markdown:
```python
html = fetch_with_cdp(WEIBO_URL)
# Save to temp file, then:
import subprocess
result = subprocess.run(
["curl", "-sL", f"https://r.jina.ai/{WEIBO_URL}"],
capture_output=True, text=True
)
print(result.stdout)
```
### 知乎 (zhihu.com)
```python
ANSWER_URL = "https://www.zhihu.com/question/123/answer/456"
# Selectors:
# h1.QuestionHeader-title — question title
# .RichContent-inner — answer body
# .AuthorInfo-name — author
```
Zhihu works with CDP but often also renders enough metadata server-side for Jina to work:
```bash
curl -sL "https://r.jina.ai/https://www.zhihu.com/question/123/answer/456"
```
Try Jina first, fall back to CDP if content is truncated.
### 飞书文档 (feishu.cn / larksuite.com)
```python
DOC_URL = "https://xxx.feishu.cn/docs/xxx"
# Feishu uses heavy virtualization — must scroll to load all content.
# Recipe:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto(DOC_URL, wait_until="domcontentloaded")
page.wait_for_selector(".docs-render-unit", timeout=15000)
# Scroll to bottom repeatedly to load lazy content
last_height = 0
for _ in range(20):
page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
page.wait_for_timeout(800)
h = page.evaluate("document.body.scrollHeight")
if h == last_height:
break
last_height = h
# Extract text
text = page.evaluate("() => document.body.innerText")
page.close()
print(text)
```
### Twitter / X
```python
TWEET_URL = "https://x.com/username/status/1234567890"
# Selectors:
# article[data-testid="tweet"] — tweet container
# div[data-testid="tweetText"] — tweet text
# div[data-testid="User-Name"] — author
# a[href$="/analytics"] — view count anchor (next sibling has stats)
```
Twitter is aggressive with anti-bot. CDP attach usually works, but set a generous wait:
```python
page.goto(url, wait_until="networkidle", timeout=45000)
page.wait_for_selector('article[data-testid="tweet"]', timeout=15000)
```
---
## Common Patterns
### Pattern 1: Scroll to load lazy content
```python
def scroll_to_bottom(page, max_steps=30, pause_ms=800):
last = 0
for _ in range(max_steps):
page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
page.wait_for_timeout(pause_ms)
h = page.evaluate("document.body.scrollHeight")
if h == last:
return
last = h
```
### Pattern 2: Screenshot a specific element
```python
element = page.locator("article").first
element.screenshot(path="/tmp/article.png")
```
### Pattern 3: Extract structured data via JavaScript
```python
data = page.evaluate("""() => {
const items = document.querySelectorAll('.list-item');
return Array.from(items).map(el => ({
title: el.querySelector('.title')?.innerText,
url: el.querySelector('a')?.href,
}));
}""")
print(data)
```
### Pattern 4: Fill a form and click
```python
page.fill("input[name=q]", "search query")
page.click("button[type=submit]")
page.wait_for_load_state("networkidle")
```
### Pattern 5: Clean HTML via Jina after extraction
When selectors are unreliable, dump the full page HTML and let Jina do the cleaning:
```python
html = page.content()
# Save to file, serve via local HTTP, or just pipe the original URL:
import subprocess
clean_md = subprocess.run(
["curl", "-sL", f"https://r.jina.ai/{url}"],
capture_output=True, text=True
).stdout
print(clean_md)
```
---
## Troubleshooting
### `connect_over_cdp` fails with `ECONNREFUSED`
Chrome is not running with remote debugging. Tell the user:
> "请先用下面的命令启动 Chrome
> `/Applications/Google\\ Chrome.app/Contents/MacOS/Google\\ Chrome --remote-debugging-port=9222 --user-data-dir=\"${DESIRECORE_ROOT}/chrome-profile\"`
> 然后手动登录需要抓取的网站,再让我继续。"
### `browser.contexts[0]` is empty
Chrome was launched but no windows are open. Ask the user to open at least one tab and navigate anywhere.
### Playwright not installed
```bash
pip3 install playwright beautifulsoup4
# No need for `playwright install` — we're attaching to existing Chrome, not downloading a new browser
```
### Site detects automation
Despite CDP attach, some sites (Cloudflare-protected, Instagram) may still detect automation. Options:
1. Use Jina Reader instead (`curl -sL https://r.jina.ai/<url>`) — often succeeds where Playwright fails
2. Ask the user to manually copy the visible content
3. Use the site's public API if available
### Content is truncated
The page uses virtualization or lazy loading. Apply Pattern 1 (scroll to bottom) before calling `page.content()`.
### `page.wait_for_selector` times out
The selector is stale — the site updated its DOM. Dump `page.content()[:5000]` and inspect manually, or fall back to Jina Reader.
---
## Security Notes
- **Never log or print cookies** from `context.cookies()` even during debugging
- **Never extract and store** the user's session tokens to files
- **Never use the CDP session** to perform writes (post, comment, like) unless the user explicitly requested it
- The `${DESIRECORE_ROOT}/chrome-profile` directory contains the user's credentials — treat it as sensitive
- If the user asks to "log in automatically", refuse and explain they must log in manually in the Chrome window; the skill only reads already-authenticated sessions
---
## When NOT to use CDP
- **Public static sites** → use L1 `WebFetch`, it's faster
- **Heavy SPAs without login walls** → use L2 Jina Reader, it's cheaper on tokens
- **You need thousands of pages** → CDP is not built for scale; look into proper scrapers
CDP is specifically the "right tool" for: **small number of pages + login required + human-like behavior needed**.

View File

@@ -95,19 +95,8 @@ curl -sL "https://r.jina.ai/$URL" > /tmp/jina-cache.md
### Handle very long articles ### Handle very long articles
Jina returns the full article in one response. For articles > 50K chars, pipe through `head` or extract specific sections with Python/awk before feeding back to the model context. Jina returns the full article in one response. For articles > 50K chars, pipe through `head` or extract specific sections with Python/awk before feeding back to the model context.
### Combine with CDP ### Combine with the built-in browser
When you use L3 CDP to fetch a login-gated page, you can pipe the resulting HTML through Jina for clean Markdown instead of parsing with BeautifulSoup: For login-gated pages, the built-in browser already has its own extraction channel with paged budgets — `BrowserAct({ action: 'page.extract-text', params: { format: 'markdown', maxBytes: 65536 } })` returns clean Markdown directly (beyond maxBytes it truncates and hands back a nextCursor). Jina can't log in on your behalf, so use Jina for public pages and `page.extract-text` for logged-in ones; there is no need to round-trip HTML through Jina anymore.
```python
html = fetch_with_cdp(url) # from references/cdp-browser.md
# Now convert via Jina (note: Jina fetches the URL itself, not your HTML)
# So this only works if the content is already visible without login:
import subprocess
md = subprocess.run(["curl", "-sL", f"https://r.jina.ai/{url}"],
capture_output=True, text=True).stdout
```
For truly login-gated content, you must parse the HTML directly (BeautifulSoup) since Jina can't log in on your behalf.
--- ---