desirecore/market
2
0
Fork 0
mirror of https://git.openapi.site/https://github.com/desirecore/market.git synced 2026-09-05 17:03:49 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(web-access): v3.0 内置浏览器能力面同步——删除 Python Playwright 回落（S31） (#85)

Browse Source 
## 变更内容 / What

浏览器升级计划 S31 最终验收信号：web-access 技能与内置受管浏览器新能力面对齐，**删除「用户手工启动调试 Chrome +
Python Playwright」回落路径**。

- **删除回落路径**：Prerequisites: Chrome CDP Setup、Layer 3 CDP Browser（Python
Playwright 模板）、Installation Note（pip install
playwright）整段删除；references/cdp-browser.md 文件删除；jina-reader.md 的 CDP 引用改为
page.extract-text
- **订正陈旧断言**：
- 「没有批量取文通道」→ BrowserSnapshot mode:text /
page.extract-text（maxBytes/cursor 分页，超出截断给 nextCursor）
- 「page.evaluate 基本不可用」→ 返回真实值（expression/awaitPromise，超预算截断标
truncated）；仍走人工闸门
- 「截图前必须 tab.activate / 串行截图 / BROWSER_TAB_HOST_NOT_FOUND」→ S36 订正：Agent
单标签会话免 activate；多标签后台 tab 秒级报 BROWSER_VIEWPORT_UNAVAILABLE；命令超时只 stop 不
close，标签页可重试
  - 「accessibility 超限即失败」→ 尊重 depth + maxBytes 截断翻页（S8）
  - 「只有整页截图」→ clip{x,y,width,height,scale≤4} + captureBeyondViewport
- **provides.tools 加 BrowserScript**（code-mode；信任级别等同 Bash）
- **新增选用规则（D4 唯一约束机制）**：反检测站点一律优先 input.*（#1808 输入拟真 +
身份一致性）；page.element 写类仅用于表单批量填充等站点不检测场景；JS 直调 el.click() 为禁止回退
- **新增 fetch.browser 配方**：page.evaluate 页面上下文跑 fetch（带 origin Cookie、同
origin、受 Grant origins 约束）——登录态取站内接口的正解
- **能力速查**：page.element 九 op / page.wait 九 until / inline wait 块 / loc=
方言 / BrowserScript / 跨源 iframe 快照（S35）
- **版本** 2.2.1 → 3.0.0（删除回落层为 breaking）；source_hash
重算；required_client_version 维持 10.0.98（新能力在正文标注 10.0.112+）

## Why

v2.x 时代回落路径存在的每一条理由（无批量取文、evaluate 不可用、截图必须串行 activate）均已被
S2–S14/S35/S36 覆盖；文档继续引导用户手工起调试 Chrome 会误导新 Agent 走已废弃路径。

双语同步修改（SKILL.md / SKILL.zh-CN.md heading 数一致，i18n-validate 通过）。

- [x] CLA
This commit is contained in:
yi-ge
2026-08-17 01:25:46 -04:00
committed by GitHub
parent 43239669e5
commit 0d4d7b208b
5 changed files with 482 additions and 868 deletions
Download Patch File Download Diff File Expand all files Collapse all files

366
skills/web-access/SKILL.md
View File

@@ -5,18 +5,17 @@ description: >-
— searching for current information, fetching public web pages, browsing
login-gated sites (微博/小红书/B站/飞书/Twitter), comparing products,
researching topics, gathering documentation, or summarizing news.
This skill orchestrates four complementary layers: (1) WebSearch + WebFetch
This skill orchestrates three complementary layers: (1) WebSearch + WebFetch
for public pages, (2) Jina Reader as the default token-optimization layer for
heavy/JS-rendered pages, (3) the governed built-in browser (isolated
BrowserSpace + Cookie import) to reach and interact with login-gated sites,
and (4) Chrome DevTools Protocol via Python Playwright as the fallback for
bulk text extraction and complex automation. Always cite source URLs.
heavy/JS-rendered pages, and (3) the governed built-in browser (isolated
BrowserSpace + Cookie import + bulk text extraction + waits + code mode) to
reach, interact with, and read login-gated sites. Always cite source URLs.
Use when 用户提到 联网搜索、上网查、
查资料、抓取网页、研究、调研、最新资讯、文档查询、对比、竞品、技术文档、
新闻、网址、URL、找一下、搜一下、查一下、小红书、B站、微博、飞书、Twitter、
推特、X、知乎、公众号、已登录、登录状态。
license: Complete terms in LICENSE.txt
version: 2.2.1
version: 3.0.0
type: procedural
risk_level: low
status: enabled
@@ -27,13 +26,14 @@ tags:
- fetch
- research
- browsing
- browser
- cdp
- playwright
provides:
tools:
- BrowserManage
- BrowserSnapshot
- BrowserAct
- BrowserScript
- BrowserImport
- BrowserShare
- SitePatternRead
@@ -41,7 +41,7 @@ provides:
- LocalBookmarks
metadata:
author: desirecore
updated_at: '2026-08-06'
updated_at: '2026-08-17'
i18n:
default_locale: en-US
source_locale: zh-CN
@@ -50,17 +50,17 @@ metadata:
- en-US
zh-CN:
name: 联网访问
short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问、研究调研工作流
description: 四层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问、Python Playwright CDP 兜底。
short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问与取文、研究调研工作流
description: 三层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问、交互与取文。
body: ./SKILL.zh-CN.md
source_hash: sha256:ce46e1982de8cd65
source_hash: sha256:20c98f047378220a
translated_by: human
en-US:
name: Web Access
short_desc: Web search, page fetching, logged-in access via the governed built-in browser, research workflows
description: A four-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, reach logged-in sites through the governed built-in browser, and fall back to Chrome CDP.
description: A three-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, and reach, interact with, and read logged-in sites through the governed built-in browser.
body: ./SKILL.md
source_hash: sha256:ce46e1982de8cd65
source_hash: sha256:20c98f047378220a
translated_by: human
market:
icon: >-
@@ -73,9 +73,8 @@ market:
stroke="url(#wa-a)" stroke-width="1"
stroke-opacity="0.35"/><path d="M2 10h16" stroke="url(#wa-a)"
stroke-width="1" stroke-opacity="0.35"/><path d="M10 2v16"
stroke="url(#wa-a)" stroke-width="1"
stroke-opacity="0.35"/><circle cx="18.5" cy="18.5" r="2.5"
stroke="#34C759" stroke-width="1.5" fill="#34C759"
stroke="url(#wa-a)" stroke-width="1" stroke-opacity="0.35"/><circle cx="18.5"
cy="18.5" r="2.5" stroke="#34C759" stroke-width="1.5" fill="#34C759"
fill-opacity="0.12"/><path d="M20.5 20.5l2 2" stroke="#34C759"
stroke-width="1.8" stroke-linecap="round"/></svg>
category: research
@@ -90,28 +89,30 @@ market:
## L0: One-line Summary
A four-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, reach login-gated sites through the governed built-in browser, and fall back to Chrome CDP.
A three-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, and reach, interact with, and read login-gated sites through the governed built-in browser (v3.0 ships bulk text extraction, waits, and code mode in-browser; the Python Playwright fallback is gone).
## L1: Overview & Use Cases
### Capability
web-access is a **procedural skill** that provides four complementary layers of web access:
web-access is a **procedural skill** that provides three complementary layers of web access:
- **L1** (WebSearch + WebFetch): public, static pages
- **L2** (Jina Reader): JS-rendered heavy pages, saving tokens by default
- **L3-fast** (governed built-in browser — **rewritten in v2.1**): reach and *interact with* logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. **Reading long article text is still L2/L3-fallback's job** — see the extraction note below
- **L3-fallback** (Chrome CDP + Python Playwright): backup for complex automation (long waits, race conditions, custom in-browser scripts)
- **L3** (governed built-in browser, capability surface completed in v3.0): reach, *interact with*, and **read** logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. Bulk text extraction (`page.extract-text`), discriminated waits (`page.wait`), and code mode (`BrowserScript`) all close the loop inside this layer
### v2.1 — governed built-in browser (default-hidden, exposed only after Skill activation)
The v2.x fourth layer — "user manually launches a debug Chrome + Python Playwright CDP" — was removed in v3.0: every reason it existed for (no bulk text channel, evaluate unusable, screenshots must activate-serialize) is now covered by the built-in browser, see the cheatsheet below.
When you call `Skill('web-access')`, the following 8 tools are injected into the current session so the LLM can drive the built-in browser directly:
### v3.0: governed built-in browser (default-hidden, exposed only after Skill activation)
When you call `Skill('web-access')`, the following 9 tools are injected into the current session so the LLM can drive the built-in browser directly:
| Tool | Purpose |
|------|---------|
| BrowserManage | Create/destroy isolated BrowserSpace, start sessions, manage tabs |
| BrowserSnapshot | `semantic` / `accessibility` / `visual` page snapshots — the primary way to read a page |
| BrowserAct | One governed action per call: navigate, click, type, scroll, screenshot, … |
| BrowserSnapshot | `semantic` / `text` / `accessibility` / `visual` snapshots — the primary way to read a page |
| BrowserAct | One governed action per call: navigate, input, extract text, wait, element ops, screenshot, … |
| BrowserScript | **Code mode**: one async JS script issues browser commands back-to-back, eliminating per-action round trips (trust level equals Bash) |
| BrowserImport | Import Cookies from the user's Chrome/Edge/Firefox/Safari profile (human-approved; **needs `browser.import.*` granted by the Host — not available to a plain `create_space` session**) |
| BrowserShare | Delegate a Space/Session to another Agent (isolated / snapshot / copy-on-write / live) |
| SitePatternRead / SitePatternWrite | Per-domain "site experience" (AgentFS three-layer) |
@@ -119,20 +120,22 @@ When you call `Skill('web-access')`, the following 8 tools are injected into the
> **Important**: before `Skill('web-access')` is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See [references/browser-tools.md](references/browser-tools.md).
>
> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+.
> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+; `page.extract-text` / `page.element` / `page.wait` / inline wait blocks / cross-origin iframe snapshots need v10.0.112+, and `BrowserScript` needs a build containing S17/S18.
### Use Cases
- The user needs to search for current information or research a specific topic
- The user needs to fetch public web content or technical documentation
- The user needs to access logged-in sites (Xiaohongshu, Bilibili, Weibo, Feishu, Twitter, etc.)
- The user needs to access logged-in sites (Xiaohongshu, Bilibili, Weibo, Feishu, Twitter, etc.) and **read the body text**
- The user needs to pull data from a site's own API in a logged-in context (lists, comments, orders, …)
- The user needs to compare products, aggregate news, or investigate API/library versions
### Core Value
- **Four-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand
- **Token optimization**: Jina Reader cuts token usage by 50–80% by default
- **Logged-in session reuse**: attach to the user's Chrome via CDP in the fallback layer, or — where the Host has granted `browser.import.*` — import their Cookies into an isolated Space via BrowserImport; either way, no re-login required
- **Three-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand
- **Token optimization**: Jina Reader cuts token usage by 50–80% by default; `page.extract-text`'s maxBytes/cursor paging keeps even long logged-in articles under control
- **Logged-in session reuse**: where the Host has granted `browser.import.*`, BrowserImport brings the user's Cookies into an isolated Space — no re-login required
- **Zero external dependencies**: no Python/Playwright install, no manually launched debug Chrome
## L2: Detailed Specification
@@ -144,53 +147,6 @@ When you complete a research task, you **MUST** cite all source URLs in your res
If any fetch fails, explicitly tell the user which URL failed and which fallback you used.
---
## Prerequisites: Chrome CDP Setup (for login-gated sites)
**Required for the L3-fallback layer** (Python Playwright) — and, in practice, still the reliable way to reuse a login. The L3-fast built-in browser can skip it only when the Host has granted `browser.import.*` so that `BrowserImport` actually works; otherwise set this up.
### One-time setup
Launch a dedicated Chrome instance with remote debugging enabled:
**macOS**:
```bash
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Linux**:
```bash
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Windows (PowerShell)**:
```powershell
& "C:\Program Files\Google\Chrome\Application\chrome.exe" `
--remote-debugging-port=9222 `
--user-data-dir="$env:USERPROFILE\.desirecore\chrome-profile"
```
After launch:
1. Manually log in to the sites you need (Xiaohongshu, Bilibili, Weibo, Feishu, …)
2. Leave this Chrome window open in the background
3. Verify the debug endpoint: `curl -s http://localhost:9222/json/version` should return JSON
### Verify CDP is ready
Before any CDP operation, always run:
```bash
curl -s http://localhost:9222/json/version | python3 -c "import sys,json; d=json.load(sys.stdin); print('CDP ready:', d.get('Browser'))"
```
If the command fails, tell the user: "Please launch Chrome with the remote debugging port enabled (see the Prerequisites section of the web-access skill)."
---
## Tool Selection Decision Tree
```
@@ -207,12 +163,12 @@ User intent
│ (Jina Reader = default for JS-rendered content, saves tokens)
│
├─ "Read this login-gated page" (Xiaohongshu/Bilibili/Weibo/Feishu/Twitter/Zhihu/WeChat)
│ ├─→ Reach it: BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
│ │ → tab.activate + page.screenshot to confirm you landed on the content
│ │ (semantic snapshot has no body text). Logged-in? BrowserImport only if
│ │ browser.import.* was granted — otherwise reuse the login via CDP below.
│ └─→ Extract the text: verify CDP ready, then python3 playwright.connect_over_cdp()
│ → page.content() → Jina Reader / BeautifulSoup
│ └─→ BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
│ → BrowserAct(page.extract-text) ← body text read out directly, maxBytes/cursor paging
│ Logged in? BrowserImport only if browser.import.* was granted
│
├─ "Pull data from the site's API in a logged-in context"
│ └─→ fetch.browser recipe: run fetch inside BrowserAct(page.evaluate) with that origin's cookies
│
├─ "API documentation / GitHub / npm package info"
│ └─→ Prefer official API endpoints over scraping HTML:
@@ -221,24 +177,19 @@ User intent
│ - PyPI: curl https://pypi.org/pypi/<pkg>/json
│
└─ "Real-time interactive task" (click, fill form, scroll, screenshot)
├─→ **Default: built-in browser** (BrowserManage → BrowserAct → BrowserSnapshot —
│ see references/browser-tools.md, no Python needed)
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when the built-in browser is insufficient
(e.g., complex race conditions, multi-event waits, long-running in-browser scripts)
└─→ built-in browser (BrowserManage → BrowserAct → BrowserSnapshot —
see references/browser-tools.md, no Python needed)
```
### Four-layer strategy summary
### Three-layer strategy summary
| Layer | Use case | Primary tool | Token cost |
|-------|----------|--------------|------------|
| L1 | Public, static | `WebFetch` | Low |
| L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) |
| **L3-fast** | **Login-gated navigation & interaction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot)** | Medium |
| L3-fallback | Complex automation (race / long-wait / custom scripts) | `Bash + Python Playwright CDP` | Medium |
| **L3** | **Login-gated navigation, interaction & extraction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript)** | Medium |
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when the built-in browser is insufficient.
---
**Default priority**: L1 for simple public pages → L2 for heavy → **L3 for login-gated (body text and in-site API data included)**.
## Supported Sites Matrix
@@ -248,16 +199,14 @@ User intent
| GitHub README, issues, PRs | `gh api` (best) → L1 WebFetch | Prefer API |
| Hacker News, Reddit | L1 WebFetch | Public content |
| Medium, Dev.to | L2 Jina Reader | JS-rendered, member gates |
| Twitter/X | L3 CDP (or L2 Jina with `x.com`) | Login required for full thread |
| Xiaohongshu (xiaohongshu.com) | L3 CDP | Login required |
| Bilibili (bilibili.com) | L3 CDP | Login needed for video desc/comments |
| Weibo (weibo.com) | L3 CDP | Long posts require login |
| Zhihu (zhihu.com) | L3 CDP | Long articles + comments require login |
| Feishu Docs (feishu.cn) | L3 CDP | Login required |
| Twitter/X | L3 (or L2 Jina with `x.com`) | Login required for full thread |
| Xiaohongshu (xiaohongshu.com) | L3 built-in browser + BrowserImport | Login required; body text via page.extract-text |
| Bilibili (bilibili.com) | L3 built-in browser + BrowserImport | Login needed for video desc/comments |
| Weibo (weibo.com) | L3 built-in browser + BrowserImport | Long posts require login |
| Zhihu (zhihu.com) | L3 built-in browser + BrowserImport | Long articles + comments require login |
| Feishu Docs (feishu.cn) | L3 built-in browser + BrowserImport | Login required |
| WeChat Official Accounts (mp.weixin.qq.com) | L2 Jina Reader | Usually public, Jina cleans better |
| LinkedIn | L3 CDP | Login wall |
---
| LinkedIn | L3 built-in browser + BrowserImport | Login wall |
## Tool Reference
@@ -280,7 +229,7 @@ WebFetch(url="https://example.com/article")
Tips:
- Results cached for 15 min
- Returns cleaned Markdown with title + URL + body
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (CDP)
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (built-in browser)
### Layer 2: Jina Reader (default for heavy pages)
@@ -298,111 +247,94 @@ Why Jina is the default token-saver:
See [references/jina-reader.md](references/jina-reader.md) for advanced endpoints and rate limits.
### Layer 3: CDP Browser (login-gated access)
### Layer 3: built-in browser (login-gated access)
Use Python Playwright's `connect_over_cdp()` to attach to the user's running Chrome (which already has login cookies). **No re-login needed.**
The full command surface, capability tiers, and boundaries are in [references/browser-tools.md](references/browser-tools.md). The loop:
**Minimal template**:
```bash
python3 << 'PY'
from playwright.sync_api import sync_playwright
1. `BrowserManage(create_space)` → `BrowserManage(start_session)` for a sessionId + first tab
2. Need login state → `BrowserImport` (only if the Host granted `browser.import.*`; without that grant the user's Cookies cannot be reused — tell the user and continue without login or abort)
3. `BrowserAct(tab.navigate)` to reach the page
4. `BrowserSnapshot(semantic)` for interactive-element `ref` handles (cross-origin iframe elements are in the same tree with globally sequential refs)
5. Interact via `input.*` (humanized trajectories) or `page.element` (bulk form writes); wait for results via `page.wait` or inline wait blocks
6. Read body text via `BrowserSnapshot(text)` or `BrowserAct(page.extract-text)`; pull API data via the fetch.browser recipe
7. `BrowserManage(close_session)` when done
TARGET_URL = "https://www.xiaohongshu.com/explore/..."
Multi-step sequences (navigate→snapshot→click→wait→extract) can be done in one `BrowserScript` run, skipping the per-action IPC round trips.
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
context = browser.contexts[0] # reuse user's default context (has cookies)
page = context.new_page()
page.goto(TARGET_URL, wait_until="domcontentloaded")
page.wait_for_timeout(2000) # let lazy content load
html = page.content()
page.close()
## L3 Cheatsheet (v3.0)
# Print first 500 chars to verify
print(html[:500])
PY
```
**Extract text via BeautifulSoup** (no Jina round-trip):
```bash
python3 << 'PY'
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto("https://www.bilibili.com/video/BV...", wait_until="networkidle")
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
title = soup.select_one("h1.video-title")
desc = soup.select_one(".video-desc")
print("Title:", title.get_text(strip=True) if title else "N/A")
print("Desc:", desc.get_text(strip=True) if desc else "N/A")
PY
```
See [references/cdp-browser.md](references/cdp-browser.md) for:
- Per-site selectors (Xiaohongshu / Bilibili / Weibo / Zhihu / Feishu)
- Scrolling & lazy-load patterns
- Screenshot & form-fill recipes
- Troubleshooting connection issues
---
## L3-fast: Built-in Browser Cheatsheet (v2.1)
**Only after you call `Skill('web-access')` will the following tools appear in `tools[]`.**
| Tool | One-line example |
|------|-----------------|
| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | One isolated Space per task |
| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | Start a session, returns sessionId + first tab |
| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | Tab / lifecycle management |
| `BrowserAct({ action: 'tab.navigate', params: { url } })` | Navigate the current tab |
| `BrowserSnapshot({ mode: 'semantic' })` | Read the page: interactive elements + `ref` handles |
| `BrowserAct({ action: 'input.click', params: { ref } })` | Click by snapshot `ref`, never by raw x/y |
| `BrowserAct({ action: 'input.text', params: { text } })` | Type into the focused element |
| `BrowserAct({ action: 'input.wheel', params: { x: 640, y: 400, deltaX: 0, deltaY: 720 } })` | Scroll to trigger lazy loading — `x`/`y` (or `ref`) is **required**, and the session needs `browser.input.pointer.wheel` |
| `BrowserAct({ action: 'tab.activate', params: { bounds } })` → `page.screenshot` | **activate first, then screenshot** |
| `BrowserImport({ action: 'discover' \| 'create_plan' \| 'dry_run' \| 'apply' \| 'rollback' \| 'list_plans' })` | Reuse the user's login cookies — those 6 are the **complete** action set; happy path is `discover → create_plan → dry_run → apply`. **Needs `browser.import.*`, which `create_space` does not grant** (see below) |
Full API and edge cases: see [references/browser-tools.md](references/browser-tools.md).
**How to read a page** — pick by what you need:
### Reading a page: pick the channel by need
| You need | Use | Note |
|----------|-----|------|
| Interactive elements + `ref` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons / inputs / links only — **no article text** |
| Article text on a small/simple page | `BrowserSnapshot({ mode: 'accessibility' })` | Returns StaticText nodes; fails with `BROWSER_RESULT_TOO_LARGE` on real content pages (2 MB result cap, and the `depth` argument is currently ignored by the host) |
| Article text on a real page | L2 Jina Reader (public) or L3-fallback Playwright (login-gated) | The built-in browser has no working bulk text-extraction channel yet |
| What the page looks like | `tab.activate` → `BrowserAct({ action: 'page.screenshot' })` | Full-page only; read it visually |
| Interactive elements + `ref` / `loc=` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons/inputs/links + per-line `[ref=eN]` and (when producible) `[loc=...]` stable selectors; cross-origin iframes in the same tree |
| Article body text | `BrowserSnapshot({ mode: 'text' })` or `BrowserAct({ action: 'page.extract-text' })` | markdown/text formats; beyond maxBytes it truncates and hands back a nextCursor for paging — no error |
| Accessibility tree | `BrowserSnapshot({ mode: 'accessibility' })` | Respects `depth` (default 50, max 100) and the maxBytes budget — truncates + pages instead of failing wholesale |
| What the page looks like | `BrowserSnapshot({ mode: 'visual' })` or `BrowserAct({ action: 'page.screenshot' })` | Pixels land directly in the result (vision models read them in place); `clip={x,y,width,height,scale}` for element-level crops (scale up to 4) and `captureBeyondViewport` for full-page capture |
`page.evaluate` is **not** an extraction channel: it needs human approval per call and its string/object return values come back as `[REDACTED:browser-runtime-value]`.
### Command surface at a glance
`BrowserAct` actions grouped by purpose (full enum in the tool schema):
- **tab.***: `navigate` / `back` / `forward` / `reload` / `activate` / `close`
- **input.***: `move` / `click` / `double-click` / `drag` / `wheel` / `touch` / `pinch` / `key` / `text` — humanized input (#1808: consistent UA/UA-CH identity + real trajectories); the preferred interaction channel on anti-bot sites
- **page.element** (discriminated op × selector, nine ops): write ops `fill` / `select-option` / `check` / `uncheck` / `scroll-into-view`; read ops `get-attribute` / `bounding-box` / `count` / `all-inner-texts`. Selectors speak the `loc=` dialect or a snapshot `ref` (with snapshotId). `fill` refuses `input[type=password]`
- **page.wait** (discriminated until, nine values): poll-type `load` / `domcontentloaded` / `networkidle` / `selector` / `url` / `timeout` return `waited:false` on timeout; event-type `request` / `response` / `download` throw on timeout. Default 10s, max 60s
- **Inline wait blocks**: `params.wait` (isomorphic to page.wait params) on `tab.navigate` / `input.click` / `input.key` / `page.element{op:"fill"}` — one receipt completes "act→wait for result", the waiter registers before the action, no cross-IPC race
- **page.evaluate**: `{ expression, awaitPromise }`, return values cross as-is (over-budget results truncate with a `truncated` flag, never throw); capability `browser.page.evaluate` sits behind the human gate (allow-all mode skips the card)
- **page.extract-text / page.screenshot / page.wait**: see the table above and the fetch.browser recipe
`loc=` selector dialect (S7/S9): `e<N>` (must carry the snapshotId of the snapshot that issued the ref), `loc=css:` / `loc=role:` / `loc=text:` / `loc=testid:`, bare CSS, composable with `internal:nth/last/scope/filter`. Unknown prefixes fail explicitly — never silently degrade to CSS.
### Choosing the interaction channel: input.* vs page.element
- **On anti-bot sites (Xiaohongshu/Weibo/Bilibili etc.) always prefer `input.*`**: it rides the #1808 humanized input pipeline (coordinate dispatch, humanized trajectories, auditable visualization) plus the identity layer that keeps UA/UA-CH free of Electron/Headless tells
- **`page.element` write ops fit bulk form filling on sites that don't detect automation**: one call fills/selects/checks, far faster than per-element input.click + input.text
- **Red line**: `page.element` deliberately has no click — pointer actions must go through `input.*`; invoking `el.click()` via JS bypasses the entire humanization investment and is an explicitly forbidden fallback
### The fetch.browser recipe: pull API data with login state
The right way to pull a site's own API (lists, comments, orders, any JSON) in a logged-in context: run `fetch` **in the page context** — it carries that origin's cookies automatically, stays same-origin, is bounded by Grant origins, and rides the existing `page.evaluate` gate. It is a wrapper usage of `BrowserAct({ action: 'page.evaluate' })`:
```yaml
BrowserAct:
action: page.evaluate
params:
expression: |
fetch('/api/v1/comments?page=1&size=20', {
headers: { accept: 'application/json' }
}).then(r => r.text())
awaitPromise: true # default true; waits for the returned Promise to settle
```
Notes:
- `tab.navigate` to any page on the site first (establishes the origin and cookies), then fire the fetch; use a relative path so it is same-origin by construction
- Return values cross as-is; for large JSON take `.text()` and slice it yourself, or page through multiple calls
- Only the current tab's origin is reachable (Grant origins constraint); for another site's API, navigate there first
- `page.evaluate` is a human-gated capability: outside allow-all mode an approval card appears — explain the purpose to the user
### Recommended flow (Xiaohongshu example)
```
1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
← an explicit list *narrows* the lease; include browser.input.pointer.wheel if you will scroll
3. Reuse the login: L3-fallback Playwright against the user's own Chrome is the default path.
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }) only works if the
Host granted browser.import.* separately — create_space alone does not. If it is denied, fall back.
← an explicit list *narrows* the lease; include browser.input.pointer.wheel to scroll
and browser.observe.snapshot to extract text
3. Reuse the login: only when the Host granted browser.import.*,
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }).
Without that grant the user's cookies cannot be reused — say so, then continue
as logged-out or abort.
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
5. BrowserSnapshot({ mode: 'semantic' }) ← element refs for interaction (no body text)
tab.activate + page.screenshot ← confirm the note actually rendered
→ extract the note text via L3-fallback Playwright
6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience
7. At task end → BrowserManage({ action: 'close_session', sessionId })
8. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
5. BrowserSnapshot({ mode: 'semantic' }) ← interaction refs (cross-origin iframes in the same tree)
6. BrowserAct({ action: 'page.extract-text', params: { format: 'markdown' } })
← body text read out directly; if too long, pass back the returned nextCursor to continue
7. Need to confirm rendering → BrowserSnapshot({ mode: 'visual' }) (pixels readable in place)
8. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience
9. At task end → BrowserManage({ action: 'close_session', sessionId })
10. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
```
---
## Site Experience Accumulation (v2.0)
## Site Experience Accumulation
When the task ends and you've discovered new anti-bot pitfalls, effective selectors, or platform quirks, call:
@@ -411,7 +343,7 @@ SitePatternWrite({
domain: "xiaohongshu.com",
scope: "agent", // agent=shared (Git-tracked, can be published); user=private
mode: "merge", // merge appends; replace overwrites
content: "## Known pitfalls\n- 2026-05: ...",
content: "## Known pitfalls\n- 2026-08: ...",
confidence: "medium"
})
```
@@ -427,8 +359,6 @@ SitePatternRead({ domain: "xiaohongshu.com" })
Content containing cookies / tokens / phone numbers / emails will **automatically downgrade scope='user'** and notify you.
---
## Common Workflows
Read [references/workflows.md](references/workflows.md) for detailed templates:
@@ -437,11 +367,9 @@ Read [references/workflows.md](references/workflows.md) for detailed templates:
- News aggregation & timelines
- API/library version investigation
Read [references/cdp-browser.md](references/cdp-browser.md) for login-gated site recipes (Xiaohongshu / Bilibili / Weibo / Zhihu / Feishu).
Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader positioning, rate limits, and advanced endpoints.
---
Read [references/browser-tools.md](references/browser-tools.md) for the full built-in browser command surface, capability tiers, and known boundaries.
## Quick Workflow: Multi-Source Research
@@ -455,27 +383,22 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
7. Report with inline [source](url) citations + a Sources list at the end
```
---
## Anti-Patterns (Avoid)
- ❌ **Using WebFetch on obviously heavy sites** — Medium, Twitter, Xiaohongshu will waste tokens or fail. Jump straight to L2/L3.
- ❌ **Launching headless Chrome instead of CDP attach** — loses user's login state, triggers anti-bot, slow cold start. Always use `connect_over_cdp()` to attach to the user's existing session.
- ❌ **Fetching one URL at a time when you need 5** — batch in a single message.
- ❌ **Trusting a single source** — cross-check ≥ 2 sources for non-trivial claims.
- ❌ **Fetching the search result page itself** — WebSearch already returns snippets; fetch the actual articles.
- ❌ **Ignoring the cache** — WebFetch caches 15 min, reuse freely.
- ❌ **Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs.
- ❌ **Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs; a logged-in site's own API goes through the fetch.browser recipe.
- ❌ **Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current.
- ❌ **Hardcoding login credentials in scripts** — always rely on the user's pre-logged CDP session.
- ❌ **Hardcoding login credentials in scripts** — login state can only come from Cookies imported via BrowserImport.
- ❌ **Citing only after the fact** — collect URLs as you fetch, not from memory afterwards.
- ❌ **(v2.1) Writing Python heredoc when the built-in browser would do** — slow, requires Python+Playwright install, and bloats context. Prefer L3-fast; fall back to Python only when the built-in browser is insufficient (race / long-wait / custom scripts).
- ❌ **(v2.1) Calling `page.screenshot` before `tab.activate`** — tabs are parked off-screen at `(-10000,-10000,1x1)` and have no compositing surface, so the capture stalls until the 30s deadline **and the tab host is destroyed**; every later call then fails with `BROWSER_TAB_HOST_NOT_FOUND`. Always `tab.activate` with real bounds first.
- ❌ **(v2.1) Reaching for `page.evaluate` to read a page** — it needs human approval on every call, its string/object return values come back as `[REDACTED:browser-runtime-value]` (only number/boolean/null survive), and anti-debug sites stall it for tens of seconds. Read pages with `BrowserSnapshot` instead.
- ❌ **(v2.1) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`.
- ❌ **(v2.1) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.
---
- ❌ **(v3.0) Using page.element for bulk interaction on anti-bot sites** — it is a direct JS call that bypasses humanized trajectories; on anti-bot sites always use `input.*`; keep `page.element` for bulk form filling where the site doesn't detect automation.
- ❌ **(v3.0) Reading body text from screenshots** — `page.extract-text` / `BrowserSnapshot(text)` hand you markdown/text with paged budgets; save screenshots for layout confirmation and CAPTCHAs where you truly must look.
- ❌ **(v3.0) Tolerating per-action round trips when BrowserScript would do** — navigate→snapshot→click→wait→extract runs as one script; remember BrowserScript's trust level equals Bash and the script source passes one human approval.
- ❌ **(v3.0) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`.
- ❌ **(v3.0) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.
## Example Interaction
@@ -484,26 +407,15 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
**Agent workflow**:
```
1. Recognize → Xiaohongshu is an L3 logged-in site
2. Check CDP: curl -s http://localhost:9222/json/version
├─ Failure → prompt the user to launch Chrome in debug mode, abort
└─ Success → continue
3. Bash: python3 connect_over_cdp script → page.goto(url) → page.content()
4. BeautifulSoup extract h1 title, .note-content, .comments
2. BrowserManage(create_space + start_session)
Need login state → if browser.import.* was granted, run the BrowserImport four steps;
otherwise tell the user the login cannot be reused and continue with the public part
3. BrowserAct(tab.navigate → the note URL)
4. BrowserAct(page.extract-text, format: markdown)
← body text read out directly; page with nextCursor if over budget
5. When returning to the user:
- Cite the original URL
- If content is long, run it through Jina to save tokens
6. Tell the user: "Fetched via your logged-in session, original link: [xhs](url)"
- Quote facts from the extracted text with source links
6. Tell the user: "Fetched via the built-in browser, original link: [xhs](url)"
7. BrowserManage(close_session)
```
---
## Installation Note
CDP features require Python + Playwright installed:
```bash
pip3 install playwright beautifulsoup4
python3 -m playwright install chromium # only needed if user hasn't installed Chrome
```
If `playwright` is not installed when the user requests a login-gated site, run the install commands in Bash and explain you're setting up the browser automation dependency.

338
skills/web-access/SKILL.zh-CN.md
View File

@@ -4,28 +4,30 @@
## L0：一句话摘要
四层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问（v2.1）、Python Playwright CDP 兜底。
三层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问与交互（v3.0 起取文/等待/脚本全部内置，不再需要 Python Playwright 兜底）。
## L1：概述与使用场景
### 能力描述
web-access 是一个**流程型技能（Procedural Skill）**，提供四层互补的联网访问能力：
web-access 是一个**流程型技能（Procedural Skill）**，提供三层互补的联网访问能力：
- **L1**（WebSearch + WebFetch）：公开页面，轻量
- **L2**（Jina Reader）：JS 渲染的重页面，默认节省 Token
- **L3-fast**（内置受管浏览器，**v2.1 重写**）：到达并*操作*登录态/交互站点——每个任务独立 BrowserSpace 隔离、零 Python 依赖、每次动作都有可审计回执。**抽取长正文仍归 L2 / L3-fallback**，见下方取文说明
- **L3-fallback**（Chrome CDP + Python Playwright）：复杂自动化场景兜底（长等待、特殊 race condition 等）
- **L3**（内置受管浏览器，v3.0 能力面补全）：到达、操作并**读取**登录态/交互站点——每个任务独立 BrowserSpace 隔离、零 Python 依赖、每次动作都有可审计回执。批量取文（`page.extract-text`）、判别式等待（`page.wait`）、代码模式（`BrowserScript`）都在本层内闭环
### v2.1 重写：内置受管浏览器（默认隐藏，激活后才暴露）
v2.x 时代的第四层「用户手工启动调试 Chrome + Python Playwright CDP」已在 v3.0 移除：它依赖的每一条理由（无批量取文通道、evaluate 不可用、截图必须串行 activate）都被内置浏览器的新能力覆盖，见下方速查。
调用 `Skill('web-access')` 加载本技能时，以下 8 个工具被注入到当前会话，让 LLM 直接驱动内置浏览器：
### v3.0：内置受管浏览器（默认隐藏，激活后才暴露）
调用 `Skill('web-access')` 加载本技能时，以下 9 个工具被注入到当前会话，让 LLM 直接驱动内置浏览器：
| 工具 | 用途 |
|------|------|
| BrowserManage | 建/销隔离 BrowserSpace、启动会话、管理标签页 |
| BrowserSnapshot | `semantic` / `accessibility` / `visual` 快照——读页面的主通道 |
| BrowserAct | 一次调用一个受管动作：导航、点击、输入、滚动、截图…… |
| BrowserSnapshot | `semantic` / `text` / `accessibility` / `visual` 四种快照——读页面的主通道 |
| BrowserAct | 一次调用一个受管动作：导航、输入、取文、等待、元素操作、截图…… |
| BrowserScript | **代码模式**：一段异步 JS 连续下发浏览器命令，消除逐动作往返（信任级别等同 Bash） |
| BrowserImport | 从用户 Chrome/Edge/Firefox/Safari 配置导入 Cookie（需人工审批；**还需 Host 授予 `browser.import.*`，普通 `create_space` 会话拿不到**） |
| BrowserShare | 把 Space/Session 委派给其他 Agent（隔离 / 快照 / 写时复制 / 实时共享） |
| SitePatternRead / SitePatternWrite | 按域名累积"站点经验"（AgentFS 三层） |
@@ -33,20 +35,22 @@ web-access 是一个**流程型技能（Procedural Skill）**，提供四层互
> **重要**：未调用 Skill('web-access') 之前，这些工具**不会**出现在 LLM 的 tools 列表里——默认对话不消耗其 token。详见 [references/browser-tools.md](references/browser-tools.md)。
>
> **v2.1 已移除**：`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用，调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本版要求客户端 v10.0.98+。
> **v2.1 已移除**：`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用，调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本版要求客户端 v10.0.98+；`page.extract-text` / `page.element` / `page.wait` / 内联 wait 块 / 跨源 iframe 快照需 v10.0.112+，`BrowserScript` 需包含 S17/S18 的更新版本。
### 使用场景
- 用户需要搜索当前信息或研究特定主题
- 用户需要抓取公开网页内容或技术文档
- 用户需要访问登录态站点（小红书、B站、微博、飞书、Twitter 等）
- 用户需要访问登录态站点（小红书、B站、微博、飞书、Twitter 等）并**读出正文**
- 用户需要在登录态下调站内接口取数（列表、评论、订单等）
- 用户需要对比产品、聚合新闻或调查 API/库版本
### 核心价值
- **四层递进**：从轻量搜索到重度 JS 渲染到登录态访问，按需选择
- **Token 优化**：Jina Reader 默认减少 50-80% Token 消耗
- **登录态复用**：兜底层 CDP 连用户 Chrome，或在 Host 已授予 `browser.import.*` 时用 BrowserImport 把 Cookie 导入隔离 Space；两条路都不必重新登录
- **三层递进**：从轻量搜索到重度 JS 渲染到登录态访问，按需选择
- **Token 优化**：Jina Reader 默认减少 50-80% Token 消耗；`page.extract-text` 的 maxBytes/cursor 分页让登录态长文也可控
- **登录态复用**：Host 授予 `browser.import.*` 时用 BrowserImport 把 Cookie 导入隔离 Space，不必重新登录
- **零外部依赖**：不再要求 Python/Playwright 安装，也不再要求用户手工启动调试 Chrome
## L2：详细规范
@@ -58,53 +62,6 @@ When you complete a research task, you **MUST** cite all source URLs in your res
If any fetch fails, explicitly tell the user which URL failed and which fallback you used.
---
## Prerequisites: Chrome CDP Setup (for login-gated sites)
**L3-fallback 层（Python Playwright）必需**，实际上也仍是复用登录态最稳的一条路。L3-fast 内置浏览器只有在 Host 已授予 `browser.import.*`、`BrowserImport` 真能用的前提下才可以跳过这一步；否则照样要配。
### One-time setup
Launch a dedicated Chrome instance with remote debugging enabled:
**macOS**:
```bash
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Linux**:
```bash
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
```
**Windows (PowerShell)**:
```powershell
& "C:\Program Files\Google\Chrome\Application\chrome.exe" `
--remote-debugging-port=9222 `
--user-data-dir="$env:USERPROFILE\.desirecore\chrome-profile"
```
After launch:
1. Manually log in to the sites you need (小红书、B站、微博、飞书 …)
2. Leave this Chrome window open in the background
3. Verify the debug endpoint: `curl -s http://localhost:9222/json/version` should return JSON
### Verify CDP is ready
Before any CDP operation, always run:
```bash
curl -s http://localhost:9222/json/version | python3 -c "import sys,json; d=json.load(sys.stdin); print('CDP ready:', d.get('Browser'))"
```
If the command fails, tell the user: "请先启动 Chrome 并开启远程调试端口（见 web-access 技能的 Prerequisites 部分）。"
---
## Tool Selection Decision Tree
```
@@ -121,12 +78,12 @@ User intent
│ (Jina Reader = default for JS-rendered content, saves tokens)
│
├─ "Read this login-gated page" (小红书/B站/微博/飞书/Twitter/知乎/公众号)
│ ├─→ 到达：BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
│ │ → tab.activate + page.screenshot 确认落到正文页
│ │ （semantic 快照不含正文，不能用来读内容）
│ │ 要登录态：仅在已授予 browser.import.* 时用 BrowserImport，否则走下面的 CDP
│ └─→ 取正文：确认 CDP 就绪后 python3 playwright.connect_over_cdp()
│ → page.content() → Jina Reader / BeautifulSoup
│ └─→ BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
│ → BrowserAct(page.extract-text) ← 正文直接读出，支持 maxBytes/cursor 分页
│ 要登录态：仅在已授予 browser.import.* 时用 BrowserImport
│
├─ "Pull data from the site's API in a logged-in context"
│ └─→ fetch.browser 配方：BrowserAct(page.evaluate) 里跑 fetch（带该 origin 的 Cookie）
│
├─ "API documentation / GitHub / npm package info"
│ └─→ Prefer official API endpoints over scraping HTML:
@@ -135,24 +92,19 @@ User intent
│ - PyPI: curl https://pypi.org/pypi/<pkg>/json
│
└─ "Real-time interactive task" (click, fill form, scroll, screenshot)
├─→ **Default: 内置受管浏览器** (BrowserManage → BrowserAct → BrowserSnapshot —
│ see references/browser-tools.md, no Python needed)
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when 内置浏览器 is insufficient
(e.g., complex race conditions, multi-event waits, long-running in-browser scripts)
└─→ 内置受管浏览器 (BrowserManage → BrowserAct → BrowserSnapshot —
see references/browser-tools.md, no Python needed)
```
### 四层策略总结
### 三层策略总结
| Layer | Use case | Primary tool | Token cost |
|-------|----------|--------------|------------|
| L1 | Public, static | `WebFetch` | Low |
| L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) |
| **L3-fast** | **登录态导航与交互 (PRIMARY)** | **内置受管浏览器（BrowserManage / BrowserAct / BrowserSnapshot）** | Medium |
| L3-fallback | 复杂自动化（race / long-wait / 自定义脚本） | `Bash + Python Playwright CDP` | Medium |
| **L3** | **登录态导航、交互与取文 (PRIMARY)** | **内置受管浏览器（BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript）** | Medium |
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when 内置浏览器不够用。
---
**Default priority**: L1 for simple public pages → L2 for heavy → **L3 for login-gated（含正文与站内接口取数）**。
## Supported Sites Matrix
@@ -162,16 +114,14 @@ User intent
| GitHub README, issues, PRs | `gh api` (best) → L1 WebFetch | Prefer API |
| Hacker News, Reddit | L1 WebFetch | Public content |
| Medium, Dev.to | L2 Jina Reader | JS-rendered, member gates |
| Twitter/X | L3 CDP (or L2 Jina with `x.com`) | Login required for full thread |
| 小红书 (xiaohongshu.com) | L3 CDP | 强制登录 |
| B站 (bilibili.com) | L3 CDP | 视频描述/评论需登录 |
| 微博 (weibo.com) | L3 CDP | 长微博需登录 |
| 知乎 (zhihu.com) | L3 CDP | 长文+评论需登录 |
| 飞书文档 (feishu.cn) | L3 CDP | 必须登录 |
| Twitter/X | L3（或 L2 Jina with `x.com`） | Login required for full thread |
| 小红书 (xiaohongshu.com) | L3 内置浏览器 + BrowserImport | 强制登录；正文走 page.extract-text |
| B站 (bilibili.com) | L3 内置浏览器 + BrowserImport | 视频描述/评论需登录 |
| 微博 (weibo.com) | L3 内置浏览器 + BrowserImport | 长微博需登录 |
| 知乎 (zhihu.com) | L3 内置浏览器 + BrowserImport | 长文+评论需登录 |
| 飞书文档 (feishu.cn) | L3 内置浏览器 + BrowserImport | 必须登录 |
| 公众号 (mp.weixin.qq.com) | L2 Jina Reader | 通常公开，Jina 处理更干净 |
| LinkedIn | L3 CDP | 登录墙 |
---
| LinkedIn | L3 内置浏览器 + BrowserImport | 登录墙 |
## Tool Reference
@@ -194,11 +144,11 @@ WebFetch(url="https://example.com/article")
Tips:
- Results cached for 15 min
- Returns cleaned Markdown with title + URL + body
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (CDP)
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (built-in browser)
### Layer 2: Jina Reader (default for heavy pages)
### Layer 2: Jina Reader（重页默认）
Jina Reader (`r.jina.ai`) is a free public proxy that renders pages server-side and returns clean Markdown. Use it as the **default** for any page where WebFetch produces garbled or truncated output, and as the **preferred** extractor for JS-heavy SPAs.
Jina Reader (`r.jina.ai`) 免费公共代理，服务端渲染页面并返回干净 Markdown。WebFetch 输出乱码/截断时的默认升级路径，JS 重页面的首选抓取器。
```bash
curl -sL "https://r.jina.ai/https://example.com/article"
@@ -212,111 +162,93 @@ Why Jina is the default token-saver:
See [references/jina-reader.md](references/jina-reader.md) for advanced endpoints and rate limits.
### Layer 3: CDP Browser (login-gated access)
### Layer 3: 内置受管浏览器（登录态与交互）
Use Python Playwright's `connect_over_cdp()` to attach to the user's running Chrome (which already has login cookies). **No re-login needed.**
完整命令面、能力档位与边界条件见 [references/browser-tools.md](references/browser-tools.md)。这里的循环是：
**Minimal template**:
```bash
python3 << 'PY'
from playwright.sync_api import sync_playwright
1. `BrowserManage(create_space)` → `BrowserManage(start_session)` 拿 sessionId + 首个 tab
2. 需要登录态 → `BrowserImport`（仅当 Host 已授予 `browser.import.*`；没有这层授权就无法复用用户 Cookie，告诉用户并按无登录态继续或放弃）
3. `BrowserAct(tab.navigate)` 到达页面
4. `BrowserSnapshot(semantic)` 拿可交互元素 `ref`（跨源 iframe 的元素也在同一棵树里，ref 全局连续编号）
5. 交互用 `input.*`（拟真轨迹）或 `page.element`（表单批量写）；等结果用 `page.wait` 或内联 wait 块
6. 取正文用 `BrowserSnapshot(text)` 或 `BrowserAct(page.extract-text)`；取接口数据用 fetch.browser 配方
7. 任务收尾 `BrowserManage(close_session)`
TARGET_URL = "https://www.xiaohongshu.com/explore/..."
多动作连续编排（导航→快照→点击→等待→取文）可用 `BrowserScript` 一段脚本完成，省去逐动作 IPC 往返。
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
context = browser.contexts[0] # reuse user's default context (has cookies)
page = context.new_page()
page.goto(TARGET_URL, wait_until="domcontentloaded")
page.wait_for_timeout(2000) # let lazy content load
html = page.content()
page.close()
## L3 速查（v3.0）
# Print first 500 chars to verify
print(html[:500])
PY
```
**Extract text via BeautifulSoup** (no Jina round-trip):
```bash
python3 << 'PY'
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto("https://www.bilibili.com/video/BV...", wait_until="networkidle")
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
title = soup.select_one("h1.video-title")
desc = soup.select_one(".video-desc")
print("Title:", title.get_text(strip=True) if title else "N/A")
print("Desc:", desc.get_text(strip=True) if desc else "N/A")
PY
```
See [references/cdp-browser.md](references/cdp-browser.md) for:
- Per-site selectors (小红书/B站/微博/知乎/飞书)
- Scrolling & lazy-load patterns
- Screenshot & form-fill recipes
- Troubleshooting connection issues
---
## L3-fast: 内置受管浏览器速查（v2.1）
**只在你调用 `Skill('web-access')` 加载本技能后，下面这组工具才会出现在 tools[] 里。**
| 工具 | 一行示例 |
|------|---------|
| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | 每个任务一个隔离 Space |
| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | 启动会话，返回 sessionId 与首个标签页 |
| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | 标签页与生命周期管理 |
| `BrowserAct({ action: 'tab.navigate', params: { url } })` | 当前标签页导航 |
| `BrowserSnapshot({ mode: 'semantic' })` | 读页面：可交互元素 + `ref` 句柄 |
| `BrowserAct({ action: 'input.click', params: { ref } })` | 按快照 `ref` 点击，不要用裸 x/y |
| `BrowserAct({ action: 'input.text', params: { text } })` | 向聚焦元素输入文本 |
| `BrowserAct({ action: 'input.wheel', params: { x: 640, y: 400, deltaX: 0, deltaY: 720 } })` | 滚动触发懒加载 —— `x`/`y`（或 `ref`）**必填**，且会话要带 `browser.input.pointer.wheel` |
| `BrowserAct({ action: 'tab.activate', params: { bounds } })` → `page.screenshot` | **先 activate 再截图** |
| `BrowserImport({ action: 'discover' \| 'create_plan' \| 'dry_run' \| 'apply' \| 'rollback' \| 'list_plans' })` | 复用用户登录态 Cookie —— 这 6 个就是**全部**动作，主流程走 `discover → create_plan → dry_run → apply`。**需要 `browser.import.*`，而 `create_space` 不会授予**（见下） |
完整 API 与边界条件见 [references/browser-tools.md](references/browser-tools.md)。
**怎么读页面** —— 按需求选通道：
### 读页面：按需求选通道
| 你要什么 | 用什么 | 说明 |
|----------|--------|------|
| 可交互元素 + `ref` 句柄 | `BrowserSnapshot({ mode: 'semantic' })` | 只有按钮/输入框/链接，**不含正文文本** |
| 小页面的正文 | `BrowserSnapshot({ mode: 'accessibility' })` | 返回 StaticText 节点；真实内容页会报 `BROWSER_RESULT_TOO_LARGE`（结果上限 2 MB，且 `depth` 参数当前被宿主忽略） |
| 真实页面的正文 | L2 Jina Reader（公开页）或 L3-fallback Playwright（登录态） | 内置浏览器目前没有可用的批量取文通道 |
| 页面长什么样 | `tab.activate` → `BrowserAct({ action: 'page.screenshot' })` | 只有整页截图，靠看图读 |
| 可交互元素 + `ref` / `loc=` 句柄 | `BrowserSnapshot({ mode: 'semantic' })` | 按钮/输入框/链接 + 每行 `[ref=eN]` 与（能产出时）`[loc=...]` 稳定选择器；跨源 iframe 元素同树 |
| 页面正文 | `BrowserSnapshot({ mode: 'text' })` 或 `BrowserAct({ action: 'page.extract-text' })` | markdown/text 两种格式；超 maxBytes 截断并给 nextCursor 续读（cursor 分页），不报错 |
| 辅助功能树 | `BrowserSnapshot({ mode: 'accessibility' })` | 尊重 `depth`（默认 50 上限 100）与 maxBytes 预算，超出截断+翻页，不再整体报错 |
| 页面长什么样 | `BrowserSnapshot({ mode: 'visual' })` 或 `BrowserAct({ action: 'page.screenshot' })` | 像素直接进结果（视觉模型当场看）；支持 `clip={x,y,width,height,scale}` 元素级裁剪放大（scale 最大 4）与 `captureBeyondViewport` 整页截图 |
`page.evaluate` **不是**取文通道：每次调用需人工审批，且字符串/对象返回值会被替换成 `[REDACTED:browser-runtime-value]`。
### 命令面速览
`BrowserAct` 的 `action` 按用途分组（完整枚举见工具 schema）：
- **tab.***：`navigate` / `back` / `forward` / `reload` / `activate` / `close`
- **input.***：`move` / `click` / `double-click` / `drag` / `wheel` / `touch` / `pinch` / `key` / `text` —— 走输入拟真（#1808：UA/UA-CH 身份一致 + 真实轨迹），反检测站点的交互首选
- **page.element**（判别式 op × selector，九 op）：写类 `fill` / `select-option` / `check` / `uncheck` / `scroll-into-view`；读类 `get-attribute` / `bounding-box` / `count` / `all-inner-texts`。selector 说 `loc=` 方言或快照 `ref`（配 snapshotId）。`fill` 拒绝 `input[type=password]`
- **page.wait**（判别式 until，九种）：轮询型 `load` / `domcontentloaded` / `networkidle` / `selector` / `url` / `timeout` 超时返回 `waited:false`；事件型 `request` / `response` / `download` 超时抛错。默认 10s、上限 60s
- **内联 wait 块**：`tab.navigate` / `input.click` / `input.key` / `page.element{op:"fill"}` 的 `params.wait`（形态与 page.wait 同构）——一条回执完成「动作→等结果」，等待器先于动作注册，无跨 IPC 竞态
- **page.evaluate**：`{ expression, awaitPromise }`，返回值原样过界（超预算截断并标 `truncated`，不抛错）；能力档 `browser.page.evaluate` 走人工闸门（allow-all 模式免卡片）
- **page.extract-text / page.screenshot / page.wait**：见上表与 fetch.browser 配方
`loc=` 选择器方言（S7/S9）：`e<序号>`（须配签发该 ref 的快照 snapshotId）、`loc=css:` / `loc=role:` / `loc=text:` / `loc=testid:`、裸 CSS，可叠 `internal:nth/last/scope/filter`。未知前缀显式报错，绝不静默降级。
### 交互通道选用：input.* vs page.element
- **反检测站点（小红书/微博/B站等）一律优先 `input.*`**：走 #1808 的输入行为拟真（坐标派发、拟真轨迹、可视化可审计），配套身份一致性层让 UA/UA-CH 不带 Electron/Headless 痕迹
- **`page.element` 写类适用于表单批量填充等站点不检测的场景**：一次调用完成 fill/select-option/check，比逐元素 input.click+input.text 快得多
- **红线**：`page.element` 刻意不含 click——指针动作必须走 `input.*`，用 JS 直调 `el.click()` 会绕开全部拟真投入，属于明确禁止的回退
### fetch.browser 配方：带登录态取接口数据
登录态下取站内接口（列表、评论、订单等 JSON）的正解：在**页面上下文**里跑 `fetch`——自动带该 origin 的 Cookie，同 origin、受 Grant origins 约束，走 `page.evaluate` 已有闸门。这是 `BrowserAct({ action: 'page.evaluate' })` 的封装用法：
```yaml
BrowserAct:
action: page.evaluate
params:
expression: |
fetch('/api/v1/comments?page=1&size=20', {
headers: { accept: 'application/json' }
}).then(r => r.text())
awaitPromise: true # 默认 true；表达式返回 Promise 时等它 settle
```
要点：
- 先 `tab.navigate` 到该站任意页面（建立 origin 与 Cookie），再发 fetch；路径写相对路径，天然同 origin
- 返回值原样过界；大 JSON 用 `.text()` 拿原文自己截取，或分页多次取
- 只能访问当前 tab origin（Grant origins 约束）；跨站接口请先导航过去
- `page.evaluate` 属人工闸门能力：非 allow-all 模式会弹审批卡，向用户说明用途即可
### 推荐流程（小红书示例）
```
1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
← 显式列表是做减法；要滚动就必须把 browser.input.pointer.wheel 列进去
3. 复用登录态：默认走 L3-fallback Playwright 连用户自己的 Chrome。
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }) 只有在 Host 另行
授予 browser.import.* 时才走得通 —— 光靠 create_space 拿不到；被拒就直接回落。
← 显式列表是做减法；要滚动就把 browser.input.pointer.wheel 列进去，
要取文就带 browser.observe.snapshot
3. 复用登录态：仅在 Host 已授予 browser.import.* 时走
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' })。
没有这层授权就无法复用用户 Cookie——如实告诉用户，按无登录态继续或放弃。
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
5. BrowserSnapshot({ mode: 'semantic' }) ← 只拿交互用的元素 ref（不含正文）
tab.activate + page.screenshot ← 确认确实渲染出了笔记
→ 正文抽取走 L3-fallback Playwright
6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← 读累积经验
7. 任务结束 → BrowserManage({ action: 'close_session', sessionId })
8. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
5. BrowserSnapshot({ mode: 'semantic' }) ← 拿交互元素 ref（跨源 iframe 同树）
6. BrowserAct({ action: 'page.extract-text', params: { format: 'markdown' } })
← 正文直接读出；太长就传上一页返回的 nextCursor 续读
7. 需要确认渲染效果 → BrowserSnapshot({ mode: 'visual' })（像素直接可看）
8. SitePatternRead({ domain: 'xiaohongshu.com' }) ← 读累积经验
9. 任务结束 → BrowserManage({ action: 'close_session', sessionId })
10. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
```
---
## 站点经验积累（v2.0 新增）
## 站点经验积累
任务结束如果发现新的反爬陷阱、有效选择器、平台特征，调用：
@@ -325,7 +257,7 @@ SitePatternWrite({
domain: "xiaohongshu.com",
scope: "agent", // agent=共享（受 Git 管理，发布给其他用户）；user=私有
mode: "merge", // merge 追加，replace 覆盖
content: "## 已知陷阱\n- 2026-05: ...",
content: "## 已知陷阱\n- 2026-08: ...",
confidence: "medium"
})
```
@@ -341,8 +273,6 @@ SitePatternRead({ domain: "xiaohongshu.com" })
含 cookie / token / 手机号 / 邮箱时 SitePatternWrite **自动降级 scope='user'** 并提示。
---
## Common Workflows
Read [references/workflows.md](references/workflows.md) for detailed templates:
@@ -351,11 +281,9 @@ Read [references/workflows.md](references/workflows.md) for detailed templates:
- 新闻聚合与时间线 (News aggregation)
- API/库版本调查 (Library version investigation)
Read [references/cdp-browser.md](references/cdp-browser.md) for login-gated site recipes (小红书/B站/微博/知乎/飞书).
Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader positioning, rate limits, and advanced endpoints.
---
Read [references/browser-tools.md](references/browser-tools.md) for the full built-in browser command surface, capability tiers, and known boundaries.
## Quick Workflow: Multi-Source Research
@@ -369,27 +297,22 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
7. Report with inline [source](url) citations + a Sources list at the end
```
---
## Anti-Patterns (Avoid)
- ❌ **Using WebFetch on obviously heavy sites** — Medium, Twitter, 小红书 will waste tokens or fail. Jump straight to L2/L3.
- ❌ **Launching headless Chrome instead of CDP attach** — loses user's login state, triggers anti-bot, slow cold start. Always use `connect_over_cdp()` to attach to the user's existing session.
- ❌ **Fetching one URL at a time when you need 5** — batch in a single message.
- ❌ **Trusting a single source** — cross-check ≥ 2 sources for non-trivial claims.
- ❌ **Fetching the search result page itself** — WebSearch already returns snippets; fetch the actual articles.
- ❌ **Ignoring the cache** — WebFetch caches 15 min, reuse freely.
- ❌ **Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs.
- ❌ **Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs; 登录态站内接口走 fetch.browser 配方.
- ❌ **Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current.
- ❌ **Hardcoding login credentials in scripts** — always rely on the user's pre-logged CDP session.
- ❌ **Hardcoding login credentials in scripts** — 登录态只能来自 BrowserImport 导入的 Cookie.
- ❌ **Citing only after the fact** — collect URLs as you fetch, not from memory afterwards.
- ❌ **(v2.1) 在能用内置浏览器时仍写 Python heredoc** — 慢、依赖 Python+Playwright 安装、上下文体积大。优先 L3-fast；只在内置浏览器不够（race / 长等待 / 自定义脚本）时才回退。
- ❌ **(v2.1) 没 `tab.activate` 就直接 `page.screenshot`** — 标签页默认停在窗口外 `(-10000,-10000,1x1)`，没有合成表面，截图会卡满 30 秒 deadline **并把标签页宿主打掉**，之后所有调用都报 `BROWSER_TAB_HOST_NOT_FOUND`。务必先用真实 bounds `tab.activate`。
- ❌ **(v2.1) 用 `page.evaluate` 读页面** — 每次调用都需人工审批，返回的字符串/对象会被治理策略替换为 `[REDACTED:browser-runtime-value]`（只有 number/boolean/null 能穿透），且反调试站点会把它挂起几十秒。读页面请用 `BrowserSnapshot`。
- ❌ **(v2.1) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')`。
- ❌ **(v2.1) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级，但你不该故意往 agent 层写敏感信息。
---
- ❌ **(v3.0) 在反检测站点用 page.element 批量交互** — 它是 JS 直调，绕开拟真轨迹；反检测站点一律 `input.*`，`page.element` 只用于不检测场景的表单批量填充。
- ❌ **(v3.0) 用截图靠"看图读字"取正文** — `page.extract-text` / `BrowserSnapshot(text)` 直接给 markdown/text，带分页预算；截图留给版面确认与验证码这类必须看图的场合。
- ❌ **(v3.0) 逐动作往返还能忍时不换 BrowserScript** — 导航→快照→点击→等待→取文五连用一段脚本完成；但记住 BrowserScript 信任级别等同 Bash，脚本源码要过一次人工审批。
- ❌ **(v3.0) 任务结束发现新陷阱却不写 site-pattern** — 下次同 Agent 再做相同任务会重复踩坑。任何"花了 2+ 步才搞清楚的细节"都值得 `SitePatternWrite(scope='agent', mode='merge')`。
- ❌ **(v3.0) 把含 cookie / 手机号的内容写到 scope='agent'** — 这层会被 Git 提交、可能发布到市场。SitePatternWrite 会自动降级，但你不该故意往 agent 层写敏感信息。
## Example Interaction
@@ -398,26 +321,15 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
**Agent workflow**:
```
1. 识别 → 小红书是 L3 登录态站点
2. 检查 CDP：curl -s http://localhost:9222/json/version
├─ 失败 → 提示用户启动 Chrome 调试模式，终止
└─ 成功 → 继续
3. Bash: python3 connect_over_cdp 脚本 → page.goto(url) → page.content()
4. BeautifulSoup 提取 h1 title、.note-content、.comments
2. BrowserManage(create_space + start_session)
需要登录态 → 已授予 browser.import.* 就走 BrowserImport 四步；
没授予 → 告诉用户无法复用登录态，按公开可见部分继续
3. BrowserAct(tab.navigate → 笔记 URL)
4. BrowserAct(page.extract-text, format: markdown)
← 正文直接读出；超预算就按 nextCursor 续读
5. 返回给用户时：
- 引用原 URL
- 若内容很长，用 Jina 清洗一遍节省 token
6. 告知用户：「已通过你的登录态抓取，原链接：[xhs](url)」
- 引用正文事实，标注来源链接
6. 告知用户：「已通过内置浏览器抓取，原链接：[xhs](url)」
7. BrowserManage(close_session)
```
---
## Installation Note
CDP features require Python + Playwright installed:
```bash
pip3 install playwright beautifulsoup4
python3 -m playwright install chromium # only needed if user hasn't installed Chrome
```
If `playwright` is not installed when the user requests a login-gated site, run the install commands in Bash and explain you're setting up the browser automation dependency.

301
skills/web-access/references/browser-tools.md
View File

@@ -1,26 +1,15 @@
# 内置受管浏览器工具速查（L3-fast）
# 内置受管浏览器工具速查（L3）
> v2.1 起，本层从「cdp-proxy 驱动用户自己的 Chrome」改为「DesireCore 内置受管浏览器」。
> v2.1 起本层从「cdp-proxy 驱动用户自己的 Chrome」改为「DesireCore 内置受管浏览器」；v3.0 补全能力面（批量取文 / 元素操作 / 等待 / 代码模式）并移除 Python Playwright 回落。
> 每个任务跑在独立 BrowserSpace 里（Cookie / Storage / 缓存互不串扰），每个动作都经过
> Capability → Grant → Lease → Origin → Host fencing 校验并留下可审计回执。
>
> 要求客户端 **v10.0.98+**。旧的 `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` /
> `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` /
> `BrowserCloseTab` 与其背后的 cdp-proxy 已停用，调用会直接返回
> 要求客户端 **v10.0.98+**。`page.extract-text` / `page.element` / `page.wait` / 内联 wait 块 /
> 跨源 iframe 快照需 **v10.0.112+**；`BrowserScript` 需包含 S17/S18 的版本。旧的
> `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` /
> `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 与其背后的 cdp-proxy 已停用，调用会直接返回
> 「该旧 BrowserXxx/cdp-proxy 入口已停用；请改用 BrowserManage、BrowserSnapshot 或 BrowserAct」。
## 何时用内置浏览器 vs Python Playwright
| 场景 | 推荐 |
|------|------|
| 到达并操作登录态站点（小红书 / B站 / 微博 / 飞书 / 知乎） | **内置浏览器**（登录态优先走 L3-fallback CDP；仅在已授予 `browser.import.*` 时才用 BrowserImport 导入 Cookie） |
| 抽取登录态站点的长正文 | Python Playwright（内置浏览器没有批量取文通道，见下） |
| 简单点击 / 滚动 / 截图 | **内置浏览器** |
| 多个任务要互不串扰地并行 | **内置浏览器**（一任务一 Space） |
| 需要复杂等待逻辑（wait_for_selector + race condition） | Python Playwright（cdp-browser.md） |
| 需要在浏览器内运行长时间脚本（>30 s） | Python Playwright（内置浏览器单条命令 30 s deadline） |
| 需要对元素做放大裁剪截图 | Python Playwright（内置浏览器只有整页截图） |
## 前置条件
1. 客户端 v10.0.98+，`~/.desirecore/config/browser.json` 里 `electron-embedded` 或
@@ -68,27 +57,52 @@ BrowserManage:
### BrowserSnapshot
读页面的**主通道**。返回可交互元素及其 `ref` 句柄、`rect`、`disabled` 状态。
读页面的**主通道**。四种 mode：
```yaml
BrowserSnapshot:
mode: semantic # semantic（默认）| accessibility | visual
mode: semantic # semantic（默认）| text | accessibility | visual
sessionId: bss_xxx # 当前 Agent 有多个会话时用于消歧
tabId: btab_xxx
tabId: btab_xxx # 非活动标签页时用于消歧
options: {...} # 各 mode 专属参数，见下
```
提示：
- `semantic` 只列 button / input / a 等可交互元素，**不含图片和正文文本节点**
- 元素 `name` 往往是 placeholder（如「请输入」），无标签时只能靠 `rect.y` 排序定位
- 页面重排后旧 `ref` 会失效——**每次交互前重新取快照**
- `accessibility` 能拿到 StaticText 正文，但**只在很简单的页面上可用**：实测
example.com 正常返回，维基百科条目一律 `BROWSER_RESULT_TOO_LARGE`（回执上限 2 MB），
且 schema 里的 `depth` 参数目前被宿主忽略（硬编码 depth=50），调小也没用
- 因此**取真实页面正文仍要靠 L2 Jina Reader（公开页）或 L3-fallback Playwright（登录态）**
**semantic** —— 可交互元素清单（button / input / a 等）+ `ref` 句柄。每行形如
`[ref=e12] [loc=...] 按钮 提交`：
- `loc=` 是该元素的稳定选择器（S7）：仅当能产出时才给出，五级优先级 data-testid →
唯一 CSS 安全 id → name 属性 → 链接 href（剥 query 前缀匹配）→ role+name
- 跨源 iframe（OOPIF）的元素并入同一棵树，ref 全局连续编号（S35）
- 页面重排后旧 `ref` 会失效——**每次交互前重新取快照**；带 ref 的命令要同时给签发该
ref 的快照 `snapshotId`
- `options`：`scope`（viewport=视口内元素优先截断）、`maxElements`（默认 400 上限 2000）、
`maxBytes`（默认 256KB 上限 1MB）、`cursor` 翻页、`limit`
**text** —— 正文批量取文通道之一（S3/S4）：
```yaml
BrowserSnapshot:
mode: text
options:
format: markdown # markdown（默认，保留标题/列表/表格）| text（纯文本）
maxBytes: 65536 # 默认 65536，最大 524288；超出不报错，截断并返回 nextCursor
cursor: t1:...:... # 上一页返回的 nextCursor，同参数续读
scope: full_page # full_page（默认）| viewport | ref（配 ref + snapshotId 取子树）
includeLinks: true # 链接目标写进正文（只写 http/https）
includeTables: true # 表格保留逐行结构
```
**accessibility** —— AX 树（S8 起**尊重 `depth`、预算截断不炸**）：
`options.depth`（默认 50 上限 100）、`maxElements`、`maxBytes`、`cursor`。真实内容页不再
`BROWSER_RESULT_TOO_LARGE` 整体失败——超预算截断给 nextCursor 续读。
**visual** —— 截图（像素直接进结果）：
`options.format`（png/jpeg）、`quality`、`captureBeyondViewport`（整页）、
`clip={x,y,width,height,scale}`（元素级裁剪放大，`scale` 最大 4）。
### BrowserAct
一次调用一个受管动作。完整 `action` 见工具 schema，常用的：
一次调用一个受管动作。`action` 分组：
**tab.*** —— `navigate` / `back` / `forward` / `reload` / `activate` / `close`：
```yaml
BrowserAct:
@@ -96,10 +110,15 @@ BrowserAct:
params: { url: https://www.xiaohongshu.com/explore/... }
```
**input.*** —— `move` / `click` / `double-click` / `drag` / `wheel` / `touch` / `pinch` /
`key` / `text`。走 #1808 输入拟真（坐标派发、拟真轨迹、身份一致性），**反检测站点的交互首选**：
```yaml
BrowserAct:
action: input.click
params: { ref: bref_xxx } # 用快照 ref；坐标会因页面重排失效
params:
ref: e12 # 语义快照签发的短号 ref（不要裸 x/y）
snapshotId: snap-xxx # 签发该 ref 的快照回执里的 snapshotId，必带
```
```yaml
@@ -114,37 +133,120 @@ BrowserAct:
params: { deltaX: 0, deltaY: 720, x: 640, y: 400 }
```
```yaml
BrowserAct:
action: tab.activate # 截图前必须先做这一步
params: { bounds: { x: 0, y: 0, width: 1280, height: 900 } }
```
带元素 ref 的 `input.*` 必须在 params 顶层同时带同一条快照回执的 `snapshotId`——ref 序号跨
快照会重复，缺 snapshotId 一律拒绝执行。
**page.element** —— 判别式元素命令（op × selector，九 op，S10）。**表单批量填充等站点
不检测场景用**；反检测站点交互走 input.*：
```yaml
BrowserAct:
action: page.screenshot
params: { format: png } # 结果落 artifact store，回执给 artifact.id / sha256 / bytes
action: page.element
params:
op: fill # 写类：fill | select-option | check | uncheck | scroll-into-view
# 读类：get-attribute | bounding-box | count | all-inner-texts
selector: loc=css:input#email # loc= 方言或 e12 / ref=e12（配 snapshotId）
value: user@example.com # fill / select-option 用
snapshotId: bsnp_xxx # selector 用 ref 时必带
```
- 写类走 `browser.input.keyboard` 能力档（与 input.text 同档）；读类走 `browser.observe.snapshot` 只读档
- `fill` 对 `input[type=password]` **一律拒绝**
- `selector` 方言：`e<序号>` / `ref=e<序号>`、`loc=css:` / `loc=role:` / `loc=text:` /
`loc=testid:`、裸 CSS，可叠 `internal:nth/last/scope/filter`。未知前缀（如 `loc=xpath:`）显式报错，绝不静默降级
**page.wait** —— 判别式等待（until，九种，S11）。等待不改页面状态，走只读档：
```yaml
BrowserAct:
action: page.wait
params:
until: networkidle # load | domcontentloaded | networkidle | selector | url | timeout
# | request | response | download
timeoutMs: 10000 # 默认 10000，上限 60000
idleMs: 500 # networkidle 静默窗，默认 500
```
- 轮询型（load/domcontentloaded/networkidle/selector/url/timeout）超时返回 `waited: false`，不抛错
- 事件型（request/response/download）超时**抛错**
- `waitForFunction` 刻意不在枚举里——任意 JS 走 page.evaluate 的能力档与审批
**内联 wait 块**（S14）—— `tab.navigate` / `input.click` / `input.key` / `page.element{op:"fill"}`
支持 `params.wait`（形态与 page.wait 参数同构），一条回执完成「动作→等结果」，等待器先于动作注册，
不跨两次 IPC 出竞态：
```yaml
BrowserAct:
action: tab.navigate
params:
url: https://example.com/login
wait: { until: networkidle, timeoutMs: 15000 }
```
**page.evaluate** —— 页面上下文求值（S2 起返回真实值）：
```yaml
BrowserAct:
action: page.evaluate
params:
expression: document.title
awaitPromise: true # 默认 true
```
- 返回值**原样过界**（字符串/对象/数组都真实返回）；超预算截断并标 `truncated`，不抛错
- 能力档 `browser.page.evaluate` 属人工闸门：非 allow-all 模式每次调用弹审批卡
- 登录态取站内接口走 [fetch.browser 配方]（见下节）
**page.extract-text** —— 与 `BrowserSnapshot mode:text` 同款取文通道，作为动作下发：
scope（viewport/full_page/ref）+ ref + snapshotId、format（markdown/text）、maxBytes（默认
65536 最大 524288）、cursor（`^t1:` 续读）、includeLinks、includeTables。超出预算截断 + nextCursor。
**page.screenshot** —— 与 `BrowserSnapshot mode:visual` 同款：`format`、`quality`、
`captureBeyondViewport`、`clip={x,y,width,height,scale}`。结果落 artifact store，回执给
artifact.id / sha256 / bytes；截图像素直接进结果（见「截图」节）。
### BrowserScript（代码模式，S17+S18）
一段异步 JS 在 Worker 里跑，通过注入的 `page` / `tab` / `input` / `snapshot` / `console` /
`performance` 门面连续下发命令——**导航→快照→点击→等待→取文一气呵成**，消除逐动作往返：
```yaml
BrowserScript:
code: |
const snap = await snapshot.semantic();
receipt.log(snap.content);
await input.click({ ref: 'e12', snapshotId: snap.snapshotId });
await page.wait({ until: 'networkidle' });
receipt.log(await page['extract-text']({ format: 'markdown' }));
sessionId: bss_xxx # 可选，多会话消歧
tabId: btab_xxx # 可选，缺省用会话活动 tab
totalBudgetMs: 180000 # 默认 180000，上限 600000；审批等待不计入
maxCommands: 500 # 每次运行命令条数上限
```
- ❗**信任级别等同 Bash**：Worker 不是沙箱，脚本可访问 Node fs/net/child_process。两层审批：
脚本源码先过一次与 Bash 同档的人类审批（allow-all / ask-external 豁免；脚本源码不可记忆），
其内每个受管浏览器能力再按「能力 × 本次运行」各问一次
- helper 白名单 24 个：`snapshot.*` / `page.extract-text` / `tab.*` / `input.*` /
`page.evaluate` / `page.wait` / `page.element` / `console.read` / `performance.metrics`；
`console.log(page)` 可输出各 helper 的 signature/params/example 文档
- 会话被用户接管、Lease 轮换 ⇒ **硬停**：后续命令一律不再下发
- 需要 `browser.*` 相应能力；`receipt.log(...)` 或 `console.log` 输出文本
### BrowserImport（**需人工审批 + 额外授权**）
把用户浏览器里的登录态 Cookie 导入当前 Space。
把用户浏览器里的登录态 Cookie 导入当前 Space——**唯一的登录态复用通道**。
> **前置条件（先看这里，别直接试）**：`browser.import.discover` /
> `browser.import.cookies.inspect` / `browser.import.cookies` 三个能力**不在 `agentDefault` 里**，
> 而 `BrowserManage(create_space)` 建的 Agent grant 就是按 `agentDefault` 签的。也就是说
> **仅靠 create_space / start_session 走不通 BrowserImport**，必须由 Host/用户侧另行授予 import
> 能力（`agentElevated` 或 Workbench 路径）。没有这层授权就别在这条路上耗——**直接回落
> L3-fallback（Python Playwright 连用户已登录的 Chrome）**，那是当前更稳的登录态复用方式。
> 能力（`agentElevated` 或 Workbench 路径）。没有这层授权就别在这条路上耗——如实告诉用户
> 「当前无法复用你的登录态」，按无登录态继续或放弃。
动作枚举只有这 6 个：`discover` | `create_plan` | `dry_run` | `apply` | `rollback` | `list_plans`
（**没有 `plan`**）。完整流程是 `discover → create_plan → dry_run → apply`：
```yaml
BrowserImport:
action: discover # 列出可导入的来源，返回不透明 sourceProfileId
```
```yaml
BrowserImport:
action: create_plan # 域名授权、来源、冲突策略都在这一步定死
@@ -155,57 +257,85 @@ BrowserImport:
conflictStrategy: newer-wins # keep-target | replace-target | newer-wins | fail-on-conflict
```
```yaml
BrowserImport:
action: dry_run # 先看命中多少条，再决定要不要真的写入
planId: bimp_xxx # create_plan 返回的 ID，前缀是 bimp_
```
```yaml
BrowserImport:
action: apply # 只认 planId；域名/策略在 create_plan 时已固化
planId: bimp_xxx
```
写坏了用 `action: rollback` + 同一个 `planId` 回退；`list_plans` 查当前 Space 的历史 plan。
解密与过滤全在 Host 侧完成，**Cookie 值不会进入 Agent 上下文或审计日志**。
`dry_run` 先看命中多少条再 `apply`（只认 `create_plan` 返回的 `planId`，前缀 `bimp_`）；写坏了
用 `rollback` + 同一个 planId 回退；`list_plans` 查历史。解密与过滤全在 Host 侧完成，
**Cookie 值不会进入 Agent 上下文或审计日志**。
### BrowserShare
把 Space / Session 委派给另一个 Agent，`shareMode` 可选 `snapshot`（只读副本）、
`copy-on-write`（写时复制）、`live-shared`（实时共享）、`handoff`（移交控制权）。
## fetch.browser 配方：带登录态取接口数据
站内接口（列表、评论、订单等 JSON）在登录态下的正解——**页面上下文跑 `fetch`**，自动带该
origin 的 Cookie，同 origin、受 Grant origins 约束，走 `page.evaluate` 已有闸门：
```yaml
BrowserAct:
action: page.evaluate
params:
expression: |
fetch('/api/v1/comments?page=1&size=20', {
headers: { accept: 'application/json' }
}).then(r => r.text())
awaitPromise: true
```
- 先 `tab.navigate` 到该站任意页面建立 origin 与 Cookie，再发 fetch；路径写**相对路径**
- 只能访问当前 tab origin；跨站接口先导航过去
- 大 JSON 用 `.text()` 拿原文自己截取，或分页多次取
- 非 allow-all 模式会弹审批卡（page.evaluate 人工闸门），向用户说明用途即可
## 交互通道选用：input.* vs page.element（反检测决策）
| 场景 | 用什么 | 原因 |
|------|--------|------|
| 反检测站点（小红书/微博/B站等）的一切点击/输入 | **`input.*`** | 坐标派发 + #1808 拟真轨迹 + 身份一致性（UA/UA-CH 无 Electron/Headless 痕迹） |
| 表单批量填充等**站点不检测**的场景 | **`page.element` 写类** | 一条命令完成 fill/select-option/check，比逐元素 input.* 快得多 |
| 需要元素属性/坐标/计数/批量文本 | `page.element` 读类 | 只读档（browser.observe.snapshot），无写审批 |
| 任何「用 JS 直调 el.click()」的想法 | ❌ 禁止 | 绕开全部拟真投入（S10 红线）；指针动作只走 input.* |
## 截图：像素直接给你，通常不需要再 Read
> **需要客户端 10.0.100+。** 更老的版本上截图只回元数据；
> 若需要画面信息请改用 `semantic` / `accessibility` 快照，或回落 Jina / Playwright。
`BrowserSnapshot mode:visual` 与 `BrowserAct page.screenshot` 会把截图像素作为 image 块
**直接放进工具结果**——视觉模型当场就能看，不必再调 `Read`。
`BrowserSnapshot` `mode: visual` 与 `BrowserAct` `page.screenshot` 会把截图像素作为
image 块**直接放进工具结果**——视觉模型当场就能看，不必再调 `Read`。
只有这几种情况才需要走 `result.artifact.absolutePath` 再 `Read` 一次：
- 结果里明确写了「截图已保存，但…未附带像素」（超出单图预算、或体积过大）
- 你需要的是**原始分辨率**，而附带的像素被压缩过
对同一张图既看 image 块又 `Read` 一遍，只会让它在上下文里占两份。
- **元素级裁剪放大**：`options.clip={x,y,width,height,scale}`（`scale` 最大 4）。先用
`semantic` 快照拿到元素坐标，再截那一块并放大——验证码、小按钮在整页截图里只有几十像素，
看不清时用它。**不必走 `cdp.raw`**
- **整页截图**：`captureBeyondViewport: true`（超出视口部分也截）
- 只有结果里明确写了「截图已保存，但…未附带像素」（超预算/体积过大）或需要原始分辨率时，
才走 `result.artifact.absolutePath` 再 `Read` 一次
当前模型不支持视觉输入时，结果会明说「你看不到它的内容」——**此时不要凭空描述画面**，
改用 `semantic` / `accessibility` 快照拿页面信息。
改用 `semantic` / `text` 快照拿页面信息。
## 视口与 tab.activate（S36 起）
Agent 会话（actor ≠ user）的标签页**常驻离屏原位，保住合成表面**，不再停放成 1×1：
| 场景 | 行为 |
|------|------|
| embedded，单标签会话 | 无需 `tab.activate` 即可截图/取视口（presentation 即呈现） |
| embedded，多标签会话的后台 tab | 仍需先 `tab.activate`；未激活的视口快判**秒级**报 `BROWSER_VIEWPORT_UNAVAILABLE`，不再挂满 30s，也不会毁掉标签页 |
| standalone | 后台 tab 可直接截图 |
多 Space 依然可以并发导航/快照/输入，互不串扰。
## 已知边界（照做，别试探）
| 边界 | 说明 |
|------|------|
| **截图前必须 `tab.activate`** | 标签页默认停在 `(-10000,-10000,1x1)`，没有合成表面。直接截图会卡满 30 s deadline，**并把标签页宿主打掉**，之后全部报 `BROWSER_TAB_HOST_NOT_FOUND` |
| **同时只有一个可见标签页** | `tab.activate` 绑定主窗口、全局互斥。多 Space 可以并发导航/快照/输入，但截图必须逐个 activate 串行 |
| **`page.evaluate` 基本不可用** | 每次调用需人工审批；返回的字符串/对象被替换为 `[REDACTED:browser-runtime-value]`（只有 number/boolean/null 穿透）；反调试站点会把它挂起几十秒。读页面用 `BrowserSnapshot` |
| **`cdp.raw` 需人工审批** | `browser.raw_cdp.*` 属于永远人工闸门的能力，无人值守流程用不了。**元素级裁剪不必走它**（需 10.0.100+）——`BrowserSnapshot` 的 `options.clip={x,y,width,height,scale}` 直接支持（`scale` 最大 4）：先用 `semantic` 快照拿到元素坐标，再截那一块并放大。验证码、小按钮在整页截图里只有几十像素，看不清时用它 |
| **没有批量取文通道** | `semantic` 不含正文，`accessibility` 在真实页面上超限，`page.evaluate` 被审批+脱敏。要正文请回落 Jina / Playwright |
| **单条命令 30 s deadline** | 超时即判 `browser.host.gone`，会话作废 |
| **用户真实操作会抢控制权** | 在可见标签页上**点击、按键、滚轮、触摸**会触发 `trusted-user-input` 并递增 control epoch，打断 Agent 的租约。**鼠标只是划过不会**（默认策略 `intentional-input`，需 10.0.100+；更老版本上划过也会打断），所以用户看着页面移动光标是安全的 |
| **artifact 不要走 `/save`** | 该接口会弹系统「另存为」对话框等人点。**用回执里的 `result.artifact.absolutePath`**（需 10.0.100+；更老版本没有这个字段，只能放弃直接读，改用页面内提取）（工具会把它登记进本次会话的可读白名单），不要自己拼路径——`${DESIRECORE_ROOT}` 这类变量在路径展开里不认（只认 `~` / `$HOME` / `$USERPROFILE`），拼出来是相对路径、`Read` 会报「文件不存在」。默认保留 24 小时 |
| **`page.evaluate` 走人工闸门** | 能力 `browser.page.evaluate` 属 always-human-gate：非 allow-all 模式每次弹审批卡。返回值已原样过界（超预算截断标 `truncated`），登录态取接口走 fetch.browser 配方 |
| **`cdp.raw` 需人工审批** | `browser.raw_cdp.*` 属于永远人工闸门的能力，无人值守流程用不了。元素级裁剪 / 整页截图用 `clip` / `captureBeyondViewport`，别走它 |
| **`page.element` 没有 click** | 指针动作必须走 `input.*`（拟真轨迹、可审计）；JS 直调 `el.click()` 是明确禁止的回退 |
| **`fill` 拒绝密码框** | `input[type=password]` 一律拒绝——密码输入只走 `input.text`（拟真键入） |
| **单条命令 30 s deadline** | 超时只掐掉**这一条命令**（stop 加载），标签页仍可用，重试即可；不再打掉宿主。等待类命令超时连 stop 都跳过 |
| **多标签后台 tab 先 activate** | embedded 多标签会话里后台 tab 截图前先 `tab.activate`，否则秒级 `BROWSER_VIEWPORT_UNAVAILABLE` |
| **用户真实操作会抢控制权** | 在可见标签页上**点击、按键、滚轮、触摸**会触发 `trusted-user-input` 并递增 control epoch，打断 Agent 的租约。**鼠标只是划过不会**（默认策略 `intentional-input`），所以用户看着页面移动光标是安全的 |
| **BrowserScript 信任级别等同 Bash** | 脚本可访问 Node fs/net/child_process；源码过一次与 Bash 同档审批 + 每能力一问；接管/Lease 轮换即硬停 |
| **artifact 不要走 `/save`** | 该接口会弹系统「另存为」对话框等人点。**用回执里的 `result.artifact.absolutePath`**（工具会把它登记进本次会话的可读白名单），不要自己拼路径——`${DESIRECORE_ROOT}` 这类变量在路径展开里不认（只认 `~` / `$HOME` / `$USERPROFILE`），拼出来是相对路径、`Read` 会报「文件不存在」。默认保留 24 小时 |
## SitePatternRead / SitePatternWrite
@@ -228,17 +358,18 @@ SitePatternWrite:
| 错误 | 原因 | 解决 |
|------|------|------|
| `该旧 BrowserXxx/cdp-proxy 入口已停用` | 还在调 v2.0 的旧工具 | 改用 BrowserManage / BrowserSnapshot / BrowserAct |
| `BROWSER_TAB_HOST_NOT_FOUND` | 标签页宿主已销毁（多因上一条命令超时） | 重建 Session；检查是否漏了 `tab.activate` |
| `BROWSER_COMMAND_DEADLINE_EXCEEDED` | 单条命令超 30 s | 截图先 activate；避免 `page.evaluate` |
| `BROWSER_TAB_HOST_NOT_FOUND` | 标签页宿主真的没了（渲染进程崩溃/销毁） | 重建 Session。命令超时**不再**引发此错误——超时只 stop 这一条命令，标签页保留 |
| `BROWSER_COMMAND_DEADLINE_EXCEEDED` | 单条命令超 30 s | 标签页仍停在上一页/空白页，直接重试；慢加载常态不算故障 |
| `BROWSER_VIEWPORT_UNAVAILABLE` | embedded 多标签会话的后台 tab 没有可用视口 | 先 `tab.activate` 再截图（秒级快判，不是挂死） |
| `BROWSER_TOOL_SESSION_FORBIDDEN` | 会话已关闭/崩溃，或不属于当前 Agent | 重新 `list_sessions`，必要时重建 |
| `BROWSER_TOOL_SESSION_AMBIGUOUS` | 当前 Agent 有多个会话且未传 sessionId | 显式传 `sessionId` |
| `BROWSER_HUMAN_APPROVAL_REQUIRED` | 触到人工闸门能力（evaluate / raw_cdp / 上传下载 / Cookie 导入等） | 向用户说明用途并等待审批，或换用无需审批的路径 |
| `BROWSER_RESULT_TOO_LARGE` | 回执超过 2 MB（`accessibility` 快照最常见） | 改用 `semantic` 快照 + 截图；取正文回落 Jina / Playwright |
| `BROWSER_HUMAN_APPROVAL_REQUIRED` | 触到人工闸门能力（evaluate / raw_cdp / 上传下载 / Cookie 导入等） | 向用户说明用途并等待审批，或换用无需审批的路径（取文用 extract-text，裁剪用 clip） |
| `BROWSER_RESULT_TOO_LARGE` | 回执超过结果上限 | text/semantic/accessibility 都有 maxBytes + cursor 分页，调小 maxBytes 续读即可，不再整体失败 |
## 调用链路
```
Agent → BrowserManage/Snapshot/Act → browser-use service（Capability/Grant/Lease/Policy 校验）
Agent → BrowserManage/Snapshot/Act/Script → browser-use service（Capability/Grant/Lease/Policy 校验）
→ BrowserHost（electron-embedded 或 standalone-managed）→ Chromium
↑ 每步产出带 digest 的回执，写入审计事件流
```

330
skills/web-access/references/cdp-browser.md
View File

@@ -1,330 +0,0 @@
# CDP Browser Access — Login-Gated Sites Manual
Detailed recipes for accessing sites that require the user's login session, via Chrome DevTools Protocol (CDP) + Python Playwright.
**Precondition**: Chrome is already running with `--remote-debugging-port=9222` and the user has manually logged in to the target sites. See the main SKILL.md `Prerequisites` section for the launch command.
---
## Why CDP attach, not headless
| Approach | Login state | Anti-bot | Speed | Cost |
|----------|-------------|----------|-------|------|
| Headless Playwright (new context) | ❌ Empty cookies | ❌ Flagged as bot | Slow cold start | Re-login pain |
| `playwright.chromium.launch(headless=False)` | ❌ Fresh profile | ⚠ Sometimes flagged | Slow | Same |
| **CDP attach (`connect_over_cdp`)** | ✅ User's real cookies | ✅ Looks human | Instant | Zero friction |
**Rule**: For any login-gated site, always attach to the user's running Chrome.
---
## Core Template
Every CDP script follows this shape:
```python
from playwright.sync_api import sync_playwright
def fetch_with_cdp(url: str, wait_selector: str | None = None) -> str:
"""Attach to user's Chrome via CDP, fetch URL, return HTML."""
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
# browser.contexts[0] is the user's default context (with cookies)
context = browser.contexts[0]
page = context.new_page()
try:
page.goto(url, wait_until="domcontentloaded", timeout=30000)
if wait_selector:
page.wait_for_selector(wait_selector, timeout=10000)
else:
page.wait_for_timeout(2000) # generic settle
return page.content()
finally:
page.close()
# DO NOT call browser.close() — that would close the user's Chrome!
if __name__ == "__main__":
html = fetch_with_cdp("https://example.com")
print(html[:1000])
```
**Critical**: Never call `browser.close()` when using CDP attach — you'd kill the user's Chrome. Only close the page you opened.
---
## Site Recipes
### 小红书 (xiaohongshu.com)
```python
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
NOTE_URL = "https://www.xiaohongshu.com/explore/XXXXXXXX"
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto(NOTE_URL, wait_until="domcontentloaded")
page.wait_for_selector("#detail-title", timeout=10000)
page.wait_for_timeout(1500) # let images/comments load
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
title = (soup.select_one("#detail-title") or {}).get_text(strip=True) if soup.select_one("#detail-title") else None
desc = (soup.select_one("#detail-desc") or {}).get_text(" ", strip=True) if soup.select_one("#detail-desc") else None
author = soup.select_one(".author-wrapper .username")
print("Title:", title)
print("Author:", author.get_text(strip=True) if author else None)
print("Desc:", desc)
```
**Selectors** (may drift over time — update if they fail):
- Title: `#detail-title`
- Description: `#detail-desc`
- Author: `.author-wrapper .username`
- Images: `.swiper-slide img`
- Comments: `.parent-comment .content`
### B站 (bilibili.com)
```python
from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup
VIDEO_URL = "https://www.bilibili.com/video/BVxxxxxxxxx"
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto(VIDEO_URL, wait_until="networkidle")
page.wait_for_timeout(2000)
html = page.content()
page.close()
soup = BeautifulSoup(html, "html.parser")
print("Title:", soup.select_one("h1.video-title").get_text(strip=True) if soup.select_one("h1.video-title") else None)
print("UP:", soup.select_one(".up-name").get_text(strip=True) if soup.select_one(".up-name") else None)
print("Desc:", soup.select_one(".desc-info-text").get_text(" ", strip=True) if soup.select_one(".desc-info-text") else None)
```
**Tip**: For B站 evaluations, the [公开 API](https://api.bilibili.com/x/web-interface/view?bvid=XXXX) often returns JSON without needing CDP. Try it first:
```bash
curl -s "https://api.bilibili.com/x/web-interface/view?bvid=BVxxxxxxxxx" | python3 -m json.tool
```
### 微博 (weibo.com)
```python
WEIBO_URL = "https://weibo.com/u/1234567890" # or /detail/xxx
# Same CDP template
# Selectors:
# .Feed_body_3R0rO .detail_wbtext_4CRf9 — post text
# .ALink_default_2ibt1 — user link
# article[aria-label="微博"] — each feed item
```
**Note**: Weibo uses React + heavy obfuscation. Selectors change frequently. If selectors fail, pipe the HTML through Jina for clean Markdown:
```python
html = fetch_with_cdp(WEIBO_URL)
# Save to temp file, then:
import subprocess
result = subprocess.run(
["curl", "-sL", f"https://r.jina.ai/{WEIBO_URL}"],
capture_output=True, text=True
)
print(result.stdout)
```
### 知乎 (zhihu.com)
```python
ANSWER_URL = "https://www.zhihu.com/question/123/answer/456"
# Selectors:
# h1.QuestionHeader-title — question title
# .RichContent-inner — answer body
# .AuthorInfo-name — author
```
Zhihu works with CDP but often also renders enough metadata server-side for Jina to work:
```bash
curl -sL "https://r.jina.ai/https://www.zhihu.com/question/123/answer/456"
```
Try Jina first, fall back to CDP if content is truncated.
### 飞书文档 (feishu.cn / larksuite.com)
```python
DOC_URL = "https://xxx.feishu.cn/docs/xxx"
# Feishu uses heavy virtualization — must scroll to load all content.
# Recipe:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp("http://localhost:9222")
page = browser.contexts[0].new_page()
page.goto(DOC_URL, wait_until="domcontentloaded")
page.wait_for_selector(".docs-render-unit", timeout=15000)
# Scroll to bottom repeatedly to load lazy content
last_height = 0
for _ in range(20):
page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
page.wait_for_timeout(800)
h = page.evaluate("document.body.scrollHeight")
if h == last_height:
break
last_height = h
# Extract text
text = page.evaluate("() => document.body.innerText")
page.close()
print(text)
```
### Twitter / X
```python
TWEET_URL = "https://x.com/username/status/1234567890"
# Selectors:
# article[data-testid="tweet"] — tweet container
# div[data-testid="tweetText"] — tweet text
# div[data-testid="User-Name"] — author
# a[href$="/analytics"] — view count anchor (next sibling has stats)
```
Twitter is aggressive with anti-bot. CDP attach usually works, but set a generous wait:
```python
page.goto(url, wait_until="networkidle", timeout=45000)
page.wait_for_selector('article[data-testid="tweet"]', timeout=15000)
```
---
## Common Patterns
### Pattern 1: Scroll to load lazy content
```python
def scroll_to_bottom(page, max_steps=30, pause_ms=800):
last = 0
for _ in range(max_steps):
page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
page.wait_for_timeout(pause_ms)
h = page.evaluate("document.body.scrollHeight")
if h == last:
return
last = h
```
### Pattern 2: Screenshot a specific element
```python
element = page.locator("article").first
element.screenshot(path="/tmp/article.png")
```
### Pattern 3: Extract structured data via JavaScript
```python
data = page.evaluate("""() => {
const items = document.querySelectorAll('.list-item');
return Array.from(items).map(el => ({
title: el.querySelector('.title')?.innerText,
url: el.querySelector('a')?.href,
}));
}""")
print(data)
```
### Pattern 4: Fill a form and click
```python
page.fill("input[name=q]", "search query")
page.click("button[type=submit]")
page.wait_for_load_state("networkidle")
```
### Pattern 5: Clean HTML via Jina after extraction
When selectors are unreliable, dump the full page HTML and let Jina do the cleaning:
```python
html = page.content()
# Save to file, serve via local HTTP, or just pipe the original URL:
import subprocess
clean_md = subprocess.run(
["curl", "-sL", f"https://r.jina.ai/{url}"],
capture_output=True, text=True
).stdout
print(clean_md)
```
---
## Troubleshooting
### `connect_over_cdp` fails with `ECONNREFUSED`
Chrome is not running with remote debugging. Tell the user:
> "请先用下面的命令启动 Chrome：
> `/Applications/Google\\ Chrome.app/Contents/MacOS/Google\\ Chrome --remote-debugging-port=9222 --user-data-dir=\"${DESIRECORE_ROOT}/chrome-profile\"`
> 然后手动登录需要抓取的网站，再让我继续。"
### `browser.contexts[0]` is empty
Chrome was launched but no windows are open. Ask the user to open at least one tab and navigate anywhere.
### Playwright not installed
```bash
pip3 install playwright beautifulsoup4
# No need for `playwright install` — we're attaching to existing Chrome, not downloading a new browser
```
### Site detects automation
Despite CDP attach, some sites (Cloudflare-protected, Instagram) may still detect automation. Options:
1. Use Jina Reader instead (`curl -sL https://r.jina.ai/<url>`) — often succeeds where Playwright fails
2. Ask the user to manually copy the visible content
3. Use the site's public API if available
### Content is truncated
The page uses virtualization or lazy loading. Apply Pattern 1 (scroll to bottom) before calling `page.content()`.
### `page.wait_for_selector` times out
The selector is stale — the site updated its DOM. Dump `page.content()[:5000]` and inspect manually, or fall back to Jina Reader.
---
## Security Notes
- **Never log or print cookies** from `context.cookies()` even during debugging
- **Never extract and store** the user's session tokens to files
- **Never use the CDP session** to perform writes (post, comment, like) unless the user explicitly requested it
- The `${DESIRECORE_ROOT}/chrome-profile` directory contains the user's credentials — treat it as sensitive
- If the user asks to "log in automatically", refuse and explain they must log in manually in the Chrome window; the skill only reads already-authenticated sessions
---
## When NOT to use CDP
- **Public static sites** → use L1 `WebFetch`, it's faster
- **Heavy SPAs without login walls** → use L2 Jina Reader, it's cheaper on tokens
- **You need thousands of pages** → CDP is not built for scale; look into proper scrapers
CDP is specifically the "right tool" for: **small number of pages + login required + human-like behavior needed**.

15
skills/web-access/references/jina-reader.md
View File

@@ -95,19 +95,8 @@ curl -sL "https://r.jina.ai/$URL" > /tmp/jina-cache.md
### Handle very long articles
Jina returns the full article in one response. For articles > 50K chars, pipe through `head` or extract specific sections with Python/awk before feeding back to the model context.
### Combine with CDP
When you use L3 CDP to fetch a login-gated page, you can pipe the resulting HTML through Jina for clean Markdown instead of parsing with BeautifulSoup:
```python
html = fetch_with_cdp(url) # from references/cdp-browser.md
# Now convert via Jina (note: Jina fetches the URL itself, not your HTML)
# So this only works if the content is already visible without login:
import subprocess
md = subprocess.run(["curl", "-sL", f"https://r.jina.ai/{url}"],
capture_output=True, text=True).stdout
```
For truly login-gated content, you must parse the HTML directly (BeautifulSoup) since Jina can't log in on your behalf.
### Combine with the built-in browser
For login-gated pages, the built-in browser already has its own extraction channel with paged budgets — `BrowserAct({ action: 'page.extract-text', params: { format: 'markdown', maxBytes: 65536 } })` returns clean Markdown directly (beyond maxBytes it truncates and hands back a nextCursor). Jina can't log in on your behalf, so use Jina for public pages and `page.extract-text` for logged-in ones; there is no need to round-trip HTML through Jina anymore.
---