mirror of
https://git.openapi.site/https://github.com/desirecore/market.git
synced 2026-09-05 20:03:43 +08:00
feat(web-access): v3.0 内置浏览器能力面同步——删除 Python Playwright 回落(S31) (#85)
## 变更内容 / What
浏览器升级计划 S31 最终验收信号:web-access 技能与内置受管浏览器新能力面对齐,**删除「用户手工启动调试 Chrome +
Python Playwright」回落路径**。
- **删除回落路径**:Prerequisites: Chrome CDP Setup、Layer 3 CDP Browser(Python
Playwright 模板)、Installation Note(pip install
playwright)整段删除;references/cdp-browser.md 文件删除;jina-reader.md 的 CDP 引用改为
page.extract-text
- **订正陈旧断言**:
- 「没有批量取文通道」→ BrowserSnapshot mode:text /
page.extract-text(maxBytes/cursor 分页,超出截断给 nextCursor)
- 「page.evaluate 基本不可用」→ 返回真实值(expression/awaitPromise,超预算截断标
truncated);仍走人工闸门
- 「截图前必须 tab.activate / 串行截图 / BROWSER_TAB_HOST_NOT_FOUND」→ S36 订正:Agent
单标签会话免 activate;多标签后台 tab 秒级报 BROWSER_VIEWPORT_UNAVAILABLE;命令超时只 stop 不
close,标签页可重试
- 「accessibility 超限即失败」→ 尊重 depth + maxBytes 截断翻页(S8)
- 「只有整页截图」→ clip{x,y,width,height,scale≤4} + captureBeyondViewport
- **provides.tools 加 BrowserScript**(code-mode;信任级别等同 Bash)
- **新增选用规则(D4 唯一约束机制)**:反检测站点一律优先 input.*(#1808 输入拟真 +
身份一致性);page.element 写类仅用于表单批量填充等站点不检测场景;JS 直调 el.click() 为禁止回退
- **新增 fetch.browser 配方**:page.evaluate 页面上下文跑 fetch(带 origin Cookie、同
origin、受 Grant origins 约束)——登录态取站内接口的正解
- **能力速查**:page.element 九 op / page.wait 九 until / inline wait 块 / loc=
方言 / BrowserScript / 跨源 iframe 快照(S35)
- **版本** 2.2.1 → 3.0.0(删除回落层为 breaking);source_hash
重算;required_client_version 维持 10.0.98(新能力在正文标注 10.0.112+)
## Why
v2.x 时代回落路径存在的每一条理由(无批量取文、evaluate 不可用、截图必须串行 activate)均已被
S2–S14/S35/S36 覆盖;文档继续引导用户手工起调试 Chrome 会误导新 Agent 走已废弃路径。
双语同步修改(SKILL.md / SKILL.zh-CN.md heading 数一致,i18n-validate 通过)。
- [x] CLA
This commit is contained in:
@@ -5,18 +5,17 @@ description: >-
|
||||
— searching for current information, fetching public web pages, browsing
|
||||
login-gated sites (微博/小红书/B站/飞书/Twitter), comparing products,
|
||||
researching topics, gathering documentation, or summarizing news.
|
||||
This skill orchestrates four complementary layers: (1) WebSearch + WebFetch
|
||||
This skill orchestrates three complementary layers: (1) WebSearch + WebFetch
|
||||
for public pages, (2) Jina Reader as the default token-optimization layer for
|
||||
heavy/JS-rendered pages, (3) the governed built-in browser (isolated
|
||||
BrowserSpace + Cookie import) to reach and interact with login-gated sites,
|
||||
and (4) Chrome DevTools Protocol via Python Playwright as the fallback for
|
||||
bulk text extraction and complex automation. Always cite source URLs.
|
||||
heavy/JS-rendered pages, and (3) the governed built-in browser (isolated
|
||||
BrowserSpace + Cookie import + bulk text extraction + waits + code mode) to
|
||||
reach, interact with, and read login-gated sites. Always cite source URLs.
|
||||
Use when 用户提到 联网搜索、上网查、
|
||||
查资料、抓取网页、研究、调研、最新资讯、文档查询、对比、竞品、技术文档、
|
||||
新闻、网址、URL、找一下、搜一下、查一下、小红书、B站、微博、飞书、Twitter、
|
||||
推特、X、知乎、公众号、已登录、登录状态。
|
||||
license: Complete terms in LICENSE.txt
|
||||
version: 2.2.1
|
||||
version: 3.0.0
|
||||
type: procedural
|
||||
risk_level: low
|
||||
status: enabled
|
||||
@@ -27,13 +26,14 @@ tags:
|
||||
- fetch
|
||||
- research
|
||||
- browsing
|
||||
- browser
|
||||
- cdp
|
||||
- playwright
|
||||
provides:
|
||||
tools:
|
||||
- BrowserManage
|
||||
- BrowserSnapshot
|
||||
- BrowserAct
|
||||
- BrowserScript
|
||||
- BrowserImport
|
||||
- BrowserShare
|
||||
- SitePatternRead
|
||||
@@ -41,7 +41,7 @@ provides:
|
||||
- LocalBookmarks
|
||||
metadata:
|
||||
author: desirecore
|
||||
updated_at: '2026-08-06'
|
||||
updated_at: '2026-08-17'
|
||||
i18n:
|
||||
default_locale: en-US
|
||||
source_locale: zh-CN
|
||||
@@ -50,17 +50,17 @@ metadata:
|
||||
- en-US
|
||||
zh-CN:
|
||||
name: 联网访问
|
||||
short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问、研究调研工作流
|
||||
description: 四层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器登录态访问、Python Playwright CDP 兜底。
|
||||
short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问与取文、研究调研工作流
|
||||
description: 三层联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问、交互与取文。
|
||||
body: ./SKILL.zh-CN.md
|
||||
source_hash: sha256:ce46e1982de8cd65
|
||||
source_hash: sha256:20c98f047378220a
|
||||
translated_by: human
|
||||
en-US:
|
||||
name: Web Access
|
||||
short_desc: Web search, page fetching, logged-in access via the governed built-in browser, research workflows
|
||||
description: A four-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, reach logged-in sites through the governed built-in browser, and fall back to Chrome CDP.
|
||||
description: A three-layer web-access toolkit — search public pages, fetch heavy pages via Jina Reader, and reach, interact with, and read logged-in sites through the governed built-in browser.
|
||||
body: ./SKILL.md
|
||||
source_hash: sha256:ce46e1982de8cd65
|
||||
source_hash: sha256:20c98f047378220a
|
||||
translated_by: human
|
||||
market:
|
||||
icon: >-
|
||||
@@ -73,9 +73,8 @@ market:
|
||||
stroke="url(#wa-a)" stroke-width="1"
|
||||
stroke-opacity="0.35"/><path d="M2 10h16" stroke="url(#wa-a)"
|
||||
stroke-width="1" stroke-opacity="0.35"/><path d="M10 2v16"
|
||||
stroke="url(#wa-a)" stroke-width="1"
|
||||
stroke-opacity="0.35"/><circle cx="18.5" cy="18.5" r="2.5"
|
||||
stroke="#34C759" stroke-width="1.5" fill="#34C759"
|
||||
stroke="url(#wa-a)" stroke-width="1" stroke-opacity="0.35"/><circle cx="18.5"
|
||||
cy="18.5" r="2.5" stroke="#34C759" stroke-width="1.5" fill="#34C759"
|
||||
fill-opacity="0.12"/><path d="M20.5 20.5l2 2" stroke="#34C759"
|
||||
stroke-width="1.8" stroke-linecap="round"/></svg>
|
||||
category: research
|
||||
@@ -90,28 +89,30 @@ market:
|
||||
|
||||
## L0: One-line Summary
|
||||
|
||||
A four-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, reach login-gated sites through the governed built-in browser, and fall back to Chrome CDP.
|
||||
A three-layer web-access toolkit — search public pages, optimize fetches via Jina Reader, and reach, interact with, and read login-gated sites through the governed built-in browser (v3.0 ships bulk text extraction, waits, and code mode in-browser; the Python Playwright fallback is gone).
|
||||
|
||||
## L1: Overview & Use Cases
|
||||
|
||||
### Capability
|
||||
|
||||
web-access is a **procedural skill** that provides four complementary layers of web access:
|
||||
web-access is a **procedural skill** that provides three complementary layers of web access:
|
||||
|
||||
- **L1** (WebSearch + WebFetch): public, static pages
|
||||
- **L2** (Jina Reader): JS-rendered heavy pages, saving tokens by default
|
||||
- **L3-fast** (governed built-in browser — **rewritten in v2.1**): reach and *interact with* logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. **Reading long article text is still L2/L3-fallback's job** — see the extraction note below
|
||||
- **L3-fallback** (Chrome CDP + Python Playwright): backup for complex automation (long waits, race conditions, custom in-browser scripts)
|
||||
- **L3** (governed built-in browser, capability surface completed in v3.0): reach, *interact with*, and **read** logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. Bulk text extraction (`page.extract-text`), discriminated waits (`page.wait`), and code mode (`BrowserScript`) all close the loop inside this layer
|
||||
|
||||
### v2.1 — governed built-in browser (default-hidden, exposed only after Skill activation)
|
||||
The v2.x fourth layer — "user manually launches a debug Chrome + Python Playwright CDP" — was removed in v3.0: every reason it existed for (no bulk text channel, evaluate unusable, screenshots must activate-serialize) is now covered by the built-in browser, see the cheatsheet below.
|
||||
|
||||
When you call `Skill('web-access')`, the following 8 tools are injected into the current session so the LLM can drive the built-in browser directly:
|
||||
### v3.0: governed built-in browser (default-hidden, exposed only after Skill activation)
|
||||
|
||||
When you call `Skill('web-access')`, the following 9 tools are injected into the current session so the LLM can drive the built-in browser directly:
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| BrowserManage | Create/destroy isolated BrowserSpace, start sessions, manage tabs |
|
||||
| BrowserSnapshot | `semantic` / `accessibility` / `visual` page snapshots — the primary way to read a page |
|
||||
| BrowserAct | One governed action per call: navigate, click, type, scroll, screenshot, … |
|
||||
| BrowserSnapshot | `semantic` / `text` / `accessibility` / `visual` snapshots — the primary way to read a page |
|
||||
| BrowserAct | One governed action per call: navigate, input, extract text, wait, element ops, screenshot, … |
|
||||
| BrowserScript | **Code mode**: one async JS script issues browser commands back-to-back, eliminating per-action round trips (trust level equals Bash) |
|
||||
| BrowserImport | Import Cookies from the user's Chrome/Edge/Firefox/Safari profile (human-approved; **needs `browser.import.*` granted by the Host — not available to a plain `create_space` session**) |
|
||||
| BrowserShare | Delegate a Space/Session to another Agent (isolated / snapshot / copy-on-write / live) |
|
||||
| SitePatternRead / SitePatternWrite | Per-domain "site experience" (AgentFS three-layer) |
|
||||
@@ -119,20 +120,22 @@ When you call `Skill('web-access')`, the following 8 tools are injected into the
|
||||
|
||||
> **Important**: before `Skill('web-access')` is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See [references/browser-tools.md](references/browser-tools.md).
|
||||
>
|
||||
> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+.
|
||||
> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+; `page.extract-text` / `page.element` / `page.wait` / inline wait blocks / cross-origin iframe snapshots need v10.0.112+, and `BrowserScript` needs a build containing S17/S18.
|
||||
|
||||
### Use Cases
|
||||
|
||||
- The user needs to search for current information or research a specific topic
|
||||
- The user needs to fetch public web content or technical documentation
|
||||
- The user needs to access logged-in sites (Xiaohongshu, Bilibili, Weibo, Feishu, Twitter, etc.)
|
||||
- The user needs to access logged-in sites (Xiaohongshu, Bilibili, Weibo, Feishu, Twitter, etc.) and **read the body text**
|
||||
- The user needs to pull data from a site's own API in a logged-in context (lists, comments, orders, …)
|
||||
- The user needs to compare products, aggregate news, or investigate API/library versions
|
||||
|
||||
### Core Value
|
||||
|
||||
- **Four-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand
|
||||
- **Token optimization**: Jina Reader cuts token usage by 50–80% by default
|
||||
- **Logged-in session reuse**: attach to the user's Chrome via CDP in the fallback layer, or — where the Host has granted `browser.import.*` — import their Cookies into an isolated Space via BrowserImport; either way, no re-login required
|
||||
- **Three-layer progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand
|
||||
- **Token optimization**: Jina Reader cuts token usage by 50–80% by default; `page.extract-text`'s maxBytes/cursor paging keeps even long logged-in articles under control
|
||||
- **Logged-in session reuse**: where the Host has granted `browser.import.*`, BrowserImport brings the user's Cookies into an isolated Space — no re-login required
|
||||
- **Zero external dependencies**: no Python/Playwright install, no manually launched debug Chrome
|
||||
|
||||
## L2: Detailed Specification
|
||||
|
||||
@@ -144,53 +147,6 @@ When you complete a research task, you **MUST** cite all source URLs in your res
|
||||
|
||||
If any fetch fails, explicitly tell the user which URL failed and which fallback you used.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites: Chrome CDP Setup (for login-gated sites)
|
||||
|
||||
**Required for the L3-fallback layer** (Python Playwright) — and, in practice, still the reliable way to reuse a login. The L3-fast built-in browser can skip it only when the Host has granted `browser.import.*` so that `BrowserImport` actually works; otherwise set this up.
|
||||
|
||||
### One-time setup
|
||||
|
||||
Launch a dedicated Chrome instance with remote debugging enabled:
|
||||
|
||||
**macOS**:
|
||||
```bash
|
||||
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
|
||||
--remote-debugging-port=9222 \
|
||||
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
|
||||
```
|
||||
|
||||
**Linux**:
|
||||
```bash
|
||||
google-chrome \
|
||||
--remote-debugging-port=9222 \
|
||||
--user-data-dir="${DESIRECORE_ROOT}/chrome-profile"
|
||||
```
|
||||
|
||||
**Windows (PowerShell)**:
|
||||
```powershell
|
||||
& "C:\Program Files\Google\Chrome\Application\chrome.exe" `
|
||||
--remote-debugging-port=9222 `
|
||||
--user-data-dir="$env:USERPROFILE\.desirecore\chrome-profile"
|
||||
```
|
||||
|
||||
After launch:
|
||||
1. Manually log in to the sites you need (Xiaohongshu, Bilibili, Weibo, Feishu, …)
|
||||
2. Leave this Chrome window open in the background
|
||||
3. Verify the debug endpoint: `curl -s http://localhost:9222/json/version` should return JSON
|
||||
|
||||
### Verify CDP is ready
|
||||
|
||||
Before any CDP operation, always run:
|
||||
```bash
|
||||
curl -s http://localhost:9222/json/version | python3 -c "import sys,json; d=json.load(sys.stdin); print('CDP ready:', d.get('Browser'))"
|
||||
```
|
||||
|
||||
If the command fails, tell the user: "Please launch Chrome with the remote debugging port enabled (see the Prerequisites section of the web-access skill)."
|
||||
|
||||
---
|
||||
|
||||
## Tool Selection Decision Tree
|
||||
|
||||
```
|
||||
@@ -207,12 +163,12 @@ User intent
|
||||
│ (Jina Reader = default for JS-rendered content, saves tokens)
|
||||
│
|
||||
├─ "Read this login-gated page" (Xiaohongshu/Bilibili/Weibo/Feishu/Twitter/Zhihu/WeChat)
|
||||
│ ├─→ Reach it: BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
|
||||
│ │ → tab.activate + page.screenshot to confirm you landed on the content
|
||||
│ │ (semantic snapshot has no body text). Logged-in? BrowserImport only if
|
||||
│ │ browser.import.* was granted — otherwise reuse the login via CDP below.
|
||||
│ └─→ Extract the text: verify CDP ready, then python3 playwright.connect_over_cdp()
|
||||
│ → page.content() → Jina Reader / BeautifulSoup
|
||||
│ └─→ BrowserManage(create_space/start_session) → BrowserAct(tab.navigate)
|
||||
│ → BrowserAct(page.extract-text) ← body text read out directly, maxBytes/cursor paging
|
||||
│ Logged in? BrowserImport only if browser.import.* was granted
|
||||
│
|
||||
├─ "Pull data from the site's API in a logged-in context"
|
||||
│ └─→ fetch.browser recipe: run fetch inside BrowserAct(page.evaluate) with that origin's cookies
|
||||
│
|
||||
├─ "API documentation / GitHub / npm package info"
|
||||
│ └─→ Prefer official API endpoints over scraping HTML:
|
||||
@@ -221,24 +177,19 @@ User intent
|
||||
│ - PyPI: curl https://pypi.org/pypi/<pkg>/json
|
||||
│
|
||||
└─ "Real-time interactive task" (click, fill form, scroll, screenshot)
|
||||
├─→ **Default: built-in browser** (BrowserManage → BrowserAct → BrowserSnapshot —
|
||||
│ see references/browser-tools.md, no Python needed)
|
||||
└─→ Fallback: CDP + Python Playwright (references/cdp-browser.md) when the built-in browser is insufficient
|
||||
(e.g., complex race conditions, multi-event waits, long-running in-browser scripts)
|
||||
└─→ built-in browser (BrowserManage → BrowserAct → BrowserSnapshot —
|
||||
see references/browser-tools.md, no Python needed)
|
||||
```
|
||||
|
||||
### Four-layer strategy summary
|
||||
### Three-layer strategy summary
|
||||
|
||||
| Layer | Use case | Primary tool | Token cost |
|
||||
|-------|----------|--------------|------------|
|
||||
| L1 | Public, static | `WebFetch` | Low |
|
||||
| L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) |
|
||||
| **L3-fast** | **Login-gated navigation & interaction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot)** | Medium |
|
||||
| L3-fallback | Complex automation (race / long-wait / custom scripts) | `Bash + Python Playwright CDP` | Medium |
|
||||
| **L3** | **Login-gated navigation, interaction & extraction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript)** | Medium |
|
||||
|
||||
**Default priority**: L1 for simple public pages → L2 for heavy → **L3-fast for login-gated** → L3-fallback only when the built-in browser is insufficient.
|
||||
|
||||
---
|
||||
**Default priority**: L1 for simple public pages → L2 for heavy → **L3 for login-gated (body text and in-site API data included)**.
|
||||
|
||||
## Supported Sites Matrix
|
||||
|
||||
@@ -248,16 +199,14 @@ User intent
|
||||
| GitHub README, issues, PRs | `gh api` (best) → L1 WebFetch | Prefer API |
|
||||
| Hacker News, Reddit | L1 WebFetch | Public content |
|
||||
| Medium, Dev.to | L2 Jina Reader | JS-rendered, member gates |
|
||||
| Twitter/X | L3 CDP (or L2 Jina with `x.com`) | Login required for full thread |
|
||||
| Xiaohongshu (xiaohongshu.com) | L3 CDP | Login required |
|
||||
| Bilibili (bilibili.com) | L3 CDP | Login needed for video desc/comments |
|
||||
| Weibo (weibo.com) | L3 CDP | Long posts require login |
|
||||
| Zhihu (zhihu.com) | L3 CDP | Long articles + comments require login |
|
||||
| Feishu Docs (feishu.cn) | L3 CDP | Login required |
|
||||
| Twitter/X | L3 (or L2 Jina with `x.com`) | Login required for full thread |
|
||||
| Xiaohongshu (xiaohongshu.com) | L3 built-in browser + BrowserImport | Login required; body text via page.extract-text |
|
||||
| Bilibili (bilibili.com) | L3 built-in browser + BrowserImport | Login needed for video desc/comments |
|
||||
| Weibo (weibo.com) | L3 built-in browser + BrowserImport | Long posts require login |
|
||||
| Zhihu (zhihu.com) | L3 built-in browser + BrowserImport | Long articles + comments require login |
|
||||
| Feishu Docs (feishu.cn) | L3 built-in browser + BrowserImport | Login required |
|
||||
| WeChat Official Accounts (mp.weixin.qq.com) | L2 Jina Reader | Usually public, Jina cleans better |
|
||||
| LinkedIn | L3 CDP | Login wall |
|
||||
|
||||
---
|
||||
| LinkedIn | L3 built-in browser + BrowserImport | Login wall |
|
||||
|
||||
## Tool Reference
|
||||
|
||||
@@ -280,7 +229,7 @@ WebFetch(url="https://example.com/article")
|
||||
Tips:
|
||||
- Results cached for 15 min
|
||||
- Returns cleaned Markdown with title + URL + body
|
||||
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (CDP)
|
||||
- If body < 200 chars or looks garbled → escalate to Layer 2 (Jina) or Layer 3 (built-in browser)
|
||||
|
||||
### Layer 2: Jina Reader (default for heavy pages)
|
||||
|
||||
@@ -298,111 +247,94 @@ Why Jina is the default token-saver:
|
||||
|
||||
See [references/jina-reader.md](references/jina-reader.md) for advanced endpoints and rate limits.
|
||||
|
||||
### Layer 3: CDP Browser (login-gated access)
|
||||
### Layer 3: built-in browser (login-gated access)
|
||||
|
||||
Use Python Playwright's `connect_over_cdp()` to attach to the user's running Chrome (which already has login cookies). **No re-login needed.**
|
||||
The full command surface, capability tiers, and boundaries are in [references/browser-tools.md](references/browser-tools.md). The loop:
|
||||
|
||||
**Minimal template**:
|
||||
```bash
|
||||
python3 << 'PY'
|
||||
from playwright.sync_api import sync_playwright
|
||||
1. `BrowserManage(create_space)` → `BrowserManage(start_session)` for a sessionId + first tab
|
||||
2. Need login state → `BrowserImport` (only if the Host granted `browser.import.*`; without that grant the user's Cookies cannot be reused — tell the user and continue without login or abort)
|
||||
3. `BrowserAct(tab.navigate)` to reach the page
|
||||
4. `BrowserSnapshot(semantic)` for interactive-element `ref` handles (cross-origin iframe elements are in the same tree with globally sequential refs)
|
||||
5. Interact via `input.*` (humanized trajectories) or `page.element` (bulk form writes); wait for results via `page.wait` or inline wait blocks
|
||||
6. Read body text via `BrowserSnapshot(text)` or `BrowserAct(page.extract-text)`; pull API data via the fetch.browser recipe
|
||||
7. `BrowserManage(close_session)` when done
|
||||
|
||||
TARGET_URL = "https://www.xiaohongshu.com/explore/..."
|
||||
Multi-step sequences (navigate→snapshot→click→wait→extract) can be done in one `BrowserScript` run, skipping the per-action IPC round trips.
|
||||
|
||||
with sync_playwright() as p:
|
||||
browser = p.chromium.connect_over_cdp("http://localhost:9222")
|
||||
context = browser.contexts[0] # reuse user's default context (has cookies)
|
||||
page = context.new_page()
|
||||
page.goto(TARGET_URL, wait_until="domcontentloaded")
|
||||
page.wait_for_timeout(2000) # let lazy content load
|
||||
html = page.content()
|
||||
page.close()
|
||||
## L3 Cheatsheet (v3.0)
|
||||
|
||||
# Print first 500 chars to verify
|
||||
print(html[:500])
|
||||
PY
|
||||
```
|
||||
|
||||
**Extract text via BeautifulSoup** (no Jina round-trip):
|
||||
```bash
|
||||
python3 << 'PY'
|
||||
from playwright.sync_api import sync_playwright
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
with sync_playwright() as p:
|
||||
browser = p.chromium.connect_over_cdp("http://localhost:9222")
|
||||
page = browser.contexts[0].new_page()
|
||||
page.goto("https://www.bilibili.com/video/BV...", wait_until="networkidle")
|
||||
html = page.content()
|
||||
page.close()
|
||||
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
title = soup.select_one("h1.video-title")
|
||||
desc = soup.select_one(".video-desc")
|
||||
print("Title:", title.get_text(strip=True) if title else "N/A")
|
||||
print("Desc:", desc.get_text(strip=True) if desc else "N/A")
|
||||
PY
|
||||
```
|
||||
|
||||
See [references/cdp-browser.md](references/cdp-browser.md) for:
|
||||
- Per-site selectors (Xiaohongshu / Bilibili / Weibo / Zhihu / Feishu)
|
||||
- Scrolling & lazy-load patterns
|
||||
- Screenshot & form-fill recipes
|
||||
- Troubleshooting connection issues
|
||||
|
||||
---
|
||||
|
||||
## L3-fast: Built-in Browser Cheatsheet (v2.1)
|
||||
|
||||
**Only after you call `Skill('web-access')` will the following tools appear in `tools[]`.**
|
||||
|
||||
| Tool | One-line example |
|
||||
|------|-----------------|
|
||||
| `BrowserManage({ action: 'create_space', name, persistence: 'ephemeral' })` | One isolated Space per task |
|
||||
| `BrowserManage({ action: 'start_session', spaceId, capabilities })` | Start a session, returns sessionId + first tab |
|
||||
| `BrowserManage({ action: 'list_tabs' \| 'create_tab' \| 'close_session' })` | Tab / lifecycle management |
|
||||
| `BrowserAct({ action: 'tab.navigate', params: { url } })` | Navigate the current tab |
|
||||
| `BrowserSnapshot({ mode: 'semantic' })` | Read the page: interactive elements + `ref` handles |
|
||||
| `BrowserAct({ action: 'input.click', params: { ref } })` | Click by snapshot `ref`, never by raw x/y |
|
||||
| `BrowserAct({ action: 'input.text', params: { text } })` | Type into the focused element |
|
||||
| `BrowserAct({ action: 'input.wheel', params: { x: 640, y: 400, deltaX: 0, deltaY: 720 } })` | Scroll to trigger lazy loading — `x`/`y` (or `ref`) is **required**, and the session needs `browser.input.pointer.wheel` |
|
||||
| `BrowserAct({ action: 'tab.activate', params: { bounds } })` → `page.screenshot` | **activate first, then screenshot** |
|
||||
| `BrowserImport({ action: 'discover' \| 'create_plan' \| 'dry_run' \| 'apply' \| 'rollback' \| 'list_plans' })` | Reuse the user's login cookies — those 6 are the **complete** action set; happy path is `discover → create_plan → dry_run → apply`. **Needs `browser.import.*`, which `create_space` does not grant** (see below) |
|
||||
|
||||
Full API and edge cases: see [references/browser-tools.md](references/browser-tools.md).
|
||||
|
||||
**How to read a page** — pick by what you need:
|
||||
### Reading a page: pick the channel by need
|
||||
|
||||
| You need | Use | Note |
|
||||
|----------|-----|------|
|
||||
| Interactive elements + `ref` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons / inputs / links only — **no article text** |
|
||||
| Article text on a small/simple page | `BrowserSnapshot({ mode: 'accessibility' })` | Returns StaticText nodes; fails with `BROWSER_RESULT_TOO_LARGE` on real content pages (2 MB result cap, and the `depth` argument is currently ignored by the host) |
|
||||
| Article text on a real page | L2 Jina Reader (public) or L3-fallback Playwright (login-gated) | The built-in browser has no working bulk text-extraction channel yet |
|
||||
| What the page looks like | `tab.activate` → `BrowserAct({ action: 'page.screenshot' })` | Full-page only; read it visually |
|
||||
| Interactive elements + `ref` / `loc=` handles | `BrowserSnapshot({ mode: 'semantic' })` | Buttons/inputs/links + per-line `[ref=eN]` and (when producible) `[loc=...]` stable selectors; cross-origin iframes in the same tree |
|
||||
| Article body text | `BrowserSnapshot({ mode: 'text' })` or `BrowserAct({ action: 'page.extract-text' })` | markdown/text formats; beyond maxBytes it truncates and hands back a nextCursor for paging — no error |
|
||||
| Accessibility tree | `BrowserSnapshot({ mode: 'accessibility' })` | Respects `depth` (default 50, max 100) and the maxBytes budget — truncates + pages instead of failing wholesale |
|
||||
| What the page looks like | `BrowserSnapshot({ mode: 'visual' })` or `BrowserAct({ action: 'page.screenshot' })` | Pixels land directly in the result (vision models read them in place); `clip={x,y,width,height,scale}` for element-level crops (scale up to 4) and `captureBeyondViewport` for full-page capture |
|
||||
|
||||
`page.evaluate` is **not** an extraction channel: it needs human approval per call and its string/object return values come back as `[REDACTED:browser-runtime-value]`.
|
||||
### Command surface at a glance
|
||||
|
||||
`BrowserAct` actions grouped by purpose (full enum in the tool schema):
|
||||
|
||||
- **tab.***: `navigate` / `back` / `forward` / `reload` / `activate` / `close`
|
||||
- **input.***: `move` / `click` / `double-click` / `drag` / `wheel` / `touch` / `pinch` / `key` / `text` — humanized input (#1808: consistent UA/UA-CH identity + real trajectories); the preferred interaction channel on anti-bot sites
|
||||
- **page.element** (discriminated op × selector, nine ops): write ops `fill` / `select-option` / `check` / `uncheck` / `scroll-into-view`; read ops `get-attribute` / `bounding-box` / `count` / `all-inner-texts`. Selectors speak the `loc=` dialect or a snapshot `ref` (with snapshotId). `fill` refuses `input[type=password]`
|
||||
- **page.wait** (discriminated until, nine values): poll-type `load` / `domcontentloaded` / `networkidle` / `selector` / `url` / `timeout` return `waited:false` on timeout; event-type `request` / `response` / `download` throw on timeout. Default 10s, max 60s
|
||||
- **Inline wait blocks**: `params.wait` (isomorphic to page.wait params) on `tab.navigate` / `input.click` / `input.key` / `page.element{op:"fill"}` — one receipt completes "act→wait for result", the waiter registers before the action, no cross-IPC race
|
||||
- **page.evaluate**: `{ expression, awaitPromise }`, return values cross as-is (over-budget results truncate with a `truncated` flag, never throw); capability `browser.page.evaluate` sits behind the human gate (allow-all mode skips the card)
|
||||
- **page.extract-text / page.screenshot / page.wait**: see the table above and the fetch.browser recipe
|
||||
|
||||
`loc=` selector dialect (S7/S9): `e<N>` (must carry the snapshotId of the snapshot that issued the ref), `loc=css:` / `loc=role:` / `loc=text:` / `loc=testid:`, bare CSS, composable with `internal:nth/last/scope/filter`. Unknown prefixes fail explicitly — never silently degrade to CSS.
|
||||
|
||||
### Choosing the interaction channel: input.* vs page.element
|
||||
|
||||
- **On anti-bot sites (Xiaohongshu/Weibo/Bilibili etc.) always prefer `input.*`**: it rides the #1808 humanized input pipeline (coordinate dispatch, humanized trajectories, auditable visualization) plus the identity layer that keeps UA/UA-CH free of Electron/Headless tells
|
||||
- **`page.element` write ops fit bulk form filling on sites that don't detect automation**: one call fills/selects/checks, far faster than per-element input.click + input.text
|
||||
- **Red line**: `page.element` deliberately has no click — pointer actions must go through `input.*`; invoking `el.click()` via JS bypasses the entire humanization investment and is an explicitly forbidden fallback
|
||||
|
||||
### The fetch.browser recipe: pull API data with login state
|
||||
|
||||
The right way to pull a site's own API (lists, comments, orders, any JSON) in a logged-in context: run `fetch` **in the page context** — it carries that origin's cookies automatically, stays same-origin, is bounded by Grant origins, and rides the existing `page.evaluate` gate. It is a wrapper usage of `BrowserAct({ action: 'page.evaluate' })`:
|
||||
|
||||
```yaml
|
||||
BrowserAct:
|
||||
action: page.evaluate
|
||||
params:
|
||||
expression: |
|
||||
fetch('/api/v1/comments?page=1&size=20', {
|
||||
headers: { accept: 'application/json' }
|
||||
}).then(r => r.text())
|
||||
awaitPromise: true # default true; waits for the returned Promise to settle
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `tab.navigate` to any page on the site first (establishes the origin and cookies), then fire the fetch; use a relative path so it is same-origin by construction
|
||||
- Return values cross as-is; for large JSON take `.text()` and slice it yourself, or page through multiple calls
|
||||
- Only the current tab's origin is reachable (Grant origins constraint); for another site's API, navigate there first
|
||||
- `page.evaluate` is a human-gated capability: outside allow-all mode an approval card appears — explain the purpose to the user
|
||||
|
||||
### Recommended flow (Xiaohongshu example)
|
||||
|
||||
```
|
||||
1. BrowserManage({ action: 'create_space', name: 'xhs-note', persistence: 'ephemeral' })
|
||||
2. BrowserManage({ action: 'start_session', spaceId, capabilities: [...] })
|
||||
← an explicit list *narrows* the lease; include browser.input.pointer.wheel if you will scroll
|
||||
3. Reuse the login: L3-fallback Playwright against the user's own Chrome is the default path.
|
||||
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }) only works if the
|
||||
Host granted browser.import.* separately — create_space alone does not. If it is denied, fall back.
|
||||
← an explicit list *narrows* the lease; include browser.input.pointer.wheel to scroll
|
||||
and browser.observe.snapshot to extract text
|
||||
3. Reuse the login: only when the Host granted browser.import.*,
|
||||
BrowserImport({ action: 'discover' → 'create_plan' → 'dry_run' → 'apply' }).
|
||||
Without that grant the user's cookies cannot be reused — say so, then continue
|
||||
as logged-out or abort.
|
||||
4. BrowserAct({ action: 'tab.navigate', params: { url: 'https://www.xiaohongshu.com/explore/abc123' } })
|
||||
5. BrowserSnapshot({ mode: 'semantic' }) ← element refs for interaction (no body text)
|
||||
tab.activate + page.screenshot ← confirm the note actually rendered
|
||||
→ extract the note text via L3-fallback Playwright
|
||||
6. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience
|
||||
7. At task end → BrowserManage({ action: 'close_session', sessionId })
|
||||
8. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
|
||||
5. BrowserSnapshot({ mode: 'semantic' }) ← interaction refs (cross-origin iframes in the same tree)
|
||||
6. BrowserAct({ action: 'page.extract-text', params: { format: 'markdown' } })
|
||||
← body text read out directly; if too long, pass back the returned nextCursor to continue
|
||||
7. Need to confirm rendering → BrowserSnapshot({ mode: 'visual' }) (pixels readable in place)
|
||||
8. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience
|
||||
9. At task end → BrowserManage({ action: 'close_session', sessionId })
|
||||
10. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content })
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Site Experience Accumulation (v2.0)
|
||||
## Site Experience Accumulation
|
||||
|
||||
When the task ends and you've discovered new anti-bot pitfalls, effective selectors, or platform quirks, call:
|
||||
|
||||
@@ -411,7 +343,7 @@ SitePatternWrite({
|
||||
domain: "xiaohongshu.com",
|
||||
scope: "agent", // agent=shared (Git-tracked, can be published); user=private
|
||||
mode: "merge", // merge appends; replace overwrites
|
||||
content: "## Known pitfalls\n- 2026-05: ...",
|
||||
content: "## Known pitfalls\n- 2026-08: ...",
|
||||
confidence: "medium"
|
||||
})
|
||||
```
|
||||
@@ -427,8 +359,6 @@ SitePatternRead({ domain: "xiaohongshu.com" })
|
||||
|
||||
Content containing cookies / tokens / phone numbers / emails will **automatically downgrade scope='user'** and notify you.
|
||||
|
||||
---
|
||||
|
||||
## Common Workflows
|
||||
|
||||
Read [references/workflows.md](references/workflows.md) for detailed templates:
|
||||
@@ -437,11 +367,9 @@ Read [references/workflows.md](references/workflows.md) for detailed templates:
|
||||
- News aggregation & timelines
|
||||
- API/library version investigation
|
||||
|
||||
Read [references/cdp-browser.md](references/cdp-browser.md) for login-gated site recipes (Xiaohongshu / Bilibili / Weibo / Zhihu / Feishu).
|
||||
|
||||
Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader positioning, rate limits, and advanced endpoints.
|
||||
|
||||
---
|
||||
Read [references/browser-tools.md](references/browser-tools.md) for the full built-in browser command surface, capability tiers, and known boundaries.
|
||||
|
||||
## Quick Workflow: Multi-Source Research
|
||||
|
||||
@@ -455,27 +383,22 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
|
||||
7. Report with inline [source](url) citations + a Sources list at the end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns (Avoid)
|
||||
|
||||
- ❌ **Using WebFetch on obviously heavy sites** — Medium, Twitter, Xiaohongshu will waste tokens or fail. Jump straight to L2/L3.
|
||||
- ❌ **Launching headless Chrome instead of CDP attach** — loses user's login state, triggers anti-bot, slow cold start. Always use `connect_over_cdp()` to attach to the user's existing session.
|
||||
- ❌ **Fetching one URL at a time when you need 5** — batch in a single message.
|
||||
- ❌ **Trusting a single source** — cross-check ≥ 2 sources for non-trivial claims.
|
||||
- ❌ **Fetching the search result page itself** — WebSearch already returns snippets; fetch the actual articles.
|
||||
- ❌ **Ignoring the cache** — WebFetch caches 15 min, reuse freely.
|
||||
- ❌ **Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs.
|
||||
- ❌ **Scraping when an API exists** — GitHub, npm, PyPI, Wikipedia all have JSON APIs; a logged-in site's own API goes through the fetch.browser recipe.
|
||||
- ❌ **Forgetting the year in time-sensitive queries** — "best AI models" returns 2023 results; "best AI models 2026" returns current.
|
||||
- ❌ **Hardcoding login credentials in scripts** — always rely on the user's pre-logged CDP session.
|
||||
- ❌ **Hardcoding login credentials in scripts** — login state can only come from Cookies imported via BrowserImport.
|
||||
- ❌ **Citing only after the fact** — collect URLs as you fetch, not from memory afterwards.
|
||||
- ❌ **(v2.1) Writing Python heredoc when the built-in browser would do** — slow, requires Python+Playwright install, and bloats context. Prefer L3-fast; fall back to Python only when the built-in browser is insufficient (race / long-wait / custom scripts).
|
||||
- ❌ **(v2.1) Calling `page.screenshot` before `tab.activate`** — tabs are parked off-screen at `(-10000,-10000,1x1)` and have no compositing surface, so the capture stalls until the 30s deadline **and the tab host is destroyed**; every later call then fails with `BROWSER_TAB_HOST_NOT_FOUND`. Always `tab.activate` with real bounds first.
|
||||
- ❌ **(v2.1) Reaching for `page.evaluate` to read a page** — it needs human approval on every call, its string/object return values come back as `[REDACTED:browser-runtime-value]` (only number/boolean/null survive), and anti-debug sites stall it for tens of seconds. Read pages with `BrowserSnapshot` instead.
|
||||
- ❌ **(v2.1) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`.
|
||||
- ❌ **(v2.1) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.
|
||||
|
||||
---
|
||||
- ❌ **(v3.0) Using page.element for bulk interaction on anti-bot sites** — it is a direct JS call that bypasses humanized trajectories; on anti-bot sites always use `input.*`; keep `page.element` for bulk form filling where the site doesn't detect automation.
|
||||
- ❌ **(v3.0) Reading body text from screenshots** — `page.extract-text` / `BrowserSnapshot(text)` hand you markdown/text with paged budgets; save screenshots for layout confirmation and CAPTCHAs where you truly must look.
|
||||
- ❌ **(v3.0) Tolerating per-action round trips when BrowserScript would do** — navigate→snapshot→click→wait→extract runs as one script; remember BrowserScript's trust level equals Bash and the script source passes one human approval.
|
||||
- ❌ **(v3.0) Discovering new pitfalls and not writing a site-pattern** — next time the same Agent runs the task, it'll repeat the same mistakes. Anything that took 2+ steps to figure out is worth `SitePatternWrite(scope='agent', mode='merge')`.
|
||||
- ❌ **(v3.0) Writing cookies / phone numbers to scope='agent'** — that layer is Git-tracked and may be published to the marketplace. SitePatternWrite auto-downgrades, but don't deliberately write secrets to the agent layer.
|
||||
|
||||
## Example Interaction
|
||||
|
||||
@@ -484,26 +407,15 @@ Read [references/jina-reader.md](references/jina-reader.md) for Jina Reader posi
|
||||
**Agent workflow**:
|
||||
```
|
||||
1. Recognize → Xiaohongshu is an L3 logged-in site
|
||||
2. Check CDP: curl -s http://localhost:9222/json/version
|
||||
├─ Failure → prompt the user to launch Chrome in debug mode, abort
|
||||
└─ Success → continue
|
||||
3. Bash: python3 connect_over_cdp script → page.goto(url) → page.content()
|
||||
4. BeautifulSoup extract h1 title, .note-content, .comments
|
||||
2. BrowserManage(create_space + start_session)
|
||||
Need login state → if browser.import.* was granted, run the BrowserImport four steps;
|
||||
otherwise tell the user the login cannot be reused and continue with the public part
|
||||
3. BrowserAct(tab.navigate → the note URL)
|
||||
4. BrowserAct(page.extract-text, format: markdown)
|
||||
← body text read out directly; page with nextCursor if over budget
|
||||
5. When returning to the user:
|
||||
- Cite the original URL
|
||||
- If content is long, run it through Jina to save tokens
|
||||
6. Tell the user: "Fetched via your logged-in session, original link: [xhs](url)"
|
||||
- Quote facts from the extracted text with source links
|
||||
6. Tell the user: "Fetched via the built-in browser, original link: [xhs](url)"
|
||||
7. BrowserManage(close_session)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Installation Note
|
||||
|
||||
CDP features require Python + Playwright installed:
|
||||
|
||||
```bash
|
||||
pip3 install playwright beautifulsoup4
|
||||
python3 -m playwright install chromium # only needed if user hasn't installed Chrome
|
||||
```
|
||||
|
||||
If `playwright` is not installed when the user requests a login-gated site, run the install commands in Bash and explain you're setting up the browser automation dependency.
|
||||
|
||||
Reference in New Issue
Block a user