diff --git a/skills/web-access/SKILL.md b/skills/web-access/SKILL.md index 934bc0f..745f9f6 100644 --- a/skills/web-access/SKILL.md +++ b/skills/web-access/SKILL.md @@ -9,13 +9,13 @@ description: >- for public pages, (2) Jina Reader as the default token-optimization layer for heavy/JS-rendered pages, and (3) the governed built-in browser (isolated BrowserSpace + Cookie import + bulk text extraction + waits + code mode) to - reach, interact with, and read login-gated sites, and (4) the user's own Chrome over CDP when they ask for it by name. Always cite source URLs. + reach, interact with, and read login-gated sites, and (4) the user's named external Chromium browser (Chrome/Edge/Chromium) over CDP when requested. Always cite source URLs. Use when 用户提到 联网搜索、上网查、 查资料、抓取网页、研究、调研、最新资讯、文档查询、对比、竞品、技术文档、 新闻、网址、URL、找一下、搜一下、查一下、小红书、B站、微博、飞书、Twitter、 推特、X、知乎、公众号、已登录、登录状态。 license: Complete terms in LICENSE.txt -version: 3.2.0 +version: 3.4.0 type: procedural risk_level: low status: enabled @@ -31,6 +31,7 @@ tags: provides: tools: - BrowserManage + - BrowserExternalProbe - BrowserSnapshot - BrowserAct - BrowserScript @@ -41,7 +42,7 @@ provides: - LocalBookmarks metadata: author: desirecore - updated_at: '2026-08-21' + updated_at: '2026-08-29' i18n: default_locale: en-US source_locale: zh-CN @@ -51,16 +52,16 @@ metadata: zh-CN: name: 联网访问 short_desc: 联网搜索、网页抓取、内置受管浏览器登录态访问与取文、研究调研工作流 - description: 联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问与取文,以及用户点名时接管他自己的 Chrome。 + description: 联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问与取文,以及用户点名时接管他自己的 Chrome/Edge/Chromium。 body: ./SKILL.zh-CN.md - source_hash: sha256:4d3bc4221b2d6b09 + source_hash: sha256:1704b973e3a90e89 translated_by: human en-US: name: Web Access short_desc: Web search, page fetching, logged-in access via the governed built-in browser, research workflows - description: A web-access toolkit — search public pages, fetch heavy pages via Jina Reader, reach and read logged-in sites through the governed built-in browser, and drive the user's own Chrome over CDP on request. + description: A web-access toolkit — search public pages, fetch heavy pages via Jina Reader, reach and read logged-in sites through the governed built-in browser, and drive the user's named Chrome/Edge/Chromium over CDP on request. body: ./SKILL.md - source_hash: sha256:4d3bc4221b2d6b09 + source_hash: sha256:1704b973e3a90e89 translated_by: human market: icon: >- @@ -78,7 +79,7 @@ market: fill-opacity="0.12"/> category: research - required_client_version: 10.0.98 + required_client_version: 10.0.128 maintainer: name: DesireCore Official verified: true @@ -89,7 +90,7 @@ market: ## L0: One-line Summary -A web-access toolkit — search public pages, optimize fetches via Jina Reader, reach/interact with/read login-gated sites through the governed built-in browser, and drive the user's own Chrome over CDP when they ask for it by name. +A web-access toolkit — search public pages, optimize fetches via Jina Reader, reach/interact with/read login-gated sites through the governed built-in browser, and drive the user's named Chrome/Edge/Chromium over CDP when requested. ## L1: Overview & Use Cases @@ -101,17 +102,18 @@ web-access is a **procedural skill** that provides four complementary layers of - **L2** (Jina Reader): JS-rendered heavy pages, saving tokens by default - **L3** (governed built-in browser, capability surface completed in v3.0): reach, *interact with*, and **read** logged-in / interactive sites — isolated BrowserSpace per task, zero Python dependency, every action carries a signed receipt. Bulk text extraction (`page.extract-text`), discriminated waits (`page.wait`), and code mode (`BrowserScript`) all close the loop inside this layer -- **L3-external** (the user's own Chrome, attached via CDP + Python Playwright): **take this route when the user names their own browser** — their login state, their window, visible to them the whole time and theirs to take over at any moment +- **L3-external** (the user's named Chrome/Edge/Chromium, attached via CDP + Python Playwright): **take this route only when the user names their own browser or explicitly accepts it after you explain why** — its isolated DesireCore profile login state, its visible window, and the user's ability to take over at any moment A note of history on L3-external: v3.0 deleted it outright, on the grounds that "every technical reason it existed for (no bulk text channel, evaluate unusable, screenshots must activate-serialize) is now covered by the built-in browser". That technical judgement was correct — **as a fallback for when the built-in browser isn't enough, it genuinely isn't needed any more**. But the deletion took with it a completely different use case: the user wanting *their own* browser. That has nothing to do with capability, and the built-in browser cannot stand in for it, so v3.2 restores it as a peer option **triggered by user intent**. Note it is no longer a fallback; see "Two browsers — pick by user intent" below. ### v3.0: governed built-in browser (default-hidden, exposed only after Skill activation) -When you call `Skill('web-access')`, the following 9 tools are injected into the current session so the LLM can drive the built-in browser directly: +When you call `Skill('web-access')`, the following tools are injected into the current session. The `Browser*` tools drive the built-in browser; `BrowserExternalProbe` only inspects external-browser prerequisites: | Tool | Purpose | |------|---------| | BrowserManage | Create/destroy isolated BrowserSpace, start sessions, manage tabs | +| BrowserExternalProbe | Read-only check for installed Chrome/Edge/Chromium and loopback CDP readiness; never launches or reads a profile | | BrowserSnapshot | `semantic` / `text` / `accessibility` / `visual` snapshots — the primary way to read a page | | BrowserAct | One governed action per call: navigate, input, extract text, wait, element ops, screenshot, … | | BrowserScript | **Code mode**: one async JS script issues browser commands back-to-back, eliminating per-action round trips (trust level equals Bash) | @@ -122,7 +124,7 @@ When you call `Skill('web-access')`, the following 9 tools are injected into the > **Important**: before `Skill('web-access')` is called, none of these tools appear in the LLM tools list — default conversations don't pay their token cost. See [references/browser-tools.md](references/browser-tools.md). > -> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". Requires client v10.0.98+; `page.extract-text` / `page.element` / `page.wait` / inline wait blocks / cross-origin iframe snapshots need v10.0.112+, and `BrowserScript` needs a build containing S17/S18. +> **Removed in v2.1**: `BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` and the cdp-proxy behind them are retired. Calling them now returns "该旧 BrowserXxx/cdp-proxy 入口已停用". This skill version requires client v10.0.128+; `page.extract-text` / `page.element` / `page.wait` / inline wait blocks / cross-origin iframe snapshots were introduced in v10.0.112+, and `BrowserScript` needs a build containing S17/S18. ### Use Cases @@ -137,7 +139,7 @@ When you call `Skill('web-access')`, the following 9 tools are injected into the - **Layered progression**: from lightweight search to heavy JS rendering to logged-in access — pick on demand; plus the user's own browser whenever they name it - **Token optimization**: Jina Reader cuts token usage by 50–80% by default; `page.extract-text`'s maxBytes/cursor paging keeps even long logged-in articles under control - **Logged-in session reuse**: where the Host has granted `browser.import.*`, BrowserImport brings the user's Cookies into an isolated Space — no re-login required -- **Zero external dependencies by default**: the built-in browser needs no Python/Playwright install and no manually launched debug Chrome (L3-external does, and only when the user asks for it) +- **Zero external dependencies by default**: the built-in browser needs no Python/Playwright install and no manually launched external Chromium browser (L3-external does, and only after explicit user intent) ## L2: Detailed Specification @@ -149,51 +151,41 @@ When you complete a research task, you **MUST** cite all source URLs in your res If any fetch fails, explicitly tell the user which URL failed and which fallback you used. -## Prerequisites: Chrome CDP Setup (L3-external only) +## Prerequisites: external browser + CDP (L3-external only) -**Only needed when taking the L3-external route** (the user named their own browser). The built-in -browser has no prerequisites. +**Only needed when taking the L3-external route.** The built-in browser has no prerequisites. -### One-time setup +### Always probe before attaching -Have the user launch Chrome with remote debugging enabled: +Map the user's words to an exact probe request: -**macOS**: -```bash -/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \ - --remote-debugging-port=9222 \ - --user-data-dir="${DESIRECORE_ROOT}/chrome-profile" -``` +- “my Chrome” → `BrowserExternalProbe({ requestedBrowser: 'chrome' })` +- “my Edge” → `BrowserExternalProbe({ requestedBrowser: 'edge' })` +- “my external/system browser” without a product name → `BrowserExternalProbe({ requestedBrowser: 'any' })` +- “local browser” is ambiguous → ask whether they mean the built-in browser or their external browser before probing -**Linux**: -```bash -google-chrome \ - --remote-debugging-port=9222 \ - --user-data-dir="${DESIRECORE_ROOT}/chrome-profile" -``` +Handle the structured result exactly: -**Windows (PowerShell)**: -```powershell -& "C:\Program Files\Google\Chrome\Application\chrome.exe" ` - --remote-debugging-port=9222 ` - --user-data-dir="$env:USERPROFILE\.desirecore\chrome-profile" -``` +| status | Required response | +|---|---| +| `ready` | Continue with Playwright `connect_over_cdp`; name the detected external browser honestly. For `any`, the already-ready endpoint is the prepared choice even if other products are installed | +| `browser_not_installed` | Say the requested browser was not detected. If `alternatives` is non-empty, ask whether the user wants one of them; **never switch automatically** | +| `debug_port_closed` | Show the returned `launchCommand`, ask the user to launch it and log in manually, then wait and probe again | +| `browser_choice_required` | No endpoint is ready and multiple external browsers are installed. List id/name only and ask which one the user wants; probe that exact choice next | +| `browser_mismatch` | Say which browser is actually on the port and which one was requested; ask the user to correct the port or explicitly approve the other browser | +| `invalid_cdp_endpoint` | Explain that something is listening on the port but it is not a valid Chrome DevTools endpoint; do not attach | +| `host_unavailable` | Explain that this Agent Service cannot inspect the user's desktop host; do not assume a browser is installed or silently use the built-in browser | -After launch: -1. The user logs in manually to the sites they need -2. That Chrome window stays open -3. Verify the debug endpoint: `curl -s http://localhost:9222/json/version` should return JSON +`launchCommand` uses an isolated DesireCore profile. After launch: -### Verify readiness before every operation +1. The user logs in manually to the sites they need. +2. That external browser window stays open. +3. Call `BrowserExternalProbe` again. Only `ready` authorizes the CDP attach attempt. -```bash -curl -s http://localhost:9222/json/version | python3 -c "import sys,json; d=json.load(sys.stdin); print('CDP ready:', d.get('Browser'))" -``` +Do not replace this probe with `curl`: a refused connection cannot distinguish “browser not installed” +from “browser installed but debugging disabled”, and a random HTTP service must not be accepted as CDP. -If it fails, tell the user: "请先启动 Chrome 并开启远程调试端口(见 web-access 技能的 Prerequisites 部分)" -— **then wait for them.** Don't switch to the built-in browser just because it could also do the job. - -⚠️ When attached over CDP, **never call `browser.close()`** — that would close the user's own Chrome. +⚠️ When attached over CDP, **never call `browser.close()`** — that would close the user's own external browser. Only close the page you opened. Full recipes in [references/cdp-browser.md](references/cdp-browser.md). --- @@ -229,8 +221,8 @@ User intent │ └─ "Real-time interactive task" (click, fill form, scroll, screenshot) ├─→ **User named "my own / my machine's / the external browser"** → L3-external: - │ verify CDP is ready (see Prerequisites), then python3 playwright.connect_over_cdp() - │ If it isn't ready, give them the launch command and wait — don't quietly switch to the built-in one + │ BrowserExternalProbe(exact requested browser), then connect_over_cdp() only on `ready` + │ Otherwise follow the status guidance and wait — don't quietly switch to the built-in one └─→ **Otherwise (default)**: built-in browser (BrowserManage → BrowserAct → BrowserSnapshot — see references/browser-tools.md, no Python needed) ``` @@ -241,10 +233,10 @@ DesireCore can drive **two** browsers. They are peer options: | | L3 built-in governed browser | L3-external — the user's own browser | |---|---|---| -| What it is | A browser instance inside the app (the `Browser*` tools) | The Chrome installed on the user's machine, attached via CDP + Python Playwright | +| What it is | A browser instance inside the app (the `Browser*` tools) | The user-named Chrome/Edge/Chromium on their machine, attached via CDP + Python Playwright | | Login state | Isolated; needs `browser.import.*` granted by the Host before `BrowserImport` can pull cookies | **Literally the user's own session** — nothing to import | | Can the user see it | Agent tabs are offscreen by default; must be presented to the workbench | **It's their own window** — visible throughout, theirs to take over | -| Prerequisite | None | User must launch Chrome with `--remote-debugging-port=9222` (see Prerequisites) | +| Prerequisite | None | `BrowserExternalProbe` must report `ready`; otherwise follow its exact status guidance | | Default | ✅ yes | When the user names it | > The login-state row is easy to misread as "the built-in browser can't reuse the user's login @@ -258,13 +250,13 @@ DesireCore can drive **two** browsers. They are peer options: **The layer is chosen by user intent, not by technical difficulty.** v3.0 deleted this layer as "a fallback for when the built-in browser isn't enough" — and as a fallback, it really isn't needed any more. But that deletion also removed a **different** use case: the user wanting *their own* -browser. That has nothing to do with capability — their login state lives in their Chrome, and they +browser. That has nothing to do with capability — the login state they establish lives in their named external browser, and they want to watch it happen and take over when they choose. The built-in browser cannot stand in for that. **If the user named one, use the one they named:** -- "my own / my machine's / the external browser / my Chrome" → **L3-external**. Verify CDP readiness - per Prerequisites first; if it isn't ready, give them the launch command and wait. **Do not switch +- "my own / my machine's / the external browser / my Chrome" → **L3-external**. Probe the exact + requested browser first; if it isn't ready, follow the structured status and wait. **Do not switch to the built-in browser just because it could also do the job** - "the built-in browser", or nothing named → **L3 built-in** (default, no prerequisites) - Genuinely unclear which they mean → ask, don't guess @@ -280,7 +272,7 @@ When what they asked for and what you're giving differ, the wording has to make | L1 | Public, static | `WebFetch` | Low | | L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) | | **L3** | **Login-gated navigation, interaction & extraction (PRIMARY)** | **built-in browser (BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript)** | Medium | -| L3-external | **User named their own browser**; or their personal login state is needed and `BrowserImport` is unavailable | `Bash + Python Playwright connect_over_cdp` (see references/cdp-browser.md) | Medium | +| L3-external | **User named their own browser**, or explicitly accepted this route after you explained why it is needed | `BrowserExternalProbe` → `Bash + Python Playwright connect_over_cdp` (see references/cdp-browser.md) | Medium | **Default priority**: L1 for simple public pages → L2 for heavy → **L3 for login-gated (body text and in-site API data included)**. @@ -354,7 +346,36 @@ The full command surface, capability tiers, and boundaries are in [references/br 4. `BrowserSnapshot(semantic)` for interactive-element `ref` handles (cross-origin iframe elements are in the same tree with globally sequential refs) 5. Interact via `input.*` (humanized trajectories) or `page.element` (bulk form writes); wait for results via `page.wait` or inline wait blocks 6. Read body text via `BrowserSnapshot(text)` or `BrowserAct(page.extract-text)`; pull API data via the fetch.browser recipe -7. `BrowserManage(close_session)` when done +7. Wrap up: **confirm first, then decide whether to close** — see "Wrapping up: before you report, before you close" below + +### Wrapping up: before you report, before you close + +**1. Before reporting a browser result to the user, take one more snapshot to confirm the page is still there.** + +Never report based on an earlier successful navigation. The session may have been terminated since — +most often by the resource quota guard (`BROWSER_RESOURCE_QUOTA_EXCEEDED`), which heavy JS sites hit +easily. This happened in the field: the agent navigated successfully, reported "✅ opened, the page is +visible in the built-in browser panel", while the session had already crashed and the user was staring +at a blank panel. + +One `BrowserSnapshot` (`text` or `visual`) before reporting surfaces this: if the session is gone, the +tool returns `BROWSER_TOOL_SESSION_TERMINATED` with the reason. When that happens, **tell the user the +session was interrupted**, then decide whether to retry, switch to a lighter page, or hand it back — +never present the earlier success as the current state. + +**2. Do not close the session unconditionally. The test is whether the user still needs to look at it.** + +| Task nature | Wrap-up | +|---|---| +| Demo / interactive / user wants to see the result | **Keep the session** and tell them where the page is, so they can take over | +| Pure data extraction, text already retrieved | Close it and free the resources | +| User explicitly said "close it when done" | Close it | + +Showing the built-in browser to the user depends on the workbench presenting that session; close it and +there is nothing left to show. This happened in the field: the agent finished a two-minute demo and +immediately called `close_session`, leaving the user with an empty panel — nothing it did was wrong, it +just left no chance to look. **When in doubt, keep it**: keeping costs a little resource, closing too +early costs the user the whole run. Multi-step sequences (navigate→snapshot→click→wait→extract) can be done in one `BrowserScript` run, skipping the per-action IPC round trips. @@ -427,8 +448,9 @@ Notes: ← body text read out directly; if too long, pass back the returned nextCursor to continue 7. Need to confirm rendering → BrowserSnapshot({ mode: 'visual' }) (pixels readable in place) 8. SitePatternRead({ domain: 'xiaohongshu.com' }) ← read accumulated experience -9. At task end → BrowserManage({ action: 'close_session', sessionId }) -10. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) +9. Before reporting → one more BrowserSnapshot to confirm the page is still there (the session may have been quota-terminated) +10. At task end → if the user still wants to look, **keep the session** and say where the page is; close only for pure extraction +11. If you find a new pitfall → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) ``` ## Site Experience Accumulation diff --git a/skills/web-access/SKILL.zh-CN.md b/skills/web-access/SKILL.zh-CN.md index 7ada3f4..665808f 100644 --- a/skills/web-access/SKILL.zh-CN.md +++ b/skills/web-access/SKILL.zh-CN.md @@ -4,7 +4,7 @@ ## L0:一句话摘要 -联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问与交互,以及用户点名时接管他自己的 Chrome。 +联网访问工具包——搜索公开页面、Jina 优化抓取、内置受管浏览器完成登录态访问与交互,以及用户点名时接管他自己的 Chrome/Edge/Chromium。 ## L1:概述与使用场景 @@ -16,17 +16,18 @@ web-access 是一个**流程型技能(Procedural Skill)**,提供四层互 - **L2**(Jina Reader):JS 渲染的重页面,默认节省 Token - **L3**(内置受管浏览器,v3.0 能力面补全):到达、操作并**读取**登录态/交互站点——每个任务独立 BrowserSpace 隔离、零 Python 依赖、每次动作都有可审计回执。批量取文(`page.extract-text`)、判别式等待(`page.wait`)、代码模式(`BrowserScript`)都在本层内闭环 -- **L3-external**(用户自己的 Chrome,经 CDP + Python Playwright 接管):**用户点名要用他自己那个浏览器时走这条**——他的登录态、他的窗口、他能全程看着并随时接管 +- **L3-external**(用户点名的 Chrome/Edge/Chromium,经 CDP + Python Playwright 接管):**仅在用户点名,或 Agent 解释原因后用户明确同意时走这条**——DesireCore 隔离外部 Profile 中由用户手工登录形成的状态、可见窗口,以及用户随时接管的能力 关于 L3-external 的一段历史:v3.0 曾把它整个删掉,理由是「它存在的每一条技术理由(无批量取文通道、evaluate 不可用、截图必须串行 activate)都已被内置浏览器覆盖」。那个技术判断没错,**作为「内置浏览器不够用时的兜底」它确实不再需要**。但删除时顺带丢掉了一个完全不同的用例:用户想用**他自己那个**浏览器。这跟能力够不够无关,内置浏览器替代不了,所以 v3.2 把它作为一条**由用户意图触发**的平级选择恢复回来——注意它不再是 fallback,判据见下方「两个浏览器,按用户意图选」。 ### v3.0:内置受管浏览器(默认隐藏,激活后才暴露) -调用 `Skill('web-access')` 加载本技能时,以下 9 个工具被注入到当前会话,让 LLM 直接驱动内置浏览器: +调用 `Skill('web-access')` 加载本技能时会注入以下工具。`Browser*` 工具驱动内置浏览器;`BrowserExternalProbe` 只检查外部浏览器前置条件: | 工具 | 用途 | |------|------| | BrowserManage | 建/销隔离 BrowserSpace、启动会话、管理标签页 | +| BrowserExternalProbe | 只读检查 Chrome/Edge/Chromium 安装与 loopback CDP 就绪状态;绝不启动浏览器或读取 Profile | | BrowserSnapshot | `semantic` / `text` / `accessibility` / `visual` 四种快照——读页面的主通道 | | BrowserAct | 一次调用一个受管动作:导航、输入、取文、等待、元素操作、截图…… | | BrowserScript | **代码模式**:一段异步 JS 连续下发浏览器命令,消除逐动作往返(信任级别等同 Bash) | @@ -37,7 +38,7 @@ web-access 是一个**流程型技能(Procedural Skill)**,提供四层互 > **重要**:未调用 Skill('web-access') 之前,这些工具**不会**出现在 LLM 的 tools 列表里——默认对话不消耗其 token。详见 [references/browser-tools.md](references/browser-tools.md)。 > -> **v2.1 已移除**:`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用,调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本版要求客户端 v10.0.98+;`page.extract-text` / `page.element` / `page.wait` / 内联 wait 块 / 跨源 iframe 快照需 v10.0.112+,`BrowserScript` 需包含 S17/S18 的更新版本。 +> **v2.1 已移除**:`BrowserListTabs` / `BrowserNavigate` / `BrowserEval` / `BrowserClick` / `BrowserScreenshot` / `BrowserScroll` / `BrowserSetFiles` / `BrowserCloseTab` 及其背后的 cdp-proxy 已停用,调用会返回「该旧 BrowserXxx/cdp-proxy 入口已停用」。本技能版本要求客户端 v10.0.128+;`page.extract-text` / `page.element` / `page.wait` / 内联 wait 块 / 跨源 iframe 快照自 v10.0.112 起可用,`BrowserScript` 需包含 S17/S18 的更新版本。 ### 使用场景 @@ -52,7 +53,7 @@ web-access 是一个**流程型技能(Procedural Skill)**,提供四层互 - **分层递进**:从轻量搜索到重度 JS 渲染到登录态访问,按需选择;用户点名时还可直接用他自己的浏览器 - **Token 优化**:Jina Reader 默认减少 50-80% Token 消耗;`page.extract-text` 的 maxBytes/cursor 分页让登录态长文也可控 - **登录态复用**:Host 授予 `browser.import.*` 时用 BrowserImport 把 Cookie 导入隔离 Space,不必重新登录 -- **默认零外部依赖**:内置浏览器不要求 Python/Playwright,也不要求用户手工启动调试 Chrome(L3-external 需要,且仅在用户点名时才用) +- **默认零外部依赖**:内置浏览器不要求 Python/Playwright,也不要求用户手工启动调试外部 Chromium 浏览器(L3-external 需要,且仅在明确用户意图后才用) ## L2:详细规范 @@ -64,50 +65,42 @@ When you complete a research task, you **MUST** cite all source URLs in your res If any fetch fails, explicitly tell the user which URL failed and which fallback you used. -## Prerequisites: Chrome CDP Setup(仅 L3-external 需要) +## Prerequisites:外部浏览器 + CDP(仅 L3-external 需要) -**只有走 L3-external(用户点名要用他自己的浏览器)时才需要这一步。** 内置浏览器零前置条件。 +**只有走 L3-external 时才需要。** 内置浏览器零前置条件。 -### One-time setup +### 每次连接前必须先结构化探测 -让用户带远程调试端口启动 Chrome: +按用户原话映射精确请求: -**macOS**: -```bash -/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \ - --remote-debugging-port=9222 \ - --user-data-dir="${DESIRECORE_ROOT}/chrome-profile" -``` +- 「我的 Chrome」→ `BrowserExternalProbe({ requestedBrowser: 'chrome' })` +- 「我的 Edge」→ `BrowserExternalProbe({ requestedBrowser: 'edge' })` +- 只说「我的外部/系统浏览器」、没点产品名 → `BrowserExternalProbe({ requestedBrowser: 'any' })` +- 「本地浏览器」语义两可 → 先问是内置浏览器还是用户自己的外部浏览器,再探测 -**Linux**: -```bash -google-chrome \ - --remote-debugging-port=9222 \ - --user-data-dir="${DESIRECORE_ROOT}/chrome-profile" -``` +严格按结构化状态处理: -**Windows (PowerShell)**: -```powershell -& "C:\Program Files\Google\Chrome\Application\chrome.exe" ` - --remote-debugging-port=9222 ` - --user-data-dir="$env:USERPROFILE\.desirecore\chrome-profile" -``` +| status | 必须采取的动作 | +|---|---| +| `ready` | 才能继续 Playwright `connect_over_cdp`,并如实说出检测到的外部浏览器;`any` 已有 ready 端口时,即使还安装了其他产品,也以该端口作为用户已准备的选择 | +| `browser_not_installed` | 明确说未检测到用户点名的浏览器;若有 `alternatives`,询问是否改用其中之一,**绝不自动替换** | +| `debug_port_closed` | 展示返回的 `launchCommand`,请用户启动并手工登录,然后等待并重新 probe | +| `browser_choice_required` | 没有 ready 端口且检测到多个外部浏览器;只列 id/name 并询问用户选哪个,再 probe 精确选择 | +| `browser_mismatch` | 说明端口上实际是什么、用户点名的是什么;让用户修正端口或明确同意改用实际浏览器 | +| `invalid_cdp_endpoint` | 说明端口虽有服务但不是合法 Chrome DevTools 端点;不得连接 | +| `host_unavailable` | 说明当前 Agent Service 无法探测用户桌面宿主;不得猜已安装浏览器,也不得静默改用内置浏览器 | -启动后: -1. 用户在这个 Chrome 里手工登录需要的站点 -2. 这个 Chrome 窗口保持开着 -3. 验证调试端点:`curl -s http://localhost:9222/json/version` 应返回 JSON +`launchCommand` 使用 DesireCore 专属隔离 Profile。启动后: -### 每次操作前先验就绪 +1. 用户在该外部浏览器里手工登录所需站点。 +2. 外部浏览器窗口保持打开。 +3. 再调用一次 `BrowserExternalProbe`;只有 `ready` 才允许尝试 CDP attach。 -```bash -curl -s http://localhost:9222/json/version | python3 -c "import sys,json; d=json.load(sys.stdin); print('CDP ready:', d.get('Browser'))" -``` +不得用 `curl` 替代本探测:连接拒绝无法区分「未安装浏览器」与「已安装但没开调试」,普通 HTTP 服务也不能冒充 CDP。 -失败就告诉用户:「请先启动 Chrome 并开启远程调试端口(见 web-access 技能的 Prerequisites 部分)」, -**然后等他**——不要因为内置浏览器也能做就擅自改用内置的。 +必须按上表的结构化状态提示用户并等待;不要因为内置浏览器也能做就擅自改用内置的。 -⚠️ 用 CDP attach 时**绝不能调 `browser.close()`**,那会关掉用户自己的 Chrome;只关你开的 page。 +⚠️ 用 CDP attach 时**绝不能调 `browser.close()`**,那会关掉用户自己的外部浏览器;只关你开的 page。 完整配方见 [references/cdp-browser.md](references/cdp-browser.md)。 --- @@ -143,8 +136,8 @@ User intent │ └─ "Real-time interactive task" (click, fill form, scroll, screenshot) ├─→ **用户点名「我本机的 / 我自己的 / 外部的浏览器」** → L3-external: - │ 先验 CDP 就绪(见 Prerequisites),再 python3 playwright.connect_over_cdp() - │ 没就绪就给启动命令并等他,不要擅自改用内置浏览器 + │ BrowserExternalProbe(精确点名的浏览器),仅 `ready` 后 connect_over_cdp() + │ 其余状态按表处理并等待,不要擅自改用内置浏览器 └─→ **其余情况(默认)**:内置受管浏览器 (BrowserManage → BrowserAct → BrowserSnapshot — see references/browser-tools.md, no Python needed) ``` @@ -155,10 +148,10 @@ DesireCore 能驱动**两个**浏览器,它们是平级的选项: | | L3 内置受管浏览器 | L3-external 用户自己的浏览器 | |---|---|---| -| 是什么 | 应用内的浏览器实例(`Browser*` 工具族) | 用户机器上装的 Chrome,经 CDP + Python Playwright 接管 | +| 是什么 | 应用内的浏览器实例(`Browser*` 工具族) | 用户点名的 Chrome/Edge/Chromium,经 CDP + Python Playwright 接管 | | 登录态 | 独立隔离;需 Host 授予 `browser.import.*` 才能用 `BrowserImport` 导 Cookie | **就是用户本人的登录态**,无需导入 | | 用户能看到吗 | Agent 开的标签页默认离屏,需展示到工作台 | **就在用户自己的窗口里**,他能全程看着、随时接管 | -| 前置条件 | 无 | 用户需先带 `--remote-debugging-port=9222` 启动 Chrome(见 Prerequisites) | +| 前置条件 | 无 | `BrowserExternalProbe` 必须返回 `ready`;否则严格按结构化状态处理 | | 默认 | ✅ 是 | 用户点名时 | > 登录态那一栏容易读成「内置浏览器复用不了用户的登录态」——不是那个意思。准确说法是 @@ -169,13 +162,13 @@ DesireCore 能驱动**两个**浏览器,它们是平级的选项: **选层判据是用户意图,不是技术难度。** v3.0 把这一层当作「内置浏览器不够用时的兜底」删掉过, 那个技术判断本身没错(取文、evaluate、截图这些内置浏览器现在都能做),但它顺带删掉的是一个 -**完全不同的用例**:用户想用**他自己那个**浏览器。那跟能力够不够无关——他的登录态在他自己的 -Chrome 里,他想亲眼看着操作、随时接管。这个需求内置浏览器替代不了。 +**完全不同的用例**:用户想用**他自己那个**浏览器。那跟能力够不够无关——他在点名的外部浏览器中 +建立登录态,想亲眼看着操作、随时接管。这个需求内置浏览器替代不了。 **用户点名了就按点名的来:** -- 说「我本机的 / 我自己的 / 外部浏览器 / 我的 Chrome」→ 走 **L3-external**。先按 - Prerequisites 验 CDP 就绪;没就绪就告诉他启动命令并等他,**不要因为「内置浏览器也能做」 +- 说「我本机的 / 我自己的 / 外部浏览器 / 我的 Chrome」→ 走 **L3-external**。先 probe + 用户精确点名的浏览器;没就绪就按结构化状态处理并等待,**不要因为「内置浏览器也能做」 就擅自改用内置的** - 说「内置浏览器」或没点名 → 走 **L3 内置浏览器**(默认,零前置条件) - 拿不准他指哪个 → 问一句,别猜 @@ -190,7 +183,7 @@ Chrome 里,他想亲眼看着操作、随时接管。这个需求内置浏览 | L1 | Public, static | `WebFetch` | Low | | L2 | JS-heavy, long articles, token savings | `Bash curl r.jina.ai` | **Lowest** (Markdown pre-cleaned) | | **L3** | **登录态导航、交互与取文 (PRIMARY)** | **内置受管浏览器(BrowserManage / BrowserAct / BrowserSnapshot / BrowserScript)** | Medium | -| L3-external | **用户点名要用他自己的浏览器**;或需要他本人的登录态而 `BrowserImport` 不可用 | `Bash + Python Playwright connect_over_cdp`(见 references/cdp-browser.md) | Medium | +| L3-external | **用户点名要用他自己的浏览器**,或 Agent 解释原因后用户明确同意改走此路径 | `BrowserExternalProbe` → `Bash + Python Playwright connect_over_cdp`(见 references/cdp-browser.md) | Medium | **Default priority**: L1 for simple public pages → L2 for heavy → **L3 for login-gated(含正文与站内接口取数)**。 @@ -264,10 +257,36 @@ See [references/jina-reader.md](references/jina-reader.md) for advanced endpoint 4. `BrowserSnapshot(semantic)` 拿可交互元素 `ref`(跨源 iframe 的元素也在同一棵树里,ref 全局连续编号) 5. 交互用 `input.*`(拟真轨迹)或 `page.element`(表单批量写);等结果用 `page.wait` 或内联 wait 块 6. 取正文用 `BrowserSnapshot(text)` 或 `BrowserAct(page.extract-text)`;取接口数据用 fetch.browser 配方 -7. 任务收尾 `BrowserManage(close_session)` +7. 收尾:**先确认、再决定关不关**,见下方「收尾:汇报之前与关闭之前」 多动作连续编排(导航→快照→点击→等待→取文)可用 `BrowserScript` 一段脚本完成,省去逐动作 IPC 往返。 +### 收尾:汇报之前与关闭之前 + +**一、向用户汇报浏览器结果之前,先用一次快照确认页面仍在。** + +不要拿上一次成功的导航结果直接汇报。会话可能在那之后被系统终止——最常见的是页面持续占用 +过多资源触发配额保护(`BROWSER_RESOURCE_QUOTA_EXCEEDED`),重 JS 站点很容易命中。真机发生过: +Agent 导航成功后直接汇报「✅ 已打开,页面已可在内置浏览器面板中查看」,而会话早已 crashed, +用户看到的是一片空白。 + +汇报前补一次 `BrowserSnapshot`(`text` 或 `visual` 均可)即可暴露这类情况:会话若已终止, +工具会返回 `BROWSER_TOOL_SESSION_TERMINATED` 并说明原因。此时**如实告诉用户会话中断了**, +再决定是重试、换更轻量的页面,还是交由用户处理——绝不能把先前的成功当作现状。 + +**二、不要无条件 `close_session`。判据是「用户还需不需要看」。** + +| 任务性质 | 收尾 | +|---|---| +| 演示 / 交互 / 用户要看结果 | **保留会话**,并告诉用户页面停在哪、可以直接接管 | +| 纯数据抓取,正文已取回 | 关闭,释放资源 | +| 用户明确说「用完关掉」 | 关闭 | + +内置浏览器的画面要展示给用户,靠的是工作台把会话呈现出来;会话一关,用户就什么都看不到了。 +真机发生过:Agent 用两分钟做完演示随即 `close_session`,用户回头去看只剩空白面板——它做的事 +没错,只是没留给用户任何查看的机会。拿不准时**保留**:留着最多占一点资源,关早了用户就得 +从头再来一遍。 + ## L3 速查(v3.0) ### 读页面:按需求选通道 @@ -336,8 +355,9 @@ BrowserAct: ← 正文直接读出;太长就传上一页返回的 nextCursor 续读 7. 需要确认渲染效果 → BrowserSnapshot({ mode: 'visual' })(像素直接可看) 8. SitePatternRead({ domain: 'xiaohongshu.com' }) ← 读累积经验 -9. 任务结束 → BrowserManage({ action: 'close_session', sessionId }) -10. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) +9. 汇报前 → 再来一次 BrowserSnapshot 确认页面仍在(会话可能已被配额终止) +10. 任务结束 → 用户还要看就**保留会话**并告知页面位置;纯抓取才 close_session +11. 如发现新陷阱 → SitePatternWrite({ domain, scope: 'agent', mode: 'merge', content }) ``` ## 站点经验积累 diff --git a/skills/web-access/references/cdp-browser.md b/skills/web-access/references/cdp-browser.md index 40cdd14..f61c564 100644 --- a/skills/web-access/references/cdp-browser.md +++ b/skills/web-access/references/cdp-browser.md @@ -1,8 +1,19 @@ # CDP Browser Access — Login-Gated Sites Manual -Detailed recipes for accessing sites that require the user's login session, via Chrome DevTools Protocol (CDP) + Python Playwright. +Detailed recipes for accessing sites through the user-approved external Chromium browser, via Chrome DevTools Protocol (CDP) + Python Playwright. -**Precondition**: Chrome is already running with `--remote-debugging-port=9222` and the user has manually logged in to the target sites. See the main SKILL.md `Prerequisites` section for the launch command. +**Precondition**: `BrowserExternalProbe` has returned `ready` for the exact browser the user requested, that browser is running with its DesireCore-isolated profile, and the user has manually logged in to the target sites. Never infer readiness from a failed/successful `curl`; see the main SKILL.md status table. + +The probe distinguishes these cases before Playwright is involved: + +- requested browser not installed (`browser_not_installed`) +- browser installed but debug port closed (`debug_port_closed`) +- multiple browsers installed with no ready endpoint (`browser_choice_required`) +- a different browser owns the port (`browser_mismatch`) +- a non-CDP service owns the port (`invalid_cdp_endpoint`) +- desktop host cannot be inspected (`host_unavailable`) + +Only `ready` permits `connect_over_cdp`. An alternative browser is a suggestion requiring user approval, never an automatic fallback. --- @@ -12,24 +23,34 @@ Detailed recipes for accessing sites that require the user's login session, via |----------|-------------|----------|-------|------| | Headless Playwright (new context) | ❌ Empty cookies | ❌ Flagged as bot | Slow cold start | Re-login pain | | `playwright.chromium.launch(headless=False)` | ❌ Fresh profile | ⚠ Sometimes flagged | Slow | Same | -| **CDP attach (`connect_over_cdp`)** | ✅ User's real cookies | ✅ Looks human | Instant | Zero friction | +| **CDP attach (`connect_over_cdp`)** | ✅ Cookies from the DesireCore-isolated external profile where the user logged in manually | ✅ Looks human | Instant | Zero friction | -**Rule**: For any login-gated site, always attach to the user's running Chrome. +**Rule**: Attach only when the user named the external browser or explicitly accepted this route after +you explained why. BrowserImport being unavailable does not itself authorize switching to the user's +external browser. --- ## Core Template -Every CDP script follows this shape: +Every CDP script follows this shape. `PROBE_PORT` must be the numeric `port` from the latest `ready` +`BrowserExternalProbe` result; never silently fall back to 9222 after probing another port. ```python from playwright.sync_api import sync_playwright -def fetch_with_cdp(url: str, wait_selector: str | None = None) -> str: - """Attach to user's Chrome via CDP, fetch URL, return HTML.""" +PROBE_PORT: int | None = None # Assign the exact BrowserExternalProbe ready result port. + +def cdp_url(port: int | None) -> str: + if not isinstance(port, int) or not 1 <= port <= 65535: + raise ValueError("invalid CDP port") + return f"http://127.0.0.1:{port}" + +def fetch_with_cdp(url: str, cdp_port: int, wait_selector: str | None = None) -> str: + """Attach to the user-approved external Chromium profile, fetch URL, return HTML.""" with sync_playwright() as p: - browser = p.chromium.connect_over_cdp("http://localhost:9222") - # browser.contexts[0] is the user's default context (with cookies) + browser = p.chromium.connect_over_cdp(cdp_url(cdp_port)) + # contexts[0] is the DesireCore-isolated external profile where the user logged in manually. context = browser.contexts[0] page = context.new_page() try: @@ -41,14 +62,18 @@ def fetch_with_cdp(url: str, wait_selector: str | None = None) -> str: return page.content() finally: page.close() - # DO NOT call browser.close() — that would close the user's Chrome! + # DO NOT call browser.close() — that would close the user's external browser! if __name__ == "__main__": - html = fetch_with_cdp("https://example.com") + if PROBE_PORT is None: + raise RuntimeError( + "Run BrowserExternalProbe first and assign the port from its ready result to PROBE_PORT." + ) + html = fetch_with_cdp("https://example.com", PROBE_PORT) print(html[:1000]) ``` -**Critical**: Never call `browser.close()` when using CDP attach — you'd kill the user's Chrome. Only close the page you opened. +**Critical**: Never call `browser.close()` when using CDP attach — you'd kill the user's external browser. Only close the page you opened. --- @@ -63,7 +88,7 @@ from bs4 import BeautifulSoup NOTE_URL = "https://www.xiaohongshu.com/explore/XXXXXXXX" with sync_playwright() as p: - browser = p.chromium.connect_over_cdp("http://localhost:9222") + browser = p.chromium.connect_over_cdp(cdp_url(PROBE_PORT)) page = browser.contexts[0].new_page() page.goto(NOTE_URL, wait_until="domcontentloaded") page.wait_for_selector("#detail-title", timeout=10000) @@ -96,7 +121,7 @@ from bs4 import BeautifulSoup VIDEO_URL = "https://www.bilibili.com/video/BVxxxxxxxxx" with sync_playwright() as p: - browser = p.chromium.connect_over_cdp("http://localhost:9222") + browser = p.chromium.connect_over_cdp(cdp_url(PROBE_PORT)) page = browser.contexts[0].new_page() page.goto(VIDEO_URL, wait_until="networkidle") page.wait_for_timeout(2000) @@ -109,7 +134,7 @@ print("UP:", soup.select_one(".up-name").get_text(strip=True) if soup.select_ print("Desc:", soup.select_one(".desc-info-text").get_text(" ", strip=True) if soup.select_one(".desc-info-text") else None) ``` -**Tip**: For B站 evaluations, the [公开 API](https://api.bilibili.com/x/web-interface/view?bvid=XXXX) often returns JSON without needing CDP. Try it first: +**Tip**: When the user did not require a specific browser, the [公开 API](https://api.bilibili.com/x/web-interface/view?bvid=XXXX) often returns JSON without needing CDP. Try it first. For an explicit external-browser request, ask before replacing that route with the API. ```bash curl -s "https://api.bilibili.com/x/web-interface/view?bvid=BVxxxxxxxxx" | python3 -m json.tool @@ -127,10 +152,10 @@ WEIBO_URL = "https://weibo.com/u/1234567890" # or /detail/xxx # article[aria-label="微博"] — each feed item ``` -**Note**: Weibo uses React + heavy obfuscation. Selectors change frequently. If selectors fail, pipe the HTML through Jina for clean Markdown: +**Note**: Weibo uses React + heavy obfuscation. If the user did not require a specific browser and selectors fail, Jina can clean the page. For an explicit external-browser request, ask before changing the execution route: ```python -html = fetch_with_cdp(WEIBO_URL) +html = fetch_with_cdp(WEIBO_URL, PROBE_PORT) # Save to temp file, then: import subprocess result = subprocess.run( @@ -157,7 +182,7 @@ Zhihu works with CDP but often also renders enough metadata server-side for Jina curl -sL "https://r.jina.ai/https://www.zhihu.com/question/123/answer/456" ``` -Try Jina first, fall back to CDP if content is truncated. +When no browser was specified, try Jina first and fall back to the built-in browser if content is truncated. For an explicit external-browser request, do not replace that route without user approval. ### 飞书文档 (feishu.cn / larksuite.com) @@ -170,7 +195,7 @@ DOC_URL = "https://xxx.feishu.cn/docs/xxx" from playwright.sync_api import sync_playwright with sync_playwright() as p: - browser = p.chromium.connect_over_cdp("http://localhost:9222") + browser = p.chromium.connect_over_cdp(cdp_url(PROBE_PORT)) page = browser.contexts[0].new_page() page.goto(DOC_URL, wait_until="domcontentloaded") page.wait_for_selector(".docs-render-unit", timeout=15000) @@ -259,7 +284,8 @@ page.wait_for_load_state("networkidle") ### Pattern 5: Clean HTML via Jina after extraction -When selectors are unreliable, dump the full page HTML and let Jina do the cleaning: +When no browser was specified and selectors are unreliable, dump the full page HTML and let Jina do +the cleaning. For an explicit external-browser request, ask before replacing that route: ```python html = page.content() @@ -276,27 +302,35 @@ print(clean_md) ## Troubleshooting -### `connect_over_cdp` fails with `ECONNREFUSED` +### `connect_over_cdp` fails after a `ready` probe -Chrome is not running with remote debugging. Tell the user: -> "请先用下面的命令启动 Chrome: -> `/Applications/Google\\ Chrome.app/Contents/MacOS/Google\\ Chrome --remote-debugging-port=9222 --user-data-dir=\"${DESIRECORE_ROOT}/chrome-profile\"` -> 然后手动登录需要抓取的网站,再让我继续。" +Do not guess that the browser is merely closed. Call `BrowserExternalProbe` again: + +- `debug_port_closed` → show its current `launchCommand` and wait for the user +- `browser_mismatch` → report the actual/requested products and ask the user to correct or approve the change +- `browser_not_installed` → report that exact installation fact; alternatives require explicit approval +- `invalid_cdp_endpoint` → tell the user the port is not a valid CDP endpoint +- still `ready` → report the Playwright attach failure separately; do not switch to the built-in browser ### `browser.contexts[0]` is empty -Chrome was launched but no windows are open. Ask the user to open at least one tab and navigate anywhere. +The approved external Chromium browser is running but no windows are open. Ask the user to open at least one tab and navigate anywhere. ### Playwright not installed ```bash -pip3 install playwright beautifulsoup4 -# No need for `playwright install` — we're attaching to existing Chrome, not downloading a new browser +python3 -m venv "${DESIRECORE_ROOT}/runtime/external-browser-playwright" +"${DESIRECORE_ROOT}/runtime/external-browser-playwright/bin/pip" install 'playwright==1.55.0' beautifulsoup4 +# No need for `playwright install` — we're attaching to an existing browser, not downloading one ``` +Keep the environment isolated to DesireCore; do not install Playwright globally. A missing Playwright +dependency does not change a `ready` browser/CDP result and never authorizes fallback to BrowserManage. + ### Site detects automation -Despite CDP attach, some sites (Cloudflare-protected, Instagram) may still detect automation. Options: +Despite CDP attach, some sites (Cloudflare-protected, Instagram) may still detect automation. If the +user explicitly requested their external browser, ask before changing routes. Otherwise, options are: 1. Use Jina Reader instead (`curl -sL https://r.jina.ai/`) — often succeeds where Playwright fails 2. Ask the user to manually copy the visible content 3. Use the site's public API if available @@ -307,7 +341,9 @@ The page uses virtualization or lazy loading. Apply Pattern 1 (scroll to bottom) ### `page.wait_for_selector` times out -The selector is stale — the site updated its DOM. Dump `page.content()[:5000]` and inspect manually, or fall back to Jina Reader. +The selector is stale — the site updated its DOM. Dump `page.content()[:5000]` and inspect manually. +Only fall back to Jina Reader when no browser was specified; for an explicit external-browser request, +ask before changing the execution route. --- @@ -316,8 +352,8 @@ The selector is stale — the site updated its DOM. Dump `page.content()[:5000]` - **Never log or print cookies** from `context.cookies()` even during debugging - **Never extract and store** the user's session tokens to files - **Never use the CDP session** to perform writes (post, comment, like) unless the user explicitly requested it -- The `${DESIRECORE_ROOT}/chrome-profile` directory contains the user's credentials — treat it as sensitive -- If the user asks to "log in automatically", refuse and explain they must log in manually in the Chrome window; the skill only reads already-authenticated sessions +- `${DESIRECORE_ROOT}/browser/external-profiles/` contains the manually established isolated login state — treat it as sensitive +- If the user asks to "log in automatically", refuse and explain they must log in manually in the approved external browser window; the skill only reads already-authenticated sessions ---