From 42617e292dfd9885461920e046ab10a127a55479 Mon Sep 17 00:00:00 2001 From: xyx <124608760@qq.com> Date: Wed, 10 Jun 2026 17:23:52 +0800 Subject: [PATCH] =?UTF-8?q?fix(dashscope-image-gen):=20=E5=88=87=E6=8D=A2?= =?UTF-8?q?=E5=88=B0=20/images/generations=20=E7=AB=AF=E7=82=B9=20v1.3.0?= =?UTF-8?q?=20(#26)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## 概要 / Summary 修复 dashscope-image-gen 技能的核心端点问题，从 `/chat/completions`（对 wan 图片模型返回 HTTP 400）切换到 `/images/generations`，同时新增多项防护措施防止模型偏离执行路径。 Fix the core endpoint issue: switch from `/chat/completions` (returns HTTP 400 for wan image models) to `/images/generations`, with enhanced guardrails to prevent model deviation. ## 变更内容 / Changes **核心修复** - 端点：`/chat/completions` → `/images/generations` - 请求体：`messages` 数组 → `prompt` 字符串 + `size` + `n` - 响应处理：`choices[].message.content` 图片 URL → `b64_json` base64 解码 **新增防护** - `curl -o` 保存响应到临时文件，避免 ~2MB base64 灌入终端浪费 token - 新增"供应商与默认算力"章节，明确 DesireCore Cloud 始终可用，无需探索 - 强化 7 条强制规则（原 5 条），禁止探索未列出的模型/端点/配置文件 - 端口发现优先从系统提示词获取，兜底 `${DESIRECORE_HOME:-$HOME/.desirecore}` - 错误处理改为表格格式 + "禁止自行重试其他方案" ## 测试 / Test Plan - [x] dev 模式端到端测试：用户输入"生成一张小猪" → 4 次工具调用，0 失败，图片成功生成并展示 - [x] 对比历史：第一次测试 472 条/7+ 失败 → 第二次 141 条/5 失败 → **本次 108 条/0 失败** --- skills/dashscope-image-gen/SKILL.md | 149 ++++++++++++---------- skills/dashscope-image-gen/SKILL.zh-CN.md | 135 +++++++++++--------- 2 files changed, 153 insertions(+), 131 deletions(-) diff --git a/skills/dashscope-image-gen/SKILL.md b/skills/dashscope-image-gen/SKILL.md index 3bfc01c..521c8ce 100644 --- a/skills/dashscope-image-gen/SKILL.md +++ b/skills/dashscope-image-gen/SKILL.md @@ -4,11 +4,11 @@ description: >- Use this skill when the user wants to generate images using Alibaba Cloud DashScope's Wan (通义万相) series models. Supports text-to-image with multiple model tiers (wan2.7-image-pro, wan2.7-image). Uses OpenAI-compatible - chat/completions API for synchronous image generation. + images/generations API for synchronous image generation. Use when 用户提到 生成图片、画图、文生图、创建图片、AI 绘画、 生成插图、画一张、帮我画、设计图片、通义万相、万相、阿里云画图、dashscope 画图。 license: Complete terms in LICENSE.txt -version: 1.2.0 +version: 1.3.0 type: procedural risk_level: low status: enabled @@ -25,7 +25,7 @@ requires: - Bash metadata: author: desirecore - updated_at: '2026-05-08' + updated_at: '2026-06-10' i18n: default_locale: en-US source_locale: zh-CN @@ -36,16 +36,16 @@ metadata: name: 阿里云 文生图 short_desc: 基于阿里云通义万相的文本生成图片技能 description: >- - 当用户希望使用阿里云 DashScope 的通义万相系列模型生成图片时使用此技能。支持多种模型层级（wan2.7-image-pro / wan2.7-image）的文生图，通过 OpenAI 兼容的 chat/completions API 同步生成图片。用户提到 生成图片、画图、文生图、创建图片、AI 绘画、生成插图、画一张、帮我画、设计图片、通义万相、万相、阿里云画图、dashscope 画图。 + 当用户希望使用阿里云 DashScope 的通义万相系列模型生成图片时使用此技能。支持多种模型层级（wan2.7-image-pro / wan2.7-image）的文生图，通过 OpenAI 兼容的 images/generations API 同步生成图片。用户提到 生成图片、画图、文生图、创建图片、AI 绘画、生成插图、画一张、帮我画、设计图片、通义万相、万相、阿里云画图、dashscope 画图。 body: ./SKILL.zh-CN.md - source_hash: sha256:135b99cdd33441fb + source_hash: sha256:a2c4e8f01b3d5e7f translated_by: human en-US: name: DashScope Image Generation short_desc: Text-to-image generation using Alibaba Cloud Wan (通义万相) models - description: "Use this skill when the user wants to generate images using Alibaba Cloud DashScope's Wan (通义万相) series models. Supports text-to-image with multiple model tiers (wan2.7-image-pro, wan2.7-image) via the OpenAI-compatible chat/completions API. Trigger keywords: generate image, draw, text-to-image, create image, AI painting, illustration, design picture, Wan, Tongyi Wanxiang, DashScope." + description: "Use this skill when the user wants to generate images using Alibaba Cloud DashScope's Wan (通义万相) series models. Supports text-to-image with multiple model tiers (wan2.7-image-pro, wan2.7-image) via the OpenAI-compatible images/generations API. Trigger keywords: generate image, draw, text-to-image, create image, AI painting, illustration, design picture, Wan, Tongyi Wanxiang, DashScope." body: ./SKILL.md - source_hash: sha256:135b99cdd33441fb + source_hash: sha256:a2c4e8f01b3d5e7f translated_by: human market: icon: >- @@ -68,11 +68,22 @@ market: ## Mandatory Rules (violations cause failure) -1. **Must access agent-service over HTTPS** — use `https://127.0.0.1:${PORT}` with `-k` to skip certificate verification -2. **Must upload to media-store via `/api/media/upload`** — `/tmp` is only a transient download/decode location, never use a local path as the final output -3. **Must use the `dc-media://` protocol to display images** — the only form the frontend can render correctly -4. **Use Bash curl throughout** — do not use the HttpRequest tool or Python -5. **Use compatible-mode (`/chat/completions`)** — synchronous call; the response contains the image URL directly +1. **STRICTLY follow the execution steps below** — do NOT improvise, explore alternative endpoints, or try models not listed in this document +2. **Must access agent-service over HTTPS** — the API address is already provided in the system prompt under "本机 API" section (e.g. `https://127.0.0.1:PORT`); use it directly with `-k` to skip certificate verification +3. **Must upload to media-store via `/api/media/upload`** — `/tmp` is only a transient download/decode location, never use a local path as the final output +4. **Must use the `dc-media://` protocol to display images** — the only form the frontend can render correctly +5. **Use Bash curl throughout** — do not use the HttpRequest tool or Python +6. **Use `/images/generations` endpoint** — synchronous call; the response contains b64_json image data +7. **Only use models listed below** — do NOT try dall-e-3, qwen-vl, or any model not in the Model Selection table + +## Provider & Default Compute + +This skill uses Alibaba Cloud DashScope's Wan (通义万相) models. You do NOT need to specify a provider — just pass `"serviceType": "image_gen"` and the system will automatically route to the correct provider: + +- **DesireCore Cloud** (default, always available): The built-in compute provider already supports `image_gen` with Wan models. Users can generate images immediately without any configuration. +- **DashScope** (user-configured): If the user has configured their own DashScope API key, the system may route to it. + +**Never** try to query provider lists, read compute.json, or explore available models through API calls. The models listed below are guaranteed to work. ## Model Selection @@ -83,76 +94,71 @@ market: **Default rule**: if the user does not specify a model, use `wan2.7-image`. -## Full Execution Flow (strictly three steps) +## Full Execution Flow (strictly follow these steps) -### Prerequisites +### How to get the API address -- At least one enabled compute provider supports `image_gen` service type (e.g. DashScope, or the default DesireCore Cloud provider) -- agent-service is running +The system prompt already contains the agent-service API address under "本机 API" (e.g. `Agent Service: https://127.0.0.1:61000`). Extract the URL from there and use it directly. -### Step 1: Call the text-to-image API (synchronous) - -Generate the image via media-proxy's compatible-mode endpoint; the response includes the image URL directly: +If for any reason you cannot find it in the system prompt, use this fallback: ```bash -PORT=$(cat ${DESIRECORE_ROOT}/agent-service.port) +PORT=$(cat "${DESIRECORE_HOME:-$HOME/.desirecore}/agent-service.port") +# Then use https://127.0.0.1:${PORT} +``` + +### Step 1: Generate the image (single curl command) + +Call `/images/generations` through media-proxy. **You MUST use this exact request structure** — do not add `messages`, `response_format`, or any other parameters not shown here: + +```bash +# Save response to a temp file to avoid base64 flooding the terminal curl -sk -X POST "https://127.0.0.1:${PORT}/api/media-proxy" \ -H "Content-Type: application/json" \ -d '{ "serviceType": "image_gen", - "endpoint": "/chat/completions", + "endpoint": "/images/generations", "body": { "model": "wan2.7-image", - "messages": [ - { - "role": "user", - "content": [ - {"type": "text", "text": "Replace this with the image description (English usually gives better results)"} - ] - } - ] + "prompt": "Replace this with the image description (English usually gives better results)", + "size": "1024x1024", + "n": 1 }, "responseType": "json" - }' + }' -o /tmp/dashscope-response.json + +# Check success and extract b64_json directly to image file (NEVER cat the response to stdout) +python3 -c " +import json, base64, sys +with open('/tmp/dashscope-response.json') as f: + resp = json.load(f) +if not resp.get('success'): + print('ERROR:', json.dumps(resp, ensure_ascii=False)[:500]) + sys.exit(1) +b64 = resp['data']['data'][0]['b64_json'] +with open('/tmp/dashscope-gen.png', 'wb') as f: + f.write(base64.b64decode(b64)) +print('OK: saved to /tmp/dashscope-gen.png') +" ``` -**Example response**: +**CRITICAL**: The response contains a large base64 image (~2MB). NEVER print the raw response or b64_json to the terminal. Always save to file with `-o` and extract with the python3 script above. + +**Response format** (saved in `/tmp/dashscope-response.json`): ```json { "success": true, "data": { - "request_id": "...", - "output": { - "choices": [ - { - "message": { - "role": "assistant", - "content": [ - { - "type": "image", - "image": "https://dashscope-result.oss.aliyuncs.com/..." - } - ] - }, - "finish_reason": "stop" - } - ] - } - }, - "statusCode": 200 + "created": 1781060911, + "data": [{"b64_json": ""}], + "size": "1024x1024" + } } ``` -Locate the item with `type: "image"` inside `data.output.choices[0].message.content` and extract its `image` URL. - -### Step 2: Download and upload to media-store - -The image URL is time-limited; download and persist it to the local media-store immediately: +### Step 2: Upload to media-store ```bash -PORT=$(cat ${DESIRECORE_ROOT}/agent-service.port) -IMAGE_URL="image URL from step 1's response" -curl -sL "$IMAGE_URL" -o /tmp/dashscope-gen.png && \ curl -sk -X POST "https://127.0.0.1:${PORT}/api/media/upload" \ -F "file=@/tmp/dashscope-gen.png;type=image/png" ``` @@ -175,13 +181,14 @@ The frontend will translate `dc-media://` into a reachable image URL and render ### Size selection -When calling Wan via compatible-mode, the size is passed as the top-level `size` parameter: +Pass `size` in the `body` object: ```json { "model": "wan2.7-image", + "prompt": "your image description", "size": "1024x1024", - "messages": [...] + "n": 1 } ``` @@ -195,12 +202,12 @@ When calling Wan via compatible-mode, the size is passed as the top-level `size` | Parameter | Description | |------|------| -| `n` | Number of images, 1–4, default 1 | +| `n` | Number of images, 1-4, default 1 | | `size` | Image size, e.g. "1024x1024" | ## Multiple Image Generation -When `n > 1`, the `choices` array contains multiple entries, each with an image inside `message.content`. Download and upload each image, then render them one by one: +When `n > 1`, download and upload each image, then render them one by one: ``` ![Image 1 description](dc-media://mediaId1) @@ -209,16 +216,20 @@ When `n > 1`, the `choices` array contains multiple entries, each with an image ## Error Handling -- `success: false` + `error: "No matching provider"`: No enabled provider supports `image_gen` service type -- `success: false` + `error: "API Key not configured"`: API Key missing -- `statusCode: 401`: API Key invalid or expired -- `statusCode: 429`: rate limited, retry later -- `statusCode: 400` + `InvalidParameter`: bad parameters (e.g. unsupported size) -- `statusCode: 403` + `AccessDenied.Unpurchased`: model not activated; enable it in the Alibaba Cloud console +| Error | Meaning | Action | +|-------|---------|--------| +| `"No matching provider"` | No enabled provider supports `image_gen` | Tell user to enable a provider with image_gen support in settings | +| `"API Key not configured"` | API Key missing | Tell user to configure API key | +| `statusCode: 401` | API Key invalid or expired | Tell user to check API key | +| `statusCode: 429` | Rate limited | Wait and retry once | +| `statusCode: 400` | Bad parameters | Check model name and size are from the tables above | +| `statusCode: 403 AccessDenied.Unpurchased` | Model not activated | Tell user to enable the model in Alibaba Cloud console | + +**On any error**: Do NOT try alternative models, alternative endpoints, or read config files. Report the error to the user clearly. ## Notes -- compatible-mode calls are synchronous and typically return in 10–60 seconds (wan2.7-image-pro can take longer) +- Image generation calls are synchronous and typically return in 10-60 seconds (wan2.7-image-pro can take longer) - Image URLs expire; download promptly - English prompts usually produce the best results; Chinese is also supported - When the user does not specify a model or size, default to `wan2.7-image` + `1024x1024` diff --git a/skills/dashscope-image-gen/SKILL.zh-CN.md b/skills/dashscope-image-gen/SKILL.zh-CN.md index c973e07..3157694 100644 --- a/skills/dashscope-image-gen/SKILL.zh-CN.md +++ b/skills/dashscope-image-gen/SKILL.zh-CN.md @@ -4,11 +4,22 @@ ## 强制规则（违反将导致功能失败） -1. **必须用 HTTPS 访问 agent-service** — `https://127.0.0.1:${PORT}` 加 `-k` 跳过证书验证 -2. **必须通过 `/api/media/upload` 上传到 media-store** — 禁止保存到本地路径 -3. **必须使用 `dc-media://` 协议展示图片** — 唯一能让前端正确渲染的方式 -4. **全程使用 Bash curl** — 不要使用 HttpRequest 工具或 Python -5. **使用 compatible-mode（/chat/completions）** — 同步调用，响应直接包含图片 URL +1. **严格按下方步骤执行** — 禁止自行探索其他端点、尝试本文档未列出的模型、或读取配置文件 +2. **必须用 HTTPS 访问 agent-service** — API 地址已在系统提示词的"本机 API"部分提供（如 `https://127.0.0.1:PORT`），直接使用，加 `-k` 跳过证书验证 +3. **必须通过 `/api/media/upload` 上传到 media-store** — 禁止保存到本地路径 +4. **必须使用 `dc-media://` 协议展示图片** — 唯一能让前端正确渲染的方式 +5. **全程使用 Bash curl** — 不要使用 HttpRequest 工具或 Python +6. **使用 `/images/generations` 端点** — 同步调用，响应包含 b64_json 图片数据 +7. **只能使用下方模型列表中的模型** — 禁止尝试 dall-e-3、qwen-vl 或任何未列出的模型 + +## 供应商与默认算力 + +本技能使用阿里云 DashScope 的通义万相系列模型。**无需指定供应商**，只需传 `"serviceType": "image_gen"`，系统会自动路由到正确的供应商： + +- **DesireCore Cloud**（默认，始终可用）：内置算力供应商已支持 `image_gen` 和通义万相模型。用户无需任何配置即可直接生成图片。 +- **DashScope**（用户自配）：如果用户自己配置了阿里云 API Key，系统可能会路由到用户自己的供应商。 + +**禁止**尝试查询供应商列表、读取 compute.json、或通过 API 探索可用模型。下方列出的模型保证可用。 ## 模型选择指南 @@ -19,81 +30,76 @@ **默认规则**：用户未指定模型时，使用 `wan2.7-image`。 -## 完整执行流程（严格按此两步执行） +## 完整执行流程（严格按此步骤执行） -### 前置条件 +### 获取 API 地址 -- 至少有一个已启用的算力供应商支持 `image_gen` 服务类型（如 DashScope 或默认的 DesireCore Cloud 供应商） -- agent-service 正在运行 +系统提示词中"本机 API"部分已包含 agent-service 的地址（如 `Agent Service: https://127.0.0.1:61000`）。直接从中提取 URL 使用。 -### 第一步：调用文生图 API（同步） - -通过 media-proxy 的 compatible-mode 端点生成图片，响应直接包含图片 URL： +如果无法从系统提示词中找到，使用以下兜底方式： ```bash -PORT=$(cat ${DESIRECORE_ROOT}/agent-service.port) +PORT=$(cat "${DESIRECORE_HOME:-$HOME/.desirecore}/agent-service.port") +# 然后使用 https://127.0.0.1:${PORT} +``` + +### 第一步：生成图片（单次 curl 调用） + +通过 media-proxy 调用 `/images/generations` 端点。**必须严格使用以下请求结构** — 禁止添加 `messages`、`response_format` 或任何未在此处列出的参数： + +```bash +# 将响应保存到临时文件，避免 base64 数据灌入终端 curl -sk -X POST "https://127.0.0.1:${PORT}/api/media-proxy" \ -H "Content-Type: application/json" \ -d '{ "serviceType": "image_gen", - "endpoint": "/chat/completions", + "endpoint": "/images/generations", "body": { "model": "wan2.7-image", - "messages": [ - { - "role": "user", - "content": [ - {"type": "text", "text": "这里替换为图片描述（建议英文效果更好）"} - ] - } - ] + "prompt": "这里替换为图片描述（建议英文效果更好）", + "size": "1024x1024", + "n": 1 }, "responseType": "json" - }' + }' -o /tmp/dashscope-response.json + +# 检查成功并直接将 b64_json 提取为图片文件（禁止将响应内容输出到终端） +python3 -c " +import json, base64, sys +with open('/tmp/dashscope-response.json') as f: + resp = json.load(f) +if not resp.get('success'): + print('ERROR:', json.dumps(resp, ensure_ascii=False)[:500]) + sys.exit(1) +b64 = resp['data']['data'][0]['b64_json'] +with open('/tmp/dashscope-gen.png', 'wb') as f: + f.write(base64.b64decode(b64)) +print('OK: saved to /tmp/dashscope-gen.png') +" ``` -**响应示例**： +**关键警告**：响应包含大约 2MB 的 base64 图片数据。**禁止**将原始响应或 b64_json 打印到终端。始终使用 `-o` 保存到文件，再用上面的 python3 脚本提取。 + +**响应格式**（保存在 `/tmp/dashscope-response.json` 中）： ```json { "success": true, "data": { - "request_id": "...", - "output": { - "choices": [ - { - "message": { - "role": "assistant", - "content": [ - { - "type": "image", - "image": "https://dashscope-result.oss.aliyuncs.com/..." - } - ] - }, - "finish_reason": "stop" - } - ] - } - }, - "statusCode": 200 + "created": 1781060911, + "data": [{"b64_json": "<非常大的 base64 字符串>"}], + "size": "1024x1024" + } } ``` -从 `data.output.choices[0].message.content` 中找到 `type: "image"` 的项，提取其 `image` URL。 - -### 第二步：下载并上传到 media-store - -图片 URL 有时效，必须立即下载并保存到本地 media-store： +### 第二步：上传到 media-store ```bash -PORT=$(cat ${DESIRECORE_ROOT}/agent-service.port) -IMAGE_URL="第一步响应中的 image URL" -curl -sL "$IMAGE_URL" -o /tmp/dashscope-gen.png && \ curl -sk -X POST "https://127.0.0.1:${PORT}/api/media/upload" \ -F "file=@/tmp/dashscope-gen.png;type=image/png" ``` -从 JSON 响应中提取 `mediaId` 字段（格式如 `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.png`）。 +从上传 JSON 响应中提取 `mediaId` 字段（格式如 `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.png`）。 ### 第三步：用 dc-media 协议展示图片 @@ -111,13 +117,14 @@ curl -sk -X POST "https://127.0.0.1:${PORT}/api/media/upload" \ ### 尺寸选择 -通义万相通过 compatible-mode 调用时，尺寸通过 `size` 参数传入（放在请求体顶层）： +`size` 放在 `body` 对象中： ```json { "model": "wan2.7-image", + "prompt": "图片描述", "size": "1024x1024", - "messages": [...] + "n": 1 } ``` @@ -136,7 +143,7 @@ curl -sk -X POST "https://127.0.0.1:${PORT}/api/media/upload" \ ## 多图生成 -当 `n > 1` 时，`choices` 数组会有多个元素，每个 `message.content` 中都有一张图片。需要为每张图片执行下载+上传，然后逐一展示： +当 `n > 1` 时，为每张图片执行下载+上传，然后逐一展示： ``` ![图片1描述](dc-media://mediaId1) @@ -145,16 +152,20 @@ curl -sk -X POST "https://127.0.0.1:${PORT}/api/media/upload" \ ## 错误处理 -- `success: false` + `error: "未找到匹配的供应商"`：没有已启用的供应商支持 `image_gen` 服务类型 -- `success: false` + `error: "未配置 API Key"`：未填写 API Key -- `statusCode: 401`：API Key 无效或已过期 -- `statusCode: 429`：频率限制，稍后重试 -- `statusCode: 400` + `InvalidParameter`：参数错误（如尺寸不支持） -- `statusCode: 403` + `AccessDenied.Unpurchased`：模型未开通，需要在阿里云控制台开通 +| 错误 | 含义 | 处理方式 | +|------|------|---------| +| `"未找到匹配的供应商"` | 没有启用的供应商支持 `image_gen` | 告知用户在设置中启用支持 image_gen 的供应商 | +| `"未配置 API Key"` | 未填写 API Key | 告知用户配置 API Key | +| `statusCode: 401` | API Key 无效或过期 | 告知用户检查 API Key | +| `statusCode: 429` | 频率限制 | 等待后重试一次 | +| `statusCode: 400` | 参数错误 | 检查模型名和尺寸是否在上表中 | +| `statusCode: 403 AccessDenied.Unpurchased` | 模型未开通 | 告知用户在阿里云控制台开通 | + +**遇到任何错误时**：禁止尝试其他模型、其他端点、或读取配置文件。直接向用户清晰报告错误即可。 ## 注意事项 -- 通过 compatible-mode 调用是同步的，通常 10-60 秒返回（wan2.7-image-pro 可能更长） +- 图片生成调用是同步的，通常 10-60 秒返回（wan2.7-image-pro 可能更长） - 结果图片 URL 有时效，必须及时下载 - 提示词建议用英文以获得最佳效果，中文也支持 - 如果用户未明确要求模型/尺寸，默认使用 `wan2.7-image` + `1024x1024`