feat(annotate): 框内文字标记[1]为最可能目标;用户消息内提示精简为一行(已定位直接作答),完整协议细节下沉知识包

This commit is contained in:
Aisland
2026-09-01 23:55:05 +08:00
parent 2d034732ff
commit d8b209ec92
3 changed files with 8 additions and 8 deletions
+4 -4
View File
@@ -16744,7 +16744,7 @@ function fillHostInput(text2) {
}
function confirmAnnot() {
const s = annotState;
const text2 = "\u{1F4CC} \u6807\u6CE8-" + s.resp + "\n" + (s.stat || "") + "\n\u8981\u6C42\uFF1A" + (s.draft.trim() || "\uFF08\u672A\u586B\u5199\uFF09") + "\n\u2014\u2014\u3010\u5DE5\u4F5C\u53F0\u6807\u6CE8\u3011\u8FD9\u662F\u7ED9 AI \u7684\u5B9A\u4F4D\u6807\u6CE8\uFF1A\u2460 \u80FD\u67E5\u770B\u622A\u56FE\u6216\u6253\u5F00\u5BF9\u5E94\u7A97\u53E3\uFF08" + s.resp + "\uFF09\u5C31\u5148\u6838\u5B9E\u4F4D\u7F6E\u518D\u56DE\u7B54\uFF1B\u2461 \u4E0D\u80FD\u67E5\u770B\u753B\u9762\u65F6\uFF0C\u53EA\u95EE\u4E00\u6761\u6700\u5173\u952E\u7684\u95EE\u9898\u786E\u8BA4\u4F4D\u7F6E\uFF0C\u4E0D\u8981\u6CDB\u6CDB\u731C\u6D4B\u3002";
const text2 = "\u{1F4CC} \u6807\u6CE8-" + s.resp + "\n" + (s.stat || "") + "\n\u8981\u6C42\uFF1A" + (s.draft.trim() || "\uFF08\u672A\u586B\u5199\uFF09") + "\n\u3010\u5DE5\u4F5C\u53F0\u6807\u6CE8\xB7\u5DF2\u5B9A\u4F4D\uFF0C\u76F4\u63A5\u4F5C\u7B54\uFF0C\u65E0\u9700\u518D\u95EE\uFF1B\u4EC5\u5F53\u786E\u5B9E\u65E0\u6CD5\u4ECE\u6807\u6CE8\u5B9A\u4F4D\u65F6\u624D\u95EE\u4E00\u6B21\u3011";
const ok = fillHostInput(text2);
if (!ok) {
try {
@@ -16811,7 +16811,7 @@ function AnnotationOverlay() {
stat += "\uFF0C\u7A97\u53E3\u5185 (" + rx1 + "%," + ry1 + "%)\u2192(" + rx2 + "%," + ry2 + "%)";
}
const texts = elementsInBox(L, T2, R, B);
if (texts.length > 0) stat += "\uFF0C\u6846\u5185\u6587\u5B57\uFF1A" + texts.map((t, i) => "[" + (i + 1) + "]" + t).join(" ");
if (texts.length > 0) stat += "\uFF0C\u6846\u5185\u6587\u5B57\uFF1A" + texts.map((t, i) => "[" + (i + 1) + "]" + t + (i === 0 ? "\u2190\u6700\u53EF\u80FD\u76EE\u6807" : "")).join(" ");
} else {
stat = "\u5C4F\u5E55\u5750\u6807 (" + Math.round(e.clientX) + ", " + Math.round(e.clientY) + ")";
if (paneEl) {
@@ -20045,8 +20045,8 @@ var KNOWLEDGE_PACK = [
" /api/worktable/git\uFF08git \u72B6\u6001\uFF09\u3001/api/worktable/site\uFF08\u9759\u6001\u6258\u7BA1\uFF09\u3002",
"- \u6539\u5B8C\u63D2\u4EF6\u5728 01_content \u76EE\u5F55\u6267\u884C npm run build\uFF1B\u91CD\u542F dsh web \u6216\u6D4F\u89C8\u5668 F5 \u751F\u6548\u3002",
"- \u6240\u6709\u4EA7\u51FA\u6587\u4EF6\u4E00\u5F8B\u653E\u8FDB\u672C\u4EFB\u52A1\u6807\u6CE8\u7684\u9879\u76EE\u6587\u4EF6\u5939\uFF0C\u4FDD\u6301\u7528\u6237\u76EE\u5F55\u5E72\u51C0\u3002",
"- \u6807\u6CE8\u534F\u8BAE\uFF1A\u6536\u5230\u300C\u{1F4CC} \u6807\u6CE8-\u7A97\u53E3N \u2026\uFF08\u5750\u6807/\u5143\u7D20\uFF09\u2026\u8981\u6C42\uFF1A\u2026\u300D= \u5DE5\u4F5C\u53F0\u6807\u6CE8\uFF08\u7A97\u53E3\u7F16\u53F7\u7EA6\u5B9A\uFF1A\u5DE6\u680F\u2192\u9876\u884C\u2192\u4E3B\u884C\uFF0C\u4ECE 1 \u8D77\uFF09\u3002",
" \u5904\u7406\uFF1A\u80FD\u67E5\u770B\u622A\u56FE\u6216\u6253\u5F00\u5BF9\u5E94\u7A97\u53E3\u5219\u5148\u6838\u5B9E\u4F4D\u7F6E\u518D\u56DE\u7B54\uFF1B\u4E0D\u80FD\u67E5\u770B\u753B\u9762\u65F6\uFF0C\u53EA\u95EE\u4E00\u6761\u6700\u5173\u952E\u7684\u95EE\u9898\u786E\u8BA4\u4F4D\u7F6E\uFF0C\u4E0D\u8981\u6CDB\u6CDB\u731C\u6D4B\u3002"
"- \u6807\u6CE8\u534F\u8BAE\uFF1A\u6536\u5230\u300C\u{1F4CC} \u6807\u6CE8-\u7A97\u53E3N \u2026\uFF08\u5750\u6807/\u5143\u7D20/\u6846\u5185\u6587\u5B57\u2026\uFF09\u2026\u300D= \u5DE5\u4F5C\u53F0\u6807\u6CE8\uFF08\u7A97\u53E3\u7F16\u53F7\u7EA6\u5B9A\uFF1A\u5DE6\u680F\u2192\u9876\u884C\u2192\u4E3B\u884C\uFF0C\u4ECE 1 \u8D77\uFF09\u3002",
" \u5904\u7406\uFF1A\u6807\u6CE8\u5DF2\u6807\u660E\u6846\u9009\u533A\u57DF\u4E0E\u300C\u6700\u53EF\u80FD\u76EE\u6807\u300D\u65F6\u76F4\u63A5\u4F5C\u7B54\u3001\u4E0D\u8981\u4E8C\u6B21\u8BE2\u95EE\uFF08\u4EC5\u5F53\u786E\u5B9E\u65E0\u6CD5\u4ECE\u6807\u6CE8\u5B9A\u4F4D\u65F6\u624D\u95EE\u4E00\u6B21\uFF09\uFF1B\u80FD\u67E5\u770B\u622A\u56FE\u6216\u6253\u5F00\u7A97\u53E3\u5219\u5148\u6838\u5B9E\u518D\u56DE\u7B54\u3002"
].join("\n");
function buildWindowTaskText(projectId, projectName, windowLabel, requirement, folder, mode) {
const win = windowLabel || "\u4E00\u4E2A\u5185\u5BB9\u7A97";
+2 -2
View File
@@ -793,8 +793,8 @@ const KNOWLEDGE_PACK = [
' /api/worktable/git(git 状态)、/api/worktable/site(静态托管)。',
'- 改完插件在 01_content 目录执行 npm run build;重启 dsh web 或浏览器 F5 生效。',
'- 所有产出文件一律放进本任务标注的项目文件夹,保持用户目录干净。',
'- 标注协议:收到「📌 标注-窗口N …(坐标/元素)…要求:…」= 工作台标注(窗口编号约定:左栏→顶行→主行,从 1 起)。',
' 处理:能查看截图或打开对应窗口则先核实位置再回答;不能查看画面时,只问一条最关键的问题确认位置,不要泛泛猜测。',
'- 标注协议:收到「📌 标注-窗口N …(坐标/元素/框内文字…)…」= 工作台标注(窗口编号约定:左栏→顶行→主行,从 1 起)。',
' 处理:标注已标明框选区域与「最可能目标」时直接作答、不要二次询问(仅当确实无法从标注定位时才问一次);能查看截图或打开窗口则先核实再回答。',
].join('\n')
/** 组装窗口任务提示词(新建/发送两模式共用;导出到 window 供自测校验) */
+2 -2
View File
@@ -409,7 +409,7 @@ function fillHostInput(text: string): boolean {
function confirmAnnot() {
const s = annotState
const text = '📌 标注-' + s.resp + '\n' + (s.stat || '') + '\n要求:' + (s.draft.trim() || '(未填写)')
+ '\n——【工作台标注】这是给 AI 的定位标注:① 能查看截图或打开对应窗口(' + s.resp + ')就先核实位置再回答;② 不能查看画面时,只问一条最关键的问题确认位置,不要泛泛猜测。'
+ '\n【工作台标注·已定位,直接作答,无需再问;仅当确实无法从标注定位时才问一次】'
const ok = fillHostInput(text)
if (!ok) {
try {
@@ -471,7 +471,7 @@ function AnnotationOverlay() {
stat += ',窗口内 (' + rx1 + '%,' + ry1 + '%)→(' + rx2 + '%,' + ry2 + '%)'
}
const texts = elementsInBox(L, T, R, B)
if (texts.length > 0) stat += ',框内文字:' + texts.map((t, i) => '[' + (i + 1) + ']' + t).join(' ')
if (texts.length > 0) stat += ',框内文字:' + texts.map((t, i) => '[' + (i + 1) + ']' + t + (i === 0 ? '←最可能目标' : '')).join(' ')
} else {
stat = '屏幕坐标 (' + Math.round(e.clientX) + ', ' + Math.round(e.clientY) + ')'
if (paneEl) {