Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2882,7 +2882,7 @@ Known limitations: printing a raw-HTML annotate session prints highlight stripes

An agent can ask the reviewer a question inside a plan or document with a `:::question` (pick one), `:::question-multi` (pick any) or `:::question-text` (free text) directive; the reviewer answers on a card in place. Approved design: a spec plus the owner's approval, which overrides it (design record, not in this repo). Shipped as core+ui (#1637), editor (#1638) and the agent side (prompt, skill, archive, WebMCP).

- **Grammar and data** live in `@plannotator/core/question-block` (shim `packages/shared/question-block.ts`, vendored to Pi): `parseQuestionBlock` (tolerant, returns null rather than throwing; an unparseable block falls back to the ordinary `Callout`), `indexQuestionBlocks`, `questionKey` (`q-` + hash8 of kind + prompt, so a reworded prompt is a new question), `QuestionAnswer` + the fail-closed `parseQuestionAnswer`, `questionStatus`, `formatQuestionAnswersSection`, and `QUESTION_AUTHORING_GUIDE`. `- [x]` means **settled** (the agent's echo of an earlier answer), `Recommended:` names the recommendation, "Other…", note and skip are always offered, and nothing is required. A `Replaces: <message id>/<key>` line (the Inbox's replaced questions) is read into `replaces` only when every token on it is id-shaped; it is then never the prompt or context and does not change the key, and any other `Replaces:` line stays prose.
- **Grammar and data** live in `@plannotator/core/question-block` (shim `packages/shared/question-block.ts`, vendored to Pi): `parseQuestionBlock` (tolerant, returns null rather than throwing; an unparseable block falls back to the ordinary `Callout`), `indexQuestionBlocks`, `questionKey` (`q-` + hash8 of kind + prompt, so a reworded prompt is a new question), `QuestionAnswer` + the fail-closed `parseQuestionAnswer`, `questionStatus`, `formatQuestionAnswersSection`, and `QUESTION_AUTHORING_GUIDE`. `- [x]` means **settled** (the agent's echo of an earlier answer), `Recommended:` names the recommendation, "Other…", note and skip are always offered, and nothing is required. A `Replaces: <message id>/<key>` line (the Inbox's replaced questions) is read into `replaces` only when every token on it is id-shaped; it is then never the prompt or context and does not change the key, and any other `Replaces:` line stays prose. A picked label is kept whole: `parseQuestionAnswer` caps each `selected` label at `MAX_QUESTION_SELECTED_CHARS` (the body cap), never at the 200-character `MAX_QUESTION_CHOICE_LABEL_CHARS` (that one only shortens a plain-bullet label at a sentence end), because task-list labels are never cut and a shorter cap made a long choice unpickable (the Inbox answered `422 … is not one of the question's choices`; fixed after 0.28.17). `canonicalQuestionAnswer` maps an answer cut at 200 by an older build back to its one choice.
- **An answer is one annotation** (`Annotation.questionAnswer`, id `ann-question-<key>`), so drafts, reload, undo, the panel and host persistence carry it with no second store. The highlighter skips these rows; every exporter prints them first under "Answers to your questions" (with an "Unanswered" list) and excludes them from the numbered feedback count. The editor adds the "N/M answered" header chip, the Questions section in the annotations panel, and plan review's **Send answers** primary (`isAnswersOnlyFeedback` in `packages/editor/questionDecision.ts`).
- **`plan.answered`.** Send answers posts `/api/deny` with `answersOnly: true` (`planDenyFeedbackFields`) and an unframed export. Both plan servers pass the flag onto the decision only when it is boolean `true`; every plan-deny consumer (the Claude Code / Codex / Gemini / Copilot / Vibe hook outputs and the `opencode-plan` JSON in `apps/hook/server/index.ts`, OpenCode 1 and 2 through `submit-plan-executor.ts`, Pi's `index.ts`) builds its message with `composePlanDeniedMessage` (`packages/shared/prompts.ts`), which picks `prompts.plan.answered` (default `DEFAULT_PLAN_ANSWERED_PROMPT`: "The user answered the questions in your plan…", plus the remove-or-`[x]` resubmit rule) for an answers-only deny and the unchanged denied prompt otherwise. A custom `plan.denied` does not replace the answered prompt. A deny that mixes answers with comments or edits keeps the denied prompt. Compatibility: a server that predates the flag ignores it and wraps the same export in its denied prompt; an older UI never sends it.
- **Claude Code Approve drops answers** (its hook allow carries no message); the existing "Feedback won't be sent" warning names them. No workaround in v1 (owner decision 1).
Expand Down
7 changes: 5 additions & 2 deletions packages/core/inbox-questions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import {
MAX_QUESTION_PROMPT_CHARS,
parseQuestionAnswer,
parseQuestionBlock,
questionChoicesNeedCanonicalLabels,
type QuestionAnswer,
type QuestionChoice,
type QuestionExportItem,
Expand Down Expand Up @@ -308,7 +309,9 @@ export function checkInboxAnswer(

function exportItem(record: InboxQuestionRecord, number: number): QuestionExportItem {
const settled = record.parsed.choices.filter((c) => c.settled).map((c) => c.label);
const hasAliases = record.parsed.aliases.some((a) => a !== null);
// The export prints current labels for answers stored under an older name
// or cut at 200 characters by a build through 0.28.17.
const needsChoices = questionChoicesNeedCanonicalLabels(storedChoices(record));
return {
key: record.key,
number,
Expand All @@ -318,7 +321,7 @@ function exportItem(record: InboxQuestionRecord, number: number): QuestionExport
...(record.parsed.suggested_text === null ? {} : { suggestedText: record.parsed.suggested_text }),
settled: settled.length > 0,
settledLabels: settled,
...(hasAliases
...(needsChoices
? {
choices: record.parsed.choices.map((c, i) => ({
label: c.label,
Expand Down
79 changes: 79 additions & 0 deletions packages/core/question-block.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -663,3 +663,82 @@ describe("Replaces: lines (the Plannotator Inbox)", () => {
expect(parseQuestionBlock("question-text", `Why?\nReplaces: ${OLD}.`)!.replaces).toEqual([OLD]);
});
});

// The owner's 0.28.17 report: a task-list choice longer than 200 characters
// could never be picked in the Inbox ("… is not one of the question's
// choices") and never showed as picked in plan review, because the answer's
// `selected` was cut to 200 characters while the label was not.
const LONG_SPIKE =
"Spike first. Find out what the sign-in grants (a Claude subscription paying for turns the platform runs?) and whether Anthropic allows it for a third-party product; the placeholder stays off the product until both answers are in";
// The same text as a long label set off from a short "why" by ` - ` / ` — `:
// the label before the separator is still over 200 characters.
const LONG_DASH = `${LONG_SPIKE} - keeps it reversible`;
const LONG_EM = `${LONG_SPIKE} — keeps it reversible`;
// The short-lead form (` - ` right after the lead) splits as always.
const SHORT_LEAD = "Spike first - find out what the sign-in grants (a Claude subscription paying for turns the platform runs?) and whether Anthropic allows it for a third-party product; the placeholder stays off the product until both answers are in";

describe("long task-list choices (picked as written)", () => {
const block = (choice: string) => `Which way?\n\n- [ ] ${choice}\n- [ ] Ship it now`;

for (const [name, choice] of [
["no separator", LONG_SPIKE],
["a ' - ' separator", LONG_DASH],
["a ' — ' separator", LONG_EM],
] as const) {
test(`${name}: the label round-trips through parseQuestionAnswer whole`, () => {
const q = parseQuestionBlock("question", block(choice))!;
const label = q.choices[0].label;
expect(label.length).toBeGreaterThan(200);
const saved = parseQuestionAnswer({ v: 1, key: q.key, kind: q.kind, prompt: q.prompt, selected: [label] })!;
expect(saved.selected).toEqual([label]);
expect(questionStatus(q, saved)).toBe("answered");
const index = indexQuestionBlocks([{ id: "b", type: "directive", directiveKind: "question", content: block(choice), startLine: 1 }]);
const out = formatQuestionAnswersSection(questionExportItems(index), [saved]);
expect(out).toContain(`Answer: ${label}`);
});
}

test("the separators keep their label/description split", () => {
for (const choice of [LONG_DASH, LONG_EM]) {
const q = parseQuestionBlock("question", block(choice))!;
expect(q.choices[0]).toMatchObject({ label: LONG_SPIKE, description: "keeps it reversible" });
}
const short = parseQuestionBlock("question", block(SHORT_LEAD))!;
expect(short.choices[0].label).toBe("Spike first");
expect(short.choices[0].description).toStartWith("find out what");
});

test("an answer an older build cut at 200 characters maps back to the one choice it names", () => {
const q = parseQuestionBlock("question", block(LONG_SPIKE))!;
const cut = LONG_SPIKE.slice(0, 200);
const stored: QuestionAnswer = { v: 1, key: q.key, kind: q.kind, prompt: q.prompt, selected: [cut] };
expect(canonicalQuestionAnswer(q, stored).selected).toEqual([LONG_SPIKE]);
// Two choices sharing their first 200 characters: ambiguous, left as stored.
const twin = parseQuestionBlock("question", `Which way?\n\n- [ ] ${LONG_SPIKE}\n- [ ] ${LONG_SPIKE.slice(0, 200)} but later`)!;
expect(canonicalQuestionAnswer(twin, stored).selected).toEqual([cut]);
});

test("the export prints the full label for an answer an older build cut at 200 characters", () => {
const index = indexQuestionBlocks([{ id: "b", type: "directive", directiveKind: "question", content: block(LONG_SPIKE), startLine: 1 }]);
const q = index[0].question;
const cut: QuestionAnswer = { v: 1, key: q.key, kind: q.kind, prompt: q.prompt, selected: [LONG_SPIKE.slice(0, 200)] };
const out = formatQuestionAnswersSection(questionExportItems(index), [cut]);
expect(out).toContain(`Answer: ${LONG_SPIKE}`);
});

test("a long label adds no looser matching: only its exact 200-character cut maps", () => {
const q = parseQuestionBlock("question", block(LONG_SPIKE))!;
expect(q.choices.some((c) => c.aliases)).toBe(false);
const stored = (label: string): QuestionAnswer => ({ v: 1, key: q.key, kind: q.kind, prompt: q.prompt, selected: [label] });
// Case or punctuation never maps here, as in a question with no long label.
const loose = stored("ship it now.");
expect(canonicalQuestionAnswer(q, loose)).toBe(loose);
const cutLoose = stored(LONG_SPIKE.slice(0, 200).toUpperCase());
expect(canonicalQuestionAnswer(q, cutLoose)).toBe(cutLoose);
});

test("selected stays bounded: a label longer than any block body is cut", () => {
const parsed = parseQuestionAnswer(answer({ selected: ["x".repeat(30_000)] }))!;
expect(parsed.selected[0].length).toBe(20_000);
});
});
59 changes: 47 additions & 12 deletions packages/core/question-block.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,15 @@ export interface QuestionDecisionLink {
export const MAX_QUESTION_BODY_CHARS = 20_000;
export const MAX_QUESTION_CHOICES = 20;
export const MAX_QUESTION_PROMPT_CHARS = 400;
/** The longest label a plain-bullet choice takes before the rest moves to
* its description (see capChoiceLabel). Task-list labels are not cut. */
export const MAX_QUESTION_CHOICE_LABEL_CHARS = 200;
/** The cap on each `selected` label of an answer. A picked label is a choice
* label, and a choice lives in a block body of at most
* MAX_QUESTION_BODY_CHARS, so any label a block can hold round-trips whole.
* (Through 0.28.17 this was MAX_QUESTION_CHOICE_LABEL_CHARS, which cut a
* task-list label over 200 characters, so it could never be picked.) */
export const MAX_QUESTION_SELECTED_CHARS = MAX_QUESTION_BODY_CHARS;
export const MAX_QUESTION_OTHER_CHARS = 2000;
export const MAX_QUESTION_TEXT_CHARS = 4000;
export const MAX_QUESTION_NOTE_CHARS = 2000;
Expand Down Expand Up @@ -258,10 +266,11 @@ const splitBoldLead = (raw: string): { label: string; description: string } | nu
return { label, description };
};

/** A label longer than an answer may quote (`MAX_QUESTION_CHOICE_LABEL_CHARS`)
* could never show as picked, so it is cut at its first sentence end (else
* its last space) within the cap; the rest leads the description. Nothing
* is dropped. */
/** A plain-bullet label longer than `MAX_QUESTION_CHOICE_LABEL_CHARS` is cut
* at its first sentence end (else its last space) within the cap; the rest
* leads the description, so the card shows a short name. Nothing is
* dropped. Task-list labels keep their full length (an answer quotes up to
* `MAX_QUESTION_SELECTED_CHARS`). */
const capChoiceLabel = (split: { label: string; description?: string }): { label: string; description?: string } => {
const { label, description } = split;
if (label.length <= MAX_QUESTION_CHOICE_LABEL_CHARS) return split;
Expand Down Expand Up @@ -845,7 +854,7 @@ export const parseQuestionAnswer = (value: unknown): QuestionAnswer | null => {
if (v.sourceLine !== undefined && !(Number.isInteger(v.sourceLine) && (v.sourceLine as number) > 0)) return null;

const selected = [...new Set((v.selected as string[])
.map((s) => cap(s.trim(), MAX_QUESTION_CHOICE_LABEL_CHARS))
.map((s) => cap(s.trim(), MAX_QUESTION_SELECTED_CHARS))
.filter(Boolean))]
.slice(0, MAX_QUESTION_CHOICES);
const other = typeof v.other === 'string' ? cap(v.other, MAX_QUESTION_OTHER_CHARS) : '';
Expand Down Expand Up @@ -880,15 +889,26 @@ export const canonicalQuestionAnswer = <A extends QuestionAnswer>(
answer: A,
): A => {
if (answer.selected.length === 0) return answer;
if (!question.choices.some((c) => c.aliases?.length)) return answer;
// Answers are capped on validation, so a long alias is matched capped too.
const names = question.choices.map((c) =>
(c.aliases ?? []).flatMap((alias) => [alias, cap(alias.trim(), MAX_QUESTION_CHOICE_LABEL_CHARS)]));
const labels = new Set(question.choices.map((c) => c.label));
if (answer.selected.every((stored) => labels.has(stored))) return answer;
// The one choice a stored label names, or -1 when none or several do: an
// ambiguous label (two bullets that shared a first line, or names that only
// differ in case or emphasis) stays as stored rather than picking one.
const only = (hits: number[]): number => (hits.length === 1 ? hits[0] : -1);
// Through 0.28.17 an answer's labels were cut at
// MAX_QUESTION_CHOICE_LABEL_CHARS: a stored label that is exactly a long
// label's first 200 characters names that choice. Exact only, never
// normalized, so a question with a long label matches nothing else looser.
const cutLabels = question.choices.map((c) => legacyCutName(c.label));
const cutHit = (stored: string): number =>
only(question.choices.flatMap((_, i) => (cutLabels[i] === stored ? [i] : [])));
const hasAliases = question.choices.some((c) => c.aliases?.length);
// Answers were capped on validation, so a long alias is matched cut too.
const names = question.choices.map((c) =>
(c.aliases ?? []).flatMap((alias) => {
const cut = legacyCutName(alias);
return cut === null ? [alias] : [alias, cut];
}));
const exactHit = (stored: string): number =>
only(question.choices.flatMap((_, i) => (names[i].includes(stored) ? [i] : [])));
const normalizedHit = (stored: string): number => {
Expand All @@ -900,15 +920,30 @@ export const canonicalQuestionAnswer = <A extends QuestionAnswer>(
let changed = false;
const selected = answer.selected.map((stored) => {
if (labels.has(stored)) return stored;
let hit = exactHit(stored);
if (hit === -1) hit = normalizedHit(stored);
let hit = hasAliases ? exactHit(stored) : -1;
if (hit === -1) hit = cutHit(stored);
if (hit === -1 && hasAliases) hit = normalizedHit(stored);
if (hit === -1) return stored;
changed = true;
return question.choices[hit].label;
});
return changed ? { ...answer, selected: [...new Set(selected)] } : answer;
};

/** A name as a build through 0.28.17 stored it in an answer when it was
* longer than MAX_QUESTION_CHOICE_LABEL_CHARS (cut to that length), else
* null (such a name was stored whole). */
const legacyCutName = (name: string): string | null => {
const trimmed = name.trim();
return trimmed.length > MAX_QUESTION_CHOICE_LABEL_CHARS ? cap(trimmed, MAX_QUESTION_CHOICE_LABEL_CHARS) : null;
};

/** Whether the export needs a question's choices to print an answer's
* current label: a choice has older names, or a label long enough that an
* older build stored it cut. */
export const questionChoicesNeedCanonicalLabels = (choices: ReadonlyArray<Pick<QuestionChoice, 'label' | 'aliases'>>): boolean =>
choices.some((c) => (c.aliases?.length ?? 0) > 0 || legacyCutName(c.label) !== null);

/** A choice was picked, "Other…" was filled, or free text was written. */
export const isQuestionAnswered = (answer: QuestionAnswer): boolean =>
answer.selected.length > 0 || !!answer.other?.trim() || !!answer.text?.trim();
Expand Down Expand Up @@ -1068,7 +1103,7 @@ export const questionExportItems = (indexed: ReadonlyArray<IndexedQuestion>): Qu
...(question.suggestedText ? { suggestedText: question.suggestedText } : {}),
settled: question.choices.some((c) => c.settled),
settledLabels: question.choices.filter((c) => c.settled).map((c) => c.label),
...(question.choices.some((c) => c.aliases) ? { choices: question.choices.map(({ label, aliases }) => ({ label, ...(aliases ? { aliases } : {}) })) } : {}),
...(questionChoicesNeedCanonicalLabels(question.choices) ? { choices: question.choices.map(({ label, aliases }) => ({ label, ...(aliases ? { aliases } : {}) })) } : {}),
}));

export const QUESTION_ANSWERS_HEADING = 'Answers to your questions';
Expand Down
Loading
Loading