Strict Chat-Completion Compatibility

Este conteúdo não está disponível em sua língua ainda.

Different LLM provider APIs enforce conflicting message structure requirements. TomoriBot normalizes role alternation, prefix completion, and assistant media behind a shared seam in src/providers/utils/strictChatCompat.ts so built-in and custom-endpoint proxies can communicate with strict backends.

The three normalizations

Normalization	Setting	Mechanism
Role alternation	`strict_role_alternation`	Merges consecutive same-role messages into one and prepends a synthetic leading user turn (`[System: Conversation start]`) when the dialogue opens with an assistant turn. Tool-bearing turns act as merge boundaries so `tool_calls` and `tool_call_id` metadata are preserved.
Prefix completion	`supports_prefix_completion`	Stamps `prefix: true` on the trailing assistant message (`applyAssistantPrefixCompletion`) to activate vendor continuation modes (DeepSeek, Z.ai).
Media relocation	Shared serialization policy	Moves assistant media into following synthetic user turns (`relocateAssistantMediaContextItems`) with sender attribution. This gives adapters a common representation that strict backends can accept.

Orthogonal compatibility toggles

Role alternation and prefix completion address independent backend constraints and remain decoupled:

A proxy fronting Claude requires role alternation enabled and prefix completion disabled; sending prefix: true causes request rejection.
A proxy fronting DeepSeek or vLLM continue modes requires prefix completion enabled, without needing same-role turn merging.

A single bundled toggle would prevent configuring backends that need only one of these behaviors.

Assistant prefill capability

supports_assistant_prefill records whether the backend accepts and continues a trailing assistant turn instead of rejecting it with an HTTP 400 error (such as Claude 4.6+ or Gemini 3.5 Flash-Lite) or restarting output from the beginning. It is distinct from prefix completion: prefix completion designates the prefix: true request field, while assistant prefill designates whether the model accepts the trailing turn.

Tool conflicts on DeepSeek: DeepSeek rejects prefix: true in requests that also carry tools. When tool calling is active, the prefill resolver (resolvePrefillBlocker in src/utils/chat/assistantPrefill.ts) skips server prefills and converts /respond prefills into written prompt instructions.
Tool loop folding: When a model initiates a tool call, trailing prefill text is folded into the first tool call’s assistant message (foldPrefillIntoToolHistory). This prevents consecutive assistant turns that strict-alternation backends would reject.
Persona’s own message last: A turn that starts without new user input (a self-triggered or system-initiated turn) can build history that ends on the persona’s own delivered message. applyAssistantPrefill then appends a user directive asking for the next message, because only this attempt’s own prefill may end the request. NovelAI is exempt: text completion continues from a trailing persona line by design.

Media relocation

The structured request builders apply media relocation independently of the alternation and prefix-completion toggles. When dialogue history includes images sent by an assistant (such as generated images), the media parts are extracted into an immediately following synthetic user message:

[System: The following image was sent by {name}.]

When no sender name is available, the notice falls back to [System: The following image was sent]. Relocation runs on structured context items before provider-specific serialization, preserving participant indices and mention metadata.

Prompt layout and cache identity

Serializers use isPromptSystemItem() from the prompt-layout registry for system routing. A system role alone does not determine placement for tagged items. Untagged preset nodes keep their authored roles. Normalization preserves the cacheable-prefix marker through media relocation and same-role merging, including when the boundary item ends in media.

Direct Anthropic and Anthropic models through OpenRouter request two five-minute cache breakpoints: the system-message end and the final serialized item above the tail slot. Each is admitted only when the local text estimate reaches the model minimum; missing boundaries or undersized prefixes omit that marker. src/providers/utils/anthropicCache.ts owns this policy. Explicit rejection of cache_control removes it for a targeted retry.

resolveToolCallId() in src/providers/utils/toolCallIds.ts preserves usable unique provider IDs and generates deterministic IDs in interaction-history order otherwise. OpenRouter, OpenAI-compatible, and Anthropic serializers reuse the resolved ID for the call and result, so appending tool-loop steps does not change IDs already serialized into the prefix.

Resolution and persistence

Compatibility settings are stored as boolean columns on llms and custom_endpoints:

Built-in providers: Seeded with required flags in src/db/seed/catalog/models.ts (Anthropic enables alternation; DeepSeek, Z.ai, and Z.ai Coding enable prefix completion). supports_assistant_prefill is configured per measured model.
Custom endpoints: Expose these settings as checkboxes under Chat Completion Compatibilities in /providers and /setup. Values sync to the custom endpoint’s synthetic llms row via upsertSyntheticCustomLlm.
Runtime safety net: providerRequiresAlternation() and providerRequiresPrefixCompletion() in strictChatCompat.ts OR-combine with stored database flags, preventing mis-seeded catalog rows from emitting invalid bodies.
Verification gate: bun run check-seed-catalogs validates required flags at boot and in CI (collectStrictChatFlagViolations in src/db/seed/catalog/modelSeed.ts) to keep built-in requirements synchronized.

Source pointers

src/providers/utils/strictChatCompat.ts: Message normalization functions (mergeConsecutiveSameRole, ensureLeadingUserTurn, applyAssistantPrefixCompletion, relocateAssistantMediaContextItems).
src/utils/chat/assistantPrefill.ts: Prefill capability resolution and tool conflict detection (resolvePrefillBlocker).
src/db/seed/catalog/models.ts: Seed definitions for built-in provider model flags.
src/db/seed/catalog/modelSeed.ts: Seed verification gates (collectStrictChatFlagViolations).