Multimodal Context Hygiene with a Jinja Chat Template
The first time I noticed an image tool result being carried into a later prompt, the waste was hard to miss. A long textual payload remained in the conversation after the useful work was finished, and then I found another copy inside an assistant tool call that had decoded or saved the same bytes. I was asking the model to drag old media work through turns that no longer needed it.
That sort of history gets expensive quickly. Base64 expands binary data, and the rendered text still occupies context and participates in the model’s attention and KV-cache work. I am deliberately not putting a percentage on the effect because it depends on the payload, tokenizer, context size, cache behavior, and runtime. I did not need a benchmark to establish the basic engineering fact that repeating thousands of characters of old encoded media is more work than leaving them out.
My first instinct was to strip the payload in the model proxy. I had just spent the previous installment establishing the proxy as a passive diagnostic instrument, though. The moment it started rewriting requests, I could no longer trust it as evidence of what the client had sent. The filtering had to happen somewhere else, and for the Qwen route I was working with, the selected Jinja chat template was the right boundary.
The Template Is Part of the Model Interface
An OpenAI-compatible request is not yet the prompt the model consumes. It is structured input made up of messages, roles, tool definitions, tool calls, and sometimes native image or video parts. The chat template turns all of that into the model’s grammar. In this case, that grammar belongs to the Qwen 3.5 family and includes the <|im_start|> and <|im_end|> turn envelope, Qwen vision placeholders, thinking layout, tool-call serialization, validation rules, and the final assistant generation boundary. Calling it a generic ChatML template would hide most of the compatibility contract I actually care about.
I kept the upstream stock template unchanged and built the media-history policy as a derivative. That gave me a real fallback instead of something I would have to reconstruct from memory, and it gave the tests a useful control: ordinary text and tool history should render identically under both templates. The proxy continues to observe the exchange without changing it, while the selected template performs the intentional transformation when the runtime constructs the prompt. If I want the proxy’s diagnostic render to match what llama.cpp constructs, both paths have to reference the same file.
That separation is simple on paper and important in operation. The Hands-On companion makes both rendering paths visible in a small stock-versus-derived test bench.
| Boundary | Responsibility |
|---|---|
| Agent or application | Sends OpenAI-compatible messages and tool history |
| Passive model proxy | Forwards the request and produces diagnostic views without rewriting it |
| Selected Qwen template | Applies the reviewed media-history policy while rendering the model prompt |
| Model runtime | Consumes the rendered prompt and processes native structured vision data |
History Is Structured, Even When the Payload Is Text
My first version kept only the latest image-like payload. It proved the idea, but it was too broad in one direction and too narrow in the other. A long string containing an image signature is not automatically an image result, and deleting only the payload can leave a broken conversational exchange behind.
The current template starts with a prepass over the messages. It looks for explicit tool results whose content is a string longer than 4,096 characters and whose structure and markers identify the relevant media type. For images, the content must look like an image result and contain a PNG, JPEG, or image data-URL marker. Audio and video require their corresponding result labels and data-URL markers.
The 4,096-character threshold is only a routing gate. It is not a truncation size, a token estimate, or proof that the content is valid base64. Jinja is not a media parser, and I do not want the template pretending it validated data that it merely recognized by shape and marker. A false positive can be just as damaging as missed cleanup, so one synthetic fixture contains a long ordinary diagnostic result with an incidental PNG signature. It stays in the rendered prompt because the surrounding structure does not identify it as an image result. You can exercise the exact greater-than boundary at 4,096 and 4,097 characters in the lab.
The prepass builds the selection policy before the normal Qwen rendering loop begins:
{%- if message.role == 'tool'
and message.content is string
and message.content|length > 4096 %}
{# Classify explicit textual media results and record retention state. #}
{%- endif %}
That fragment is intentionally incomplete. I would rather give readers the reviewed file than turn a few selected snippets into a puzzle they have to reconstruct. The complete attributed template, stock fallback, fixtures, expected results, renderer, tests, notice, checksums, and Apache-2.0 license are available together later in this article.
Images, Audio, and Video Do Not Share One Retention Rule
The current policy removes every textual image tool result, including the newest one. That sounds aggressive until textual history and native structured vision are treated as different things. Textual image results are often enormous, may already be truncated, and cannot reliably reconstruct a usable image for the model. Keeping the newest damaged string does not make it useful. Native structured image parts are different: the model engine handles their data separately while the template emits Qwen’s vision placeholder, so those parts remain eligible.
Audio and video follow a different rule in the current policy. Older explicit textual results are removed while the latest explicit result of each type remains. This is a bounded choice based on the shapes exercised in the tested route, not a general claim that the latest media payload is always sufficient for every model or application.
The result is easier to understand as a selection flow than as a pile of string checks:
The important division is not simply old versus new. It is textual tool history versus native structured multimodal input, followed by a media-specific retention rule.
Remove the Exchange, Not Just the Payload
A tool result does not exist by itself. The assistant requested the tool, the tool returned a result, and later turns may refer back to the exchange. If I remove only the result, the prompt can retain an orphaned assistant call asking for work that apparently never completed. The template therefore removes media-producing assistant calls and their associated tool results as complete exchanges. It also removes assistant calls carrying recognizable media bytes or explicitly decoding base64, along with the following results that belong to those calls. That catches the second copy that started this investigation in the first place: the image appeared as a tool result, then an assistant-side operation copied or decoded the same bytes again.
The ordering logic deserves tests because message history is not always tidy. A truncated result can still contain enough structure to identify the exchange. A malformed system message must still fail with the stock Qwen ordering error. A normal tool exchange must remain untouched. Removing context is only a win when the remaining prompt preserves the model’s grammar and the application’s meaning.
What the Fixtures Actually Prove
I built the publication companion around seven invented fixtures instead of trying to sanitize a screenshot from my own environment. Each fixture generates obvious canary strings locally and contains no real prompt, conversation, model response, or media artifact. The Hands-On companion lets you compare all seven fixtures without dumping their long payloads.
| Fixture | Evidence it provides |
|---|---|
| Native structured media | One image placeholder and one video placeholder remain while the invented data URL does not render as prompt text |
| Textual image exchanges | Both image results, both producing calls, and the assistant-side decode exchange are absent |
| Latest audio and video | Older explicit exchanges are absent and the latest explicit exchanges remain |
| Truncated image history | The truncation marker, image marker, and producing call are removed together |
| Incidental signature collision | A long ordinary result remains because image-result structure is absent |
| Malformed ordering | Rendering fails with the normal Qwen system-message ordering error |
| Ordinary text and tools | The derived render is byte-identical to the packaged stock render |
The companion tests also verify provenance checksums. The packaged stock fallback is byte-equivalent to the immutable upstream Qwen template after removing its single added trailing newline. The derivative carries a prominent modification notice, and the Apache-2.0 license travels with the package. These are rendering tests: they establish how the supplied template handles the supplied message shapes under the tested Jinja environment, but they do not prove output quality, token savings, latency improvement, or compatibility with an arbitrary model artifact. A generation canary still has to run against the exact model and runtime selected for deployment.
Get the Complete Template Package
The complete package is available as one ZIP download. It contains the exact nine documented files and excludes interpreter caches and bytecode. The same files are available below through the source viewer, so you can inspect them before downloading anything.
Qwen-3_5-media-history-template.jinja jinja View source
{#
Modified Qwen 3.5 media-history chat template.
Upstream: Qwen/Qwen3.5-27B chat_template.jinja
Revision: feea018b31f89dc0950e61da42577a7a4ab09169
Source: https://huggingface.co/Qwen/Qwen3.5-27B/blob/feea018b31f89dc0950e61da42577a7a4ab09169/chat_template.jinja
License: Apache License 2.0. See LICENSE-APACHE-2.0.txt and NOTICE.md.
Modifications: adds media-history policy that removes textual image tool
exchanges and assistant-side media byte copies, retains only the latest explicit
audio and video tool results, and preserves native structured image and video
parts. The policy uses structural and marker checks with a 4,096-character
detection threshold. Jinja does not validate base64 data.
#}
{%- set image_count = namespace(value=0) %}
{%- set video_count = namespace(value=0) %}
{%- macro render_content(content, do_vision_count, is_system_content=false, message_index=-1) %}
{%- if content is string %}
{{- content }}
{%- elif content is iterable and content is not mapping %}
{%- for item in content %}
{%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain images.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set image_count.value = image_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Picture ' ~ image_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
{%- elif 'video' in item or item.type == 'video' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain videos.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set video_count.value = video_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Video ' ~ video_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
{%- elif 'text' in item %}
{{- item.text }}
{%- else %}
{{- raise_exception('Unexpected item type in content.') }}
{%- endif %}
{%- endfor %}
{%- elif content is none or content is undefined %}
{{- '' }}
{%- else %}
{{- raise_exception('Unexpected content type.') }}
{%- endif %}
{%- endmacro %}
{%- if not messages %}
{{- raise_exception('No messages provided.') }}
{%- endif %}
{%- set media_state = namespace(last_audio_message_index=-1, last_video_message_index=-1) %}
{%- for message in messages %}
{%- if message.role == 'tool' and message.content is string and message.content|length > 4096 %}
{%- if ('audio' in message.content or 'audios' in message.content) and 'data:audio/' in message.content %}
{%- set media_state.last_audio_message_index = loop.index0 %}
{%- endif %}
{%- if ('video' in message.content or 'videos' in message.content) and 'data:video/' in message.content %}
{%- set media_state.last_video_message_index = loop.index0 %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set skip_state = namespace(indices=[]) %}
{%- for message in messages %}
{%- set message_index = loop.index0 %}
{%- set content_state = namespace(has_media=false, keep_media=false) %}
{%- if message.role == 'tool' and message.content is string and message.content|length > 4096 %}
{%- if 'images' in message.content and ('iVBORw0KGgo' in message.content or '/9j/' in message.content or 'data:image/' in message.content) %}
{%- set content_state.has_media = true %}
{%- endif %}
{%- if ('audio' in message.content or 'audios' in message.content) and 'data:audio/' in message.content %}
{%- set content_state.has_media = true %}
{%- if message_index == media_state.last_audio_message_index %}{%- set content_state.keep_media = true %}{%- endif %}
{%- endif %}
{%- if ('video' in message.content or 'videos' in message.content) and 'data:video/' in message.content %}
{%- set content_state.has_media = true %}
{%- if message_index == media_state.last_video_message_index %}{%- set content_state.keep_media = true %}{%- endif %}
{%- endif %}
{%- endif %}
{%- if content_state.has_media and not content_state.keep_media %}
{%- set _ = skip_state.indices.append(message_index) %}
{%- if message_index > 0 and messages[message_index - 1].role == 'assistant' %}
{%- set _ = skip_state.indices.append(message_index - 1) %}
{%- endif %}
{%- endif %}
{%- if message.role == 'assistant' and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
{%- set tool_state = namespace(has_media=false) %}
{%- for original_tool_call in message.tool_calls %}
{%- set tool_call = original_tool_call.function if original_tool_call.function is defined else original_tool_call %}
{%- if tool_call.arguments is string %}
{%- if 'data:image/' in tool_call.arguments or 'iVBORw0KGgo' in tool_call.arguments or '/9j/' in tool_call.arguments or 'data:audio/' in tool_call.arguments or 'data:video/' in tool_call.arguments or 'base64.b64decode' in tool_call.arguments or 'base64 --decode' in tool_call.arguments or 'base64 -d' in tool_call.arguments %}
{%- set tool_state.has_media = true %}
{%- endif %}
{%- elif tool_call.arguments is mapping %}
{%- for args_name, args_value in tool_call.arguments|items %}
{%- set args_text = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
{%- if 'data:image/' in args_text or 'iVBORw0KGgo' in args_text or '/9j/' in args_text or 'data:audio/' in args_text or 'data:video/' in args_text or 'base64.b64decode' in args_text or 'base64 --decode' in args_text or 'base64 -d' in args_text %}
{%- set tool_state.has_media = true %}
{%- endif %}
{%- endfor %}
{%- endif %}
{%- endfor %}
{%- if tool_state.has_media %}
{%- set _ = skip_state.indices.append(message_index) %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set following_state = namespace(skip_tools=false) %}
{%- for message in messages %}
{%- if message.role == 'assistant' %}
{%- set following_state.skip_tools = loop.index0 in skip_state.indices %}
{%- elif message.role == 'tool' and following_state.skip_tools %}
{%- set _ = skip_state.indices.append(loop.index0) %}
{%- else %}
{%- set following_state.skip_tools = false %}
{%- endif %}
{%- endfor %}
{%- if tools and tools is iterable and tools is not mapping %}
{{- '<|im_start|>system\n' }}
{{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>" }}
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{%- if content %}
{{- '\n\n' + content }}
{%- endif %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- else %}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" %}
{%- set content = render_content(message.content, false, false, index)|trim %}
{%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if ns.multi_step_tool %}
{{- raise_exception('No user query found in messages.') }}
{%- endif %}
{%- for message in messages %}
{%- if loop.index0 not in skip_state.indices %}
{%- set content = render_content(message.content, true, false, loop.index0)|trim %}
{%- if message.role == "system" %}
{%- if not loop.first %}
{{- raise_exception('System message must be at the beginning.') }}
{%- endif %}
{%- elif message.role == "user" %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- set reasoning_content = reasoning_content|trim %}
{%- if loop.index0 > ns.last_query_index %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{%- if loop.first %}
{%- if content|trim %}
{{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- else %}
{{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- else %}
{{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- if tool_call.arguments is defined %}
{%- for args_name, args_value in tool_call.arguments|items %}
{{- '<parameter=' + args_name + '>\n' }}
{%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
{%- if args_value|length > 4096 and ('data:image/' in args_value or 'iVBORw0KGgo' in args_value or '/9j/' in args_value or 'data:audio/' in args_value or 'data:video/' in args_value) %}
{{- '[Embedded media payload omitted from tool call]' }}
{%- else %}
{{- args_value }}
{%- endif %}
{{- '\n</parameter>\n' }}
{%- endfor %}
{%- endif %}
{{- '</function>\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.previtem and loop.previtem.role != "tool" %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if not loop.last and loop.nextitem.role != "tool" %}
{{- '<|im_end|>\n' }}
{%- elif loop.last %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- else %}
{{- raise_exception('Unexpected message role.') }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- else %}
{{- '<think>\n' }}
{%- endif %}
{%- endif %}
Qwen-3_5-stock-template.jinja jinja View source
{%- set image_count = namespace(value=0) %}
{%- set video_count = namespace(value=0) %}
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
{%- if content is string %}
{{- content }}
{%- elif content is iterable and content is not mapping %}
{%- for item in content %}
{%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain images.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set image_count.value = image_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Picture ' ~ image_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
{%- elif 'video' in item or item.type == 'video' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain videos.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set video_count.value = video_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Video ' ~ video_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
{%- elif 'text' in item %}
{{- item.text }}
{%- else %}
{{- raise_exception('Unexpected item type in content.') }}
{%- endif %}
{%- endfor %}
{%- elif content is none or content is undefined %}
{{- '' }}
{%- else %}
{{- raise_exception('Unexpected content type.') }}
{%- endif %}
{%- endmacro %}
{%- if not messages %}
{{- raise_exception('No messages provided.') }}
{%- endif %}
{%- if tools and tools is iterable and tools is not mapping %}
{{- '<|im_start|>system\n' }}
{{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>" }}
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{%- if content %}
{{- '\n\n' + content }}
{%- endif %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- else %}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" %}
{%- set content = render_content(message.content, false)|trim %}
{%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if ns.multi_step_tool %}
{{- raise_exception('No user query found in messages.') }}
{%- endif %}
{%- for message in messages %}
{%- set content = render_content(message.content, true)|trim %}
{%- if message.role == "system" %}
{%- if not loop.first %}
{{- raise_exception('System message must be at the beginning.') }}
{%- endif %}
{%- elif message.role == "user" %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- set reasoning_content = reasoning_content|trim %}
{%- if loop.index0 > ns.last_query_index %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{%- if loop.first %}
{%- if content|trim %}
{{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- else %}
{{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- else %}
{{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- if tool_call.arguments is defined %}
{%- for args_name, args_value in tool_call.arguments|items %}
{{- '<parameter=' + args_name + '>\n' }}
{%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
{{- args_value }}
{{- '\n</parameter>\n' }}
{%- endfor %}
{%- endif %}
{{- '</function>\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.previtem and loop.previtem.role != "tool" %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if not loop.last and loop.nextitem.role != "tool" %}
{{- '<|im_end|>\n' }}
{%- elif loop.last %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- else %}
{{- raise_exception('Unexpected message role.') }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- else %}
{{- '<think>\n' }}
{%- endif %}
{%- endif %}
README.md markdown View source
# Qwen 3.5 Media-History Template Companion
This package is the practical companion for Part 7 of the Local-First Agent Operations series. It gives Hermes users running a matching Qwen model route a reviewable chat template, a stock fallback, sanitized fixtures, expected results, provenance checks, and rollback instructions.
It is not a generic ChatML template. Qwen uses `<|im_start|>` and `<|im_end|>` as its ChatML turn envelope, and its tool calling is Hermes-like, but this complete file also depends on Qwen vision placeholders, thinking layout, tool-call serialization, and Jinja runtime helpers. The media-history policy may be ported to another ChatML-style template only after preserving that model's control tokens, multimodal markers, reasoning rules, tool schema, and assistant generation boundary.
## Package Contents
| File | Purpose |
| --- | --- |
| `Qwen-3_5-media-history-template.jinja` | Attributed Apache-2.0 derivative with media-history pruning |
| `Qwen-3_5-stock-template.jinja` | Unmodified fallback baseline, apart from one trailing newline |
| `fixtures.json` | Seven invented, compact fixtures expanded locally into long synthetic media markers |
| `expected-results.json` | Expected retained, omitted, counted, error, and stock-equivalence results |
| `render_fixture.py` | Renders one fixture for manual inspection |
| `test_media_history_template.py` | Self-contained checksum and behavior tests |
| `NOTICE.md` | Immutable provenance, modification notice, checksums, and license boundary |
| `LICENSE-APACHE-2.0.txt` | Apache License 2.0 text |
## Supported Input Assumptions
The tested runtime supplies Jinja 3.1.6 and the `raise_exception` helper used by the Qwen template. The render context includes `messages`, `tools`, `add_generation_prompt`, `enable_thinking`, `add_vision_id`, `bos_token`, and `eos_token`.
Messages use the Qwen 3.5-family shape exercised by the fixtures:
- roles are `system`, `user`, `assistant`, or `tool`;
- content is a string, `null`, or a list of structured text, image, or video parts;
- assistant `tool_calls` are iterable and each call exposes a function name and argument mapping;
- tool results are textual strings when the media-history policy inspects them;
- native structured images and videos are processed separately by the model engine after the template emits Qwen vision placeholders.
The template has also worked with the matching Qwen 3.6 prompt format in the publisher's environment. That is an operational observation, not universal compatibility evidence. Render and generation acceptance are still required for the exact model artifact and runtime you use.
## What the Policy Does
Textual image tool results are removed as complete exchanges, including the most recent result. They are often truncated before they reach history and cannot be reconstructed reliably. Native structured image and video parts remain eligible and render Qwen vision placeholders.
For explicit textual audio and video results, only the latest result of each type remains. Older associated assistant calls and tool results are removed. Assistant tool calls that carry recognizable media bytes or explicitly decode base64 are also removed.
Textual tool-result detection requires content longer than 4,096 characters. Image detection also requires image-result structure plus a PNG, JPEG, or `data:image/` marker. Audio and video detection require their corresponding result labels and data-URL markers. Jinja does not validate base64. The threshold and marker checks are routing policy, not proof that the payload is valid media.
## Stock Versus Derived Behavior
| Input history | Stock Qwen template | Media-history derivative |
| --- | --- | --- |
| Ordinary text and tool history | Retained | Byte-identical rendered prompt in the supplied fixture |
| Native structured image or video part | Qwen vision placeholder retained | Qwen vision placeholder retained |
| Textual image tool result | Retained as tool-response text | Entire image-producing exchange removed |
| Assistant-side media byte or decode call | Retained | Associated assistant call and following tool result removed |
| Multiple explicit textual audio results | All retained | Only latest explicit audio exchange retained |
| Multiple explicit textual video results | All retained | Only latest explicit video exchange retained |
| Long incidental PNG signature without image-result structure | Retained | Retained |
| Malformed system-message ordering | Template error | Same template error |
## Verify Before Installing
Run the checksum and behavior suite with the same application-owned Python environment used by LLM-Ops-Kit:
```bash
python -m unittest -v test_media_history_template.py
```
Render individual sanitized fixtures before touching a model profile:
```bash
python render_fixture.py native_structured_media
python render_fixture.py textual_image_exchanges
python render_fixture.py ordinary_text_tool_history
```
The fixture payloads are invented. Long base64-like strings are generated locally from obvious canary markers and repeated letters; they are not real images, audio, video, prompts, conversations, or model output.
If you have a sanitized captured OpenAI-compatible request, render it with the LLM-Ops-Kit proxy tool and compare stock with derived output:
```bash
model-proxy render \
--input sanitized-request.json \
--chat-template ./Qwen-3_5-stock-template.jinja
model-proxy render \
--input sanitized-request.json \
--chat-template ./Qwen-3_5-media-history-template.jinja
```
Do not use a private production capture for a public comparison. Remove hostnames, users, paths, tokens, conversation content, media, tool arguments, and generated artifacts first.
## Install for a Tested llama.cpp Route
Keep both files together in an operator-owned template directory. Point the Qwen model profile at the derivative and ensure the runtime starts llama.cpp with Jinja and the same chat-template file:
```text
--jinja --chat-template-file /path/to/Qwen-3_5-media-history-template.jinja
```
For an LLM-Ops-Kit model profile, the relevant shape is:
```json
{
"template": {
"enabled": true,
"path": "/path/to/Qwen-3_5-media-history-template.jinja"
}
}
```
Plan the restart, apply it, and verify status using your configured stack and component names:
```bash
llmops component plan restart STACK:MODEL_COMPONENT
llmops component restart STACK:MODEL_COMPONENT
llmops component status STACK:MODEL_COMPONENT
```
If the passive model proxy renders diagnostic prompts, point its `--chat-template` setting at the same file. Otherwise the diagnostic view can disagree with what llama.cpp constructs.
Hermes continues sending its OpenAI-compatible message and tool history to the selected Qwen route. The filtering happens when the model runtime renders that history. This package does not require rewriting Hermes requests in a proxy.
## Acceptance and Rollback
Before replacement, retain the model's bundled template, the exact model revision, and one sanitized render fixture. Compare stock and derivative output, then run a bounded generation canary that covers ordinary text, tool calling, native structured vision, and the media-history case you actually need.
If role boundaries, thinking behavior, tool calls, native vision, or generation termination changes unexpectedly, restore the stock fallback immediately:
```json
{
"template": {
"enabled": true,
"path": "/path/to/Qwen-3_5-stock-template.jinja"
}
}
```
Restart the model component and repeat the same sanitized render and generation canaries. Do not keep the derivative merely because it reduces a prompt. Correct model behavior remains the acceptance gate.
## License and Provenance
The Qwen baseline and this derivative are distributed under Apache License 2.0. Read `NOTICE.md` for the immutable upstream URL, revision, modification notice, and checksums. Read `LICENSE-APACHE-2.0.txt` for the license terms.
fixtures.json json View source
{
"native_structured_media": {
"description": "Native structured image and video parts remain eligible for the model engine.",
"payload": {
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "Describe the invented media."},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,STRUCTURED_IMAGE_CANARY"}},
{"type": "video", "video": "invented-video.mp4"}
]
}
],
"tools": [],
"add_generation_prompt": true
}
},
"textual_image_exchanges": {
"description": "Every textual image tool exchange and an assistant-side decode copy are removed.",
"payload": {
"messages": [
{"role": "user", "content": "Create two invented images."},
{
"role": "assistant",
"content": "first image generation call",
"tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "generate first image"}}}]
},
{"role": "tool", "content": "{\"images\":[\"{{PNG_OLD}}\"]}"},
{
"role": "assistant",
"content": "assistant-side image byte copy",
"tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "base64.b64decode(invented_buffer)"}}}]
},
{"role": "tool", "content": "invented decode result"},
{
"role": "assistant",
"content": "second image generation call",
"tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "generate second image"}}}]
},
{"role": "tool", "content": "{\"images\":[\"{{PNG_NEW}}\"]}"},
{"role": "user", "content": "Continue without the textual image history."}
],
"tools": [],
"add_generation_prompt": true
}
},
"latest_audio_video": {
"description": "Only the latest explicit textual audio and video results remain.",
"payload": {
"messages": [
{"role": "user", "content": "Create invented audio and video."},
{"role": "assistant", "content": "old audio generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "old audio"}}}]},
{"role": "tool", "content": "{\"audio\":\"{{AUDIO_OLD}}\"}"},
{"role": "assistant", "content": "latest audio generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "latest audio"}}}]},
{"role": "tool", "content": "{\"audio\":\"{{AUDIO_NEW}}\"}"},
{"role": "assistant", "content": "old video generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "old video"}}}]},
{"role": "tool", "content": "{\"video\":\"{{VIDEO_OLD}}\"}"},
{"role": "assistant", "content": "latest video generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "latest video"}}}]},
{"role": "tool", "content": "{\"video\":\"{{VIDEO_NEW}}\"}"},
{"role": "user", "content": "Continue with only the latest explicit media results."}
],
"tools": [],
"add_generation_prompt": true
}
},
"truncated_image_history": {
"description": "A long truncated textual image result is removed as a complete exchange.",
"payload": {
"messages": [
{"role": "user", "content": "Create an invented image."},
{"role": "assistant", "content": "truncated image generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "generate truncated image"}}}]},
{"role": "tool", "content": "{\"images\":[\"{{TRUNCATED_IMAGE}}\"]}"},
{"role": "user", "content": "Continue after truncation."}
],
"tools": [],
"add_generation_prompt": true
}
},
"incidental_signature_collision": {
"description": "A long ordinary tool result containing an incidental PNG signature is retained when no image-result structure is present.",
"payload": {
"messages": [
{"role": "user", "content": "Inspect an invented diagnostic."},
{"role": "assistant", "content": "ordinary diagnostic call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "inspect diagnostic"}}}]},
{"role": "tool", "content": "{\"note\":\"{{INCIDENTAL_SIGNATURE}}\"}"},
{"role": "user", "content": "Keep the ordinary result."}
],
"tools": [],
"add_generation_prompt": true
}
},
"malformed_ordering": {
"description": "A system message after the first turn fails template validation.",
"payload": {
"messages": [
{"role": "user", "content": "Start normally."},
{"role": "system", "content": "This invented system message is deliberately misplaced."}
],
"tools": [],
"add_generation_prompt": true
}
},
"ordinary_text_tool_history": {
"description": "Ordinary text and tool history renders exactly like the stock baseline.",
"payload": {
"messages": [
{"role": "user", "content": "Check the invented service."},
{"role": "assistant", "content": "checking status", "tool_calls": [{"type": "function", "function": {"name": "service_status", "arguments": {"scope": "local"}}}]},
{"role": "tool", "content": "{\"ok\":true}"},
{"role": "assistant", "content": "The invented service is healthy."},
{"role": "user", "content": "Thank you."}
],
"tools": [],
"add_generation_prompt": true
}
}
}
expected-results.json json View source
{
"native_structured_media": {
"status": "rendered",
"contains": ["Describe the invented media.", "<|image_pad|>", "<|video_pad|>"],
"absent": ["STRUCTURED_IMAGE_CANARY"],
"counts": {"<|image_pad|>": 1, "<|video_pad|>": 1}
},
"textual_image_exchanges": {
"status": "rendered",
"contains": ["Continue without the textual image history."],
"absent": ["PNG_OLD_CANARY", "PNG_NEW_CANARY", "first image generation call", "second image generation call", "assistant-side image byte copy", "base64.b64decode", "invented decode result"]
},
"latest_audio_video": {
"status": "rendered",
"contains": ["AUDIO_NEW_CANARY", "VIDEO_NEW_CANARY", "latest audio generation call", "latest video generation call"],
"absent": ["AUDIO_OLD_CANARY", "VIDEO_OLD_CANARY", "old audio generation call", "old video generation call"]
},
"truncated_image_history": {
"status": "rendered",
"contains": ["Continue after truncation."],
"absent": ["TRUNCATED_IMAGE_CANARY", "OUTPUT TRUNCATED", "truncated image generation call"]
},
"incidental_signature_collision": {
"status": "rendered",
"contains": ["INCIDENTAL_SIGNATURE_CANARY", "ordinary diagnostic call", "Keep the ordinary result."],
"absent": []
},
"malformed_ordering": {
"status": "error",
"error": "System message must be at the beginning."
},
"ordinary_text_tool_history": {
"status": "rendered",
"contains": ["Check the invented service.", "service_status", "{\"ok\":true}", "The invented service is healthy.", "Thank you."],
"absent": [],
"matches_stock": true
}
}
render_fixture.py python View source
#!/usr/bin/env python
"""Render one sanitized fixture with the packaged Qwen media-history template."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from typing import Any
import jinja2
ROOT = Path(__file__).resolve().parent
DEFAULT_TEMPLATE = ROOT / "Qwen-3_5-media-history-template.jinja"
FIXTURES = ROOT / "fixtures.json"
EXPANSIONS = {
"{{PNG_OLD}}": "iVBORw0KGgoPNG_OLD_CANARY" + ("A" * 5000),
"{{PNG_NEW}}": "iVBORw0KGgoPNG_NEW_CANARY" + ("B" * 5000),
"{{AUDIO_OLD}}": "data:audio/wav;base64,AUDIO_OLD_CANARY" + ("C" * 5000),
"{{AUDIO_NEW}}": "data:audio/wav;base64,AUDIO_NEW_CANARY" + ("D" * 5000),
"{{VIDEO_OLD}}": "data:video/mp4;base64,VIDEO_OLD_CANARY" + ("E" * 5000),
"{{VIDEO_NEW}}": "data:video/mp4;base64,VIDEO_NEW_CANARY" + ("F" * 5000),
"{{TRUNCATED_IMAGE}}": (
"iVBORw0KGgoTRUNCATED_IMAGE_CANARY\n\n"
"... [OUTPUT TRUNCATED - invented fixture] ...\n\n"
+ ("G" * 5000)
),
"{{INCIDENTAL_SIGNATURE}}": "iVBORw0KGgoINCIDENTAL_SIGNATURE_CANARY" + ("H" * 5000),
}
def expand(value: Any) -> Any:
if isinstance(value, str):
for marker, replacement in EXPANSIONS.items():
value = value.replace(marker, replacement)
return value
if isinstance(value, list):
return [expand(item) for item in value]
if isinstance(value, dict):
return {key: expand(item) for key, item in value.items()}
return value
def raise_exception(message: str) -> None:
raise jinja2.TemplateError(message)
def load_template(path: Path) -> jinja2.Template:
environment = jinja2.Environment(
undefined=jinja2.ChainableUndefined,
trim_blocks=False,
lstrip_blocks=False,
autoescape=False,
)
environment.globals["raise_exception"] = raise_exception
return environment.from_string(path.read_text(encoding="utf-8"))
def render_case(case: dict[str, Any], template_path: Path = DEFAULT_TEMPLATE) -> str:
payload = expand(case["payload"])
template = load_template(template_path)
return template.render(
messages=payload["messages"],
tools=payload.get("tools", []),
add_generation_prompt=payload.get("add_generation_prompt", True),
enable_thinking=payload.get("enable_thinking", True),
add_vision_id=payload.get("add_vision_id", False),
bos_token=payload.get("bos_token", ""),
eos_token=payload.get("eos_token", ""),
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("case", help="Fixture name from fixtures.json")
parser.add_argument("--template", type=Path, default=DEFAULT_TEMPLATE)
args = parser.parse_args()
cases = json.loads(FIXTURES.read_text(encoding="utf-8"))
if args.case not in cases:
parser.error(f"unknown fixture: {args.case}")
try:
print(render_case(cases[args.case], args.template))
except jinja2.TemplateError as exc:
print(f"TEMPLATE ERROR: {exc}")
return 2
return 0
if __name__ == "__main__":
raise SystemExit(main())
test_media_history_template.py python View source
#!/usr/bin/env python
from __future__ import annotations
import hashlib
import json
import unittest
from pathlib import Path
import jinja2
from render_fixture import ROOT, render_case
UPSTREAM_SHA256 = "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715"
STOCK_SHA256 = "d2cb9a5730cdd5f44bce3ada2dc1b0e00c6c59788b6d1c4d8d49c40a274dffb0"
DERIVATIVE_SHA256 = "162671aeaf5e2c39966816dae53e5e6f8ac0dfb97d53f34094afe74e44b2fae6"
def sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
class MediaHistoryTemplateTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.cases = json.loads((ROOT / "fixtures.json").read_text(encoding="utf-8"))
cls.expected = json.loads((ROOT / "expected-results.json").read_text(encoding="utf-8"))
def test_provenance_checksums(self) -> None:
stock = (ROOT / "Qwen-3_5-stock-template.jinja").read_bytes()
derivative = (ROOT / "Qwen-3_5-media-history-template.jinja").read_bytes()
self.assertEqual(sha256(stock), STOCK_SHA256)
self.assertEqual(sha256(stock.removesuffix(b"\n")), UPSTREAM_SHA256)
self.assertEqual(sha256(derivative), DERIVATIVE_SHA256)
def test_sanitized_fixture_contracts(self) -> None:
for name, case in self.cases.items():
with self.subTest(case=name):
expected = self.expected[name]
if expected["status"] == "error":
with self.assertRaisesRegex(jinja2.TemplateError, expected["error"]):
render_case(case)
continue
rendered = render_case(case)
for marker in expected.get("contains", []):
self.assertIn(marker, rendered)
for marker in expected.get("absent", []):
self.assertNotIn(marker, rendered)
for marker, count in expected.get("counts", {}).items():
self.assertEqual(rendered.count(marker), count)
if expected.get("matches_stock"):
stock_rendered = render_case(
case,
ROOT / "Qwen-3_5-stock-template.jinja",
)
self.assertEqual(rendered, stock_rendered)
def test_fixture_and_expectation_names_match(self) -> None:
self.assertEqual(set(self.cases), set(self.expected))
if __name__ == "__main__":
unittest.main()
NOTICE.md markdown View source
# Attribution and Modification Notice
`Qwen-3_5-media-history-template.jinja` is a modified form of Qwen's `chat_template.jinja` from `Qwen/Qwen3.5-27B`.
Upstream source:
- Repository: `Qwen/Qwen3.5-27B`
- Immutable revision: `feea018b31f89dc0950e61da42577a7a4ab09169`
- Source file: `https://huggingface.co/Qwen/Qwen3.5-27B/blob/feea018b31f89dc0950e61da42577a7a4ab09169/chat_template.jinja`
- Raw source: `https://huggingface.co/Qwen/Qwen3.5-27B/resolve/feea018b31f89dc0950e61da42577a7a4ab09169/chat_template.jinja`
- Upstream license: Apache License 2.0
The derivative adds a media-history policy that:
- removes every textual image tool result and its associated assistant tool exchange;
- removes assistant tool calls that explicitly carry or decode historical media bytes;
- retains only the latest explicit textual audio result and latest explicit textual video result;
- preserves native structured image and video message parts and their Qwen vision placeholders;
- requires media-result structure, known data markers, and a content length greater than 4,096 characters for textual tool-result detection.
The derivative does not validate base64 data. It does not claim compatibility with arbitrary ChatML, Llama, Mistral, Gemma, OpenAI, or other model-family templates.
## SHA-256 Checksums
| Artifact | SHA-256 |
| --- | --- |
| Immutable upstream `chat_template.jinja` bytes | `a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715` |
| Packaged stock fallback with one trailing newline | `d2cb9a5730cdd5f44bce3ada2dc1b0e00c6c59788b6d1c4d8d49c40a274dffb0` |
| Published attributed media-history derivative | `162671aeaf5e2c39966816dae53e5e6f8ac0dfb97d53f34094afe74e44b2fae6` |
The packaged stock fallback is byte-equivalent to the immutable upstream file after removing its single trailing newline. The test suite verifies both forms and the derivative checksum without requiring network access.
The template files and modifications in this companion are distributed under Apache License 2.0. They are not presented as solely covered by the surrounding repository's MIT license. See `LICENSE-APACHE-2.0.txt`.
LICENSE-APACHE-2.0.txt text View source
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
A Small Change Still Needs a Rollback Path
Replacing a model’s bundled chat template may look like a small configuration change, but it is not cosmetic. One missing delimiter, changed tool schema, damaged reasoning boundary, or misplaced generation marker can make a healthy model behave as though the entire serving stack is broken.
My acceptance sequence begins with rendering sanitized fixtures under both templates. Ordinary text and tool history must match the stock baseline, and the media cases must show only the intended differences. I then run bounded generation canaries for ordinary text, tool calling, native structured vision, and the media-history case that motivated the change. Only after those checks would I point the model profile at the derivative and restart the selected component through the normal plan, apply, and status path. If the passive proxy renders diagnostic prompts, it receives the same template selection so its evidence agrees with the runtime. The optional lab section shows how to compare those two diagnostic renders without claiming that render-only mode proves live forwarding.
Rollback is simply the stock template kept beside the derivative. Restoring that path and repeating the same canaries is faster and safer than editing the derivative under pressure, which is exactly when I am least likely to make a careful template repair. The companion README records the profile shape, restart sequence, checksums, acceptance gates, and stock rollback procedure.
Hermes continues sending its OpenAI-compatible message and tool history. It does not need a request-rewriting plugin for this policy because the selected matching Qwen runtime applies the policy during prompt rendering. The template has also worked with a matching later Qwen prompt format in my environment, but I treat that as an operational observation rather than a support claim. I would not drop this file into Llama, Mistral, Gemma, an OpenAI model, or an arbitrary ChatML-style runtime. The pruning policy can be ported, but the destination template must preserve that model’s own control tokens, multimodal markers, reasoning rules, tool schema, and generation boundary. Similar-looking envelopes are not a compatibility test.
Current State
The Qwen 3.5-family media-history derivative is implemented and covered by the current LLM-Ops-Kit regression suite. The publication companion adds an attributed derivative, stock fallback, Apache-2.0 license and notice, immutable provenance, checksums, seven synthetic fixtures, explicit expected results, a manual renderer, and self-contained tests. Its three test methods pass under Jinja 3.1.6, and the underlying production suite covers the central structured-media and passive-proxy boundaries.
There is bounded operational evidence for the matching Qwen route used in my environment and a successful Hermes history replay. That does not establish universal model compatibility or a benchmark result. The package is presented as a tested technical companion, not a generic replacement for another model’s bundled template.
Next Work
What remains is narrower and easier to name. I want an explicit final-artifact generation pass against the selected model revision, followed by repeated checks for ordinary tools, native vision, reasoning boundaries, and termination. More general tool-call reduction stays deferred because it can change semantics in ways this media-history cleanup does not.
The next main installment returns to the larger operations story: how the collection of scripts became an operator-ready LLM-Ops-Kit with typed adapters, dependency-aware plans, immutable releases, and rollback that does not depend on remembering which shell command happened to work last time.
Join the Discussion
Comments for this post live in GitHub Discussions. That keeps moderation in one place and gives the conversation a stable home.