Hands-On: Put Multimodal Context Hygiene Through a Test Bench
In the main Part 7 article, I explained why media-history filtering belongs in the selected Qwen Jinja template instead of inside a model proxy that is supposed to remain passive. This companion lets you take that explanation apart and see whether it actually holds up.
I did not want to publish another page of source code and ask readers to accept it on faith. The useful part is running the same invented conversation through both templates and asking plain questions: Did the old image payload disappear? Did the latest audio result and native vision placeholder remain? Did ordinary tool history stay byte-for-byte identical?
That is the whole point of this lab. You do not need a model, GPU, private capture, or production agent configuration; Python and Jinja are enough for the core exercise. If LLM-Ops-Kit is already installed, the optional final section sends the same sanitized request through the model-proxy renderer. It shows where the proxy fits without pretending that the proxy performs the filtering.
What You Will Build and Verify
The package contains both Qwen templates, seven invented fixtures, expected results, two small Python tools, and two test modules. inspect_fixture.py renders each fixture twice and reports a bounded comparison without printing the long synthetic payloads.
The lab gives you enough evidence to verify these boundaries yourself:
| Experiment | Result that must hold |
|---|---|
| Ordinary text and tool history | Stock and derived renders match exactly |
| Textual image history | Every image-producing call and result pair is absent from the derived render |
| Explicit textual audio and video history | Only the latest result of each type remains |
| Native structured image and video parts | Qwen image and video placeholders remain |
| Incidental PNG-looking text | Ordinary tool output remains when image-result structure is absent |
| Invalid system-message ordering | Both templates reject it with the intended validation error |
| Provenance | The packaged stock and derivative checksums match the reviewed files |
The character counts in this lab are measurements of invented rendered strings. They are not token counts, cost estimates, latency results, or generation-quality scores. I am using them because they make the transformation visible without pretending that one synthetic fixture predicts a production workload.
Step 1: Unpack the Lab in Its Own Directory
Download the complete twelve-file Hands-On 7A package and expand it into a fresh working directory. Do not mix it into a model installation yet. I want the first pass to be about understanding the template behavior before a runtime or live model can complicate the result.
Every file is also available below through the site’s standard source viewer. The disclosures stay collapsed until you choose one, and each file can be downloaded separately.
Qwen-3_5-media-history-template.jinja jinja View source
{#
Modified Qwen 3.5 media-history chat template.
Upstream: Qwen/Qwen3.5-27B chat_template.jinja
Revision: feea018b31f89dc0950e61da42577a7a4ab09169
Source: https://huggingface.co/Qwen/Qwen3.5-27B/blob/feea018b31f89dc0950e61da42577a7a4ab09169/chat_template.jinja
License: Apache License 2.0. See LICENSE-APACHE-2.0.txt and NOTICE.md.
Modifications: adds media-history policy that removes textual image tool
exchanges and assistant-side media byte copies, retains only the latest explicit
audio and video tool results, and preserves native structured image and video
parts. The policy uses structural and marker checks with a 4,096-character
detection threshold. Jinja does not validate base64 data.
#}
{%- set image_count = namespace(value=0) %}
{%- set video_count = namespace(value=0) %}
{%- macro render_content(content, do_vision_count, is_system_content=false, message_index=-1) %}
{%- if content is string %}
{{- content }}
{%- elif content is iterable and content is not mapping %}
{%- for item in content %}
{%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain images.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set image_count.value = image_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Picture ' ~ image_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
{%- elif 'video' in item or item.type == 'video' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain videos.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set video_count.value = video_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Video ' ~ video_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
{%- elif 'text' in item %}
{{- item.text }}
{%- else %}
{{- raise_exception('Unexpected item type in content.') }}
{%- endif %}
{%- endfor %}
{%- elif content is none or content is undefined %}
{{- '' }}
{%- else %}
{{- raise_exception('Unexpected content type.') }}
{%- endif %}
{%- endmacro %}
{%- if not messages %}
{{- raise_exception('No messages provided.') }}
{%- endif %}
{%- set media_state = namespace(last_audio_message_index=-1, last_video_message_index=-1) %}
{%- for message in messages %}
{%- if message.role == 'tool' and message.content is string and message.content|length > 4096 %}
{%- if ('audio' in message.content or 'audios' in message.content) and 'data:audio/' in message.content %}
{%- set media_state.last_audio_message_index = loop.index0 %}
{%- endif %}
{%- if ('video' in message.content or 'videos' in message.content) and 'data:video/' in message.content %}
{%- set media_state.last_video_message_index = loop.index0 %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set skip_state = namespace(indices=[]) %}
{%- for message in messages %}
{%- set message_index = loop.index0 %}
{%- set content_state = namespace(has_media=false, keep_media=false) %}
{%- if message.role == 'tool' and message.content is string and message.content|length > 4096 %}
{%- if 'images' in message.content and ('iVBORw0KGgo' in message.content or '/9j/' in message.content or 'data:image/' in message.content) %}
{%- set content_state.has_media = true %}
{%- endif %}
{%- if ('audio' in message.content or 'audios' in message.content) and 'data:audio/' in message.content %}
{%- set content_state.has_media = true %}
{%- if message_index == media_state.last_audio_message_index %}{%- set content_state.keep_media = true %}{%- endif %}
{%- endif %}
{%- if ('video' in message.content or 'videos' in message.content) and 'data:video/' in message.content %}
{%- set content_state.has_media = true %}
{%- if message_index == media_state.last_video_message_index %}{%- set content_state.keep_media = true %}{%- endif %}
{%- endif %}
{%- endif %}
{%- if content_state.has_media and not content_state.keep_media %}
{%- set _ = skip_state.indices.append(message_index) %}
{%- if message_index > 0 and messages[message_index - 1].role == 'assistant' %}
{%- set _ = skip_state.indices.append(message_index - 1) %}
{%- endif %}
{%- endif %}
{%- if message.role == 'assistant' and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
{%- set tool_state = namespace(has_media=false) %}
{%- for original_tool_call in message.tool_calls %}
{%- set tool_call = original_tool_call.function if original_tool_call.function is defined else original_tool_call %}
{%- if tool_call.arguments is string %}
{%- if 'data:image/' in tool_call.arguments or 'iVBORw0KGgo' in tool_call.arguments or '/9j/' in tool_call.arguments or 'data:audio/' in tool_call.arguments or 'data:video/' in tool_call.arguments or 'base64.b64decode' in tool_call.arguments or 'base64 --decode' in tool_call.arguments or 'base64 -d' in tool_call.arguments %}
{%- set tool_state.has_media = true %}
{%- endif %}
{%- elif tool_call.arguments is mapping %}
{%- for args_name, args_value in tool_call.arguments|items %}
{%- set args_text = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
{%- if 'data:image/' in args_text or 'iVBORw0KGgo' in args_text or '/9j/' in args_text or 'data:audio/' in args_text or 'data:video/' in args_text or 'base64.b64decode' in args_text or 'base64 --decode' in args_text or 'base64 -d' in args_text %}
{%- set tool_state.has_media = true %}
{%- endif %}
{%- endfor %}
{%- endif %}
{%- endfor %}
{%- if tool_state.has_media %}
{%- set _ = skip_state.indices.append(message_index) %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set following_state = namespace(skip_tools=false) %}
{%- for message in messages %}
{%- if message.role == 'assistant' %}
{%- set following_state.skip_tools = loop.index0 in skip_state.indices %}
{%- elif message.role == 'tool' and following_state.skip_tools %}
{%- set _ = skip_state.indices.append(loop.index0) %}
{%- else %}
{%- set following_state.skip_tools = false %}
{%- endif %}
{%- endfor %}
{%- if tools and tools is iterable and tools is not mapping %}
{{- '<|im_start|>system\n' }}
{{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>" }}
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{%- if content %}
{{- '\n\n' + content }}
{%- endif %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- else %}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" %}
{%- set content = render_content(message.content, false, false, index)|trim %}
{%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if ns.multi_step_tool %}
{{- raise_exception('No user query found in messages.') }}
{%- endif %}
{%- for message in messages %}
{%- if loop.index0 not in skip_state.indices %}
{%- set content = render_content(message.content, true, false, loop.index0)|trim %}
{%- if message.role == "system" %}
{%- if not loop.first %}
{{- raise_exception('System message must be at the beginning.') }}
{%- endif %}
{%- elif message.role == "user" %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- set reasoning_content = reasoning_content|trim %}
{%- if loop.index0 > ns.last_query_index %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{%- if loop.first %}
{%- if content|trim %}
{{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- else %}
{{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- else %}
{{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- if tool_call.arguments is defined %}
{%- for args_name, args_value in tool_call.arguments|items %}
{{- '<parameter=' + args_name + '>\n' }}
{%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
{%- if args_value|length > 4096 and ('data:image/' in args_value or 'iVBORw0KGgo' in args_value or '/9j/' in args_value or 'data:audio/' in args_value or 'data:video/' in args_value) %}
{{- '[Embedded media payload omitted from tool call]' }}
{%- else %}
{{- args_value }}
{%- endif %}
{{- '\n</parameter>\n' }}
{%- endfor %}
{%- endif %}
{{- '</function>\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.previtem and loop.previtem.role != "tool" %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if not loop.last and loop.nextitem.role != "tool" %}
{{- '<|im_end|>\n' }}
{%- elif loop.last %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- else %}
{{- raise_exception('Unexpected message role.') }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- else %}
{{- '<think>\n' }}
{%- endif %}
{%- endif %}
Qwen-3_5-stock-template.jinja jinja View source
{%- set image_count = namespace(value=0) %}
{%- set video_count = namespace(value=0) %}
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
{%- if content is string %}
{{- content }}
{%- elif content is iterable and content is not mapping %}
{%- for item in content %}
{%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain images.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set image_count.value = image_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Picture ' ~ image_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
{%- elif 'video' in item or item.type == 'video' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain videos.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set video_count.value = video_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Video ' ~ video_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
{%- elif 'text' in item %}
{{- item.text }}
{%- else %}
{{- raise_exception('Unexpected item type in content.') }}
{%- endif %}
{%- endfor %}
{%- elif content is none or content is undefined %}
{{- '' }}
{%- else %}
{{- raise_exception('Unexpected content type.') }}
{%- endif %}
{%- endmacro %}
{%- if not messages %}
{{- raise_exception('No messages provided.') }}
{%- endif %}
{%- if tools and tools is iterable and tools is not mapping %}
{{- '<|im_start|>system\n' }}
{{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>" }}
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{%- if content %}
{{- '\n\n' + content }}
{%- endif %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- else %}
{%- if messages[0].role == 'system' %}
{%- set content = render_content(messages[0].content, false, true)|trim %}
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == "user" %}
{%- set content = render_content(message.content, false)|trim %}
{%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if ns.multi_step_tool %}
{{- raise_exception('No user query found in messages.') }}
{%- endif %}
{%- for message in messages %}
{%- set content = render_content(message.content, true)|trim %}
{%- if message.role == "system" %}
{%- if not loop.first %}
{{- raise_exception('System message must be at the beginning.') }}
{%- endif %}
{%- elif message.role == "user" %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- set reasoning_content = reasoning_content|trim %}
{%- if loop.index0 > ns.last_query_index %}
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{%- if loop.first %}
{%- if content|trim %}
{{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- else %}
{{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- else %}
{{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- if tool_call.arguments is defined %}
{%- for args_name, args_value in tool_call.arguments|items %}
{{- '<parameter=' + args_name + '>\n' }}
{%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
{{- args_value }}
{{- '\n</parameter>\n' }}
{%- endfor %}
{%- endif %}
{{- '</function>\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.previtem and loop.previtem.role != "tool" %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- content }}
{{- '\n</tool_response>' }}
{%- if not loop.last and loop.nextitem.role != "tool" %}
{{- '<|im_end|>\n' }}
{%- elif loop.last %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- else %}
{{- raise_exception('Unexpected message role.') }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- else %}
{{- '<think>\n' }}
{%- endif %}
{%- endif %}
README.md markdown View source
# Hands-On 7A: Qwen Media-History Template Test Bench
This self-contained lab package accompanies Hands-On 7A of the Local-First Agent Operations series. It gives readers a reviewable Qwen chat template, an unchanged stock fallback, sanitized fixtures, expected results, a bounded stock-versus-derived inspector, executable tests, provenance checks, and rollback instructions.
It is not a generic ChatML template. Qwen uses `<|im_start|>` and `<|im_end|>` as its ChatML turn envelope, and its tool calling is Hermes-like, but this complete file also depends on Qwen vision placeholders, thinking layout, tool-call serialization, and Jinja runtime helpers. The media-history policy may be ported to another ChatML-style template only after preserving that model's control tokens, multimodal markers, reasoning rules, tool schema, and assistant generation boundary.
## Package Contents
| File | Purpose |
| --- | --- |
| `Qwen-3_5-media-history-template.jinja` | Attributed Apache-2.0 derivative with media-history pruning |
| `Qwen-3_5-stock-template.jinja` | Unmodified fallback baseline, apart from one trailing newline |
| `fixtures.json` | Seven invented, compact fixtures expanded locally into long synthetic media markers |
| `expected-results.json` | Expected retained, omitted, counted, error, and stock-equivalence results |
| `render_fixture.py` | Renders one fixture for manual inspection |
| `inspect_fixture.py` | Produces a bounded stock-versus-derived comparison without printing synthetic long payloads |
| `test_media_history_template.py` | Self-contained checksum and behavior tests |
| `test_inspect_fixture.py` | Verifies the bounded inspector and its reader-facing claims |
| `requirements.txt` | Pins the Jinja version used by the standalone lab |
| `NOTICE.md` | Immutable provenance, modification notice, checksums, and license boundary |
| `LICENSE-APACHE-2.0.txt` | Apache License 2.0 text |
## Supported Input Assumptions
The tested runtime supplies Jinja 3.1.6 and the `raise_exception` helper used by the Qwen template. The render context includes `messages`, `tools`, `add_generation_prompt`, `enable_thinking`, `add_vision_id`, `bos_token`, and `eos_token`.
Messages use the Qwen 3.5-family shape exercised by the fixtures:
- roles are `system`, `user`, `assistant`, or `tool`;
- content is a string, `null`, or a list of structured text, image, or video parts;
- assistant `tool_calls` are iterable and each call exposes a function name and argument mapping;
- tool results are textual strings when the media-history policy inspects them;
- native structured images and videos are processed separately by the model engine after the template emits Qwen vision placeholders.
The template has also worked with the matching Qwen 3.6 prompt format in the publisher's environment. That is an operational observation, not universal compatibility evidence. Render and generation acceptance are still required for the exact model artifact and runtime you use.
## What the Policy Does
Textual image tool results are removed as complete exchanges, including the most recent result. They are often truncated before they reach history and cannot be reconstructed reliably. Native structured image and video parts remain eligible and render Qwen vision placeholders.
For explicit textual audio and video results, only the latest result of each type remains. Older associated assistant calls and tool results are removed. Assistant tool calls that carry recognizable media bytes or explicitly decode base64 are also removed.
Textual tool-result detection requires content longer than 4,096 characters. Image detection also requires image-result structure plus a PNG, JPEG, or `data:image/` marker. Audio and video detection require their corresponding result labels and data-URL markers. Jinja does not validate base64. The threshold and marker checks are routing policy, not proof that the payload is valid media.
## Stock Versus Derived Behavior
| Input history | Stock Qwen template | Media-history derivative |
| --- | --- | --- |
| Ordinary text and tool history | Retained | Byte-identical rendered prompt in the supplied fixture |
| Native structured image or video part | Qwen vision placeholder retained | Qwen vision placeholder retained |
| Textual image tool result | Retained as tool-response text | Entire image-producing exchange removed |
| Assistant-side media byte or decode call | Retained | Associated assistant call and following tool result removed |
| Multiple explicit textual audio results | All retained | Only latest explicit audio exchange retained |
| Multiple explicit textual video results | All retained | Only latest explicit video exchange retained |
| Long incidental PNG signature without image-result structure | Retained | Retained |
| Malformed system-message ordering | Template error | Same template error |
## Verify Before Installing
Run the complete checksum, behavior, inspector, and threshold suite:
```bash
python -m unittest -v test_media_history_template.py test_inspect_fixture.py
```
Render individual sanitized fixtures before touching a model profile:
```bash
python render_fixture.py native_structured_media
python render_fixture.py textual_image_exchanges
python render_fixture.py ordinary_text_tool_history
```
Compare every stock and derived render without printing the long synthetic payloads:
```bash
python inspect_fixture.py
python inspect_fixture.py textual_image_exchanges --json
```
The inspector reports rendered character counts, character deltas, expectation checks, and Qwen vision-placeholder counts. Character deltas are measurements of the supplied synthetic renders. They are not token, latency, generation-quality, or cost benchmarks.
The fixture payloads are invented. Long base64-like strings are generated locally from obvious canary markers and repeated letters; they are not real images, audio, video, prompts, conversations, or model output.
If you have a sanitized captured OpenAI-compatible request, render it with the LLM-Ops-Kit proxy tool and compare stock with derived output:
```bash
model-proxy render \
--input sanitized-request.json \
--chat-template ./Qwen-3_5-stock-template.jinja
model-proxy render \
--input sanitized-request.json \
--chat-template ./Qwen-3_5-media-history-template.jinja
```
Do not use a private production capture for a public comparison. Remove hostnames, users, paths, tokens, conversation content, media, tool arguments, and generated artifacts first.
## Install for a Tested llama.cpp Route
Keep both files together in an operator-owned template directory. Point the Qwen model profile at the derivative and ensure the runtime starts llama.cpp with Jinja and the same chat-template file:
```text
--jinja --chat-template-file /path/to/Qwen-3_5-media-history-template.jinja
```
For an LLM-Ops-Kit model profile, the relevant shape is:
```json
{
"template": {
"enabled": true,
"path": "/path/to/Qwen-3_5-media-history-template.jinja"
}
}
```
Plan the restart, apply it, and verify status using your configured stack and component names:
```bash
llmops component plan restart STACK:MODEL_COMPONENT
llmops component restart STACK:MODEL_COMPONENT
llmops component status STACK:MODEL_COMPONENT
```
If the passive model proxy renders diagnostic prompts, point its `--chat-template` setting at the same file. Otherwise the diagnostic view can disagree with what llama.cpp constructs.
Hermes continues sending its OpenAI-compatible message and tool history to the selected Qwen route. The filtering happens when the model runtime renders that history. This package does not require rewriting Hermes requests in a proxy.
## Acceptance and Rollback
Before replacement, retain the model's bundled template, the exact model revision, and one sanitized render fixture. Compare stock and derivative output, then run a bounded generation canary that covers ordinary text, tool calling, native structured vision, and the media-history case you actually need.
If role boundaries, thinking behavior, tool calls, native vision, or generation termination changes unexpectedly, restore the stock fallback immediately:
```json
{
"template": {
"enabled": true,
"path": "/path/to/Qwen-3_5-stock-template.jinja"
}
}
```
Restart the model component and repeat the same sanitized render and generation canaries. Do not keep the derivative merely because it reduces a prompt. Correct model behavior remains the acceptance gate.
## License and Provenance
The Qwen baseline and this derivative are distributed under Apache License 2.0. Read `NOTICE.md` for the immutable upstream URL, revision, modification notice, and checksums. Read `LICENSE-APACHE-2.0.txt` for the license terms.
fixtures.json json View source
{
"native_structured_media": {
"description": "Native structured image and video parts remain eligible for the model engine.",
"payload": {
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "Describe the invented media."},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,STRUCTURED_IMAGE_CANARY"}},
{"type": "video", "video": "invented-video.mp4"}
]
}
],
"tools": [],
"add_generation_prompt": true
}
},
"textual_image_exchanges": {
"description": "Every textual image tool exchange and an assistant-side decode copy are removed.",
"payload": {
"messages": [
{"role": "user", "content": "Create two invented images."},
{
"role": "assistant",
"content": "first image generation call",
"tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "generate first image"}}}]
},
{"role": "tool", "content": "{\"images\":[\"{{PNG_OLD}}\"]}"},
{
"role": "assistant",
"content": "assistant-side image byte copy",
"tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "base64.b64decode(invented_buffer)"}}}]
},
{"role": "tool", "content": "invented decode result"},
{
"role": "assistant",
"content": "second image generation call",
"tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "generate second image"}}}]
},
{"role": "tool", "content": "{\"images\":[\"{{PNG_NEW}}\"]}"},
{"role": "user", "content": "Continue without the textual image history."}
],
"tools": [],
"add_generation_prompt": true
}
},
"latest_audio_video": {
"description": "Only the latest explicit textual audio and video results remain.",
"payload": {
"messages": [
{"role": "user", "content": "Create invented audio and video."},
{"role": "assistant", "content": "old audio generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "old audio"}}}]},
{"role": "tool", "content": "{\"audio\":\"{{AUDIO_OLD}}\"}"},
{"role": "assistant", "content": "latest audio generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "latest audio"}}}]},
{"role": "tool", "content": "{\"audio\":\"{{AUDIO_NEW}}\"}"},
{"role": "assistant", "content": "old video generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "old video"}}}]},
{"role": "tool", "content": "{\"video\":\"{{VIDEO_OLD}}\"}"},
{"role": "assistant", "content": "latest video generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "latest video"}}}]},
{"role": "tool", "content": "{\"video\":\"{{VIDEO_NEW}}\"}"},
{"role": "user", "content": "Continue with only the latest explicit media results."}
],
"tools": [],
"add_generation_prompt": true
}
},
"truncated_image_history": {
"description": "A long truncated textual image result is removed as a complete exchange.",
"payload": {
"messages": [
{"role": "user", "content": "Create an invented image."},
{"role": "assistant", "content": "truncated image generation call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "generate truncated image"}}}]},
{"role": "tool", "content": "{\"images\":[\"{{TRUNCATED_IMAGE}}\"]}"},
{"role": "user", "content": "Continue after truncation."}
],
"tools": [],
"add_generation_prompt": true
}
},
"incidental_signature_collision": {
"description": "A long ordinary tool result containing an incidental PNG signature is retained when no image-result structure is present.",
"payload": {
"messages": [
{"role": "user", "content": "Inspect an invented diagnostic."},
{"role": "assistant", "content": "ordinary diagnostic call", "tool_calls": [{"type": "function", "function": {"name": "terminal", "arguments": {"command": "inspect diagnostic"}}}]},
{"role": "tool", "content": "{\"note\":\"{{INCIDENTAL_SIGNATURE}}\"}"},
{"role": "user", "content": "Keep the ordinary result."}
],
"tools": [],
"add_generation_prompt": true
}
},
"malformed_ordering": {
"description": "A system message after the first turn fails template validation.",
"payload": {
"messages": [
{"role": "user", "content": "Start normally."},
{"role": "system", "content": "This invented system message is deliberately misplaced."}
],
"tools": [],
"add_generation_prompt": true
}
},
"ordinary_text_tool_history": {
"description": "Ordinary text and tool history renders exactly like the stock baseline.",
"payload": {
"messages": [
{"role": "user", "content": "Check the invented service."},
{"role": "assistant", "content": "checking status", "tool_calls": [{"type": "function", "function": {"name": "service_status", "arguments": {"scope": "local"}}}]},
{"role": "tool", "content": "{\"ok\":true}"},
{"role": "assistant", "content": "The invented service is healthy."},
{"role": "user", "content": "Thank you."}
],
"tools": [],
"add_generation_prompt": true
}
}
}
expected-results.json json View source
{
"native_structured_media": {
"status": "rendered",
"contains": ["Describe the invented media.", "<|image_pad|>", "<|video_pad|>"],
"absent": ["STRUCTURED_IMAGE_CANARY"],
"counts": {"<|image_pad|>": 1, "<|video_pad|>": 1}
},
"textual_image_exchanges": {
"status": "rendered",
"contains": ["Continue without the textual image history."],
"absent": ["PNG_OLD_CANARY", "PNG_NEW_CANARY", "first image generation call", "second image generation call", "assistant-side image byte copy", "base64.b64decode", "invented decode result"]
},
"latest_audio_video": {
"status": "rendered",
"contains": ["AUDIO_NEW_CANARY", "VIDEO_NEW_CANARY", "latest audio generation call", "latest video generation call"],
"absent": ["AUDIO_OLD_CANARY", "VIDEO_OLD_CANARY", "old audio generation call", "old video generation call"]
},
"truncated_image_history": {
"status": "rendered",
"contains": ["Continue after truncation."],
"absent": ["TRUNCATED_IMAGE_CANARY", "OUTPUT TRUNCATED", "truncated image generation call"]
},
"incidental_signature_collision": {
"status": "rendered",
"contains": ["INCIDENTAL_SIGNATURE_CANARY", "ordinary diagnostic call", "Keep the ordinary result."],
"absent": []
},
"malformed_ordering": {
"status": "error",
"error": "System message must be at the beginning."
},
"ordinary_text_tool_history": {
"status": "rendered",
"contains": ["Check the invented service.", "service_status", "{\"ok\":true}", "The invented service is healthy.", "Thank you."],
"absent": [],
"matches_stock": true
}
}
render_fixture.py python View source
#!/usr/bin/env python
"""Render one sanitized fixture with the packaged Qwen media-history template."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from typing import Any
import jinja2
ROOT = Path(__file__).resolve().parent
DEFAULT_TEMPLATE = ROOT / "Qwen-3_5-media-history-template.jinja"
FIXTURES = ROOT / "fixtures.json"
EXPANSIONS = {
"{{PNG_OLD}}": "iVBORw0KGgoPNG_OLD_CANARY" + ("A" * 5000),
"{{PNG_NEW}}": "iVBORw0KGgoPNG_NEW_CANARY" + ("B" * 5000),
"{{AUDIO_OLD}}": "data:audio/wav;base64,AUDIO_OLD_CANARY" + ("C" * 5000),
"{{AUDIO_NEW}}": "data:audio/wav;base64,AUDIO_NEW_CANARY" + ("D" * 5000),
"{{VIDEO_OLD}}": "data:video/mp4;base64,VIDEO_OLD_CANARY" + ("E" * 5000),
"{{VIDEO_NEW}}": "data:video/mp4;base64,VIDEO_NEW_CANARY" + ("F" * 5000),
"{{TRUNCATED_IMAGE}}": (
"iVBORw0KGgoTRUNCATED_IMAGE_CANARY\n\n"
"... [OUTPUT TRUNCATED - invented fixture] ...\n\n"
+ ("G" * 5000)
),
"{{INCIDENTAL_SIGNATURE}}": "iVBORw0KGgoINCIDENTAL_SIGNATURE_CANARY" + ("H" * 5000),
}
def expand(value: Any) -> Any:
if isinstance(value, str):
for marker, replacement in EXPANSIONS.items():
value = value.replace(marker, replacement)
return value
if isinstance(value, list):
return [expand(item) for item in value]
if isinstance(value, dict):
return {key: expand(item) for key, item in value.items()}
return value
def raise_exception(message: str) -> None:
raise jinja2.TemplateError(message)
def load_template(path: Path) -> jinja2.Template:
environment = jinja2.Environment(
undefined=jinja2.ChainableUndefined,
trim_blocks=False,
lstrip_blocks=False,
autoescape=False,
)
environment.globals["raise_exception"] = raise_exception
return environment.from_string(path.read_text(encoding="utf-8"))
def render_case(case: dict[str, Any], template_path: Path = DEFAULT_TEMPLATE) -> str:
payload = expand(case["payload"])
template = load_template(template_path)
return template.render(
messages=payload["messages"],
tools=payload.get("tools", []),
add_generation_prompt=payload.get("add_generation_prompt", True),
enable_thinking=payload.get("enable_thinking", True),
add_vision_id=payload.get("add_vision_id", False),
bos_token=payload.get("bos_token", ""),
eos_token=payload.get("eos_token", ""),
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("case", help="Fixture name from fixtures.json")
parser.add_argument("--template", type=Path, default=DEFAULT_TEMPLATE)
args = parser.parse_args()
cases = json.loads(FIXTURES.read_text(encoding="utf-8"))
if args.case not in cases:
parser.error(f"unknown fixture: {args.case}")
try:
print(render_case(cases[args.case], args.template))
except jinja2.TemplateError as exc:
print(f"TEMPLATE ERROR: {exc}")
return 2
return 0
if __name__ == "__main__":
raise SystemExit(main())
inspect_fixture.py python View source
#!/usr/bin/env python
"""Compare sanitized fixtures under the stock and media-history templates."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from typing import Any, Dict, List, Optional
import jinja2
from render_fixture import ROOT, render_case
STOCK_TEMPLATE = ROOT / "Qwen-3_5-stock-template.jinja"
DERIVATIVE_TEMPLATE = ROOT / "Qwen-3_5-media-history-template.jinja"
EXPECTED_RESULTS = ROOT / "expected-results.json"
def render_safely(case: Dict[str, Any], template_path: Path) -> Dict[str, Any]:
"""Render one fixture and return bounded output metadata."""
try:
rendered = render_case(case, template_path)
except jinja2.TemplateError as exc:
return {"characters": None, "error": str(exc), "rendered": None}
return {"characters": len(rendered), "error": None, "rendered": rendered}
def inspect_case(
name: str,
case: Dict[str, Any],
expected: Dict[str, Any],
) -> Dict[str, Any]:
"""Compare one fixture and verify its published marker expectations."""
stock = render_safely(case, STOCK_TEMPLATE)
derived = render_safely(case, DERIVATIVE_TEMPLATE)
rendered: Optional[str] = derived["rendered"]
required_present: List[str] = expected.get("contains", [])
required_absent: List[str] = expected.get("absent", [])
counts: Dict[str, int] = expected.get("counts", {})
present_passed = 0 if rendered is None else sum(
marker in rendered for marker in required_present
)
absent_passed = 0 if rendered is None else sum(
marker not in rendered for marker in required_absent
)
count_passed = 0 if rendered is None else sum(
rendered.count(marker) == count for marker, count in counts.items()
)
stock_characters = stock["characters"]
derived_characters = derived["characters"]
character_delta = None
exact_match = False
if stock_characters is not None and derived_characters is not None:
character_delta = derived_characters - stock_characters
exact_match = stock["rendered"] == derived["rendered"]
return {
"case": name,
"description": case["description"],
"expected_status": expected["status"],
"expected_error": expected.get("error"),
"matches_stock_required": bool(expected.get("matches_stock")),
"stock_characters": stock_characters,
"derived_characters": derived_characters,
"character_delta": character_delta,
"exact_match": exact_match,
"stock_error": stock["error"],
"derived_error": derived["error"],
"present_checks": {"passed": present_passed, "total": len(required_present)},
"absent_checks": {"passed": absent_passed, "total": len(required_absent)},
"count_checks": {"passed": count_passed, "total": len(counts)},
"image_placeholders": 0 if rendered is None else rendered.count("<|image_pad|>"),
"video_placeholders": 0 if rendered is None else rendered.count("<|video_pad|>"),
}
def inspect_fixtures(selected: Optional[str] = None) -> List[Dict[str, Any]]:
"""Inspect all fixtures, or one named fixture."""
cases = json.loads((ROOT / "fixtures.json").read_text(encoding="utf-8"))
expected = json.loads(EXPECTED_RESULTS.read_text(encoding="utf-8"))
if selected is not None:
if selected not in cases:
choices = ", ".join(sorted(cases))
raise ValueError(f"unknown fixture: {selected}; choose from: {choices}")
names = [selected]
else:
names = list(cases)
return [inspect_case(name, cases[name], expected[name]) for name in names]
def check_summary(summary: Dict[str, Any]) -> str:
"""Return a compact result label for one inspection summary."""
if summary["expected_status"] == "error":
expected_error = summary["expected_error"]
errors_match = (
expected_error is not None
and summary["stock_error"] is not None
and summary["derived_error"] is not None
and expected_error in summary["stock_error"]
and expected_error in summary["derived_error"]
)
return "expected error" if errors_match else "check failed"
if summary["stock_error"] is not None or summary["derived_error"] is not None:
return "check failed"
checks = (
summary["present_checks"],
summary["absent_checks"],
summary["count_checks"],
)
checks_pass = all(item["passed"] == item["total"] for item in checks)
stock_contract_passes = (
not summary["matches_stock_required"] or summary["exact_match"]
)
return "pass" if checks_pass and stock_contract_passes else "check failed"
def print_table(summaries: List[Dict[str, Any]]) -> None:
"""Print a bounded human-readable comparison table."""
print(f"{'fixture':30} {'stock':>9} {'derived':>9} {'delta':>9} {'result':>14}")
print(f"{'-' * 30} {'-' * 9} {'-' * 9} {'-' * 9} {'-' * 14}")
for summary in summaries:
stock = "error" if summary["stock_characters"] is None else str(summary["stock_characters"])
derived = "error" if summary["derived_characters"] is None else str(summary["derived_characters"])
delta = "n/a" if summary["character_delta"] is None else str(summary["character_delta"])
print(
f"{summary['case']:30} {stock:>9} {derived:>9} "
f"{delta:>9} {check_summary(summary):>14}"
)
print("\nCharacter deltas describe these synthetic renders. They are not token or cost measurements.")
def main() -> int:
"""Run the fixture inspector command."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("case", nargs="?", help="Inspect one fixture instead of all fixtures")
parser.add_argument("--json", action="store_true", help="Write the bounded summary as JSON")
args = parser.parse_args()
try:
summaries = inspect_fixtures(args.case)
except ValueError as exc:
parser.error(str(exc))
if args.json:
print(json.dumps(summaries, indent=2, sort_keys=True))
else:
print_table(summaries)
return 0 if all(check_summary(summary) in {"pass", "expected error"} for summary in summaries) else 1
if __name__ == "__main__":
raise SystemExit(main())
test_media_history_template.py python View source
#!/usr/bin/env python
from __future__ import annotations
import hashlib
import json
import unittest
from pathlib import Path
import jinja2
from render_fixture import ROOT, render_case
UPSTREAM_SHA256 = "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715"
STOCK_SHA256 = "d2cb9a5730cdd5f44bce3ada2dc1b0e00c6c59788b6d1c4d8d49c40a274dffb0"
DERIVATIVE_SHA256 = "162671aeaf5e2c39966816dae53e5e6f8ac0dfb97d53f34094afe74e44b2fae6"
def sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
class MediaHistoryTemplateTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.cases = json.loads((ROOT / "fixtures.json").read_text(encoding="utf-8"))
cls.expected = json.loads((ROOT / "expected-results.json").read_text(encoding="utf-8"))
def test_provenance_checksums(self) -> None:
stock = (ROOT / "Qwen-3_5-stock-template.jinja").read_bytes()
derivative = (ROOT / "Qwen-3_5-media-history-template.jinja").read_bytes()
self.assertEqual(sha256(stock), STOCK_SHA256)
self.assertEqual(sha256(stock.removesuffix(b"\n")), UPSTREAM_SHA256)
self.assertEqual(sha256(derivative), DERIVATIVE_SHA256)
def test_sanitized_fixture_contracts(self) -> None:
for name, case in self.cases.items():
with self.subTest(case=name):
expected = self.expected[name]
if expected["status"] == "error":
with self.assertRaisesRegex(jinja2.TemplateError, expected["error"]):
render_case(case)
continue
rendered = render_case(case)
for marker in expected.get("contains", []):
self.assertIn(marker, rendered)
for marker in expected.get("absent", []):
self.assertNotIn(marker, rendered)
for marker, count in expected.get("counts", {}).items():
self.assertEqual(rendered.count(marker), count)
if expected.get("matches_stock"):
stock_rendered = render_case(
case,
ROOT / "Qwen-3_5-stock-template.jinja",
)
self.assertEqual(rendered, stock_rendered)
def test_fixture_and_expectation_names_match(self) -> None:
self.assertEqual(set(self.cases), set(self.expected))
if __name__ == "__main__":
unittest.main()
test_inspect_fixture.py python View source
#!/usr/bin/env python
from __future__ import annotations
import unittest
from inspect_fixture import check_summary, inspect_fixtures
from render_fixture import render_case
class FixtureInspectorTests(unittest.TestCase):
def test_all_packaged_fixtures_satisfy_their_contracts(self) -> None:
summaries = inspect_fixtures()
self.assertEqual(len(summaries), 7)
self.assertTrue(
all(check_summary(summary) in {"pass", "expected error"} for summary in summaries)
)
def test_ordinary_history_matches_stock_exactly(self) -> None:
summary = inspect_fixtures("ordinary_text_tool_history")[0]
self.assertTrue(summary["exact_match"])
self.assertEqual(summary["character_delta"], 0)
def test_required_stock_match_cannot_report_pass_when_render_differs(self) -> None:
summary = inspect_fixtures("ordinary_text_tool_history")[0]
summary["exact_match"] = False
self.assertEqual(check_summary(summary), "check failed")
def test_textual_image_history_is_removed(self) -> None:
summary = inspect_fixtures("textual_image_exchanges")[0]
self.assertLess(summary["character_delta"], 0)
self.assertEqual(summary["absent_checks"]["passed"], summary["absent_checks"]["total"])
def test_native_media_keeps_qwen_placeholders(self) -> None:
summary = inspect_fixtures("native_structured_media")[0]
self.assertEqual(summary["image_placeholders"], 1)
self.assertEqual(summary["video_placeholders"], 1)
def test_malformed_ordering_reports_the_template_error(self) -> None:
summary = inspect_fixtures("malformed_ordering")[0]
self.assertEqual(check_summary(summary), "expected error")
self.assertIn("System message must be at the beginning.", summary["stock_error"])
self.assertIn("System message must be at the beginning.", summary["derived_error"])
def test_declared_error_requires_matching_stock_and_derived_failures(self) -> None:
summary = inspect_fixtures("malformed_ordering")[0]
summary["stock_error"] = None
summary["derived_error"] = "Unrelated template failure"
self.assertEqual(check_summary(summary), "check failed")
def test_textual_media_threshold_is_strictly_greater_than_4096(self) -> None:
prefix = '{"images":["iVBORw0KGgoTHRESHOLD_CANARY'
suffix = '"]}'
def threshold_case(length: int) -> dict:
content = prefix + ("A" * (length - len(prefix) - len(suffix))) + suffix
return {
"payload": {
"messages": [
{"role": "user", "content": "Create an invented image."},
{
"role": "assistant",
"content": "threshold image call",
"tool_calls": [
{
"type": "function",
"function": {
"name": "terminal",
"arguments": {
"command": "generate invented image"
},
},
}
],
},
{"role": "tool", "content": content},
{"role": "user", "content": "Continue."},
],
"tools": [],
"add_generation_prompt": True,
}
}
self.assertIn("THRESHOLD_CANARY", render_case(threshold_case(4096)))
self.assertNotIn("THRESHOLD_CANARY", render_case(threshold_case(4097)))
if __name__ == "__main__":
unittest.main()
NOTICE.md markdown View source
# Attribution and Modification Notice
`Qwen-3_5-media-history-template.jinja` is a modified form of Qwen's `chat_template.jinja` from `Qwen/Qwen3.5-27B`.
Upstream source:
- Repository: `Qwen/Qwen3.5-27B`
- Immutable revision: `feea018b31f89dc0950e61da42577a7a4ab09169`
- Source file: `https://huggingface.co/Qwen/Qwen3.5-27B/blob/feea018b31f89dc0950e61da42577a7a4ab09169/chat_template.jinja`
- Raw source: `https://huggingface.co/Qwen/Qwen3.5-27B/resolve/feea018b31f89dc0950e61da42577a7a4ab09169/chat_template.jinja`
- Upstream license: Apache License 2.0
The derivative adds a media-history policy that:
- removes every textual image tool result and its associated assistant tool exchange;
- removes assistant tool calls that explicitly carry or decode historical media bytes;
- retains only the latest explicit textual audio result and latest explicit textual video result;
- preserves native structured image and video message parts and their Qwen vision placeholders;
- requires media-result structure, known data markers, and a content length greater than 4,096 characters for textual tool-result detection.
The derivative does not validate base64 data. It does not claim compatibility with arbitrary ChatML, Llama, Mistral, Gemma, OpenAI, or other model-family templates.
## SHA-256 Checksums
| Artifact | SHA-256 |
| --- | --- |
| Immutable upstream `chat_template.jinja` bytes | `a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715` |
| Packaged stock fallback with one trailing newline | `d2cb9a5730cdd5f44bce3ada2dc1b0e00c6c59788b6d1c4d8d49c40a274dffb0` |
| Published attributed media-history derivative | `162671aeaf5e2c39966816dae53e5e6f8ac0dfb97d53f34094afe74e44b2fae6` |
The packaged stock fallback is byte-equivalent to the immutable upstream file after removing its single trailing newline. The test suite verifies both forms and the derivative checksum without requiring network access.
The template files and modifications in this companion are distributed under Apache License 2.0. They are not presented as solely covered by the surrounding repository's MIT license. See `LICENSE-APACHE-2.0.txt`.
LICENSE-APACHE-2.0.txt text View source
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Create the working directory, download the archive, and inspect the inventory:
mkdir -p qwen-media-history-lab
cd qwen-media-history-lab
curl -fsSLo lab.zip https://unixwzrd.ai/assets/code/agent-optimization/post-07a/hands-on-07a-qwen-media-history-test-bench.zip
unzip lab.zip
rm lab.zip
ls -1
You should see both templates, fixtures, expectations, renderer, inspector, tests, the pinned Jinja requirement, and the provenance and license documents. Keep NOTICE.md and LICENSE-APACHE-2.0.txt with the templates. The derivative is Apache-2.0 material with an attribution and modification boundary.
Step 2: Create a Small Python Environment
I prefer to run a lab like this in a disposable virtual environment. It keeps the one Python dependency visible and makes cleanup straightforward:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python -c 'import jinja2; print(jinja2.__version__)'
The reviewed package pins Jinja 3.1.6 because that is the version exercised by the companion and the matching application-owned runtime. The template also expects the Hugging Face-style raise_exception helper. The standalone renderer supplies that helper locally, so you do not need a model server to reach the validation paths.
If Jinja is already available in an environment you trust, you can use that environment instead. I would still check the version and run the tests before drawing conclusions from a render.
Step 3: Run the Contract Before Inspecting Pretty Output
Start with the tests:
python -m unittest -v test_media_history_template.py test_inspect_fixture.py
The expected result is eleven passing tests. Three cover template provenance, fixture behavior, and fixture-name agreement. Eight cover the reader-facing inspector, including ordinary-history equivalence, negative aggregate-result cases, textual-image removal, native placeholder retention, the intended malformed-ordering error, and the strict 4,096-character boundary.
A rendered prompt can look perfectly plausible while quietly dropping a role boundary, tool result, or assistant-generation marker. These tests do not prove that the model will produce a good answer, but they do prove agreement with the published fixture contract.
Step 4: Compare All Seven Fixtures Without Dumping the Payloads
Run the bounded inspector:
python inspect_fixture.py
You should see this table:
fixture stock derived delta result
------------------------------ --------- --------- --------- --------------
native_structured_media 172 172 0 pass
textual_image_exchanges 10975 156 -10819 pass
latest_audio_video 21263 10723 -10540 pass
truncated_image_history 5475 137 -5338 pass
incidental_signature_collision 5419 5419 0 pass
malformed_ordering error error n/a expected error
ordinary_text_tool_history 409 409 0 pass
Those numbers are deterministic for the packaged fixtures and templates. The negative delta is simply the derived character count minus the stock character count. It tells me how much invented rendered text the policy removed in this particular fixture, and nothing by itself about tokenizer behavior, KV-cache allocation, latency, cost, or answer quality.
The zero-delta cases matter too. Native structured media keeps its Qwen placeholders. The incidental-signature fixture stays intact because a marker without image-result structure is not enough to classify an image exchange.
Step 5: Inspect One Decision in Detail
Ask the inspector for the textual-image case as JSON:
python inspect_fixture.py textual_image_exchanges --json
The result reports 10,975 stock characters, 156 derived characters, and seven passing absence checks without printing the canary expansions. I can see whether the contract held without turning the test log into another synthetic media archive.
Now use the lower-level renderer to prove that the markers really are present under the stock template and absent under the derivative:
python render_fixture.py textual_image_exchanges \
--template ./Qwen-3_5-stock-template.jinja \
| grep -oE 'PNG_(OLD|NEW)_CANARY' \
| sort -u
python render_fixture.py textual_image_exchanges \
--template ./Qwen-3_5-media-history-template.jinja \
| grep -oE 'PNG_(OLD|NEW)_CANARY' \
| sort -u
The stock command prints both canary names. The derivative command prints nothing. Because grep returns a nonzero status when it finds no match, that second pipeline may also leave a nonzero shell status. In this one inspection, that absence is the expected result.
This is also where complete-exchange removal becomes easier to understand. The derived render does not merely replace the long tool result with a note. It removes the associated assistant tool call, the textual result, and the assistant-side decode exchange covered by the fixture. Leaving the call while deleting only the result would create a malformed history that looks smaller but no longer tells a coherent story.
Step 6: Check Native Media and Ordinary History
Inspect the two cases that must remain useful:
python inspect_fixture.py native_structured_media --json
python inspect_fixture.py ordinary_text_tool_history --json
The native case reports one <|image_pad|> and one <|video_pad|>. The long data URL itself is not placed into the rendered prompt, which is normal for this Qwen contract. The model engine receives and processes the structured media outside the textual template output.
The ordinary-history case reports exact_match: true and a character delta of zero. That comparison is the control. An optimization that removes obvious media payloads but changes routine tool calling, thinking layout, role boundaries, or assistant generation behavior is not a safe drop-in improvement.
Step 7: Touch the 4,096-Character Boundary
The template only considers textual media results longer than 4,096 characters. That threshold is a routing gate. It is not base64 validation and it is not a general definition of media.
Run this temporary experiment from the lab directory:
python - <<'PY'
from render_fixture import render_case
prefix = '{"images":["iVBORw0KGgoTHRESHOLD_CANARY'
suffix = '"]}'
for length in (4096, 4097):
content = prefix + ('A' * (length - len(prefix) - len(suffix))) + suffix
case = {
'payload': {
'messages': [
{'role': 'user', 'content': 'Create an invented image.'},
{
'role': 'assistant',
'content': 'threshold image call',
'tool_calls': [
{
'type': 'function',
'function': {
'name': 'terminal',
'arguments': {'command': 'generate invented image'},
},
}
],
},
{'role': 'tool', 'content': content},
{'role': 'user', 'content': 'Continue.'},
],
'tools': [],
'add_generation_prompt': True,
}
}
rendered = render_case(case)
state = 'retained' if 'THRESHOLD_CANARY' in rendered else 'removed'
print(f'{length}: {state}')
PY
The expected output is:
4096: retained
4097: removed
I like this experiment because it leaves very little room for vague language. The condition is greater than 4,096, not greater than or equal to it, and the other image-result structure and marker checks still apply. One character changes the routing decision. That is exactly the sort of edge I would rather capture in a canary than explain in a comment and hope everybody interprets the same way.
Step 8: See Where the Passive Model Proxy Fits
The core lab calls Jinja directly because a proxy or live model would only distract from the policy being tested. In an installed LLM-Ops-Kit route, the passive proxy can render a diagnostic view while forwarding the original OpenAI-compatible request upstream unchanged. The model runtime then uses the selected template to construct the actual prompt. Both rendering paths must point at the same template file if I expect those two views to agree.
If model-proxy is installed, create a sanitized expanded request from the packaged fixture:
python - <<'PY' > ./sanitized-media-request.json
import json
from render_fixture import expand
with open('fixtures.json', encoding='utf-8') as handle:
cases = json.load(handle)
print(json.dumps(expand(cases['textual_image_exchanges']['payload'])))
PY
If jq is available, you can inspect the request structure without printing message content or synthetic media strings:
jq '{message_count: (.messages | length), roles: [.messages[].role], assistant_tool_calls: ([.messages[] | select(.role == "assistant") | .tool_calls[]?] | length), tool_results: ([.messages[] | select(.role == "tool")] | length), structured_image_parts: ([.messages[].content? | arrays | .[] | select((.type? == "image") or has("image_url") or has("image"))] | length), structured_video_parts: ([.messages[].content? | arrays | .[] | select((.type? == "video") or has("video"))] | length)}' sanitized-media-request.json
For this fixture, the structural inventory reports eight messages, three assistant tool calls, three tool results, and no native structured image or video parts. The roles array shows the turn order without exposing message content. This check is optional because the core lab does not require jq.
Render it first with the stock template and then with the derivative, keeping the two diagnostic files separate:
model-proxy render \
--input ./sanitized-media-request.json \
--chat-template ./Qwen-3_5-stock-template.jinja \
--log ./stock-render-metrics.ndjson \
--raw-request-log ./stock-request.log \
--rendered-prompt-log ./stock-rendered.log
model-proxy render \
--input ./sanitized-media-request.json \
--chat-template ./Qwen-3_5-media-history-template.jinja \
--log ./derived-render-metrics.ndjson \
--raw-request-log ./derived-request.log \
--rendered-prompt-log ./derived-rendered.log
Now compare only the invented canaries:
grep -oE 'PNG_(OLD|NEW)_CANARY' stock-rendered.log | sort -u
grep -oE 'PNG_(OLD|NEW)_CANARY' derived-rendered.log | sort -u
python - <<'PY'
import json
def framed_payload(path):
text = open(path, encoding='utf-8').read()
start = text.index('\n') + 1
end = text.rindex('\n=== RAW_REQUEST END')
return json.loads(text[start:end])
assert framed_payload('stock-request.log') == framed_payload('derived-request.log')
print('raw request payloads match')
PY
The stock diagnostic contains the textual image canaries and the derived diagnostic does not. The raw request payloads match after the timestamped frame headers and footers are excluded. Those timestamps are expected to differ because these are two separate render-only runs.
Render-only mode does not start the proxy or send bytes across a network, so this exercise does not independently prove passive forwarding. It shows that selecting a different diagnostic template changes the rendered view without changing the input payload recorded by the tool. The proxy’s byte-preservation contract is covered by its production regressions and by Hands-On 6A. In a live route, the filtering still happens when the selected template renders message history, not while the proxy forwards the request.
This exercise writes only invented data, but its rendered and raw logs are still content-bearing artifacts. I would not repeat it with a private capture merely because the command happens to be convenient.
Step 9: Clean Up the Lab Evidence
The standalone tests use temporary files and clean those up themselves. The optional proxy exercise creates the request and log files named above. Remove those invented artifacts when you are finished, then leave the virtual environment:
rm -f ./sanitized-media-request.json \
./stock-render-metrics.ndjson \
./stock-request.log \
./stock-rendered.log \
./derived-render-metrics.ndjson \
./derived-request.log \
./derived-rendered.log
deactivate
If you skipped the optional proxy exercise, those files will not exist and there is nothing to remove. Keep the package if you want a known comparison baseline, and keep the stock fallback beside the derivative if you move on to a private runtime canary. A rollback file stored somewhere else is not much of a rollback plan.
What This Lab Does Not Prove
The lab proves behavior against seven invented fixture shapes. It does not prove better answers, universal Qwen compatibility, portability to another ChatML-style model, tokenizer-specific savings, KV-cache allocation, generation speed, or cost.
It also does not make the teaching package a production rollout procedure. A real change needs the exact model revision, exact runtime, bundled stock template, sanitized render canaries, ordinary text and tool-call checks, native vision checks, bounded generation acceptance, restart planning, health verification, and a tested rollback. If any role boundary, tool call, thinking behavior, native vision path, or generation terminator changes unexpectedly, restore the stock template and investigate.
Current State
The separate Hands-On 7A package contains the technically approved stock and derived Qwen templates, seven sanitized fixtures, expected results, provenance and license material, the original renderer and tests, and a bounded inspector with eight additional tests. All eleven companion tests pass under Python with Jinja 3.1.6. The inspector produces the documented character counts and marker checks, and both the permanent threshold regression and temporary reader exercise produce the documented 4,096 and 4,097 results. The original nine-file Part 7 package remains byte-stable.
The tutorial, inspector, tests, requirements file, package, and two diagrams passed technical review before being staged here. The package remains a teaching artifact rather than a production template rollout.
Next Work
The lab uses its own twelve-file archive, leaving the approved Part 7 download unchanged. A later revision may add a bounded model-generation canary after the render-only contract remains stable across the exact model and runtime under test.
The next production-facing step is still a bounded model and runtime canary, not a larger synthetic benchmark. The lesson I want readers to take away is simpler: give the same invented history to the stock and derived paths, verify what changed, verify what did not, and keep the passive observer out of the mutation business.
Join the Discussion
Comments for this post live in GitHub Discussions. That keeps moderation in one place and gives the conversation a stable home.