🔍 Read the full analysis: Could A Single Weight Edit Help Qwen Escape JSON Tabs? on ThorstenMeyerAI.com
Get audio and creator gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
An experiment run by OpenAI’s GPT-6 Astra as a Codex agent changed one matrix in Qwen3-4B-Instruct-2507 to target errors escaping tabs and other characters in JSON. On a frozen 120-case test, exact answers rose from 49 to 70, with 21 cases fixed and none newly broken. The result is limited to synthetic cases, and another task showed no improvement.
An experiment led by OpenAI’s GPT-6 Astra changed one matrix in the open Qwen3-4B-Instruct-2507 model and raised its exact score on a frozen test of text escaping in JSON from 49 of 120 cases to 70. The narrow result suggests a direct weight edit can help with a specific failure pattern, but it does not establish a general improvement: the cases were synthetic, 50 final answers remained wrong, and the work has not received an independent review.
The experiment targeted a specific error: copying short texts containing tabs and other special characters into JSON without altering them. A real tab must be represented as the escape sequence \t inside JSON and must decode back to the original tab. The test required exact recovery of the source text, not merely syntactically valid JSON. The original Qwen model answered 16 of 32 screening cases exactly. A more explicit prompt also scored 16 of 32, according to the experiment report.
The edit was a calculated rank-one update to the output projection of the model’s final multilayer perceptron (MLP) layer. The report says no gradient training was used and the model’s other 397 stored tensors remained bit-identical to the originals. On the final test, which had been frozen before the result was measured, the edited model answered 70 cases exactly, compared with 49 for the unedited model: a gain of 21 cases, or 17.5 percentage points. The report records no cases that changed from correct to incorrect.
The improvement did not carry over to every task. On a separate test of extracting a table row, both versions scored 0 of 30. The final JSON-escaping test also still had 50 incorrect responses after the edit. The experiment’s authoring account says GPT-6 Astra planned and carried out the work as a Codex agent, including inspecting the weights, creating and freezing tests, calculating the update and reporting the results.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Gain Limited to JSON Escaping
The result matters as a small test of whether a model’s behavior can be changed by editing its stored weights directly, rather than by retraining the model or relying only on a different prompt. The reported improvement on a previously frozen test makes the finding more informative than a result measured only on examples used to devise the edit. It remains a single experiment on one model and one narrow task, however; it does not show that one-matrix edits can reliably improve other capabilities.
JSON formatting errors can affect software that passes structured data between services, tools and models. In this test, a response counted only when parsing recovered the original text exactly, so escaping a tab incorrectly was treated as a failure even if the output looked close. The score gain is relevant to that precise reliability problem. The remaining errors and the zero score on table-row extraction also show why the result should not be read as a broad upgrade to Qwen’s performance.
Products Worth Considering
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, Black
LG UltraGear 32GS60QC-B 32-inch Curved QHD Gaming Monitor (180Hz, 1ms, 1000R, AMD FreeSync, HDR10)
As an affiliate, we earn on qualifying purchases.
The Earlier Qwen Test
This was the third experiment in a week involving AI systems examining or editing model weights, according to the report published by ThorstenMeyerAI.com on November 5, 2026. In a previous attempt, GPT-6 Astra tested the same Qwen model on extracting fields from synthetic business messages. Both the original and edited versions scored 97 of 200 on that attempt’s frozen final test. The report says many invalid outputs in that experiment contained raw tab characters in JSON strings, providing the lead for the new, narrower test.
The week also included a separate experiment in which Claude edited a local model called Ornith. The report says GPT-6 Astra later reviewed that work and identified faults in its implementation and claims. Claude was not involved in the Qwen experiment described here; the source says Claude was used to research and draft the article about it. The previous review is relevant as a reminder that the current experiment has not yet had the same second-system check.
“Codex’s own experiment”
— ThorstenMeyerAI.com experiment report
Independent Checks Still Pending
The reported scores come from the experiment’s own records, and no independent review of this third experiment had been completed at publication, according to the source. The report describes the cases as synthetic and does not establish how the result would transfer to real software inputs, other prompts, other JSON tasks or other models. It also does not provide, in the supplied material, enough detail to assess how representative the test cases are of everyday JSON use.
The reported 21-case gain is tied to the test and comparison described by the experiment. It does not show that the model is generally more accurate, nor that the edit has no side effects outside the measured cases. The source says the other 397 tensors were unchanged, but the supplied material does not report a broad evaluation of the edited model’s performance or behavior. Whether the improvement persists across independent test sets remains unknown.
Replication and Broader Testing
The immediate next step would be an independent check of the experiment’s code, saved weight change and test results, similar to the review performed on the earlier weight-editing attempt. Further tests would be needed to see whether the gain holds on new, independently prepared examples and whether it affects other tasks. The source does not announce a schedule for that review or a follow-up experiment.
Until those checks are available, the result is best treated as a narrow, reported improvement on a synthetic JSON-escaping test, not as a general performance claim about Qwen. The table-row result and the 50 remaining errors are part of the current record, alongside the increase from 49 to 70 exact answers.
Key Questions
What did the weight edit change?
It applied a calculated rank-one update to the output projection of Qwen3-4B-Instruct-2507’s final MLP layer. The report says no gradient training was used and the other 397 stored tensors were unchanged.
How much did the JSON test score improve?
On the frozen final test, exact answers rose from 49 of 120 to 70 of 120, a gain of 21 cases or 17.5 percentage points. The report says no previously correct cases became incorrect.
Did the edit improve other tasks?
Not in the separate table-row extraction test reported here: both versions scored 0 of 30. The source does not report broad testing across other capabilities.
Has another reviewer verified the result?
No independent review had been completed at the time of publication, according to the source. The test cases were synthetic, and the result has not yet been shown to generalize to other data or settings.
Source: ThorstenMeyerAI.com
![New Developer Video For ‘Lords Of The Fallen II’ Relishes In Combat Overhaul [Watch]](https://strongmocha.com/wp-content/uploads/2026/11/new-developer-video-for-lords-of-the-fallen-ii-relishes-in-combat-overhaul-watch-featured-260x140.jpg "New Developer Video For ‘Lords Of The Fallen II’ Relishes In Combat Overhaul [Watch] 14")