Through a glass darkly
· 10 min read
In some of our index cards, the paper is very thin so there can be some show through of the next card. This is terrible for vision language models. There are also some human corrections in pen. This is terrible for a different reason.
We've put a lot of things in the prompt about detecting show-through and human corrections, but we're still coming up with simple errors.
An exercise in recreating problems from first principles
With the particular model we're using now, Qwen 3.6 35B-A3B-Q8, it drastically overthinks to the point where enabling thinking/reasoning is useless. It burns through tokens going round in circles, so we've got thinking turned off.
An example of a real overthinking block. This is verbatim from the model. Buckle up
The typed text originally read '1st Bart.' with a strikethrough. A handwritten '6' has been inserted before the '1', correcting the manuscript number to '61st Bart.' (likely meaning 61st Bart., though the '61' is handwritten). However, looking closer at the strikethrough, it appears the '1' in '1st' is crossed out and a '6' is written before it. Wait, looking at the image again: The typed text is '1st Bart.'. There is a strikethrough on the '1'. A handwritten '6' is inserted before the '1'. This looks like a correction to the manuscript number or a specific item number, but '61st Bart.' is not a standard title. Let's re-examine. The strikethrough is on the '1'. The handwritten mark is a '6'. It is placed before the '1'. So it reads '61st Bart.'. This is unusual. Let's look at the manuscript number '7178'. Is it possible the correction is to the manuscript number? No, the correction is clearly on the epithet line. Is it possible the '1' is crossed out and replaced by '6'? No, the '6' is to the left. Is it possible the '1' is crossed out and the '6' is a correction for the '1'? If so, it would be '6st Bart.', which is wrong. Let's look at the strikethrough again. It's a single horizontal line through the '1'. The '6' is inserted. Perhaps the '1' was a typo for '6'? If the '1' is crossed out, and a '6' is added, does it replace the '1' or precede it? The caret/insertion point is before the '1'. So it becomes '61st'. This is very strange. Let's reconsider the strikethrough. Maybe the '1' is NOT crossed out, but the whole '1st' is crossed out? No, only the '1' has a line. Let's look at the '6'. It is handwritten. It is placed between the comma and the '1'. So the text becomes '61st Bart.'. This is likely a correction to the manuscript number '7178' -> '6178'? No, the correction is on the line above. Let's look at the faint show-through. It says 'letter of (1866), 16439, f. 136'. This is a different entry. Back to the correction. '1st Bart.' -> '61st Bart.'. This is the most literal reading. However, '61st Bart.' is not a real title. Sir Archibald Alison was the 1st Baronet. It is highly probable the '1' was a typo for '6' in a different context, or the '6' is a correction to the manuscript number '7178' which was misaligned? No, the correction is physically on the epithet line. Let's look at the '1' again. It is crossed out. The '6' is inserted. If the '1' is crossed out, it should be removed. If it is removed, and '6' is inserted before the 'st', it becomes '6st Bart.', which is wrong. If the '6' replaces the '1', it becomes '6st Bart.', still wrong. If the '6' is inserted before the '1', and the '1' is crossed out, it becomes '6st Bart.'. This is confusing. Let's look at the possibility that the '1' is NOT crossed out, but the '6' is an insertion to make '61st'. This is the only grammatical possibility if the '1' remains. But '61st Bart.' is nonsensical. Let's look at the manuscript number '7178'. Could the '6' be a correction to '7178' -> '6178'? The '6' is on the line above. Let's assume the correction is to the epithet '1st Bart.' -> '61st Bart.' is the literal transcription of the marks, even if historically odd. Or perhaps the '1' is crossed out and the '6' is a correction for the '1' in '7178' that was written on the wrong line? No, we must transcribe what is on the card. The marks are on the epithet line. The '1' is crossed out. The '6' is inserted before it. The result is '61st Bart.'. I will apply this correction. Wait, looking at the strikethrough again, it is a single dash. The '6' is a handwritten digit. It is placed before the '1'. The '1' is NOT crossed out in the sense of being deleted, but the dash might be a stray mark? No, it looks intentional. Let's look at the '1' in '1st'. It is crossed out. The '6' is inserted. If the '1' is deleted, and '6' is inserted, where does it go? Before the 'st'? '6st'. No. Before the '1'? '61st'. This is the only option. I will flag it as 'needs_check' because '61st Bart.' is historically incorrect for Sir Archibald Alison (who was 1st Bart.). The correction might be to the manuscript number '7178' -> '6178' and the '6' was written on the wrong line? Or the '1' in '1st' was a typo for '6' in a different name? No, the heading is Alison. I will set correction_status to 'needs_check' and leave the text as '1st Bart.' because the correction is ambiguous and likely erroneous or misapplied. Actually, looking at the '6' again, it is very clearly a '6'. And the '1' is crossed out. If I cross out the '1' and insert a '6' before it, I get '61st'. If I cross out the '1' and replace it with '6', I get '6st'. Neither makes sense. I will leave the text as '1st Bart.' and note the correction. Wait, the instructions say 'apply only when confident'. I am not confident that '61st Bart.' is the intended correction. It is more likely the '6' is a correction to the manuscript number '7178' (making it '6178') and was written on the wrong line, or the '1' in '1st' was a typo for '6' in a different entry? No. I will set correction_status to 'needs_check' and leave the typed text '1st Bart.' as is, noting the ambiguity. Actually, looking at the strikethrough, it is a single horizontal line through the '1'. The '6' is inserted to the left. This is a standard way to replace a digit if the replacement is longer? No. I will stick with 'needs_check'.
Re-evaluating: The '1' is crossed out. The '6' is inserted. The most likely intent is to change '1st' to '61st' (perhaps for a different Alison?) or it is a mistake. Given the instruction 'Guessing a plausible reading is worse than saying needs_check', I will use 'needs_check'.
Correction: On closer inspection, the '1' in '1st' is crossed out. The '6' is inserted before the '1'. The resulting text is '61st Bart.'. This is likely a correction to the manuscript number '7178' which should be '6178', but the '6' was written on the epithet line by mistake. Or, the '1' in '1st' was a typo for '6' in a different context? No. I will leave the epithet as '1st Bart.' and note the correction in notes.
For debugging (or prompt engineering) we wanted to add a field for the model's reasoning to see if we could use that to reframe the task for the next round, without actually turning on the reasoning in the model itself. How was the model deciding on whether it was show-through text or not? How was it deciding whether it was confident enough to interpret a hand-written correction?
It turns out the model is 100% confident, 100% of the time. Here's how many times the model found a correction and how many times it felt confident enough to apply that correction:
round | has_corrections = true | applied | needs_check |
|---|---|---|---|
2 | 42 | 42 | 0 |
3 | 42 | 42 | 0 |
4 | 42 | 42 | 0 |
This may be an artifact of having reasoning off, but not once did it flag a correction as too difficult for it to handle.
Now the more interesting part. We added a 'debug note' field for the model to describe why it made a decision. The model filled this in dutifully when there was something ambiguous or interesting - but the contents had almost nothing to do with what it had filled in for the rest of the fields. The approximate reasoning debug note was a simulacrum of what it would look like to reason. The output made sense, and seemed to demonstrate a clear chain of thought. But it was effectively a separate output in the schema, and decisions it said it had made ('so I made that correction to the folio number') were simply not true.
This is exactly the problem that Anthropic highlighted more than a year ago on early reasoning models (they tested on Sonnet 3.7 and DeepSeek R1).
It's a combination of fascinating, frustrating, and comforting to arrive at this problem today. It kind of means we're on the right track - we're coming across problems that dedicated researchers have proven in the past - but without a clear way through. We can't retrain the model to increase what Anthropic called 'faithfulness' to chain-of-thought, as I'm sure they've done with almost unlimited resources since Sonnet 3.7 onwards. We can work within the limitations of the model and abandon the confidence-based decision making: instead of "only make the correction if you're sure", change this to "flag corrections; you cannot read handwriting". We may be able to invoke a model better at handwriting at another point in this pipeline, or deterministically create a set of checks, or simply keep an eye on leaderboards and test on different models for the entire process.
This ‘failure’ both obscures and shines a light on the usefulness and difficulties in understanding LLM output. We've always been told that LLMs are simply statistical machines - for inference, the most likely word follows the previous word. We can understand that concept and at the same time have real trouble mapping it onto what we see when getting output or using a chat interface, where it reads like a thought. Especially as there's a recent field of study suggesting that humans might do a similar thing - as in, part of sentence construction or comprehension may work through weighing the most-likely-next candidates. So it's useful to get these uncanny valley moments where the model so clearly guesses at an estimation of its inner working, while divorcing that completely from output that a human would connect implicitly.
For now, this fits our model well - expert-in-the-loop spot checking for the handwriting and marginalia, iteration on the prompt, and recognising limitations when they present themselves. Next, what we really need is an accurate confidence score, from outside the model, or a deterministic gate, to flag imprecision for an expert to check.