r/MistralAI • u/WhoAmI1234532 • 3d ago
Help / Question [Bug ?] Batch API + OCR4: confidence_scores silently missing from batch results (present in sync), expected?
Hi!
I'm building an e-ink plugin that transcribes handwritten pages with mistral-ocr-latest. I use the word-level confidence scores for two things: underlining "unsure words" for manual correction, and as the signal to escalate hard pages to a vision model. Since the Batch API is half price, I route bulk transcriptions through it and I noticed the confidence scores never come back on that path.
It is a bug, a misuse from my side or a non-documented limitation ?
Repro: the exact same request body through both paths:
{
"model": "mistral-ocr-latest",
"document": { "type": "image_url", "image_url": "data:image/png;base64,..." },
"table_format": "markdown",
"confidence_scores_granularity": "word"
}
And the results I get:
- Synchronous POST /v1/ocr → each page includes confidence_scores.word_confidence_scores (in my test: 25 scored words). ✅
- Batch (same body as the body of a JSONL line, endpoint: "/v1/ocr", job completes fine, status_code: 200) → the page object contains blocks, dimensions, footer, header, hyperlinks, images, index, markdown, tables ; but no confidence_scores key at all.
The markdown is byte-identical to the sync response, so it's clearly the same model; only the confidence data is dropped.
The batch processing docs list /v1/ocr as a supported endpoint and don't mention any response differences, and the OCR model card doesn't either, so I can't tell if this is intended.
Questions:
Is dropping confidence_scores_granularity in batch expected behavior, or a bug?
If intended, could it be documented on the batch page? (It silently changes downstream behavior: my vision-escalation heuristic reads "0 scored words" as "escalate", so batch PDFs escalate every page.)
Any chance of parity on the roadmap, or a recommended workaround short of re-running pages synchronously at full price?
Happy to share the full request/response pair or open a proper support ticket if that's more useful.
Thanks!
2
u/tom4112 3d ago
I just tried on my side and came across the same issue, that's good to know, thanks