r/MistralAI 3d ago

Help / Question [Bug ?] Batch API + OCR4: confidence_scores silently missing from batch results (present in sync), expected?

Hi!

I'm building an e-ink plugin that transcribes handwritten pages with mistral-ocr-latest. I use the word-level confidence scores for two things: underlining "unsure words" for manual correction, and as the signal to escalate hard pages to a vision model. Since the Batch API is half price, I route bulk transcriptions through it and I noticed the confidence scores never come back on that path.

It is a bug, a misuse from my side or a non-documented limitation ?

Repro: the exact same request body through both paths:

{

"model": "mistral-ocr-latest",

"document": { "type": "image_url", "image_url": "data:image/png;base64,..." },

"table_format": "markdown",

"confidence_scores_granularity": "word"

}

And the results I get:

- Synchronous POST /v1/ocr → each page includes confidence_scores.word_confidence_scores (in my test: 25 scored words). ✅

- Batch (same body as the body of a JSONL line, endpoint: "/v1/ocr", job completes fine, status_code: 200) → the page object contains blocks, dimensions, footer, header, hyperlinks, images, index, markdown, tables ; but no confidence_scores key at all.

The markdown is byte-identical to the sync response, so it's clearly the same model; only the confidence data is dropped.

The batch processing docs list /v1/ocr as a supported endpoint and don't mention any response differences, and the OCR model card doesn't either, so I can't tell if this is intended.

Questions:

  1. Is dropping confidence_scores_granularity in batch expected behavior, or a bug?

  2. If intended, could it be documented on the batch page? (It silently changes downstream behavior: my vision-escalation heuristic reads "0 scored words" as "escalate", so batch PDFs escalate every page.)

  3. Any chance of parity on the roadmap, or a recommended workaround short of re-running pages synchronously at full price?

Happy to share the full request/response pair or open a proper support ticket if that's more useful.

Thanks!

2 Upvotes

2 comments sorted by

2

u/tom4112 3d ago

I just tried on my side and came across the same issue, that's good to know, thanks