Repository navigation
Wrong advance widths for non-ASCII characters in non-embedded standard-14 fonts with /WinAnsiEncoding (© gets the width of quotesingle) #403
Description
Summary
For a non-embedded standard-14 font that declares /Encoding /WinAnsiEncoding and has no /Widths (and no /DW), pdf_resource<PAGE_FONT>::get_width() looks the raw character code up in the AFM's built-in Standard encoding instead of using the glyph named by the declared encoding. Every code above 0x7F where WinAnsi and Standard differ gets the width of a different glyph, which shifts all following glyphs — in the parsed text cells and in the rendered page image.
This is the cause of the diacritics problem in docling-project/docling#4497 (exported images changed when docling 2.123.0 made the threaded docling-parse backend the default).
Minimal reproduction
A 729-byte PDF: Times-Roman, /WinAnsiEncoding, no /Widths, font size 100. Each line shows X<c>X, so the middle glyph's advance is the word width minus two X (722 units each).
Generator (stdlib only)
chars = [0xA9, 0xED, 0xE9, 0x41] # copyright, iacute, eacute, A
content = "\n".join(
f"BT /F1 100 Tf 50 {700 - 150 * i} Td <58{c:02X}58> Tj ET" for i, c in enumerate(chars)
).encode("latin-1")
objs = [
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] /Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R >>",
b"<< /Length %d >>\nstream\n" % len(content) + content + b"\nendstream",
b"<< /Type /Font /Subtype /Type1 /BaseFont /Times-Roman /Encoding /WinAnsiEncoding >>",
]
out, offsets = bytearray(b"%PDF-1.4\n"), []
for i, body in enumerate(objs, 1):
offsets.append(len(out))
out += b"%d 0 obj\n" % i + body + b"\nendobj\n"
xref = len(out)
out += b"xref\n0 %d\n0000000000 65535 f \n" % (len(objs) + 1)
out += b"".join(b"%010d 00000 n \n" % o for o in offsets)
out += b"trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n" % (len(objs) + 1, xref)
open("min_winansi_times.pdf", "wb").write(out)Advance of the middle glyph in font units, docling-parse 7.22.2 (from the word-cell width) vs pdfium (from consecutive char origins):
| char (WinAnsi code) | Times-Roman AFM | docling-parse | pdfium | what docling-parse used |
|---|---|---|---|---|
© (0xA9) |
760 | 180 | 760 | Standard 0xA9 = quotesingle |
í (0xED) |
278 | 500 | 278 | Standard 0xED is unassigned → fallback |
é (0xE9) |
444 | 722 | 444 | Standard 0xE9 = Oslash |
A (0x41) |
722 | 722 | 722 | same glyph in both encodings |
The decoded text is correct (©, í, é); only the widths are wrong.
Two defects
get_width()ignores the declared encoding. With no/Widthsor/DWit callsbfont.has(c)/bfont.get_width(c)with the raw code, and AFM codes follow the font's built-in encoding.get_glyph_name()already resolves the declared base encoding first (the0xE1→Æfix), but the width path does not.- The AFM parser drops unencoded glyphs.
base_font::initialise()only stores entries withnumb >= 0, and in the standard AFMs every accented Latin letter andcopyrightisC -1(e.g.C -1 ; WX 278 ; N iacute). So these glyphs have no metrics at all, and a name-based lookup would fail too. (name_to_width/name_to_bboxare filled but never read.)
Real-world impact
On the PDF from docling#4497 (doi:10.1590/S0103-50531999000300013, four pages of Times-Roman/Italic/Bold, all /WinAnsiEncoding, none embedded, no /Widths), the header © 1999 Soc. Bras. Química gets a 1.3 pt © cell and 1999 shifted left by about 4 pt; in the rendered image © overlaps 1999 and the letters after í move. Any older Latin-script PDF set in non-embedded standard fonts with WinAnsi encoding is affected (é, ç, ã, ü, …).
Suggested fix
- In
base_font::initialise(), keepC -1entries in the name-keyed maps. - In
get_width()(andnonzero_fallback_width()), when the font has an explicit base encoding or/Differences, resolveget_glyph_name(c)first and use the name-keyed width; keep the code-based lookup for fonts that use their built-in encoding.
I'd like to send a PR for this, with the PDF above as a regression test asserting the cell widths. #370 also touches base_font.h (decoding, not widths), so I'd rebase on whichever lands first.
Activity
wittjeff commentedon Oct 16, 2026
Reproduced on docling-parse 7.22.1 with the 729-byte PDF above and on the SciELO paper from docling#4497 (page 1: the
©cell is 1.25 pt wide at a 6.96 pt font size, i.e. 180 units;íin Química gets 500 units where Times-Italic has 278).Both defects are where you describe them on current
main(7.22.3):pdf_resource<PAGE_FONT>::get_width()goesbfont.has(c)→bfont.get_width(c)with the raw code, thenbfont.has(get_string(c)), which cannot help becausebase_font::initialise()only stores entries withnumb >= 0. That drops 166 of the 315 glyphs in every Latin AFM (Times-Roman.afmet al.; Symbol drops 1, ZapfDingbats 0).name_to_width/name_to_bboxare filled only for the encoded glyphs and are never read.One more site uses the raw code the same way and shows in the rendered image independently of the advance:
pdf_states/text.h(around theglyph_rectblock, ~line 915) callsfont.has_char_bbox(glyph_code)/get_char_bbox(glyph_code), which indexnumb_to_bboxof the AFM. The bbox travels in thetext_instruction, andrenderer<BLEND2D>::calculate_glyph_bbox_adjustmentfits every non-ASCII glyph into it whenfit_glyph_bbox_to_targetis on (the default). So on the repro,©is scaled intoquotesingle's box (B 48 431 133 676) andéintoOslash's (B 34 -80 688 734); the squeezed©and the stretchedéare visible next to a pdfium render.í(no Standard glyph at 0xED) keeps its natural shape. Resolving the glyph name first fixes this path too, provided theC -1entries land inname_to_bboxas well asname_to_width.Two notes for the PR:
base_font::has_glyph_name()on top ofname_to_widthwith the comment "every encoded glyph of the metrics file is entered with its width". Keeping theC -1entries widens that to every glyph of the file, which is what its Symbol check wants anyway (Symbol's only unencoded glyph isapple), so the two changes compose, but the comment should follow.get_glyph_name()returns the base-encoding name only whenhas_explicit_encodingis set and the encoding is one of the four named ones. A font with/Differencesbut no/BaseEncodingfalls back to the program's builtin encoding per 9.6.6.2 for the codes/Differencesdoes not mention, so for a non-symbolic non-embedded font that case still needs the Standard table rather than the AFM code, which for the standard 14 is the same thing. Nothing to do there, just confirming the branch condition is right for the width path too.usercjr commentedon Oct 29, 2026
Thanks @wittjeff for reproducing this and for finding the third site in
text.h— that explains the squeezed©and stretchedéI saw in the rendered images, so the fix should resolve the glyph name before the bbox lookup as well.Two additions:
ASCII is affected too. WinAnsi
0x27isquotesingle, but the AFM's code 39 isquoteright. With the same minimal PDF approach (Times-Roman,/WinAnsiEncoding, no/Widths), docling-parse gives the apostrophe 333 units where Times-Roman'squotesingleis 180 (pdfium: 180).0x60maps toquoteleftinstead ofgrave; both are 333 wide in Times-Roman, so only the bbox differs there. So any English text with apostrophes in these fonts is shifted, not just accented letters.Regression dataset (rev
21374211). Following #428 I scanned all 482 PDFs for pages that show text in a non-embedded simple font with no/Widthsand a WinAnsi/MacRoman base encoding, using a character whose code maps to a different glyph than in StandardEncoding. Only 4 pages qualify, each with one or a few such characters:regression/annots_01.pdf€ Â “ Ã ' ”regression/annots_02.pdf'regression/13728511855768238546-20.pdf—regression/2020020019307-7.pdf'(monospaced, so only the bbox changes)So the dataset barely exercises this path, and the document from docling#4497, where it affects every accented letter and the copyright line, isn't in it.
@cau-git, how would you like this submitted under #428? It touches
page_font.h/base_font.h(metrics, not glyph-to-Unicode decoding), so I'm not sure whether it belongs to the font topic after #431 or can go as a standalone PR. For evidence I'd cite the four dataset pages above with before/after cell diffs, and I can open a dataset PR adding a real document that shows it at scale if its license allows redistribution.