The invoice says €561. Your PDF library says €961.

Part 2 of a series on how files attack AI models and agents. Part 1: Hidden text in PDFs: what your LLM reads and your users don’t.

The material for this series came out of building file intake for AI agents at FoxNose. Every technique here is one we had to reproduce, measure and design for in our own pipeline.

Open this invoice and it asks for 561.00 EUR. Hand it to pdftotext, pypdf, PyMuPDF or PDFium and it asks for 961.00 EUR. The person who approved the page and the agent that booked the payment are now 400 EUR apart, and neither of them made a mistake.

Part 1 of this series ended with a 40-line script that flags text a person can’t see. I ran it on this invoice and it reported nothing, because every digit in that total is drawn in solid black. Then I ran it over 519 real PDFs and it flagged 173 of the 494 that have text. So this part is also a correction of the last one.

One entry in a table

A PDF doesn’t store text the way a text file does. It stores glyph codes and a font. To draw the page, a renderer looks up the shape for each code. To extract text, a library looks up the same code in a separate table, the font’s ToUnicode map, which exists only so that copy, search and extraction work. Nothing in the file format requires the two to agree.

Part 1 was about text with no ink: white, off the page, under a shape. This is the opposite case. The ink is right there, and it’s the extracted text that’s wrong.

All the code here, with pinned versions and the output we got, is in one gist. The invoices carry no real attack, only one changed digit and a harmless note.

pip install pymupdf==1.27.1 pypdf==6.7.5 pypdfium2==4.30.0 pillow numpy
brew install poppler tesseract   # or: apt install poppler-utils tesseract-ocr

The core of make_invoices.py is one override in that table. Code 0x35, the glyph for “5”, maps to U+0039, the digit 9:

page.insert_text((60, 200), "Total due: 561.00 EUR", fontsize=14, fontname="hebo")
font = next(f[0] for f in page.get_fonts() if "Helvetica-Bold" in f[3])
cmap = doc.get_new_xref()
doc.update_object(cmap, "<<>>")
doc.update_stream(cmap, b"""/CIDInit /ProcSet findresource begin
12 dict begin begincmap
/CMapName /Swap def /CMapType 2 def
1 begincodespacerange <00> <FF> endcodespacerange
2 beginbfrange <20> <34> <0020> <36> <7E> <0036> endbfrange
1 beginbfchar <35> <0039> endbfchar
endcmap CMapName currentdict /CMap defineresource pop end end
""")
doc.xref_set_key(font, "ToUnicode", f"{cmap} 0 R")

The page next to what the libraries return. All four return the same wrong total.

Source	Total on swap.pdf
The page, as drawn by poppler, MuPDF and PDFium	561.00 EUR
tesseract 5.5.0, reading that page	561.00 EUR
pdftotext 26.03.0	961.00 EUR
pypdf 6.7.5	961.00 EUR
PyMuPDF 1.27.1	961.00 EUR
PDFium (pypdfium2 4.30.0 and 5.14.0)	961.00 EUR

Part 1’s advice to run your detector on the same extractor your pipeline uses doesn’t help here, because all four extractors agree. There’s no disagreement to notice unless something reads the page itself.

Handing the PDF to a model doesn’t settle it either. Which total it reports depends on what the provider’s API does with the file, and that’s the next part of this series.

Nobody has to attack you for this to happen

Font-map trouble sounds like an attacker’s trick. So I compared text layers with OCR on the first three pages of a corpus of 519 public PDFs we use for testing: tax guides, US public laws, EU reports, NASA memos, slide decks. 493 of them have a text layer on those pages, 1,307 pages and 529,059 words in all.

A slide deck exported through macOS Quartz has a page that reads “Total responses”. All four libraries return this:

7RWDOUHVSRQVHV

That’s “Totalresponses” with every character code shifted down by 29, and the space gone as well. An EU publication prints “Europe Direct is a service to help you find answers” on its back cover, and the extractors return:

$TQNOD«#HQDBS«HR«@«RDQUHBD«SN«GDKO«XNT«ɅMC«@MRVDQR«

Both are benign. The first file has a ToUnicode map and the map is wrong. The second has none, so the libraries fall back on raw character codes. A simple shift test found these 2 files out of 493, and it would miss any mapping that isn’t a constant shift, so treat that as a floor. It’s still enough to say that a pipeline trusting its text layer will sometimes feed the model noise with no attacker involved.

Proof stamps are the milder version. US public laws carry lines like VerDate Sep 11 2014 in the text layer, and an IRS guide, Publication 590-A, has a printer’s line off the edge of its pages. pypdf returns it on page 1, and the other three libraries don’t:

Userid: CPM Schema: tipx Leadpct: 100% Pt. size: 10  Draft  Ok to Print

My check from Part 1, on real files

The Part 1 script calls a span invisible when the pixels under it are nearly flat, and it also flags fonts under 4 pt and anything off the page. On these 519 files, reading up to 40 pages of each, it flagged 14,964 of 1,456,867 spans, in 173 of the 494 files with text.

Some of those flags are right in the narrow sense, such as the proof stamps above. Others are the script’s own fault. In a sample of 60 flagged files, reading up to 10 pages of each, 30 percent of the “no ink” spans sat just under its fixed cutoff, with 30 to 59 grey levels of difference under them. One example is a light grey sidebar that a person can read. Another 6 percent were zero-height boxes that have nothing to measure. I didn’t go through all 14,964 by hand, so I can’t give you a false-positive rate, only the fraction of files a reviewer would have to open.

A flag on 35 percent of ordinary documents means reviewers stop reading the flags. I called that script a sketch at the time, which was accurate, but I hadn’t measured how rough a sketch it was.

It also has a blind spot I should have caught. It reads spans with PyMuPDF, so it can only question what PyMuPDF returns, and that’s the next section.

Which page?

Part 1 compared the text layer against “the rendered page”, as if there were one. These are the other two invoices from make_invoices.py. clip.pdf draws a harmless note inside a clipping path of zero area. print_only.pdf puts it in an optional-content layer that is off on screen and on in print.

	Text extraction returns the note	Page shows the note
clip.pdf	pdftotext, pypdf, PDFium; not PyMuPDF	PDFium 126 only; not PDFium 156, poppler or MuPDF
print_only.pdf	pdftotext, pypdf, PDFium; not PyMuPDF	poppler only; not MuPDF or PDFium

The first row is the one that surprised me. The same engine paints that note in build 6462 (pypdfium2 4.30.0) and doesn’t in build 8076 (pypdfium2 5.14.0). I had “PDFium paints this” written down as a fact about the engine. It’s a fact about one version.

The Part 1 script passes both files. PyMuPDF doesn’t return the note, so the script never looks at it, while pdftotext and pypdf hand it straight to whatever comes next.

OCR isn’t a person either

So read the page with OCR and use that? It does fix the invoice: tesseract reads 561. It also has its own idea of what’s visible. I drew the same 9-word line in grey on a white A4 page, alone and then below four lines of black text, rendered it at 216 dpi and ran tesseract 5.5.0 on its defaults:

Grey level	Contrast with white	Line alone on the page	Same line with black text above
0.50	3.98	9 of 9 words	9 of 9
0.55	3.35	9 of 9	9 of 9
0.60	2.85	9 of 9	0 of 9
0.98	1.04	9 of 9	0 of 9

At 0.98 a person sees a blank page, and OCR reads every word. At 0.60 a person reads the line without trouble, and OCR drops all of it once there’s black text on the page, most likely because the black text pulls its binarisation threshold down. The boundary moves with what else is on the page. This is one engine on one synthetic page, and other engines will land elsewhere, but I wouldn’t expect any of them to land on “whatever a person sees”.

OCR also fails on real pages in less tidy ways. The page with the worst score in my run was from a 2014 German whitepaper: 94 percent of its text layer had no match. I read the first few words, “Anmerkungen / Fragen des Layouters”, notes for the layout designer, and wrote down a nice example of draft notes left behind in a text layer. Then I opened the page. The notes are right there, a tiny white outline on a black panel that tesseract skipped. The text layer was correct and OCR was wrong.

A comparison that catches the swap, and what it costs

compare_layers.py in the gist reads the text layer with PyMuPDF, runs tesseract on the rendered page, and looks for each word near its position in the OCR output. OCR is noisy, so words match loosely, the way most diffs of this kind do:

difflib.SequenceMatcher(None, "961.00", "561.00").ratio()   # 0.83

With a similarity cutoff of 0.75, the swapped total passes. One wrong digit in a six-character token looks like ordinary OCR noise, and the tolerance you need for words is the gap this attack fits through. So numbers of three or more digits have to match exactly:

for x0, y0, x1, y1, word, *_ in page.get_text("words"):
    native = norm(word)
    box = fitz.Rect(x0, y0, x1, y1) * page.rotation_matrix
    pad = max(box.height, 6)
    near = [t for cx, cy, t in read
            if box.x0 - pad <= cx <= box.x1 + pad and box.y0 - pad <= cy <= box.y1 + pad]
    if len(digits(native)) >= 3:
        if digits(native) not in digits("".join(near)):
            yield "number", word, near[:4]
    elif not any(similar(native, t) for t in near):
        yield "word", word, near[:4]

On the invoice:

page 1: number text layer '961.00', page has ['due', '561.00', 'eur']

The first version of this script reported 97 percent of the words on a rotated EU report as unmatched, because PyMuPDF’s word boxes ignore page rotation and I hadn’t applied rotation_matrix. With that fixed the same page has 12 misses in 249 words. Here’s what it does on the whole corpus, where nothing was planted:

Rule	Items that failed	Pages (of 1,307)	Files (of 493)
Words, loose match	13,511 of 529,059	983	452
Numbers, exact match	1,271 of 22,890	257	199

As a gate on whole files this is useless: it fires on 40 percent of them through the number rule alone. More than half of the failed numbers I stored (622 of 1,151) had no OCR word near them at all. In the sample I read, those were usually table cells tesseract skipped. The word rule also takes 2,286 hits from non-Latin text, because I ran the English model only.

The boundaries of those numbers: first three pages of each file, tesseract’s defaults, one extractor. It says nothing about how often this is abused in the wild. I found no attack in the corpus and I wasn’t looking for one.

What I’d do in a pipeline today

1. Pick one source for the model’s text and write down which. If the model gets the text layer, you’re trusting a table the file’s author wrote. If it gets OCR, you inherit OCR’s errors, but what it reads at least came from the same pixels a person can check. If you hand the PDF to a provider’s API, test which of the two it uses with a file like this one. The documentation doesn’t always tell you.

2. Compare only what you act on. A whole-page diff drowns you. A field you pay, book or store is short, so check that one exact: the total, the IBAN, the invoice number, the date. Read it from the text layer and from OCR, and send it to a person when they differ.

3. Treat a mismatch as a finding, and keep the file. Expect broken fonts and OCR misses first, as above. A flag is a fact about a file, as in Part 1, and the reviewer needs to see both values.

4. Pin your renderer and test it when it moves. Keep a few layer fixtures like these in your test suite. A renderer upgrade changed what clip.pdf shows, and nothing in your own code would tell you.

5. Run detectors on the extractor your model reads from. Part 1 said this. The Part 1 script didn’t follow it for anyone using pdftotext or pypdf.

What we took from this

A PDF page is three things: what the text layer claims, what one renderer at one version paints, and what a person can read. Part 1 treated the last two as one. FoxNose Files treats them as separate, and that decides what happens to each case in this article:

Case	What the agent gets	What the file’s owner gets
Swapped font map	The digits on the page, read by OCR. The text layer never reaches it	No finding yet
Broken font map, like 7RWDO	The words on the page, read by OCR	No finding yet
Text in a zero-area clip	Nothing: the words are withheld and painted out of the page image	A suspicious finding
Print-only layer, or text with no ink	Nothing	A hidden-text finding
Off-page proof line	Nothing	A hidden-text finding
Tiny or very faint text	Words under a legibility floor are withheld and painted out	A finding on the page

The design behind that table:

• For a PDF, the agent’s text comes from OCR of the rendered page. The text layer is read only as evidence to compare against, so a swapped font map can’t put 961 in front of an agent, and the answer doesn’t depend on how a provider’s API treats a PDF.
• The renderer is pinned, and it’s the same render everywhere. OCR, the page image the agent can see and the checks all work from one set of pixels. Where the file format lets renderers disagree, as with a zero-area clip, a separate structural check looks at the PDF itself and doesn’t depend on what got painted.
• The safe copy of a file is rebuilt from the redacted page images. It has no text layer left to disagree with anything.

Now the limits. We don’t yet raise a finding for a swapped font map: the agent’s text is unaffected, but the owner isn’t told, and the numbers above show why a cheap version of that check would mostly report noise. The legibility floor is a measurement, and we don’t claim it holds against a page built to beat it. And when we ran a 62-page IRS publication through the pipeline, it raised 62 hidden-text findings, one per page, all of them that printer’s line. By mechanism alone, off-page text can’t be told apart from a hidden prompt, so we left it flagged, and the collection’s policy decides what happens next.

All of this is one slice of what Files does with an upload. Before anything is rendered it refuses known malware, Office macros, remote templates, DDE fields, polyglots and zip bombs. It flags invisible Unicode such as tag characters and bidi controls, and it screens visible text for prompt injection. It strips metadata from images, reports JavaScript and embedded files in PDFs, and renders documents in a sandbox with no network. Text that’s plainly visible still reaches the agent, so all of it arrives marked as untrusted, with the verdict and findings beside it.

FoxNose Files isn’t available yet; you can get notified when it launches.