[ykhrustalev]

Overview

LFM2-VL, LFM2.5-VL, d1-3B and d1-omni cut a large image into 512x512 tiles plus a small thumbnail. llama.cpp did this differently from the reference Python code (transformers Lfm2VlImageProcessor, d1-omni's vision.py) in two ways:

• Tiles: Python stretches the image to fill the tile grid. llama.cpp kept the image's shape and filled the rest with black bars. This PR stretches for all LFM2 models, as llama.cpp already does for d1-omni (model : add LiquidAI/d1-omni-600M decision model #30114).
• Layout: Python rounds halves to even and uses double precision when it picks the tile grid and the thumbnail size. llama.cpp used std::round and float, so 31,926 of the 16.8M image sizes up to 4096x4096 got another grid or thumbnail, and another number of tokens. For example, a 720x1600 image got 2x5 tiles instead of 2x4. This PR rounds as Python does, like mtmd: fix LFM2 image tiling threshold #27057 already did for should_tile.

Only LFM2 code changes; other models are not affected.

Additional information

Reproduction (prompt tokens: master 2812, this PR 2298, transformers 2298):

python3 -c "from PIL import Image; Image.new('RGB', (720, 1600), 'gray').save('img.png')"
llama-server -hf LiquidAI/LFM2.5-VL-450M-GGUF:F16 --port 8080 &
until curl -sf localhost:8080/health > /dev/null; do sleep 1; done
echo '{"max_tokens": 1, "messages": [{"role": "user", "content": [{"type": "image_url", "image_url": {"url": "data:image/png;base64,'$(base64 < img.png | tr -d '\n')'"}}]}]}' > req.json
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' --data-binary @req.json | jq .usage.prompt_tokens
python3 -c "from PIL import Image; from transformers import AutoProcessor; p = AutoProcessor.from_pretrained('LiquidAI/LFM2.5-VL-450M'); t = p.apply_chat_template([{'role': 'user', 'content': [{'type': 'image'}]}], add_generation_prompt=True); print(len(p(images=[Image.open('img.png')], text=t)['input_ids'][0]))"

With a 400x600 image: master 258, this PR 239, transformers 239.

Accuracy: the largest probability difference to the Python reference (fp32; d1: over the answer options, LFM2-VL: over transformers' top-20 next tokens), and the number of items with another prompt length. Master 50e3e3e, Apple M3 Ultra (Metal), F16 models:

model, test set	master	this PR
LFM2.5-VL-450M, 52 images	0.193, 14	0.014, 0
LFM2-VL-450M, 52 images	0.503, 14, 2 top-1 flips	0.145, 0, 0
d1-omni-600M, 96 image questions	0.954, 42, 4 flipped answers	0.010, 0, 0
d1-3B, 96 image questions	0.356, 36	0.029, 6 (see below)
d1-3B, 450 POPE questions	0.056, 5	0.009, 0

• LFM2-VL-450M stays at 0.145 because its published mmproj has no clip.vision.image_resize_algo, so llama.cpp resizes it with bilinear while transformers uses bicubic. With the key set to bicubic: 0.444 -> 0.003.
• The new layout code gives the same result as the Python code for every image size up to 4096x4096. Where the layout changes, 27,834 sizes get fewer tokens (up to 2048 fewer) and 4,092 get more: most by under 100 tokens, 523 by 256 to 1024 tokens (shapes that sit exactly between two grids, where Python picks the bigger grid).
• One case gets worse for d1-3B: its own runner shrinks images above 1 megapixel before tiling (720x1600 becomes 686x1526, 2x5 tiles). Master matched that by accident; this PR gives 2x4, as transformers does for the full image. 6 test items change: 3 worse (largest 0.019 -> 0.029), 3 better. Adding the shrink step is out of scope.

Speed follows the number of tokens. Whole-request time, this PR vs master 50e3e3e (F16, -np 1), on an H100 GPU (CUDA) and a 30-core Xeon 8358 (CPU):

image (prompt tokens, master -> this PR)	CUDA (LFM2.5-VL / d1-3B)	CPU (LFM2.5-VL / d1-3B)
504x1120 (2817 -> 761)	-65.0% / -69.4%	-73.6% / -72.6%
720x1600 (2817 -> 2303)	-16.6% / -17.3%	-16.3% / -17.8%
640x480, 1024x768, 2000x1500, 3024x4032 (unchanged)	-2.8% to +0.8%	within noise
1000x1200 (1797 -> 2568)	+29.9% / +36.4%	+43.0% / +42.9%
560x960 (771 -> 1799)	+90.6% / +111%	+124% / +131%

Stretching costs nothing: preparing a 1024x768 image takes 25.9 ms instead of 31.0 ms (LFM2.5-VL, CUDA build).

Tests: test-mtmd-impl gets 4 thumbnail-size cases and 4 grid cases with the values transformers gives; master's code fails 6 of the 13 checks. The stretch is a loader setting and has no unit test. For a 1024x768 image on LFM2.5-VL-450M, master's tiles differ from transformers in 97.5% of the pixel values (up to 255, the black bars); this PR's tiles are identical.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES , Claude is being used to run the investigation, implementation and the tests, human supervised every stage and directed the agent.

[ngxson]

could you please rebase this?