All posts
AI

Before You Swap Haiku 4.5 for 5.5, Check Prefill, Sampling Settings and Thinking Blocks

ZKO blog tile: AI

Haiku 5.5's price cut makes the swap tempting. Do not do it with a model name change alone. Anthropic's what's new page lists breaking changes from Haiku 4.5.

Three will catch most codebases. Assistant message prefill returns an error, so end your messages with a user turn. Sampling parameters can return an error, so omit temperature, top_p and top_k. Manual extended thinking with budget_tokens also errors, and you replace it with adaptive thinking.

Our take: the quieter change will cost you money. Haiku 5.5 keeps thinking blocks from all earlier assistant turns in context, where Haiku 4.5 kept only the latest. Multi-turn conversations carry more input tokens than the roughly 30 percent tokenizer increase alone explains. Anthropic points to thinking block clearing if that bites.

Two more small ones: a response can begin with a thinking block, so select content by type and never by position. And thinking text is omitted unless you set thinking.display to summarized.

Run your test suite and recount tokens before you quote a saving to anyone.