LLM

The Problem LLMs hallucinate. They return wrong types, invalid values, and malformed data. A rating meant to be 1-5 comes back as 10. An email field contains “not provided”. A required field is missing entirely. In Python, Pydantic is the standard solution - define a model, validate the output, catch errors before they crash your app. But what about Go? Why Existing Go Libraries Don’t Cut It Go has validation libraries. It has JSON schema libraries. But none of them solve the LLM output problem end-to-end. ...

The promise of a standardized interface for LLMs via OpenAI-compatible endpoints is compelling. In theory, it allows for a plug-and-play architecture where switching models is as trivial as changing a base_url. In practice, this compatibility is often an illusion. I’ve spent the past year building a multi-provider LLM backend, and the pattern is always the same: things work for basic text generation, then break the moment you need production-critical features. ...

The Problem LLMs stream JSON token by token. Your structured output arrives as: {"project": {"name": "Mo {"project": {"name": "Mobile App", "status": "in_prog {"project": {"name": "Mobile App", "status": "in_progress"}, "tasks": [{"title": "UI Redes ... Standard encoding/json fails on every chunk except the last: json.Unmarshal([]byte(`{"project": {"name": "Mo`), &result) // error: unexpected end of JSON input This was recently highlighted by swyx as a #1 or #2 performance issue in AI applications. You’re forced to wait for the complete response before showing anything to users - negating the entire point of streaming with json mode or structured output. ...