Fast Native LLM Inference in Coni: The Poem Generator
In our ongoing quest to make Coni the most capable, dependency-free, and relentlessly fun functional language out there, we’ve hit a massive milestone. We didn’t just write an AI script—we went fully native.
Today, we’re unveiling our first fully native AI example: The Poem Generator.
Using the highly capable Qwen 2.5 0.5b instruct model formatted as a raw GGUF binary, this new Coni script takes a theme, invokes a local LLM, computes a beautiful poem, and spits out characters at a shockingly fast rate of over 70 tokens per second (TPS).