You don't always need a language model to classify intent

The task at the AI coding workshop I did a few weeks ago was to bring an AI language model into a game. In Sands of Ra, that’s the wishes: you type any wish, and Claude decides what happens. But that needs Claude Code running on your own computer, and the wishes are one of the game’s main features, so I didn’t put it online.

To put it online, something else had to read the wishes. I assumed I’d need something like BERT, which I used in my master’s thesis, Autonomous ATC, to sort pilots’ requests into types.

Instead, I trained a much smaller model: an intent classifier, a bag-of-n-grams logistic regression. It doesn’t write anything, it just sorts each wish into one of the 14 things the council can grant. How it works is on the project page.

It trains in under three seconds, where fine-tuning BERT took about 15 minutes, and it’s a 200 KB download instead of about 440 MB. On wishes it hasn’t seen, it’s correct 90% of the time, and inference takes about 0.02 milliseconds a wish, in your browser. Claude took 4 to 5 seconds.

Where I think it fits

There’s a big rush for AI right now, and at work I’ve helped build chatbots on models like Haiku. Doing this made me think something this light could be the default instead. A lot of what those chatbots do day to day is put a message into one of a few boxes, and my guess, depending on the job, is that a model like this could handle 70 to 80% of it.

It only does one kind of job. The answers have to be a fixed list, and you need a few thousand examples. It can’t write answers, or handle questions nobody planned for. But it could have replaced BERT for sorting the pilots’ requests in my thesis.

It doesn’t have to be one or the other, either. A system could be a hybrid: the model scores every option, so it knows when it’s unsure, and only those messages go on to a big model. It’s very simple, and it would save a lot of money and time.