TandemLLM
Open models, working as agents, on an engine of my own.
What TandemLLM is
TandemLLM began as an inference engine and grew into the environment around it, a first-level harness. A request goes in and a router reads what it wants and how much reasoning it deserves. The task gets the skills and packages written for it, notes from earlier sessions when memory is on, and tools that run in a sandbox that belongs to this one conversation. While the model writes, a conductor reads along and steps in when a tool call is broken or a reasoning loop will not end.
Under that sits the engine. Small drafters guess the next tokens, the big model checks the whole guess in one pass, and a guess it agrees with is a free token. A wrong one costs time. Caches keep the KV and recurrent state of a long conversation, so the next turn starts where the last one stopped. The text that comes out is the model's own.
It serves several open models today, some from my own EXL3 packs, and nothing in the design depends on one model or one machine. The aim is a stack where models of different sizes work in tandem, each doing the part it is good at.
The path of one request
- RoutingA router reads the request, decides what kind of task it is and how much reasoning it deserves.
- Skills and memoryThe task gets the skills and packages written for it, and notes from earlier sessions when memory is on.
- Sandboxed toolsShell, files, web and MCP servers run in a sandbox that belongs to this one conversation, behind permission prompts.
- State cachesThe KV and recurrent state of the last turn is still there for the next one, in memory or on disk, so a long conversation resumes instead of reading it all again.
- Draft and verifySmall drafters propose a tree of tokens. The big model checks the whole tree in one pass and keeps every guess it agrees with.
- MonitorWhile the model writes, the conductor reads along. It closes a reasoning loop, repairs a broken tool call and checks each call against its schema.