Koog: A JVM framework for building predictable, fault-tolerant and enterprise-ready AI agents (4.6k stars)
What it does
All text below is quoted directly from the project README.
>Koog is a JVM (Java and Kotlin) framework for building predictable, fault-tolerant and enterprise-ready AI agents across all platforms, from backend services to Android and iOS, JVM, and even in-browser environments. Koog is based on our AI products expertise and provides proven solutions for complex LLM and AI problems
Key features:
- Multiplatform development: Deploy agents across JVM, JS, WasmJS, Android, and iOS targets using Kotlin Multiplatform.
- Reliability and fault-tolerance: Handle failures with built-in retries and restore the agent state at specific points during execution with the agent persistence feature.
- Intelligent history compression: Optimize token usage while maintaining context in long-running conversations using advanced built-in history compression techniques.
- Enterprise-ready integrations: Utilize integration with popular JVM frameworks such as Spring Boot and Ktor to embed Koog into your applications.
- LLM switching and seamless history adaptation: Switch to a different LLM at any point without losing the existing conversation history, or reroute between multiple LLM providers.
- Model Context Protocol integration: Use Model Context Protocol (MCP) tools in AI agents.
- Knowledge retrieval and memory: Retain and retrieve knowledge across conversations using vector embeddings and RAG.
- Flexible graph workflows: Design complex agent behaviors using intuitive graph-based workflows.
>implementation("ai.koog:koog-agents:1.3.0")
Repository
JetBrains/koog
Repository stats
|Metric|Value|
|:-|:-|
|Stars|4,645|
|Forks|502|
|Last release|1.3.0 (2026-09-24)|
|Languages|Kotlin 97.3%, Java 2.6%, Other 0.1%|
|Contributors|150|
|AI contributor accounts (Claude / Codex / Cursor / Copilot)|@claude (6 commits), @Copilot (3 commits), @cursoragent (1 commits)|
|AI co-authored commits|Claude: 6, Cursor: 1, Copilot: 3|
Main contributor
|Field|Value|
|:-|:-|
|Name|Anastasiia Zarechneva|
|GitHub|@zarechneva-a|
|Commits to this repo|177 (top contributor)|
|Bio|QA engineer @ JetBrains|
|Company / Location|JetBrains / Germany|
|Followers|66|
──────────────────────────────
📌 If you want to browse more repos like this, we've organized them by category and rankings here: bestgithubrepositories.com
🗂️ We're also archiving repos where possible, so useful projects don't disappear when they're removed from GitHub.
5 points
An AI agent that asks before it pays: human approval for tool calls in Kotlin
Updated Oct 2026 for 0.1.0-alpha12: MCP tools, retries, tracing and a SQLite checkpointer are now in.
An agent that only talks is easy to trust. An agent with a refund tool is not. The model decides when to call it and with which amount, and the same question can get a different answer tomorrow.
I don't think a better prompt solves that. "Never refund without permission" is a sentence the model usually follows, and "usually" is not good enough for money. What solves it is structure around the model: ordinary code that decides where the model may act alone and where it has to wait.
This article builds that in Kotlin, in five steps: an agent, a pause before it acts, a pause that survives a restart, a test without a model, and a rule the model never gets to see. It uses Telar, an open-source library for agent workflows that I am writing. Every example here was compiled and run against the current release, 0.1.0-alpha12.
1. The agent
A tool is a function the model may call. Its input is a @Serializable class, and the library builds the schema the model reads from that class.
@Serializable
data class OrderLookup(
@Description("The order number, for example \"1001\"") val orderId: String,
)
@Serializable
data class RefundRequest(
@Description("The order number, for example \"1001\"") val orderId: String,
@Description("The amount to pay back, in euros") val euros: Int,
)
fun helpDesk(model: ChatModel, pay: suspend (RefundRequest) -> Unit): CompiledGraph<AgentState> {
val orderStatus = Tool<OrderLookup>("order_status", "Returns what was ordered, what it cost and when it arrived.") { lookup ->
"Order ${lookup.orderId}: one margherita for 9 euros, delivered 70 minutes late." // your database here
}
val refund = Tool<RefundRequest>("refund", "Pays money back for an order.") { request ->
pay(request)
"Paid back ${request.euros} euros for order ${request.orderId}."
}
return toolAgent(
model,
tools = listOf(orderStatus, refund),
system = "You work at the help desk of a pizzeria. Look an order up before you decide anything about it.",
)
}
toolAgent is the usual loop: the model answers or asks for tools, the tools run, and their results go back to the model, until it has an answer.
val model = AnthropicChatModel(HttpClient(), apiKey = System.getenv("ANTHROPIC_API_KEY"), model = "claude-opus-5-5")
val agent = helpDesk(model, pay = { payments.refund(it.orderId, it.euros) })
val state = agent.invoke(AgentState("My pizza for order 1001 came 70 minutes late. I want my money back.")).state
println(state.answer)
This works, and it pays whenever the model thinks it should. That is the part to fix.
2. Pause before the tools run
The agent is not a black box. It is a graph of two nodes, model and tools, with an arrow back from one to the other. And a graph can be told to stop before a node:
val config = GraphConfig(
threadId = "ticket-42",
checkpointer = MemoryCheckpointer<AgentState>(),
interruptBefore = setOf("tools"),
)
Now invoke returns GraphResult.Interrupted each time the model has asked for tools and none of them has run yet. The calls that wait are in the state, with their names and their inputs:
suspend fun answer(
agent: CompiledGraph<AgentState>,
question: String,
config: GraphConfig<AgentState>,
approve: suspend (List<ToolCall>) -> Boolean,
): AgentState {
var result = agent.invoke(AgentState(question), config)
while (result is GraphResult.Interrupted) {
val refunds = result.state.messages.pendingToolCalls().filter { it.name == "refund" }
result = if (refunds.isEmpty() || approve(refunds)) {
// Nothing risky, or a person said yes: run the calls.
agent.resume(config)
} else {
// A person said no: answer the call yourself. A call that already has a result is not run.
agent.resume(config) { state ->
state.copy(messages = state.messages + refunds.map {
ChatMessage.ToolResult(it.id, it.name, "A person declined this refund.", isError = true)
})
}
}
}
return result.state
}
Three things happen in that loop:
- Lookups pass without a question. The code decides which tools need a person, by name. Nobody is asked to approve reading an order.
- A yes runs the call as the model wrote it. approve gets the calls, so your screen can show refund {"orderId":"1001","euros":9} before anyone clicks.
- A no goes back to the model as a result. The model reads "A person declined this refund." and writes the customer an answer that fits. The conversation stays whole.
The difference from a prompt is where the guarantee lives. The refund function is not reached unless approve returned true. The model cannot talk its way past an if.
3. The pause survives a restart
A person may answer in a minute or after lunch, and the server may be deployed in between. Telar saves a checkpoint after every step, so the only change is where it is saved:
val config = GraphConfig(
threadId = "ticket-42",
// One JSON file per thread. AgentState is @Serializable, so nothing else is needed.
checkpointer = FileCheckpointer(Path("checkpoints"), KotlinxStateSerializer<AgentState>()),
interruptBefore = setOf("tools"),
)
After a restart, lastResult reads where a thread stopped without running anything:
when (val result = agent.lastResult(config)) {
is GraphResult.Interrupted -> showApproval(result.state.messages.pendingToolCalls()) // still waiting for a person
is GraphResult.Completed -> showAnswer(result.state.answer)
null -> showEmptyForm() // this ticket has never run
}
agent.resume(config) then continues in the new process, from the step where the old one stopped.
One thing to know before you rely on this for payments: a tool call that finished is saved with the step, so resuming after a failure or a restart gives it its saved result instead of paying twice. A call that was cut off in the middle, by a crash during the payment itself, can run again. Give a side effect like a refund an idempotency key, for example made from the order number.
There are checkpointers for memory, files, the browser's localStorage, and SQLite through SQLDelight on every platform. Checkpointer is a small interface, so a server database is one class away; PostgreSQL is an open issue if you want to help.
4. Test it without a model
ChatModel is an interface with one function, so a model that follows a script is a lambda. This one looks the order up, asks for the refund, and then answers according to what came back:
val scripted = ChatModel { request ->
val results = request.messages.filterIsInstance<ChatMessage.ToolResult>()
ChatResponse(
when (results.size) {
0 -> ChatMessage.Assistant(toolCalls = listOf(ToolCall("call-1", "order_status", buildJsonObject { put("orderId", "1001") })))
1 -> ChatMessage.Assistant(toolCalls = listOf(ToolCall("call-2", "refund", buildJsonObject { put("orderId", "1001"); put("euros", 9) })))
else -> ChatMessage.Assistant(if (results.last().isError) "I am sorry, I cannot refund this order." else "Your 9 euros are on their way.")
},
)
}
And the property that matters becomes a unit test that runs in milliseconds, with no key and no network:
@Test
fun aRefundThatAPersonDeclinesIsNotPaid() = runTest {
val paid = mutableListOf<RefundRequest>()
val agent = helpDesk(scripted, pay = { paid += it })
val config = GraphConfig(checkpointer = MemoryCheckpointer<AgentState>(), interruptBefore = setOf("tools"))
val state = answer(agent, "My pizza for order 1001 came 70 minutes late. I want my money back.", config, approve = { false })
assertEquals(emptyList(), paid)
assertEquals("I am sorry, I cannot refund this order.", state.answer)
}
The model is the part you cannot predict. Everything around it is plain Kotlin on an immutable state, and you test it the way you test any other Kotlin.
5. Rules go in code, not in the prompt
Some rules have no exceptions. Say orders older than 30 days are never refunded. You could write that into the system prompt and hope. Or the model never sees those orders.
toolLoop is the same agent as a part of a graph of your own, with the conversation in a state you define. model, orderStatus and refund are the ones from step 1:
data class Complaint(
val orderId: String,
val text: String,
val daysAgo: Int,
val messages: List<ChatMessage> = emptyList(),
val reply: String = "",
)
val desk = StateGraph<Complaint> {
val tooOld = node("too_old") { it.copy(reply = "Orders older than 30 days cannot be refunded.") }
val send = node("send") { it.copy(reply = it.reply.ifEmpty { it.messages.last().text }) }
val agent = toolLoop(
model = model,
tools = listOf(orderStatus, refund),
// Where the conversation is in your state, and how to add messages to it.
messages = { it.messages },
append = { complaint, new -> complaint.copy(messages = complaint.messages + new) },
firstMessage = { "Order ${it.orderId}: ${it.text}" },
then = send,
)
conditionalEdge(START, targets = setOf(agent, tooOld)) { if (it.daysAgo > 30) tooOld else agent }
tooOld then send
send then END
}.compile()
An old order takes the arrow to too_old, and the model is not called at all. No tokens, no risk, and a test that proves it.
compile() checks the drawing before anything runs. A node that nothing leads to, or an arrow to a node that does not exist, fails there and not in production at the fourth step of a run.
Where this runs
All of the above is Kotlin Multiplatform. The core depends only on kotlinx-coroutines, and the same graph runs on the JVM, Android, iOS, desktop, and in the browser as JS or Wasm. The models are Claude and anything with the API of OpenAI (OpenAI, Gemini, Ollama, Groq and others) on Ktor client, and any LangChain4j model on the JVM.
To see graphs like these run without writing code, there is a small game in the browser in which every stage is a Telar graph. It needs no account and no API key.
What is missing
Telar is an early alpha, and I would rather say so here than have you find out:
- The API can still change before 1.0.
- A message holds text only: no images or files yet.
- A long conversation is sent to the model in full every time.
- Tracing has spans for runs and nodes, but not yet for each model call and tool call.
- The full list is on the roadmap.
- If you need a complete agent platform today, Koog from JetBrains is the safer choice. The site has a comparison with Koog, LangGraph4j and LangChain4j.
Telar is for the case this article is about: the workflow around the model is the hard part, and you want it to be ordinary, typed, testable Kotlin.
Try it
// build.gradle.kts. `@Serializable` needs the plugin kotlin("plugin.serialization").
implementation("dev.deeptelar:telar-agent:0.1.0-alpha12")
implementation("dev.deeptelar:telar-anthropic:0.1.0-alpha12") // or telar-openai: OpenAI, Gemini, Ollama, ...
implementation("dev.deeptelar:telar-checkpoint-file:0.1.0-alpha12") // step 3
implementation("dev.deeptelar:telar-serialization:0.1.0-alpha12") // step 3
implementation("io.ktor:ktor-client-cio:3.6.0") // any Ktor engine
- Code: <deeptelar/telar>
- Documentation and tutorial: <deeptelar.github.io>
- The guides behind this article: Agents with tools and Human-in-the-loop
It is early, which makes this the best time for feedback. If something in the API looks wrong to you, open an issue or tell me in the comments.
1 reaction
The JVM Speaks AI: Cobrainer's Journey with Kotlin, Koog and Exposed
---
title: "The JVM Speaks AI: Cobrainer's Journey with Kotlin, Koog and Exposed"
published: true
tags: kotlin, ai, koog, jvm
cover_image: dev-posts-0b9b92.gitlab.io
---
One afternoon, one of our LLM providers started throttling us. A large customer had triggered a burst of AI-assisted work, and the provider's capacity in our region could not keep up. Our prompt executor followed its fallback chain, the circuit breaker took the affected models out of rotation, and each request moved to a second provider without interrupting the workflow. There was no incident to handle and no change in system behaviour for the customer. We noticed it only the next morning on our LLM observability dashboard.
We built all of our AI integration in Kotlin. A controversial decision? We do not think so. In 2025, one could still argue that AI integration required Python. By 2026, that debate is no longer a useful basis for architecture decisions. Python remains important for research and model training, but production AI is mostly about integrating APIs, enforcing contracts, and operating reliable systems. Every prompt, agent, and structured-output contract in our core platform lives on the JVM: one Kotlin monorepo with types shared end to end, built on Koog, Exposed, coroutines, and kotlinx.serialization.
TL;DR
- Kotlin lets six developers and their coding agents work in one stack. Sealed types, kotlinx.serialization, null safety, and coroutines make contracts and control flow explicit.
- Koog was the best fit because its lightweight abstractions build on Kotlin instead of hiding it. We moved the proof of concept into production for all customers in one day.
- Production reliability comes from cross-provider fallback chains, circuit breakers, LLMObs, and contract tests. Removing blocking code from coroutine paths also increased throughput by 5×.
We at Cobrainer
Cobrainer is a Munich-based B2B HR-tech company that grew out of a Technical University of Munich research project in 2013. We serve enterprises across industries and regions. Our AI-native Skills Platform creates and governs living Job and Skill Architectures: structured models of an organization's functional roles, levels, tasks, and skills. They answer three basic questions: Which distinct jobs exist in the organization? What is needed to perform them? How do they relate to each other? A patented Skills Graph and ML/NLP power the platform.
Our customers use it for skills-based (AI) workforce transformation: internal mobility, recruiting, learning, workforce planning, and EU pay-transparency readiness, with integrations into SAP SuccessFactors, Workday, ServiceNow and other HCM systems.
Our Summer 2026 release includes a context-aware Personal Agent for the Job Architecture and the wider platform, agentic background tasks, agentic (g)RAG, and collaborative multilingual commenting for governance. The AI can resolve structural and language-specific comments on job profiles in more than 25 languages.
Our agentic engineering setup centres on six developers with end-to-end ownership, from cloud infrastructure through backend and frontend to AI. Coding agents help us explore the repository, implement changes, run checks, and investigate operational issues. Together, we serve more than 10,000 active users at companies around the world.
Small, fast-moving team, big surface. That constraint explains almost every technology decision in this article.
Kotlin for AI-powered features
Why Kotlin? Reliability meets velocity
The control plane and core domain of our platform use Kotlin and Spring Boot. The stack is type-safe and null-safe end to end, with kotlinx.serialization for wire contracts, Exposed for SQL, and coroutines for non-blocking I/O. A central monolith is the entry point for the platform, and everything lives in one monorepo. With six engineers working across the full system, every additional deployable adds work to development, operations, and CI.
A large monolith rewards explicitness. Its behaviour can span distant files, and coding agents cannot safely fill gaps left by implicit conventions. Kotlin's types and compiler checks make those relationships visible. Koog follows the same principle with small abstractions that keep contracts, capabilities, and control flow in the code. This helps developers review changes and gives coding agents firmer boundaries in which to work.
When our LLM work moved from isolated features to core platform components, we asked whether it justified a separate stack. The models sit behind HTTP APIs, so another runtime would not improve their output. It would give us a second deployment pipeline, add serialization boundaries between the AI layer and domain model, and remove compile-time guarantees at the boundary where LLMs introduce the most uncertainty.
We wrote down what we actually needed from an LLM framework:
- Lean and explicit. The framework should behave like a library while keeping control flow visible in the code.
- A release cadence that matches innovation speed and model churn. New models and paradigms arrive monthly, so infrequent framework releases become a bottleneck.
- Clean provider and model-capability abstractions. We need multiple providers, and their models support different features.
- Type safety end to end. An LLM's output should enter the domain as a typed value or not at all.
We tested these requirements by building working implementations instead of comparing feature matrices.
The turning point: why Koog fits Kotlin
We built and ran production implementations with Spring AI, LangChain4j, and Koog. The important difference was not the length of their feature lists. It was how clearly each framework expressed its behaviour in Kotlin for developers, coding agents, and production tooling.
Spring AI is the most comprehensive and deeply integrated Spring option. It offers a robust ecosystem and good portability for teams already using Spring Boot. We still use it for our MCP endpoints, where its integration with REST infrastructure and the Spring Security filter chain is exactly what we need. In the agentic core, however, it added another layer between our code and the model when we wanted direct control and rapid iteration.
LangChain4j is a large, feature-rich library that brings LangChain concepts to the JVM. It helps teams move quickly, but we found failures inside a chain difficult to trace back to the underlying LLM call. We eventually replaced the last utilities that depended on it with a few dozen lines of Kotlin.
Koog changed the direction of the evaluation because it made Kotlin itself the foundation of the integration. Our @Serializable domain types become structured-output contracts directly. Sealed hierarchies model the possible results, and exhaustive when expressions force every caller to handle them. Coroutines carry naturally through provider calls, retries, and agent workflows. Koog's DSL builds on familiar Kotlin features such as extension functions and lambdas with receivers, so agent graphs remain normal Kotlin code that can be traced, changed, and verified with the compiler.
The provider abstraction was equally important. Koog represents model capabilities explicitly instead of reducing every model to a lowest common denominator. We can adapt temperature, token limits, and structured-output handling for each model while keeping the same domain contract and call site. These decisions are visible in types and code instead of being hidden in configuration or prompts. When a developer or agent investigates a failure, the underlying request remains accessible rather than disappearing behind a chain of abstractions.
That combination gave us the level of abstraction we wanted: enough structure to avoid rebuilding provider clients and agent orchestration, but close enough to Kotlin that our existing tools, types, and debugging habits still applied. A coding agent can trace a change from a domain type through its LLM contract to every consumer without crossing languages or repositories. Koog did not introduce a second programming model into the codebase. It made the existing one more useful to both humans and agents.
Once we chose Koog, we cleaned up the proof of concept and rolled it out to all customers in one day. That speed came from keeping the domain model, serialization, concurrency model, tests, and deployment pipeline we already understood.
AI in production: structured output as the workhorse
In production, much of an LLM's value comes from filling data structures that domain logic can trust, rather than generating free-form chat. Koog has powered our Job Architecture generation since summer 2025, and structured output is the foundation of that core product capability.
The contract is a Kotlin sealed hierarchy with kotlinx.serialization. Koog turns it into a JSON schema (docstrings included, via @LLMDescription), gets the provider to comply, and hands back a typed value:
@Serializable
enum class ChangeKind { RENAME, ARCHIVE, MOVE }
@LLMDescription("A single proposed change to the job architecture")
@Serializable
sealed interface ProposedChange {
val kind: ChangeKind
@Serializable
@SerialName("rename")
data class Rename(
@property:LLMDescription("The item to rename")
val itemId: String,
@property:LLMDescription("New title, in the tenant's base language")
val newTitle: String,
) : ProposedChange {
override val kind get() = ChangeKind.RENAME
}
@Serializable
@SerialName("archive")
data class Archive(
@property:LLMDescription("Items to archive, including their descendants")
val itemIds: List<String>,
) : ProposedChange {
override val kind get() = ChangeKind.ARCHIVE
}
// ...
}
Requesting it is one call. If a weaker model returns almost-JSON, a StructureFixingParser repairs it with a lightweight model before we ever see it:
val plan = executor.executeStructured<ChangePlan>(
prompt = prompt("change-planner") {
system(plannerInstructions)
user(request)
},
model = model,
fixingParser = StructureFixingParser(model = lightweightModel, retries = 2),
).getOrThrow().data
plan.changes.forEach { change ->
when (change) { // The compiler enforces exhaustive coverage.
is ProposedChange.Rename -> preview(change.itemId, change.newTitle)
is ProposedChange.Archive -> confirmArchive(change.itemIds)
is ProposedChange.Move -> validateMove(change)
}
}
Kotlin provides specific safeguards here. Null safety exposes schema drift at compile time: add a required field to the contract and every construction site that omits it stops compiling. A when expression over a sealed result is exhaustive, so a new change kind breaks the build until every consumer handles it. Default arguments and small DSLs keep configuration readable, allowing us to review prompt code like domain code.
We also balance traffic across providers whose models differ in capabilities, and Koog represents those differences explicitly. A provider that uses an existing API format requires little integration code:
// xAI uses the OpenAI wire format.
@Serializable
object XAI : LLMProvider("xai", "xAI")
val xaiClient = OpenAILLMClient(
apiKey = settings.apiKey,
settings = OpenAIClientSettings(baseUrl = "https://api.x.ai"),
)
Our executor adds three behaviours to Koog's PromptExecutor: retry with backoff, a config-driven fallback chain, and a per-model circuit breaker. Before each attempt, it adapts the prompt to the target model's declared capabilities:
fun adaptPromptFor(model: LLModel, prompt: Prompt): Prompt = prompt
.letIf(!model.supports(LLMCapability.Temperature)) { it.withoutTemperature() }
.letIf(prompt.maxTokens > model.maxOutputTokens) { it.cappedTo(model.maxOutputTokens) }
// Model can't do native structured output? Ask for it in the prompt
// instead. The sealed contract and fixing parser stay identical.
.letIf(!model.supports(prompt.schemaCapability)) { it.withSchemaAsUserMessage() }
The call site never knows which provider answered. That indirection is what made the silent rebalancing from the opening of this article a non-event.
Growing into an agentic platform
At Cobrainer, AI powers skill inference, Job Architecture generation, transcription, chat agents, and agentic (g)RAG. Our Job Architecture chat agent brings several of these capabilities together.
The agent lets users create, review, and maintain a job architecture conversationally: "merge these two levels", "draft the missing task descriptions for this job family", or "archive everything nobody has touched since the reorg". At a high level, it is a supervisor tool loop with deterministic sub-workflows. The LLM decides what to do, while a Koog strategy graph routes every consequential action through code we control. A human must approve every change.
The agent is our one exception to the monolith. It runs in the Agent Runtime, a reactive Spring Boot Kotlin service designed for long-running conversations, streaming, and structured concurrency. The monolith remains the single entry point for the platform, including REST and MCP. Both services live in the same monorepo, so they share types directly and the compiler checks their contract. SurrealDB stores checkpoints for resumable turns, message history, and three layers of memory: a small window of recent conversation chunks, a running session briefing, and a fact-based long-term graph. Platform data remains in Postgres behind the monolith, accessed through Exposed.
Koog's graph DSL makes the tool loop almost declarative. We use marker tools that the model can call but that never execute directly. A guarded edge intercepts the call and routes it into a deterministic sub-workflow:
val toolLoop = strategy<String, String>("supervisor") {
val callLlm by nodeLLMRequest()
val executeTool by nodeExecuteTool()
val sendResult by nodeLLMSendToolResult()
val editFlow = installEditCapability(approvalGate)
val bulkFlow = installBulkCapability(approvalGate)
// Marker tools are visible to the model but never executed.
// The edge routes into a deterministic sub-workflow instead.
edge((callLlm forwardTo editFlow).soleMarker(EDIT_TOOL_ID))
edge((callLlm forwardTo bulkFlow).soleMarker(BULK_TOOL_ID))
// Edges match in declaration order, so the generic loop comes last.
edge(callLlm forwardTo executeTool onToolCalls { true })
edge(callLlm forwardTo nodeFinish onAssistantMessage { true })
edge(executeTool forwardTo sendResult)
}
The same graph drives the progress UI. Instead of streaming tokens, we show meaningful progress. Koog's event hooks fire as the agent moves through nodes and tools, and each transition maps to a localized status such as "Analyzing your architecture…" or "Preparing changes for review…". Showing graph progress improved perceived latency more than token streaming would have for this workflow:
install(EventHandler) {
onNodeExecutionStarting { ctx -> emitStatus(ctx.node.name) }
onToolCallStarting { ctx -> emitToolStatus(ctx.toolName) }
onAgentExecutionFailed { ctx -> events.close(ctx.error) }
}
Security is where the two frameworks meet. The agent accesses tools only through central MCP servers, which Spring AI serves inside the monolith. Every tool call therefore passes through the same Spring Security filter chain as our REST APIs. We run one MCP server per permission scope (read, architect, reviewer), guard each at the path level, and give the agent a short-lived delegated token so it always acts as the user. Permissions filter the available tools before any LLM call. If a user cannot edit the architecture, their agent never sees editing tools. This reuses our existing authorization infrastructure without relying on prompts for access control.
Koog runs the agent in the Agent Runtime, Spring AI exposes and protects its tools in the monolith, Exposed and Postgres store platform data, and SurrealDB stores agent memory. The result is two JVM services in one monorepo, sharing one type system.
!High-level architecture of the Job Architecture agent
Discoveries and challenges
Production use also exposed several important problems and lessons.
A 5× throughput gain from coroutine cleanup. Our most difficult performance problem had nothing to do with Koog or LLMs. Legacy blocking code inside coroutines starved dispatchers under AI-driven load, causing vague and load-dependent slowdowns. Once we found and removed it, throughput on the affected paths increased by 5×.
Which leads to the sharpest lesson we can offer:
> Do not leave runBlocking in production request paths. This applies even to a blocking Spring Boot servlet stack where it appears to work. Under load, it can deadlock a dispatcher without crashing the application; requests simply stop completing.
Build deep fallback chains across providers. Several fallback models from the same provider still share one failure domain. Our chains cross provider boundaries, and per-model circuit breakers skip throttled or unhealthy routes so retries reach a viable model quickly. We trace every attempt and fallback in LLMObs, including latency, token usage, and errors. That makes it possible to reconstruct a request across providers and distinguish a model failure from a parser or application problem. This setup paid off as soon as one provider lost capacity.
Contract-test the LLM layer. Structured output depends on our schema, the framework's schema generation, and each provider's implementation. We run end-to-end structured-output tests with real calls and parsing for every provider, as well as LLM contract tests in CI. Together with LLMObs, these checks form our AI operations loop: observe production behaviour, investigate failures with full context, and validate model or provider changes before rollout. This lets developers and agents move quickly without turning model churn into guesswork.
Final words
Building AI features in Kotlin once sounded unconventional. Today, its explicit contracts and lightweight abstractions give our six developers and their coding agents a system that is fast to change, safe to refactor, and straightforward to operate.
If you are considering AI development on the JVM, the ecosystem is ready. Have a look at Koog, Exposed, and Kotlin. We thank the JetBrains teams behind all three for their frequent releases and responsiveness.
Questions, disagreements, and production stories are welcome in the comments. We read them.
3 reactions
I built Telar, an open-source Kotlin Multiplatform library for AI agent workflows as typed graphs (early alpha, feedback wanted)
I wanted to build agent workflows in Kotlin the way LangGraph does it in Python: as a graph where each step is visible and you decide what runs next. I also wanted it to run on small devices and in the browser, not only on a server, so I wrote my own. It is called Telar, it is Apache 2.0, and it is an early alpha.
The idea in one example
The state is your own data class, a node is a suspend function that returns a copy of it, and arrows connect the nodes. This workflow stops before money moves and continues when a person says yes:
data class Refund(val orderId: String, val amount: Int, val approved: Boolean = false, val status: String = "new")
val refunds = StateGraph<Refund> {
val check = node("check") { it.copy(status = if (it.amount > 1000) "too large" else "checked") }
val pay = node("pay") { it.copy(status = if (it.approved) "paid" else "rejected") }
val decline = node("decline") { it.copy(status = "declined") }
START then check
conditionalEdge(check, targets = setOf(pay, decline)) { if (it.status == "checked") pay else decline }
pay then END
decline then END
}.compile() // a node nothing leads to, or a target that is not declared, fails here
suspend fun main() {
val config = GraphConfig(threadId = "order-1001", checkpointer = MemoryCheckpointer<Refund>(), interruptBefore = setOf("pay"))
// runs "check", then waits before "pay"
refunds.invoke(Refund("1001", amount = 250), config)
// a person said yes
val result = refunds.resume(config) { it.copy(approved = true) }
// paid
println(result.state.status)
}
That part needs only telar-core, which depends on kotlinx-coroutines and nothing else.
What it does today
- Graphs: loops, parallel nodes, subgraphs, and a stream of events while a run is going
- Save points: a checkpoint after every step, in memory, in a file, or in the browser's localStorage. A run can pause, survive a restart, and be forked from an earlier step
- Agents: a ready-made tool-calling agent (toolAgent). You give it a model and a list of tools, and a tool's input is a @Serializable class that becomes the JSON schema
- Models: Claude, and anything with the OpenAI API (OpenAI, Gemini, Ollama, Groq and so on) on Ktor client, plus an adapter for LangChain4j models on the JVM
- Targets: JVM, Android, iOS, macOS, Linux, Windows, JS and Wasm
What it is not
Stable, or complete. There is no MCP, no tracing, no retry policy and no database checkpointer yet, and the API can still change before 1.0.
If you need a finished agent platform today, Koog is the safer choice, and LangGraph4j and LangChain4j are both at 1.x on the JVM. Telar is for the case where the workflow itself is the hard part and you want its state to be ordinary Kotlin. There is a comparison table on the site, and I would like to hear where it is wrong.
Try it
To see graphs run without writing code or bringing an API key, there is a small game in the browser, written in Compose Multiplatform for Wasm, where every stage is a Telar graph.
- Code: deeptelar/telar
- Docs and tutorial: deeptelar.github.io
- Demo: deeptelar.github.io
The feedback that would help me most right now is about the API: the graph DSL, how state is merged when nodes run in parallel, and what you would need before trying it in a real project.
5 comments
Mentions
Koog: A JVM framework for building predictable, fault-tolerant and enterprise-ready AI agents (4.6k stars) What it does All text below is quoted directly from the project README. >Koog is a JVM (Java and Kotlin) framework for building predictable, fault-tolerant and enterprise-ready AI agents across all platforms, from backend services to Android and iOS, JVM, and even in-browser environments. Koog is based on our AI products expertise and provides proven solutions for complex LLM and AI problems Key features: - Multiplatform development: Deploy agents across JVM, JS, WasmJS, Android, and iOS targets using Kotlin Multiplatform. - Reliability and fault-tolerance: Handle failures with built-in retries and restore the agent state at specific points during execution with the agent persistence feature. - Intelligent history compression: Optimize token usage while maintaining context in long-running conversations using advanced built-in history compression techniques. - Enterprise-ready integrations: Utilize integration with popular JVM frameworks such as Spring Boot and Ktor to embed Koog into your applications. - LLM switching and seamless history adaptation: Switch to a different LLM at any point without losing the existing conversation history, or reroute between multiple LLM providers. - Model Context Protocol integration: Use Model Context Protocol (MCP) tools in AI agents. - Knowledge retrieval and memory: Retain and retrieve knowledge across conversations using vector embeddings and RAG. - Flexible graph workflows: Design complex agent behaviors using intuitive graph-based workflows. >implementation("ai.koog:koog-agents:1.3.0") Repository JetBrains/koog Repository stats |Metric|Value| |:-|:-| |Stars|4,645| |Forks|502| |Last release|1.3.0 (2026-09-24)| |Languages|Kotlin 97.3%, Java 2.6%, Other 0.1%| |Contributors|150| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|@claude (6 commits), @Copilot (3 commits), @cursoragent (1 commits)| |AI co-authored commits|Claude: 6, Cursor: 1, Copilot: 3| Main contributor |Field|Value| |:-|:-| |Name|Anastasiia Zarechneva| |GitHub|@zarechneva-a| |Commits to this repo|177 (top contributor)| |Bio|QA engineer @ JetBrains| |Company / Location|JetBrains / Germany| |Followers|66| ────────────────────────────── 📌 If you want to browse more repos like this, we've organized them by category and rankings here: bestgithubrepositories.com 🗂️ We're also archiving repos where possible, so useful projects don't disappear when they're removed from GitHub.
An AI agent that asks before it pays: human approval for tool calls in Kotlin Updated Oct 2026 for 0.1.0-alpha12: MCP tools, retries, tracing and a SQLite checkpointer are now in. An agent that only talks is easy to trust. An agent with a refund tool is not. The model decides when to call it and with which amount, and the same question can get a different answer tomorrow. I don't think a better prompt solves that. "Never refund without permission" is a sentence the model usually follows, and "usually" is not good enough for money. What solves it is structure around the model: ordinary code that decides where the model may act alone and where it has to wait. This article builds that in Kotlin, in five steps: an agent, a pause before it acts, a pause that survives a restart, a test without a model, and a rule the model never gets to see. It uses Telar, an open-source library for agent workflows that I am writing. Every example here was compiled and run against the current release, 0.1.0-alpha12. 1. The agent A tool is a function the model may call. Its input is a @Serializable class, and the library builds the schema the model reads from that class. @Serializable data class OrderLookup( @Description("The order number, for example \"1001\"") val orderId: String, ) @Serializable data class RefundRequest( @Description("The order number, for example \"1001\"") val orderId: String, @Description("The amount to pay back, in euros") val euros: Int, ) fun helpDesk(model: ChatModel, pay: suspend (RefundRequest) -> Unit): CompiledGraph<AgentState> { val orderStatus = Tool<OrderLookup>("order_status", "Returns what was ordered, what it cost and when it arrived.") { lookup -> "Order ${lookup.orderId}: one margherita for 9 euros, delivered 70 minutes late." // your database here } val refund = Tool<RefundRequest>("refund", "Pays money back for an order.") { request -> pay(request) "Paid back ${request.euros} euros for order ${request.orderId}." } return toolAgent( model, tools = listOf(orderStatus, refund), system = "You work at the help desk of a pizzeria. Look an order up before you decide anything about it.", ) } toolAgent is the usual loop: the model answers or asks for tools, the tools run, and their results go back to the model, until it has an answer. val model = AnthropicChatModel(HttpClient(), apiKey = System.getenv("ANTHROPIC_API_KEY"), model = "claude-opus-5-5") val agent = helpDesk(model, pay = { payments.refund(it.orderId, it.euros) }) val state = agent.invoke(AgentState("My pizza for order 1001 came 70 minutes late. I want my money back.")).state println(state.answer) This works, and it pays whenever the model thinks it should. That is the part to fix. 2. Pause before the tools run The agent is not a black box. It is a graph of two nodes, model and tools, with an arrow back from one to the other. And a graph can be told to stop before a node: val config = GraphConfig( threadId = "ticket-42", checkpointer = MemoryCheckpointer<AgentState>(), interruptBefore = setOf("tools"), ) Now invoke returns GraphResult.Interrupted each time the model has asked for tools and none of them has run yet. The calls that wait are in the state, with their names and their inputs: suspend fun answer( agent: CompiledGraph<AgentState>, question: String, config: GraphConfig<AgentState>, approve: suspend (List<ToolCall>) -> Boolean, ): AgentState { var result = agent.invoke(AgentState(question), config) while (result is GraphResult.Interrupted) { val refunds = result.state.messages.pendingToolCalls().filter { it.name == "refund" } result = if (refunds.isEmpty() || approve(refunds)) { // Nothing risky, or a person said yes: run the calls. agent.resume(config) } else { // A person said no: answer the call yourself. A call that already has a result is not run. agent.resume(config) { state -> state.copy(messages = state.messages + refunds.map { ChatMessage.ToolResult(it.id, it.name, "A person declined this refund.", isError = true) }) } } } return result.state } Three things happen in that loop: - Lookups pass without a question. The code decides which tools need a person, by name. Nobody is asked to approve reading an order. - A yes runs the call as the model wrote it. approve gets the calls, so your screen can show refund {"orderId":"1001","euros":9} before anyone clicks. - A no goes back to the model as a result. The model reads "A person declined this refund." and writes the customer an answer that fits. The conversation stays whole. The difference from a prompt is where the guarantee lives. The refund function is not reached unless approve returned true. The model cannot talk its way past an if. 3. The pause survives a restart A person may answer in a minute or after lunch, and the server may be deployed in between. Telar saves a checkpoint after every step, so the only change is where it is saved: val config = GraphConfig( threadId = "ticket-42", // One JSON file per thread. AgentState is @Serializable, so nothing else is needed. checkpointer = FileCheckpointer(Path("checkpoints"), KotlinxStateSerializer<AgentState>()), interruptBefore = setOf("tools"), ) After a restart, lastResult reads where a thread stopped without running anything: when (val result = agent.lastResult(config)) { is GraphResult.Interrupted -> showApproval(result.state.messages.pendingToolCalls()) // still waiting for a person is GraphResult.Completed -> showAnswer(result.state.answer) null -> showEmptyForm() // this ticket has never run } agent.resume(config) then continues in the new process, from the step where the old one stopped. One thing to know before you rely on this for payments: a tool call that finished is saved with the step, so resuming after a failure or a restart gives it its saved result instead of paying twice. A call that was cut off in the middle, by a crash during the payment itself, can run again. Give a side effect like a refund an idempotency key, for example made from the order number. There are checkpointers for memory, files, the browser's localStorage, and SQLite through SQLDelight on every platform. Checkpointer is a small interface, so a server database is one class away; PostgreSQL is an open issue if you want to help. 4. Test it without a model ChatModel is an interface with one function, so a model that follows a script is a lambda. This one looks the order up, asks for the refund, and then answers according to what came back: val scripted = ChatModel { request -> val results = request.messages.filterIsInstance<ChatMessage.ToolResult>() ChatResponse( when (results.size) { 0 -> ChatMessage.Assistant(toolCalls = listOf(ToolCall("call-1", "order_status", buildJsonObject { put("orderId", "1001") }))) 1 -> ChatMessage.Assistant(toolCalls = listOf(ToolCall("call-2", "refund", buildJsonObject { put("orderId", "1001"); put("euros", 9) }))) else -> ChatMessage.Assistant(if (results.last().isError) "I am sorry, I cannot refund this order." else "Your 9 euros are on their way.") }, ) } And the property that matters becomes a unit test that runs in milliseconds, with no key and no network: @Test fun aRefundThatAPersonDeclinesIsNotPaid() = runTest { val paid = mutableListOf<RefundRequest>() val agent = helpDesk(scripted, pay = { paid += it }) val config = GraphConfig(checkpointer = MemoryCheckpointer<AgentState>(), interruptBefore = setOf("tools")) val state = answer(agent, "My pizza for order 1001 came 70 minutes late. I want my money back.", config, approve = { false }) assertEquals(emptyList(), paid) assertEquals("I am sorry, I cannot refund this order.", state.answer) } The model is the part you cannot predict. Everything around it is plain Kotlin on an immutable state, and you test it the way you test any other Kotlin. 5. Rules go in code, not in the prompt Some rules have no exceptions. Say orders older than 30 days are never refunded. You could write that into the system prompt and hope. Or the model never sees those orders. toolLoop is the same agent as a part of a graph of your own, with the conversation in a state you define. model, orderStatus and refund are the ones from step 1: data class Complaint( val orderId: String, val text: String, val daysAgo: Int, val messages: List<ChatMessage> = emptyList(), val reply: String = "", ) val desk = StateGraph<Complaint> { val tooOld = node("too_old") { it.copy(reply = "Orders older than 30 days cannot be refunded.") } val send = node("send") { it.copy(reply = it.reply.ifEmpty { it.messages.last().text }) } val agent = toolLoop( model = model, tools = listOf(orderStatus, refund), // Where the conversation is in your state, and how to add messages to it. messages = { it.messages }, append = { complaint, new -> complaint.copy(messages = complaint.messages + new) }, firstMessage = { "Order ${it.orderId}: ${it.text}" }, then = send, ) conditionalEdge(START, targets = setOf(agent, tooOld)) { if (it.daysAgo > 30) tooOld else agent } tooOld then send send then END }.compile() An old order takes the arrow to too_old, and the model is not called at all. No tokens, no risk, and a test that proves it. compile() checks the drawing before anything runs. A node that nothing leads to, or an arrow to a node that does not exist, fails there and not in production at the fourth step of a run. Where this runs All of the above is Kotlin Multiplatform. The core depends only on kotlinx-coroutines, and the same graph runs on the JVM, Android, iOS, desktop, and in the browser as JS or Wasm. The models are Claude and anything with the API of OpenAI (OpenAI, Gemini, Ollama, Groq and others) on Ktor client, and any LangChain4j model on the JVM. To see graphs like these run without writing code, there is a small game in the browser in which every stage is a Telar graph. It needs no account and no API key. What is missing Telar is an early alpha, and I would rather say so here than have you find out: - The API can still change before 1.0. - A message holds text only: no images or files yet. - A long conversation is sent to the model in full every time. - Tracing has spans for runs and nodes, but not yet for each model call and tool call. - The full list is on the roadmap. - If you need a complete agent platform today, Koog from JetBrains is the safer choice. The site has a comparison with Koog, LangGraph4j and LangChain4j. Telar is for the case this article is about: the workflow around the model is the hard part, and you want it to be ordinary, typed, testable Kotlin. Try it // build.gradle.kts. `@Serializable` needs the plugin kotlin("plugin.serialization"). implementation("dev.deeptelar:telar-agent:0.1.0-alpha12") implementation("dev.deeptelar:telar-anthropic:0.1.0-alpha12") // or telar-openai: OpenAI, Gemini, Ollama, ... implementation("dev.deeptelar:telar-checkpoint-file:0.1.0-alpha12") // step 3 implementation("dev.deeptelar:telar-serialization:0.1.0-alpha12") // step 3 implementation("io.ktor:ktor-client-cio:3.6.0") // any Ktor engine - Code: <deeptelar/telar> - Documentation and tutorial: <deeptelar.github.io> - The guides behind this article: Agents with tools and Human-in-the-loop It is early, which makes this the best time for feedback. If something in the API looks wrong to you, open an issue or tell me in the comments.
The JVM Speaks AI: Cobrainer's Journey with Kotlin, Koog and Exposed --- title: "The JVM Speaks AI: Cobrainer's Journey with Kotlin, Koog and Exposed" published: true tags: kotlin, ai, koog, jvm cover_image: dev-posts-0b9b92.gitlab.io --- One afternoon, one of our LLM providers started throttling us. A large customer had triggered a burst of AI-assisted work, and the provider's capacity in our region could not keep up. Our prompt executor followed its fallback chain, the circuit breaker took the affected models out of rotation, and each request moved to a second provider without interrupting the workflow. There was no incident to handle and no change in system behaviour for the customer. We noticed it only the next morning on our LLM observability dashboard. We built all of our AI integration in Kotlin. A controversial decision? We do not think so. In 2025, one could still argue that AI integration required Python. By 2026, that debate is no longer a useful basis for architecture decisions. Python remains important for research and model training, but production AI is mostly about integrating APIs, enforcing contracts, and operating reliable systems. Every prompt, agent, and structured-output contract in our core platform lives on the JVM: one Kotlin monorepo with types shared end to end, built on Koog, Exposed, coroutines, and kotlinx.serialization. TL;DR - Kotlin lets six developers and their coding agents work in one stack. Sealed types, kotlinx.serialization, null safety, and coroutines make contracts and control flow explicit. - Koog was the best fit because its lightweight abstractions build on Kotlin instead of hiding it. We moved the proof of concept into production for all customers in one day. - Production reliability comes from cross-provider fallback chains, circuit breakers, LLMObs, and contract tests. Removing blocking code from coroutine paths also increased throughput by 5×. We at Cobrainer Cobrainer is a Munich-based B2B HR-tech company that grew out of a Technical University of Munich research project in 2013. We serve enterprises across industries and regions. Our AI-native Skills Platform creates and governs living Job and Skill Architectures: structured models of an organization's functional roles, levels, tasks, and skills. They answer three basic questions: Which distinct jobs exist in the organization? What is needed to perform them? How do they relate to each other? A patented Skills Graph and ML/NLP power the platform. Our customers use it for skills-based (AI) workforce transformation: internal mobility, recruiting, learning, workforce planning, and EU pay-transparency readiness, with integrations into SAP SuccessFactors, Workday, ServiceNow and other HCM systems. Our Summer 2026 release includes a context-aware Personal Agent for the Job Architecture and the wider platform, agentic background tasks, agentic (g)RAG, and collaborative multilingual commenting for governance. The AI can resolve structural and language-specific comments on job profiles in more than 25 languages. Our agentic engineering setup centres on six developers with end-to-end ownership, from cloud infrastructure through backend and frontend to AI. Coding agents help us explore the repository, implement changes, run checks, and investigate operational issues. Together, we serve more than 10,000 active users at companies around the world. Small, fast-moving team, big surface. That constraint explains almost every technology decision in this article. Kotlin for AI-powered features Why Kotlin? Reliability meets velocity The control plane and core domain of our platform use Kotlin and Spring Boot. The stack is type-safe and null-safe end to end, with kotlinx.serialization for wire contracts, Exposed for SQL, and coroutines for non-blocking I/O. A central monolith is the entry point for the platform, and everything lives in one monorepo. With six engineers working across the full system, every additional deployable adds work to development, operations, and CI. A large monolith rewards explicitness. Its behaviour can span distant files, and coding agents cannot safely fill gaps left by implicit conventions. Kotlin's types and compiler checks make those relationships visible. Koog follows the same principle with small abstractions that keep contracts, capabilities, and control flow in the code. This helps developers review changes and gives coding agents firmer boundaries in which to work. When our LLM work moved from isolated features to core platform components, we asked whether it justified a separate stack. The models sit behind HTTP APIs, so another runtime would not improve their output. It would give us a second deployment pipeline, add serialization boundaries between the AI layer and domain model, and remove compile-time guarantees at the boundary where LLMs introduce the most uncertainty. We wrote down what we actually needed from an LLM framework: - Lean and explicit. The framework should behave like a library while keeping control flow visible in the code. - A release cadence that matches innovation speed and model churn. New models and paradigms arrive monthly, so infrequent framework releases become a bottleneck. - Clean provider and model-capability abstractions. We need multiple providers, and their models support different features. - Type safety end to end. An LLM's output should enter the domain as a typed value or not at all. We tested these requirements by building working implementations instead of comparing feature matrices. The turning point: why Koog fits Kotlin We built and ran production implementations with Spring AI, LangChain4j, and Koog. The important difference was not the length of their feature lists. It was how clearly each framework expressed its behaviour in Kotlin for developers, coding agents, and production tooling. Spring AI is the most comprehensive and deeply integrated Spring option. It offers a robust ecosystem and good portability for teams already using Spring Boot. We still use it for our MCP endpoints, where its integration with REST infrastructure and the Spring Security filter chain is exactly what we need. In the agentic core, however, it added another layer between our code and the model when we wanted direct control and rapid iteration. LangChain4j is a large, feature-rich library that brings LangChain concepts to the JVM. It helps teams move quickly, but we found failures inside a chain difficult to trace back to the underlying LLM call. We eventually replaced the last utilities that depended on it with a few dozen lines of Kotlin. Koog changed the direction of the evaluation because it made Kotlin itself the foundation of the integration. Our @Serializable domain types become structured-output contracts directly. Sealed hierarchies model the possible results, and exhaustive when expressions force every caller to handle them. Coroutines carry naturally through provider calls, retries, and agent workflows. Koog's DSL builds on familiar Kotlin features such as extension functions and lambdas with receivers, so agent graphs remain normal Kotlin code that can be traced, changed, and verified with the compiler. The provider abstraction was equally important. Koog represents model capabilities explicitly instead of reducing every model to a lowest common denominator. We can adapt temperature, token limits, and structured-output handling for each model while keeping the same domain contract and call site. These decisions are visible in types and code instead of being hidden in configuration or prompts. When a developer or agent investigates a failure, the underlying request remains accessible rather than disappearing behind a chain of abstractions. That combination gave us the level of abstraction we wanted: enough structure to avoid rebuilding provider clients and agent orchestration, but close enough to Kotlin that our existing tools, types, and debugging habits still applied. A coding agent can trace a change from a domain type through its LLM contract to every consumer without crossing languages or repositories. Koog did not introduce a second programming model into the codebase. It made the existing one more useful to both humans and agents. Once we chose Koog, we cleaned up the proof of concept and rolled it out to all customers in one day. That speed came from keeping the domain model, serialization, concurrency model, tests, and deployment pipeline we already understood. AI in production: structured output as the workhorse In production, much of an LLM's value comes from filling data structures that domain logic can trust, rather than generating free-form chat. Koog has powered our Job Architecture generation since summer 2025, and structured output is the foundation of that core product capability. The contract is a Kotlin sealed hierarchy with kotlinx.serialization. Koog turns it into a JSON schema (docstrings included, via @LLMDescription), gets the provider to comply, and hands back a typed value: @Serializable enum class ChangeKind { RENAME, ARCHIVE, MOVE } @LLMDescription("A single proposed change to the job architecture") @Serializable sealed interface ProposedChange { val kind: ChangeKind @Serializable @SerialName("rename") data class Rename( @property:LLMDescription("The item to rename") val itemId: String, @property:LLMDescription("New title, in the tenant's base language") val newTitle: String, ) : ProposedChange { override val kind get() = ChangeKind.RENAME } @Serializable @SerialName("archive") data class Archive( @property:LLMDescription("Items to archive, including their descendants") val itemIds: List<String>, ) : ProposedChange { override val kind get() = ChangeKind.ARCHIVE } // ... } Requesting it is one call. If a weaker model returns almost-JSON, a StructureFixingParser repairs it with a lightweight model before we ever see it: val plan = executor.executeStructured<ChangePlan>( prompt = prompt("change-planner") { system(plannerInstructions) user(request) }, model = model, fixingParser = StructureFixingParser(model = lightweightModel, retries = 2), ).getOrThrow().data plan.changes.forEach { change -> when (change) { // The compiler enforces exhaustive coverage. is ProposedChange.Rename -> preview(change.itemId, change.newTitle) is ProposedChange.Archive -> confirmArchive(change.itemIds) is ProposedChange.Move -> validateMove(change) } } Kotlin provides specific safeguards here. Null safety exposes schema drift at compile time: add a required field to the contract and every construction site that omits it stops compiling. A when expression over a sealed result is exhaustive, so a new change kind breaks the build until every consumer handles it. Default arguments and small DSLs keep configuration readable, allowing us to review prompt code like domain code. We also balance traffic across providers whose models differ in capabilities, and Koog represents those differences explicitly. A provider that uses an existing API format requires little integration code: // xAI uses the OpenAI wire format. @Serializable object XAI : LLMProvider("xai", "xAI") val xaiClient = OpenAILLMClient( apiKey = settings.apiKey, settings = OpenAIClientSettings(baseUrl = "https://api.x.ai"), ) Our executor adds three behaviours to Koog's PromptExecutor: retry with backoff, a config-driven fallback chain, and a per-model circuit breaker. Before each attempt, it adapts the prompt to the target model's declared capabilities: fun adaptPromptFor(model: LLModel, prompt: Prompt): Prompt = prompt .letIf(!model.supports(LLMCapability.Temperature)) { it.withoutTemperature() } .letIf(prompt.maxTokens > model.maxOutputTokens) { it.cappedTo(model.maxOutputTokens) } // Model can't do native structured output? Ask for it in the prompt // instead. The sealed contract and fixing parser stay identical. .letIf(!model.supports(prompt.schemaCapability)) { it.withSchemaAsUserMessage() } The call site never knows which provider answered. That indirection is what made the silent rebalancing from the opening of this article a non-event. Growing into an agentic platform At Cobrainer, AI powers skill inference, Job Architecture generation, transcription, chat agents, and agentic (g)RAG. Our Job Architecture chat agent brings several of these capabilities together. The agent lets users create, review, and maintain a job architecture conversationally: "merge these two levels", "draft the missing task descriptions for this job family", or "archive everything nobody has touched since the reorg". At a high level, it is a supervisor tool loop with deterministic sub-workflows. The LLM decides what to do, while a Koog strategy graph routes every consequential action through code we control. A human must approve every change. The agent is our one exception to the monolith. It runs in the Agent Runtime, a reactive Spring Boot Kotlin service designed for long-running conversations, streaming, and structured concurrency. The monolith remains the single entry point for the platform, including REST and MCP. Both services live in the same monorepo, so they share types directly and the compiler checks their contract. SurrealDB stores checkpoints for resumable turns, message history, and three layers of memory: a small window of recent conversation chunks, a running session briefing, and a fact-based long-term graph. Platform data remains in Postgres behind the monolith, accessed through Exposed. Koog's graph DSL makes the tool loop almost declarative. We use marker tools that the model can call but that never execute directly. A guarded edge intercepts the call and routes it into a deterministic sub-workflow: val toolLoop = strategy<String, String>("supervisor") { val callLlm by nodeLLMRequest() val executeTool by nodeExecuteTool() val sendResult by nodeLLMSendToolResult() val editFlow = installEditCapability(approvalGate) val bulkFlow = installBulkCapability(approvalGate) // Marker tools are visible to the model but never executed. // The edge routes into a deterministic sub-workflow instead. edge((callLlm forwardTo editFlow).soleMarker(EDIT_TOOL_ID)) edge((callLlm forwardTo bulkFlow).soleMarker(BULK_TOOL_ID)) // Edges match in declaration order, so the generic loop comes last. edge(callLlm forwardTo executeTool onToolCalls { true }) edge(callLlm forwardTo nodeFinish onAssistantMessage { true }) edge(executeTool forwardTo sendResult) } The same graph drives the progress UI. Instead of streaming tokens, we show meaningful progress. Koog's event hooks fire as the agent moves through nodes and tools, and each transition maps to a localized status such as "Analyzing your architecture…" or "Preparing changes for review…". Showing graph progress improved perceived latency more than token streaming would have for this workflow: install(EventHandler) { onNodeExecutionStarting { ctx -> emitStatus(ctx.node.name) } onToolCallStarting { ctx -> emitToolStatus(ctx.toolName) } onAgentExecutionFailed { ctx -> events.close(ctx.error) } } Security is where the two frameworks meet. The agent accesses tools only through central MCP servers, which Spring AI serves inside the monolith. Every tool call therefore passes through the same Spring Security filter chain as our REST APIs. We run one MCP server per permission scope (read, architect, reviewer), guard each at the path level, and give the agent a short-lived delegated token so it always acts as the user. Permissions filter the available tools before any LLM call. If a user cannot edit the architecture, their agent never sees editing tools. This reuses our existing authorization infrastructure without relying on prompts for access control. Koog runs the agent in the Agent Runtime, Spring AI exposes and protects its tools in the monolith, Exposed and Postgres store platform data, and SurrealDB stores agent memory. The result is two JVM services in one monorepo, sharing one type system. !High-level architecture of the Job Architecture agent Discoveries and challenges Production use also exposed several important problems and lessons. A 5× throughput gain from coroutine cleanup. Our most difficult performance problem had nothing to do with Koog or LLMs. Legacy blocking code inside coroutines starved dispatchers under AI-driven load, causing vague and load-dependent slowdowns. Once we found and removed it, throughput on the affected paths increased by 5×. Which leads to the sharpest lesson we can offer: > Do not leave runBlocking in production request paths. This applies even to a blocking Spring Boot servlet stack where it appears to work. Under load, it can deadlock a dispatcher without crashing the application; requests simply stop completing. Build deep fallback chains across providers. Several fallback models from the same provider still share one failure domain. Our chains cross provider boundaries, and per-model circuit breakers skip throttled or unhealthy routes so retries reach a viable model quickly. We trace every attempt and fallback in LLMObs, including latency, token usage, and errors. That makes it possible to reconstruct a request across providers and distinguish a model failure from a parser or application problem. This setup paid off as soon as one provider lost capacity. Contract-test the LLM layer. Structured output depends on our schema, the framework's schema generation, and each provider's implementation. We run end-to-end structured-output tests with real calls and parsing for every provider, as well as LLM contract tests in CI. Together with LLMObs, these checks form our AI operations loop: observe production behaviour, investigate failures with full context, and validate model or provider changes before rollout. This lets developers and agents move quickly without turning model churn into guesswork. Final words Building AI features in Kotlin once sounded unconventional. Today, its explicit contracts and lightweight abstractions give our six developers and their coding agents a system that is fast to change, safe to refactor, and straightforward to operate. If you are considering AI development on the JVM, the ecosystem is ready. Have a look at Koog, Exposed, and Kotlin. We thank the JetBrains teams behind all three for their frequent releases and responsiveness. Questions, disagreements, and production stories are welcome in the comments. We read them.
I built Telar, an open-source Kotlin Multiplatform library for AI agent workflows as typed graphs (early alpha, feedback wanted) I wanted to build agent workflows in Kotlin the way LangGraph does it in Python: as a graph where each step is visible and you decide what runs next. I also wanted it to run on small devices and in the browser, not only on a server, so I wrote my own. It is called Telar, it is Apache 2.0, and it is an early alpha. The idea in one example The state is your own data class, a node is a suspend function that returns a copy of it, and arrows connect the nodes. This workflow stops before money moves and continues when a person says yes: data class Refund(val orderId: String, val amount: Int, val approved: Boolean = false, val status: String = "new") val refunds = StateGraph<Refund> { val check = node("check") { it.copy(status = if (it.amount > 1000) "too large" else "checked") } val pay = node("pay") { it.copy(status = if (it.approved) "paid" else "rejected") } val decline = node("decline") { it.copy(status = "declined") } START then check conditionalEdge(check, targets = setOf(pay, decline)) { if (it.status == "checked") pay else decline } pay then END decline then END }.compile() // a node nothing leads to, or a target that is not declared, fails here suspend fun main() { val config = GraphConfig(threadId = "order-1001", checkpointer = MemoryCheckpointer<Refund>(), interruptBefore = setOf("pay")) // runs "check", then waits before "pay" refunds.invoke(Refund("1001", amount = 250), config) // a person said yes val result = refunds.resume(config) { it.copy(approved = true) } // paid println(result.state.status) } That part needs only telar-core, which depends on kotlinx-coroutines and nothing else. What it does today - Graphs: loops, parallel nodes, subgraphs, and a stream of events while a run is going - Save points: a checkpoint after every step, in memory, in a file, or in the browser's localStorage. A run can pause, survive a restart, and be forked from an earlier step - Agents: a ready-made tool-calling agent (toolAgent). You give it a model and a list of tools, and a tool's input is a @Serializable class that becomes the JSON schema - Models: Claude, and anything with the OpenAI API (OpenAI, Gemini, Ollama, Groq and so on) on Ktor client, plus an adapter for LangChain4j models on the JVM - Targets: JVM, Android, iOS, macOS, Linux, Windows, JS and Wasm What it is not Stable, or complete. There is no MCP, no tracing, no retry policy and no database checkpointer yet, and the API can still change before 1.0. If you need a finished agent platform today, Koog is the safer choice, and LangGraph4j and LangChain4j are both at 1.x on the JVM. Telar is for the case where the workflow itself is the hard part and you want its state to be ordinary Kotlin. There is a comparison table on the site, and I would like to hear where it is wrong. Try it To see graphs run without writing code or bringing an API key, there is a small game in the browser, written in Compose Multiplatform for Wasm, where every stage is a Telar graph. - Code: deeptelar/telar - Docs and tutorial: deeptelar.github.io - Demo: deeptelar.github.io The feedback that would help me most right now is about the API: the graph DSL, how state is merged when nodes run in parallel, and what you would need before trying it in a real project.