Built in Go · Wired through Kafka · License the whole way down
Scalytics · KafClaw · Apache 2.0
Multi-agent runtime on Apache Kafka. Self-hosted. Apache 2.0.
The agent runtime that produces and consumes from any Kafka cluster you already run. Confluent, Redpanda, MSK, Apache Kafka, KafScale. Typed JSON envelopes, shared memory, distributed skill routing. Onboard new agents in production without restarting another component.
agent_a
Go · planner
agent_b
Python · retriever
agent_c
Rust · tool
Slack
channel
Telegram
channel
Web UI
channel
JSON
Envelope
any
Language
none
Lock-in
Apache 2.0
License
Built in Go · Wired through Kafka · License the whole way down
What KafClaw commits to that comparable runtimes don't
KafClaw is a small set of architectural decisions that make multi-agent systems usable for engineering teams who own the consequences of their infrastructure choices. These commitments hold across every release, every wire envelope, and every deployment mode. They are documented in the source, enforced in CI, and visible at the protocol layer.
Agents communicate through typed JSON envelopes on Kafka topics, not through in-process function calls or a proprietary message bus. Every interaction has an offset, every decision has a trace, every audit row already lives inside your retention policy. The cluster you already monitor is the cluster that runs the agents.
The envelope is JSON. The schema is documented. A Python script with kafka-python participates as a first-class member of the group. So does a Rust worker, a Node service, or a shell pipeline running kcat. Agents are not bound to a single language, runtime, or LLM vendor.
Knowledge published to the group memory topic is indexed locally by every subscribing agent. There is no central RAG service to scale, no vector database to operate, no shared cache that fails the whole fleet. Each agent owns its retrieval surface, populated by group events.
OpenAI, OpenRouter, on-prem vLLM through Scalytics Copilot, anything that speaks the OpenAI chat protocol. The LLM provider is an interface, not a foundation. Switching the model behind an agent does not require rewriting tools, prompts, or coordination logic.
A correlation ID and a trace span flow through every envelope. A decision made by Agent B answering Agent A using a memory item written by Agent C is reconstructable in one query. The local timeline DB writes synchronously, the audit topic writes to Kafka, and both stay in lockstep.
Run a single agent on a laptop with no Kafka at all. Add Kafka and the agent joins the group. Add the orchestrator and you get hierarchies, zones, and discovery. Each step is opt-in. The license is the same one Apache Kafka itself ships under.
Each commitment is a red line in the build. They are documented in the source, enforced in tests, and visible at the wire. A buyer who wants to verify them does not need a sales call. The repository is at github.com/KafClaw/KafClaw.
The system
KafClaw runs as a Go binary that produces and consumes typed JSON envelopes on a structured Kafka topic hierarchy. Heterogeneous agents (Go, Python, Rust, shell scripts using kcat) participate through the same wire format. The orchestrator handles hierarchy, zones, and discovery. The local timeline DB handles trace spans, sessions, and per-agent retrieval.
Agents
Any language
Go, Python, Rust, kcat
JSON envelope schema
no SDK required
→ produce
Topics
group.<name>.*
announce · requests
responses · traces
memory · audit
→ route
Orchestrator
Zones & hierarchy
discovery roster
parent-child delegation
public · shared · private
→ govern
Runtime
Per-agent state
SQLite timeline
tool registry sandbox
vector index, policy engine
This is a wire protocol, not a pub-sub wrapper. Every envelope carries a correlation ID, a trace span, and a timestamp. Reconstruction of any agent decision is one query against the audit topic. Onboarding a new agent in production requires no downtime, no central registry update, no orchestrator restart. The roster topic converges, the agent comes online, traffic flows.
LOCAL DESKTOP
For developers building, testing, or running a personal assistant on one machine. Standalone mode keeps the timeline DB, tool registry, and channel bridges. Add Kafka later and the same binary joins a group.
LOCAL · KAFKA
For teams running a small set of cooperating agents on a shared Kafka cluster. Group mode adds typed envelopes, shared memory, skill routing, and distributed tracing. The orchestrator stays optional.
REMOTE · HEADLESS
For platform teams running KafClaw as a managed service inside their boundary. Headless mode binds the gateway to the network, requires a bearer token, and exposes the API for orchestrator-driven workloads.
Where it sits in your stack
KafClaw connects to any system that speaks the Kafka protocol and operates directly on your existing topics as a producer and consumer. Your brokers continue to handle transport and durability. ACLs, schema registry, and audit pipelines remain unchanged. KafClaw introduces a runtime that reads from those topics, processes events, and writes results back into the same streams. No new infrastructure. No changes to how your platform is operated.
YOUR PLATFORM
Confluent · Redpanda
MSK · Apache Kafka
untouched
existing ACLs preserved
existing audit preserved
→ produce / consume
NEW NAMESPACE
group.<name>.*
new topics under existing
topic policy
standard Kafka ACLs apply
→ subscribe
KAFCLAW
Agent runtime
Go binary on K8s
standard Kafka client
no managed control plane
For teams running Confluent today, KafClaw is additive. It produces and consumes through the standard client surface, including SASL/SCRAM, mTLS, schema registry, and the rest of the Confluent Cloud configuration. The agent topics are new namespaces under your existing topic policy.
For teams running self-hosted Apache Kafka, KafClaw is a Go binary that respects the topic conventions you already publish to. The orchestrator runs alongside your existing brokers. The timeline DB stays local to each agent. No additional broker dependency.
The wire protocol is the wire protocol. Whatever cluster speaks Kafka, KafClaw connects to it. Redpanda, Amazon MSK, IBM Event Streams, on-prem Apache Kafka. The same binary, the same envelope schemas, the same operator surface.
For teams adopting KafScale for backup, disaster recovery, or AI workload isolation from production Confluent, KafClaw treats it the same as any other broker. The agent runtime is neutral about where the broker lives.
Scalytics is a Confluent partner. KafClaw is positioned as the agent runtime that runs on top of any Kafka platform, including Confluent and Confluent Cloud. The deployment pattern is additive, not adversarial. The repository is at github.com/KafClaw/KafClaw.
Honest comparison · May 2026
Most agent frameworks coordinate inside one process or one machine. That works for a notebook. It does not survive the operational requirements of production: multi-host, multi-language, audit, retry, replay, and policy enforcement at the boundary. KafClaw is for the multi-host case. For a single-process graph, the frameworks below are perfectly good and KafClaw can sit underneath them.
| Framework | License | Coordination | Multi-host | Audit at the wire | Best for |
|---|---|---|---|---|---|
| KafClaw | Apache 2.0 | Apache Kafka topics | Yes, native | Yes, every envelope | Multi-agent production, audit, replay |
| LangGraph | MIT | In-process graph | Application-level | Application-level | Single-process workflow graphs |
| AutoGen | CC-BY 4.0 / MIT | Python in-process | Application-level | Application-level | Research, agent debate, .NET |
| CrewAI | MIT | Python in-process | Application-level | Application-level | Role-based crews, single host |
| OpenAI Assistants | Proprietary | Vendor API | Vendor-hosted | Vendor logs | OpenAI-only managed agents |
| Google ADK | Apache 2.0 | In-process or Vertex | Application-level | Application-level | Vertex AI ecosystem |
LangGraph, AutoGen, CrewAI, and Google ADK are excellent for in-process orchestration. They sit one layer below KafClaw: a Python agent built with LangGraph can publish a response envelope to a KafClaw group topic and become part of a multi-host system without changing its internal graph. The frameworks compose; the choice is not exclusive.
Architecture commitments
A buyer running multi-agent workloads for an audited business process reads architectural commitments more carefully than they read benchmarks. Each line below is enforced in CI, documented in the source, and visible at the protocol layer.
Deployment posture
KafClaw is designed for environments where a vendor cloud dependency is a dealbreaker. No external service calls in the runtime path. No telemetry back to the maintainers. The deployment unit is a Go binary on the substrate you already operate. Sovereignty and network boundaries stay your decision.
Single Go binary, around 30 MB. Linux, macOS, Raspberry Pi (arm64). Standalone mode runs on a developer laptop with ./kafclaw agent -m "hello", no Kafka required.
Helm chart for Kubernetes. Docker Compose files (docker-compose.yml, docker-compose.group.yml) for local development. systemd service files for bare metal. Electron desktop build for personal-assistant deployments.
SQLite locally for the timeline DB, sessions, and the per-agent vector index. Kafka topics for transport. S3-compatible storage for shared-memory artifacts that exceed the Kafka message size limit.
local for single-agent desktop work. local-kafka for connected group collaboration. remote for headless server deployments behind a bearer token. Configuration is environment variables. The same binary runs all three.
Operates against any reachable Kafka cluster. No Scalytics phone-home. LLM endpoints can be on-prem (vLLM, Ollama, Scalytics Copilot). Container images mirror to your registry. The runtime never reaches out.
Built-in kshark command for Kafka connectivity, topic inspection, partition layout, and group membership. Apache 2.0 source on GitHub: inspect, fork, audit before purchase. There is no enterprise-only branch.
Procurement questions
The questions platform leads, security architects, and CTOs raise in every early call. Answered here so the briefing time can go to your actual problem.
No. KafClaw produces and consumes from any cluster that speaks the Apache Kafka protocol, including Confluent, Confluent Cloud, Redpanda, Amazon MSK, and self-hosted Apache Kafka. Treat the agent topics as new namespaces under your existing topic policy.
As a standard Kafka client. SASL/SCRAM, mTLS, schema registry, and the rest of the Confluent surface are configured through the standard environment variables. The agent topics live alongside your existing topics under your existing ACL model. Scalytics is a Confluent partner; this is the supported coexistence pattern.
No. The provider layer is an interface. OpenAI, OpenRouter, and OpenAI-compatible self-hosted endpoints are supported out of the box. Scalytics Copilot covers the on-prem case for organizations that need reasoning to stay inside the firewall.
Two layers. At the wire, Kafka ACLs apply to the agent topic hierarchy exactly as they do to any other topics. At the application, the orchestrator supports zones with public, shared, and private visibility, isolating groups unless a zone bridge is configured.
An onboarding protocol on a dedicated control topic. Optional challenge and response handshake gates membership. Roster management uses heartbeat-based liveness, so dead agents drop out without operator intervention. Live agents come online without a restart of any other component.
Those frameworks coordinate agents inside one process. KafClaw coordinates agents across processes, machines, and languages using Kafka as the wire. They compose: a LangGraph agent in Python can publish a response envelope to a KafClaw group topic and become part of a multi-host system without changing its internal graph.
Yes. The repository is open at github.com/KafClaw/KafClaw under Apache 2.0. Envelope schemas, the topic protocol, and configuration keys are all documented at kafclaw.scalytics.io. There is no enterprise-only branch.
Architecture review for multi-agent platforms, hardening for regulated environments, integration with Apache Wayang for federated execution, integration with Scalytics Copilot for private LLM hosting, and 24/7 enterprise support contracts. The KafClaw repository remains Apache 2.0 and the project's roadmap is independent of any specific support engagement.
Where this is in the build
The technical buyer reads this section more carefully than the case studies. A platform that admits what is shipped and what is in flight is one the buyer can plan against. A platform that claims everything is ready is one the buyer assumes is hiding the integration cost.
SHIPPED
Local agent loop with LLM provider abstraction. Group mode over Kafka with typed envelopes and correlation IDs. Orchestrator with zones, hierarchy, and discovery. Tool registry with sandboxed filesystem, shell, memory, and web tools. Policy engine for classification, token quotas, and rate limits. Scheduler, timeline DB on SQLite, per-agent vector index. KShark Kafka diagnostics built into the binary. Helm chart, Docker Compose, Electron desktop build.
IN FLIGHT
Slack and Microsoft Teams bridges are landing in the v2 release line. Enterprise auth bridge for OIDC and SAML is in active development. Native MCP server adapter inside the tool registry, schema registry integration for envelope evolution, and a versioned skill marketplace are on the roadmap and tracked in the GitHub project.
OPEN
We are working with platform teams running multi-agent workloads on top of Confluent, Apache Kafka, or KafScale who need typed wire envelopes, audited memory, and language-neutral coordination. If your gap is the gap described on this page, request a briefing.
Who builds this
KafClaw is developed and maintained by Scalytics, the company founded by the original inventors of Apache Wayang, now an Apache Top-Level Project for federated data processing. The same architectural philosophy, compute where the data lives on standards you can audit, runs through every part of KafClaw.
The founders served in executive and senior engineering roles at companies whose data and AI problems shaped KafClaw's design priorities. The CTO previously served as Chief Digital Architect at E.ON, one of Europe's largest energy infrastructure operators. The wider team includes Apache committers and former engineers from Confluent, Cloudera, and the regulated enterprise environments listed below. Scalytics is an active Confluent partner. KafClaw is positioned as the agent runtime that runs on top of any Kafka platform: Confluent, Confluent Cloud, Redpanda, MSK, Apache Kafka, and KafScale.
Founder and team work history: Allianz · E.ON · Cloudera · Scout24 · Confluent · McKinsey
45 minutes. The wire envelopes, your Kafka topology, the integration path for your existing agent code. No slides, no discovery questionnaire. An architect on the call. Or read the source first.
▸ Book a briefing ▸ Read the source ▸ Explore docs at kafclaw.scalytics.io