Two rankings of the same four agent frameworks disagree, and that gap is where most framework decisions go wrong. GitHub stars pushed LangGraph past CrewAI through early 2026. The tool-use benchmarks reorder the list the moment you run the models on hardware you own.
I built this comparison the slow way. Four open-source frameworks, five local models served through Ollama, three standard tool-use benchmarks, and two cloud baselines for calibration. Cost figures trace back to published IBM Research, Gartner, McKinsey, and Deloitte reporting rather than vendor marketing. Every chart on this page is interactive and reads from that dataset. The star-growth chart tracks all four repositories from 2024 through Q1 2026, the radar chart scores each framework across eight dimensions of real-world utility, and the success-rate chart plots 500 standardized tasks per category against model sizes from 1B to 70B with GPT-4o as the ceiling.
The stakes here are not academic. Most agent deployments funnel data through cloud APIs. Every prompt, every tool call, every reasoning trace flowing through third-party servers. For organizations processing sensitive data or proprietary business logic, that architecture creates unacceptable exposure under GDPR, CCPA, and sector-specific compliance frameworks. Running the whole stack locally erases that exposure and narrows the question to something you can actually measure. Are local models good enough yet, and which framework extracts the most from them?
Subscribe to the newsletter for future AI engineering deep dives.
The short answer
The AI agent market is growing at a 35% CAGR. Tractica projects the conversational AI platform market will exceed $9 billion by 2025. Gartner reports over 75% of large enterprises plan to deploy AI agents within the next two years. And up to 80% of customer interactions in retail will route through AI agents by 2026. Those forecasts explain the funding. They say nothing about which framework survives contact with a local model.
The measured picture splits four ways. LangGraph overtook CrewAI in GitHub stars during Q1 2026 and scores highest on multi-agent orchestration and production readiness, paid for with roughly three times the code a simple ReAct agent needs elsewhere. Smolagents holds the tightest local model integration because HuggingFace built it against its own pipelines with no adapter layer. CrewAI trades explicit control for speed of assembly. AutoGen keeps a large install base from Microsoft's early multi-agent research push.
On models, Qwen 2.5 32B reaches 82.6% on BFCL v3 and beats Mistral Large 2 while running entirely on local hardware. Qwen 3.5 7B hits 71.2% on the same benchmark and runs at 45 tokens per second on an Apple M4, which covers single-tool agents. Simple tool calls turn useful at 7B with a 78% success rate. Multi-step ReAct needs 14B or more to clear 65%. Multi-agent pipelines need 32B or more for the coordinating agent. Every number behind those claims sits in the charts and tables below, alongside the code that produced them.
The Agent Framework Landscape
Four frameworks dominate the open-source agent ecosystem in Q1 2026. Each takes a fundamentally different approach to building autonomous workflows, and that architectural choice determines what you can build with it.
LangGraph surpassed CrewAI in GitHub stars during early 2026, driven by enterprise adoption and its graph-based architecture that maps cleanly to production requirements like audit trails and rollback points. CrewAI still grows steadily, favored by teams that want fast iteration without learning graph theory. AutoGen maintains a large install base from Microsoft's early push into multi-agent research. Smolagents, the newest entrant from HuggingFace (which has crossed 30 million model downloads), shows the steepest relative growth because it fills a gap the others don't. It writes and executes Python code as its primary action mechanism rather than calling predefined tool functions.
Framework Architecture Breakdown
| Framework | Org | Version | Stars | Approach | Local LLM | Multi-Agent | Learning Curve |
|---|---|---|---|---|---|---|---|
| LangGraph | LangChain | 0.3.x | 39.2K | Graph-based state machines | Full (via ChatOllama) | Native graph nodes | Steep |
| CrewAI | CrewAI Inc. | 0.80+ | 33.8K | Role-based agent crews | Via Ollama adapter | Native crew system | Easy |
| AutoGen | Microsoft | 0.4.x | 33.0K | Conversational agents | Via OpenAI-compat API | Group chat pattern | Moderate |
| Smolagents | HuggingFace | 1.x | 16.8K | Code-first, minimal abstraction | Native HF integration | Basic delegation | Easy |
The table distills months of testing into the metrics that matter for framework selection. Version numbers reflect the state of active development. "Local LLM Support" distinguishes native integration from requiring an adapter layer, which adds latency and failure points.
Capability Analysis
Raw star counts don't tell you which framework to pick for a specific project. The radar chart below maps each framework across eight dimensions that determine real-world utility.
LangGraph scores highest on multi-agent orchestration and production-readiness because its graph abstraction enforces explicit state management. You define nodes (agents, tools, checkpoints), edges (transitions, conditions), and the framework handles execution, persistence, and replay. The tradeoff is complexity. A simple ReAct agent takes 40 lines in Smolagents and 120 in LangGraph.
CrewAI inverts that tradeoff. Define a "crew" of agents with roles, goals, and backstory text. The framework infers coordination patterns. This works remarkably well for standard workflows but becomes opaque when you need to debug a failure in a five-agent pipeline.
Smolagents leads in local LLM support because it was built by HuggingFace for their own model ecosystem. No adapter needed. Point it at a local model, define tools as Python functions, and it generates executable code instead of JSON tool calls. This code-first approach produces more reliable outputs from smaller models because code generation is a stronger capability in most LLMs than structured JSON output.
Tool-Use Benchmarks: Local Models vs Cloud
The defining capability of an agent is reliable tool use. An LLM that can't consistently generate correct function calls, parse responses, and decide what to do next isn't an agent. It's an autocomplete engine with extra steps.
I ran three standard benchmarks across five local models and two cloud baselines. The test matters because by 2026, up to 80% of customer interactions in retail will run through AI agents (Gartner), and those agents need to call APIs reliably.
The data reveals two critical thresholds. Below 7B parameters, tool-use accuracy falls off a cliff. Models can't reliably follow the function-calling format. Above 32B parameters, local models achieve 80%+ accuracy across all three benchmarks, closing to within 8-10 percentage points of GPT-4o and Claude 3.5.
Qwen 2.5 32B stands out. At 82.6% on BFCL v3, it outperforms Mistral Large 2 on the most rigorous benchmark while running entirely on local hardware. The practical implication is clear. You no longer need cloud APIs for production-grade tool use if you can run a 32B model.
The 7B Sweet Spot
For teams that can't dedicate 40+ GB of RAM to a single model, the 7B tier deserves attention. Qwen 3.5 7B hits 71.2% on BFCL v3, enough for single-tool agents that call well-defined APIs. It runs at 45 tokens per second on Apple M4, fast enough for interactive applications.
# Example: Qwen 3.5 7B agent with Smolagents
from smolagents import CodeAgent, OllamaModel, tool
model = OllamaModel(model_id="qwen3.5:latest")
@tool
def search_docs(query: str) -> str:
"""Search the internal documentation index."""
# Your retrieval logic here
return retrieve_relevant_docs(query)
@tool
def create_ticket(title: str, priority: str) -> str:
"""Create a support ticket in the system."""
return create_jira_ticket(title, priority)
agent = CodeAgent(
tools=[search_docs, create_ticket],
model=model,
max_steps=5,
)
result = agent.run("Find docs about auth failures and create a P2 ticket")Agent Architecture Patterns
Framework choice matters less than architecture choice. A well-designed ReAct loop in Smolagents will outperform a poorly-structured graph in LangGraph. The table below maps the six dominant patterns to their ideal use cases and minimum model requirements.
| Pattern | Description | Best For | Framework | Min Local Model |
|---|---|---|---|---|
| ReAct (Reason + Act) | LLM reasons about the task, selects a tool, observes the result, and repeats | General-purpose agents, Q&A with tools | All frameworks | Qwen 3.5 7B+ |
| Plan-and-Execute | Creates full plan first, then executes steps sequentially with replanning on failure | Complex multi-step tasks, research workflows | LangGraph, AutoGen | Qwen 2.5 32B+ |
| Multi-Agent Supervisor | Supervisor agent delegates to specialized worker agents and aggregates results | Enterprise workflows, diverse tool sets | LangGraph, CrewAI | 32B+ for supervisor |
| Code-Writing Agent | Agent writes and executes Python code rather than calling predefined tools | Data analysis, dynamic problem-solving | Smolagents, AutoGen | DeepSeek-Coder 32B |
| RAG Agent | Retrieves context from vector store before generating, with self-reflection on relevance | Knowledge bases, document Q&A | LangGraph, Smolagents | Any 7B+ with embedding model |
| Human-in-the-Loop | Agent pauses at decision points for human approval before executing actions | High-stakes decisions, financial ops | LangGraph, AutoGen | Any supported model |
ReAct: The Universal Starting Point
Every agent framework implements some version of ReAct (Reason, Act, Observe). The LLM receives a task, thinks about what tool to call, calls it, observes the result, and decides whether to continue or return an answer. This loop handles 80% of real-world agent use cases.
# LangGraph ReAct agent with Ollama
from langchain_ollama import ChatOllama
from langgraph.prebuilt import create_react_agent
llm = ChatOllama(model="qwen2.5:32b", temperature=0)
tools = [search_tool, calculator_tool, email_tool]
agent = create_react_agent(
model=llm,
tools=tools,
prompt="You are a research assistant. Use tools to answer questions accurately.",
)
result = agent.invoke({
"messages": [("user", "What was NVIDIA's revenue last quarter?")]
})Plan-and-Execute: For Complex Multi-Step Tasks
When a task requires more than 3-4 tool calls, ReAct loops tend to lose coherence. The model forgets earlier observations or repeats the same action. Plan-and-Execute solves this by separating planning from execution. A planning LLM creates a full step-by-step plan. An executor LLM follows the plan step by step, reporting results back to the planner for potential re-planning.
This pattern demands a stronger model (32B minimum for the planner) but produces significantly more reliable outputs on complex tasks like research reports, data analysis pipelines, and multi-system integrations.
Multi-Agent Supervisor: Enterprise Scale
The supervisor pattern assigns specialized agents to specific domains. A routing agent receives the user request, determines which specialist should handle it, delegates the work, and aggregates results. This maps naturally to enterprise organizations where different teams own different systems.
# CrewAI multi-agent crew
from crewai import Agent, Task, Crew
researcher = Agent(
role="Research Analyst",
goal="Find and verify data from multiple sources",
llm="ollama/qwen2.5:32b",
)
writer = Agent(
role="Technical Writer",
goal="Transform research into clear, structured content",
llm="ollama/qwen3.5:latest",
)
research_task = Task(
description="Research the latest AI agent framework benchmarks",
agent=researcher,
)
writing_task = Task(
description="Write a technical summary of the research findings",
agent=writer,
context=[research_task],
)
crew = Crew(agents=[researcher, writer], tasks=[research_task, writing_task])
result = crew.kickoff()Success Rates by Model Size
The relationship between model parameters and agent task success follows a sigmoid curve, not a linear one. There's a critical mass of capability needed for each type of agentic behavior, and below that threshold, adding parameters doesn't help much.
The data tells a specific story for each task category.
Simple tool calls reach useful reliability (78%) at 7B parameters. This covers most chatbot-with-tools scenarios. A customer support agent that looks up order status, checks inventory, or searches a knowledge base works fine at 7B.
Multi-step ReAct requires 14B+ to cross 65% reliability. Below that, the model loses track of the observation-action-reasoning chain after 2-3 iterations. At 32B, you get 79%, which is production-viable for internal tools where occasional failures are acceptable.
Multi-agent pipelines demand 32B+ for the coordinating agent. Worker agents can run smaller models because they handle focused, single-domain tasks. A 70B supervisor with 7B workers produces better results than four 32B agents of equal capability because the planning bottleneck sits at the coordinator level.
Autonomous research presents the hardest challenge. Even GPT-4o only hits 80% on standardized research tasks. Local models at 70B reach 68%. This gap narrows with better prompting and structured output constraints, but truly open-ended research still benefits from frontier model capabilities.
Cost Analysis: Local Agents vs Cloud Deployment
The economics of agentic workflows split into two categories. Initial deployment costs for autonomous agent systems run $50,000 to $100,000 for AI framework setup and training data preparation (IBM Research, 2023), compared to $500,000 to $1 million for custom traditional workflow systems (Gartner). A mid-range GPU setup costs $5,000 to $10,000 (NVIDIA), while cloud-based managed AI services start at $10,000 to $20,000 for small-scale deployment (AWS).
Ongoing costs tell the real story. Traditional workflow maintenance runs $50,000 to $100,000 annually. Agent-based systems cost $30,000 to $60,000 per year for continuous model training and retraining (McKinsey). A company that previously required 10 human operators for a workflow can save $250,000 annually by switching to autonomous agents (Deloitte). Cloud scaling costs approximately $5,000 to $10,000 per month. Running the same workloads locally on owned hardware eliminates that recurring expense entirely.
Building a Local Agent Stack
Here's the practical setup for running agent workflows entirely on local hardware.
1. Install Ollama and Pull Models
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull models - start with the 7B for fast iteration
ollama pull qwen3.5:latest
# Pull the 32B for production agent tasks
ollama pull qwen2.5:32b2. Choose Your Framework
For first-time agent builders, start with Smolagents. Its code-first approach produces intuitive results, and the HuggingFace integration means zero configuration for local models.
pip install smolagentsFor production systems that need checkpointing, human-in-the-loop, and audit trails, use LangGraph.
pip install langgraph langchain-ollamaFor rapid prototyping of multi-agent teams where development speed outweighs fine-grained control, use CrewAI.
pip install crewai3. Start with ReAct, Graduate to Plan-and-Execute
Every agent project should begin with the simplest architecture that could work. Build a single ReAct agent with 1-3 tools. Validate that your local model handles the tool calling format reliably. Then add complexity only when the simple approach demonstrably fails.
The most common mistake in agent development is over-engineering the orchestration layer before validating that the underlying model can handle the task at all.
4. Monitor and Debug
Agent failures are harder to debug than traditional software because the failure mode is often "the model made a bad decision" rather than a clear exception. All four frameworks provide some form of trace logging. Use it.
# LangGraph: stream events for debugging
async for event in agent.astream_events(
{"messages": [("user", "Analyze Q1 revenue")]},
version="v2"
):
if event["event"] == "on_tool_start":
print(f"Calling tool: {event['name']}")
elif event["event"] == "on_tool_end":
print(f"Tool result: {event['data']}")RAG Agents: The Practical Starting Point
Retrieval-Augmented Generation is the most common first agent project, and the market data confirms why. The global RAG pipeline market is growing at a 45% CAGR, with enterprise data volumes hitting 15 terabytes per month processed through RAG systems by 2026 (up from 3 TB in 2021). Over 90% of RAG pipelines now integrate with at least one external API. Healthcare accounts for 18% of deployments, finance 35%, and tech/IT services lead with a 60% CAGR in adoption.
A RAG agent adds two capabilities beyond basic retrieval. First, it decides whether to search at all, skipping retrieval for questions it can answer from its training data. Second, it evaluates the relevance of retrieved documents and can reformulate the query if initial results are poor.
# Smolagents RAG agent with local embedding
from smolagents import CodeAgent, OllamaModel, tool
import chromadb
client = chromadb.PersistentClient(path="./vector_store")
collection = client.get_collection("company_docs")
model = OllamaModel(model_id="qwen3.5:latest")
@tool
def search_knowledge_base(query: str) -> str:
"""Search the company knowledge base for relevant documentation."""
results = collection.query(query_texts=[query], n_results=5)
return "\n---\n".join(results["documents"][0])
agent = CodeAgent(
tools=[search_knowledge_base],
model=model,
max_steps=3,
system_prompt="Answer questions using the knowledge base. If the search results don't contain the answer, say so clearly.",
)The 7B model handles RAG well because the retrieval step constrains the output. The model doesn't need to recall facts from training data. It needs to read provided context and synthesize an answer, which is closer to reading comprehension than knowledge recall.
What Comes Next
The agent framework landscape will consolidate in 2026. Right now, four major frameworks serve overlapping use cases. By year-end, expect clearer specialization. LangGraph is positioned to own the production/enterprise tier. Smolagents will likely dominate the HuggingFace ecosystem and research community. CrewAI and AutoGen will compete for the accessible middle ground.
McKinsey estimates that up to 30% of jobs will be partially or fully automated through AI agents by 2026. That creates enormous demand for skilled professionals who can build, deploy, and monitor these systems. The market for conversational AI platforms alone will exceed $9 billion (Tractica), and over 50% of hospitals will adopt AI-driven diagnostic tools (IBM Watson Health projections).
The more important trend is model capability. Every six months, the minimum model size needed for reliable agentic behavior drops. Tasks that required 70B parameters in early 2025 work at 32B in early 2026. By late 2026, 14B models may handle multi-step ReAct with 75%+ reliability.
That trajectory means local agent deployment moves from "possible for enthusiasts" to "default for privacy-conscious organizations" within this calendar year. The frameworks are ready. The models are ready. The remaining gap is operational maturity, specifically the monitoring, debugging, and failover patterns that match the standards teams expect from traditional software infrastructure.
Subscribe for updates on AI agent engineering, local LLM benchmarks, and production deployment patterns.