Full HTML export (tables preserved)

From Theory to Agent Team: Building a CrewAI-Powered Solidity Security Auditor

2026-04-13

In the previous article, I laid out the case for adversarial prompt engineering in Solidity auditing. The principles were clear: give the model a role that isn't "helper," force structured output, separate attack modeling from remediation, and never skip the verification step.

But there was a gap. Principles are useful. A running system is more useful.

So let's close it.

This article is about taking that entire framework, the Planner, the Analyzer, the Verifier, and the Report Writer, and turning it into an actual multi-agent team that you can run against a real contract. We're using CrewAI as the orchestration layer. By the end of the follow-up piece, you'll have a working pipeline that takes a .sol file as input and produces a professional audit report as output.

This article covers the thinking: how the agent team is designed, what each agent does, and what their prompts look like. The next article covers the building: project structure, the CrewAI configuration files, how context flows between agents, and how to run the whole thing locally.

Series map: prior prompt-engineering article, this design article, then implementation article

Let's start at the beginning.

Why CrewAI?

Before we get into agents, a quick word on the framework choice, because it matters.

CrewAI is a Python framework for orchestrating teams of AI agents. Each agent has a role, a goal, and a backstory that shapes how the LLM reasons. Tasks are defined separately, with explicit expected outputs. Agents are assigned tasks, tasks can receive context from prior tasks, and the whole thing runs as a sequential (or hierarchical) pipeline.

This maps almost perfectly to how a real audit team works.

In a human audit engagement, you don't have one person doing everything. You have a lead who scopes the work, a specialist who runs tools, a researcher who digs deep, a red teamer who builds PoCs, a reviewer who challenges findings, and a writer who produces the report. Each hand-off is explicit. Each role has a different adversarial posture.

CrewAI models exactly that structure. And for security work specifically, that separation of roles isn't just organizational convenience: it's a safety property. When your Exploit Crafter is explicitly told not to think about fixes, it stops self-censoring. When your Verification Agent is explicitly told to be skeptical of every finding, it catches false positives that a single-agent approach would let through.

The alternative, one massive prompt doing everything, produces the worst of all worlds: analysis that's simultaneously not adversarial enough, not specific enough, and not verifiable.

The Audit Pipeline

Here's the team we're building. Six agents, running sequentially, each passing its output as context to the next:

Six-agent audit pipeline from contract input through planner, static analyst, hunter, exploit crafter, verifier, to report writer

No agent sees the output of a later agent. No agent mixes analysis with remediation. No finding reaches the report without passing through a dedicated skeptic.

Let me walk through each one.

Agent 1: The Contract Planner

Role: Senior Smart Contract Audit Planner

The Planner does something that most developers skip when they ask AI to "just review the code": it builds the map before it starts looking.

In the previous article, I made the case that protocol context is the most important part of any audit prompt, more important than the code itself. The Planner operationalizes that argument. Before any vulnerability analysis starts, it reads the entire contract and produces a structured document covering:

This last output, the risk surface, becomes the analysis queue for every downstream agent. It's how you prevent the Vulnerability Hunter from spending 80% of its reasoning budget on low-risk getter functions.

Prompt Summary

The Planner is explicitly told not to find bugs yet. Its only job is the map. The key instruction that shapes its behavior:

"You do NOT look for bugs yet. You build the map that guides every agent after you."

The backstory frames it as someone who has seen exploits that were "obvious in hindsight once the full picture was understood." This pushes the LLM toward thoroughness over speed, and toward understanding design intent rather than just listing code patterns.

Output is structured markdown, human-readable, but precise enough for downstream agents to extract signal from.

Agent 2: The Static Analysis Interpreter

Role: Static Analysis Specialist

In the previous article, the recommended workflow was: run Slither and 4naly3er first, inject their output as context into your analysis prompts, and use the LLM for what static tools genuinely cannot do.

That's the right workflow when you have those tools installed and integrated. But for this demo crew, we're keeping the stack dependency-free: no external tool API keys required.

The Static Analysis Agent bridges that gap. It doesn't run Slither. It simulates what Slither would flag, using its training knowledge of Slither's 150+ detectors, common Solidity patterns, and how those detectors behave on the code it's reading.

Static tools baseline versus adversarial gaps and triage

This is more useful than it sounds for two reasons.

First, it produces a baseline finding list with triage verdicts (GENUINE, FALSE_POSITIVE, or NEEDS_DEEPER_ANALYSIS), which prevents the Vulnerability Hunter from re-covering the same ground inefficiently.

Second, and more importantly, it explicitly identifies the gaps: vulnerability categories that static tools cannot reliably detect: logic errors, economic attack paths, protocol invariant violations, multi-step reentrancy through unexpected callback chains. These gaps become the Vulnerability Hunter's priority queue.

You're not replacing Slither. You're modeling its epistemic limits, which shapes what comes next.

Prompt Summary

The key framing:

"You treat tool output as a starting point, never a conclusion, and your real value is in correctly triaging what the tools surface versus what they miss entirely."

Output is a JSON array of findings (each with triage, severity, category, justification, location) followed by a written gap analysis. The JSON format is intentional: it makes findings machine-processable without any extra parsing work, which matters when you start building production tooling on top of this pipeline.

Agent 3: The Vulnerability Hunter

Role: Adversarial Smart Contract Security Auditor

This is the core analysis engine of the pipeline. The Vulnerability Hunter receives the Planner's architecture map and the Static Analyst's findings + gap analysis as context, then performs a deep adversarial scan.

The key word is adversarial. The default assumption is that the contract is vulnerable. The Hunter's job is to prove it.

It covers all six attack categories from the previous article:

Six attack categories: reentrancy, access control, arithmetic, logic, economic, denial of service

Reentrancy: traces every external call, checks Checks-Effects-Interactions compliance, considers cross-function and read-only reentrancy vectors.

Access Control: verifies every privileged function is properly gated, looks for missing modifiers, incorrect role checks, and ownership transfer attack paths.

Arithmetic: checks for overflow and underflow (including within unchecked blocks in Solidity ^0.8.x), division by zero, and rounding errors in fee and share calculations.

Logic Errors: examines state update ordering, invariant preservation across all code paths, and edge cases at boundary values (zero, max uint256).

Economic / Incentive Attacks: models fund drainage paths, flash loan vectors, and front-running or sandwich attack opportunities.

Denial of Service: looks for gas griefing, unbounded loops, and ways an attacker could permanently block withdrawals or key protocol functions.

Each finding is returned as a structured JSON object with id, severity, category, affected_function, description, attack_preconditions, attack_vector, impact, and confidence.

Prompt Summary

The adversarial framing from the previous article, operationalized:

"Your default assumption is that this contract is vulnerable. You do not look for what the code does. You look for what it does that the developer did not intend."

Three structural rules enforce rigor: do not fabricate findings, state explicitly when a category is clean and why, and put uncertain findings in a separate NEEDS_REVIEW array rather than mixing them with confirmed issues. This prevents the hallucination problem that plagues single-agent audit approaches: the tendency to generate plausible-sounding but non-exploitable findings when asked to find bugs.

Agent 4: The Exploit Crafter

Role: White-Hat Exploit Researcher

In the previous article, there was a specific warning about mixing attack modeling with remediation in the same prompt:

"When you mix attack modeling with remediation in the same prompt, the model self-censors."

The Exploit Crafter is the structural solution to that problem. It receives confirmed vulnerability findings and does exactly one thing: model the attack. No fixes. No mitigations. Full attacker perspective.

For each Critical or High finding, the Exploit Crafter produces a complete attack scenario covering:

For Medium findings, a simplified scenario. For anything that turns out to be theoretically possible but practically unexecutable, an explicit THEORETICAL label with reasoning.

Prompt Summary

The enforced separation is the defining feature of this agent's prompt:

"Do NOT suggest fixes. Only model the attack. Do NOT write complete runnable Solidity: pseudocode only."

The pseudocode constraint is deliberate. Full Solidity attack contracts are out of scope for an article-accompanying demo, but pseudocode is enough to validate exploitability and communicate the attack path clearly to a human reviewer, which is the actual goal.

Agent 5: The Verification Agent

Role: Skeptical Senior Auditor

This is the most underused pattern in AI-assisted auditing and, from the previous article's perspective, the most important one.

The Verification Agent receives every finding from the Vulnerability Hunter and every attack scenario from the Exploit Crafter, then systematically challenges them. It applies a fixed challenge checklist to each one:

For each finding, the verdict is one of four options: CONFIRMED, FALSE_POSITIVE, DOWNGRADED, or NEEDS_HUMAN_REVIEW. Every verdict comes with explicit reasoning and a reference to the specific code evidence that determined it.

Security audit verdicts: confirmed, false positive, downgraded, needs human review

The final output is a verified findings list plus a summary count: confirmed, false positives eliminated, downgraded, escalated to human. That count is one of the most valuable things in the entire pipeline, because it tells you exactly how much noise the AI generated versus how much signal survived adversarial review.

Prompt Summary

The backstory shapes the entire disposition of this agent:

"You have seen junior auditors and AI systems hallucinate vulnerabilities, misread code flow, and miss access control checks that invalidate entire attack paths. You have no ego about overturning a finding if the evidence doesn't hold."

That last sentence matters. Without it, LLMs tend to defer to their own previous outputs. By explicitly framing the Verification Agent as someone who expects to overturn findings, you're engineering a disposition that counteracts that tendency.

Agent 6: The Report Writer

Role: Professional Blockchain Security Report Author

By the time the Report Writer runs, the heavy work is done. It receives the verified findings list, the exploit scenarios, the audit plan, and the original contract code, and transforms all of it into a publication-ready report.

The structure is fixed and intentional:

Prompt Summary

Two tone requirements pull in opposite directions and both matter:

"Technical sections: precise, no hedging on confirmed findings. Executive summary: readable for a non-Solidity CTO."

And the hard constraint that keeps the report credible:

"Never speculate. Only include demonstrably exploitable confirmed findings."

The report saves to audit_report.md in the project root, ready to read, share, or publish.

What the Full Pipeline Looks Like in Practice

To make this concrete, here's the contract the pipeline runs against in the demo: a deliberately vulnerable ETH vault with three intentional security issues:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

contract VulnerableVault {
    mapping(address => uint256) public balances;
    address public owner;
    bool public paused;

    constructor() { owner = msg.sender; }

    // VULN-001: Reentrancy: external call before state update
    function withdraw(uint256 amount) external {
        require(!paused, "Vault is paused");
        require(balances[msg.sender] >= amount, "Insufficient balance");
        (bool success, ) = msg.sender.call{value: amount}(""); // ← call first
        require(success, "Transfer failed");
        balances[msg.sender] -= amount; // ← state update after, too late
    }

    function deposit() external payable { balances[msg.sender] += msg.value; }

    // VULN-002: Missing access control: any address can pause
    function setPaused(bool _paused) external {
        // missing: require(msg.sender == owner, "Not owner");
        paused = _paused;
    }

    // VULN-003: Inconsistent accounting: owner drains without zeroing balances
    function emergencyWithdraw() external {
        require(msg.sender == owner, "Not owner");
        payable(owner).transfer(address(this).balance); // drains ETH
        // user balances[x] remain unchanged; funds appear present but are gone
    }

    receive() external payable {}
}

Three vulnerabilities. Two of them are Critical/High and immediately exploitable. One of them, the emergencyWithdraw accounting inconsistency, is a design-level flaw that static tools consistently miss because the code itself has no syntax errors and no obvious pattern violation. It's the kind of finding that requires understanding protocol intent, which is exactly what the agent pipeline is designed to surface.

The six agents run sequentially on this contract. Each one builds on the context of the ones before it. The final output, the audit report, captures all three findings with attack scenarios, PoC pseudocode, and concrete remediation guidance.

In the next article, we'll walk through the complete implementation: the project structure, the CrewAI configuration files in full, how context chaining works in code, and how to run the crew locally on your own contracts.

Key Design Decisions, Summarized

Before the implementation piece, here's the reasoning behind the major choices: the things that might look arbitrary but aren't.

Sequential process, not hierarchical. CrewAI supports both. Hierarchical adds a manager agent that routes tasks dynamically. For auditing, that's unnecessary complexity. The pipeline is linear by design: each stage genuinely depends on the previous one, and there's no routing decision to make.

Six agents instead of three. The previous article described three roles: Planner, Analyzer, Reviewer. We split that into six because the distinctions matter for prompt quality. The Static Analyst and Vulnerability Hunter have fundamentally different dispositions: one interprets known patterns, the other reasons adversarially about unknown ones. The Exploit Crafter and Verification Agent are also intentionally separate: one builds the attack, the other tears it down. Combining them produces worse outputs from both.

No external tool dependencies. Slither, Mythril, and 4naly3er are all simulated through LLM reasoning. This means the crew runs with a single API key and zero additional setup. In the follow-up, we'll discuss how to wire up the real tools as CrewAI BaseTool wrappers: that's the natural production upgrade path, but it's not required to get the pipeline running and producing real findings.

Structured JSON throughout. Every agent that produces findings returns JSON. This is the lesson from the previous article: free-form responses in a security context are dangerous. JSON makes findings deduplicate cleanly, survive hand-offs between agents without information loss, and feed directly into report generation without heroic string parsing.

The NEEDS_HUMAN_REVIEW verdict. The Verification Agent can escalate findings rather than forcing a binary confirm/reject. This is the human checkpoint from the previous article's workflow, built into the agent's output contract. Any finding marked NEEDS_HUMAN_REVIEW with human_escalation_required: true is explicitly flagged in the report for senior human auditor attention.

What's Next

The next article goes into the implementation. You'll see the complete agents.yaml, tasks.yaml, crew.py, and main.py files. We'll walk through how context chaining works in CrewAI: how the Vulnerability Hunter gets both the Planner's map and the Static Analyst's gap analysis in its context window without you having to manually wire anything together. And we'll look at the actual output the crew produces on VulnerableVault, so you can see how the pipeline's reasoning compares to what a human auditor would flag.

The goal isn't to replace the auditor. It's to give auditors and developers who can't afford an audit a structured, adversarial, verifiable first pass that catches the things that matter.

This article is installment 2 in the AI-Assisted Smart Contract Auditing series on smarthinking.tech (design of the CrewAI pipeline; implementation follows). The full source code is available at github.com/baties/Smart-Contract-Security.