Building a Real AI Audit Team for Solidity Smart Contracts
If you read Part 1: CrewAI pipeline design, you already know the plan. Six agents, each with a completely different job, passing their work down the pipeline until a professional audit report lands in your project folder. Now let's build it.
This part is about the actual code. We will walk through the project structure, look at the most important parts of each file, and run the whole thing against a deliberately vulnerable contract. You don't need to be a CrewAI expert to follow along, and by the end you'll have a working local setup.
The full source is on GitHub: github.com/baties/Smart-Contract-Security
This series: Prompt engineering for Solidity audits · Part 1: agent team design · Part 2: implementation (this article)
A Quick Word on CrewAI
If you haven't heard of CrewAI before, here's the short version. It's a Python framework that lets you define a team of AI agents, give each one a distinct role and goal, assign them tasks, and then wire those tasks together so each agent's output automatically becomes the next agent's context. Think of it as a way to turn a long, complicated prompt into an actual team where every member has a clear responsibility.
The reason it fits security auditing so well is that the roles genuinely matter here. An agent that is told "you are an attacker, find every way to drain funds" produces completely different reasoning than one told "you are a skeptical reviewer, challenge every finding." CrewAI lets you enforce that separation cleanly, without hacks.
Project Structure
After cloning the repo, this is what you are working with:
sc_security/
├── src/sc_security/
│ ├── config/
│ │ ├── agents.yaml ← who each agent is
│ │ └── tasks.yaml ← what each agent does
│ ├── crew.py ← wires agents and tasks together
│ ├── main.py ← entry point, loads your contract
│ └── tools/ ← empty for now, ready for Slither later
├── contracts/
│ └── VulnerableVault.sol ← the demo contract with 3 bugs
└── pyproject.toml
Four files do all the work. Let's go through each one.
Defining the Agents
The agents.yaml file is where each agent gets its identity. CrewAI uses three fields: role, goal, and backstory. The backstory is not decoration. It shapes how the LLM reasons throughout the entire task. Compare these two approaches to a vulnerability researcher:
Generic approach:
vulnerability_hunter:
role: Security Researcher
goal: Find vulnerabilities in smart contracts
backstory: You are a security expert.
What we actually use:
vulnerability_hunter:
role: >
Adversarial Smart Contract Security Auditor
goal: >
Perform a deep adversarial analysis of the target contract using the audit
plan and static analysis baseline as context. Identify every exploitable
vulnerability, especially those that automated tools miss: logic errors,
economic attacks, and protocol invariant violations.
backstory: >
You are an elite smart contract security researcher who thinks like an
attacker. Your default assumption is that every contract you see is
vulnerable; your job is to prove it. You have found reentrancy bugs in
audited protocols, identified oracle manipulation paths that no static
tool could detect, and broken economic invariants through multi-step
transaction sequences. You do not look for what the code does. You look
for what it does that the developer did not intend.
The difference in output quality is significant. The second version sets an adversarial mindset from the start, references specific attack types the agent should be thinking about, and closes with that key line: "you do not look for what the code does, you look for what it does that the developer did not intend." That single sentence shifts the entire reasoning posture.
All six agents follow this same pattern. The Verification Agent, for example, is explicitly framed as someone who expects to overturn findings:
verification_agent:
backstory: >
You are the most skeptical person on any audit team. You have seen junior
auditors and AI systems hallucinate vulnerabilities, misread code flow,
and miss access control checks that invalidate entire attack paths.
You have no ego about overturning a finding if the evidence does not hold.
That framing is what makes the false positive filter actually work.
Defining the Tasks
The tasks.yaml file is where you write the actual instructions each agent receives. This is essentially the prompt engineering layer, and it is where the principles from Part 1 of this series become concrete.
A few patterns worth pointing out.
Structured output is enforced. Every analysis task specifies exactly what format the output should take. The Vulnerability Hunter, for example, is told to return a JSON object for each finding:
hunt_vulnerabilities_task:
description: >
...
For each vulnerability found, return a JSON object:
{
"id": "VULN-001",
"title": "Short descriptive title",
"severity": "Critical | High | Medium | Low",
"category": "Reentrancy | AccessControl | Arithmetic | Logic | Economic | DoS",
"affected_function": "functionName()",
"description": "Technical explanation of the flaw",
"attack_preconditions": "What the attacker needs before exploiting",
"attack_vector": "Step-by-step exploitation path",
"impact": "Financial and protocol-level consequences",
"confidence": "High | Medium | Low"
}
This is not optional formatting preference. Structured output is what makes findings survive the handoff between agents cleanly. The Verification Agent can only challenge something it can actually read and parse.
Attack modeling is kept separate from remediation. The Exploit Crafter task includes this line:
craft_exploits_task:
description: >
...
RULES:
- Do NOT suggest fixes. Only model the attack.
- Do NOT write complete runnable Solidity: pseudocode only.
As covered in Part 1, mixing attack modeling with remediation causes the LLM to self-censor its attack paths. Keeping them in separate tasks with separate agents solves that.
The contract code flows through as a variable. Every task receives {contract_code} as an input variable, injected at runtime by CrewAI. This means you can point the pipeline at any .sol file and the entire prompt chain updates automatically.
Wiring It Together in crew.py
This is where the agents and tasks connect. The crew.py file has three sections: agent definitions, task definitions, and the crew assembly.
@CrewBase
class ScSecurity():
@agent
def contract_planner(self) -> Agent:
return Agent(
config=self.agents_config['contract_planner'],
verbose=True
)
@agent
def vulnerability_hunter(self) -> Agent:
return Agent(
config=self.agents_config['vulnerability_hunter'],
verbose=True
)
# ... remaining agents follow the same pattern
Each agent pulls its configuration directly from agents.yaml via the config parameter. No duplication, no hardcoded strings.
The more interesting part is how context flows between tasks:
@task
def hunt_vulnerabilities_task(self) -> Task:
return Task(
config=self.tasks_config['hunt_vulnerabilities_task'],
context=[self.plan_audit_task(), self.interpret_static_analysis_task()]
)
@task
def verify_findings_task(self) -> Task:
return Task(
config=self.tasks_config['verify_findings_task'],
context=[self.hunt_vulnerabilities_task(), self.craft_exploits_task()]
)
That context parameter is the key. It tells CrewAI which previous task outputs should be included in this agent's context window. The Vulnerability Hunter sees both the Planner's architecture map and the Static Analyst's baseline before it starts looking for bugs. The Verification Agent sees both the findings and the exploit scenarios before it starts challenging them.
This is how you get an agent team that actually builds on each other's work rather than just running in parallel and ignoring what everyone else produced.
The final crew assembly is straightforward:
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True,
)
Sequential process means each agent waits for the previous one to finish. No agent sees the output of a later agent. That ordering is intentional and important.
The Sample Contract
The demo contract is VulnerableVault.sol, a deliberately broken ETH vault with three bugs placed at different severity levels and in different vulnerability categories.
contract VulnerableVault {
mapping(address => uint256) public balances;
address public owner;
bool public paused;
// VULN-001: Reentrancy: external call before state update
function withdraw(uint256 amount) external {
require(!paused, "Vault is paused");
require(balances[msg.sender] >= amount, "Insufficient balance");
(bool success, ) = msg.sender.call{value: amount}(""); // call first
require(success, "Transfer failed");
balances[msg.sender] -= amount; // state update after; too late
}
// VULN-002: Missing access control: any address can pause the vault
function setPaused(bool _paused) external {
// missing: require(msg.sender == owner, "Not owner");
paused = _paused;
}
// VULN-003: Owner drains ETH without zeroing user balances
function emergencyWithdraw() external {
require(msg.sender == owner, "Not owner");
payable(owner).transfer(address(this).balance);
// user balances remain unchanged; funds appear present but are gone
}
}
VULN-001 and VULN-002 are the kind of things Slither catches. VULN-003 is more interesting. The code has no syntax errors, no obvious pattern violation, and will pass a surface-level review. The flaw is in the accounting logic: the owner can drain all ETH from the contract while user balances mappings remain untouched, so every user still thinks they have funds. You only catch this if you understand what the contract is supposed to guarantee.
Running It
Install the dependencies and run the crew against the default sample contract:
git clone https://github.com/baties/Smart-Contract-Security.git
cd Smart-Contract-Security/sc_security
pip install -e .
cp .env.example .env
# add your OpenAI or Anthropic key to .env
sc_security
To audit your own contract:
sc_security path/to/YourContract.sol
The pipeline runs all six agents sequentially. With verbose=True, you will see each agent's reasoning as it works. When it finishes, the complete audit report is saved to audit_report.md in your working directory.
On the demo contract, the crew found all three vulnerabilities, including VULN-003 which requires understanding protocol intent rather than pattern matching. The Verification Agent confirmed all three and flagged none as false positives. The report came back with an executive summary, a risk table, full findings with PoC pseudocode, and concrete remediation code for each issue.
What's Missing (Intentionally)
This pipeline runs entirely on LLM reasoning. That is a feature for simplicity and portability, but in a production audit setup you would want to extend it.
The tools/ folder is already there and empty for this reason. The most valuable next step is wrapping Slither as a CrewAI BaseTool and giving it to the Static Analyst agent, so it runs real detectors before the simulated analysis kicks in. The prompt is already designed to accept Slither's JSON output; it just defaults to simulation when the tool isn't present.
Similarly, the Verification Agent's NEEDS_HUMAN_REVIEW verdict was built for a reason. In a real engagement you would put a human checkpoint between the Verification Agent and the Report Writer for any Critical finding. CrewAI supports that natively via human_input=True on a task.
These are natural extensions rather than gaps. The core pipeline is complete and functional as is.
Wrapping Up
Two articles, one working system. The principles from Part 1 (adversarial mindset, structured output, separated attack modeling and verification) all have a direct implementation in the code you just read.
The full source is at github.com/baties/Smart-Contract-Security. Clone it, run it on your own contracts, and see what it finds.
If you extend it with Slither integration, a Foundry PoC runner, or a human-in-the-loop checkpoint, we would genuinely love to see what you build.
This article is Part 2 of the AI-Assisted Smart Contract Auditing series on smarthinking.tech