Advertise on podcast: AI Security Podcast

If your AI solution is just helping humans process the same amount of alerts a little faster, you haven't transformed anything, you've just created a faster hamster wheel.In this episode, Ashish and Caleb speak with Heather Ceylan, CISO at Box.com, about how she is leading a true, developer-first AI transformation within her security organization . Heather reveals the five strategic "AI Bets" Box is making. We dive into the reality of building an AI SOC, discussing how Box achieved a 38% automated triage rate for Tier 1 alerts, and why teaching AI not to hallucinate requires treating prompts like strict policy engines .The conversation also tackles the build vs. buy dilemma. Heather explains why she prefers to have her team build custom AI solutions (at least until vendors can out-innovate her engineers) and shares her biggest disappointment when evaluating AI security startups Questions asked: (00:00) Introduction(02:50) Who is Heather Ceylan? (CISO at Box.com) (04:20) Transformation vs. Acceleration: Eliminating Classes of Work (06:00) Building an AI SOC: Achieving 38% Automated Triage (07:20) Controlling Hallucinations: Prompts as Policy Engines (09:30) The Buy vs. Build Debate for CISOs (14:00) Why Security Architecture Must Be Machine Consumable (16:50) The Problem with 3rd Party Risk Management (18:20) Box's "5 AI Bets" Framework (21:30) Will AI Replace SOC Analysts? Why Teams Are Embracing the Change (23:50) Continuous Pen Testing & Evaluating AI Startups (26:30) The Biggest Pitching Mistake Startups Make with CISOs (30:20) Shadow AI: When the Business Starts Building Its Own Apps (37:30) Personalized Software: The LEGO Brick Model of Security Agents (41:50) Fun Questions: Crocodile Jerky and Tim Tam Slams (44:20) Hobbies & Family: Raising Two Boys and Surviving the Chaos (45:30) Favorite Restaurant: Meyhouse (Turkish Cuisine in Palo Alto) Resources discussed during the episode: Heather's LinkedIn NewsletterHeather's post RSA blog5 Big AI Betshttps://blog.box.com/big-cybersecurity-bets-part1https://blog.box.com/big-cybersecurity-bets-part-2https://blog.box.com/big-security-bet-3-ai-redefines-vulnerability-managementhttps://blog.box.com/5-big-cybersecurity-bets-4-scaling-security-architecture-ai-first-worldhttps://blog.box.com/5-big-cybersecurity-bets-continuous-adversarial-validation

In this episode, Ashish and Caleb discuss the internet-breaking preview of Project Mythos, an unreleased AI model from Anthropic that has shown an unprecedented, terrifying ability to reason through code and automatically generate working zero-day exploits .We dive into the conversations surrounding Project Glasswing, Anthropic's initiative to share this model with select partners (like Palo Alto and CrowdStrike) before public release, allowing them a 100-day window to patch critical vulnerabilities . Caleb explains why this level of AI reasoning isn't just hype: early testers are reporting that Mythos is not only finding zero-days, but actively detecting dormant intrusions within their own networks .If you are a CISO or security practitioner, this episode talks about it all. We discuss why the traditional 30-day patch cycle is dead, why "assuming breach" is now mandatory, and why 60% of legacy security vendors might not survive this shift . Questions asked: (00:00) Introduction: The Hype Around Anthropic's Project Mythos (04:00) What is Project Mythos? (Reasoning and Finding Zero-Days) (06:50) Project Glasswing: The 100-Day Partner Patch Window (08:30) The Controversy: Did Anthropic Pick the Right Partners? (12:30) Why Anthropic Doesn't Have the Compute to Scan the Whole Internet (15:10) The Insider View: Mythos is Finding Dormant Intrusions (16:30) Why 60% of Security Vendors Will Go Away (19:30) Hype vs. Reality: GeoHot's Comments on Small Models (21:30) Eliminating False Positives in Static Code Analysis (23:50) The Zero-Day Clock: Time to Exploit Drops to Under 6 Hours (25:50) The Ethics of Zero-Days: Should Mythos Be Released at All? (34:30) The CISO Action Plan: Speeding Up Patching (Hours vs. Days) (44:50) The 3rd Party SaaS Problem: What to Do When You Can't Patch (46:10) "Assume Breach": Why Deception (Honeypots) is the New Priority (57:30) Empowering Non-Tech Teams to Build Detections (01:02:10) AI Makes Cheesy "Hacker Movies" a Reality Resources mentioned during the episode: Assessing Claude Mythos Preview’s cybersecurity capabilities Project Glasswing Zero Day Clock

With over 70 startups claiming to have built the perfect "AI SOC Analyst" or "AI Threat Hunter," how do you separate the real products from the vaporware? Recorded live at Decibel RSAC Founder Festival, Ashish and Caleb hosted a heated panel with Edward Wu (Founder & CEO, Dropzone AI) and Lou Manousos (Co-Founder & CEO, Ent AI). The group debates the controversial claim that AI can provide 100% threat prevention and exposes the dirty secret of the industry: Many AI startups are "cheating" by hiding human analysts behind their software.If you were a CISO or security practitioner navigating the vendor floor at RSA, this episode provides a BS-detector framework. Learn why an AI wrapper around Claude Code isn't enough, why "consistency" is the ultimate test for AI agents, and how to verify if a startup actually has real-world, paying enterprise deployments (and not just friendly design partners) . Questions asked: (00:00) Introduction: Live with Decibel(01:30) Meet the Panel: Edward Wu (Dropzone) & Lou Manousos (Ent) (03:40) The Great Debate: Has the Industry Given Up on Prevention? (05:50) What Has AI Actually Solved? (Repetitive Work vs. Context) (09:00) How to Spot BS on the RSA Show Floor (11:30) Defining an AI Agent: Chatbots vs. Threat Hunters (13:40) The Claude Code Problem: Is Your Product Just a Wrapper? (16:50) The 80% Accuracy Trap & Why Consistency is Key (21:30) Proving ROI: Evaluating AI Agents Like Human Employees (24:50) The Dirty Secret: Humans Hiding Behind AI Startups (26:30) Spotting Fake Customer Logos (28:30) Audience Q&A: Scaling the SOC vs. Replacing Humans (36:10) Forward Deployed Engineering & Personalized Software (40:30) Reimagining Security Architecture from the Inside Out (43:30) How Ent Detects Remote Workers Outsourcing Their Jobs (45:30) Final Thoughts: Asking Vendors for Real Proof Points

RSA Conference 2026 is here and the AI agent hype machine is louder than ever. In this episode, Ashish and Caleb cut through the noise and arm CISOs, practitioners, and security teams with a clear-eyed view of what's actually happening in AI security this year. From the vendor floor at RSAC to the future of internal security automation, Caleb and Ashish speak about why 70% of "AI agent security" vendors can't even define what an agent is, why security team consolidation around 2–3 major platforms (plus internal AI capability) may be the most underrated CISO strategy of 2026, and why the window from vulnerability disclosure to live exploitation has collapsed from months to under two days. They also explore the emerging idea of a centralised AI automation function inside security teams and why the future of security isn't buying more point solutions, it's building internal AI capability on top of a standardised vendor stack. Questions asked: (00:00) Introduction: Preparing for RSAC 2026(03:50) The Year of the "AI Agent" Marketing Hype (06:50) The Secret to AI Context: Enterprise Search (Glean) (09:50) Why Your SOC Needs a Centralized AI Platform Team (13:30) The #1 Question to Ask Vendors at RSAC: API Access (16:50) The Myth of MCP (Model Context Protocol) as the Gold Standard (20:50) Why RSAC is Too Noisy: Vibe Coding & 1,000 New Startups (22:30) Is Capital Raised the Only Signal of Trust? (24:50) Prediction: CISOs Will Fire 500 Vendors and Consolidate (30:50) The Build vs. Buy Debate for AI Security Features (35:50) Surviving RSAC: Sorting Signal from Noise (38:50) The Problem with "End-to-End" AI Agent Claims (41:50) Are AI-Driven Attacks Real? (44:50) The Zero-Day Clock: From 5 Months to 2 Days (48:50) RSAC Events: Live Recordings and CISO Panels Resources spoken about during the episode: RSAC 2026 BSidesSF 2026 Glean Zero Day Clock

Did Anthropic just kill the AppSec industry? Following the announcement of Claude Code Security, a tool that finds, reasons about, and fixes code vulnerabilities, major security stocks dropped by 8% .In this episode of the AI Security Podcast, Ashish and Caleb break down the reality behind the hype. Caleb explains why using AI for SAST (Static Application Security Testing) is "a no-brainer," noting that many open-source projects and startups have already been doing exactly what Anthropic announced . We discuss why this actually validates the shift toward AI-automated remediation.The conversation goes deeper into the future of the cybersecurity market: Will giant foundation models start acquiring security companies? Will they offer "premium gas" (cheaper tokens) for building on their platforms? And most importantly, what does this mean for AppSec engineers whose jobs involve triaging false positives? Questions asked: (00:00) Introduction: The Claude Code Security Announcement(02:50) What is Claude Code Security? (Finding & Reasoning about VULNs) (03:50) Market Overreaction: Why Security Stocks Dropped 8% (05:10) Why AI-Powered SAST is Not New (OpenAI & Open Source doing it already) (07:20) Will AI Take AppSec Jobs? (Triaging False Positives) (09:00) "Shift Left" on Steroids: Auto-Fixing and PR Submission (11:30) The Threat to Legacy Vendors: Why CrowdStrike's Moat is Safe (14:30) Historical Context: AI is the New Calculator/Typewriter (18:20) The "Gasoline" Theory: Foundation Models as Fuel (21:00) Will Anthropic Acquire Security Startups? (26:30) Anthropic's Go-To-Market Strategy: Building AI SOCs (33:30) Startup Survival: Can Innovation Outpace Big Tech? (41:30) The Future of Threat Intel: Is the Legacy Moat Disappearing? (48:20) Negotiating with Vendors using AI Leverage (53:30) Using Evals for Organizational Anomaly Detection

What if you could automate your entire work life with a personal AI Chief of Staff? In this episode, Caleb Sima reveals "Pepper," his custom-built AI agent to Ashish that manages emails, schedules meetings, and even hires other AI experts to solve problems for him . Using Claude Code and a "vibe coding" approach, Caleb built a multi-agent system over a single holiday weekend, without writing a single line of Rust code himself . We discuss how he used this same method to build a black-box testing agent that auto-files bugs on GitHub and even designed the branding for his venture fund, White Rabbit . We explore why "intelligence is becoming a commodity," and how you can survive by becoming an architect of AI agents rather than just a worker Questions asked: (00:00) Introduction(03:20) Meet "Pepper": Caleb's AI Chief of Staff (05:40) How Pepper Dynamically Hires "Expert" Agents (07:30) Pepper Builds its Own Tools (MCP Servers) (11:50) Do You Need to Be a Coder to Do This? (12:50) Using "Claude Superpowers" to Orchestrate Agents (16:50) Automating a Venture Fund: Branding White Rabbit with AI (20:50) Building a "Black Box" Testing Agent in Rust (Without Knowing Rust) (28:50) The Developer Who Went Skiing While AI Did His Job (32:20) The Coming "App Sprawl" Crisis in Enterprise Security (36:00) Security Risks: Managing Shared Memory & Context (41:20) The Future of Work: Is Intelligence Becoming a Commodity? (44:50) Why Plumbers are Safe from AI

This is a forward-looking episode, as Ashish Rajan and Caleb Sima break down the 8 critical predictions shaping the future of AI security in 2026 We explore the impending "Age of Zombies", a crisis where thousands of unmaintainable, "vibe-coded" internal tools begin to rot as employees churn . We also unpack controversial theory about the "circular economy" of token costs, suggesting that major providers are artificially keeping prices high to avoid a race to the bottom . The conversation dives deep into the shift from individual AI features to centralized AI Platforms , the reality of the Capability Plateau where models are getting "better but not different" , and the hilarious yet concerning story of Anthropic’s Claude not being able to operate a simple office vending machine without resorting to socialism or buying stun guns Questions asked: (00:00) Introduction: 2026 Predictions(02:50) Prediction 1: The Capability Plateau (Why models feel the same) (05:30) Consumer vs. Enterprise: Why OpenAI wins consumer, but Anthropic wins code (09:40) Prediction 2: The "Evil Conspiracy" of High AI Costs (12:50) Prediction 3: The Rise of the Centralized AI Platform Team (15:30) The "Free License" Trap: Microsoft Copilot & Enterprise fatigue (20:40) Prediction 4: Hyperscalers Shift from Features to Platforms (AWS Agents) (23:50) Prediction 5: Agent Hype vs. Reality (Netflix & Instagram examples) (27:00) Real-World Use Case: Auto-Fixing 1,000 Vulnerabilities in 2 Days (31:30) Prediction 6: Vibe Coding is Replacing Security Vendors (34:30) Prediction 7: Prompt Injection is Still the #1 Unsolved Threat (43:50) Prediction 8: The "Confused Deputy" Identity Problem (51:30) The "Zombie Tool" Crisis: Why Vibe Coded Tools will Rot (56:00) The Claude Vending Machine Failure: Why Operations are Harder than Code

Is your organization stuck in "read-only" mode with AI agents? You're not alone. In this episode, Dev Rishi (GM of AI at Rubrik, formerly CEO of Predibase) joins Ashish and Caleb to dissect why enterprise AI adoption is stalling at the experimentation phase and how to safely move to production . Dev reveals the three biggest fears holding IT leaders back: shadow agents, lack of real-time governance, and the inability to "undo" catastrophic mistakes . We dive deep into the concept of "Agent Rewind", a capability to roll back changes made by rogue AI agents, like deleting a production database and why this remediation layer is critical for trust . The conversation also explores the technical architecture needed for safe autonomous agents, including the debate between MCP (Model Context Protocol) and A2A (Agent to Agent) standards . Dev explains why traditional "anomaly detection" fails for AI and proposes a new model of AI-driven policy enforcement using small language models (SLMs) as judges . Questions asked: (00:00) Introduction(02:50) Who is Dev Rishi? From Predibase to Rubrik(04:00) The Shift from Fine-Tuning to Foundation Models (07:20) Enterprise AI Use Cases: Background Checks & Call Centers (11:30) The 4 Phases of AI Adoption: Where are most companies? (13:50) The 3 Biggest Fears of IT Leaders: Shadow Agents, Governance, & Undo (18:20) "Agent Rewind": How to Undo a Rogue Agent's Actions (23:00) Why Agents are Stuck in "Read-Only" Mode (27:40) Why Anomaly Detection Fails for AI Security (30:20) Using AI Judges (SLMs) for Real-Time Policy Enforcement (34:30) LLM Firewalls vs. Bespoke Policy Enforcement (44:00) Identity for Agents: Scoping Permissions & Tools (46:20) MCP vs. A2A: Which Protocol Wins? (48:40) Why A2A is Technically Superior but MCP Might Win

It's the season finale of the AI Security Podcast! Ashish Rajan and Caleb Sima look back at their 2025 predictions and reveal that they went 9 for 9. We wrap up the year by dissecting exactly what the industry got right (and wrong) about the trajectory of AI, providing a definitive "state of the union" for AI security. We analyze why SOC Automation became the undisputed king of real-world AI impact in 2025 , while mature AI production systems failed to materialize beyond narrow use cases due to skyrocketing costs and reliability issues . They also review the accuracy of their forecasts on the rise of AI Red Teaming , the continued overhyping of Agentic AI , and why Data Security emerged as a critical winner in a geo-locked world . Looking ahead to 2026, the conversation shifts to bold new predictions: the inevitable bursting of the "AI Bubble" as valuations detach from reality and the rise of self-fine-tuning models . We also explore the controversial idea that the "AI Engineer" is merely a rebrand for data scientists and a lot more… Questions asked: (00:00) Introduction: 2025 Season Wrap Up(02:50) State of AI Utility in late 2025: From coding to daily tasks(09:30) 2025 Report Card: Mature AI Production Systems? (Verdict: Correct)(10:45) The Cost Barrier: Why Production AI is Expensive(13:50) 2025 Report Card: SOC Automation is #1 (Verdict: Correct)(16:00) 2025 Report Card: The Rise of AI Red Teaming (Verdict: Correct)(17:20) 2025 Report Card: AI in the Browser & OS(21:00) Security Reality: Prompt Injection is still the #1 Risk(22:30) 2025 Report Card: Data Security is the Winner(24:45) 2025 Report Card: Geo-locking & Data Sovereignty(28:00) 2026 Outlook: Age Verification & Adult Content Models(33:00) 2025 Report Card: "Agentic AI" is Overhyped (Verdict: Correct)(39:50) 2025 Report Card: CISOs Should NOT Hire "AI Engineers" Yet(44:00) The "AI Engineer" is just a rebranded Data Scientist(46:40) 2026 Prediction: Self-Training & Self-Fine-Tuning Models(47:50) 2026 Prediction: The AI Bubble Will Burst(49:50) Bold Prediction: Will OpenAI Disappear?(01:01:20) Final Thoughts: Looking ahead to Season 4

Cloudflare announced this year that AI bots must pay to crawl content. In this episode, Ashish Rajan and Caleb Sima dive deep into what this means for the future of the "open web" and why search engines as we know them might be dying . We explore Cloudflare's new model where websites can whitelist AI crawlers in exchange for payment, effectively putting a price tag on the world's information . Caleb spoke about the potential security implications, predicting a shift towards a web that requires strict identity and authentication for both humans and AI agents . The conversation also covers Cloudflare's new open-source browser, Ladybird, positioning itself as a competitor to the dominant Chromium engine . Is this the beginning of Web 3.0 where "information becomes currency"? Tune in to understand the massive shifts coming to browser security, AI agent identity, and the economics of the internet . Questions asked: (00:00) Introduction(01:55) Cloudflare's Announcement: Blocking AI Bots Unless They Pay (03:50) Why Search Engines Are Dying & The "Oracle" of AI (05:40) How the Payment Model Works: Bidding for Content Access (09:30) Will This Adoption Come from Enterprise or Bloggers?(11:45) Security Implications: The Web Requires Identity & Auth (13:50) Phase 2: Cloudflare's New Browser "Ladybird" vs. Chromium (19:00) Moving from B2B to Consumer: Paying Per Article via Browser (21:50) Managing AI Agent Identity: Who is Buying This Dinner? (23:20) Why Did We Switch to Chrome? (Performance vs. Memory) (27:00) Jony Ive & Sam Altman's AI Device: The Future Interface? (30:20) Google's Response: New Tools like "Opal" to Compete with n8n (33:15) The Controversy: Is This the End of the Free Open Web? (36:20) The New Economics of the Internet: Information as Currency Resources discussed during the interview: Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model

Should you build your own AI security tools or buy from a vendor? In this episode, Ashish Rajan and Caleb Sima dive deep into the "Build vs. Buy" debate, sparked by Google DeepMind's release of CodeMender, an AI agent that autonomously finds, root-causes, and patches software vulnerabilities . While building an impressive AI prototype is easy, maintaining and scaling it into a production-grade security product is "very, very difficult" and often leads to failure after 18 months of hidden costs and consistency issues . We get into the incentives driving internal "AI sprawl," where security teams build tools just to secure budget and promotions, potentially fueling an AI bubble waiting to pop . We also discuss the "overhyped" state of AI security marketing, why nobody can articulate the specific risks of "agentic AI," and the future where third-party security products use AI to automatically personalize themselves to your environment, eliminating the need for manual tuning . Questions asked: (00:00) Introduction: The "Most Innovative" Episode Ever(01:40) DeepMind's CodeMender: Autonomously Finding & Patching Vulnerabilities(05:00) The "Build vs. Buy" Debate: Can You Just Slap an LLM on It?(06:50) The Prototype Trap: Why Internal AI Tools Fail at Scale(11:15) The "Data Lake" Argument: Can You Replace a SIEM with DIY AI?(14:30) Bank of America vs. Capital One: Are Banks Building AI Products?(18:30) The Failure of Traditional Threat Intel & Building Your Own(23:00) Perverse Incentives: Why Teams Build AI Tools for Promotions & Budget(26:30) The Coming AI Bubble Pop & The Fate of "AI Wrapper" Startups(31:30) AI Sprawl: Repeating the Mistakes of Cloud Adoption(33:15) The Frustration with "Agentic AI" Hype & Buzzwords(38:30) The Future: AI Platforms & Auto-Personalized Security Products(46:20) Secure Coding as a Black Box: The End of DevSecOps?

What does it take to build a fully autonomous AI system that can find, verify, and patch vulnerabilities in open-source software? Michael Brown, Principal Security Engineer at Trail of Bits, joins us to go behind the scenes of the 3-year DARPA AI Cyber Challenge (AICC), where his team's agent, "Buttercup," won second place. Michael, a self-proclaimed "AI skeptic," shares his surprise at how capable LLMs were at generating high-quality patches . However, he also shared the most critical lesson from the competition: "AI was actually the commodity" The real differentiator wasn't the AI model itself, but the "best of both worlds" approach, robust engineering, intelligent scaffolding, and using "AI where it's useful and conventional stuff where it's useful" . This is a great listen for any engineering or security team building AI solutions. We cover the multi-agent architecture of Buttercup, the real-world costs and the open-source future of this technology . Questions asked: (00:00) Introduction: The DARPA AI Hacking Challenge(03:00) Who is Michael Brown? (Trail of Bits AI/ML Research)(04:00) What is the DARPA AI Cyber Challenge (AICC)?(04:45) Why did the AICC take 3 years to run?(07:00) The AICC Finals: Trail of Bits takes 2nd place(07:45) The AICC Goal: Autonomously find AND patch open source(10:45) Competition Rules: No "virtual patching"(11:40) AICC Scoring: Finding vs. Patching(14:00) The competition was fully autonomous(14:40) The 3-month sprint to build Buttercup v1(15:45) The origin of the name "Buttercup" (The Princess Bride)(17:40) The original (and scrapped) concept for Buttercup(20:15) The critical difference: Finding vs. Verifying a vulnerability(26:30) LLMs were allowed, but were they the key?(28:10) Choosing LLMs: Using OpenAI for patching, Anthropic for fuzzing(30:30) What was the biggest surprise? (An AI skeptic is blown away)(32:45) Why the latest models weren't always better(35:30) The #1 lesson: The importance of high-quality engineering(39:10) Scaffolding vs. AI: What really won the competition?(40:30) Key Insight: AI was the commodity, engineering was the differentiator(41:40) The "Best of Both Worlds" approach (AI + conventional tools)(43:20) Pro Tip: Don't ask AI to "boil the ocean"(45:00) Buttercup's multi-agent architecture (Engineer, Security, QA)(47:30) Can you use Buttercup for your enterprise? (The $100k+ cost)(48:50) Buttercup is open source and runs on a laptop(51:30) The future of Buttercup: Connecting to OSS-Fuzz(52:45) How Buttercup compares to commercial tools (RunSybil, XBOW)(53:50) How the 1st place team (Team Atlanta) won(56:20) Where to find Michael Brown & Buttercup Resources discussed during the interview: Trail of BitsButtercup (Open Source Project)DARPA AI Cyber Challenge (AICC)Movie: The Princess Bride

Anthropic's August 2025 AI Threat Intelligence report is out, and it paints a fascinating picture of how attackers are really using large language models like Claude Code. In this episode, Ashish Rajan and Caleb Sima dive deep into the 10 case studies, revealing a landscape where AI isn't necessarily creating brand new attack vectors, but is dramatically lowering the bar and professionalizing existing ones. The discussion covers shocking examples, from "biohacking" attacks using AI for sophisticated extortion strategies , to North Korean IT workers completely dependent on AI, simulating technical competence to successfully gain and maintain employment at Fortune 500 companies . We also explore how AI enables the rapid development of ransomware-as-a-service and malware with advanced evasion, even by actors lacking deep technical skills . This episode is essential for anyone wanting to understand the practical realities of AI threats today, the gaps in defense, and why the volume might still be low but the potential impact is significant. Questions asked: (00:00) Introduction: Anthropic's AI Threat Report(02:20) Case Study 1: Biohacking & AI-Powered Extortion Strategy(08:15) Case Study 2: North Korean IT Workers Simulating Competence with AI(12:45) The Identity Verification Problem & Potential Solutions(16:20) Case Study 3: AI-Developed Ransomware-as-a-Service (RaaS)(17:35) How AI Lowers the Bar for Malware Creation(20:25) The Gray Area: AI Safety vs. Legitimate Security Research(25:10) Why Defense & Enterprise Adoption of AI Security is Lagging(30:20) Case Studies 4-10 Overview (Fraud, Scams, Malware Distribution, Credential Harvesting)(35:50) Multi-Lingual Attacks: Language No Longer a Barrier(36:45) Case Study: Russian Actor's Rapid Malware Deployment via AI(43:10) Key Takeaways: Early Days, But Professionalizing Existing Threats(45:20) Takeaway 2: The Need for Enterprises to Leverage AI Defensively(50:45) The Gap: Security for AI vs. AI for Security Resources discussed during the interview: Anthropic - Threat Intelligence Report August 2025

What if the prompts used in your AI systems were treated as a new class of threat indicator? In this episode, Thomas Roccia, Senior Security Researcher at Microsoft, introduces the concept of the IOPC (Indicator of Prompt Compromise), sharing that "when there is a threat actors using a GenAI model for malicious activities, then the prompt... is considered as an IOPC". The conversation dives deep into the practical application of AI in threat intelligence. Thomas shares details from his open-source projects, including NOVA, a tool for detecting adversarial prompts, and an AI agent he built to track the complex money laundering scheme from a $1.4 billion crypto hack . We also explore how AI is dramatically lowering the barrier to entry for complex tasks like reverse engineering, turning a once-niche skill into something accessible to a broader range of security professionals . Questions asked: (00:00) Introduction(02:20) Who is Thomas Roccia?(03:20) Using AI for Reverse Engineering & Malware Analysis(04:30) Building an AI Agent to Track Crypto Money Laundering(11:30) What is an IOPC (Indicator of Prompt Compromise)?(14:40) MITRE ATLAS: A TTP Framework for LLMs(18:20) NOVA: An Open-Source Tool for Detecting Malicious Prompts(23:15) Using RAG for Threat Intelligence on Data Leaks(31:00) Proximity: A New Scanner for Malicious MCP Servers(34:30) Why Good Ideas are Now More Valuable Than Execution(35:30) Real-World AI Threats: Stolen API Keys & Smart Malware(40:15) The Challenge of Building Reliable Multi-Agent Systems(48:20) How AI is Lowering the Barrier for Reverse Engineering(50:30) "Vibe Investigating": Assisting the SOC with AI(54:15) Caleb's Personal AI Agent for Document Organization Resources discussed during the call: NOVA- The Prompt Pattern Matching DEF CON 33 Talk - Where’s My Crypto, Dude? The Ultimate Guide to Crypto Money Laundering

Welcome to the 2025 State of AI Security. This year, the conversation has moved beyond simple prompt injection to a far more complex threat: attacking the entire ecosystem surrounding the LLM. In this deep-dive discussion, offensive security experts Jason Haddix (Arcanum Information Security) and Daniel Miessler (Unsupervised Learning) break down the real-world attack vectors they're seeing in the wild. The conversation explores why prompt injection remains an unsolved problem and how the LLM is now being used as a delivery system to attack internal developers and connected applications. We also tackle the critical challenge of incident response, questioning how you can detect or investigate a malicious prompt when privacy regulations in some regions prevent logging and observability. This episode is a must-listen for anyone looking to understand the true offensive and defensive landscape of AI security, from the DARPA Cyber Challenge to the race for AI to control the browser. Questions asked: (00:00) Introduction(02:22) Who are Jason Haddix & Daniel Miessler?(03:40) The State of AI Security in 2025(06:20) It's All About the "Scaffolding", Not Just the Model(08:30) Why Prompt Injection is a Fundamental, Unsolved Problem(10:45) "Attacking the Ecosystem": Using the LLM as a Delivery System(12:45) The New Enterprise Protocol: Prompts in English(15:10) The Incident Response Dilemma: How Do You Detect Malicious Prompts?(16:50) The Challenge of Logging: When Privacy Laws Block Observability(21:30) Has Data Poisoning Become a Major Threat?(27:20) How Far Can Autonomous AI Go in Hacking Today?(28:30) An Inside Look at the DARPA AI Cyber Challenge (AIxCC)(40:45) Are Attackers Actually Using AI in the Wild?(47:30) The Evolution of the "Script Kitty" in the Age of AI(51:00) Would AGI Solve Security? The Problem of Politics & Context(59:15) Context is King: Why Prompt Engineering is a Critical Skill(01:03:30) What are the Best LLMs for Security & Productivity?(01:05:40) The Next Frontier: Why AI is Racing to Own the Browser(01:20:20) Does Using AI to Write Content Erode Trust?