
Advertise on podcast: Detection at Scale
Rating
5from
This podcast has
78 episodes
Language
EnglishPublisher
Panther LabsExplicit
No
Date created
2021/08/24
Latest episode
2026/05/12
Average duration
37 min.
Release period
22 days
Description
The Detection at Scale Podcast is dedicated to helping security practitioners and their teams succeed at managing and responding to threats at a modern, cloud scale. Hosted by Jack Naglieri, Founder and CTO at Panther, every episode is focused on actionable takeaways to help you get ahead of the curve and prepare for the trends and technologies shaping the future.
Unlock Detection at Scale podcast Email contact info,
Listeners & Audience details
Email contact information
Direct podcast contact details

Listeners
Audience numbers & engagement insights

Audience details
Podcast Insights

Podcast episodes
Check latest episodes from Detection at Scale podcast
Closing The Alert vs. Closing The Loop: How AI Is Reinventing the SOC
2026/05/12
This week is a special episode of the Detection at Scale podcast. I’m usually the one asking the questions, but this time I’m in the guest chair, hosted by Julian Giuca, Panther’s Chief Product Officer. Our conversation covers the journey of building Panther’s AI SOC platform: From the evolution of the SOC from human-led to AI-enabled, the shifts of the last few years that took LLMs from “autocomplete-plus-plus” into genuinely useful agents, how security teams are actually adopting this technology in production, and what we’ve learned building these systems as the operating patterns keep evolving.
The podcast traces back to two early bets, the security data lake and detection-as-code, which we made in 2018 to solve the data scale problem before AI emerged as the next wave. Many years later, those choices turned out to be the exact foundation AI agents needed: detection logic they could read and modify, and a query layer to access huge amounts of helpful context. The work since then has been figuring out what changes when agents are the primary readers of detections, where teams actually want to land on the autonomy spectrum, and what it means for a system to close the loop rather than just close the alert.
One frame I keep coming back to from this conversation is that the risk of not adopting AI in the SOC is greater than the risk of an agent making a mistake. That has not been true for any prior generation of automation, and it changes the calculus for how aggressively teams should move. I hope you enjoy the conversation! Please leave a comment with your thoughts.
Why detections written for humans fail agents: Most detections hand an analyst a step-by-step runbook — check this, then this, then make a call. When you give that structure to an agent, you’re wasting the technology. What works is describing the threat model, the evidence you’d want to see, and the judgment criteria a senior analyst would apply, then letting the agent reason from there.
Closing the alert vs. closing the loop: Closing an alert clears the queue. Closing the loop means the system gets smarter every time it runs. An agent without native access to your detection logic can triage, but it cannot improve the underlying detection that fired, which means alert volume stays flat or grows because nothing is actually learning.
The three-part inflection that moved AI from code autocomplete to agentic work: Reasoning models, tool calling, and MCP converging together is what made agents capable of doing real investigative work in the SOC — not any single capability in isolation, but the combination of all three arriving at roughly the same time.
Architectural prerequisites most teams don’t have yet: Python-based detections and a centralized cloud data warehouse weren’t built with agents in mind, but they created exactly the foundation agents need — detection logic they can read and modify, and a data layer they can query and federate out from via MCP into sources like BigQuery, Elastic, or Snowflake.
Agent autonomy tolerance as a function of workflow risk: There’s no single right threshold for how much independence to give an agent. High-stakes workflows need tighter guardrails; routine triage can run with more autonomy. Teams applying one blanket policy across everything are either underusing the technology or taking on more risk than they realize.
From 50% to 110% alert coverage: Going from monitoring half your alerts to exceeding full coverage — with that extra capacity running as proactive threat hunting agents around the clock — isn’t theoretical. It doesn’t mean fewer security people; it means you need people who know how to work with agents, prompt them well, and encode your team’s expertise into something that scales.
Google's Michael Sinno on Autonomous Detection at 7 Trillion Logs Per Day
2026/02/24
What does it actually take to automate security operations when you're processing 7 trillion log lines daily and a single missed threat could compromise billions of users? Michael Sinno, Director of Detection & Response at Google, explains how his team handles this with less than 1% requiring human intervention through strategic AI implementation. He explores Google's methodical approach to AI autonomy, including fine-tuned models trained on golden datasets, validation through overseer agents, and the critical distinction between traditional automation and agentic AI that exercises judgment.
Michael also discusses groundbreaking work with Sec-Gemini and Timesketch that enables forensic analysis to surface attack patterns humans would never detect manually. Michael shares concrete metrics like reducing executive incident notifications from 30 minutes to 90 seconds, achieving 95% precision in ticket deduplication, and automating vulnerability coordination from hours to minutes.
Topics discussed:
Processing 7 trillion log lines daily with less than 1% of a million annual tickets requiring human intervention at Google
Strategic evolution from AI-assisted to AI-led to autonomous security operations using fine-tuned models and golden datasets
Building modular detection agents as pluggable components that can be combined like Legos for specific security use cases
Implementing quality assurance through overseer agents that review other agents' work to ensure precision in security decisions
Reducing executive incident notifications from 30 minutes to 90 seconds using AI-powered summarization and context gathering
Achieving 95% precision in ticket deduplication while managing the trade-off between precision and 38% recall rates
Integrating Sec-Gemini with Timesketch to surface attack patterns in forensic investigations that humans would never find manually
Shifting from traditional detection and response to infer-and-interrupt models that contain threats immediately before escalation
Automating vulnerability coordination workflows from hours to minutes through AI-powered data collection and impact analysis
Distinguishing between traditional automation and agentic AI that exercises judgment rather than following if-then logic
Setting a stretch goal of 70% automation in operations work while focusing humans on novel and complex security challenges
Measuring success through time-to-mitigation metrics and evaluating AI performance against human baseline capabilities
Listen to more episodes:
Apple
Spotify
YouTube
Website
Block's CISO James Nettesheim on How 40% of Their Detections Are Now Written with AI
2026/02/10
What if the real risk isn't adopting AI agents, but refusing to? James Nettesheim, CISO & Head of Enterprise Technology at Block, argues that principled risk-taking beats playing it safe. James shares Block's journey co-designing the Model Context Protocol with Anthropic and building Goose, their open-source general-purpose agent that enables anyone in the company to write security detections using natural language.
James also explores Block's Binary Intelligent Triage system achieving 99.9% accuracy, their data safety levels framework, and practical strategies for balancing autonomous AI capabilities with human oversight. James offers candid insights about implementing AI security principles, the evolution from tool experts to domain experts, and why open source remains fundamental to Block's mission of economic empowerment and technological innovation.
Topics discussed:
Co-designing of MCP with Anthropic and developing of Goose as an open-source general-purpose AI agent
Implementing prompt injection defenses and adversarial AI concepts to harden Goose against malicious instructions and attacks
Rolling out AI responsibly through data safety levels modeled after CDC bio-contamination protocols for sensitive data handling
Democratizing detection engineering by enabling anyone at Block to write detections using natural language
Achieving 40% of new detections created with AI assistance through recipes, playbooks, and automated tuning capabilities
Building Binary Intelligent Triage system that analyzes historical alerts and investigations to achieve 99.9% automated triage accuracy
Balancing autonomous AI capabilities with human oversight, requiring PR reviews and maintaining accountability for agent-generated code
Transitioning from tool expertise to domain expertise as the future skill set needed for detection and response professionals
Block's commitment to open source development driven by economic empowerment mission and desire to build accessible financial tools
Listen to more episodes:
Apple
Spotify
YouTube
Website
Compass' Ryan Glynn on Why LLMs Shouldn't Make Security Decisions — But Should Power Them
2026/01/27
Ryan Glynn, Staff Security Engineer at Compass, has a practical AI implementation strategy for security operations. His team built machine learning models that removed 95% of on-call burden from phishing triage by combining traditional ML techniques with LLM-powered semantic understanding.
He also explores where AI agents excel versus where deterministic approaches still win, why tuning detection rules beats prompt-engineering agents, and how to build company-specific models that solve your actual security problems rather than chasing vendor promises about autonomous SOCs.
Topics discussed:
Language models excel at documentation and semantic understanding of log data for security analysis purposesUsing LLMs to create binary feature flags for machine learning models enables more flexible detection engineeringAgentic SOC platforms sometimes claim to analyze data they aren't actually querying accurately in practiceTuning detection rules directly proves more reliable than trying to prompt-engineer agent analysis behaviorIntent classification in email workflows helps automate triage of forwarded and reported phishing attempts effectivelyCustom ML models addressing company-specific burdens can achieve 95% reduction in analyst workload for targeted problemsAlert tagging systems with simple binary classifications enable better feedback loops for AI-assisted detection tuningContext gathering costs in security make efficiency critical when deploying AI agents across diverse data sourcesQuery language complexity across SIEM platforms creates challenges for general-purpose LLM code generation capabilitiesExplainable machine learning models remain essential for security decisions requiring human oversight and accountabilityListen to more episodes:
Apple
Spotify
YouTube
Website
Veeva Systems' Mike Vetri on Building Threat Operations Teams and AI-Powered Investigations
2026/01/13
Mike Vetri, Sr. Director of Security Operations at Veeva Systems, reflects on transforming SOC investigations through AI-powered data aggregation and building threat operations teams with the analytical mindset required for proactive defense. Mike introduces the C3 Matrix framework for prioritizing security efforts across centers of gravity, crown jewels, and capability enablers, and explains the seven Ds of cyber defense from discovery through deception operations.
Drawing from 10+ years of Air Force cyber intelligence experience, Mike details why threat operations requires fundamentally different system-two thinking than detection engineering, and how this discipline shift moves organizations from reactive firefighting to proactive threat anticipation. He covers practical examples of AI cutting investigation time by aggregating data from multiple tools, the importance of defense in personnel for operational resilience, and strategies for preventing analyst burnout while maintaining effective security operations.
Topics discussed:
How AI transforms insider threat investigations by aggregating workstation logs, browsing history, and DLP alerts into single queries
The C3 Matrix framework prioritizes security controls across centers of gravity, crown jewels, and capability enablers based on organizational impact and recoverability
Why threat operations requires system-two analytical thinking fundamentally different from the engineering mindset
The seven Ds of cyber defense: discover, detect, deny, disrupt, degrade, destroy, and deception operations for comprehensive threat mitigation
How deception operations provide the most accurate intelligence by studying adversary behavior in controlled environments
The distinction between threat intelligence and threat operations, and why mature SOCs need teams focused on proactive defense strategies
Defense in personnel ensures multiple team members can handle each security capability, preventing single points of failure
Time-sensitive investigation scenarios where AI delivers maximum ROI by eliminating the need to manually query dozens of security tools
The evolution of cyber threats from technical attacks to psychological warfare using AI to challenge human judgment and decision-making
Why security culture must extend beyond traditional boundaries as AI-powered threats increasingly target HR processes, financial operations, and business functions
Listen to more episodes:
Apple
Spotify
YouTube
Website
Trustpilot's Gary Hunter on Structuring Security Knowledge for AI Success
2025/12/23
Gary Hunter, Head of Security Operations at Trustpilot, built a security team from scratch at a company synonymous with trust. Gary shares how his ten-person team leverages AI agents across alert triage, multimodal brand protection, and incident response.
He explores why he and his team treat AI agents like interns with codified guardrails, why competitive prompt testing reveals the best approaches, and how restricting AI to specific documentation sets prevents confusion. Gary also offers his tips on building weatherproof team members who adapt to any technology shift and reflects on why constraints breed creativity in resource-limited environments.
Topics discussed:
Building security operations from scratch by identifying pain points, understanding technology gaps, and systematically increasing detection coverage and visibility
Leveraging AI agents for alert triage and workflows to enable teams to run as fast as attackers while maintaining appropriate human oversight
Implementing competitive prompt testing by running multiple AI models to identify the most effective approach before deployment
Creating cultural buy-in for AI adoption by empowering team members to contribute prompts and democratizing learning across skill levels
Using AI for multimodal brand protection, analyzing screenshots and HTML content to score potential infringements and automate response workflows appropriately
Treating AI agents like interns, codifying processes, and limiting tool access based on what you'd delegate to junior team members
Building detection strategies that focus on behaviors and crown jewels while using AI to triage noisy but potentially valuable alerts
Documenting institutional knowledge concisely rather than overwhelming AI models with extensive documentation that creates conflicting or irrelevant responses
Shifting team focus from alert triaging to high-impact prevention work, vendor management, and building relationships across the business
Listen to more episodes:
Apple
Spotify
YouTube
Website
Vjaceslavs Klimovs on Why 40% of Security Work Lacks Threat Models
2025/12/09
Vjaceslavs Klimovs, Distinguished Engineer at CoreWeave, reflects on building security programs in AI infrastructure companies operating at massive scale. He explores how security observability must be the foundation of any program, how to ensure all security work connects to concrete threat models, and why AI agents will make previously tolerable security gaps completely unacceptable.
Vjaceslavs also discusses CoreWeave's approach to host integrity from firmware to user space, the transition from SOC analysts to detection engineers, and building AI-first detection platforms. He shares insights on where LLMs excel in security operations, from customer questionnaires to forensic analysis, while emphasizing the continued need for deterministic controls in compliance-regulated environments.
Topics discussed:
The importance of security observability as the foundation for any security program, even before data is perfectly parsed.
Why 40 to 50 percent of security work across the industry lacks connection to concrete threat models or meaningful risk reduction.
The prioritization framework for detection over prevention in fast-moving environments due to lower organizational friction.
How AI agents will expose previously tolerable security gaps like over-provisioned access, bearer tokens, and lack of source control.
Building an AI-first detection platform with assistance for analysis, detection writing, and forensic investigations.
The transition from traditional SOC analyst tiers to full-stack detection engineering with end-to-end ownership of verticals.
Strategic use of LLMs for customer questionnaires, design doc refinement, and forensic analysis.
Why authentication and authorization systems cannot rely on autonomous AI decision-making in compliance-regulated environments requiring strong accountability.
GreenSky's Ken Bowles on Auditing Controls before They Silently Fail
2025/11/25
Over his 15-year journey through healthcare and financial services security, Ken Bowles, now Director of Security Operations at GreenSky, has collected a plethora of practical strategies for prioritizing crown jewels, managing cloud over-permissions, and building SOCs that scale effectively. He reflects on transforming security operations through AI and intelligent automation and discusses how AI is reducing analyst investigation time dramatically.
Ken also asserts the importance of auditing security controls before they silently fail. The conversation touches on the evolving role of the MITRE framework, the concept of signaling versus alerting, and why embracing AI might be the best career move for security professionals navigating rapid technological change in cloud environments.
Topics discussed:
Building security operations programs around crown jewels and scaling outward to manage the most critical assets first.
Managing over-permissions in cloud environments that have snowballed across multiple administrators without proper governance.
Using AI to reduce analyst investigation time from 30 minutes to seconds through intelligent data enrichment and context.
Creating true single-pane-of-glass visibility by connecting security tools and data sources for more effective threat detection.
Training new security analysts with AI assistance to bridge knowledge gaps in SQL, SOAR platforms, and log analysis.
Documenting institutional knowledge while encouraging analysts to trust their intuition when something doesn't look right.
Understanding the limitations of impossible travel alerts and using AI to establish user behavior baselines for accurate detection.
Applying the MITRE framework as a guideline rather than gospel, adapting detection strategies to specific organizational needs.
Implementing signaling approaches that label security-relevant events without creating alert fatigue for security operations teams.
Auditing security controls regularly to catch configuration drift and ensure protective measures remain effective over time.
Listen to more episodes:
Apple
Spotify
YouTube
Website
FanDuel's Tyler Martin on the Bronze-Silver-Gold Path to Autonomous Security Triage
2025/11/11
Tyler Martin, Senior Director of Enterprise Security Engineering & Operations at FanDuel, reflects on revolutionizing security operations by replacing traditional analyst tiers with security engineers supported by custom AI agents. Tyler shares the architecture behind SAGE, FanDuel's phishing automation system, and explains how his team progressed from human-in-the-loop validation to fully autonomous triage through bronze-silver-gold maturity stages.
The conversation explores practical challenges like context enrichment, implementing user personas connected to IDP and HRIS systems, and choosing between RAG versus CAG models for knowledge augmentation. Tyler also discusses shifts in detection strategy, arguing for leaner detection catalogs with just-in-time, query-based rules over maintaining point-in-time codified detections that no longer address active risks.
Topics discussed:
Restructuring security operations teams to include only security engineers while AI agents handle traditional level 1-3 triage work.
Building Security Analysis and Guided Escalation, an AI-powered phishing automation system that reduced manual ticket volume.
Implementing bronze-silver-gold maturity stages for AI triage: manual validation, automated closures with oversight, and full autonomous operations.
Enriching AI agents with organizational context through connections to IDP systems, HRIS platforms, and user behavior analytics.
Creating user personas that encode access patterns, permissions, security groups, and typical behaviors to improve AI decision-making accuracy.
Designing incident response automation that spins up Slack channels, Zoom bridges, recordings, and comprehensive documentation through simple commands.
Eliminating 90% of missing PIR action items through automated documentation capture and stakeholder tagging in Confluence.
Shifting detection strategy from maintaining large MITRE-mapped catalogs to just-in-time query-based rules written by AI agents.
Balancing signal volume and enrichment data against inference costs while avoiding context rot that degrades LLM performance.
Evaluating RAG versus CAG models for knowledge augmentation and exploring multi-agent architectures with supervisory oversight layers.
Listen to more episodes:
Apple
Spotify
YouTube
Website
Live Oak Bank's George Werbacher on AI As SecOps' Single Pane of Glass
2025/10/28
George Werbacher, Head of Security Operations at Live Oak Bank, reviews the practical realities of implementing AI agents in security operations, sharing his journey from exploring tools like Cursor and Claude Code to building custom agents in-house. He also reflects on the challenges of moving from local development to production-ready systems with proper durability and retry logic.
The conversation explores how AI is changing the security analyst role from alert analysis to deeper investigation work, why SOAR platforms face significant disruption, and how MCP servers enable natural language interactions across security tools. George offers pragmatic advice on cutting through AI hype, emphasizing that agents augment rather than replace human expertise while dramatically lowering barriers to automation and query language mastery.
Through technical insights and leadership perspective, George illuminates how security teams can embrace AI to improve operational efficiency and mean time to detect without inflating budgets, while maintaining the critical human judgment that effective security demands.
Topics discussed:
Understanding AI's role in augmenting security analysts rather than replacing them, shifting roles toward investigation and threat hunting.
Building custom AI agents using Python and exploring frameworks like LangChain to solve specific SecOps use cases.
Managing moving agents from local development to production, including retry logic, failbacks, and durability requirements.
Implementing MCP servers to enable natural language interactions with security tools, eliminating the need to learn multiple query languages.
Navigating AI hype by focusing on solving specific problems and understanding what agents can realistically accomplish.
Predicting SOAR platform disruption as agents take over enrichment, orchestration, and response with simpler automation approaches.
Removing platform barriers by enabling analysts to use natural language rather than mastering specific tools or query languages.
Exploring context management, prompt engineering, and conversation history techniques essential for building effective agentic systems.
Adopting tools like Cursor and Claude Code to empower technical security professionals without deep coding backgrounds.
Listen to more episodes:
Apple
Spotify
YouTube
Website
Ochsner Health's Andrew Casazza on When AI Becomes the Hammer Looking for Nails
2025/10/14
Andrew Casazza, AVP of Cyber Security Operations at Ochsner Health, explores how healthcare organizations navigate FDA-approved medical devices running on legacy operating systems, implement AI-powered security tools while maintaining HIPAA compliance, and respond to threats that now move from initial compromise to malicious action in seconds rather than hours.
Andrew gives Jack his insights on building effective security programs in heavily regulated industries, emphasizing the importance of visibility, automation with guardrails, and keeping humans in the loop for critical decisions while leveraging AI to handle the speed and scale of modern threats.
Topics discussed:
Unique security challenges in healthcare environments where medical devices run on legacy operating systems that cannot be easily updated.
Strategies for monitoring and securing systems that cannot have traditional security agents installed due to FDA regulations and medical certification requirements.
Leveraging AI and automation in security operations while navigating HIPAA regulations and protecting patient data from external training models.
Implementing human-in-the-loop approaches where AI performs initial analysis and triage while escalating critical decisions to human analysts.
Understanding the privacy and compliance implications of AI tools that may use customer data for model training and improvement.
The dramatic reduction in threat-actor dwell time from hours or days to minutes or seconds.
Building effective SOAR automation playbooks to handle repetitive cases and reduce noise while focusing attention on bigger threats.
Establishing appropriate guardrails for AI-powered security tools to prevent unintended consequences while enabling automated response capabilities.
The importance of being curious and maintaining broad knowledge across multiple domains to become more effective.
Listen to more episodes:
Apple
Spotify
YouTube
Website
Cisco Meraki's Stephen Gubenia on How to Crawl-Walk-Run to AI-Powered SecOps
2025/09/23
Stephen Gubenia, Head of Detection Engineering for Threat Response for Cisco Meraki, shares his evolution from managing overwhelming alert volumes as a one-person security team to architecting sophisticated automated systems that handle everything from enrichment to containment.
Stephen discusses the organizational changes needed for successful AI adoption, including top-down buy-in and proper training programs that help team members understand AI as a productivity multiplier rather than a job threat.
The conversation also explores Stephen’s practical "crawl, walk, run" methodology for responsibly implementing AI agents, the critical importance of maintaining human oversight through auditable workflows, and how security teams can transition from reactive alert management to strategic agent supervision.
Topics discussed:
Evolution from manual security operations to AI-powered agentic workflows that eliminate repetitive tasks and enable strategic focus.
Implementation of the "crawl, walk, run" methodology for gradually introducing AI agents with proper human oversight and validation.
Building enrichment agents that automatically gather threat intelligence and OSINT data instead of manual investigations.
Development of reasoning models that can dynamically triage alerts, run additional queries, and recommend investigation steps.
Automated containment workflows that can perform endpoint isolation and other response actions while maintaining appropriate guardrails.
Essential foundations including proper logging pipelines, alerting systems, and detection logic required before implementing AI automation.
Human-in-the-loop strategies that transition from per-alert review to periodic auditing and agent management oversight.
Organizational change management including top-down buy-in, training programs, and addressing fears about AI replacing jobs.
Future of detection engineering with AI-assisted rule development, gap analysis, and customized detection libraries.
Learning recommendations for cybersecurity professionals to develop AI literacy through reputable sources and consistent daily practice.
Listen to more episodes:
Apple
Spotify
YouTube
Website
Databricks' Dave Herrald on Building SOCs with Data Lakes & Focused AI Agents
2025/09/04
Dave Herrald, Global Head of Cybersecurity GTM at Databricks, tells Jack about transforming security operations through modern data lake architectures and strategic AI implementation. He discusses the practical benefits of separating storage from compute, giving security teams direct control over data retention while maintaining operational flexibility.
The conversation explores how organizations can move beyond traditional SIEM limitations by leveraging cost-effective data lake storage with advanced analytics capabilities. They touch on AI agents in security, where Dave advocates for focused agents over broad analyst replacement approaches. He also addresses common concerns about hallucinations, framing them as engineering challenges rather than insurmountable obstacles, and shares real-world examples of successful agent implementations.
Topics discussed:
Moving from traditional SIEM architectures to modern data lake approaches for cost-effective security analytics and data control.
Implementing focused AI agents for specific security tasks like context gathering rather than attempting broad analyst replacement.
Leveraging graph analytics for security operations including CMDB visualization, breach scoping, and vulnerability prioritization across enterprise environments.
Addressing AI hallucinations through prompt engineering and proper context management rather than avoiding AI implementation entirely.
Building detection capabilities using SQL and Python for analytics that provide supersets of traditional SIEM query languages.
Creating normalization frameworks using standards like OCSF to enable consistent data analytics across diverse security data sources.
Developing career resilience in security through mission-focused thinking, continuous AI learning, and building practical skills.
Comparing modern AI agents to traditional SOAR platforms for automation effectiveness and maintenance requirements.
Establishing data governance and access controls in security data lakes while maintaining operational flexibility and cost effectiveness.
Listen to more episodes:
Apple
Spotify
YouTube
Website
Tines' Matt Muller on AI-Assisted Security Operations and Modernizing the SOC
2025/08/21
Matt Muller, Field CISO at Tines, knows all about revolutionizing security operations through strategic AI integration and intelligent automation. In his conversation with Jack, Matt explores how traditional SOC models create problematic feedback loops where junior analysts make critical decisions while senior practitioners handle escalations, limiting learning and growth opportunities.
Instead, Matt envisions AI-assisted workflows where senior expertise gets encoded into intelligent systems that teach junior team members while they work, transforming security operations from reactive alert-chasing to proactive strategic defense. He also emphasizes communication skills, relationship building, and moving beyond being perceived as the team of no to become strategic enablers.
Topics discussed:
Evolution from banning ChatGPT to strategic AI integration in security operations, emphasizing augmentation over replacement strategies.
Model Context Protocol implementation challenges and the importance of safe-by-default approaches when integrating emerging AI technologies into production.
Traditional SOC tier models create problematic feedback loops where junior analysts make critical decisions but lack learning opportunities.
AI-assisted workflows can transform security operations by encoding senior expertise into systems that teach while automating routine tasks.
Practical approaches to AI adoption including demystification techniques, validation methods, and breaking complex problems into manageable components.
Strategic implementation of AI agents in security workflows, particularly for non-deterministic tasks like phishing investigation and alert triage.
Importance of maintaining human oversight and guardrails when deploying AI systems in critical security operations and incident response.
Communication skills and relationship building as fundamental competencies for security practitioners working with both AI systems and human stakeholders.
Safe experimentation with AI technologies through controlled environments and understanding system limitations before production deployment.
Listen to more episodes:
Apple
Spotify
YouTube
Website
Illumio's Erik Bloch on Getting Security Fundamentals Right Before Adding AI
2025/07/15
In this episode of Detection at Scale, Jack speaks with Erik Bloch, VP of Security, Illumio, about why most security operations teams aren't ready for AI tools and what fundamental processes must be in place first. Erik challenges the industry's obsession with new technologies, sharing stories from his experience transforming underperforming security teams at major companies like Cisco, Salesforce, and Atlassian.
His conversation with Jack explores how to measure what actually matters in security operations, from team capacity utilization to business outcome dispositions, and why proper ticketing systems and actionable metrics are prerequisites for any advanced tooling to be effective.
Topics discussed:
The importance of establishing fundamental processes like ticketing systems and metrics before implementing AI tools in security operations.
How to measure team capacity utilization and resource allocation to identify when security operations teams are operating beyond sustainable levels.
Why traditional security metrics like mean time to detect are often vanity metrics that don't provide actionable business intelligence.
The critical need for security leaders to communicate in business language with concrete data rather than anecdotal risk assessments.
How managed service providers will likely be the first to successfully adopt AI tools due to their standardized processes.
The challenge of proving AI tool effectiveness when most organizations lack baseline metrics to measure improvement against established benchmarks.
Why security teams gravitate toward building custom tools and how this impacts their approach to adopting commercial AI solutions.
The role of MCP in enabling security teams to create their own agents and integrate multiple tools.
How AI should focus on eliminating routine tasks like phishing email analysis rather than trying to catch advanced persistent threats.
The framework for implementing AI tools by starting with business outcomes, defining metrics, identifying capabilities, and then inserting automation.
Listen to more episodes:
Apple
Spotify
YouTube
Website
Podcast reviews
Read Detection at Scale podcast reviews
Dan1777999877 2023/05/02
Incredible!
It’s obvious Jack puts extraordinary effort in covering salient topics and finding guests that are authentic and truly care about being a positive for...
YouShouldCode1 2021/08/24
Love it
High scale threat detection is such a unique problem that more and more struggle with every day - so great to see someone focusing on helping us pract...
Podcast sponsorship advertising
Start advertising on Detection at Scale relevant audience podcasts
You may also like to advertise on these Podcasts

4.712855557
True Crime All The Time
Emash Digital

4.8160261996
Watch What Crappens
Ben Mandelker & Ronnie Karam

4.639331173
Modern Wisdom
Chris Williamson

4.66267310
True Crime with Kendall Rae
Mile Higher Media & Audioboom Studios

4.73456789
The Ancients
History Hit

4.89197320
The Why Files: Operation Podcast
The Why Files: Operation Podcast

51088171
Middle Children
Jessie Jolles and Chris Burns

4.8265172000
Morning Wire
The Daily Wire

4.518808383
American Scandal
Audible

4.689571234
Get Sleepy: Sleep meditation and stories
Slumber Studios