Advertise on podcast: Software Engineering Institute (SEI) Podcast Series

AI components, whether traditional machine learning or generative AI, are embedded in many present-day systems. However, despite advances in automation, infrastructure, and tooling for developing AI components, many do not leave the prototype stage or reach production because they fail to meet overall system quality expectations. In our latest podcast from the Carnegie Mellon University Software Engineering Institute, Rachel Brower-Sinning and Robert Edman, both machine learning scientists in the SEI's Tactical Edge and AI-Enabled Systems Initiative, sit down with Grace Lewis, the initiative's lead and an SEI principal researcher, to discuss a quality model for machine learning components to support their proper testing and evaluation. The research team that developed the model also included Alex Derr, Sebastián Echeverría, and Ipek Ozkaya from the SEI, and, from Carnegie Mellon University, Kate Maffey. Colin Beaudoin, a PhD intern at the SEI in the summer of 2025 who is now an assistant professor at Fairfield University, also worked on the model.

The recent explosion in large language model (LLM) technology has highlighted the challenges of using public generative artificial intelligence tools in classified environments, especially for software analysis. Currently, software analysis falls on the shoulders of static analysis tools and manual code review, which tend to provide limited technical depth and are often time-consuming in practice. In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Ryan Karl, an SEI embedded engineer, and John Robert, deputy director of the SEI's Software Solutions Division, discuss their work on using LLMs in unclassified environments to rapidly develop tools that accelerate software analysis in classified environments with improved accuracy for certain software analysis tasks.

Software-defined warfare is today's reality for national security, shifting the emphasis in military operations from hardware to software. In the latest podcast from the Carnegie Mellon University Software Engineering Institute, SEI director Paul Nielsen recently sat down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss the evolution of software-defined warfare and the ways in which software engineering practices can meaningfully address the challenges of implementing software on the battlefield.

In aviation, waypoints guide pilots through complex flight plans, providing some structure but maintaining flexibility. Kevin Dooley, a senior Agile transformation leader at the SEI, adopted this concept to solve one of defense acquisition's most persistent challenges: synchronizing dozens of interdependent teams without drowning in administrative noise and overhead. In the latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Dooley, who co-developed the Waypoints Framework with Air Force Major Adam Satterfield, sits down with Eileen Wrubel, SEI technical director for Smart Software Acquisition to discuss Waypoints and how it can help teams visualize their work and own processes so they can start collaborating.

Experimentation and validation of LLM performance is critical when building LLM-driven systems that must reliably deliver a service, from customer service chat bots to intelligence analysis tools. To help teams meet the need for rigorous evaluation methods, a research team in the SEI's AI Division led by Violet Turri has developed the Evaluating Large Language Models (ELM) library, which is built on best practices for LLM evaluation and benchmarking. In the latest episode from the Carnegie Mellon University Software Engineering Institute, Turri sits down with Katie Robinson, a design researcher also in the SEI's AI division, to discuss the ELM library, which turns evaluation from an ad-hoc process into a repeatable, extensible framework.

Data poisoning—where adversaries tamper with training data to corrupt model behavior—poses significant risks as AI adoption expands across critical sectors. Organizations without mechanisms in place to detect or prevent data poisoning are open to an avenue of attack that, once exploited, is difficult to remediate. Machine unlearning and model retraining are not always viable or effective solutions. In today's operational climate, where threat actors look to influence models and degrade the trust of users through incorrect behaviors, preventing data poisoning is more important than ever.  In this episode of the SEI Podcast Series, Julie Lawler and James Cunningham—AI security researchers at Carnegie Mellon University's Software Engineering Institute—discuss the growing threat of data poisoning in AI systems and highlight emerging mitigation strategies, including chain-of-custody controls.

As recently as December 2025, the Carnegie Mellon University Software Engineering Institute (SEI's) CERT Coordination Center (CERT/CC) documented a UEFI-related vulnerability in certain motherboard models, illustrating that early-boot firmware behavior continues to present security challenges despite requiring local physical access to exploit. While CERT/CC reported seven UEFI vulnerabilities in 2025, that number remains small compared to reported vulnerabilities in other software. However, the consequences of a potential UEFI attack are often more serious given the extremely high privileges UEFI firmware possesses. In our latest SEI Podcast, Vijay Sarvepalli, a senior information security architect specializing in vulnerability and threat analysis in CERT, sits down with Michael Winter, deputy technical director of threat analysis in CERT, to discuss research and mitigation of UEFI vulnerabilities and discuss a new tool, the CERT UEFI parser, an open source tool that uses program analysis to reveal the architecture of UEFI software, and explore this veiled source of vulnerabilities.

In February 2026, Paul Nielsen announced that he will transition out of his role as director and chief executive officer of the Software Engineering Institute (SEI) at Carnegie Mellon University. During Nielsen's tenure, the SEI has marked major institutional milestones that underscore its enduring role in strengthening the security, resilience, and reliability of the nation's software- and AI-intensive systems. The institute recently celebrated 40 years of innovation and saw its contract renewed, which paved the way for CMU to operate the SEI for another five years. In our latest SEI podcast, Nielsen recently sat down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss his legacy at the SEI, the impact of mentors, and the importance of encouraging scientists and engineers to do their best work.

Cybersecurity staffing shortages are a major concern in the government given the increasingly sophisticated cyber attacks on the nation's critical infrastructure. In the FY2023 National Defense Authorization Act (NDAA), Congress tasked the Pentagon with finding flexible options to address cyber staffing needs. The Pentagon commissioned the SEI to conduct an independent study to assess the feasibility and advisability of creating a civilian cybersecurity reserve (CCR) that could harness cyber expertise from the private sector to mobilize a mission-ready workforce capable of operating in contested environments. In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), the lead authors on the report, Marie Baker, a technical manager in the SEI's CERT Division, and Chris May, technical director of the CERT Cyber Mission Readiness directorate, sit down with Mike Winter, deputy technical director of threat analysis, to discuss their findings.

While Stanford University found that AI investments, optimism, and accessibility are rising, a recent MIT report suggests that 95 percent of organizations are realizing no returns on their generative AI investments. Research from Accenture found that only 8 percent of companies are scaling AI at an enterprise level and embedding the technology into core business strategy to maximize value. Mismatched expectations, misaligned applications, and poorly executed or untested implementation practices—not the technology itself—often keep organizations from realizing immediate value from an AI investment. For AI to increase efficiency, productivity, and value while conserving resources and lowering overall costs, organizations need to shift their focus from hype-driven experimentation to foundational capabilities and practical, measurable outcomes. In our latest podcast from the Carnegie Mellon University Software Engineering Institute, Dr. Ipek Ozkaya, technical director of AI-Native Software Engineering, sits down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss their work on an AI Adoption Maturity Model that organizations can use to create a roadmap for predictable AI adoption and realization of AI benefits.

In October 2025, CyberPress reported a critical security vulnerability in the Redis Server, an open-source in-memory database that allowed authenticated attackers to achieve remote code execution through a use-after-free flaw in the Lua scripting engine. In 2024, another prominent temporal memory safety flaw was found in the Netfilter subsystem in the Linux kernel: CVE-2024-1086. Bugs related to temporal memory safety, such as use-after-free and double-free vulnerabilities, are challenging issues in C and C++ code. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Lori Flynn, a senior software security researcher in the SEI's CERT Division, and David Svoboda, a senior software engineer, also in CERT, sit down with Tim Chick, technical manager of CERT's Applied Systems Group, to discuss recent updates to the Pointer Ownership Model for C, a modeling framework designed to improve the ability of developers to statically analyze C programs for errors involving temporal memory.

On November 7, the Department of War released an acquisition transformation strategy that seeks to remove bureaucratic hurdles and streamline acquisition processes to enable even more rapid adoption of technologies, including artificial intelligence. Getting AI into the hands of warfighters requires disciplined AI Engineering. In this podcast from the Carnegie Mellon University Software Engineering Institute, Carol Smith, lead of human-centered research in the SEI's AI Division, and Brigid O'Hearn, the SEI's lead of software modernization policy for the Department of War, sit down with Eileen Wrubel, the SEI's technical director of Transforming Software Acquisition Policy and Practice, to discuss AI Engineering challenges and guidance in the defense acquisition space.

Organizations, including the U.S. military, are increasingly adopting cloud deployments for their flexibility and cost savings. The shared security model utilized by cloud service providers removes some of the adopting organization's responsibility for system administration and security. But it leaves them on the hook for monitoring hosted applications and resources. Cloud flow logs are a valuable source of data for supporting these security responsibilities and attaining situational awareness. The SEI has a long history of supporting flow log collection and analysis, including tools for collection in Azure and AWS. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), two leading researchers in this area, principal researcher Tim Shimeall and security data analyst Ikem Okafo, both with the SEI's CERT Division, sit down with Dan Ruef, technical manager of the CERT Division's Network Situational Awareness Group, to discuss how to enhance security with cloud flow analysis as well as available tools and resources.

From early 2022 through late 2024, a group of threat actors publicly known as APT28 exploited known vulnerabilities, such as CVE-2022-38028, to remotely and wirelessly access sensitive information from a targeted company network. This attack did not require any hardware to be placed in the vicinity of the targeted company's network as the attackers were able to execute remotely from thousands of miles away. With the ubiquity of Wi-Fi, cellular networks, and Internet of Things (IoT) devices, the attack surface of communications-related vulnerabilities that can compromise data is extremely large and constantly expanding.   In the latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI) Joseph McIlvenny, a senior research scientist, and Michael Winter, vulnerability analysis technical manager, both with the SEI's CERT Division, discuss common radio frequency (RF) attacks and investigate how software and cybersecurity play key roles in preventing and mitigating these exploitations.

Modern data analytic methods and tools—including artificial intelligence (AI) and machine learning (ML) classifiers—are revolutionizing prediction capabilities and automation through their capacity to analyze and classify data. To produce such results, these methods depend on correlations. However, an overreliance on correlations can lead to prediction bias and reduced confidence in AI outputs.  Drift in data and concept, evolving edge cases, and emerging phenomena can undermine the correlations that AI classifiers rely on. As the U.S. government increases its use of AI classifiers and predictors, these issues multiply (or use increase again). Subsequently, users may grow to distrust results. To address inaccurate erroneous correlations and predictions, we need new methods for ongoing testing and evaluation of AI and ML accuracy. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Nicholas Testa, a senior data scientist in the SEI's Software Solutions Division (SSD), and Crisanne Nolan, and Agile transformation engineer, also in SSD, sit down with Linda Parker Gates, Principal Investigator for this research and initiative lead for Software Acquisition Pathways at the SEI, to discuss the AI Robustness (AIR) tool, which allows users to gauge AI and ML classifier performance with data-based confidence.