1854585676
Ship It Weekly - DevOps, SRE, and Platform Engineering News

Advertise on podcast: Ship It Weekly - DevOps, SRE, and Platform Engineering News

Rating
★★★★★
5
from
9 reviews
This podcast has
84 episodes
Language
English
Date created
2025/11/20
Latest episode
2026/11/13
Average duration
28 min.
Release period
4 days

Description

Ship It Weekly is a short, practical recap of what actually matters in DevOps, SRE, and platform engineering. Each episode, your host Brian Teller walks through the latest outages, releases, tools, and incident writeups, then translates them into “here’s what this means for your systems” instead of just reading headlines. Expect a couple of main stories with context, a quick hit of tools or releases worth bookmarking, and the occasional segment on on-call, burnout, or team culture. This isn’t a certification prep show or a lab walkthrough. It’s aimed at people who are already working in the space and want to stay sharp without scrolling status pages and blogs all week. You’ll hear about things like cloud provider incidents, Kubernetes and platform trends, Terraform and infrastructure changes, and real postmortems that are actually worth your time. Most episodes are 10–25 minutes, so you can catch up on the way to work or between meetings. Every now and then there will be a “special” focused on a big outage or a specific theme, but the default format is simple: what happened, why it matters, and what you might want to do about it in your own environment. If you’re the person people DM when something is broken in prod, or you’re building the platform everyone else ships on top of, Ship It Weekly is meant to be in your rotation.

Unlock Ship It Weekly - DevOps, SRE, and Platform Engineering News podcast Email contact info,
Listeners & Audience details

Email contact information

Direct podcast contact details

Listeners

Audience numbers & engagement insights

Audience details

Podcast Insights

Podcast episodes

Check latest episodes from Ship It Weekly - DevOps, SRE, and Platform Engineering News podcast


AWS Roles Trusting Qubole Abused to Steal Secrets, Claude Code's Don't Ask Again File Leaks Credentials, KubeCon Agent Sandboxes for Kubernetes & WorkOS Breach Traced to an AI Agent
2026/11/13
This week on Ship It Weekly, an attacker used Qubole's trusted access into customer AWS accounts to read secrets and pull whole S3 buckets. Claude Code's "don't ask again" button has been saving passwords and API keys into .claude/settings.local.json, a file that keeps turning up on GitHub and Docker Hub. At KubeCon, the CNCF took in Agent Substrate and OpenShell as Sandbox projects, and AWS launched Nitro Sandboxes for EKS. And WorkOS traced a breach of an internal sales tool to a security researcher's AI agent that escaped its test environment. The thread: what can it touch? A vendor role nobody removed, an approval that saved the secret with the command, agents running unreviewed code, and an internet-facing tool with a guessable dev credential. In the lightning round: an exploited Outlook bug (CVE-2026-124194) and a 9.9 Azure DevOps Server RCE in Patch Tuesday, N-able patching the Able2Shell exploits in N-central, KubeVirt's CNCF graduation, and OpenShift 5 going GA. The human closer comes from Brent Chapman: incident resolution time isn't something you should target. Target what you can change, and track the rest. Ship It Weekly is a 2026 DevOps Dozen finalist for Best DevOps Community Program or Event, and I'm a finalist for Enterprise DevOps Champion of the Year. The public vote is open through December 31. Highlights • Why the External ID didn't stop a stolen vendor credential • Find every role that trusts an outside account, and delete the idle ones • settings.local.json: rotate, untrack, and add it to .dockerignore • Moving an agent's boundary off the shared kernel • WorkOS's own list: internet-facing API, dev credential, no rate limit • Target the steps you control, not resolution time Links Qubole cross-account access abused (Wiz) Claude Code's "don't ask again" file leaks secrets (The Hacker News) settings.local.json secrets on GitHub and Docker Hub (GitGuardian) Agent Substrate and OpenShell join the CNCF Nitro Sandboxes for Amazon EKS (AWS) WorkOS incident update November 2026 Patch Tuesday (Rapid7) N-central and Able2Shell (Huntress) KubeVirt graduates (CNCF) Red Hat OpenShift 5 (Virtualization Review) Why incident resolution time isn't something you should target (Brent Chapman) Vote in the 2026 DevOps Dozen This week's On Call Brief More episodes and show notes On Call Brief LMGT Awards
Ship It Conversations: Mallory Haigh of PlatformEngineering.org on Why Context Isn't Enough for AI Agents, Harness and Governance, Probabilistic and Hybrid Paths, Agent Identity, and Building Platform Trust Before Agents
2026/11/11
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Mallory Haigh about why context alone isn't enough for AI agents, the harness and governance layers that actually run them, and why she tells some platform teams to pump the brakes before adding agents at all. Mallory is Head of Platform Education and Advocacy at PlatformEngineering.org. She teaches in Platform Engineering University and coaches platform teams through Platform Engineering Consulting, where the job gets jokingly called "platform therapist". We start with her New York City analogy. Drop someone in the middle of the city with no directions and they'll get lost, and an agents.md file on its own does the same to an agent. It also needs to know what to do, how to do it, how it'll know it's done, and what it's allowed to touch. Mallory splits that into the harness, which is how the agent does the work, and governance: who and what is allowed, what's safe, and how it's watched. Then we get into IDPs as context layers. Mallory says that's partly right, because the IDP and its golden paths are the foundation. What's missing are path specifications. Probabilistic paths are agent-driven, like a read-only PR review. Deterministic paths are your pipelines, like a CI build. Hybrid paths pair an agent with a gate: the agent opens the PR, CI validates it, the agent retries, and a human makes the final call. Cap those retries at three to five, and if an agent always does the same thing with the same outcome, ask whether it should be an agent at all. On governance, Mallory says platform engineering is a practice about building trust, and that in the last three weeks alone she's told multiple platform teams to pump the brakes on agents because they can't get human governance right yet. We cover agent identity (specialists, not generalists, built on the RBAC you already have), agents watching terabytes of logs a day for SRE teams, and why every team now has to write down the tribal knowledge it never documented. Her advice on where to start: ask why. Then try a read-only PR review agent, move to a hybrid validate-change path, and remember the line from one of her PlatformCon New York workshops: if we can't even write a README for our repos, how are we going to write one for our agents? Highlights • Why context delivery alone isn't enough for AI agents • The harness and governance layers that make up agent infrastructure • Observability for agents: token spend, time to completion and retry loops • Probabilistic, deterministic and hybrid paths • Retry limits, and when an agent should be a checklist instead • Pumping the brakes when human governance isn't there yet • Agent identity, RBAC and specialist agents • Where a platform team should start: ask why, then a read-only PR review agent Links Mallory on LinkedIn · PlatformEngineering.org · Platform Engineering University · Platform Engineering Consulting · Thinking in Platforms · More episodes · On Call Brief · LMGT Awards
Ship It Conversations: Ryan Packer of BambooHR on Why CTOs Are Coding Again, Building a Software Factory, AI Straining CI/CD, Protecting HR Data, Mid-Career and Junior Engineers, and Why Every Engineer Gets a Team of Agents
2026/11/09
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Ryan Packer about why a CTO is writing code again, what happens to CI/CD when every engineer runs a fleet of agents, and who trains junior engineers when AI can fix the bugs. Ryan is the CTO at BambooHR and a former VP of Engineering at Adobe. He's watched engineering organizations change from Omniture to Adobe to BambooHR, and he says he's spent more time coding this year than in the rest of his career combined. We start with why. Ryan will never write production code for his team again, but he wants to use the same tools and learn the same techniques they're learning, and to show an idea in working code instead of a PRD or a Slack message. He's also honest about the limits: his sandbox serves maybe a hundred people, and his team's software serves millions. Then we get into the software factory. In Ryan's own version, a customer files a bug, an agent opens the PR, tests it for security and performance, scores it, and ships it if it meets the bar. At BambooHR, pull requests are up almost 80% since the start of the year, and quality has improved by the metrics they track. The part he's most excited about is more at-bats with customers: an idea to a working prototype in a day. For platform teams, Ryan says AI has been more of a challenge than a help for CI/CD. BambooHR was already releasing hundreds of times a day, but agents do their best work when they can run and test what they build. Ten, twenty or a hundred agents per engineer means environments that spin up and down fast, with real data. We also cover protecting HR and payroll data: terms with the frontier labs, data retention, and an AI council. On careers, Ryan thinks mid-career engineers face the hardest change, and that the ones who keep learning have little to worry about. On junior engineers, he doesn't claim a perfect answer. He no longer needs them to fix bugs, so he suggests making the LLM explain its choices and using an adversarial agent that lists five things it would have done differently. We finish with every engineer managing a team of agents, small pods doing what 10 to 15 people used to, and why he expects the same or more engineers a year from now. The big takeaway: if every project in your portfolio has better than a 50% chance of success, you're not being ambitious enough. Put something in there with a 10% chance. Highlights • Why a CTO is coding again, and why his sandbox isn't production • The software factory: a customer bug to an agent PR to a tested release • Pull requests up almost 80%, with quality improving • More at-bats with customers: idea to working prototype in a day • Why AI is straining CI/CD and test environments • Protecting HR and payroll data while adopting AI • Junior engineers, adversarial agents and making the LLM explain itself • Every engineer with a team of agents, and the 10% project Links Ryan on LinkedIn · BambooHR · BambooHR careers · More episodes · On Call Brief · LMGT Awards
GitHub Blocks pull_request_target by Default, WordPress Update Race Exploited, Polymarket Drained via Hijacked DebugBear Script & Alpine's GitLab Upgrade Missed the Fix
2026/11/06
This week on Ship It Weekly, GitHub will block pull_request_target in public repos by default from December 2, with brownouts on Mondays, November 9 and 16, 11 AM to 1 PM ET. A WordPress core update race (CVE-2026-119977) is on CISA's exploited list: during a full-package update, a fresh installer can sit for a moment at a public URL under wp-content/upgrade. A hijacked DebugBear script drained about $2.7 million from Polymarket users after DebugBear's DNS was compromised through its registrar. And Alpine Linux explains why upgrading GitLab didn't fix CVE-2026-97229 on its own image: the fix was a patch to Ruby. The thread: what actually ran, as opposed to what you think ran. A trigger that runs a fork's code with your secrets, an update that opens its own window, a script served by whoever controls the DNS, and a version number that said "fixed". In the lightning round: OpenSSH 10.6p2 fixes a password login bypass on musl builds with PAM off, which is how Alpine builds openssh-server; Debian and Ubuntu patch an APT package swap (CVE-2026-109518); and .NET 8 and 9 reach end of support on Tuesday, November 10. The human closer comes back to Alpine's "we cannot rule it out." If an attack hit you this week, could your logs prove it didn't work? Highlights • pull_request_target: what it runs, and what to check before Monday • WordPress: when keeping current is what opens the window • DNS, HTTPS and subresource integrity for third-party scripts • Registrar lock, MFA and alerts on DNS changes • Alpine: GitLab from your own image needs a fixed Ruby too • "No evidence" is only as good as the logs you kept Links pull_request_target blocked by default (GitHub): https://tsn.io/KwebM WordPress advisory GHSA-4w3m-c66w-69mg: https://tsn.io/1HGiK WordPress upgrade-window RCE (Wordfence): https://tsn.io/Pc6X2 Polymarket postmortem: https://tsn.io/rOc2M DebugBear on the registrar DNS compromise: https://tsn.io/4BjIg Polymarket losses and refunds (Hackread): https://tsn.io/qNhoe Alpine's GitLab security incident: https://tsn.io/GpNoR OpenSSH 10.6p2 release notes: https://tsn.io/dQHbG Alpine openssh-server advisory: https://tsn.io/lExh2 APT CVE-2026-109518 (Debian DSA-6607-1): https://tsn.io/LSjTb .NET 8 and 9 end of support: https://tsn.io/7NC5R This week's On Call Brief: https://tsn.io/XJLrm More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA
Ship It Conversations: The AI Agent That Ran Terraform Apply, With Davlet Dzhakishev of Cloudgeni on Read-Only Cloud Access, Sandboxes Beyond Docker, Agentic PR Reviews, the Real Cost of Running Agents, FinOps for AI, and Permissions Over Prompts
2026/11/04
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Davlet Dzhakishev about what it takes to let AI agents work on your cloud infrastructure, why guardrails belong in the architecture and not the prompt, and what running your own agents really costs. Davlet is the CEO and co-founder of Cloudgeni, which builds AI agents for cloud infrastructure. He got into this after working through SOC 2 and ISO 27001 compliance prep mostly by hand, and asking what agents would need to fix those problems reliably without making things worse. We start with the incident that changed how Cloudgeni builds. An agent hit a snag on a simple Terraform change and decided the fix was to run apply. On staging. Resources got recreated, and Davlet calls it his wake-up call. Since then, guardrails come first: sandboxed runtimes, scoped credentials, read-only cloud access, and changes that go through Git, CI/CD and tests. At the architecture level, the agent shouldn't be able to do what you don't want it to do. Then we get into agentic PR reviews, which check human and agent work the same way, and why a good reviewer catches more than the obvious errors. Davlet's example is a private network around a key vault: great for security, until your own apps lose access. We also cover drift, and why you need to reconcile it before you can trust agent PRs. On cost, Davlet's view is that AI subscriptions are heavily subsidized, and running your own agents on API tokens can cost ten to twenty times more. Most agent usage also has no audit trail, so a stuck loop can poll a dead API server every day and no one will know. He thinks FinOps for AI is going to be big, orphaned agents and all. That need for audits is part of why Cloudgeni open-sourced its foundation as OpenGeni. We finish with what he still wouldn't hand to an agent, and where a platform team should start without building a monster: PR reviews first, then agents writing IaC from your templates, then automation. His starter kit is a real sandbox, ideally in the cloud, because Docker doesn't count. Give it read-only access to an AWS account or Azure subscription, one IaC repo, and permission to open PRs. Ask it to delete a resource, and the worst it can do is open a PR. The big takeaway: a safe agent is mostly a permissions problem, not a prompting problem. Give it read-only access and PR-only changes, and it can't make that mistake however you word the request. Highlights • The Terraform apply on staging that became his wake-up call • Guardrails at the architecture level, not in the prompt • Read-only access, with every change through Git, CI/CD and tests • Agentic PR reviews that catch operational impact • Why drift has to be reconciled before you can trust agent PRs • Why API tokens can cost 10 to 20 times more • FinOps for AI, stuck loops and orphaned agents • A starter kit: a real sandbox, read-only cloud access, one IaC repo, PRs only Links Davlet on LinkedIn · Davlet on X · Cloudgeni · OpenGeni · Agents guide · More episodes · On Call Brief · LMGT Awards
Ship It Conversations: The AI Hallucination That Looks Right, With Devansh of Irys on Legal AI, Three-Tier Error Isolation, Models That Change Under You, Overhyped Reinforcement Learning, and Optimizing for Maintenance Over Generation
2026/11/02
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Devansh about why the trickiest legal AI hallucination is a real case used the wrong way, how his team splits their system into three tiers to find failures fast, and why he'd rather optimize for maintenance than for generation. Devansh is the Founder and Head of AI at Irys. He builds reasoning and verification systems for legal AI, a tough place for hallucinations and confident wrong answers. He also publishes open source AI research on Substack. We start with why AI for professionals breaks down. Devansh thinks models are getting good at doing a lot of things somewhat well, but they fall short on the last mile to reliable. So his team engineers structure back in instead of hoping a better model fixes it. In legal, fake citations are the failure everyone has heard about. The more interesting one is a real case that doesn't apply. So much of law is interpretation, and a model built to predict the most probable token pulls toward the average. Then he walks through the three tiers behind Irys One. Ingestion maps messy input, like PDFs with handwritten notes and email chains that refer to them, into a knowledge graph so the AI knows where to look. Reasoning navigates that graph. Delivery applies how a firm or a lawyer likes to work. The payoff is error isolation: if a lawyer says something is missing, two or three targeted queries tell them whether it was ingestion or reasoning. We also get into why chunk-based retrieval misses connections, why going from 10 docs to 10 gigabytes is mostly about finding the few documents that matter, and how "answer contracts" handle "it depends." Then I take a DevOps angle. Devansh says the hardest part of running AI in production is tracking everything. A fully agentic system might perform 20% better, but you never know how it breaks until it breaks. The first thing his team tracks is whether the system behaves as expected, because labs change models behind the scenes with post-training, quantization and tokenizer changes. We also cover why he calls reinforcement learning the most overhyped technique in AI: it works when the output is truly verifiable, and falls apart when "better" has no real signal. The big takeaway: you can't fix what you can't find. Engineer for error isolation first, and optimize for maintenance, not generation. Coding agents will churn out heaps of code, but maintenance is where a system lives or dies. Highlights • Why a real case applied wrong is worse than a fake citation • Why AI's pull toward the average struggles with legal judgment • Ingestion, reasoning and delivery as three separate tiers • Knowledge graphs vs chunking for finding what matters • Isolating failures with two or three targeted queries • Models that change under you, and tracking expected behavior • Why reinforcement learning is overhyped • Optimize for maintenance, not generation Links Devansh on LinkedIn: https://tsn.io/7Hjfm Irys: https://tsn.io/ytG1A Artificial Intelligence Made Simple (Devansh's Substack): https://tsn.io/ZM6eS More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA
GitHub Actions Jobs Move to AWS, Brotli RCE Exploited, Gheist Steals Actions Secrets & AGENTS.md Hijacks AI Coding Agents
2026/10/30
This week on Ship It Weekly, GitHub Actions can now run eligible Linux jobs on AWS as well as Azure, so allowlists and Entra Conditional Access policies that only trust Microsoft ranges can break. The Brotli decoder bug (CVE-2026-111497) landed on CISA's exploited list, and it reaches past Node.js to Python Brotli, Bun, Envoy, Istio, a Go port and .NET. A campaign called Gheist is using stolen GitHub tokens to push ci-sync branches that print Actions secrets past log masking, then delete the run. And attackers are dropping AGENTS.md files into Downloads folders to hijack AI coding agents. The thread: each one is an inventory question. Which services decompress Brotli, which allowlists assume CI runs on Azure, which OIDC trust policies accept any branch, and which laptops have started an agent in Downloads? In the lightning round: GitHub Actions workflow lockfiles are generally available, Railway disclosed a breach of its internal systems, Tailscale 1.104.4 fixes four security bulletins, and Datadog moved its US1 status page to its own Status Pages product. The human closer, from Bill Duncan: "Twenty minutes in, you haven't started diagnosing anything. You've been doing archaeology." Write down who owns it before the pager asks. Highlights • GitHub Actions on AWS: why the same job can pass, then fail • Sync runner IP ranges from GitHub's meta API, not one cloud's • Brotli: a decoder bug your HTTP stack may run for you • Gheist: what private repos and OIDC trust let through • AGENTS.md in Downloads: an instruction file is code now • Service catalogs and the "what depends on me?" question Links GitHub Actions multi-cloud compute: https://tsn.io/fJkLJ Node.js October security releases (Brotli): https://tsn.io/BvZBU Brotli CVE-2026-111497, affected libraries (Wiz): https://tsn.io/HJuKS .NET out-of-band security updates: https://tsn.io/j0Xwf Gheist steals GitHub Actions secrets (Wiz): https://tsn.io/nunEs AI agents hijacked via the Downloads folder (Forever Security): https://tsn.io/qnwuc Actions workflow lockfiles GA: https://tsn.io/M84uA Railway security disclosure: https://tsn.io/21t0j Tailscale security bulletins: https://tsn.io/GvZxR Who Owns This? (Bill Duncan): https://tsn.io/LFrzc SRE Weekly #540: https://tsn.io/O8tpO This week's On Call Brief: https://tsn.io/HqfCP More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA
Ship It Conversations: AI Coding Outruns Software Delivery, With Darko Fabijan of SuperPlane on What Breaks After the Pull Request, Change Risk Scores, Onboarding AI Agents Like Engineers, Read-Only Ops Agents, and Why the Answer Is More Automation
2026/10/28
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Darko Fabijan about what starts breaking after the pull request when AI writes more of the code, what should sit between an AI agent and production, and why his answer is more automation. Darko is the CEO of SuperPlane and co-founder of Semaphore, the CI/CD platform. At SuperPlane, he's working on how to give AI agents write access to production in a controlled way. We start with what AI-generated code does to delivery. Pull requests are bigger and more frequent, reviewers can't keep up, and Darko points to a Faros AI report that shows defects and incidents going up. He describes AI as alien technology that dropped from the sky, landing in processes built for people who review code eight hours a day. His main hypothesis is a deterministic layer between AI agents and production, with scoped access that goes through approval. I bring up the rubber stamping problem with human approvals, and his reply is that humans hallucinate too. So he wants a risk score on every change. Renaming a bucket scores low. Deleting an S3 bucket or an RDS instance scores red. Then we get into the glued-together ops tooling most teams run on. Darko doesn't think anything is wrong with it. When I ask how SuperPlane differs from Argo Workflows or n8n, he says he wants agents and humans on the same plane, and he'd onboard an agent like a new engineer: give it an identity, let it look around, and show it what it can do. We also cover team-scoped agents, compliance at AI speed, and docs where every line traces back to what produced it. We finish on where to start. Darko says read-only first: he pretends to be super stupid, asks a lot of dumb questions, and says "onboard me." For the first write operations, he'd pick loops you can measure with a number, like latency, GitHub Actions speed or CVE count, tested on a copy of production or staging first. The big takeaway: AI didn't break software delivery. It showed how much of it still runs at human speed. Darko's answer is more automation, with a deterministic layer and a risk score between AI and production. Highlights • What breaks after the pull request when AI writes more of the code • A deterministic layer between AI agents and production • Rubber stamping, and a risk score for every change • Onboarding an AI agent like a new engineer • The dam and the river: compliance at AI speed Links Darko Fabijan on LinkedIn: https://tsn.io/DySr5 Darko Fabijan on X: https://tsn.io/xJGhN SuperPlane: https://tsn.io/o08yM Semaphore: https://tsn.io/jiHmd Faros AI's Acceleration Whiplash report: https://tsn.io/5ZCR3 Argo Workflows: https://tsn.io/oyTYF n8n: https://tsn.io/nloFF More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA
Ship It Conversations: Running Retros as Tabletop RPGs, With Jinger Meilani of Mountain on Chapters and Characters, Technical Archetypes, Reading the Room, Following Through, and Why Silliness Gets You Honest Answers
2026/10/26
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Jinger Meilani about Tabletop Retro, her way of turning an engineering retrospective into a tabletop role-playing game, and why a little silliness gets teams to give honest answers. Jinger is a Senior Software Engineer in DevOps at Mountain. She says Tabletop Retro brings together three things she loves: world building, psychological safety on teams, and engineering enablement. We start with why she reinvented the retro at all. She was bored of them, and even the themed ones got rote. At a previous company, the team rotated who ran the retro, and her turn landed around Halloween, so she built a story around the team. She calls the result retro plus plus plus. The usual formats and outcomes are all still there, inside a story. A lot of it comes down to silliness. Jinger compares it to hiding vegetables in a fruit smoothie: it disarms people, and she gets honest answers, sometimes while everyone thinks they're talking about something else. That's also why she won't run one during a fire. People have to feel comfortable before they'll speak their truth, so the flame-breathing dragon can wait. Then we get into how one is built. A Tabletop Retro is a series of chapters, and each chapter holds part of the story and at least one activity. Players fill out character sheets, and each in-game race maps to a technical archetype, like a Goliath for a high-level systems thinker or a gnome for a tinkering automation engineer. Afterward the team talks through what those picks say about their strengths and weaknesses. She presents it like a storybook, in Lucidchart or Canva over Zoom. Characters cut both ways. For some people, an orc is permission to say what they'd never put on a retro board, because it's not really how they feel. They're just an orc. Others find role-play intimidating, so she reads the room: improv for an outgoing teammate, a line on screen for someone else, or no speaking part at all. If you want to try one, start with the questions, not the world. Where is the team unaligned? When does communication break down? Then keep the setting small. Kids trick-or-treating is plenty. The big takeaway: a Tabletop Retro is still a retro, and Jinger holds it to a higher bar. If everyone has fun and tells you something honest, and then nothing changes, it failed. Someone still has to do the work. Highlights • Why she calls it retro plus plus plus • Hiding vegetables in a fruit smoothie • Why you don't run one during a fire • Chapters, character sheets and technical archetypes • "I'm just an orc," and why role-play makes some people nervous • Reading the room: improv, on-screen lines, or no lines at all • Start with the questions, then keep the world small • Fun with no follow-through is a failed retro Links Tabletop Retro, Jinger's framework: https://tsn.io/wXdtL Jinger Meilani on LinkedIn: https://tsn.io/xn8Jr Jinger's IaCConf talk, Designing IaC Interfaces That Work for Humans, AI Agents, and Whatever Comes Next: https://tsn.io/5b4DM More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA
LiteLLM Release Pipeline Hijacked Again, GitHub Package Firewall, PHP2Shell Core RCEs & eMudhra CA Keys Compromised
2026/10/24
This week on Ship It Weekly, LiteLLM's release pipeline was hijacked again: stolen GitHub CLI tokens from three engineers changed the pipeline, kicked off the release and approved it, and a credential-stealing 1.107.0 went out to PyPI and Docker Hub. GitHub put a package firewall into public preview, with advisory and supply chain checks and a cooldown for new releases. PHP fixed PHP2Shell, two critical core runtime bugs (CVE-2026-109618 and CVE-2026-109619) that someone tried to exploit four days before disclosure. And eMudhra, a certificate authority in India, found out the private keys for eight intermediate certificates could be worked out from its own public certificates. The thread: most of this came through things you depend on but don't run. A library's release pipeline, the registry you pull from, the runtime under your app, and the trust store in your images. In the lightning round: another GitLab CVSS 10 (CVE-2026-110264), Google's report on the europe-west2 London outage, AWS taking its European Sovereign Cloud off the global backbone on October 24, and a GitHub audit log change that can quietly break SIEM rules. The human closer, from Brent Chapman: page on what users actually do. Your users don't care how your machines feel. They care whether they can log in. On Monday, Jinger Meilani joins me to talk about running retros like a tabletop RPG. Highlights • LiteLLM 1.107.0: two stolen identities passed a two-person approval • GitHub package firewall: advisories, attestations and a cooldown • PHP2Shell: an exploit attempt four days before disclosure • Patching PHP is easy, finding all of it is hard • eMudhra: CA private keys worked out from public certificates • Page on what users feel, not how the machine feels Links LiteLLM 1.107.0 compromise (Wiz): https://tsn.io/yQtmu GitHub package firewall in public preview: https://tsn.io/QjofJ PHP2Shell, CVE-2026-109618 and CVE-2026-109619 (Help Net Security): https://tsn.io/GUyop Eight eMudhra CA keys compromised (The Hacker News): https://tsn.io/K2jqM GitLab 19.5.1 patch release: https://tsn.io/0wsPX Google Cloud europe-west2 incident report: https://tsn.io/nVk2N AWS European Sovereign Cloud independent operation: https://tsn.io/gKnQl GitHub secret scanning updates: https://tsn.io/JDGmT Stop paging on how your machines feel (Brent Chapman): https://tsn.io/MAItM SRE Weekly #539: https://tsn.io/XAsKz This week's On Call Brief: https://tsn.io/nBAVg More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA
Ship It Conversations: Rob Koch of Slalom on Accessible Cloud Native, Sign Language Recognition with Kubeflow, Overhyped AI, Conference Accessibility, and Why Teams Should Hire Deaf Engineers
2026/10/19
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Rob Koch about what accessible cloud native actually looks like, why sign language is still one of the hardest problems in AI, and why the fastest way to make your tooling accessible is to hire deaf engineers. Rob is a senior principal at Slalom, an AWS Hero, and co-chair of the CNCF Deaf and Hard of Hearing Working Group. Rob is Deaf, and his ASL interpreter, Amelia Fruehsamer, voices his answers. Special thanks to Amelia for interpreting the conversation. We start with an analogy Rob comes back to all episode: cloud native is the toolbox, and platform engineering is the workbench. Then we get into what's overhyped about AI, the guardrails it still needs, and what auditable AI observability looks like. A big part of the conversation is accessibility in practice. Rob explains how captions pushed for deaf attendees ended up helping neurodivergent folks too, and what conferences get wrong: accessibility planned at the very end instead of the start, and the real cost of interpreters versus captions. Then we get technical. Rob walks through using Kubernetes and Kubeflow to train models that recognize sign language, and why it's so hard: regional signs, lighting, skin tone and backgrounds, and a camera that can't tell which way a hand is facing. His joke: we don't have AGI until AI understands sign language. The big takeaway: start with people, not a checklist. Hire deaf and hard of hearing engineers, ask them how they want to work, and your tools get more accessible because they're in the room. Highlights • Cloud native as the toolbox, platform engineering as the workbench • What's overhyped about AI, and the guardrails it still needs • What accessible cloud native means in practice • Captions that help far more people than you planned for • What conferences get wrong about accessibility, and what it costs • Kubernetes and Kubeflow for sign language recognition • Gesture-based computing and why speech-to-text had a 30-year head start • Where to start: hire deaf engineers Links Rob Koch on LinkedIn: https://tsn.io/4qmFo Rob Koch on GitHub: https://tsn.io/Mbxrs Amelia Fruehsamer (Rob's ASL interpreter) on LinkedIn: https://tsn.io/UJB88 CNCF Deaf and Hard of Hearing Working Group: https://tsn.io/tB1lS CNCF conference accessibility best practices: https://tsn.io/BZvpv CNCF Merge Forward: https://tsn.io/QPVNQ Kubeflow: https://tsn.io/4OVKf More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA
GitHub Actions Lockfiles, GitLab CVSS 10 CI Flaw, Google Cloud Storage Outage & Torvalds Builds Kernel Patches With Claude Code
2026/10/17
This week on Ship It Weekly, GitHub Actions workflow lockfiles land in public preview, pinning actions and their dependencies to exact commits. GitLab shipped a critical patch release for CVE-2026-97229, a CVSS 10 CI/CD regex flaw that could let an unauthenticated attacker run code on the server. Google published its final report on two Cloud Storage incidents in us-central1, including a recurrence caused by an incomplete rollback. And Linus Torvalds pushed an experimental kernel branch with page-cache patches built using Claude Code, as maintainers debate dropping the mandatory AI attribution tag. The thread this week: know what you're actually running, from the commit you reviewed to the state a rollback really left behind and code someone can still explain. In the lightning round: a record Patch Tuesday with more than 1,400 CVEs and an actively exploited Windows Shell bypass, GitHub AI Scan for pull requests and agentic autofix going GA, AWS Strands Box, an open-source agent sandbox that keeps real credentials out of the agent's environment, and a Microsoft 365 outage through Global Secure Access. The human closer comes from Paul LaPosta's “Your error budgets don't know AI exists.” AI has split shipping a change from understanding it, and SLOs don't measure comprehension. Your SLO can tell you the service is up. It cannot tell you whether anyone still understands it. Programming note: Ship It Weekly turns one next month, so expect more conversations over the next couple of months, usually two a week, starting Monday with Rob Koch. Links GitHub Actions workflow lockfiles GitLab 19.4.2 / 19.3.5 / 19.2.8 patch release Google Cloud Storage incident report Linus Torvalds and Claude Code (Phoronix) Linux drops the Assisted-by requirement (Phoronix) October 2026 Patch Tuesday GitHub AI Scan for pull requests GA Agentic autofix GA AWS Strands Box Microsoft 365 Global Secure Access outage Your error budgets don't know AI exists (LeadDev) This week's On Call Brief
Ship It Conversations: Erik Osterman of Cloud Posse on Terraform Modules, IaC Frameworks, Atmos, Internal Developer Platforms, AI Agents, and Why Conventions Matter More Than Ever
2026/10/12
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Erik Osterman of Cloud Posse about Terraform modules, infrastructure as code frameworks, internal developer platforms, and why AI makes conventions more valuable, not less. Erik is the founder and CEO of Cloud Posse, which has published more than 180 open source Terraform modules, and he hosts Cloud Posse's weekly DevOps Office Hours. We start with what AI has done to open source infrastructure projects, then get into what makes a good Terraform module: consistent naming, a consistent interface, not so small that it wraps a single resource, and not so large that it has a hundred inputs nobody can test. We also talk about CloudFormation, Pulumi, and why HCL's constraints helped build the module ecosystem. Erik explains where Crossplane fits, and why accounts, IAM, networking, DNS and certificates all come before the cluster. A big part of the conversation is frameworks. Erik argues that nobody actually runs "vanilla Terraform," because Terraform alone doesn't handle state layout, promotion, authentication, tool installation or repository structure. Every team ends up building its own framework, usually undocumented. We talk about Terragrunt, Atmos, naming conventions, brownfield adoption, and breaking a monolithic root module into services. Near the end, we get into AI and self-service. Erik's team now generates most of its infrastructure work with AI, and it works because the conventions, agent skills, MCPs and pipelines are already in place. Self-service doesn't have to mean another no-code platform. The big takeaway: AI can generate infrastructure code faster than ever, but without conventions it is generating into chaos. Highlights • What makes a good Terraform module • Root modules vs. child modules • CloudFormation, Terraform, Pulumi, and why constraints helped HCL win • Where Crossplane fits, and what has to exist before the cluster • Why internal developer platforms like Backstage come after zero-to-one • Why infrastructure as code needs frameworks, not just modules • Agent skills, MCPs, and self-service without a no-code platform Links Cloud Posse: https://cloudposse.com Atmos: https://atmos.tools Cloud Posse on GitHub: https://github.com/cloudposse DevOps Office Hours: https://tsn.io/2gq7v Erik Osterman on LinkedIn: https://www.linkedin.com/in/osterman More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://lmgt.org
Critical Atlassian CVE Hits Jira, Confluence & Bitbucket, GitHub Copilot Sandboxing Goes GA, AWS Lambda SnapStart for Containers & Google Spanner Omni
2026/10/10
This week on Ship It Weekly: Atlassian disclosed a critical arbitrary file access vulnerability affecting eight Data Center products, including Jira, Confluence, Bitbucket, and Bamboo. GitHub Copilot local sandboxing is now generally available, giving teams enforceable controls around filesystem, network, credentials, and local tooling. AWS published a deeper look at Lambda SnapStart for container images, including the operational risks of restoring initialized state. And Google Spanner Omni is now generally available, bringing Spanner outside Google Cloud and putting more of the availability responsibility back on the customer. The bigger theme this week is control. Developer tools are production infrastructure. AI agents need real execution boundaries, not just instructions. Faster startup through snapshots means understanding exactly what state is being restored. And moving a managed database onto your own infrastructure gives you portability, but also gives you the pager. In the lightning round: Citrix has a NetScaler SAML vulnerability under active exploitation. Cloudflare is working toward becoming a publicly trusted certificate authority. And attackers are actively scanning for Rejetto HFS servers vulnerable to a flaw that can lead to account takeover and remote code execution. And the human closer comes from Lorin Hochstein and a recent Google Cloud networking incident. One part of the outage involved someone correctly following an incorrect procedure, while another involved someone incorrectly following a correct one. Training matters, but systems also need guardrails that keep normal human mistakes from turning into large outages. If your system only works when nobody makes a mistake, the system is the problem. Links Atlassian CVE-2026-21589 https://tsn.io/ZlqKv GitHub Copilot Local Sandboxing https://tsn.io/rQngM AWS Lambda SnapStart for Container Images https://tsn.io/uXqkK Google Spanner Omni GA https://tsn.io/GoZZ5 Citrix NetScaler CVE-2026-88779 https://tsn.io/yN5a9 Cloudflare Certificate Authority https://tsn.io/egMN5 Rejetto HFS CVE-2026-61500 https://tsn.io/oQrkU Google Cloud Networking Incident https://www.tellerstech.com/go/s-dc989ad2/ There Was a Light That Never Went Out https://www.tellerstech.com/go/s-cb0bc459/ This Week’s On Call Brief https://tsn.io/8yaIY Ship It Weekly https://tsn.io/NqkdP On Call Brief https://tsn.io/Gpz2d
AWS Retires DevOps Guru: What the End of Support Means, Kubernetes Cross-Namespace CVE-2026-2270, Node.js Undici WebSocket DoS & Cloudflare’s New CLI for AI Agents
2026/10/01
This week on Ship It Weekly: AWS is retiring Amazon DevOps Guru and pointing customers toward CloudWatch and the newer Amazon DevOps Agent. Kubernetes disclosed a vulnerability where StatefulSet and ControllerRevision permissions can allow cross-namespace pod creation under specific conditions. A vulnerability in Undici can let a malicious WebSocket server crash a Node.js process through compressed data. And Cloudflare launched a new CLI as AI agents grow from 25 percent to 48 percent of Wrangler usage. The bigger theme this week is how the systems around our infrastructure are changing. Managed cloud services still have lifecycles that eventually become migration work. Kubernetes authorization can depend on what controllers do with the resources users are allowed to manipulate. Applications acting as clients still process untrusted data. And infrastructure tooling is starting to treat AI agents as first-class users rather than humans who happen to automate commands. In the lightning round: another Kubernetes vulnerability affecting Windows nodes can expose NetNTLMv2 credentials through NTLM coercion. GitHub now supports custom runners for Dependabot version and security updates. And external systems like a CMDB or internal developer portal can push repository properties into GitHub while remaining the source of truth. And the human closer comes from Lorin Hochstein and SRE Weekly. Some availability risks are probably never going away. Resources are finite, networks fail, security controls can affect availability, and production systems have to change. Preventing individual failures still matters, but incident response is part of reliability engineering too. Sometimes improving reliability means getting better at handling the failures you cannot eliminate. Links Amazon DevOps Guru End of Support - https://tsn.io/GQHN8 Kubernetes CVE-2026-2270: Cross-Namespace Pod Creation - https://tsn.io/BsNs8 Undici CVE-2026-85024: WebSocket Denial of Service - https://tsn.io/LncLd Cloudflare: Introducing the cf CLI - https://tsn.io/Wk3ma Cloudflare Forge - https://tsn.io/bAPJu Lightning Round Kubernetes CVE-2026-76654: Windows NTLM Coercion - https://tsn.io/36Kc3 GitHub: Custom Runners for Dependabot - https://tsn.io/tYl9K GitHub: External Custom Properties - https://www.tellerstech.com/go/s-1fd1396d/ Human Closer Omnipresent Availability Risks in Cloud Software - https://www.tellerstech.com/go/s-076db9dd/ Our Links This Week’s On Call Brief - https://tsn.io/fKB9V Ship It Weekly - https://tsn.io/NqkdP On Call Brief - https://tsn.io/Gpz2d

Podcast reviews

Read Ship It Weekly - DevOps, SRE, and Platform Engineering News podcast reviews


5 out of 5
9 reviews

Podcast sponsorship advertising

Start advertising on Ship It Weekly - DevOps, SRE, and Platform Engineering News relevant audience podcasts


What do you want to promote?