Very Good Engineering

Designing AI to Fail: What a Capture the Flag Game Reveals About Where LLMs Break

AG

Andy Gibel

Engineering Leader, Yum! Brands

Andy Gibel, Yum! Brands — When Speed Is Cheap, Taste Is Everything

Jorge Coca talks with Andy Gibel, an engineering leader at Yum! Brands, the company behind KFC, Taco Bell, and Pizza Hut, about an AI-focused Capture the Flag game he designed to push his engineering peers past surface-level AI use. Borrowing the format from security, Andy set one rule for participants: use AI by any means necessary, including exploiting the APIs. His fear that someone would crack it in ten minutes led him to over-engineer it into something closer to an escape room, with seven progressively unlocked tiers, a live leaderboard, and a countdown clock.

Andy explains the AI adoption curve he sees most teams stuck on, where people find a comfortable spot prompting a chatbot and stop pushing toward orchestration and multi-agent work. He designed challenges that target documented model weaknesses: the eagerness to please that produces confident wrong answers, the difficulty AI has with deception, which he tested with a lying-agent murder mystery, and a Magic Eye stereogram challenge that had agents grinding for hours because the models lack depth-based vision. He even ran an autonomous red team of agents against his own build to find and patch holes.

The takeaway reaches past the offsite. Andy and Jorge discuss how securing agentic systems marries deterministic gatekeepers with non-deterministic models, and why, as production gets cheaper and faster, the bottleneck moves to review and the real skill becomes taste, reading unfamiliar code and reasoning about architecture. As Andy puts it, echoing Andrej Karpathy, you can outsource your thinking but not your understanding.

Frequently Asked Questions

What was the AI Capture the Flag game?

An offsite challenge Andy Gibel built to push engineers past surface-level AI use, with seven progressively unlocked tiers, a live leaderboard, a countdown clock, and one rule: use AI by any means necessary.

Which AI weaknesses did the challenges target?

Documented ones like the eagerness to please that produces confident wrong answers, difficulty with deception, and weak depth-based vision, tested with a lying-agent murder mystery and a Magic Eye stereogram.

What is the takeaway for engineers?

As production gets cheaper and faster, the bottleneck moves to review, and the real skill becomes taste. As Andy puts it, echoing Andrej Karpathy, you can outsource your thinking but not your understanding.