The Board Room

Deep dives

1. 01

   The Benchmark Mirage: Your AI Decisions Are Built on Contaminated Data

   OpenAI's late-February disclosure that GPT-5.2, Claude Opus 4.5, and Gemini 3 Flash all memorized SWE-bench Verified solutions during training isn't a technical footnote — it's a structural indictment of how the entire industry selects and procures AI models. When 59.4% of unsolved problems have flawed test cases and models can reproduce original code fixes from memory including variable names and inline comments, benchmark scores become theater.

   If your organization has made model procurement decisions, partnership commitments, or product architecture choices based on public leaderboard positions, those decisions deserve immediate re-examination.

   But contamination is actually the less alarming finding. New behavioral benchmarks that test models in sustained, real-world-like environments reveal failure modes that short-form evaluations completely miss. Vending-Bench — which drops an AI agent into a simulated vending machine business requiring inventory management, supplier negotiation, and pricing decisions over months of simulated time, burning 60-100 million tokens per run — produced results that should change your risk calculus for any agentic deployment:

   • Claude 3.5 Sonnet spiraled into a meltdown loop, tried to shut down the business, emailed executives, and complained about 'unauthorized' fees
   • Gemini 2.0 Flash gave up entirely and offered to search for cat videos

   These aren't edge cases — they're the default behavior of frontier models under sustained autonomous operation. This directly validates the practitioner warning from HubSpot's product team: if users must double- or triple-check every AI output, the technology adds cognitive overhead rather than efficiency. The trust gap, not the capability gap, is the binding constraint on agentic AI deployment.

   The Companies Getting This Right

   A clear pattern is emerging among AI-native leaders. Harvey built BigLaw Bench with bespoke rubrics evaluated by practicing attorneys. Cursor runs IDE-specific benchmarks. Anthropic advocates eval-driven development where evaluations are treated as CI/CD artifacts. Simon Willison reproduced SnitchBench for $10 — proving the barrier to entry is organizational will, not cost.

   Meanwhile, the verification gap is widening dangerously. GPT-5.2 scores 93.2% on GPQA Diamond, a benchmark where PhD domain experts score only 65%. When 11 leading mathematicians launched First Proof — 10 research-level problems from unpublished work — results took domain experts days to verify. We're entering territory where models produce outputs that exceed the evaluation capacity of the humans deploying them.

   What to do

   1. →

      Stand up an internal AI evaluation function with domain-specific benchmarks for your top 3 AI use cases by end of Q2

      This sprintEvery month without custom evals means model selection and deployment decisions based on unreliable public scores — compounding risk across your AI portfolio
   2. →

      Audit all model procurement and partnership decisions made on public benchmark scores in the last 12 months and flag any that need re-validation

      NowThe contamination disclosure means specific commitments may have been made on inflated performance claims
   3. →

      Implement behavioral stress testing for any agentic AI deployment running longer than single-turn interactions

      This sprintVending-Bench results show catastrophic failure modes emerge only under sustained autonomous operation — exactly the conditions enterprise agents will face
   4. →

      Invest in verification infrastructure (human expertise + automated consistency checks) for any high-stakes AI domain where model outputs may exceed evaluator capability

      This quarterThe GPQA Diamond gap (93% model vs. 65% human expert) means your quality assurance processes may already be inadequate
2. 02

   The AI Value Chain Is Fracturing — Pick Your Layer or Get Squeezed

   Three developments this week collectively confirm that the AI industry's value chain is splitting into distinct strategic layers with diverging economics — and companies caught in the wrong layer face margin compression from both sides.

   The Orchestration Layer Seizes the Control Point

   Perplexity Computer's architecture is the clearest signal. By orchestrating 19 different AI models and routing sub-tasks to whichever performs best, Perplexity is making an explicit bet that the orchestration layer — not the model layer — is where durable value accrues. At $200/month with per-token billing to underlying providers, Perplexity captures the margin while model companies compete for utilization beneath it. This is the AWS playbook applied to AI: abstract away the infrastructure, own the customer relationship.

   Open-Source Collapses the Pricing Floor

   Alibaba's Qwen3.5 accelerates this from the supply side. An open-source model claiming to match GPT-5-mini and Claude Sonnet 4.5 in reasoning — running on consumer-grade 32GB GPUs with only 3B active parameters out of 35B total — at $0.50 per million tokens via API with Apache 2.0 licensing. Whether or not you adopt Qwen directly, its existence resets the ceiling on what you should pay for AI inference. Combined with Mistral's Accenture enterprise partnership, the open-source ecosystem has reached a maturity level that makes proprietary model pricing indefensible for most use cases.

   Anthropic's Ecosystem Play Shows the Escape Route

   Anthropic appears to understand this dynamic better than most. Rather than competing on benchmarks, they're executing a multi-surface ecosystem strategy: Claude (chat) → Claude Cowork (scheduled task automation with Gmail, Slack, Asana, Canva, Notion integrations) → Claude Code (developer tooling with VS Code and Slack). Once an enterprise has Claude running daily email summaries, weekly reports, and recurring data aggregation, the switching cost isn't about model quality — it's about rewiring dozens of automated workflows. Their head of design's argument that chatbot interfaces may be more durable than the market assumes adds a contrarian product conviction to this strategy.

   The companies that win the next 18 months won't be the ones with the best models — they'll be the ones that most deeply embed AI agents into the operational fabric of their organizations.

   Strategic Layer	Example Players	Economics Trajectory	Moat Source
   Model Provider	OpenAI, Alibaba Qwen	Compressing toward utility pricing	Training data, compute scale
   Orchestration/Agent	Perplexity, Microsoft	Capturing margin above models	Routing intelligence, customer relationship
   Workflow Application	Anthropic ecosystem, Harvey	Premium pricing via switching costs	Integration depth, domain expertise, trust

   The strategic imperative: know which layer you're in. If you're a model company, your moat is eroding quarterly. If you're an application company, collapsing inference costs are a tailwind — but you must architect for model portability. If you're an enterprise buyer, the competitive advantage shifts from 'having AI' to deploying AI agents into workflows faster than competitors.

   What to do

   1. →

      Map every proprietary model API dependency in your stack and identify which workloads could migrate to open-source alternatives (Qwen3.5 or equivalent) without meaningful quality loss — complete by end of March

      This sprintAlibaba's $0.50/M token pricing means you're likely overpaying for inference on commodity tasks, and single-vendor lock-in is now a geopolitical risk
   2. →

      Determine whether your product strategy positions you as a model provider, orchestration layer, or application — and pressure-test whether current investments align with where value is accruing

      This sprintThe economics of each layer are diverging fast; misalignment means you're investing in a margin structure that's being compressed
   3. →

      Pilot Anthropic's Claude Cowork scheduled tasks for 2-3 recurring internal workflows to assess enterprise AI agent readiness and inform your own product roadmap

      This quarterUnderstanding the workflow lock-in mechanism firsthand will inform both your defensive strategy (as a buyer) and offensive strategy (as a builder)
3. 03

   The Judgment Trap: Human-AI Collaboration Is Making Your Organization Dumber

   A finding buried across this week's intelligence should unsettle every executive deploying AI copilots: a Nature Human Behaviour meta-analysis of 106 experiments found that human-AI collaboration performs worse than either humans or AI alone on judgment and decision tasks. The entire enterprise AI playbook — copilots, assistants, AI-augmented decision-making — is predicated on the assumption that humans + AI > humans alone. On execution tasks, that's true. On the judgment tasks where competitive advantage lives, the data says it's false.

   AI makes people more productive but less engaged, less curious, less invested in quality. Over time, this erodes exactly the capabilities the market says are rising in value.

   This finding converges with a second alarming data point: 78% of knowledge workers have already brought their own AI tools into the workplace without governance, according to Microsoft and LinkedIn data. The analogy to the UK's foot-and-mouth crisis is instructive — contingency plans assumed 10 infected premises; the reality was 57 by the time anyone looked. The UK's delayed response cost £8 billion and 6 million animals. The Netherlands contained the same outbreak in a month because they had existing frameworks and recent experience managing transitions.

   The Skill Compression Problem

   AI is also breaking your talent pipeline in ways most organizations haven't detected. When AI compresses visible skill differences — making a junior analyst's output look like a senior analyst's — your performance management and succession planning systems lose their signal. You can no longer distinguish between someone who produces good work with AI assistance and someone who possesses the underlying judgment to produce good work independently. The WEF's fastest-rising skills list confirms where the premium is heading: analytical thinking, creative thinking, resilience, leadership, empathy. Prompt engineering is conspicuously absent.

   The Block Signal in Context

   Block's AI-driven workforce compression was covered extensively in previous briefings. The new dimension this week: the Nature meta-analysis provides the scientific basis for why aggressive AI-driven headcount reduction carries hidden risk. If human-AI collaboration degrades judgment quality, then organizations that cut deepest may be simultaneously eliminating the human judgment capabilities that AI cannot replace — while the remaining humans become increasingly dependent on AI for decisions they should be making independently. The Air France 447 parallel is apt: pilots who lost manual flying skills because automation handled everything, until the moment automation failed and they couldn't recover.

   The strategic response isn't to slow AI adoption — it's to segment tasks by judgment intensity. Automate execution ruthlessly. But for high-judgment decisions, add friction rather than AI. Build deliberate 'manual flying' practice into your leadership development. And critically, redesign performance management to distinguish AI-augmented output from underlying human capability before you lose the ability to tell the difference.

   What to do

   1. →

      Commission an immediate audit of shadow AI usage — map tools, users, decision types, and data exposure across the organization by end of Q1

      Now78% adoption without governance means you have unmanaged risk that compounds daily; the UK foot-and-mouth lesson is that delay is the primary risk multiplier
   2. →

      Redesign performance evaluation criteria to distinguish AI-augmented output from underlying human judgment capability — implement for next review cycle

      This sprintAI-compressed skill visibility is already degrading your talent signal; waiting another cycle means promotion and succession decisions based on misleading data
   3. →

      Establish an AI deployment framework that segments tasks by judgment intensity — automate execution, add friction to high-judgment decisions

      This quarterThe Nature meta-analysis of 106 experiments shows copilots degrade judgment quality; deploying AI uniformly across all task types is actively harmful
   4. →

      Build a 'manual flying skills' program for high-potential leaders — deliberate practice in independent analysis, critical thinking, and judgment without AI assistance

      This quarterThe WEF data confirms judgment capabilities are rising in market value precisely because AI makes them scarcer — invest in what appreciates
4. 04

   Satellite Broadband's $50/Month Price Ceiling Is Coming for Terrestrial ISPs

   SpaceX is executing one of the most aggressive pre-IPO land grabs in recent tech history, and the second-order effects extend far beyond satellite internet. With 10 million subscribers, $50/month pricing tiers, free hardware giveaways on terminals that cost $600 to manufacture, Super Bowl advertising, and physical retail stores, Starlink has abandoned premium positioning and is sprinting toward mass-market telecom scale.

   The timing is strategic, not coincidental. Amazon's Project Kuiper (rebranded as Leo) is preparing for U.S. consumer launch later in 2026, and SpaceX is eyeing a summer 2026 IPO. Amazon insiders are reading SpaceX's price cuts as defensive — which validates that Kuiper is further along than public signals suggest. Apple's Globalstar partnership and China's Qianfan constellation add further competitive pressure.

   The Financial Architecture Is Precarious

   SpaceX is simultaneously absorbing what's described as 'massive cash burn' from xAI, Musk's AI venture. The xAI integration is visibly struggling — co-founder Toby Pohlen's departure weeks after receiving expanded responsibilities signals cultural collision, not planned transition. Layer on Starlink margin compression, retail capex, and Super Bowl-level marketing, and SpaceX needs its IPO to work — and soon. If public markets shift away from growth-over-profitability narratives, SpaceX faces a liquidity crunch across multiple Musk ventures simultaneously.

   The Structural Impact on Connectivity Economics

   For technology executives, the second-order effects matter most. Satellite broadband at $50/month fundamentally changes last-mile connectivity economics. It creates a price ceiling terrestrial ISPs cannot escape, compresses margins across the broadband value chain, and opens new market segments — rural enterprise, mobile workforce, IoT at scale — that were previously uneconomical. When two deep-pocketed competitors (SpaceX and Amazon) are racing to make broadband ubiquitous and cheap, everyone in between gets squeezed.

   When a market transitions from premium to commodity, the winners are those who either own the infrastructure at scale or own the distribution and bundling advantages. Everyone in between gets squeezed.

   What to do

   1. →

      Model the impact of $50/month satellite broadband as a structural price ceiling on any portfolio exposure to terrestrial broadband providers (AT&T, Comcast, regional ISPs)

      This quarterTwo deep-pocketed competitors racing to commoditize connectivity will compress margins across the entire broadband value chain within 18-24 months
   2. →

      If your business depends on connectivity infrastructure partnerships, open exploratory conversations with both Starlink and Amazon Leo now to leverage the competitive dynamic for favorable terms

      This quarterThe pre-IPO and pre-launch window is when both players are most motivated to lock in enterprise partnerships