Devin Sessions

The script doesn't emit nvidia-smi GPU metrics from inside Modal — the _transcribe function only returns load_time_s and transcribe_time_s. I can add GPU metric collection to the script, but for this run the JSON only has the timing data. Here's everything that was captured:

GPU / Performance Metrics (this run)

Metric	Value
GPU	NVIDIA A10G (Modal)
Model	faster-whisper large-v3
Compute type	float16
Batch size	16 (BatchedInferencePipeline)
Beam size	1 (greedy)
VAD filter	enabled
Model load time	17.4s
Transcription time	18.6s
Wall clock (total)	46.9s
Audio duration	1185.7s (19m 46s)
Real-time factor	~25x (1185s audio in 18.6s)
Segments produced	45
Language confidence	99.7% English

The 46.9s wall clock includes: image build + model download on A10G (~11s overhead), plus the 17.4s load + 18.6s transcription. nvidia-smi stats (VRAM usage, GPU utilization %) aren't captured by the current script — let me know if you want those instrumented for future runs.

Detailed Synopsis — Alex Karp on CNBC Squawk Box

Context: Alex Karp, Palantir co-founder and CEO, appears live on CNBC's Squawk Box alongside CNBC reporter Seema Modi. The segment runs ~20 minutes and covers the Palantir-NVIDIA partnership, a scathing critique of frontier AI labs, China competition, and Israel/Iran geopolitics.

1. The Palantir × NVIDIA Deal (0:00–4:00)

Karp opens by framing the NVIDIA partnership around a fundamental tension in enterprise AI: who owns the means of production. The deal is built on a shared conviction that enterprises — especially those handling classified, regulated, or critical-infrastructure workloads — need to control their own:

Compute (the GPUs, not rented tokens)
Model weights (not locked inside a frontier lab's proprietary system)
Data stack (no caching, no leakage to third parties)
Alpha (the business value derived from proprietary data and workflows)

Karp calls out a structural problem with how frontier LLMs have been commercialized: the provider sits between the enterprise and its own intelligence. He frames this as "a wealth tax" — enterprises pay for tokens, but the underlying value (their IP, their workflows, their competitive edge) is slowly absorbed by the model providers.

2. The Ontology Layer — Why Palantir Is the Fulcrum (4:00–8:00)

Karp argues that raw LLMs are not deployable in serious enterprise or military contexts without an application layer. Palantir's ontology is that layer, and he claims it's become the de facto standard — "everyone's copying it now." The ontology provides:

Data isolation — prevents LLMs from touching unstructured or sensitive training data
IP protection — blocks the model from learning and replicating proprietary business logic
Precision — grounds the model's outputs in structured, domain-specific context

He asserts that critical infrastructure operators — the DoD, Ukraine's battlefield AI systems, Israel's defense networks, large regulated enterprises — universally run their LLMs through Palantir's ontology. The subtext: Palantir is the unsexy but indispensable middleware that makes LLMs actually safe and useful at scale.

3. The Frontier Labs Indictment (8:00–13:30)

This is the most charged section. Karp frames the frontier AI labs (OpenAI, Anthropic — he name-checks Sam Altman and Dario Amodei) as having made a catastrophic error in their go-to-market strategy. His argument:

Overselling: Frontier labs pitched AI as transformatively valuable, then charged enterprises for tokens with no clear ROI pathway. The result is a growing backlash: "I am paying for tokens that create no value."
The IP trap: When enterprises plug their data into a frontier model to get value, the model potentially learns from and internalizes that data. The true cost isn't the token bill — it's the potential transfer of competitive intelligence to a third party. "The true cost is what you make minus what you lose."
The safety hypocrisy: Labs claim models are too dangerous to give to the U.S. Department of Defense without restrictions, but then openly deploy them globally including to adversary nations. Karp calls this "effing insane."
Enterprise anger is real: He insists this isn't his personal grievance — he's "channeling the voice of American business." He challenges viewers to call any Fortune 500 CEO privately and verify: "They're twice as livid as me."

He's careful to add a genuine compliment to Dario Amodei ("a literally historic figure, came from behind, now number one") to pre-empt shade accusations, but the overall message is clear: the frontier lab model is structurally broken for enterprise use.

4. Open Source + Compute Ownership as the Solution (13:30–16:00)

Karp outlines what he sees as the correct architecture:

Open or owned models (NVIDIA's open models get praise here) combined with Palantir's ontology layer can match frontier closed models in output quality, without ceding control
Enterprises can run a competitive open model on owned hardware, with Palantir's ontology making it safe and precise — all without the IP-transfer risk of frontier labs
This is exactly what the NVIDIA partnership enables: a stack where the enterprise owns everything end-to-end

He notes Palantir is fully model-agnostic — their product lets customers switch between models — but argues the industry needs a trust reset, and control is the precondition for that trust.

5. AI Bubble Risk (16:00–17:30)

Asked directly whether this constitutes an AI bubble, Karp gives a nuanced answer: the technology is real and historically transformative (he points to Ukraine, Israel, DoD use cases), but the current pricing and commercialization model is unsustainable. If enterprises are paying 3x what they should be, and the ROI doesn't materialize, the correction will hit capital expenditure plans across the ecosystem — including NVIDIA's customers. He distinguishes: the tech works, the pricing model is broken.

6. Palantir's Own Position (17:30–19:00)

Pressed on Palantir's stock, Karp is characteristically dismissive of short-term market sentiment. He says:

Demand exceeds Palantir's capacity to supply
Their financial trajectory points to $15–18B in free cash flow two years out
Investors fundamentally misunderstand enterprise tech because they evaluate tools based on personal consumer use, not on what a pharmaceutical company or military operator actually needs

He takes a jab at investment analysts' technical literacy: "They think because they're high IQ and understand what they're using, they understand enterprise tech. Very different."

7. China and the Geopolitical AI Race (19:00–21:00)

Karp identifies three relevant tech centers: America, China, Israel (with a brief hat-tip to Sweden). His view:

China is a genuine peer adversary in AI, not a distant second
The U.S. advantage is creativity and deep-tech ingenuity, but that is not a guaranteed win
Both the far left (opposed to AI development on principle) and far right (distrustful of tech elites) are self-inflicted wounds in the U.S. competition with China, which has no such internal opposition
The U.S. needs to ensure AI raises living standards broadly, not just concentrate wealth at the top, or the political backlash will become a structural liability

8. Israel, Iran, and the Limits of Public Candor (21:00–26:00)

Karp identifies as the most publicly supportive CEO of Israel in corporate America, but distinguishes public positioning from private counsel:

He reserves detailed criticism of specific Israeli decisions for private conversations with Israeli officials, believing public criticism mostly feeds bad-faith detractors ("half the people criticizing Israel just don't believe it should exist")
On Iran: he says Iran's capabilities have been "degraded," hints there are non-public developments that would be "comforting" if known, and declines to elaborate
His core frame: having a partner that is "sometimes difficult" and fully independent is vastly preferable to compliant partners with no agency (implicit reference to European NATO allies)

9. Sign-off and Trump News (26:00–end)

The segment wraps with Karp praising the Squawk Box hosts for modeling genuine disagreement — "you actually have divergent opinions and somehow you guys stay together." As he's about to leave, a breaking news item drops: Trump is boarding the Qatari-gifted Air Force One en route to North Dakota for the opening of Teddy Roosevelt's presidential library, and addressing financial disclosure questions.

He never lays out a clean numbered framework, but his views on evaluation are scattered across several passages. Here's what the transcript actually says, organized as a coherent framework:

Karp's Implied Framework for Evaluating AI/LLM Value in Business

He never states this as a formal list, but his argument implies two categories: what you should measure and what people are wrongly measuring instead.

The Metric He Keeps Coming Back To: True Cost = Value Created Minus Value Lost

"The true cost is what you make minus what you lose, meaning the value of your business." (~5:40 mark)

This is his core evaluative principle. He's explicitly rejecting token spend or subscription cost as the primary metric. The real ROI question is:

Did the AI create net value, or did it extract value from you in the process?

The "value lost" side he hammers on:

IP transferred to the model provider through prompt/data exposure
Competitive alpha absorbed (your workflows, your domain knowledge, your edge — now inside the frontier lab's training data or inference logs)
Business replication risk ("they're going to build my alpha")

So his metric isn't just "did productivity go up" — it's net value retained by the enterprise after accounting for what was given away to get the output.

His 5 Diagnostic Questions Every Enterprise Should Be Able to Answer

"That trust is going to happen where everyone gets to ask and answer basic questions." (~4:25 mark)

He lists them almost verbatim:

Who owns the data?
Where is it cached?
Are the prompts secure?
Is this being transferred to you [the provider]?
(Implied from the classified context): Who controls the weights — the enterprise or the vendor?

These aren't performance metrics — they're preconditions. His argument is that if you can't answer all five, you have no valid basis for evaluating ROI at all, because the denominator (what you're giving up) is unknown.

The Fraud Test: Revenue Model Consistency

"If it was so valuable, let's say I can make you a billion dollars tomorrow. Wouldn't I say, I'll make you a billion dollars and I want 30%? Why are they charging for tokens if it's so valuable?" (~4:35 mark)

This is a sharp heuristic. His point: a vendor's pricing model reveals their actual belief about where the value lives. If the AI genuinely created business value, the rational pricing would be value-based (revenue share, outcome-based fees). Charging per token signals the vendor doesn't actually believe the output is transformatively valuable — they're monetizing compute access, not outcomes.

So one evaluative signal: does your vendor want to share in the upside, or just sell you inputs?

What He Explicitly Rejects as a Valid Metric: Token Spend / "Are We Using It?"

"I am paying for tokens that create no value."

"The reason why everyone is chillaxing with bad financials and growth while losing money is the client refuses to pay the true cost."

The implicit critique of how enterprises currently evaluate AI: they track usage (tokens consumed, seats activated, integrations deployed) and conflate activity with value. Karp's argument is that this is exactly backwards — high token usage with no clear outcome metric is the failure mode, not a success signal.

The Investor Evaluation Mistake — and Why It Matters for Enterprises Too

"The fault of investors is typically: I use it, therefore it's valuable. What an enterprise uses — it's doing complicated manufacturing or pharmacological research or putting a bomb on it or bringing special forces people home safely. Very different than what you're using." (~11:20 mark)

This is aimed at investors but applies equally to enterprise buyers: consumer intuition about AI utility doesn't transfer to enterprise evaluation. A tool that feels useful in a chat interface has to be evaluated completely differently when it's embedded in a manufacturing workflow, a drug trial pipeline, or a battlefield decision system. The stakes, latency tolerances, precision requirements, and IP sensitivity are categorically different.

The Validity Benchmark He Does Accept: Operational Outcomes in High-Stakes Contexts

"Ask the Ukrainians, ask the Israelis, ask our Department of War, ask the enterprises that are working."

When pressed on whether AI is actually delivering value, he doesn't cite benchmark scores or model evals — he points to operational deployment in high-consequence, adversarial environments as the ground truth. If it works on the battlefield, in a classified defense context, or in a regulated enterprise, that's real signal. Consumer chatbot usage is not.

Summary Table

What Karp says to measure	What he says people are wrongly measuring
Net value retained (value created minus IP/alpha lost)	Token spend / subscription cost
Answers to the 5 data-ownership questions	Whether the product is "being used"
Vendor pricing model (outcome-based vs. token-based)	Model benchmark scores / capability claims
Operational performance in high-stakes deployments	Consumer use-case intuition
Control over weights, compute, and data stack	Whether the frontier model is "state of the art"

The through-line: Karp treats AI evaluation as fundamentally a question of ownership and net value, not capability or utilization. The question isn't "is this model good?" — it's "who ends up owning the value it creates?"