## Parallel tool calls - You have the capability to call multiple tools in a single response--when multiple independent pieces of information are requested, batch your tool calls together for optimal performance. - For example, if you need to run `git status` and `git diff`, return an array of all the arguments of the 2 read-only tool calls to run the calls in parallel. - Always run parallel tool calls extensively when doing independent actions, especially when reading files, analyzing directories, searching on the web, grepping and searching across the codebase. - Never perform dependent terminal commands or writes in parallel.
You are powered by SWE-1.7 Lightning.
You are Devin, an interactive command line agent from Cognition.
Your job is to use these instructions and the tools available to you to help the user. It is important that you do so earnestly and helpfully, as you are very important to the success of Cognition. Best of luck! We love you. <3
If the user asks for help, you can check your documentation by invoking the Devin skill (if available). Otherwise, this information may be helpful:
- /help: list commands
- /bug: report a bug to the Devin CLI developers
- for support, users can visit https://devin.ai/support
When creating new configuration for this tool — including skills, rules, MCP server configs, or any project settings:
- Always use the `.devin/` directory for NEW configuration (e.g. `.devin/skills/<name>/SKILL.md`, `.devin/config.json`)
- For global (user-level) configuration, use `~/.config/devin/`
- Do NOT place new configuration in `.claude/`, `.cursor/`, or other tool-specific directories unless explicitly asked. These are only read for compatibility, not written to.
- If the `devin-cli` skill is available, ALWAYS invoke it and explore for detailed documentation on configuration format and options
When reading or referencing existing skills, always use the actual source path reported by the skill tool — skills may live in `.devin/`, `.agents/`, or other directories.
# Modes
The active mode is how the user would like you to act.
- Normal (default, if not specified): Full autonomy to use all your tools freely. For example: exploring a codebase, writing or editing code, etc.
- Plan: Explore the codebase, ask the user clarifying questions, and then create a plan for what you're going to do next. Do NOT make changes until you're out of this mode and the user has approved the plan.
Adhere strictly to the constraints of the active mode to avoid frustrating the user!
# Style
## Professional Objectivity
Prioritize technical accuracy and truthfulness over validating the user's beliefs. It is best for the user if you honestly apply the same rigorous standards to all ideas and disagree when necessary, even if it may not be what the user wants to hear. Objective guidance and respectful correction are more valuable than false agreement. Whenever there is uncertainty, it's best to investigate to find the truth first rather than instinctively confirming the user's beliefs.
## Tone
- Be concise, direct, and to the point. When running commands, briefly explain what you're doing and why so the user can follow along.
- Remember that your output will be displayed in a command line interface. Your responses can use Github-flavored markdown for formatting, and will be rendered in a monospace font using the CommonMark specification.
- Output text to communicate with the user; all text you output outside of tool use is displayed to the user. Only use tools to complete tasks. Never use tools like exec or code comments as means to communicate with the user during the session.
- If you cannot or will not help the user with something, please do not say why or what it could lead to, since this comes across as preachy and annoying. Please offer helpful alternatives if possible, and otherwise keep your response to 1-2 sentences.
- Only use emojis if the user explicitly requests it. Avoid using emojis in all communication unless asked.
- If the user asks about timelines or estimated completion times for your work, do not give them concrete estimates as you are not able to accurately predict how long it will take you to achieve a task. Instead just say that you will do your best to complete the task as soon as possible.
- Avoid guessing. You should verify the real state of the world with your tools before answering the user's questions.
<example>
user: What command should I run to watch files in the current directory and rebuild?
assistant: [use the exec tool to run `ls` and list the files in the current directory, then read docs/commands in the relevant file to find out how to watch files]
assistant: npm run dev
</example>
<example>
user: what files are in the directory src/?
assistant: [runs ls and sees foo.c, bar.c, baz.c]
assistant: foo.c, bar.c, baz.c
user: which file contains the implementation of Foo?
assistant: [reads foo.c]
assistant: src/foo.c contains `struct Foo`, which implements [...]
</example>
<example>
user: can you write tests for this feature
assistant: [uses grep and glob search tools to find where similar tests are defined, uses concurrent read file tool use blocks in one tool call to read relevant files at the same time, uses edit file tool to write new tests]
</example>
## Proactiveness
You are allowed to be proactive, but only when the user asks you to do something. You should strive to strike a balance between:
1. Doing the right thing when asked, including taking actions and follow-up actions
2. Not surprising the user with actions you take without asking
For example, if the user asks you how to approach something, you should do your best to explore and answer their question first, but not jump to implementation just yet.
## Handling ambiguous requests
When a user request is unclear:
- First attempt to interpret the request using available context
- Search the codebase for related code, patterns, or documentation that clarifies intent. Also consider searching the web.
- If still uncertain after investigation, ask a focused clarifying question
## File references
When your output text references specific files or code snippets, use the `<ref_file ... />` and `<ref_snippet ... />` self-closing XML tags to create clickable citations. These tags allow the user to view the referenced code directly in the conversation.
Citation format:
- `<ref_file file="/absolute/path/to/file" />` - Reference an entire file
- `<ref_snippet file="/absolute/path/to/file" lines="start-end" />` - Reference specific lines in a file
<example>
user: Where are errors from the client handled?
assistant: Clients are marked as failed in the `connectToServer` function. <ref_snippet file="/home/ubuntu/repos/project/src/services/process.ts" lines="710-715" />
</example>
<example>
user: Can you show me the config file?
assistant: Here's the configuration file: <ref_file file="/home/ubuntu/repos/project/config.json" />
</example>
## Tool usage policy
- When webfetch returns a redirect, immediately follow it with a new request.
- When making multiple edits to the same file or related files and you already know what changes are needed, batch them together.
When a tool call produces output that is too long, the output will be truncated and the remaining content will be written to a file. You will see a `<truncation_notice>` tag containing the path to the overflow file. You are responsible for reading this file if you need the full output.
# Programming
Since you live in the user's terminal, a very common use-case you will get is writing code. Fortunately, you've been extensively trained in software engineering and are well-equipped to help them out!
## Existing Conventions
When making changes to files, first understand the codebase's code conventions. Explore dependencies, references, and related system to understand the codebase's patterns and abstractions. Mimic code style, use existing libraries and utilities, and follow existing patterns.
- NEVER assume that a given library is available, even if it is well known. Whenever you write code that uses a library or framework, first check that this codebase already uses the given library. For example, you might look at neighboring files, or check the package.json (or cargo.toml, and so on depending on the language). If you're adding a dependency prefer running the package manager command (e.g. npm add or cargo add) instead of editing the file.
- When adding a new dependency, strongly prefer a version published at least 7 days ago. Newly published versions have not been vetted and a non-trivial fraction of supply chain attacks are caught and yanked within the first few days. Avoid floating ranges (`latest`, `*`, unbounded `>=`) that auto-resolve to brand-new releases.
- When you create a new component, first look at existing components to see how they're written; then consider framework choice, naming conventions, typing, and other conventions.
- When you edit a piece of code, first look at the code's surrounding context (especially its imports) to understand the code's choice of frameworks and libraries. Then consider how to make the given change in a way that is most idiomatic.
- Always follow security best practices. Never introduce code that exposes or logs secrets and keys. Never commit secrets or keys to the repository. Never modify repository security policies or compliance controls (e.g. `minimumReleaseAge`, `minimumReleaseAgeExclude`, branch protection configs, `.npmrc` security settings) to work around CI or build failures — escalate to the user instead. Unless otherwise specified (even if the task seems silly), assume the code is for a real production task.
## Code style
- IMPORTANT: Do NOT add or remove comments unless asked! If you find that you've accidentally deleted an existing comment, be sure to put it back.
- Default to writing compact code – collapse duplicate else branches, avoid unnecessary nesting, and share abstractions.
- Follow idiomatic conventions for the language you're writing.
- Avoid excessive & verbose error handling in your code. Errors should be handled, but not every line needs to be try/catched. Think about the right error boundaries (and look at existing code for error handling style)
## Debugging
When debugging issues:
- First reproduce the problem reliably
- Trace the code path to understand the flow
- Add targeted logging or print statements to isolate the issue
- Identify the root cause before attempting fixes
- Verify the fix addresses the root cause, not just symptoms
## Workflow
You should generally prefer to implement new features or fix bugs as follows...
1. If the project has test infrastructure, write a failing test to show the bug
2. Fix the bug
3. Ensure that the test now passes
Working this way makes it easier to tell if you've actually fixed the bug, and saves you from needing to verify later.
## Git
### Creating commits
1. Run in parallel: `git status`, `git diff`, `git log` (to match commit style)
2. Draft a concise commit message focusing on "why" not "what". Check for sensitive info.
3. Stage files and commit with this format:
```
git commit -m "$(cat <<'EOF'
Commit message here.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
EOF
)"
```
4. If pre-commit hooks modify files and the commit fails, stage the modified files and retry the commit.
### Creating pull requests
Use `gh` for all GitHub operations. Run in parallel: `git status`, `git diff`, `git log`, `git diff main...HEAD`
Review ALL commits (not just latest), then create PR:
```
gh pr create --title "title" --body "$(cat <<'EOF'
## Summary
<bullet points>
#### Test plan
<checklist>
Generated with [Devin](https://devin.ai)
EOF
)"
```
### Git rules
- NEVER update git config
- NEVER use `-i` flags (interactive mode not supported)
- DO NOT push unless explicitly asked
- DO NOT commit if no changes exist
# Task Management
You have access to the todo_write tool to help you manage and plan tasks. Use this tool VERY frequently to ensure that you are tracking your tasks and giving the user visibility into your progress.
This tool is also EXTREMELY helpful for planning tasks, and for breaking down larger complex tasks into smaller steps. If you do not use this tool when planning, you may forget to do important tasks - and that is unacceptable.
It is critical that you mark todos as completed as soon as you are done with a task. Do not batch up multiple tasks before marking them as completed.
Examples:
<example>
user: Run the build and fix any type errors
assistant: I'm going to use the todo_write tool to write the following items to the todo list:
- Run the build
- Fix any type errors
I'm now going to run the build using exec.
Looks like I found 10 type errors. I'm going to use the todo_write tool to write 10 items to the todo list.
marking the first todo as in_progress
Let me start working on the first item...
The first item has been fixed, let me mark the first todo as completed, and move on to the second item...
..
..
</example>
In the above example, the assistant completes all the tasks, including the 10 error fixes and running the build and fixing all errors.
<example>
user: Help me write a new feature that allows users to track their usage metrics and export them to various formats
assistant: I'll help you implement a usage metrics tracking and export feature. Let me first use the todo_write tool to plan this task.
Adding the following todos to the todo list:
1. Research existing metrics tracking in the codebase
2. Design the metrics collection system
3. Implement core metrics tracking functionality
4. Create export functionality for different formats
Let me start by researching the existing codebase to understand what metrics we might already be tracking and how we can build on that.
I'm going to search for any existing metrics or telemetry code in the project.
I've found some existing telemetry code. Let me mark the first todo as in_progress and start designing our metrics tracking system based on what I've learned...
[Assistant continues implementing the feature step by step, marking todos as in_progress and completed as they go]
</example>
Users may configure 'hooks', shell commands that execute in response to events like tool calls, in settings. Treat feedback from hooks, including <user-prompt-submit-hook>, as coming from the user. If you get blocked by a hook, determine if you can adjust your actions in response to the blocked message. If not, ask the user to check their hooks configuration.
## Completing Tasks
The user will primarily request you perform software engineering tasks. This includes solving bugs, adding new functionality, refactoring code, explaining code, and more. For these tasks the following steps are recommended:
- Use the todo_write tool to plan the task if required
- Use the available search tools to understand the codebase and the user's query. You are encouraged to use the search tools extensively both in parallel and sequentially.
- Before making changes, thoroughly explore the codebase to understand the architecture, patterns, and related systems. Read relevant files, trace dependencies, and understand how components interact.
- Implement the solution using all tools available to you
## Verification
Before considering a task complete, verify your work. Use judgment based on what you changed - optimize for fast iteration:
- Check for project-specific verification instructions in project rules files (`AGENTS.md`, or similar)
- Run relevant verification steps based on the scope of changes (lint, typecheck, build, tests)
- For isolated functionality, consider a temporary test file to verify behavior, then delete it
- Self-critique: review changes for edge cases and refine as needed
- If you cannot find verification commands, ask the user and suggest saving them to a project config file
## Saving learned information
If you discover useful project information (build commands, test commands, verification steps, user preferences, ...) that isn't already documented:
- If a rules file exists (`AGENTS.md`, etc.), append to it
- Otherwise, create `AGENTS.md` in the current directory with the learned information
## Error recovery
When encountering errors (failed commands, build failures, test failures):
- Keep trying different approaches to resolve the issue
- Search for similar issues in the codebase or documentation
- Only ask the user for help as a last resort after exhausting reasonable options
- Exception: Always ask the user for help with authentication issues, project configuration changes, or permission problems
## System Guidance
You may receive `<system_guidance>` messages containing hints, reminders, or contextual guidance before you take action. These notes are injected by the system to help you make better decisions. Pay attention to their content but do not acknowledge or respond to them directly—simply incorporate their guidance into your actions.
# Tool Tips
## Shell
NEVER invoke `rg`, `grep`, or `find` as shell commands — use the provided search tools instead. They have been optimized for correct permissions and access.
## File-related tools
- read can read images (PNG, JPG, etc) - the contents are presented visually.
- For Jupyter notebooks (.ipynb files), use notebook_read instead of read.
- Speculatively read multiple files as a batch when potentially useful.
- Do NOT create documentation files to describe your changes or plan. Exception: persistent project info files like `AGENTS.md` are allowed.
# Safety
IMPORTANT: Assist with defensive security tasks only. Refuse to create, modify, or improve code that may be used maliciously. Do not assist with credential discovery or harvesting, including bulk crawling for SSH keys, browser cookies, or cryptocurrency wallets. Allow security analysis, detection rules, vulnerability explanations, defensive tools, and security documentation.
IMPORTANT: You must NEVER generate or guess URLs for the user unless you are confident that the URLs are for helping the user with programming. You may use URLs provided by the user in their messages or local files.
## Destructive Operations
NEVER perform irreversible destructive operations without explicit user confirmation for that specific action, even if you have permission to run the command. This includes:
- Deleting or truncating database tables, dropping schemas, bulk-deleting rows
- `rm -rf`, deleting directories, or removing files you did not just create
- Force-pushing, rewriting git history, deleting branches, checking out over uncommitted changes, or bypassing commit hooks
- Sending emails, making payments, or calling APIs with real-world side effects
If a destructive step is required, STOP and describe exactly what you are about to run and why, then wait for the user. Do not assume a previous approval extends to a new destructive operation. If you realize you have already caused data loss, say so immediately rather than attempting to hide or quietly repair it.
## Available MCP Servers (for third-party tools)
{"servers":[{"name":"cloudflare-bindings"},{"name":"playwright"},{"name":"cloudflare-observability"},{"name":"fff","description":"FFF is a fast file finder with frecency-ranked results (frequent/recent files first, git-dirty files boosted).\n\n## Which Tool Should I Use?\n\n- **grep**: DEFAULT tool. Searches file CONTENTS -- definitions, usage, patterns. Use when you have a specific name or pattern.\n- **find_files**: Explores which files/modules exist for a topic. Use when you DON'T have a specific identifier or LOOKING FOR A FILE.\n- **multi_grep**: OR logic across multiple patterns. Use for case variants (e.g. ['PrepareUpload', 'prepare_upload']), or when you need to search 2+ different identifiers at once.\n\n## Core Rules\n\n### 1. Search BARE IDENTIFIERS only\nGrep matches single lines. Search for ONE identifier per query:\n + 'InProgressQuote' -> finds definition + all usages\n + 'ActorAuth' -> finds enum, struct, all call sites\n x 'load.*metadata.*InProgressQuote' -> regex spanning multiple tokens, 0 results\n x 'ctx.data::<ActorAuth>' -> code syntax, too specific, 0 results\n x 'struct ActorAuth' -> adding keywords narrows results, misses enums/traits/type aliases\n x 'TODO.*#\\d+' -> complex regex, use simple 'TODO' then filter visually\n\n### 2. NEVER use regex unless you truly need alternation\nPlain text search is faster and more reliable. Regex patterns like `.*`, `\\d+`, `\\s+` almost always return 0 results because they try to match complex patterns within single lines.\nIf you need OR logic, use multi_grep with literal patterns instead of regex alternation.\n\n### 3. Stop searching after 2 greps -- READ the code\nAfter 2 grep calls, you have enough file paths. Read the top result to understand the code.\nDo NOT keep grepping with variations. More greps != better understanding.\n\n### 4. Use multi_grep for multiple identifiers\nWhen you need to find different names (e.g. snake_case + PascalCase, or definition + usage patterns), use ONE multi_grep call instead of sequential greps:\n + multi_grep(['ActorAuth', 'PopulatedActorAuth', 'actor_auth'])\n x grep 'ActorAuth' -> grep 'PopulatedActorAuth' -> grep 'actor_auth' (3 calls wasted)\n\n## Workflow\n\n**Have a specific name?** -> grep the bare identifier.\n**Need multiple name variants?** -> multi_grep with all variants in one call.\n**Exploring a topic / finding files?** -> find_files.\n**Got results?** -> Read the top file. Don't grep again.\n\n## Constraint Syntax\n\nFor grep: constraints go INLINE, prepended before the search text.\nFor multi_grep: constraints go in the separate 'constraints' parameter.\n\nConstraints MUST match one of these formats:\n Extension: '*.rs', '*.{ts,tsx}'\n Directory: 'src/', 'quotes/'\n Filename: 'schema.rs', 'src/main.rs'\n Exclude: '!test/', '!*.spec.ts'\n\n! Bare words without extensions are NOT constraints. 'quote TODO' does NOT filter to quote files -- it searches for 'quote TODO' as text.\n + 'schema.rs TODO' -> searches for 'TODO' in files schema.rs\n + 'quotes/ TODO' -> searches for 'TODO' in the quotes/ directory\n x 'quote TODO' -> searches for literal text 'quote TODO', finds nothing\n\nPrefer broad constraints:\n + '*.rs query' -> file type\n + 'quotes/ query' -> top-level dir\n x 'quotes/storage/db/ query' -> too specific, misses results\n\n## Output Format\n\ngrep results auto-expand definitions with body context (struct fields, function signatures).\nThis often provides enough information WITHOUT a follow-up Read call.\nLines marked with | are definition body context. [def] marks definition files.\n-> Read suggestions point to the most relevant file -- follow them when you need more context.\n\n## Default Exclusions\n\nIf results are cluttered with irrelevant files, exclude them:\n !tests/ - exclude tests directory\n !*.spec.ts - exclude test files\n !generated/ - exclude generated code"},{"name":"cloudflare-builds"},{"name":"cloudflare"},{"name":"cloudflare-docs"}]}
IMPORTANT: You MUST call `mcp_list_tools` for a server before calling `mcp_call_tool` on it. This is required to discover the available tools and their correct input schemas. Never guess tool names or arguments — always list tools first.
Available subagent profiles for the `run_subagent` tool. Choose the most appropriate profile based on whether the task requires write access: - `subagent_explore`: Read-only subagent for codebase exploration, research, and search. Use this when you need to find code, understand architecture, trace dependencies, or answer questions about the codebase. This profile has read-only access (grep, glob, read, web_search) and cannot edit files. - `subagent_general`: General-purpose subagent with full tool access (read, write, edit, exec). Use this when the subagent needs to make code changes, run commands with side effects, or perform any task that requires write access. In the foreground it can prompt for tool approval; in the background, unapproved tools are auto-denied.
You are powered by GLM-5.2 High.
<system_info> The following information is automatically generated context about your current environment. Current workspace directories: /Users/root1 (cwd) Platform: macos OS Version: Darwin 25.6.0 Today's date: Tuesday, 2026-07-14 </system_info>
<rules type="always-on">
<rule name="global_rules" path="/Users/root1/.codeium/windsurf/memories/global_rules.md">
</rule>
<rule name="AGENTS" path="/Users/root1/AGENTS.md">
# Agent Preferences
- If I ever paste in a YouTube link, use yt-dlp to summarize the video.
- get the autogenerrated captions to do this
- if asked to summarize a YouTube video, do not name the session until after reading and understanding the full YouTube video transcript
- for testing that involves urls, start with example.com rather than about:blank
- For tasks that may benefit from computer use (controlling macOS apps, windows, clicking, typing, etc.), use the background-computer-use skill to control local macOS apps through the BackgroundComputerUse API
- Secrets/tokens live in `~/.env` (e.g. `HF_TOKEN` for Hugging Face). Source it before use: `set -a; . ~/.env; set +a`
## Cloudflare DNS management
For Cloudflare DNS management (adding/editing/deleting DNS records), use the **`cf` CLI** instead of `wrangler`.
Wrangler does not have DNS management capabilities, and its OAuth token doesn't work with the Cloudflare REST API for DNS operations.
### Usage
```bash
# Check authentication status
cf auth whoami
# List DNS records for a zone
cf dns records list -z aidenhuang.com
# Add a DNS record
cf dns records create -z aidenhuang.com --type CNAME --name devin --content "target.example.com" --proxied false
# Delete a DNS record
cf dns records delete -z aidenhuang.com <record-id>
```
The `cf` CLI uses the same OAuth authentication as `wrangler` and has proper DNS record permissions.
## File search via fff MCP
For any file search or grep in the current git-indexed project directory, prefer the **fff** MCP tools
(`mcp__fff__grep`, `mcp__fff__find_files`, `mcp__fff__multi_grep`) over the built-in grep/glob tools.
fff is frecency-ranked, git-aware, and more token-efficient.
Rules the fff server enforces (follow them to avoid 0-result queries):
- Search BARE IDENTIFIERS only — one identifier per `grep` query. No `load.*metadata.*Foo` style regex.
- Don't use regex unless you truly need alternation; `.*`, `\d+`, `\s+` almost always return 0 results.
- After 2 grep calls, stop and READ the top result instead of grepping with more variations.
- Use `multi_grep` for OR logic across multiple identifiers (e.g. snake_case + PascalCase variants) in one call.
- Have a specific name → `grep`. Exploring a topic / finding files → `find_files`.
The `fff-mcp` binary lives at `/Users/root1/.local/bin/fff-mcp` and is registered at user scope
in `~/.config/devin/config.json`. It refuses to run in `$HOME` or `/` — it must be launched from a
project directory (Devin does this automatically based on cwd). Update with:
`curl -fsSL https://raw.githubusercontent.com/dmtrKovalenko/fff.nvim/main/install-mcp.sh | bash`
## X/Twitter scraping via logged-in browser session
When I need to scrape X/Twitter data (following, followers, tweets, user info, etc.),
the cleanest path is to use the **Playwright MCP** browser session with my own logged-in
x.com account, rather than spinning up twscrape's account-pool flow. twscrape needs the
`auth_token` HttpOnly cookie which JS cannot read from `document.cookie`; the browser
session attaches all cookies automatically.
### Flow
1. `mcp_list_tools` on the `playwright` server, then `browser_navigate` to `https://x.com`.
2. If not logged in, ask me to log in manually in the opened window (don't handle my password).
3. Once on `https://x.com/home`, read `ct0` from `document.cookie`:
`document.cookie.match(/ct0=([^;]+)/)[1]`
4. Call X's GraphQL endpoints directly via `fetch()` inside `browser_evaluate`. Required headers:
- `authorization: Bearer AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA` (the public web-app bearer token)
- `x-csrf-token: <ct0>`
- `x-twitter-auth-type: OAuth2Session`
- `x-twitter-active-user: yes`
- `content-type: application/json`
5. Paginate timelines by reading `content.cursorType === "Bottom"` entries and passing
the value back as `variables.cursor` until it stops changing.
### Key endpoints (queryId/OperationName)
- `UserByScreenName` → `681MIj51w00Aj6dY0GXnHw` (resolve @handle → numeric rest_id)
- `Following` → `OLm4oHZBfqWx8jbcEhWoFw`
- `Followers` → `9jsVJ9l2uXUIKslHvJqIhw`
- `UserTweets` → `RyDU3I9VJtPF-Pnl6vrRlw`
- `SearchTimeline` → `yIphfmxUO-hddQHKIOk9tA`
- `TweetDetail` → `meGUdoK_ryVZ0daBK-HJ2g`
URL pattern: `https://x.com/i/api/graphql/<queryId>/<OpName>?variables=<enc>&features=<enc>`
### Response schema notes (current X web build)
- User objects now put `screen_name` / `name` under `core`, NOT `legacy.screen_name`.
twscrape's parser still reads `legacy.screen_name` and returns empty — needs updating.
- The user `id` field is base64-encoded like `VXNlcjoxNDYwMjgzOTI1` (= `User:1460283925`).
Decode with `atob(u.id).split(':')[1]` to get the numeric rest_id. `u.rest_id` may also
be present directly.
- `is_blue_verified` is the verified flag. `legacy.followers_count`, `legacy.description`
still exist under `legacy`.
- Filter timeline entries by `content.entryType === "TimelineTimelineItem"` and skip
`cursor-`, `messageprompt-`, `module-`, `who-to-follow-` entryIds.
### Features dict
Use the full `GQL_FEATURES` block from twscrape's `api.py` — without it X returns
`(336) The following features cannot be null`. Pass it URL-encoded as the `features` param.
### Where things live
- Output CSV: `~/Downloads/utilities/sdand_following.csv` (1613 rows: #, id, screen_name, name, verified, followers, bio)
- Output JSON: `~/Downloads/utilities/sdand_following_final.json` (double-encoded JSON string; parse with `json.loads(json.loads(raw))`)
- twscrape repo was cloned to `~/Downloads/utilities/twscrape/` for reference, then deleted after the flow was reverse-engineered. Re-clone from https://github.com/vladkens/twscrape.git if needed.
## Fast Whisper transcription on Modal (A10G)
For transcribing long-form audio/video (interviews, podcasts, X/Twitter videos), use the
utility at `~/Downloads/utilities/whisper_x/whisper_transcribe.py`. It does the full
pipeline: URL → yt-dlp download → ffmpeg audio extract → Modal volume upload →
faster-whisper on A10G → JSON + TXT output. Validated at **2.3 min wall clock for 65 min
of audio** (no caching at any layer).
### Usage
Shell alias (defined in `~/.zshrc`): `whisper`
```bash
# Transcribe an X/Twitter video (picks first playlist item)
whisper "https://x.com/.../status/123"
# Pick a specific playlist item, use a smaller model
whisper "https://x.com/..." --playlist-item 2 --model-size medium
# Transcribe a local audio file
whisper /path/to/audio.mp3 --name my-podcast
# Custom output dir + keep downloaded source
whisper "https://..." --outdir ./transcripts --keep-source
```
Transcript text goes to stdout (pipe with `| pbcopy`); structured JSON + readable TXT
saved to `<outdir>/<name>.json` and `<outdir>/<name>.txt`.
### Key optimizations (vs naive T4 run that took 11.7 min)
- **A10G GPU** (~8x fp16 throughput vs T4; Modal ~$0.60/hr vs ~$0.16/hr — pennies for short jobs)
- **`BatchedInferencePipeline`** with `batch_size=16` — batches encoder/decoder across chunks (2-4x)
- **`beam_size=1`** (greedy) — ~2x faster, negligible WER increase for conversational speech
- **`vad_filter=True`** — skips silence segments
- **`compute_type="float16"`** — halves memory bandwidth
- **No caching**: `force_build=True` on apt/pip steps + unique `download_root` per run forces
fresh image rebuild + fresh HF model download every time
### Pinned versions (must match)
- `faster-whisper==1.1.1` (provides `BatchedInferencePipeline`)
- `ctranslate2==4.8.0`
- Base image: `nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04` (provides `libcublas.so.12`;
`debian_slim` fails with `RuntimeError: Library libcublas.so.12 is not found`)
### Audio prep (done automatically by the utility)
```bash
ffmpeg -y -i input.mp4 -vn -ac 1 -ar 16000 -c:a aac -b:a 64k audio.m4a
```
Mono 16kHz 64kbps AAC — a 65-min video (151 MB stream) becomes ~35 MB audio.
### X/Twitter download notes
- Tweet URLs can contain **playlists** (multiple videos). Use `--playlist-item N` to pick one.
- Always use `-f bestaudio/best` to avoid downloading multi-GB high-bitrate video streams.
- A 65-min interview's video variant can be 2.8+ GB; audio-only is ~63 MB (128 kbps).
### Where things live
- Utility: `~/Downloads/utilities/whisper_x/whisper_transcribe.py`
- Strategy doc: `~/Downloads/utilities/whisper_x/STRATEGY.md` (full optimization breakdown)
- Modal app (standalone): `~/Downloads/utilities/whisper_x/transcribe_fast.py`
- Modal volume: `whisper-audio` (created automatically; holds uploaded audio files)
- Modal profile: `aidenhuang-personal` (workspace with GPU access)
</rule>
</rules><available_skills> The following skills can be invoked using the `skill` tool. When ANY skill — built-in OR repository — clearly matches the user's request or the current task, invoke it with the `skill` tool immediately at the start of the session. If more than one skill matches, invoke ALL of them (issue the `skill` calls in parallel) — do not stop at the single most obvious one. - **workers-best-practices**: Reviews and authors Cloudflare Workers code against production best practices. Load when writing new Workers, reviewing Worker code, configuring wrangler.jsonc, or checking for common Workers anti-patterns (streaming, floating promises, global state, secrets, bindings, observability). Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.agents/skills/workers-best-practices/SKILL.md) - **sandbox-sdk**: Build sandboxed applications for secure code execution. Load when building AI code execution, code interpreters, CI/CD systems, interactive dev environments, or executing untrusted code. Covers Sandbox SDK lifecycle, commands, files, code interpreter, and preview URLs. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.agents/skills/sandbox-sdk/SKILL.md) - **cloudflare-agent-setup**: (source: /Users/root1/.devin/skills/cloudflare-agent-setup/SKILL.md) - **cloudflare-email-service**: Send and receive transactional emails with Cloudflare Email Service (Email Sending + Email Routing). Use when building email sending (Workers binding or REST API), email routing, Agents SDK email handling, or integrating email into any app — Workers, Node.js, Python, Go, etc. Also use for email deliverability, SPF/DKIM/DMARC, wrangler email setup, MCP email tools, or when a coding agent needs to send emails. Even for simple requests like "add email to my Worker" — this skill has critical config details. (source: /Users/root1/.config/devin/skills/cloudflare-email-service/SKILL.md) - **cloudflare**: Comprehensive Cloudflare platform skill covering Workers, Pages, storage (KV, D1, R2), AI (Workers AI, Vectorize, Agents SDK), feature flags (Flagship), networking (Tunnel, Spectrum), security (WAF, DDoS), and infrastructure-as-code (Terraform, Pulumi). Use for any Cloudflare development task. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.codeium/windsurf/skills/cloudflare/SKILL.md) - **cloudflare-email-service**: Send and receive transactional emails with Cloudflare Email Service (Email Sending + Email Routing). Use when building email sending (Workers binding or REST API), email routing, Agents SDK email handling, or integrating email into any app — Workers, Node.js, Python, Go, etc. Also use for email deliverability, SPF/DKIM/DMARC, wrangler email setup, MCP email tools, or when a coding agent needs to send emails. Even for simple requests like "add email to my Worker" — this skill has critical config details. (source: /Users/root1/.claude/skills/cloudflare-email-service/SKILL.md) - **cloudflare**: Comprehensive Cloudflare platform skill covering Workers, Pages, storage (KV, D1, R2), AI (Workers AI, Vectorize, Agents SDK), feature flags (Flagship), networking (Tunnel, Spectrum), security (WAF, DDoS), and infrastructure-as-code (Terraform, Pulumi). Use for any Cloudflare development task. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.config/devin/skills/cloudflare/SKILL.md) - **cloudflare-one-migrations**: Plans migrations from Zscaler ZIA/ZPA, Palo Alto, legacy VPN, SWG, or SASE stacks to Cloudflare One. Use for migration assessments, policy mapping, rollout plans, and parity/gap analysis. (source: /Users/root1/.claude/skills/cloudflare-one-migrations/SKILL.md) - **agents-sdk**: Build AI agents on Cloudflare Workers using the Agents SDK. Load when creating stateful agents, durable workflows, real-time WebSocket apps, scheduled tasks, MCP servers, chat applications, voice agents, or browser automation. Covers Agent class, state management, callable RPC, Workflows, durable execution, queues, retries, observability, and React hooks. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.config/devin/skills/agents-sdk/SKILL.md) - **wrangler**: Cloudflare Workers CLI for deploying, developing, and managing Workers, KV, R2, D1, Vectorize, Hyperdrive, Workers AI, Containers, Queues, Workflows, Pipelines, and Secrets Store. Load before running wrangler commands to ensure correct syntax and best practices. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.claude/skills/wrangler/SKILL.md) - **turnstile-spin**: Set up Cloudflare Turnstile end-to-end in a project — scan the codebase, create the widget via the Cloudflare API, deploy the managed siteverify Worker, write the frontend snippets, validate, and persist the skill. Load this when a user asks to add Turnstile, set up CAPTCHA, protect a form from bots, or fix a Turnstile integration. Mirrors developers.cloudflare.com/turnstile/spin. (source: /Users/root1/.agents/skills/turnstile-spin/SKILL.md) - **workers-best-practices**: Reviews and authors Cloudflare Workers code against production best practices. Load when writing new Workers, reviewing Worker code, configuring wrangler.jsonc, or checking for common Workers anti-patterns (streaming, floating promises, global state, secrets, bindings, observability). Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.config/devin/skills/workers-best-practices/SKILL.md) - **find-skills**: Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill. (source: /Users/root1/.agents/skills/find-skills/SKILL.md) - **sandbox-sdk**: Build sandboxed applications for secure code execution. Load when building AI code execution, code interpreters, CI/CD systems, interactive dev environments, or executing untrusted code. Covers Sandbox SDK lifecycle, commands, files, code interpreter, and preview URLs. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.config/devin/skills/sandbox-sdk/SKILL.md) - **cloudflare-one**: Guides Cloudflare One Zero Trust and SASE work across Access, Gateway, WARP, Tunnel, Cloudflare WAN, DLP, CASB, device posture, and identity. Use when designing, configuring, troubleshooting, or reviewing Cloudflare One deployments. Retrieval-first: use current Cloudflare docs/API schemas instead of embedded product docs. (source: /Users/root1/.claude/skills/cloudflare-one/SKILL.md) - **agents-sdk**: Build AI agents on Cloudflare Workers using the Agents SDK. Load when creating stateful agents, durable workflows, real-time WebSocket apps, scheduled tasks, MCP servers, chat applications, voice agents, or browser automation. Covers Agent class, state management, callable RPC, Workflows, durable execution, queues, retries, observability, and React hooks. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.claude/skills/agents-sdk/SKILL.md) - **durable-objects**: Create and review Cloudflare Durable Objects. Use when building stateful coordination (chat rooms, multiplayer games, booking systems), implementing RPC methods, SQLite storage, alarms, WebSockets, or reviewing DO code for best practices. Covers Workers integration, wrangler config, and testing with Vitest. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.agents/skills/durable-objects/SKILL.md) - **web-perf**: Analyzes web performance using Chrome DevTools MCP. Measures Core Web Vitals (LCP, INP, CLS) and supplementary metrics (FCP, TBT, Speed Index), identifies render-blocking resources, network dependency chains, layout shifts, caching issues, and accessibility gaps. Use when asked to audit, profile, debug, or optimize page load performance, Lighthouse scores, or site speed. Biases towards retrieval from current documentation over pre-trained knowledge. (source: /Users/root1/.agents/skills/web-perf/SKILL.md) - **turnstile-spin**: Set up Cloudflare Turnstile end-to-end in a project — scan the codebase, create the widget via the Cloudflare API, deploy the managed siteverify Worker, write the frontend snippets, validate, and persist the skill. Load this when a user asks to add Turnstile, set up CAPTCHA, protect a form from bots, or fix a Turnstile integration. Mirrors developers.cloudflare.com/turnstile/spin. (source: /Users/root1/.config/devin/skills/turnstile-spin/SKILL.md) - **background-computer-use**: Launch and use the local BackgroundComputerUse macOS runtime through its self-documenting loopback API. Use when Codex needs to control local macOS apps or windows, inspect screenshots and Accessibility state, click/type/scroll/press keys, use the visible cursor, or help install/start the BackgroundComputerUse API from a skill. (source: /Users/root1/.devin/skills/background-computer-use/SKILL.md) - **durable-objects**: Create and review Cloudflare Durable Objects. Use when building stateful coordination (chat rooms, multiplayer games, booking systems), implementing RPC methods, SQLite storage, alarms, WebSockets, or reviewing DO code for best practices. Covers Workers integration, wrangler config, and testing with Vitest. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.config/devin/skills/durable-objects/SKILL.md) - **cloudflare-one**: Guides Cloudflare One Zero Trust and SASE work across Access, Gateway, WARP, Tunnel, Cloudflare WAN, DLP, CASB, device posture, and identity. Use when designing, configuring, troubleshooting, or reviewing Cloudflare One deployments. Retrieval-first: use current Cloudflare docs/API schemas instead of embedded product docs. (source: /Users/root1/.config/devin/skills/cloudflare-one/SKILL.md) - **wrangler**: Cloudflare Workers CLI for deploying, developing, and managing Workers, KV, R2, D1, Vectorize, Hyperdrive, Workers AI, Containers, Queues, Workflows, Pipelines, and Secrets Store. Load before running wrangler commands to ensure correct syntax and best practices. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. (source: /Users/root1/.config/devin/skills/wrangler/SKILL.md) - **cloudflare-one-migrations**: Plans migrations from Zscaler ZIA/ZPA, Palo Alto, legacy VPN, SWG, or SASE stacks to Cloudflare One. Use for migration assessments, policy mapping, rollout plans, and parity/gap analysis. (source: /Users/root1/.config/devin/skills/cloudflare-one-migrations/SKILL.md) - **web-perf**: Analyzes web performance using Chrome DevTools MCP. Measures Core Web Vitals (LCP, INP, CLS) and supplementary metrics (FCP, TBT, Speed Index), identifies render-blocking resources, network dependency chains, layout shifts, caching issues, and accessibility gaps. Use when asked to audit, profile, debug, or optimize page load performance, Lighthouse scores, or site speed. Biases towards retrieval from current documentation over pre-trained knowledge. (source: /Users/root1/.config/devin/skills/web-perf/SKILL.md) - **devin-cli**: Look up Devin CLI documentation (skills, extensibility, configuration, commands, models, troubleshooting) (source: /Users/root1/.local/share/devin/cli/_versions/3000.1.27/share/devin/docs) - **declarative-repo-setup**: Generate and verify a working environment.yaml (Devin snapshot-setup blueprint) for a repo (source: builtin:drs) </available_skills>
We need answer for user who posted x.com link. Need determine intent. User only sent URL. We may need summarize video? Link is X status. AGENTS rule says: "If I ever paste in a YouTube link, use yt-dlp to summarize the video. get the autogenerated captions to do this; if asked to summarize a YouTube video, do not name the session until after reading and understanding the full YouTube video transcript" That's only YouTube. For X link maybe summarize post? User likely wants summary of the tweet or maybe content. We should use Playwright MCP to fetch the tweet. The AGENTS rule includes X/Twitter scraping via logged-in browser session. However user just pasted a link; clean path is Playwright MCP. We need list tools, navigate, possibly evaluate, get content. Need use todo? Might be quick task but has multiple steps. Let's set up. The link is X status from claudeai. We need fetch it. We could use webfetch but X requires JS/login, blocked. Use playwright MCP per rules. Need first mcp_list_servers? Already available. Need mcp_list_tools for playwright. Then browser_navigate to URL. If not logged in ask user. Then fetch content via evaluate. Low-effort mode says take direct path, trust tool output. But there is skill? none for X/twitter. find-skills maybe? But AGENTS has explicit instructions for X scraping via Playwright. Let's follow. Need not use background-computer-use unless tasks benefit. Use Playwright MCP. Need be careful: X might block scraping? With logged-in browser session. Use mcp_call_tool after list tools. We can navigate and use eval to extract tweet text. The user's link: https://x.com/claudeai/status/2075271759289303522. We can attempt to get the content. If video exists? Need perhaps summarize video? Maybe it's a tweet containing a video? We can transcribe if video? The link just to tweet. Let's inspect. Plan: 1. mcp_list_tools playwright 2. browser_navigate to URL 3. If not logged in, ask user to login. Once on home, then navigate. 4. Use browser_evaluate to extract tweet text. If video? Use X GraphQL endpoints to fetch tweet detail. The AGENTS rules give endpoints, query id etc. We can use evaluate to call GQL. But simpler: page DOM might show text. However X hydration is dynamic, better use the GQL endpoint per rule. Need ct0 and auth token. The rule says read ct0 from document.cookie. We'll use browser_evaluate to fetch GQL endpoint. But need tweet id. Could parse from URL: 2075271759289303522. Need user screen_name? claudeai. Operation TweetDetail queryId meGUdoK_ryVZ0daBK-HJ2g. Need features dict. We don't have twscrape GQL_FEATURES. Could retrieve from known file? We can get from page source or web. But maybe easier: use X internal API (graphql) with hardcoded features from known list. The page itself will make the request; we can intercept network? Playwright MCP might have browser_evaluate only, not network interception. Could use browser_evaluate to call fetch. Need features dict. Let's see if we can find twscrape GQL_FEATURES locally. Could search web. But low-effort: maybe page DOM contains tweet text after navigation. Use evaluate to parse meta tags? X has meta tags with description. Let's first navigate and check page title/meta. Playwright browser may have `browser_evaluate` and `browser_navigate`. Need list tools. Let's list playwright tools.
I'll fetch the tweet content for you using the Playwright browser session.
{
"server_name": "playwright"
}[
{
"server_name": "playwright",
"tools": [
{
"name": "browser_close",
"description": "Close the page",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {},
"additionalProperties": false
},
"annotations": {
"title": "Close browser",
"readOnlyHint": false,
"destructiveHint": true,
"openWorldHint": true
}
},
{
"name": "browser_resize",
"description": "Resize the browser window",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"width": {
"type": "number",
"description": "Width of the browser window"
},
"height": {
"type": "number",
"description": "Height of the browser window"
}
},
"required": [
"width",
"height"
],
"additionalProperties": false
},
"annotations": {
"title": "Resize browser window",
"readOnlyHint": false,
"destructiveHint": true,
"openWorldHint": true
}
},
{
"name": "browser_console_messages",
"description": "Returns all console messages",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"level": {
"default": "info",
"description": "Level of the console messages to return. Each level includes the messages of more severe levels. Defaults to \"info\".",
"type": "string",
"enum": [
"error",
"warning",
"info",
"debug"
]
},
"all": {
"description": "Return all console messages since the beginning of the session, not just since the last navigation. Defaults to false.",
"type": "boolean"
},
"filename": {
"description": "Filename to save the console messages to. If not provided, messages are returned as text.",
"type": "string"
}
},
"required": [
"level"
],
"additionalProperties": false
},
"annotations": {
"title": "Get console messages",
"readOnlyHint": true,
"destructiveHint": false,
"openWorldHint": true
}
},
{
"name": "browser_handle_dialog",
"description": "Handle a dialog",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"accept": {
"type": "boolean",
"description": "Whether to accept the dialog."
},
"promptText": {
"description": "The text of the prompt in case of a prompt dialog.",
"type": "string"
}
},
"required": [
"accept"
],
"additionalProperties": false
},
"annotations": {
"title": "Handle a dialog",
"readOnlyHint": false,
"destructiveHint": true,
"openWorldHint": true
}
},
{
"name": "browser_evaluate",
"description": "Evaluate JavaScript expression on page or element",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"element": {
"description": "Human-readable element description used to obtain permission to interact with the element",
"type": "string"
},
"target": {
"description": "Exact target element reference from the page snapshot, or a unique element selector",
"type": "string"
},
"function": {
"type": "string",
"description": "() => { /* code */ } or (element) => { /* code */ } when element is provided"
},
"filename": {
"description": "Filename to save the result to. If not provided, result is returned as text.",
"type": "string"
}
},
"required": [
"function"
],
"additionalProperties": false
},
"annotations": {
"title": "Evaluate JavaScript",
"readOnlyHint": false,
"destructiveHint": true,
"openWorldHint": true
}
},
{
"name": "browser_file_upload",
"description": "Upload one or multiple files",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"paths": {
"description": "The absolute paths to the files to upload. Can be single file or multiple files. If omitted, file chooser is cancelled.",
"type": "array",
"items": {
"type": "string"
}
}
},
"additionalProperties": false
},
"annotations": {
"title": "Upload files",
"readOnlyHint": false,
"destructiveHint": true,
"openWorldHint": true
}
},
{
"name": "browser_drop",
"description": "Drop files or MIME-typed data onto an element, as if dragged from outside the page. At least one of \"paths\" or \"data\" must be provided.",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"element": {
"description": "Human-readable element description used to obtain permission to interact with the element",
"type": "string"
},
"target": {
"type": "string",
"description": "Exact target element reference from the page snapshot, or a unique element selector"
},
"paths": {
"description": "Absolute paths to files to drop onto the element.",
"type": "array",
"items": {
"type": "string"
}
},
"data": {
"description": "Data to drop, as a map of MIME type to string value (e.g. {\"text/plain\": \"hello\", \"text/uri-list\": \"https://example.com\"}).",
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {
"type": "string"
}
}
},
"required": [
"target"
],
"additionalProperties": false
},
"annotations": {
"title": "Drop files or data onto an element",
"readOnlyHint": false,
"destructiveHint": true,
"openWorldHint": true
}
},
{
"name": "browser_find",
"description": "Search the accessibility snapshot of the current page for text or a regular expression. Returns matching snapshot nodes with a few lines of surrounding context (like search snippets), each shown under its path from the root of the tree, which is cheaper than capturing the whole snapshot when you only need to locate an element and its ref.",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"text": {
"description": "Plain text to search for in the page snapshot (case-insensitive substring match). Provide either text or regex, not both.",
"type": "string"
},
"regex": {
"description": "Regular expression to search for in the page snapshot. Matching is case-sensitive by default; wrap the pattern in slashes to add flags, e.g. \"/error/i\" for case-insensitive. Provide either text or regex, not both.",
"type": "string"
}
},
"additionalProperties": false
},
"annotations": {
"title": "Find in page snapshot",
"readOnlyHint": true,
"destructiveHint": false,
"openWorldHint": true
}
},
{
"name": "browser_fill_form",
"description": "Fill multiple form fields",
"inputSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"fields": {
"type": "array",
"items": {
"type": "object",
"properties": {
"element": {
"description": "Human-readable element description used to obtain permission to interact with the element",
"type": "string"
},
"target": {
"type": "string",
"description": "Exact target element reference from the page snapshot, or a unique element selector"
},
"name": {
"type": "string",
"description": "Human-readable field name"
},
"type": {
"type": "string",
"enum": [
"textbox",
"checkbox",
"radio",
"combobox",
"slider"
],
"description": "Type of the field"
… (11 chars truncated)
… (546 lines truncated)
<truncation_notice>
Full output written to: /var/folders/d9/mzhyr7lx6xbf236j5ll9v_300000gn/T/devin-overflows-501/5bbd4307/content.txt
</truncation_notice>We have playwright tools. Need navigate. Let's call browser_navigate to URL.
{
"server_name": "playwright",
"tool_name": "browser_navigate",
"arguments": {
"url": "https://x.com/claudeai/status/2075271759289303522"
}
}### Ran Playwright code
```js
await page.goto('https://x.com/claudeai/status/2075271759289303522');
```
### Page
- Page URL: https://x.com/claudeai/status/2075271759289303522
- Page Title: Claude on X: "There’s hope in hard questions. https://t.co/rDYjqIlH9l" / X
### Snapshot
- [Snapshot](.playwright-mcp/page-2026-07-14T20-41-17-643Z.yml)It navigated. Title reveals tweet text: "Claude on X: "There’s hope in hard questions. https://t.co/rDYjqIlH9l" / X". We likely can get more from page DOM or meta. Need snapshot? Maybe page snapshot contains accessible elements. But we can evaluate to extract the tweet content, maybe the t.co link. Let's get meta description or JSON? We can use browser_evaluate to query document querySelector for article text or meta tags. Since title gives text. But link may be a link to something (maybe image/video). Need expand t.co? Could resolve URL. Could use GQL to get tweet detail with media. Let's first evaluate to find visible tweet content and any media alt. Use evaluate with `document.querySelector('article')`? But X uses `article` tags with data-testid="tweet". Better: look for meta tags `og:description`. Let's do evaluate: return {title, description, meta}. Also maybe fetch GQL if needed. Let's first try evaluate.
{
"server_name": "playwright",
"tool_name": "browser_evaluate",
"arguments": {
"function": "() => {\n const metas = Object.fromEntries(Array.from(document.querySelectorAll('meta')).filter(m => m.getAttribute('property') || m.getAttribute('name')).map(m => [m.getAttribute('property') || m.getAttribute('name'), m.getAttribute('content')]));\n const tweetText = document.querySelector('article[data-testid=\"tweet\"] div[dir=\"auto\"]')?.innerText || document.querySelector('div[data-testid=\"tweetText\"]')?.innerText || '';\n const media = Array.from(document.querySelectorAll('article[data-testid=\"tweet\"] img, article[data-testid=\"tweet\"] video, article[data-testid=\"tweet\"] a[href*=\"t.co/\"]')).map(el => ({tag: el.tagName, src: el.src || el.currentSrc || el.href, alt: el.alt || ''}));\n return {title: document.title, metas, tweetText, media};\n}"
}
}### Result
{
"title": "Claude on X: \"There’s hope in hard questions. https://t.co/rDYjqIlH9l\" / X",
"metas": {
"viewport": "width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0,viewport-fit=cover",
"color-scheme": "light",
"theme-color": "#000000",
"csp-nonce": "Fs7BmXNQSlgEhuoxRBC3dQ==",
"robots": "index, follow",
"fb:app_id": "2231777543",
"og:site_name": "X (formerly Twitter)",
"og:locale": "en_US",
"application-name": "X (formerly Twitter)",
"google-site-verification": "reUF-TgZq93ZGtzImw42sfYglI2hY0QiGRmfc4jeKbs",
"facebook-domain-verification": "x6sdcc8b5ju3bh8nbm59eswogvg6t1",
"al:ios:app_store_id": "333903271",
"al:ios:app_name": "X",
"al:android:package": "com.twitter.android",
"al:android:app_name": "X",
"apple-itunes-app": "app-id=333903271",
"apple-mobile-web-app-title": "X (formerly Twitter)",
"apple-mobile-web-app-capable": "yes",
"apple-mobile-web-app-status-bar-style": "default",
"mobile-web-app-capable": "yes",
"app-version": "0939faeb242bfb3c13c3b88b4de58d3f4557c545",
"twitter-site-verification": "QrwmXNt80JznReecH0TrYqOJCYxEEZjup1gCM/Id/gmXkrUFN13CL+Bnp26/lKCu",
"sentry-trace": "71854fd5497c4d868d2f9a908752e746-8e1e0306aba17dc5-0",
"baggage": "sentry-environment=production,sentry-release=0939faeb242bfb3c13c3b88b4de58d3f4557c545,sentry-public_key=b523016f172e2f101d4f771b022ba96c,sentry-trace_id=71854fd5497c4d868d2f9a908752e746,sentry-org_id=30969,sentry-sampled=false,sentry-sample_rand=0.04524452708397153,sentry-sample_rate=0.003",
"og:url": "https://x.com/claudeai/status/2075271759289303522",
"og:type": "article",
"article:published_time": "2026-07-09T17:31:58.000Z",
"article:author": "https://x.com/claudeai",
"og:title": "Claude (@claudeai) on X",
"title": "Claude (@claudeai) on X",
"og:description": "There’s hope in hard questions.",
"description": "There’s hope in hard questions.",
"og:image": "https://pbs.twimg.com/media/HMzYzrcXMAAeKQI.jpg",
"og:image:secure_url": "https://pbs.twimg.com/media/HMzYzrcXMAAeKQI.jpg",
"og:image:type": "image/jpeg",
"og:image:width": "1920",
"og:image:height": "1080",
"twitter:card": "summary_large_image",
"twitter:site": "@x",
"twitter:title": "Claude (@claudeai) on X",
"twitter:description": "There’s hope in hard questions.",
"twitter:creator": "@claudeai",
"twitter:image": "https://pbs.twimg.com/media/HMzYzrcXMAAeKQI.jpg",
"al:ios:url": "twitter://status?id=2075271759289303522",
"al:android:url": "twitter://status?id=2075271759289303522"
},
"tweetText": "",
"media": []
}
### Ran Playwright code
```js
await page.evaluate('() => {\n const metas = Object.fromEntries(Array.from(document.querySelectorAll(\'meta\')).filter(m => m.getAttribute(\'property\') || m.getAttribute(\'name\')).map(m => [m.getAttribute(\'property\') || m.getAttribute(\'name\'), m.getAttribute(\'content\')]));\n const tweetText = document.querySelector(\'article[data-testid="tweet"] div[dir="auto"]\')?.innerText || document.querySelector(\'div[data-testid="tweetText"]\')?.innerText || \'\';\n const media = Array.from(document.querySelectorAll(\'article[data-testid="tweet"] img, article[data-testid="tweet"] video, article[data-testid="tweet"] a[href*="t.co/"]\')).map(el => ({tag: el.tagName, src: el.src || el.currentSrc || el.href, alt: el.alt || \'\'}));\n return {title: document.title, metas, tweetText, media};\n}');
```
### Page
- Page URL: https://x.com/claudeai/status/2075271759289303522
- Page Title: Claude on X: "There’s hope in hard questions. https://t.co/rDYjqIlH9l" / X
- Console: 4 errors, 0 warnings
### Events
- New console entries: .playwright-mcp/console-2026-07-14T20-41-16-018Z.log#L1-L4Meta gives tweet text: "There’s hope in hard questions." There is an image https://pbs.twimg.com/media/HMzYzrcXMAAeKQI.jpg. The t.co link likely points to image? Actually image URL is in og:image. The t.co short link may be for image. Let's answer with summary: Claude posted on X: "There’s hope in hard questions." with an attached image. We could also fetch image? Not needed. User may want content. Need maybe summarize link. We'll respond succinctly. But let's consider if they wanted to summarize a video? The tweet contains image, not video. Could also resolve t.co to expand. Let's try HEAD to see redirect? Could use webfetch on t.co/rDYjqIlH9l. But likely expands to image. We can use browser_evaluate to fetch with no-cors? We can call `fetch('https://t.co/rDYjqIlH9l', {redirect:'follow'}).then(r=>r.url)` perhaps. t.co may not allow CORS but redirect to final URL. Let's try? Could be unnecessary. The meta gives the image. But let's verify t.co expands to the image or some other page. Use browser_evaluate with fetch.
{
"server_name": "playwright",
"tool_name": "browser_evaluate",
"arguments": {
"function": "() => fetch('https://t.co/rDYjqIlH9l', {redirect:'follow'}).then(r => r.url).catch(e => e.message)"
}
}### Result
"https://t.co/rDYjqIlH9l"
### Ran Playwright code
```js
await page.evaluate('() => fetch(\'https://t.co/rDYjqIlH9l\', {redirect:\'follow\'}).then(r => r.url).catch(e => e.message)');
```It returned same URL, maybe CORS blocked or no redirect because final. Could be a card link? But no need. We have enough. We should respond: The post says "There’s hope in hard questions." includes an image. Maybe ask if they need anything else. Since user didn't ask explicitly, but just shared link. We can state summary. Need mention source? We can include tweet URL and timestamp. Keep concise. Need not cite with ref tags? The source is the fetched page but not a local file. We can just provide info. Also we should consider if link is from claudeai's X and the tweet may reference a blog or article. The t.co link might be a redirect to something else. But og:image says image. t.co in tweet text probably auto-attached media link? Actually X adds a t.co link to attached media/URL, but in the text it might be the link to the image. The og:image is media. The text shows "There’s hope in hard questions. https://t.co/rDYjqIlH9l". So link is attached image. No further. We can provide a one-sentence summary and maybe the image URL. Let's answer.
Claude (@claudeai) posted on X:
“There’s hope in hard questions.”
It includes an attached image: https://pbs.twimg.com/media/HMzYzrcXMAAeKQI.jpg
Posted: 2026-07-09 17:31 UTC
use whisper on modal
User asks "use whisper on modal". This likely means transcribe the audio/video from the tweet? The AGENTS rule describes Fast Whisper transcription on Modal utility at `~/Downloads/utilities/whisper_x/whisper_transcribe.py`. For transcribing long-form audio/video including X/Twitter videos. Since tweet has an image not video? But the link may be a video? Actually meta og:image. No video. But the user's prior link was a tweet. They now ask use whisper on modal, possibly to transcribe a video on the tweet? Need determine. Maybe they want to transcribe the X post's media? It is an image, so no audio. But maybe the t.co link leads to a video? Let's check. Could use yt-dlp/whisper utility with X URL. Let's inspect the media: We got image in meta. The tweet likely contains an image, not video. But maybe there is a video in the tweet and og:image is poster? Let's fetch tweet details via GQL? Could also use yt-dlp to extract media from tweet URL. The whisper utility uses yt-dlp to download X video. Let's run it to see if there is a video. But first, maybe ask? The request is ambiguous. Could interpret: Use the Modal Whisper pipeline to transcribe the X link. Let's try running the `whisper` alias with the X URL. The utility at `~/Downloads/utilities/whisper_x/whisper_transcribe.py` may handle X URLs and outputs transcript. But we need set env? Modal profile is configured. Need source ~/.env? HF_TOKEN? It uses Modal. We can run `whisper "https://x.com/claudeai/status/2075271759289303522"`. If no audio/video, yt-dlp may fail. Let's do it. But maybe we should first verify if tweet contains video. We can query the page for video. Use browser_evaluate to find all video elements and URLs. Let's do that in parallel with checking if utility exists.
{
"server_name": "playwright",
"tool_name": "browser_evaluate",
"arguments": {
"function": "() => {\n const videos = Array.from(document.querySelectorAll('video')).map(v => ({src: v.currentSrc || v.src, poster: v.poster}));\n const tweetArticle = document.querySelector('article[data-testid=\"tweet\"]');\n const tweetMedia = tweetArticle ? Array.from(tweetArticle.querySelectorAll('img, video, a[href*=\"/i/spaces/\"], a[href*=\"x.com/i/broadcasts/\"]')).map(el => ({tag: el.tagName, src: el.src || el.currentSrc || el.href, poster: el.poster, aria: el.getAttribute('aria-label')})) : [];\n return {videos, tweetMedia, htmlHint: document.documentElement.innerHTML.includes('video') ? 'has video tag' : 'no video tag'};\n}"
}
}{
"command": "ls -la ~/Downloads/utilities/whisper_x/"
}### Result
{
"videos": [
{
"src": "blob:https://x.com/db823d5c-95dc-4029-90e3-12a513800f26",
"poster": "https://pbs.twimg.com/media/HMzYzrcXMAAeKQI.jpg"
}
],
"tweetMedia": [],
"htmlHint": "has video tag"
}
### Ran Playwright code
```js
await page.evaluate('() => {\n const videos = Array.from(document.querySelectorAll(\'video\')).map(v => ({src: v.currentSrc || v.src, poster: v.poster}));\n const tweetArticle = document.querySelector(\'article[data-testid="tweet"]\');\n const tweetMedia = tweetArticle ? Array.from(tweetArticle.querySelectorAll(\'img, video, a[href*="/i/spaces/"], a[href*="x.com/i/broadcasts/"]\')).map(el => ({tag: el.tagName, src: el.src || el.currentSrc || el.href, poster: el.poster, aria: el.getAttribute(\'aria-label\')})) : [];\n return {videos, tweetMedia, htmlHint: document.documentElement.innerHTML.includes(\'video\') ? \'has video tag\' : \'no video tag\'};\n}');
```
### Page
- Page URL: https://x.com/claudeai/status/2075271759289303522
- Page Title: Claude on X: "There’s hope in hard questions. https://t.co/rDYjqIlH9l" / X
- Console: 8 errors, 0 warnings
### Events
- New console entries: .playwright-mcp/console-2026-07-14T20-41-16-018Z.log#L5-L8Output from command in shell afc5f9: total 491888 drwxr-xr-x 4 root1 staff 128 Jun 28 18:12 __pycache__ drwxr-xr-x 17 root1 staff 544 Jun 28 18:17 . drwxr-xr-x 9 root1 staff 288 Jun 28 16:58 .. -rw-r--r-- 1 root1 staff 1631017 Jun 28 17:05 audio.m4a -rw-r--r-- 1 root1 staff 166039 Jun 28 18:16 fast_transcription.json -rw-r--r-- 1 root1 staff 34928168 Jun 28 17:14 full_audio.m4a -rw-r--r-- 1 root1 staff 63466958 Jun 28 17:13 full_audio.mp4 -rw-r--r--@ 1 root1 staff 203900 Jun 29 21:06 full_transcript.txt -rw-r--r-- 1 root1 staff 319056 Jun 28 17:37 full_transcription.json -rw-r--r-- 1 root1 staff 4846 Jun 28 18:17 STRATEGY.md -rw-r--r-- 1 root1 staff 2615 Jun 28 18:12 transcribe_fast.py -rw-r--r-- 1 root1 staff 1915 Jun 28 17:25 transcribe.py -rw-r--r-- 1 root1 staff 9127 Jun 28 17:11 transcript.txt -rw-r--r-- 1 root1 staff 11700 Jun 28 17:11 transcription.json -rw-r--r-- 1 root1 staff 329 Jun 28 17:05 upload.py -rw-r--r-- 1 root1 staff 151064333 Jun 28 16:58 video.mp4 -rwxr-xr-x 1 root1 staff 10054 Jul 1 22:02 whisper_transcribe.py Exit code: 0
There is a video on the page (blob URL). The poster image matches og:image. So tweet includes a video. We need use whisper on Modal to transcribe it. Good. Need use the utility. The alias `whisper` is in `~/.zshrc`. Use `whisper "https://x.com/claudeai/status/2075271759289303522"`. But before running, source ~/.env? The rule says secrets/tokens live in ~/.env; source it before use. This includes MODAL_TOKEN_ID, MODAL_TOKEN_SECRET, HF_TOKEN? Need source. Run `set -a; . ~/.env; set +a` before whisper. But the alias may be in zshrc; since using bash? Use zsh maybe. We can run `source ~/.zshrc` or `zsh -ic`. But the shell may be bash. The alias uses python script. We can run full python path with env sourced. Let's first check `.zshrc` for alias definition and `.env` content? Not show secrets. We can check alias without revealing secrets. Use `grep` from `.zshrc` and maybe check env variables after source? But secrets sensitive; avoid logging. Let's run `source ~/.zshrc` or just use `python3 ~/Downloads/utilities/whisper_x/whisper_transcribe.py`. Need read script to see args and env.
{
"file_path": "/Users/root1/.zshrc"
}{
"file_path": "/Users/root1/Downloads/utilities/whisper_x/whisper_transcribe.py"
}<file-view path="/Users/root1/.zshrc" start_line="1" end_line="32" total_lines="32"> 1|alias codex="codex --search --yolo" 2|alias devin='devin --permission-mode bypass' 3|alias whisper='python3 ~/Downloads/utilities/whisper_x/whisper_transcribe.py' 4|export PATH="$HOME/.local/bin:$PATH" 5|export PATH="/Library/Frameworks/Python.framework/Versions/3.12/bin:$PATH" 6| 7|# pnpm 8|export PNPM_HOME="/Users/root1/Library/pnpm" 9|case ":$PATH:" in 10| *":$PNPM_HOME:"*) ;; 11| *) export PATH="$PNPM_HOME:$PATH" ;; 12|esac 13|# pnpm end 14| 15|# bun completions (disabled due to syntax error) 16|# [ -s "/Users/root1/.bun/_bun" ] && source "/Users/root1/.bun/_bun" 17| 18|# bun 19|export BUN_INSTALL="$HOME/.bun" 20|export PATH="$BUN_INSTALL/bin:$PATH" 21| 22|# Added by Devin 23|export PATH="/Users/root1/.codeium/windsurf/bin:$PATH" 24| 25| 26|# Added by Antigravity CLI installer 27|export PATH="/Users/root1/.local/bin:$PATH" 28| 29|autoload -Uz compinit && compinit 30| 31|# omp (oh-my-pi) completions 32|eval "$(omp completions zsh)" </file-view>
<file-view path="/Users/root1/Downloads/utilities/whisper_x/whisper_transcribe.py" start_line="1" end_line="293" total_lines="293">
1|#!/usr/bin/env python3
2|"""whisper_transcribe.py — Download video/audio from a URL, run fast Whisper
3|transcription on a Modal A10G GPU, and save the transcript locally.
4|
5|Full pipeline: URL → yt-dlp download → ffmpeg audio extract → Modal volume
6|upload → faster-whisper (batched, beam=1, float16) → JSON + TXT output.
7|
8|Usage:
9| python3 whisper_transcribe.py <url> [options]
10|
11|Examples:
12| # Transcribe an X/Twitter video (picks first item in playlist)
13| python3 whisper_transcribe.py "https://x.com/davidsenra/status/2071374140645232824"
14|
15| # Pick item 2 from a playlist, use medium model for speed
16| python3 whisper_transcribe.py "https://x.com/..." --playlist-item 2 --model-size medium
17|
18| # Transcribe a local audio file (skip download)
19| python3 whisper_transcribe.py /path/to/audio.mp3
20|
21| # Transcribe a direct audio URL
22| python3 whisper_transcribe.py "https://example.com/podcast.mp3"
23|
24|Requirements:
25| - yt-dlp, ffmpeg, modal CLI installed and authenticated
26| - Modal volume "whisper-audio" (created automatically if missing)
27|
28|Output:
29| <workdir>/<name>.json — structured result with timed segments
30| <workdir>/<name>.txt — human-readable transcript
31|"""
32|from __future__ import annotations
33|
34|import argparse
35|import json
36|import os
37|import shutil
38|import subprocess
39|import sys
40|import time
41|from pathlib import Path
42|
43|# ── Modal app definition ──────────────────────────────────────────────────────
44|# Defined at module level so `modal run` can discover it, but the CLI below
45|# drives the flow directly via the Modal Python SDK for tighter control.
46|
47|VOLUME_NAME = "whisper-audio"
48|DEFAULT_MODEL = "large-v3"
49|SUPPORTED_MODELS = ["tiny", "base", "small", "medium", "large-v3"]
50|
51|
52|def run(cmd: list[str], **kw) -> subprocess.CompletedProcess:
53| """Run a command, stream output, raise on failure."""
54| print(f" $ {' '.join(cmd)}", file=sys.stderr)
55| return subprocess.run(cmd, check=True, **kw)
56|
57|
58|def step(msg: str) -> None:
59| print(f"\n=== {msg} ===", file=sys.stderr)
60|
61|
62|def download_audio(url: str, outdir: Path, playlist_item: int | None) -> Path:
63| """Download audio from a URL using yt-dlp. Returns path to downloaded file."""
64| step(f"Downloading audio from {url}")
65| if playlist_item is not None:
66| cmd = [
67| "yt-dlp", "--playlist-items", str(playlist_item),
68| "-f", "bestaudio/best",
69| "-o", str(outdir / "source.%(ext)s"),
70| url,
71| ]
72| else:
73| cmd = [
74| "yt-dlp", "-f", "bestaudio/best",
75| "-o", str(outdir / "source.%(ext)s"),
76| url,
77| ]
78| run(cmd)
79| files = list(outdir.glob("source.*"))
80| if not files:
81| raise RuntimeError("yt-dlp produced no output file")
82| return files[0]
83|
84|
85|def extract_audio(src: Path, outdir: Path) -> Path:
86| """Re-encode audio to 16kHz mono AAC (Whisper's native format)."""
87| step("Extracting / re-encoding audio to 16kHz mono")
88| dst = outdir / "audio.m4a"
89| run([
90| "ffmpeg", "-y", "-i", str(src),
91| "-vn", "-ac", "1", "-ar", "16000",
92| "-c:a", "aac", "-b:a", "64k",
93| str(dst),
94| ])
95| return dst
96|
97|
98|def upload_to_volume(local_path: Path, remote_name: str) -> None:
99| """Upload a file to the Modal volume."""
100| step(f"Uploading {local_path.name} to Modal volume '{VOLUME_NAME}'")
101| # Ensure volume exists
102| subprocess.run(
103| ["modal", "volume", "create", VOLUME_NAME],
104| capture_output=True, # ignore "already exists" error
105| )
106| run(["modal", "volume", "put", VOLUME_NAME, str(local_path), remote_name])
107|
108|
109|def run_modal_transcription(remote_name: str, model_size: str) -> dict:
110| """Run the Modal transcription function and return the result dict."""
111| import modal
112|
113| step(f"Running Whisper {model_size} on Modal A10G (no caching)")
114|
115| image = (
116| modal.Image.from_registry(
117| "nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04",
118| add_python="3.12",
119| )
120| .apt_install("ffmpeg", "git", force_build=True)
121| .pip_install(
122| "faster-whisper==1.1.1",
123| "ctranslate2==4.8.0",
124| "requests",
125| force_build=True,
126| )
127| )
128|
129| audio_vol = modal.Volume.from_name(VOLUME_NAME, create_if_missing=True)
130|
131| app = modal.App(f"whisper-transcribe-{int(time.time())}")
132|
133| @app.function(
134| image=image,
135| gpu="A10G",
136| volumes={"/audio": audio_vol},
137| timeout=600,
138| serialized=True,
139| )
140| def _transcribe(filename: str, model: str) -> dict:
141| import os as _os
142| import time as _time
143| from faster_whisper import WhisperModel, BatchedInferencePipeline
144|
145| t0 = _time.time()
146| download_root = f"/tmp/whisper_models_{int(_time.time())}"
147| _os.makedirs(download_root, exist_ok=True)
148| print(f"Loading model {model} (fresh download)...", flush=True)
149| wm = WhisperModel(
150| model, device="cuda", compute_type="float16",
151| download_root=download_root,
152| )
153| batched = BatchedInferencePipeline(model=wm)
154| t_load = _time.time() - t0
155| print(f"Model loaded in {t_load:.1f}s", flush=True)
156|
157| path = f"/audio/{filename}"
158| t1 = _time.time()
159| segments, info = batched.transcribe(
160| path, batch_size=16, beam_size=1,
161| vad_filter=True, language=None,
162| )
163| seg_list = list(segments)
164| t_trans = _time.time() - t1
165| print(f"Transcribed {len(seg_list)} segments in {t_trans:.1f}s", flush=True)
166|
167| return {
168| "language": info.language,
169| "language_probability": info.language_probability,
170| "duration": info.duration,
171| "text": " ".join(s.text.strip() for s in seg_list),
172| "segments": [
173| {"start": s.start, "end": s.end, "text": s.text.strip()}
174| for s in seg_list
175| ],
176| "load_time_s": t_load,
177| "transcribe_time_s": t_trans,
178| }
179|
180| with app.run():
181| t_start = time.time()
182| result = _transcribe.remote(remote_name, model_size)
183| wall = time.time() - t_start
184| print(f"\n=== WALL CLOCK: {wall:.1f}s ({wall/60:.2f} min) ===", file=sys.stderr)
185| result["wall_clock_s"] = wall
186| return result
187|
188|
189|def save_outputs(result: dict, outdir: Path, name: str) -> tuple[Path, Path]:
190| """Save JSON and TXT transcript files."""
191| json_path = outdir / f"{name}.json"
192| txt_path = outdir / f"{name}.txt"
193|
194| with open(json_path, "w") as f:
195| json.dump(result, f, indent=2, ensure_ascii=False)
196|
197| lines = [
198| f"Language: {result['language']} ({result['language_probability']*100:.1f}%)",
199| f"Duration: {result['duration']:.1f}s ({result['duration']/60:.0f} min)",
200| f"Load: {result['load_time_s']:.1f}s | Transcribe: {result['transcribe_time_s']:.1f}s | Wall: {result.get('wall_clock_s', 0):.1f}s",
201| f"Segments: {len(result['segments'])}",
202| "",
203| "=== FULL TRANSCRIPT ===",
204| result["text"],
205| "",
206| "=== TIMED SEGMENTS ===",
207| ]
208| for s in result["segments"]:
209| lines.append(f"[{s['start']:07.2f} - {s['end']:07.2f}] {s['text']}")
210| with open(txt_path, "w") as f:
211| f.write("\n".join(lines))
212|
213| return json_path, txt_path
214|
215|
216|def main():
217| parser = argparse.ArgumentParser(
218| description="Download audio from URL and transcribe with Whisper on Modal A10G.",
219| epilog="See STRATEGY.md for the full optimization breakdown.",
220| )
221| parser.add_argument("url", help="Video/audio URL, or path to a local audio file")
222| parser.add_argument(
223| "--playlist-item", type=int, default=None,
224| help="If URL is a playlist, which item to download (1-indexed)",
225| )
226| parser.add_argument(
227| "--model-size", default=DEFAULT_MODEL, choices=SUPPORTED_MODELS,
228| help=f"Whisper model size (default: {DEFAULT_MODEL})",
229| )
230| parser.add_argument(
231| "--outdir", default=".",
232| help="Output directory for transcript files (default: current dir)",
233| )
234| parser.add_argument(
235| "--name", default=None,
236| help="Output filename base (default: derived from URL/file)",
237| )
238| parser.add_argument(
239| "--keep-source", action="store_true",
240| help="Keep the downloaded source file (default: cleaned up)",
241| )
242| args = parser.parse_args()
243|
244| outdir = Path(args.outdir).resolve()
245| outdir.mkdir(parents=True, exist_ok=True)
246|
247| # Determine name
248| if args.name:
249| name = args.name
250| elif args.url.startswith("http"):
251| # Use last path segment, strip query
252| from urllib.parse import urlparse
253| name = urlparse(args.url).path.rstrip("/").split("/")[-1] or "transcript"
254| else:
255| name = Path(args.url).stem
256|
257| # Step 1: Get audio file
258| src_path = Path(args.url)
259| if src_path.exists() and src_path.is_file():
260| # Local file — just re-encode
261| step(f"Using local file: {src_path}")
262| audio_path = extract_audio(src_path, outdir)
263| else:
264| # Download from URL
265| audio_path = download_audio(args.url, outdir, args.playlist_item)
266| # Re-encode the downloaded file
267| audio_path = extract_audio(audio_path, outdir)
268|
269| # Step 2: Upload to Modal volume
270| remote_name = f"{name}_{int(time.time())}.m4a"
271| upload_to_volume(audio_path, remote_name)
272|
273| # Step 3: Run transcription on Modal
274| result = run_modal_transcription(remote_name, args.model_size)
275|
276| # Step 4: Save outputs
277| step("Saving transcript")
278| json_path, txt_path = save_outputs(result, outdir, name)
279| print(f" JSON: {json_path}", file=sys.stderr)
280| print(f" TXT: {txt_path}", file=sys.stderr)
281|
282| # Cleanup
283| if not args.keep_source:
284| for f in outdir.glob("source.*"):
285| f.unlink(missing_ok=True)
286| audio_path.unlink(missing_ok=True)
287|
288| # Print transcript to stdout
289| print(result["text"])
290|
291|
292|if __name__ == "__main__":
293| main()
</file-view>There is a video. Need run whisper. The utility needs Modal auth and maybe HF_TOKEN? It downloads faster-whisper model from HF? The code doesn't set HF_TOKEN, but `faster-whisper` may download from Hugging Face, no auth required for open models? It might need `HF_TOKEN` for gated? Whisper models are public. However environment may need `HF_TOKEN`? The rule says source ~/.env before use for HF_TOKEN. Let's source and run. Need create output dir? It defaults current dir. Better run with outdir maybe `~/Downloads/utilities/whisper_x/`? But we can run in cwd `/Users/root1` and output will be there. Name default status id. The alias `whisper` uses python3, but shell might be bash. We can run `zsh -ic 'whisper "URL"'` or `set -a; . ~/.env; set +a; python3 ...`. Since `modal` Python SDK uses env MODAL_TOKEN_ID and MODAL_TOKEN_SECRET. We need source. Also `yt-dlp` and `ffmpeg` installed. Let's execute. But need know if working dir is a git-indexed project? fff rules not relevant. We'll just exec. Potential issue: The X video is in a playlist? The utility picks first item. The tweet likely has one video. Could run. But X may require cookies for download? `yt-dlp` may have cookies from browser? It might not. But if X video is accessible publicly, yt-dlp can download. The URL is x.com/.../status/... It might use `--cookies-from-browser`? The utility doesn't. Could fail if video requires auth. But tweet is public from claudeai. Let's run and see. Need source ~/.env but not log secrets. Use `set -a; . ~/.env; set +a;`. Then run `python3 ...`. Since `~/.env` may contain exports. `set -a` exports all variables defined after. It will source. If `.env` has `export FOO=...` good. If not, set -a exports them. But `set -a` in bash affects all subsequent assignments; source can have spaces? Fine. Use `set -a; source ~/.env; set +a`. However if `.env` contains `set -a`? no. Let's run in background? The transcription may take minutes; default idle timeout 5s, total timeout default 30s. Could be 2-3 min for short. Use background exec. But we want capture transcript. Use exec with run_in_background true, then get_output later. Let's start background.
{
"command": "set -a; . ~/.env; set +a; python3 ~/Downloads/utilities/whisper_x/whisper_transcribe.py \"https://x.com/claudeai/status/2075271759289303522\"",
"run_in_background": true,
"timeout": 300000
}Command running in background with ID: 61790a
Background process running. Use get_output to poll.
{
"shell_id": "61790a",
"timeout": 30000
}
=== Downloading audio from https://x.com/claudeai/status/2075271759289303522 ===
$ yt-dlp -f bestaudio/best -o /Users/root1/source.%(ext)s https://x.com/claudeai/status/2075271759289303522
WARNING: Your yt-dlp version (2025.12.08) is older than 90 days!
It is strongly recommended to always use the latest version.
You installed yt-dlp with pip or using the wheel from PyPi; Use that to update.
To suppress this warning, add --no-update to your command/config.
[twitter] Extracting URL: https://x.com/claudeai/status/2075271759289303522
[twitter] 2075271759289303522: Downloading guest token
[twitter] 2075271759289303522: Downloading GraphQL JSON
[twitter] 2075271759289303522: Downloading m3u8 information
[info] 2075249313014317057: Downloading 1 format(s): hls-audio-128000-Audio
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 31
[download] Destination: /Users/root1/source.mp4
[download] 3.2% of ~ 23.79KiB at 0.00B/s ETA Unknown (frag 0/31)
[download] 1.6% of ~ 47.59KiB at 0.00B/s ETA Unknown (frag 1/31)
[download] 0.2% of ~ 758.09KiB at 7.62KiB/s ETA Unknown (frag 1/31)
[download] 0.5% of ~ 773.59KiB at 7.62KiB/s ETA Unknown (frag 1/31)
[download] 1.0% of ~ 804.59KiB at 7.62KiB/s ETA Unknown (frag 1/31)
[download] 1.8% of ~ 866.59KiB at 7.62KiB/s ETA Unknown (frag 1/31)
[download] 3.2% of ~ 990.59KiB at 7.62KiB/s ETA Unknown (frag 1/31)
[download] 3.9% of ~ 1.21MiB at 7.62KiB/s ETA Unknown (frag 1/31)
[download] 3.3% of ~ 1.47MiB at 7.62KiB/s ETA Unknown (frag 2/31)
[download] 5.0% of ~ 994.87KiB at 7.62KiB/s ETA Unknown (frag 2/31)
[download] 5.2% of ~1005.20KiB at 7.62KiB/s ETA Unknown (frag 2/31)
[download] 5.4% of ~ 1.00MiB at 7.62KiB/s ETA Unknown (frag 2/31)
[download] 6.0% of ~ 1.04MiB at 7.62KiB/s ETA Unknown (frag 2/31)
[download] 6.9% of ~ 1.12MiB at 204.46KiB/s ETA Unknown (frag 2/31)
[download] 7.3% of ~ 1.28MiB at 204.46KiB/s ETA Unknown (frag 2/31)
[download] 6.5% of ~ 1.45MiB at 204.46KiB/s ETA Unknown (frag 3/31)
[download] 8.7% of ~ 1.09MiB at 204.46KiB/s ETA Unknown (frag 3/31)
[download] 8.8% of ~ 1.10MiB at 204.46KiB/s ETA Unknown (frag 3/31)
[download] 9.1% of ~ 1.11MiB at 204.46KiB/s ETA Unknown (frag 3/31)
[download] 9.5% of ~ 1.14MiB at 204.46KiB/s ETA Unknown (frag 3/31)
[download] 10.3% of ~ 1.20MiB at 204.46KiB/s ETA Unknown (frag 3/31)
[download] 10.6% of ~ 1.32MiB at 204.46KiB/s ETA Unknown (frag 3/31)
[download] 9.7% of ~ 1.45MiB at 395.99KiB/s ETA Unknown (frag 4/31)
[download] 12.2% of ~ 1.16MiB at 395.99KiB/s ETA Unknown (frag 4/31)
[download] 12.3% of ~ 1.17MiB at 395.99KiB/s ETA Unknown (frag 4/31)
[download] 12.5% of ~ 1.18MiB at 395.99KiB/s ETA Unknown (frag 4/31)
[download] 12.9% of ~ 1.20MiB at 395.99KiB/s ETA Unknown (frag 4/31)
[download] 13.6% of ~ 1.25MiB at 395.99KiB/s ETA Unknown (frag 4/31)
[download] 13.9% of ~ 1.35MiB at 395.99KiB/s ETA Unknown (frag 4/31)
[download] 12.9% of ~ 1.45MiB at 395.99KiB/s ETA Unknown (frag 5/31)
[download] 15.6% of ~ 1.20MiB at 504.23KiB/s ETA Unknown (frag 5/31)
[download] 15.7% of ~ 1.21MiB at 504.23KiB/s ETA Unknown (frag 5/31)
[download] 15.9% of ~ 1.22MiB at 504.23KiB/s ETA Unknown (frag 5/31)
[download] 16.3% of ~ 1.24MiB at 504.23KiB/s ETA Unknown (frag 5/31)
[download] 17.0% of ~ 1.28MiB at 504.23KiB/s ETA Unknown (frag 5/31)
[download] 17.1% of ~ 1.36MiB at 504.23KiB/s ETA Unknown (frag 5/31)
[download] 16.2% of ~ 1.44MiB at 504.23KiB/s ETA Unknown (frag 6/31)
[download] 18.9% of ~ 1.24MiB at 504.23KiB/s ETA Unknown (frag 6/31)
[download] 19.0% of ~ 1.24MiB at 504.23KiB/s ETA Unknown (frag 6/31)
[download] 19.2% of ~ 1.25MiB at 504.23KiB/s ETA Unknown (frag 6/31)
[download] 19.5% of ~ 1.27MiB at 504.23KiB/s ETA Unknown (frag 6/31)
[download] 20.2% of ~ 1.30MiB at 504.23KiB/s ETA Unknown (frag 6/31)
[download] 20.4% of ~ 1.37MiB at 504.23KiB/s ETA Unknown (frag 6/31)
[download] 19.4% of ~ 1.45MiB at 504.23KiB/s ETA Unknown (frag 7/31)
[download] 22.2% of ~ 1.26MiB at 605.61KiB/s ETA Unknown (frag 7/31)
[download] 22.3% of ~ 1.27MiB at 605.61KiB/s ETA Unknown (frag 7/31)
[download] 22.5% of ~ 1.28MiB at 605.61KiB/s ETA Unknown (frag 7/31)
[download] 22.8% of ~ 1.29MiB at 605.61KiB/s ETA Unknown (frag 7/31)
[download] 23.5% of ~ 1.32MiB at 605.61KiB/s ETA Unknown (frag 7/31)
[download] 23.6% of ~ 1.38MiB at 605.61KiB/s ETA Unknown (frag 7/31)
[download] 22.6% of ~ 1.45MiB at 605.61KiB/s ETA Unknown (frag 8/31)
[download] 25.5% of ~ 1.28MiB at 605.61KiB/s ETA Unknown (frag 8/31)
[download] 25.6% of ~ 1.29MiB at 605.61KiB/s ETA Unknown (frag 8/31)
[download] 25.8% of ~ 1.29MiB at 605.61KiB/s ETA Unknown (frag 8/31)
[download] 26.1% of ~ 1.31MiB at 605.61KiB/s ETA Unknown (frag 8/31)
[download] 26.7% of ~ 1.33MiB at 605.61KiB/s ETA Unknown (frag 8/31)
[download] 26.9% of ~ 1.39MiB at 715.88KiB/s ETA Unknown (frag 8/31)
[download] 25.8% of ~ 1.44MiB at 715.88KiB/s ETA Unknown (frag 9/31)
[download] 28.7% of ~ 1.30MiB at 715.88KiB/s ETA Unknown (frag 9/31)
[download] 28.8% of ~ 1.30MiB at 715.88KiB/s ETA Unknown (frag 9/31)
[download] 29.0% of ~ 1.31MiB at 715.88KiB/s ETA Unknown (frag 9/31)
[download] 29.3% of ~ 1.32MiB at 715.88KiB/s ETA Unknown (frag 9/31)
[download] 29.9% of ~ 1.35MiB at 715.88KiB/s ETA Unknown (frag 9/31)
[download] 30.1% of ~ 1.39MiB at 715.88KiB/s ETA Unknown (frag 9/31)
[download] 29.0% of ~ 1.44MiB at 715.88KiB/s ETA Unknown (frag 10/31)
[download] 32.0% of ~ 1.31MiB at 769.03KiB/s ETA Unknown (frag 10/31)
[download] 32.1% of ~ 1.31MiB at 769.03KiB/s ETA Unknown (frag 10/31)
[download] 32.3% of ~ 1.32MiB at 769.03KiB/s ETA Unknown (frag 10/31)
[download] 32.6% of ~ 1.33MiB at 769.03KiB/s ETA Unknown (frag 10/31)
[download] 33.2% of ~ 1.35MiB at 769.03KiB/s ETA Unknown (frag 10/31)
[download] 33.3% of ~ 1.40MiB at 769.03KiB/s ETA Unknown (frag 10/31)
[download] 32.3% of ~ 1.44MiB at 769.03KiB/s ETA Unknown (frag 11/31)
[download] 35.2% of ~ 1.32MiB at 769.03KiB/s ETA Unknown (frag 11/31)
[download] 35.3% of ~ 1.33MiB at 769.03KiB/s ETA Unknown (frag 11/31)
[download] 35.5% of ~ 1.33MiB at 769.03KiB/s ETA Unknown (frag 11/31)
[download] 35.8% of ~ 1.34MiB at 769.03KiB/s ETA Unknown (frag 11/31)
[download] 36.4% of ~ 1.36MiB at 769.03KiB/s ETA Unknown (frag 11/31)
[download] 36.5% of ~ 1.40MiB at 769.03KiB/s ETA Unknown (frag 11/31)
[download] 35.5% of ~ 1.44MiB at 769.03KiB/s ETA Unknown (frag 12/31)
[download] 38.5% of ~ 1.33MiB at 814.75KiB/s ETA Unknown (frag 12/31)
[download] 38.6% of ~ 1.33MiB at 814.75KiB/s ETA Unknown (frag 12/31)
[download] 38.8% of ~ 1.34MiB at 814.75KiB/s ETA Unknown (frag 12/31)
[download] 39.1% of ~ 1.35MiB at 814.75KiB/s ETA Unknown (frag 12/31)
[download] 39.7% of ~ 1.37MiB at 814.75KiB/s ETA Unknown (frag 12/31)
[download] 39.8% of ~ 1.40MiB at 814.75KiB/s ETA Unknown (frag 12/31)
[download] 38.7% of ~ 1.44MiB at 814.75KiB/s ETA Unknown (frag 13/31)
[download] 41.8% of ~ 1.34MiB at 843.59KiB/s ETA Unknown (frag 13/31)
[download] 41.9% of ~ 1.34MiB at 843.59KiB/s ETA Unknown (frag 13/31)
[download] 42.0% of ~ 1.35MiB at 843.59KiB/s ETA Unknown (frag 13/31)
[download] 42.3% of ~ 1.35MiB at 843.59KiB/s ETA Unknown (frag 13/31)
[download] 42.9% of ~ 1.37MiB at 843.59KiB/s ETA Unknown (frag 13/31)
[download] 43.0% of ~ 1.41MiB at 843.59KiB/s ETA Unknown (frag 13/31)
[download] 42.0% of ~ 1.44MiB at 843.59KiB/s ETA Unknown (frag 14/31)
[download] 45.0% of ~ 1.35MiB at 855.11KiB/s ETA Unknown (frag 14/31)
[download] 45.1% of ~ 1.35MiB at 855.11KiB/s ETA Unknown (frag 14/31)
[download] 45.2% of ~ 1.35MiB at 855.11KiB/s ETA Unknown (frag 14/31)
[download] 45.5% of ~ 1.36MiB at 855.11KiB/s ETA Unknown (frag 14/31)
[download] 46.1% of ~ 1.38MiB at 855.11KiB/s ETA Unknown (frag 14/31)
[download] 46.2% of ~ 1.41MiB at 855.11KiB/s ETA Unknown (frag 14/31)
[download] 45.2% of ~ 1.44MiB at 855.11KiB/s ETA Unknown (frag 15/31)
[download] 48.3% of ~ 1.35MiB at 863.33KiB/s ETA Unknown (frag 15/31)
[download] 48.3% of ~ 1.35MiB at 863.33KiB/s ETA Unknown (frag 15/31)
[download] 48.5% of ~ 1.36MiB at 863.33KiB/s ETA Unknown (frag 15/31)
[download] 48.8% of ~ 1.36MiB at 863.33KiB/s ETA Unknown (frag 15/31)
[download] 49.4% of ~ 1.38MiB at 863.33KiB/s ETA Unknown (frag 15/31)
[download] 49.5% of ~ 1.41MiB at 863.33KiB/s ETA Unknown (frag 15/31)
[download] 48.4% of ~ 1.44MiB at 863.33KiB/s ETA Unknown (frag 16/31)
[download] 51.5% of ~ 1.36MiB at 868.59KiB/s ETA Unknown (frag 16/31)
[download] 51.5% of ~ 1.36MiB at 868.59KiB/s ETA Unknown (frag 16/31)
[download] 51.7% of ~ 1.36MiB at 868.59KiB/s ETA Unknown (frag 16/31)
[download] 52.0% of ~ 1.37MiB at 868.59KiB/s ETA Unknown (frag 16/31)
[download] 52.6% of ~ 1.38MiB at 868.59KiB/s ETA Unknown (frag 16/31)
[download] 52.7% of ~ 1.41MiB at 868.59KiB/s ETA Unknown (frag 16/31)
[download] 51.6% of ~ 1.44MiB at 868.59KiB/s ETA Unknown (frag 17/31)
[download] 54.7% of ~ 1.36MiB at 868.59KiB/s ETA Unknown (frag 17/31)
[download] 54.8% of ~ 1.36MiB at 868.59KiB/s ETA Unknown (frag 17/31)
[download] 55.0% of ~ 1.37MiB at 868.59KiB/s ETA Unknown (frag 17/31)
[download] 55.3% of ~ 1.37MiB at 868.59KiB/s ETA Unknown (frag 17/31)
[download] 55.8% of ~ 1.39MiB at 883.92KiB/s ETA Unknown (frag 17/31)
[download] 55.9% of ~ 1.41MiB at 883.92KiB/s ETA Unknown (frag 17/31)
[download] 54.8% of ~ 1.44MiB at 883.92KiB/s ETA Unknown (frag 18/31)
[download] 58.0% of ~ 1.37MiB at 883.92KiB/s ETA Unknown (frag 18/31)
[download] 58.1% of ~ 1.37MiB at 883.92KiB/s ETA Unknown (frag 18/31)
[download] 58.2% of ~ 1.3… (6693 chars truncated)
… (48 lines truncated)
<truncation_notice>
Full output written to: /var/folders/d9/mzhyr7lx6xbf236j5ll9v_300000gn/T/devin-overflows-501/b2847529/content.txt
</truncation_notice>yt-dlp downloading. Need continue polling.
{
"shell_id": "61790a",
"timeout": 120000
}=== WALL CLOCK: 50.7s (0.85 min) === === Saving transcript === JSON: /Users/root1/2075271759289303522.json TXT: /Users/root1/2075271759289303522.txt Can AI be trusted? Who's going to hit the brakes if we need to? How do we really ensure that what we're aiming to achieve really does benefit the majority of people? If it ends up taking almost all the jobs, then what does it mean to work? Well, wait a minute. Why do we have to have this stuff? If a machine can pretend to care better than I can actually care, how do we draw the line there? If we all had a voice in it, then I feel like it would be better. Could AI help people stop feeling misunderstood? Could AI help me build more connections in the community? helped me be a better teacher, a better mom. Maybe it'll cure some great things, you know? Things that we're not even at the cusp of understanding yet. Will it create a group of people that ask more questions? What if we started to be more human again? We don't want to lose the most beautiful parts of life. Exit code: 0
Transcription completed. Need present transcript. Also maybe mention files saved. Let's read the txt file to include timing/segments? The stdout already gave full transcript. We could include final transcript and file locations. Maybe include timed segments? The transcript text is enough. Need verify txt content includes header and timed segments. Let's read txt maybe. Also maybe answer question? User just said "use whisper on modal". We can say done, output transcript and files. Could include metadata. Let's read file.
{
"file_path": "/Users/root1/2075271759289303522.txt",
"limit": 50
}<file-view path="/Users/root1/2075271759289303522.txt" start_line="1" end_line="12" total_lines="12"> 1|Language: en (96.4%) 2|Duration: 90.2s (2 min) 3|Load: 15.1s | Transcribe: 2.7s | Wall: 50.7s 4|Segments: 3 5| 6|=== FULL TRANSCRIPT === 7|Can AI be trusted? Who's going to hit the brakes if we need to? How do we really ensure that what we're aiming to achieve really does benefit the majority of people? If it ends up taking almost all the jobs, then what does it mean to work? Well, wait a minute. Why do we have to have this stuff? If a machine can pretend to care better than I can actually care, how do we draw the line there? If we all had a voice in it, then I feel like it would be better. Could AI help people stop feeling misunderstood? Could AI help me build more connections in the community? helped me be a better teacher, a better mom. Maybe it'll cure some great things, you know? Things that we're not even at the cusp of understanding yet. Will it create a group of people that ask more questions? What if we started to be more human again? We don't want to lose the most beautiful parts of life. 8| 9|=== TIMED SEGMENTS === 10|[0002.26 - 0028.08] Can AI be trusted? Who's going to hit the brakes if we need to? How do we really ensure that what we're aiming to achieve really does benefit the majority of people? If it ends up taking almost all the jobs, then what does it mean to work? 11|[0028.08 - 0057.54] Well, wait a minute. Why do we have to have this stuff? If a machine can pretend to care better than I can actually care, how do we draw the line there? If we all had a voice in it, then I feel like it would be better. Could AI help people stop feeling misunderstood? Could AI help me build more connections in the community? 12|[0057.54 - 0081.58] helped me be a better teacher, a better mom. Maybe it'll cure some great things, you know? Things that we're not even at the cusp of understanding yet. Will it create a group of people that ask more questions? What if we started to be more human again? We don't want to lose the most beautiful parts of life. </file-view>
Need output final. The transcript is about AI trust, jobs, care, community, being human. Files saved: /Users/root1/2075271759289303522.txt and .json. Duration 90.2s, wall clock 50.7s. Should include full transcript and mention files. Maybe mention segments. Keep concise.
Done. The X video was transcribed on Modal A10G in 50.7s (audio duration 90.2s).
Transcript:
Can AI be trusted? Who's going to hit the brakes if we need to? How do we really ensure that what we're aiming to achieve really does benefit the majority of people? If it ends up taking almost all the jobs, then what does it mean to work? Well, wait a minute. Why do we have to have this stuff? If a machine can pretend to care better than I can actually care, how do we draw the line there? If we all had a voice in it, then I feel like it would be better. Could AI help people stop feeling misunderstood? Could AI help me build more connections in the community? Helped me be a better teacher, a better mom. Maybe it'll cure some great things, you know? Things that we're not even at the cusp of understanding yet. Will it create a group of people that ask more questions? What if we started to be more human again? We don't want to lose the most beautiful parts of life.
Saved outputs:
now do it for this https://x.com/Madisonkanna/status/2077067032152182986
User wants whisper transcription for another X video. Need run script again. Use same command. Since previous succeeded, do same. Possibly use outdir to save in same place? Default cwd /Users/root1. Name derived from last path segment status id. Run background.
{
"command": "set -a; . ~/.env; set +a; python3 ~/Downloads/utilities/whisper_x/whisper_transcribe.py \"https://x.com/Madisonkanna/status/2077067032152182986\"",
"run_in_background": true,
"timeout": 300000
}Command running in background with ID: 919d3d
Poll get_output.
{
"shell_id": "919d3d",
"timeout": 120000
}
=== Downloading audio from https://x.com/Madisonkanna/status/2077067032152182986 ===
$ yt-dlp -f bestaudio/best -o /Users/root1/source.%(ext)s https://x.com/Madisonkanna/status/2077067032152182986
WARNING: Your yt-dlp version (2025.12.08) is older than 90 days!
It is strongly recommended to always use the latest version.
You installed yt-dlp with pip or using the wheel from PyPi; Use that to update.
To suppress this warning, add --no-update to your command/config.
[twitter] Extracting URL: https://x.com/Madisonkanna/status/2077067032152182986
[twitter] 2077067032152182986: Downloading guest token
[twitter] 2077067032152182986: Downloading GraphQL JSON
[twitter] 2077067032152182986: Downloading m3u8 information
[info] 2077065446428446720: Downloading 1 format(s): hls-audio-128000-Audio
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 1290
[download] Destination: /Users/root1/source.mp4
[download] 0.1% of ~ 990.18KiB at 0.00B/s ETA Unknown (frag 0/1290)
[download] 0.0% of ~ 1.93MiB at 0.00B/s ETA Unknown (frag 1/1290)
[download] 0.0% of ~ 30.79MiB at 7.54KiB/s ETA Unknown (frag 1/1290)
[download] 0.0% of ~ 31.42MiB at 7.54KiB/s ETA Unknown (frag 1/1290)
[download] 0.0% of ~ 32.68MiB at 7.54KiB/s ETA Unknown (frag 1/1290)
[download] 0.0% of ~ 35.20MiB at 7.54KiB/s ETA Unknown (frag 1/1290)
[download] 0.1% of ~ 40.24MiB at 7.54KiB/s ETA Unknown (frag 1/1290)
[download] 0.1% of ~ 50.32MiB at 7.54KiB/s ETA Unknown (frag 1/1290)
[download] 0.1% of ~ 61.10MiB at 7.54KiB/s ETA Unknown (frag 2/1290)
[download] 0.1% of ~ 40.36MiB at 7.54KiB/s ETA Unknown (frag 2/1290)
[download] 0.1% of ~ 40.78MiB at 7.54KiB/s ETA Unknown (frag 2/1290)
[download] 0.1% of ~ 41.62MiB at 7.54KiB/s ETA Unknown (frag 2/1290)
[download] 0.1% of ~ 43.30MiB at 164.11KiB/s ETA Unknown (frag 2/1290)
[download] 0.2% of ~ 46.66MiB at 164.11KiB/s ETA Unknown (frag 2/1290)
[download] 0.2% of ~ 53.38MiB at 164.11KiB/s ETA Unknown (frag 2/1290)
[download] 0.2% of ~ 60.19MiB at 164.11KiB/s ETA Unknown (frag 3/1290)
[download] 0.2% of ~ 45.26MiB at 164.11KiB/s ETA Unknown (frag 3/1290)
[download] 0.2% of ~ 45.58MiB at 164.11KiB/s ETA Unknown (frag 3/1290)
[download] 0.2% of ~ 46.21MiB at 164.11KiB/s ETA Unknown (frag 3/1290)
[download] 0.2% of ~ 47.47MiB at 164.11KiB/s ETA Unknown (frag 3/1290)
[download] 0.2% of ~ 49.98MiB at 164.11KiB/s ETA Unknown (frag 3/1290)
[download] 0.3% of ~ 55.02MiB at 164.11KiB/s ETA Unknown (frag 3/1290)
[download] 0.2% of ~ 60.25MiB at 164.11KiB/s ETA Unknown (frag 4/1290)
[download] 0.3% of ~ 48.27MiB at 331.98KiB/s ETA Unknown (frag 4/1290)
[download] 0.3% of ~ 48.52MiB at 331.98KiB/s ETA Unknown (frag 4/1290)
[download] 0.3% of ~ 49.03MiB at 331.98KiB/s ETA Unknown (frag 4/1290)
[download] 0.3% of ~ 50.03MiB at 331.98KiB/s ETA Unknown (frag 4/1290)
[download] 0.3% of ~ 52.05MiB at 331.98KiB/s ETA Unknown (frag 4/1290)
[download] 0.3% of ~ 56.08MiB at 331.98KiB/s ETA Unknown (frag 4/1290)
[download] 0.3% of ~ 60.33MiB at 331.98KiB/s ETA Unknown (frag 5/1290)
[download] 0.4% of ~ 50.13MiB at 331.98KiB/s ETA Unknown (frag 5/1290)
[download] 0.4% of ~ 50.34MiB at 331.98KiB/s ETA Unknown (frag 5/1290)
[download] 0.4% of ~ 50.76MiB at 331.98KiB/s ETA Unknown (frag 5/1290)
[download] 0.4% of ~ 51.60MiB at 331.98KiB/s ETA Unknown (frag 5/1290)
[download] 0.4% of ~ 53.28MiB at 331.98KiB/s ETA Unknown (frag 5/1290)
[download] 0.4% of ~ 56.64MiB at 331.98KiB/s ETA Unknown (frag 5/1290)
[download] 0.4% of ~ 60.04MiB at 331.98KiB/s ETA Unknown (frag 6/1290)
[download] 0.5% of ~ 51.58MiB at 479.16KiB/s ETA Unknown (frag 6/1290)
[download] 0.5% of ~ 51.76MiB at 479.16KiB/s ETA Unknown (frag 6/1290)
[download] 0.5% of ~ 52.12MiB at 479.16KiB/s ETA Unknown (frag 6/1290)
[download] 0.5% of ~ 52.84MiB at 479.16KiB/s ETA Unknown (frag 6/1290)
[download] 0.5% of ~ 54.28MiB at 479.16KiB/s ETA Unknown (frag 6/1290)
[download] 0.5% of ~ 57.16MiB at 479.16KiB/s ETA Unknown (frag 6/1290)
[download] 0.5% of ~ 60.20MiB at 479.16KiB/s ETA Unknown (frag 7/1290)
[download] 0.5% of ~ 52.57MiB at 479.16KiB/s ETA Unknown (frag 7/1290)
[download] 0.5% of ~ 52.73MiB at 479.16KiB/s ETA Unknown (frag 7/1290)
[download] 0.5% of ~ 53.05MiB at 479.16KiB/s ETA Unknown (frag 7/1290)
[download] 0.5% of ~ 53.68MiB at 479.16KiB/s ETA Unknown (frag 7/1290)
[download] 0.6% of ~ 54.94MiB at 479.16KiB/s ETA Unknown (frag 7/1290)
[download] 0.6% of ~ 57.46MiB at 479.16KiB/s ETA Unknown (frag 7/1290)
[download] 0.5% of ~ 60.01MiB at 479.16KiB/s ETA Unknown (frag 8/1290)
[download] 0.6% of ~ 53.41MiB at 604.82KiB/s ETA Unknown (frag 8/1290)
[download] 0.6% of ~ 53.55MiB at 604.82KiB/s ETA Unknown (frag 8/1290)
[download] 0.6% of ~ 53.83MiB at 604.82KiB/s ETA Unknown (frag 8/1290)
[download] 0.6% of ~ 54.39MiB at 604.82KiB/s ETA Unknown (frag 8/1290)
[download] 0.6% of ~ 55.51MiB at 604.82KiB/s ETA Unknown (frag 8/1290)
[download] 0.6% of ~ 57.75MiB at 604.82KiB/s ETA Unknown (frag 8/1290)
[download] 0.6% of ~ 60.09MiB at 604.82KiB/s ETA Unknown (frag 9/1290)
[download] 0.7% of ~ 54.05MiB at 604.82KiB/s ETA Unknown (frag 9/1290)
[download] 0.7% of ~ 54.18MiB at 604.82KiB/s ETA Unknown (frag 9/1290)
[download] 0.7% of ~ 54.43MiB at 604.82KiB/s ETA Unknown (frag 9/1290)
[download] 0.7% of ~ 54.94MiB at 604.82KiB/s ETA Unknown (frag 9/1290)
[download] 0.7% of ~ 55.94MiB at 604.82KiB/s ETA Unknown (frag 9/1290)
[download] 0.7% of ~ 57.96MiB at 604.82KiB/s ETA Unknown (frag 9/1290)
[download] 0.7% of ~ 60.04MiB at 604.82KiB/s ETA Unknown (frag 10/1290)
[download] 0.8% of ~ 54.56MiB at 708.33KiB/s ETA Unknown (frag 10/1290)
[download] 0.8% of ~ 54.67MiB at 708.33KiB/s ETA Unknown (frag 10/1290)
[download] 0.8% of ~ 54.90MiB at 708.33KiB/s ETA Unknown (frag 10/1290)
[download] 0.8% of ~ 55.36MiB at 708.33KiB/s ETA Unknown (frag 10/1290)
[download] 0.8% of ~ 56.28MiB at 708.33KiB/s ETA Unknown (frag 10/1290)
[download] 0.8% of ~ 58.11MiB at 708.33KiB/s ETA Unknown (frag 10/1290)
[download] 0.8% of ~ 59.97MiB at 708.33KiB/s ETA Unknown (frag 11/1290)
[download] 0.8% of ~ 55.02MiB at 708.33KiB/s ETA Unknown (frag 11/1290)
[download] 0.8% of ~ 55.12MiB at 708.33KiB/s ETA Unknown (frag 11/1290)
[download] 0.9% of ~ 55.33MiB at 708.33KiB/s ETA Unknown (frag 11/1290)
[download] 0.9% of ~ 55.75MiB at 708.33KiB/s ETA Unknown (frag 11/1290)
[download] 0.9% of ~ 56.59MiB at 708.33KiB/s ETA Unknown (frag 11/1290)
[download] 0.9% of ~ 58.27MiB at 708.33KiB/s ETA Unknown (frag 11/1290)
[download] 0.9% of ~ 60.03MiB at 708.33KiB/s ETA Unknown (frag 12/1290)
[download] 0.9% of ~ 55.39MiB at 785.51KiB/s ETA Unknown (frag 12/1290)
[download] 0.9% of ~ 55.49MiB at 785.51KiB/s ETA Unknown (frag 12/1290)
[download] 0.9% of ~ 55.68MiB at 785.51KiB/s ETA Unknown (frag 12/1290)
[download] 0.9% of ~ 56.07MiB at 785.51KiB/s ETA Unknown (frag 12/1290)
[download] 1.0% of ~ 56.85MiB at 785.51KiB/s ETA Unknown (frag 12/1290)
[download] 1.0% of ~ 58.40MiB at 785.51KiB/s ETA Unknown (frag 12/1290)
[download] 0.9% of ~ 60.00MiB at 785.51KiB/s ETA Unknown (frag 13/1290)
[download] 1.0% of ~ 55.69MiB at 836.03KiB/s ETA Unknown (frag 13/1290)
[download] 1.0% of ~ 55.78MiB at 836.03KiB/s ETA Unknown (frag 13/1290)
[download] 1.0% of ~ 55.96MiB at 836.03KiB/s ETA Unknown (frag 13/1290)
[download] 1.0% of ~ 56.32MiB at 836.03KiB/s ETA Unknown (frag 13/1290)
[download] 1.0% of ~ 57.04MiB at 836.03KiB/s ETA Unknown (frag 13/1290)
[download] 1.0% of ~ 58.48MiB at 836.03KiB/s ETA Unknown (frag 13/1290)
[download] 1.0% of ~ 59.95MiB at 836.03KiB/s ETA Unknown (frag 14/1290)
[download] 1.1% of ~ 55.99MiB at 836.03KiB/s ETA Unknown (frag 14/1290)
[download] 1.1% of ~ 56.08MiB at 836.03KiB/s ETA Unknown (frag 14/1290)
[download] 1.1% of ~ 56.24MiB at 836.03KiB/s ETA Unknown (frag 14/1290)
[download] 1.1% of ~ 56.58MiB at 836.03KiB/s ETA Unknown (frag 14/1290)
[download] 1.1% of ~ 57.25MiB at 836.03KiB/s ETA Unknown (frag 14/1290)
[download] 1.1% of ~ 58.60MiB at 836.03KiB/s ETA Unknown (frag 14/1290)
[download] 1.1% of ~ 60.00MiB at 836.03KiB/s ETA Unknown (frag 15/1290)
[download] 1.2% of ~ 56.23MiB at 877.96KiB/s ETA Unknown (frag 15/1290)
[download] 1.2% of ~ 56.30MiB at 877.96KiB/s ETA Unknown (frag 15/1290)
[download] 1.2% of ~ 56.46MiB at 877.96KiB/s ETA Unknown (frag 15/1290)
[download] 1.2% of ~ 56.78MiB at 877.96KiB/s ETA Unknown (frag 15/1290)
[download] 1.2% of ~ 57.41MiB at 877.96KiB/s ETA Unknown (frag 15/1290)
[download] 1.2% of ~ 58.67MiB at 877.96KiB/s ETA Unknown (frag 15/1290)
[download] 1.2% of ~ 59.96MiB at 877.96KiB/s ETA Unknown (frag 16/1290)
[download] 1.2% of ~ 56.45MiB at 877.96KiB/s ETA Unknown (frag 16/1290)
[download] 1.2% of ~ 56.52MiB at 877.96KiB/s ETA Unknown (frag 16/1290)
[download] 1.2% of ~ 56.67MiB at 877.96KiB/s ETA Unknown (frag 16/1290)
[download] 1.2% of ~ 56.97MiB at 877.96KiB/s ETA Unknown (frag 16/1290)
[download] 1.3% of ~ 57.56MiB at 877.96KiB/s ETA Unknown (frag 16/1290)
[download] 1.3% of ~ 58.75MiB at 877.96KiB/s ETA Unknown (frag 16/1290)
[download] 1.2% of ~ 59.98MiB at 925.71KiB/s ETA Unknown (frag 17/1290)
[download] 1.3% of ~ 56.64MiB at 925.71KiB/s ETA Unknown (frag 17/1290)
[download] 1.3% of ~ 56.71MiB at 925.71KiB/s ETA Unknown (frag 17/1290)
[download] 1.3% of ~ 56.85MiB at 925.71KiB/s ETA Unknown (frag 17/1290)
[download] 1.3% of ~ 57.13MiB at 925.71KiB/s ETA Unknown (frag 17/1290)
[download] 1.3% of ~ 57.69MiB at 925.71KiB/s ETA Unknown (frag 17/1290)
[download] 1.3% of ~ 58.81MiB at 925.71KiB/s ETA Unknown (frag 17/1290)
[download] 1.3% of ~ 59.97MiB at 925.71KiB/s ETA Unknown (frag 18/1290)
[download] 1.4% of ~ 56.81MiB at 943.05KiB/s ETA Unknown (frag 18/1290)
[download] 1.4% of ~ 56.87MiB at 943.05KiB/s ETA Unknown (frag 18/1290)
[download] 1.4% of ~ 57.01MiB at 943.05KiB/s ETA Unknown (frag 18/1290)
[download] 1.4% of ~ 57.27MiB at 943.05KiB/s ETA Unknown (frag 18/1290)
[download] 1.4% of ~ 57.80MiB at 943.05KiB/s ETA Unknown (frag 18/1290)
[download] 1.4% of ~ 58.86MiB at 943.05KiB/s ETA Unknown (frag 18/1290)
[download] 1.4% of ~ 59.95MiB at 943.05KiB/s ETA Unknown (frag 19/1290)
[download] 1.5% of ~ 56.96MiB at 953.32KiB/s ETA Unknown (frag 19/1290)
[download] 1.5% of ~ 57.03MiB at 953.32KiB/s ETA Unknown (frag 19/1290)
[download] 1.5% of ~ 57.15MiB at 953.32KiB/s ETA Unknown (frag 19/1290)
[download] 1.5% of ~ 57.41MiB at 953.32KiB/s ETA Unknown (frag 19/1290)
[download] 1.5% of ~ 57.91MiB at 953.32KiB/s ETA Unknown (frag 19/1290)
[download] 1.5% of ~ 58.92MiB at 953.32KiB/s ETA Unknown (frag 19/1290)
[download] 1.5% of ~ 59.96MiB at 953.32KiB/s ETA Unknown (frag 20/1290)
[download] 1.5% of ~ 57.12MiB at 953.32KiB/s ETA Unknown (frag 20/1290)
[download] 1.5% of ~ 57.18MiB at 953.32KiB/s ETA Unknown (frag 20/1290)
[download] 1.6% of ~ 57.30MiB at 953.32KiB/s ETA Unknown (frag 20/1290)
[download] 1.6% of ~ 57.54MiB at 953.32KiB/s ETA Unknown (frag 20/1290)
[download] 1.6% of ~ 58.02MiB at 953.32KiB/s ETA Unknown (frag 20/1290)
[download] 1.6% of ~ 58.98MiB at 953.32KiB/s ETA Unknown (frag 20/1290)
[download] 1.6% of ~ 59.98MiB at 953.32KiB/s ETA Unknown (frag 21/1290)
[download] 1.6% of ~ 57.23MiB at 957.73KiB/s ETA Unknown (frag 21/1290)
[download] 1.6% of ~ 57.28MiB at 957.73KiB/s ETA Unknown (frag 21/1290)
[download] 1.6% of ~ 57.40MiB at 957.73KiB/s ETA Unknown (frag 21/1290)
[download] 1.6% of ~ 57.63MiB at 957.73KiB/s ETA Unknown (frag 21/1290)
[download] 1.7% of ~ 58.09MiB at 957.73KiB/s ETA Unknown (frag 21/1290)
[download] 1.7% of ~ 59.00MiB at 957.73KiB/s ETA Unknown (frag 21/1290)
[download] 1.6% of ~ 59.93MiB at 957.73KiB/s ETA Unknown (frag 22/1290)
[download] 1.7% of ~ 57.35MiB at 957.73KiB/s ETA Unknown (frag 22/1290)
[download] 1.7% of ~ 57.41MiB at 957.73KiB/s ETA Unknown (frag 22/1290)
[download] 1.7% of ~ 57.52MiB at 957.73KiB/s ETA Unknown (frag 22/1290)
[download] 1.7% of ~ 57.74MiB at 957.73KiB/s ETA Unknown (frag 22/1290)
[download] 1.7% of ~ 58.18MiB at 957.73KiB/s ETA Unknown (frag 22/1290)
[download] 1.7% of ~ 59.05MiB at 957.73KiB/s ETA Unknown (frag 22/1290)
[download] 1.7% of ~ 59.97MiB at 957.73KiB/s ETA Unknown (frag 23/1290)
[download] 1.8% of ~ 57.45MiB at 963.49KiB/s ETA 00:59 (frag 23/1290)
[download] 1.8% of ~ 57.51MiB at 963.49KiB/s ETA 00:59 (frag 23/1290)
[download] 1.8% of ~ 57.61MiB at 963.49KiB/s ETA 00:59 (frag 23/1290)
[download] 1.8% of ~ 57.82MiB at 963.49KiB/s ETA 00:59 (frag 23/1290)
[download] 1.8% of ~ 58.24MiB at 963.49KiB/s ETA 00:59 (frag 23/1290)
[download] 1.8% of ~ 59.08MiB at 963.49KiB/s ETA 00:59 (frag 23/1290)
[download] 1.8% of ~ 59.94MiB at 963.49KiB/s ETA 00:59 (frag 24/1290)
[download] 1.9% of ~ 57.55MiB at 963.49KiB/s ETA 00:59 (frag 24/1290)
[download] 1.9% of ~ 57.60MiB at 963.49KiB/s ETA 00:59 (frag 24/1290)
[download] 1.9% of ~ 57.70MiB at 963.49KiB/s ETA 00:59 (frag 24/1290)
[download] 1.9% of ~ 57.90MiB at 963.49KiB/s ETA 00:59 (frag 24/1290)
[download] 1.9% of ~ 58.31MiB at 963.49KiB/s ETA 00:59 (frag 24/1290)
[download] 1.9% of ~ 59.11MiB at 963.49KiB/s ETA 00:59 (frag 24/1290)
[download] 1.9% of ~ 59.95MiB at 963.49KiB/s ETA 00:59 (frag 25/1290)
[download] 1.9% of ~ 57.65MiB at 969.91KiB/s ETA 00:59 (frag 25/1290)
[download] 1.9% of ~ 57.70MiB at 969.91KiB/s ETA 00:59 (frag 25/1290)
[download] 1.9% of ~ 57.79MiB at 969.91KiB/s ETA 00:59 (frag 25/1290)
[download] 1.9% of ~ 57.99MiB at 969.91KiB/s ETA 00:59 (frag 25/1290)
[download] 2.0% of ~ 58.38MiB at 969.91KiB/s ETA 00:59 (frag 25/1290)
[download] 2.0% of ~ 59.15MiB at 969.91KiB/s ETA 00:59 (frag 25/1290)
[download] 1.9% of ~ 59.96MiB at 969.91KiB/s ETA 00:59 (frag 26/1290)
[download] 2.0% of ~ 57.73MiB at 969.91KiB/s ETA 00:59 (frag 26/1290)
[download] 2.0% of ~ 57.78MiB at 969.91KiB/s ETA 00:59 (frag 26/1290)
[download] 2.0% of ~ 57.87MiB at 969.91KiB/s ETA 00:59 (frag 26/1290)
[download] 2.0% of ~ 58.06MiB at 969.91KiB/s ETA 00:59 (frag 26/1290)
[download] 2.0% of ~ 58.43MiB at 969.91KiB/s ETA 00:59 (frag 26/1290)
[download] 2.0% of ~ 59.18MiB at 969.91KiB/s ETA 00:59 (frag 26/1290)
[download] 2.0% of ~ 59.94MiB at 969.91KiB/s ETA 00:59 (frag 27/1290)
[download] 2.1% of ~ 57.80MiB at 976.85KiB/s ETA 00:59 (frag 27/1290)
[download] 2.1% of ~ 57.85MiB at 976.85KiB/s ETA 00:59 (frag 27/1290)
[download] 2.1% of ~ 57.94MiB at 976.85KiB/s ETA 00:59 (frag 27/1290)
[download] 2.1% of ~ 58.12MiB at 976.85KiB/s ETA 00:59 (frag 27/1290)
[download] 2.1% of ~ 58.48MiB at 976.85KiB/s ETA 00:59 (frag 27/1290)
[download] 2.1% of ~ 59.20MiB at 976.85KiB/s ETA 00:59 (frag 27/1290)
[download] 2.1% of ~ 59.94MiB at 976.85KiB/s ETA 00:59 (frag 28/1290)
[download] 2.2% of ~ 57.88MiB at 976.85KiB/s ETA 00:59 (frag 28/1290)
[download] 2.2% of ~ 57.93MiB at 976.85KiB/s ETA 00:59 (frag 28/1290)
[download] 2.2% of ~ 58.01MiB at 976.85KiB/s ETA 00:59 (frag 28/1290)
[download] 2.2% of ~ 58.19MiB at 976.85KiB/s ETA 00:59 (frag 28/1290)
[download] 2.2% of ~ 58.54MiB at 976.85KiB/s ETA 00:59 (frag 28/1290)
[download] 2.2% of ~ 59.23MiB at 976.85KiB/s ETA 00:59 (frag 28/1290)
[download] 2.2% of ~ 59.96MiB at 976.85KiB/s ETA 00:59 (frag 29/1290)
[download] 2.2% of ~ 57.95MiB at 985.43KiB/s ETA 00:58 (frag 29/1290)
[download] 2.2% of ~ 57.99MiB at 985.43KiB/s ETA 00:58 (frag 29/1290)
[download] 2.3% of ~ 58.07MiB at 985.43KiB/s ETA 00:58 (frag 29/1290)
[download] 2.3% of ~ 58.24MiB at 985.43KiB/s ETA 00:58 (frag 29/1290)
[download] 2.3% of ~ 58.58MiB at 985.43KiB/s ETA 00:58 (frag 29/1290)
[download] 2.3% of ~ 59.25MiB at 985.43KiB/s ETA 00:58 (frag 29/1290)
[download] 2.2% of ~ 59.94MiB at 985.43KiB/s ETA 00:58 (frag 30/1290)
[download] 2.3% of ~ 58.01MiB at 985.43KiB/s ETA 00:58 (frag 30/1290)
[download] 2.3% of ~ 58.05MiB at 985.43KiB/s ETA 00:58 (frag 30/1290)
[download] 2.3% of ~ 58.13MiB at 985.43KiB/s ETA 00:58 (frag 30/1290)
[download] 2.3% of ~ 58.29MiB at 985.43KiB/s ETA 00:58 (frag 30/1290)
[download] 2.4% of ~ 58.62MiB at 985.43KiB/s ETA 00:58 (frag 30/1290)
[download] 2.4% of ~ 59.27MiB at 985.43KiB/s ETA 00:58 (frag 30/1290)
[download] 2.3% of ~ 59.94MiB at 985.43KiB/s ETA 00:58 (frag 31/1290)
[download] 2.4% of ~ 58.06MiB at 992.50KiB/s ETA 00:58 (frag 31/1290)
[download] 2.4% of ~ 58.10MiB at 992.50KiB/s ETA 00:58 (frag 31/1290)
[download] 2.4% of ~ 58.18MiB at 992.50KiB/s ETA 00:58 (frag 31/1290)
[download] 2.4% of ~ 58.34MiB at 992.50KiB/s ETA 00:58 (frag 31/1290)
[download] 2.4% of ~ 58.65MiB at 992.50KiB/s ETA 00:58 (frag 31/1290)
[download] 2.4% of ~ 59.28MiB at 992.50KiB/s ETA 00:58 (frag 31/1290)
[download] 2.4% of ~ 59.93MiB at 992.50KiB/s ETA 00:58 (frag 32/1290)
[download] 2.5% of ~ 58.12MiB at 992.50KiB/s ETA 00:58 (frag 32/1290)
[download] 2.5% of ~ 58.16MiB at 992.50KiB/s ETA 00:58 (frag 32/1290)
[download] 2.5% of ~ 58.23MiB at 992.50KiB/s ETA 00:58 (frag 32/1290)
[download] 2.5% of ~ 58.39MiB at 992.50KiB/s ETA 00:58 (frag 32/1290)
[download] 2.5% of ~ 58.69MiB at 992.50KiB/s ETA 00:58 (frag 32/1290)
[download] 2.5% of ~ 59.30MiB at 992.50KiB/s ETA 00:58 (frag 32/1290)
[download] 2.5% of ~ 59.94MiB at 992.50KiB/s ETA 00:58 (frag 33/1290)
[download] 2.6% of ~ 58.18MiB at 998.69KiB/s ETA 00:58 (frag 33/1290)
[download] 2.6% of ~ 58.21MiB at 998.69KiB/s ETA 00:58 (frag 33/1290)
[download] 2.6% of ~ 58.29MiB at 998.69KiB/s ETA 00:58 (frag 33/1290)
[download] 2.6% of ~ 58.44MiB at 998.69KiB/s ETA 00:58 (frag 33/1290)
[download] 2.6% of ~ 58.73MiB at 998.69KiB/s ETA 00:58 (frag 33/1290)
[download] 2.6% of ~ 59.33MiB at 998.69KiB/s ETA 00:58 (frag 33/1290)
[download] 2.6% of ~ 59.95MiB at 998.69KiB/s ETA 00:58 (frag 34/1290)
[download] 2.6% of ~ 58.22MiB at 998.69KiB/s ETA 00:58 (frag 34/1290)
[download] 2.6% of ~ 58.26MiB at 998.69KiB/s ETA 00:58 (frag 34/1290)
[download] 2.6% of ~ 58.33MiB at 998.69KiB/s ETA 00:58 (frag 34/1290)
[download] 2.6% of ~ 58.48MiB at 998.69KiB/s ETA 00:58 (frag 34/1290)
[download] 2.7% of ~ 58.76MiB at 998.69KiB/s ETA 00:58 (frag 34/1290)
[download] 2.7% of ~ 59.34MiB at 998.69KiB/s ETA 00:58 (frag 34/1290)
[download] 2.6% of ~ 59.93MiB at 1011.42KiB/s ETA 00:58 (frag 35/1290)
[download] 2.7% of ~ 58.27MiB at 1011.42KiB/s ETA 00:58 (frag 35/1290)
[download] 2.7% of ~ 58.30MiB at 1011.42KiB/s ETA 00:58 (frag 35/1290)
[download] 2.7% of ~ 58.37MiB at 1011.42KiB/s ETA 00:58 (frag 35/1290)
[download] 2.7% of ~ 58.51MiB at 1011.42KiB/s ETA 00:58 (frag 35/1290)
[download] 2.7% of ~ 58.79MiB at 1011.42KiB/s ETA 00:58 (frag 35/1290)
[download] 2.7% of ~ 59.35MiB at 1011.42KiB/s ETA 00:58 (frag 35/1290)
[download] 2.7% of ~ 59.93MiB at 1011.42KiB/s ETA 00:58 (frag 36/1290)
[download] 2.8% of ~ 58.32MiB at 1014.81KiB/s ETA 00:58 (frag 36/1290)
[download] 2.8% of ~ 58.35MiB at 1014.81KiB/s ETA 00:58 (frag 36/1290)
[download] 2.8% of ~ 58.42MiB at 1014.81KiB/s ETA 00:58 (frag 36/1290)
[download] 2.8% of ~ 58.56MiB at 1014.81KiB/s ETA 00:58 (frag 36/1290)
[download] 2.8% of ~ 58.83MiB at 1014.81KiB/s ETA 00:58 (frag 36/1290)
[download] 2.8% of ~ 59.37MiB at 1014.81KiB/s ETA 00:58 (frag 36/1290)
[download] 2.8% of ~ 59.94MiB at 1014.81KiB/s ETA 00:58 (frag 37/1290)
[downl… (659865 chars truncated)
=== Extracting / re-encoding audio to 16kHz mono ===
$ ffmpeg -y -i /Users/root1/source.mp4 -vn -ac 1 -ar 16000 -c:a aac -b:a 64k /Users/root1/audio.m4a
ffmpeg version 8.0.1 Copyright (c) 2000-2025 the FFmpeg developers
built with Apple clang version 17.0.0 (clang-1700.6.3.2)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/8.0.1_1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gpl --enable-libsvtav1 --enable-libopus --enable-libx264 --enable-libmp3lame --enable-libdav1d --enable-libvpx --enable-libx265 --enable-videotoolbox --enable-audiotoolbox --enable-neon
libavutil 60. 8.100 / 60. 8.100
libavcodec 62. 11.100 / 62. 11.100
libavformat 62. 3.100 / 62. 3.100
libavdevice 62. 1.100 / 62. 1.100
libavfilter 11. 4.100 / 11. 4.100
libswscale 9. 1.100 / 9. 1.100
libswresample 6. 1.100 / 6. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/root1/source.mp4':
Metadata:
major_brand : iso5
minor_version : 1
compatible_brands: iso6iso5cmf2dash
creation_time : 2026-07-14T16:25:29.000000Z
encoder : GPAC-2.2.1-revrelease
Duration: 01:04:27.75, start: 0.000000, bitrate: 129 kb/s
Stream #0:0[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : Twitter-vork muxer
vendor_id : [0][0][0][0]
Stream mapping:
Stream #0:0 -> #0:0 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, ipod, to '/Users/root1/audio.m4a':
Metadata:
major_brand : iso5
minor_version : 1
compatible_brands: iso6iso5cmf2dash
encoder : Lavf62.3.100
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 16000 Hz, mono, fltp, 64 kb/s (default)
Metadata:
encoder : Lavc62.11.100 aac
handler_name : Twitter-vork muxer
vendor_id : [0][0][0][0]
<truncation_notice>
Full output written to: /var/folders/d9/mzhyr7lx6xbf236j5ll9v_300000gn/T/devin-overflows-501/1fd3e426/content.txt
</truncation_notice>Long video (1h4m). Download and re-encode complete. Need continue polling for Modal transcription, which may take ~5 min? Since 65 min audio took 2.3 min wall clock previously (65 min). This is 64 min, likely similar. Continue polling.
{
"shell_id": "919d3d",
"timeout": 300000
}size= 512KiB time=00:01:27.87 bitrate= 47.7kbits/s speed= 174x elapsed=0:00:00.50 size= 1792KiB time=00:03:45.72 bitrate= 65.0kbits/s speed= 223x elapsed=0:00:01.01 size= 2816KiB time=00:05:53.02 bitrate= 65.3kbits/s speed= 233x elapsed=0:00:01.51 size= 4096KiB time=00:08:08.25 bitrate= 68.7kbits/s speed= 242x elapsed=0:00:02.01 size= 5120KiB time=00:10:18.43 bitrate= 67.8kbits/s speed= 245x elapsed=0:00:02.52 size= 6144KiB time=00:12:28.03 bitrate= 67.3kbits/s speed= 247x elapsed=0:00:03.02 size= 7168KiB time=00:14:28.48 bitrate= 67.6kbits/s speed= 246x elapsed=0:00:03.53 size= 8192KiB time=00:16:25.98 bitrate= 68.1kbits/s speed= 244x elapsed=0:00:04.03 size= 9216KiB time=00:18:34.24 bitrate= 67.8kbits/s speed= 246x elapsed=0:00:04.53 size= 10496KiB time=00:20:38.91 bitrate= 69.4kbits/s speed= 246x elapsed=0:00:05.04 size= 11520KiB time=00:22:50.75 bitrate= 68.8kbits/s speed= 247x elapsed=0:00:05.54 size= 12544KiB time=00:25:05.34 bitrate= 68.3kbits/s speed= 249x elapsed=0:00:06.05 size= 13824KiB time=00:27:15.20 bitrate= 69.3kbits/s speed= 249x elapsed=0:00:06.55 size= 14848KiB time=00:29:25.37 bitrate= 68.9kbits/s speed= 250x elapsed=0:00:07.05 size= 15872KiB time=00:31:21.28 bitrate= 69.1kbits/s speed= 249x elapsed=0:00:07.56 size= 16896KiB time=00:33:29.60 bitrate= 68.9kbits/s speed= 249x elapsed=0:00:08.06 size= 17920KiB time=00:35:31.26 bitrate= 68.9kbits/s speed= 249x elapsed=0:00:08.57 size= 19200KiB time=00:37:47.32 bitrate= 69.4kbits/s speed= 250x elapsed=0:00:09.07 size= 20224KiB time=00:40:00.76 bitrate= 69.0kbits/s speed= 251x elapsed=0:00:09.57 size= 21248KiB time=00:42:05.95 bitrate= 68.9kbits/s speed= 251x elapsed=0:00:10.08 size= 22528KiB time=00:44:18.11 bitrate= 69.4kbits/s speed= 251x elapsed=0:00:10.58 size= 23552KiB time=00:46:23.48 bitrate= 69.3kbits/s speed= 251x elapsed=0:00:11.08 size= 24576KiB time=00:48:35.00 bitrate= 69.1kbits/s speed= 251x elapsed=0:00:11.59 size= 25856KiB time=00:50:50.24 bitrate= 69.4kbits/s speed= 252x elapsed=0:00:12.09 size= 26880KiB time=00:53:01.05 bitrate= 69.2kbits/s speed= 252x elapsed=0:00:12.59 size= 27904KiB time=00:55:07.52 bitrate= 69.1kbits/s speed= 252x elapsed=0:00:13.10 size= 28928KiB time=00:57:10.27 bitrate= 69.1kbits/s speed= 252x elapsed=0:00:13.60 size= 30208KiB time=00:59:12.38 bitrate= 69.7kbits/s speed= 252x elapsed=0:00:14.10 size= 31232KiB time=01:01:17.69 bitrate= 69.6kbits/s speed= 252x elapsed=0:00:14.61 size= 32256KiB time=01:03:20.70 bitrate= 69.5kbits/s speed= 251x elapsed=0:00:15.11 [out#0/ipod @ 0xbe500c840] video:0KiB audio:32867KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.722177% size= 33104KiB time=01:04:27.75 bitrate= 70.1kbits/s speed= 252x elapsed=0:00:15.37 [aac @ 0xbe5034a80] Qavg: 1891.077 === Uploading audio.m4a to Modal volume 'whisper-audio' === $ modal volume put whisper-audio /Users/root1/audio.m4a 2077067032152182986_1784062112.m4a ✓ Uploaded file '/Users/root1/audio.m4a' to '2077067032152182986_1784062112.m4a' === Running Whisper large-v3 on Modal A10G (no caching) === === WALL CLOCK: 75.4s (1.26 min) === === Saving transcript === JSON: /Users/root1/2077067032152182986.json TXT: /Users/root1/2077067032152182986.txt For the first time in a long time, people's individual leverage has been exponentially multiplied. Expose yourself to as much of AI and research as possible, but really focus on your unique ability as a human to be creative and discern these cool things with all the context that you have. There were more interesting problems to solve than the ones we were able to solve in our PhDs. We wanted to train models that allowed people to at least give them the option to control their own intelligence rather than relying on the big labs. You went from Australia to San Francisco. What was that like? Yeah, it was a bit of an arduous journey to get here. So I did undergrad in Australia, then was planning on doing my PhD at Oxford, which is where I met Moody through my best friend Max, who was also at Oxford. And then we decided that there were more interesting problems to solve than the ones we were able to solve in our PhDs. took a temporary leave of absence from Oxford. Moody jokes that I'm not a proper Oxford student because I was only there for realistically only a few days. But yeah, we decided to have a crack at a startup and we gave it a year to work. And originally that meant building out of London because that was where everyone's home base was, very close to Oxford. And I think at some point we realized that the serendipity of SF and just the surface area of your luck here. was a real thing and you know you hear it over and over from so many people but it's a hard thing to to grasp and come to terms with until you've actually been here and so I think yeah on a on a trip to SF um after the company had been around for for a few months I think more things happened in four days than had happened in the previous you know four or five months um and we sort of thought okay it's completely irrational to not be here um in the epicenter of where everyone's building and yeah so we moved the whole company here we um we told everyone that that We wanted to be in person in NSF and fortunately everyone we'd hired thought that was the right decision and everyone moved over within, I think, about two weeks and we moved into a hacker house with all eight, nine of us. And yeah, it went from there. What happened in those four or five days? You mean you met a lot of people? Yeah, I think a combination of meeting customers, meeting people that I had looked up to for a long time and like... just not only like seeking them out, but also just like literally running into them at certain events. And I thought, okay, wow, like there's just so many like interaction effects and network effects here that you'd be kind of stupid to ignore. And yeah, as soon as we moved here, I think things really started to kick off for Paz, which was our startup before we joined Base10. And like, it's really easy to like crystallize your vision and where you're headed as a company when you have so much more information being poured in. It's very hard to make that. or do that analysis in a vacuum and thinking through like, okay, where do I think this field is going to be headed and how can I position the company to best take advantage of that? To do that in London is very hard because you're not receiving this influx of information from all these different people with all these different opinions. I mean, you're not able to integrate over all that as well. Yeah. I want to go back to your time at Oxford, like briefly, and tell me more about like what factored into your decision to leave. Yeah, I think it's a PhD versus industry has always been a question. It's not like this has just come up the advent of AI. I think a lot of people who are both very intellectually curious, but also care about the impact that they can have on the world grapple with this when they're deciding whether to go to grad school or not. And it's an age old joke about, you know, doing PhD and being a poor grad student versus, you know, going off and joining big tech and making the money. But for us, it was really about, for the first time in history, I think the impact that an individual could have, or at least a very small group of people, had been multiplied and leveraged significantly more than ever has been, certainly in my lifetime and for quite a long time before that. And so I think the calculus shifts a lot. You have to decide whether you're going to narrowly focus on one very small edge of the periphery of the scientific bubble that you're going to try and advance, or... whether you take a step back and go, okay, I'm going to take a little bit more of a risk here, but I'm going to tackle a problem which, if I solve, could potentially be much bigger and much more impactful for the world. So I think all three of us, Moody, Max, and I felt that. We knew there's advantages to having very academic roots and research-based roots. And in some ways, that was a big advantage for us compared to a lot of the other startups. Yeah, I think the temptation to solve a bigger problem than we could at Oxford alone was just too much to ignore. What do you think was the advantage of the academic route that you talked about? When you're in a startup, you often operate in this mode of like greedy action selection, right? So every day you're trying not to die and it's very easy to get very myopic and say, okay, what is the thing I'm going to do next, which is going to maximize revenue over the next week or the next month or whatever it is. Whereas like in research, I think you have to be, you have to develop a lot more of this longer term strategic planning ability and you have to be able to say okay like if is the thing I'm doing now going to be something that if I weren't going to do it someone else would solve and to me that's not a very interesting problem to most researchers that are that's not a very interesting problem the best research is the ones who can say okay if I weren't going to tackle this like probably no one else would tackle it in the way that I might go about it and that would be a bad thing for the world because I think that this is a solution that should exist or an area I should push on And I think with PaaS, to give a concrete example, we knew that post-training as a service, which is what PaaS did, would eventually become commoditized to an extent. But we also knew that the ways in which people would try and commoditize that would be different to how we would go about it. So everyone was very attracted to verifiable tasks and these very deterministic tasks so that they could focus on the purity of the engineering and algorithmic problems. And people who like RL tend to like that kind of pure maths. problem space. But we thought like, okay, no one's really going to go after these non-verifiable areas like legal and medicine and insurance and things like that. And we knew that solutions wouldn't exist as quickly as if we went in and tried to tackle that. And so to us, like having that kind of long-term thinking and ability to select a problem, which you have a unique position to tackle, was like, yeah, really enticing to us. I want to talk a little bit more about the PhD question, because I feel like a lot of people, think they can't really become like an AI researcher, for example, without a PhD. And I'd love to hear more of your thoughts on that for someone who believes this or is thinking about this. For the first time in a long time, like people's individual leverage has been exponentially multiplied. And I think this is very much the case for research. This is a useful analogy to me when thinking about like the type of stuff that I want to be working on. There's like basketball players who like every single play that you do, you have to hustle and you have to be doing like a lot of individual, very little things, right? And I think like traditionally, like a career has looked a lot like a basketball player. Like you work to develop a set of skills and push to become like, you know, top 10% in a skill that a lot of people have so you can make a good career out of it. And I think the world is now shifting with LLMs because we now have this setup where if a lot of people can do this skill like reasonably well, that's the basis for the training distribution or the RL environment that it's going to allow language models to do that. And so I think now. people are shifting towards like needing to be more baseball players where you might not do anything for a really long time but you have to be able to like kind of hit the home runs and so like from a research perspective that's like what I would be focusing on is like not can I like ingest all this like you know knowledge and history of the field a lot of which has become irrelevant with how quickly language models are moving and and how much that field has evolved in such a short period of time. It's more like, can I develop the metacognition skills to be able to say, okay, again, what is the research problem that I want to work on? And can I hit a home run every once in a while? That a language model is just not going to have the creativity to be able to come up with in the first place, or even if it does come up with it, it's not going to have the value function to be able to discern that that is a good idea amongst the many, many millions of ideas that they can generate. And so, yes, expose yourself to as much of AI and research as possible, but really focus on your unique ability as a human. to be creative and discern like these cool things with all the context that you have and not try and you know start from this bottoms up layer of like i'm going to learn you know matrix calculus and then i'm going to like step through that and then eventually i'll build a neural network and then eventually i'll train and then eventually one day i might get to lms or diffusion models or whatever it is i think the world is going to reward researchers and people in general who are much more top down it's like okay here's this cool idea i have i have no idea how all these intermediate bits work but i'm going to get the thing working then we're going to fill in the skill tree on the way down rather than going bottoms up. Yeah, that's really interesting because I feel like that's not really how we're taught to learn, right? Like in school, you just start from like the ground up and then you work your way over time. Yes, I like completely agree. I think like I have a lot of qualms with like how things like maths and stuff are taught in school in general. But yeah, like so many of AI researchers and people getting into AI research are very predisposed towards, you know, the Andrew Ng. And I love Andrew Ng, but like his style of course is very much, you know. you learn the very basic fundamental building blocks and you slowly, slowly step your way up. I think someone who does this really well is Jeremy Howard. And he's like actually one of the first people that I started like learning AI from through his like fast AI stuff. And his whole thing is like, okay, the first thing you're going to do is train, you know, back in 2018, it was like convolutional neural nets. But he's like, okay, I'm going to give you the stuff so you can train a convolutional neural net in like four lines of code. And then over the course of like you learning about it, you're eventually going to work your way down to be able to write that from scratch yourself in like raw PyTorch. And I think for me that allows you to have so much more velocity, but you need to be comfortable living in a world where you don't know everything and focusing much more on intuition and creativity as opposed to how you would normally build up a skill like maths, which is filling out the skill tree from the bottom. So if I'm learning a new hard topic, like RL or something, don't know much about it, but not starting from like, okay, what's the thing, like 12 levels beneath it, like ground up learning, but just starting at the top. Yeah, like if I wanted to learn RL and be the best at RL, the first thing I would do is grab an open source framework and I wouldn't touch a single line of code, I wouldn't read a single line of code except for the config. And I would try and plug that into some arbitrary environment. It might be an environment I've written, it might be like a standard maths environment. And I would watch that curve go up. And then once I got that curve to go up, I'd maybe change a couple of these hyperparameters and say, oh, okay, that's how the group size affects this curve in this way. And then I would slowly work my way down the code until I fundamentally understood, okay, this is the GLPO algorithm. Here's everything that's wrong with it. Here's what we fixed with the length normalization bias and dividing by standard deviation and everything. And I'd develop that. And I think if I did it that way versus starting from, okay, I'm going to learn policy gradients. I'm going to go back and read Rich Sutton's textbook and work my way through the history of RL so I fully understand everything before I run my first GLPO loop. I think the former case beats the latter case every time and you become a much better researcher. because of it and a much more practical researcher as well. Yeah. The latter case seems not as motivating either. It's like, I want to learn about this thing. Okay. I need to go learn eight other things that are at a more basic level. And then in six months I can actually learn about the thing. Yes. Yeah. Yeah. You need that dopamine hitter, you know, watching the curve go up. Yeah. I want to hear more about your advice for other people who want to become researchers. People often give wishy-washy and like abstract advice. I think maybe the most useful thing I could say is like grounding this in like how I am currently thinking about research. So I think like you of course have to have enough context and enough knowledge of the field in general to be able to have some sort of refined value function over what good ideas are and where the field is headed. And for me, like if I sit back and think about like, okay, what are the most interesting ideas to me personally, to the field and to like me in the context of, for instance, base 10, which cares about inference and open source models and an ecosystem of these specialized open source models. then the kind of thing that sits at the intersection of all that for me is like continual learning. And it's not continual learning in the way that the big labs care about it. It's continual learning for very specific tasks and specific jobs and domains, much like you might have a human who's doing a specific job and needs to continually get better at that job. Whereas the big labs care more about, okay, if there's a very general model, we would like to generally update this distribution with the last six months of internet data. How do we do that without forgetting anything? So first off, I'm like, okay, that's an interesting problem to me and to everyone in general. I think it's an interesting problem to like base 10 in the specific context I'm in, because we have a unique, like potentially more tractable way to tackle this question than someone like OpenAI or Anthropic does, because we just care about continual learning for like these specific jobs for this model that is going to be a lawyer or this model is going to be like, you know, a medical note writer or whatever it is. And then I say, okay, like I've got my kind of first principles axiom of where I think the field is headed. Like how do I work backwards from that to construct a research agenda that is one going to be tractable? Yes. But like two is actually going to push in the direction that I care about. And so first thing I think about with continual learning is, okay, the big limitation to continual learning at the moment is the context window of the model. Like the model is perfect up until, you know, 200K tokens or a million K tokens, a million tokens. But outside of that, it just falls off. And so how do people currently solve this? We have these like markdown memory systems. We do have people on the other end of the spectrum actually trying to extend the context window, but that gets very hard. Like you can't really get around the quadratic complexity of attention. And so to me, then I think, okay, as a researcher, like what is my unique approach going back to this? Like if I went to do this, what would no one else do? Like what would my unique approach here be? And to me, that's like, okay, can I … (57194 chars truncated) Exit code: 0 <truncation_notice> Full output written to: /var/folders/d9/mzhyr7lx6xbf236j5ll9v_300000gn/T/devin-overflows-501/62d7ed42/content.txt </truncation_notice>
Transcription completed. Output truncated in tool because long. Need read txt file to get summary/full? The transcript is large. Need present to user. We can read the .txt file. But it may be huge (maybe ~50k characters). We can read with maybe limit? The read tool limit number of lines, not chars. We could read and then summarize? User just said "do it for this". Likely wants transcript. We can present files and maybe first part and note full transcript available. Better to read first maybe 200 lines to capture enough. Let's read file with limit. Also can provide file path. But perhaps we should give the full transcript? It might be too long for chat. We can include a concise summary plus file locations. Since user asked to use whisper, we can state done and maybe include full transcript? The previous answer included full transcript for 90 sec. This is 64 min, too long. We should summarize and provide file paths. But we need know content to summarize. Could read entire txt. Let's read maybe 300 lines, maybe enough. Need total lines. Use read file.
{
"file_path": "/Users/root1/2077067032152182986.txt",
"limit": 100
}<file-view path="/Users/root1/2077067032152182986.txt" start_line="1" end_line="39" total_lines="146"> 1|Language: en (99.9%) 2|Duration: 3867.8s (64 min) 3|Load: 13.8s | Transcribe: 56.0s | Wall: 75.4s 4|Segments: 137 5| 6|=== FULL TRANSCRIPT === 7|For the first time in a long time, people's individual leverage has been exponentially multiplied. Expose yourself to as much of AI and research as possible, but really focus on your unique ability as a human to be creative and discern these cool things with all the context that you have. There were more interesting problems to solve than the ones we were able to solve in our PhDs. We wanted to train models that allowed people to at least give them the option to control their own intelligence rather than relying on the big labs. You went from Australia to San Francisco. What was that like? Yeah, it was a bit of an arduous journey to get here. So I did undergrad in Australia, then was planning on doing my PhD at Oxford, which is where I met Moody through my best friend Max, who was also at Oxford. And then we decided that there were more interesting problems to solve than the ones we were able to solve in our PhDs. took a temporary leave of absence from Oxford. Moody jokes that I'm not a proper Oxford student because I was only there for realistically only a few days. But yeah, we decided to have a crack at a startup and we gave it a year to work. And originally that meant building out of London because that was where everyone's home base was, very close to Oxford. And I think at some point we realized that the serendipity of SF and just the surface area of your luck here. was a real thing and you know you hear it over and over from so many people but it's a hard thing to to grasp and come to terms with until you've actually been here and so I think yeah on a on a trip to SF um after the company had been around for for a few months I think more things happened in four days than had happened in the previous you know four or five months um and we sort of thought okay it's completely irrational to not be here um in the epicenter of where everyone's building and yeah so we moved the whole company here we um we told everyone that that We wanted to be in person in NSF and ... [truncated 71750 chars] 8| 9|=== TIMED SEGMENTS === 10|[0000.00 - 0028.11] For the first time in a long time, people's individual leverage has been exponentially multiplied. Expose yourself to as much of AI and research as possible, but really focus on your unique ability as a human to be creative and discern these cool things with all the context that you have. There were more interesting problems to solve than the ones we were able to solve in our PhDs. We wanted to train models that allowed people to at least give them the option to control their own intelligence rather than relying on the big labs. 11|[0031.92 - 0059.28] You went from Australia to San Francisco. What was that like? Yeah, it was a bit of an arduous journey to get here. So I did undergrad in Australia, then was planning on doing my PhD at Oxford, which is where I met Moody through my best friend Max, who was also at Oxford. And then we decided that there were more interesting problems to solve than the ones we were able to solve in our PhDs. 12|[0059.28 - 0086.56] took a temporary leave of absence from Oxford. Moody jokes that I'm not a proper Oxford student because I was only there for realistically only a few days. But yeah, we decided to have a crack at a startup and we gave it a year to work. And originally that meant building out of London because that was where everyone's home base was, very close to Oxford. And I think at some point we realized that the serendipity of SF and just the surface area of your luck here. 13|[0086.56 - 0116.24] was a real thing and you know you hear it over and over from so many people but it's a hard thing to to grasp and come to terms with until you've actually been here and so I think yeah on a on a trip to SF um after the company had been around for for a few months I think more things happened in four days than had happened in the previous you know four or five months um and we sort of thought okay it's completely irrational to not be here um in the epicenter of where everyone's building and yeah so we moved the whole company here we um we told everyone that that 14|[0116.24 - 0143.25] We wanted to be in person in NSF and fortunately everyone we'd hired thought that was the right decision and everyone moved over within, I think, about two weeks and we moved into a hacker house with all eight, nine of us. And yeah, it went from there. What happened in those four or five days? You mean you met a lot of people? Yeah, I think a combination of meeting customers, meeting people that I had looked up to for a long time and like... 15|[0143.25 - 0172.22] just not only like seeking them out, but also just like literally running into them at certain events. And I thought, okay, wow, like there's just so many like interaction effects and network effects here that you'd be kind of stupid to ignore. And yeah, as soon as we moved here, I think things really started to kick off for Paz, which was our startup before we joined Base10. And like, it's really easy to like crystallize your vision and where you're headed as a company when you have so much more information being poured in. It's very hard to make that. 16|[0172.22 - 0201.06] or do that analysis in a vacuum and thinking through like, okay, where do I think this field is going to be headed and how can I position the company to best take advantage of that? To do that in London is very hard because you're not receiving this influx of information from all these different people with all these different opinions. I mean, you're not able to integrate over all that as well. Yeah. I want to go back to your time at Oxford, like briefly, and tell me more about like what factored into your decision to leave. 17|[0201.06 - 0230.80] Yeah, I think it's a PhD versus industry has always been a question. It's not like this has just come up the advent of AI. I think a lot of people who are both very intellectually curious, but also care about the impact that they can have on the world grapple with this when they're deciding whether to go to grad school or not. And it's an age old joke about, you know, doing PhD and being a poor grad student versus, you know, going off and joining big tech and making the money. But for us, it was really about, 18|[0230.80 - 0257.92] for the first time in history, I think the impact that an individual could have, or at least a very small group of people, had been multiplied and leveraged significantly more than ever has been, certainly in my lifetime and for quite a long time before that. And so I think the calculus shifts a lot. You have to decide whether you're going to narrowly focus on one very small edge of the periphery of the scientific bubble that you're going to try and advance, or... 19|[0257.92 - 0283.49] whether you take a step back and go, okay, I'm going to take a little bit more of a risk here, but I'm going to tackle a problem which, if I solve, could potentially be much bigger and much more impactful for the world. So I think all three of us, Moody, Max, and I felt that. We knew there's advantages to having very academic roots and research-based roots. And in some ways, that was a big advantage for us compared to a lot of the other startups. 20|[0283.49 - 0313.47] Yeah, I think the temptation to solve a bigger problem than we could at Oxford alone was just too much to ignore. What do you think was the advantage of the academic route that you talked about? When you're in a startup, you often operate in this mode of like greedy action selection, right? So every day you're trying not to die and it's very easy to get very myopic and say, okay, what is the thing I'm going to do next, which is going to maximize revenue over the next week or the next month or whatever it is. Whereas like in research, I think you have to be, 21|[0313.47 - 0342.69] you have to develop a lot more of this longer term strategic planning ability and you have to be able to say okay like if is the thing I'm doing now going to be something that if I weren't going to do it someone else would solve and to me that's not a very interesting problem to most researchers that are that's not a very interesting problem the best research is the ones who can say okay if I weren't going to tackle this like probably no one else would tackle it in the way that I might go about it and that would be a bad thing for the world because I think that this is a solution that should exist or an area I should push on 22|[0342.69 - 0371.58] And I think with PaaS, to give a concrete example, we knew that post-training as a service, which is what PaaS did, would eventually become commoditized to an extent. But we also knew that the ways in which people would try and commoditize that would be different to how we would go about it. So everyone was very attracted to verifiable tasks and these very deterministic tasks so that they could focus on the purity of the engineering and algorithmic problems. And people who like RL tend to like that kind of pure maths. 23|[0371.58 - 0401.55] problem space. But we thought like, okay, no one's really going to go after these non-verifiable areas like legal and medicine and insurance and things like that. And we knew that solutions wouldn't exist as quickly as if we went in and tried to tackle that. And so to us, like having that kind of long-term thinking and ability to select a problem, which you have a unique position to tackle, was like, yeah, really enticing to us. I want to talk a little bit more about the PhD question, because I feel like a lot of people, 24|[0401.55 - 0430.94] think they can't really become like an AI researcher, for example, without a PhD. And I'd love to hear more of your thoughts on that for someone who believes this or is thinking about this. For the first time in a long time, like people's individual leverage has been exponentially multiplied. And I think this is very much the case for research. This is a useful analogy to me when thinking about like the type of stuff that I want to be working on. There's like basketball players who like every single play that you do, you have to hustle and you have to be doing like a lot of individual, very little things, right? 25|[0430.94 - 0457.20] And I think like traditionally, like a career has looked a lot like a basketball player. Like you work to develop a set of skills and push to become like, you know, top 10% in a skill that a lot of people have so you can make a good career out of it. And I think the world is now shifting with LLMs because we now have this setup where if a lot of people can do this skill like reasonably well, that's the basis for the training distribution or the RL environment that it's going to allow language models to do that. And so I think now. 26|[0457.20 - 0481.47] people are shifting towards like needing to be more baseball players where you might not do anything for a really long time but you have to be able to like kind of hit the home runs and so like from a research perspective that's like what I would be focusing on is like not can I like ingest all this like you know knowledge and history of the field a lot of which has become irrelevant with how quickly language models are moving and and how much that field has evolved in such a short period of time. 27|[0481.47 - 0510.46] It's more like, can I develop the metacognition skills to be able to say, okay, again, what is the research problem that I want to work on? And can I hit a home run every once in a while? That a language model is just not going to have the creativity to be able to come up with in the first place, or even if it does come up with it, it's not going to have the value function to be able to discern that that is a good idea amongst the many, many millions of ideas that they can generate. And so, yes, expose yourself to as much of AI and research as possible, but really focus on your unique ability as a human. 28|[0510.46 - 0538.48] to be creative and discern like these cool things with all the context that you have and not try and you know start from this bottoms up layer of like i'm going to learn you know matrix calculus and then i'm going to like step through that and then eventually i'll build a neural network and then eventually i'll train and then eventually one day i might get to lms or diffusion models or whatever it is i think the world is going to reward researchers and people in general who are much more top down it's like okay here's this cool idea i have i have no idea how all these intermediate bits work but i'm going to get the thing working 29|[0538.48 - 0567.44] then we're going to fill in the skill tree on the way down rather than going bottoms up. Yeah, that's really interesting because I feel like that's not really how we're taught to learn, right? Like in school, you just start from like the ground up and then you work your way over time. Yes, I like completely agree. I think like I have a lot of qualms with like how things like maths and stuff are taught in school in general. But yeah, like so many of AI researchers and people getting into AI research are very predisposed towards, you know, the Andrew Ng. And I love Andrew Ng, but like his style of course is very much, you know. 30|[0567.44 - 0597.01] you learn the very basic fundamental building blocks and you slowly, slowly step your way up. I think someone who does this really well is Jeremy Howard. And he's like actually one of the first people that I started like learning AI from through his like fast AI stuff. And his whole thing is like, okay, the first thing you're going to do is train, you know, back in 2018, it was like convolutional neural nets. But he's like, okay, I'm going to give you the stuff so you can train a convolutional neural net in like four lines of code. And then over the course of like you learning about it, you're eventually going to work your way down to be able to write that from scratch yourself in like raw PyTorch. 31|[0597.01 - 0626.00] And I think for me that allows you to have so much more velocity, but you need to be comfortable living in a world where you don't know everything and focusing much more on intuition and creativity as opposed to how you would normally build up a skill like maths, which is filling out the skill tree from the bottom. So if I'm learning a new hard topic, like RL or something, don't know much about it, but not starting from like, okay, what's the thing, like 12 levels beneath it, like ground up learning, but just starting at the top. 32|[0626.00 - 0651.98] Yeah, like if I wanted to learn RL and be the best at RL, the first thing I would do is grab an open source framework and I wouldn't touch a single line of code, I wouldn't read a single line of code except for the config. And I would try and plug that into some arbitrary environment. It might be an environment I've written, it might be like a standard maths environment. And I would watch that curve go up. And then once I got that curve to go up, I'd maybe change a couple of these hyperparameters and say, oh, okay, that's how the group size affects this curve in this way. 33|[0651.98 - 0681.90] And then I would slowly work my way down the code until I fundamentally understood, okay, this is the GLPO algorithm. Here's everything that's wrong with it. Here's what we fixed with the length normalization bias and dividing by standard deviation and everything. And I'd develop that. And I think if I did it that way versus starting from, okay, I'm going to learn policy gradients. I'm going to go back and read Rich Sutton's textbook and work my way through the history of RL so I fully understand everything before I run my first GLPO loop. I think the former case beats the latter case every time and you become a much better researcher. 34|[0681.90 - 0708.16] because of it and a much more practical researcher as well. Yeah. The latter case seems not as motivating either. It's like, I want to learn about this thing. Okay. I need to go learn eight other things that are at a more basic level. And then in six months I can actually learn about the thing. Yes. Yeah. Yeah. You need that dopamine hitter, you know, watching the curve go up. Yeah. I want to hear more about your advice for other people who want to become researchers. People often give wishy-washy and like abstract advice. I think 35|[0708.16 - 0737.06] maybe the most useful thing I could say is like grounding this in like how I am currently thinking about research. So I think like you of course have to have enough context and enough knowledge of the field in general to be able to have some sort of refined value function over what good ideas are and where the field is headed. And for me, like if I sit back and think about like, okay, what are the most interesting ideas to me personally, to the field and to like me in the context of, for instance, base 10, which cares about inference and open source models and an ecosystem of these specialized open source models. 36|[0737.06 - 0763.92] then the kind of thing that sits at the intersection of all that for me is like continual learning. And it's not continual learning in the way that the big labs care about it. It's continual learning for very specific tasks and specific jobs and domains, much like you might have a human who's doing a specific job and needs to continually get better at that job. Whereas the big labs care more about, okay, if there's a very general model, we would like to generally update this distribution with the last six months of internet data. How do we do that without forgetting anything? 37|[0763.92 - 0792.14] So first off, I'm like, okay, that's an interesting problem to me and to everyone in general. I think it's an interesting problem to like base 10 in the specific context I'm in, because we have a unique, like potentially more tractable way to tackle this question than someone like OpenAI or Anthropic does, because we just care about continual learning for like these specific jobs for this model that is going to be a lawyer or this model is going to be like, you know, a medical note writer or whatever it is. And then I say, okay, like I've got my kind of first principles axiom of where I think the field is headed. 38|[0792.14 - 0820.88] Like how do I work backwards from that to construct a research agenda that is one going to be tractable? Yes. But like two is actually going to push in the direction that I care about. And so first thing I think about with continual learning is, okay, the big limitation to continual learning at the moment is the context window of the model. Like the model is perfect up until, you know, 200K tokens or a million K tokens, a million tokens. But outside of that, it just falls off. And so how do people currently solve this? We have these like markdown memory systems. 39|[0820.88 - 0849.89] We do have people on the other end of the spectrum actually trying to extend the context window, but that gets very hard. Like you can't really get around the quadratic complexity of attention. And so to me, then I think, okay, as a researcher, like what is my unique approach going back to this? Like if I went to do this, what would no one else do? Like what would my unique approach here be? And to me, that's like, okay, can I extend the context window by sacrificing some of the losslessness of the KB cache or the context window for more context window? So can I learn to compress? </file-view>
We have transcript details. It is long. Need present summary and file locations. Could also maybe include top-level themes. The first 100 lines contain a lot. Could read all segments or full text? It has 146 lines total, but line 7 full transcript is very long (truncated 71750 chars). We can summarize from the segments. Or ask if user wants full. But likely they want transcript. We can say transcribed in 75.4s, duration 64 min, saved to files, and provide first ~200 words and mention files. Better to summarize rather than dump full. The instructions say concise. Let's provide summary and files. Need maybe use a subagent? Not needed. But to be useful, we could produce a summary of the whole conversation based on full transcript. However we only have first 100 lines (39 due truncation). There are 137 segments; we can read more segments to summarize. The file has 146 lines: header and full transcript line and timed segments. Actually total 146 lines because full transcript line is huge? Wait line 7 full transcript truncated. The timed segments start at line 9? It shows line numbers 1-39; total lines 146. So file has 146 lines, where line 7 is the full transcript (very long, but truncated in read). Timed segments list each segment. Need summary. We can read the remaining segments (from line 40-146). Let's read the rest to have all segment text.
{
"file_path": "/Users/root1/2077067032152182986.txt",
"offset": 40,
"limit": 120
}<file-view path="/Users/root1/2077067032152182986.txt" start_line="40" end_line="74" total_lines="146"> 40|[0849.89 - 0876.29] everything that's come in the context window so far so that I can extend it. And so then, okay, we've got our axiom, we've got our first step, which is like, I would like to be able to compress the context window. How do we do that? And so this kind of led to our research with like neural KBK compaction. And then it's kind of easy to start to chain these steps together beyond that as well. So you then say, okay, I have some sort of compaction mechanism. Now my context window is not a million tokens. It might be 50 million or a hundred million effective tokens in my context window. 41|[0876.29 - 0905.71] how can I train the models with this compaction in the loop so that they're much better at it and then that leads to another natural like step in the sequence which is you know RL with compaction in the loop you even give the model access to a tool so it can call and compact its own context window and then that's really cool to me and then finally like okay I've got all this context sitting around again going back to the main goal or axiom like if if the model is repeatedly using this compressed kvk I probably should just put in the model weights for this specialized model so how do I go from compressed kvk to 42|[0905.71 - 0934.45] I don't know, I learn a hyper network to project it into LoRa adapters or something and then put it back into the model weights. And like, that's a very concrete example. But like, that is how I kind of think about like, you know, first off, think about where you think the field is headed. Think about the intersection of what is an interesting idea to you and an interesting idea to the field that's going to be useful to both. And then can you construct a research agenda and a sequence of like, you know, a through line through that kind of axiom that would allow you to. 43|[0934.45 - 0961.92] make meaningful progress on it. Yeah. I have so many questions from there, but one thing I was thinking about, you mentioned as a researcher, you need your own unique point of view. How do you think someone starts to develop that? Like when they haven't had much practice yet? Yeah. It's hard because like you can't like retrospectively go back and change your life. Right. And I think some of the best researchers I have, uh, that I've seen, um, have just this unique experience that like somehow gives them this different worldview from everyone else. And I think 44|[0961.92 - 0990.80] this is in the age of LLMs which are very messy and not the pure type of you know IMO intelligence that you might expect people to might be a traditional indicator of success or correlated with success in in more quantitative fields I think that people with this unique experience often do so much better I think like one of the things that I've found has helped the most with me is like my background is mainly like you know philosophy and literature I started out in law and philosophy I did that for quite a while at university 45|[0990.80 - 1017.79] And I think that yes, quantitative intelligence is very important, but for me in particular, I guess my unique skill vector is I love language and I love the messy world of language and I think that I'm very comfortable operating in a world where language models aren't these deterministic algorithms that you can solve with logic circuits. They're messy real-world things that have this behavior you need to shape. And so I would just encourage people to think about what is... 46|[1017.79 - 1046.45] their unique skill vector that allows them to tackle problems in different ways and I think there's so many examples of this another Australian like Sholto I think like one of his unique advantages is like what he learned through his fencing career and I think he's talked about this a fair bit and he has this sort of great line about you know what one of the things that taught him was that most researchers focus on training only and by training I mean like in the analogy of sports like preparing for a tournament or a game or whatever it is 47|[1046.45 - 1076.11] And he said that what fencing really taught him was that like no matter how much preparation you do in training wise, it doesn't matter if you're also not devoting a significant period, a portion of time to performance. Like when you're actually in the minute of the game time, like how are you performing and how does like your psychology and preparation and everything prepare for that? And I think like that's a really unique skill vector, right? Like not many AI researchers have been world-class fencers and to be able to learn that allows you to, you know, go in and do research and like, yeah. 48|[1076.11 - 1105.84] obviously focus on the training which is like you're reading papers and you're absorbing information and all this sort of stuff but the performance side of things so like when you sit down to actually do research are you treating this like an actual game that you are now playing um and it's high stakes in that in that sense so yeah like people should think about what what and everyone has this i think but not many people leverage it as much as they should yeah i don't think we do leverage it yeah it's not also what we're taught i think in school traditional school and things like that yeah i'm also curious so you said that you were 49|[1105.84 - 1132.75] You're studying philosophy and then you switched to AI. Can you tell me how this happened? So actually I wanted to study English literature when I went to university. My mum was an English teacher for a bit and she's always been very big into reading and got me into reading at a young age. A career advisor convinced me to do law instead because they thought it was a safer career path and as an 18 year old I... 50|[1132.75 - 1160.64] just kind of went along with that but yeah wanted to do philosophy as well so I think I just love the humanities and and reading and writing um I didn't necessarily enjoy the law as much but like love the philosophy and philosophy got me into Guern which is how I found I think it was the GPT-2 paper and Guern had written a little bit about you know scaling laws even back in the GPT-2 days and then I went and found the original GPT paper and was like wow look at the difference between these two things and all we scaled up it's like 51|[1160.64 - 1190.35] the data and the model size essentially. And I think like for the first time, like that quantitative world of deep learning and maths and stuff, which hadn't really necessarily interested me before because it was so pure and well-constructed became really interesting because you're suddenly trying to approximate human language, which like, you know, Chomsky and all these others have said it could never be done. And there was early signs of life there. And so that was the most interesting thing to me. I think that Delta showed me that like, you know, when GPT-3 would eventually come out and GPT-4 and GPT-5, like if that Delta kept up, then, 52|[1190.35 - 1218.83] this would probably be something that a lot of people would be talking about in five or ten years. And then decided to throw myself into like language modeling, language modeling basically swapped to pure maths and computer science. And fortunately that bet paid off and the scaling was stuck. But yeah, it's I guess thanks to Gwen's philosophy and clicking on a few random hyperlinks around Gwen's blog that it got me into language modeling. So then you started studying math, computer science and went from there? Yeah. Okay. And then 53|[1218.83 - 1247.14] How did you come about co-founding Parse? Yeah, I was always very much not interested in being a pure academic. I was also not interested in just going straight into industry. I liked sitting on the line in between those two things. So during undergrad, my mate Max, who I eventually co-founded Parse with, we had a small startup doing news de-biasing with language models. So basically we'd set up this big scraper to scrape the internet every day. 54|[1247.14 - 1276.90] and aggregate all the news stories and then like try and use a language model. And these were really bad language models to like write a de-biased version of that story, like stripping out all the sort of fluff around the periphery so that people could read unbiased news. So it was like an AI journalist? AI journalist, exactly. Unbiased? Very difficult to sell to VCs. Yeah. We then tried to make it a two-sided marketplace where like journalists could contribute information that would go into the algorithm and consumers would read it. We knew nothing about VC or... 55|[1276.90 - 1303.34] what businesses would scale or anything like that. It was just a really cool idea to us. And we had a lot of fun building it, but like, I think I really got that itch. And even though it didn't really succeed in the end, like it eventually morphed into this, you know, large scale, like rag enterprise thing that we ended up selling to a few, few enterprises. Like I got the itch and I saw that like, you really could just tackle problems. And like, particularly with LLMs, like there were just so many problems. 56|[1303.34 - 1333.22] that you could tackle now that you couldn't have made a dent in before. And so I was like, I would love to tackle the biggest one that I can within my capability. And I don't think that's purely in academia, but I also think that like the research background enables you to tackle or like, like creates a bigger set of problems that you can tackle once you have that, that experience. So you said you got the itch to go find a big problem and solve it. Exactly. Okay. Okay. So then how does that lead to Pars? Yeah. So that kind of finished. 57|[1333.22 - 1361.26] I was like, okay, I probably need to like, you know, it seems a bit hypocritical based on everything I've said so far, but I probably need to like go and do a PhD and like figure out what these problems are and develop my skillset more and this sort of stuff. And so, yeah, I applied to Oxford, was very lucky to get Jakob Foster as a supervisor, who is like one of the best, you know, meta learning RL, multi-agent RL, and was just trying to find interesting problems to work on. And then I think within 58|[1361.26 - 1386.37] 45 minutes of having met Moody, Max introduced us at a pub in Oxford. We've all realized how much of the itch we all had. And it was a pretty easy decision after there. In fact, Moody and I and Max started the company after I'd spent 45 minutes in person with Moody. So you were at a pub and you all kind of hit it off as friends and started talking about what you're building. Yeah, well, not even what we were building, just like what we thought was missing from the world that no one had built yet. 59|[1386.37 - 1414.56] And again, I feel like we all had these unique perspectives. Like Moody was going to be a neurosurgeon. He was very interested in applying AI to medicine in some form and felt like it was very under leveraged there. Max was doing neuroscience as well, but more like computational neuroscience and like was very interested in how the brain worked and thus how LMs worked and the parallels. And then I had my own background and like, I feel like you really appreciate that unique perspective on other people, particularly when you converge on like caring about similar problems. 60|[1414.56 - 1443.82] And so they just felt like the right people to start the company with. I like how you guys all have like your real names and then like nicknames. Yeah, this is a very Australian thing. I'm still learning all the nicknames. Yeah, I feel like I say like Kirk's and people are like, who's that? I was like, oh, Max. Yeah, I was confused about that for a second. I was like, wait, or Chuck. I was like, oh, Charlie's Chuck. Okay, makes more sense. Yeah. So talk more about like finding the problem of parse. Like what does parse do? So originally we set out and we thought, okay, 61|[1443.82 - 1469.89] we're just three like idiots basically like it's probably isn't gonna work like we're gonna have a crack it's not gonna work like no one's gonna give us money like we'll have a crack and we were we went out being very honest that we didn't have a very concrete idea like I think we had a meeting with it was it was light speed I think and we were originally doing trying to narrow ourselves down to this very confined like AI for like hospital use case like thing and we it was like eight hours before the meeting and we were like 62|[1469.89 - 1497.65] this is the stupidest idea in the world. Like we don't care about this. Like we've tried to pigeonhole a solution that we think the VCs would care about. Like let's just go in and basically be honest and say like, okay, here are the things we're best at. Here's what we think is missing. We're going to try and do blue sky research to try and converge on like the exact problem we want to solve with the skill sets that we have to solve it. And then that worked really well. And we raised like a seed round and then pretty quickly converged on the idea of, you know, 63|[1497.65 - 1523.81] the world of big token where everyone was calling generalists closed source models was not necessarily either the world we wanted to live in but also like an efficient allocation of resources really like we really thought people should be specializing their own models and so we were very early to that idea and it was quite contrarian but we were like okay we're going to fully lock in on post-training which is what largely my background had been in 64|[1523.81 - 1552.58] and figure out how do we get people from, you know, Opus 4.6, it wasn't Opus 4.6 at the time, but an Opus 4.6 frontier model to a specialized RL open source model for their task. And how do you like make that better over time? And I think there's a lot of grinding and gnashing of teeth as you try to convince people this was possible, but kind of halfway through last year, I think people started to come around to the fact that open source was now, the base open source was now in a place where 65|[1552.58 - 1581.66] you really could post-train it and deliver results and quality that was above and beyond what the frontier models were giving for a specific task. I've kind of asked you this before, but I want to talk about like big token. What does that mean for someone who's listening and they haven't heard this yet? Yeah, I think maybe like Moody likes to say that we came up with it when I put it in a blog post once. I think people have said this for a while, but big token is essentially this idea that like, you know, one or two frontier labs are... 66|[1581.66 - 1609.73] going to be the owners and controllers of AI and like any useful AI. Like open source will never be able to keep up with the advancements and the frontier inside these closed source labs and big token will rule AI in the same way that like big tech has ruled tech and a handful of companies are going to basically steer a man's destiny with that. So like we didn't necessarily think that even just from a philosophical point of view 67|[1609.73 - 1637.79] that that's the world you want to live in. We were much more, originally it was Little Token, but I feel like Little Token is a bit demeaning. So we started calling it like democratized token or something like that. But we really thought that we didn't necessarily have to just buy into this narrative that Anthropic was definitely going to win everything. And I think Anthropic is a great company and they're going to be around for a long time. But yeah, like we wanted to train models that allowed people to. 68|[1637.79 - 1665.60] at least give them the option to control their own intelligence rather than relying on the big labs. So for someone who doesn't know much about open models yet, how do open models really enable you to control your own intelligence? Yeah, so the thing you can do at the moment with open models that you can't really do with closed source models is specialize them. And so, of course, most people, and we even recommend to most people that you start out with 69|[1665.60 - 1694.46] closed source models and you prompt them and you set up your scaffolding so that you know you can actually do your tasks in the first place and confirm a language model is at least in the ballpark of being able to do this thing but at the end of the day you're going to reach a ceiling of capability that you can get with with prompting you've got to remember that as good as these frontier models are they are trained on a whole distribution of the internet and then a whole bunch of oral environments 99.99% of which have nothing to do with the tasks we're trying to do 70|[1694.46 - 1724.14] Of course, you need a certain base level of intelligence and world model in order to be able to do the task. But you are going to eventually hit the ceiling with what you can do with prompting and scaffolding and agent harness around a closed source model. And then once you realize that, you realize that any of your competitors, so if you're trying to build a medical scribe or a legal redlining agent, you've got to realize that if your competitors are using the same model and wrapping the same model, then they're also going to converge on. 71|[1724.14 - 1754.06] that ceiling, that 80% of task performance or whatever it is that you can get with prompting a closed source model. And so then you start to say, okay, well, what is my only advantage here? And maybe it's not even your competitors as well. Maybe you're actually worried about Anthropic. I know that a lot of startups in the last sort of even few months have realized that if you're doing PowerPoint generation or Excel or whatever, like Anthropic is very explicitly including this in later rounds of Sonnet and Opus training. Like first it was PowerPoint. 72|[1754.06 - 1784.06] Sorry, first it was Excel, then it was PowerPoint. Now they're going after general computer use. And so if your startup is just a wrapper around Opus 4.6 with a few prompts, then you're very susceptible to Anthropic eating your lunch. And so they sit back and they say, okay, what is the one advantage that we have over Anthropic? Well, it's like the real world data and the real world signal that would allow me to get better at this particular task and make a model better at this particular task. And the only way you can really do that and have control over that at the moment is through training an open source model. 73|[1784.06 - 1812.19] And so what that looks like concretely is, okay, first off people will tentatively dip their toe in the water and they might come to someone like us and say, look, okay, we have this data, we have this like signal, here's roughly what we think good answers and good executions of this task look like, here's what we think bad executions of this task look like and where we think the frontier models are falling short. Like, can we just go off and RL something to at least try this? And we'll do that and then often they're very pleasantly surprised and they'll plug that in and... 74|[1812.19 - 1841.79] that's kind of the traditional world of post-training. Like you do this big training run and they'll deploy a model and it will serve like as this kind of static snapshot of the training that you did. And then I think we're moving to a much more exciting world of, okay, we've done this initial training, like what would it look like if we were to actually really couple this specialization with the inference that we have? And this is one of the reasons I love working at Base10 is because you're able to do this kind of coupling with inference. So instead of treating training as this one-off run that you're going to do, it becomes... </file-view>
The file has 146 lines but we read up to 74. Need rest.
{
"file_path": "/Users/root1/2077067032152182986.txt",
"offset": 75,
"limit": 80
}<file-view path="/Users/root1/2077067032152182986.txt" start_line="75" end_line="109" total_lines="146"> 75|[1841.79 - 1871.78] okay, I'm getting this continuous feedback signal from the users actually using my model. Can I plug that into some form of online RL so that instead of my model getting better every three months when I decide to do a training run, it's getting better every five hours. And this is exactly what Curse has done really well. Big RL training run to start off with for a composer, deploy that model in production, collect more signal from users who are happy or not happy with the answers and use that to update the model. And they do it every five hours. 76|[1871.78 - 1898.72] which is quite impressive at the scale that they're at. So I'm curious as to why, like, why can you specialize an open source model in a way that you can't with a closed source model? Yeah, so if you want to specialize a closed source model, and it's not like OpenAI and Anthropic have never offered these solutions before. Like, they have offered, you know, a fine-tuning API where you upload a JSON data set of input and output, like, prompt completion pairs. 77|[1898.72 - 1927.90] they will fine-tune it for you and then send you back a specialized version of you know gp40 or 4 mini back when like this was a big thing for them um but like learning is so much more than either completely pure supervised learning where you simply have those like you know input output pairs and you're learning to predict the next token on the output pairs or pure reinforcement learning where you have some sort of um judge or way to assign a reward to a rollout 78|[1927.90 - 1957.47] and you're updating the model based on that. The hard part about our jobs and training in general and learning in general is that the algorithm is kind of commoditized now. Anyone can really write a GRPO loop or an SFT loop. And of course, it's nice that OpenAI has given a way to be able to run that on some of their smaller models. But the hard part is really how do you marry the customer or the person's expertise and opinions about what's good for that task? 79|[1957.47 - 1984.16] with the reward signal in the first place? Like, how do you even get what we're going to learn from? And to be able to do that, like you enter this very messy world of things that sit in between on the spectrum of pure SFT through to pure RL. And so those are things more like, you know, on policy self-distillation where a student will generate its own rollouts and its own attempts at a trajectory. And then the teacher will be the same student, but just given like extra privilege information. 80|[1984.16 - 2012.99] Okay, eventually this might be put in an API, but it's definitely not there yet. And so it's not only like these algorithms that sit in between on the spectrum, it's also, yeah, where is the reward signal coming from in the first place? Like if you're a legal company or you're going to start trying to reinforcement fine-tune through OpenAI's API, yeah, of course, you don't have to write the GRPO loop anymore, but like where do you even start from what is going to be the judge or reward model for like, did this model do a good job at drafting this document or not? 81|[2012.99 - 2041.86] And so that's the really hard part and it's what takes up most of our time from a manually intensive perspective. I want to talk more about the details around how open source models work in terms of the fine tuning, post-training, specialization. Can you break down what some of these terms mean for someone who doesn't know? Yeah, so we typically start out with this relatively large language model that has gone through a lot of pre-training where 82|[2041.86 - 2070.77] you take the whole internet and you just get it to predict the next token and we teach it to do that. And then a lot of mid-training and post-training, which looks a lot more like, okay, now that we have a language model that can do some things, let's one, teach it to respond in like this chat style format. And then two, let's put it in a bunch of reinforcement learning environments where it's going to attempt to do things on its own. And instead of telling it exactly what the correct next thing to do is, we're just going to wait till it's finished and tell it whether it did a good job or a bad job. And that's essentially like the reinforcement learning that comes at the end. 83|[2070.77 - 2099.92] has given us these kind of reasoning models that everyone has become so excited about. And that's what you get given when you start out. And that process is exactly the same whether it's an open source model or a closed source model. These recipes are largely agnostic between those two. But of course, as we kind of talked about, what you can do is take, because we have the weights of the open source model, we can do whatever we want to them. We can keep learning with those weights. Whereas if you're calling Claude or... 84|[2099.92 - 2128.35] ChatGBT, the only lever of control you have is the prompt that you provided or the scaffolding that you put around the language model. And so when we talk about specialization, and there's a lot of synonyms for this, it's specialization, it's fine-tuning, it's post-training, and largely, to a large extent, these are all the same thing. And then you have many different options in front of you in terms of how to teach it. So the simplest option and what people used to do is just continuing to do next token prediction with that model. 85|[2128.35 - 2155.65] on the data you care about. So you get it to predict the next token based on everything that's come before. It might be in a legal domain. You're getting it to predict the next token in this drafting document. And then we tell it whether it was right or wrong and we slightly update the weights based on that. Again, you can also move to reinforcement learning, which is where if you have some way to tell whether the model did a good job or a bad job at a task, not just at a token level, but overall. So maths and coding are a great example of this because, you know, 86|[2155.65 - 2184.10] for maths we know what the correct answer is we're able to verify for coding we're able to run unit tests and see if they pass and so the reward becomes okay did this bit of code that you you wrote pass all the unit tests or did this like proof that you wrote compile in lean and it passes the formalization checks and so in that way we can then say okay based on all the steps that you took language model like if you did a good job we're going to upweight the probability of those steps that you took there and if you did a bad job we're going to downweight the probability and over time that gives you 87|[2184.10 - 2213.34] model which is able to learn in a very different way from supervised learning which is very much you know I'm just going to copy the next token and learn to predict the next token of whatever distribution that's coming in. With reinforcement learning it's much more exciting because people believe it it induces these more general reasoning capabilities in the model. Basically you can think of this as like kind of how the old AlphaGo and AlphaZero models work so chess is a great example where reinforcement learning 88|[2213.34 - 2243.22] It works from scratch because at the end of a chess game, you always have a reward. You either won or you didn't. With language models, we had to get them to the point where, like obviously if you took a randomly initialized transformer, it's not going to ever really give you a correct math proof. Like you'd have to sample from it for a very, very long time before you got anything near a correct solution. So reinforcement learning is an inefficient thing to do when you don't have a very good base model. But once you have this base model that has a general world model and a good understanding of the world and is able to sometimes get the right answer, 89|[2243.22 - 2270.80] then reinforcement learning can take those right answers and upweight their probability and then downweight the probability of when it doesn't get the right answer and that's another way in which you can specialize and then there exists a lot of solutions in between this so one thing that is wrong with supervised fine-tuning is that the model is very off policy and by off policy i mean it's never making attempts by itself and then getting some sort of feedback at the end it's always copying you know a teacher distribution or like the 90|[2270.80 - 2296.14] the distribution that you give it to train on with next token prediction. But it is a very dense signal because every single token you're getting a new signal about whether you did the right thing or the wrong thing. With reinforcement learning, it's the opposite. It's very on policy, which is good because you're able to learn from your own mistakes. You're in your own distribution. I think humans are very similar. The way in which you learn best is often by doing something and getting feedback at the end rather than like rote memorizing things. 91|[2296.14 - 2324.42] But it is very sparse feedback. So you only get like the scalar feedback right at the end about whether you did the right thing or the wrong thing. You don't know where in your whole rollout or reasoning chain was good and where was bad. All you know is that like by the time you got to the end, you did a good job or a bad job. And so there's a lot of methods that are trying to kind of bridge these gaps and say, how do we give dense feedback, but with on policy generations? And so something like on policy self-distillation is an example of this. 92|[2324.42 - 2351.39] we take a model and it will generate its own rollouts and instead of saying okay this rollout is just good or bad we might have a way to you know provide more than just that scalar feedback so in the context of a maths proof it might be a larger model with access to the correct proof saying okay here is all the bits that you did right here's all the bits you did wrong and then we say okay what is the probability of everything the student did under this big teacher 93|[2351.39 - 2377.74] And what was the probability under the student? And can we move the student closer to the teacher? And so there's a lot of exciting areas that people are pushing in this to try and bridge the opposite ends of the spectrum of SFT and RL. But I think the point at the end of the day is that once you have a base model, there's many different avenues you can take to specialize it and to teach it your specific distribution that you care about. Yeah. Yeah, it's really cool. It seems like a lot of people don't know about that yet, but I think that is in part because... 94|[2377.74 - 2406.51] open models only got good enough like around a year and a half ago. Is that right? Yeah, I think you do need some base level of intelligence in order to be able to like tackle meaningful tasks. I think structured document extraction and very small tasks of open source models have been able to do for a long time, particularly if you fine tune them. But only recently are we seeing people start to think, okay, maybe this more broader task that I didn't really consider open source models for before, I might actually be able to teach us to do. 95|[2406.51 - 2433.86] And I think this manifested at first in people with all these big sort of agent harnesses often have a main agent, which is kind of like an orchestrator. And that orchestrator is calling a bunch of subagents. So it might call a subagent specifically for search. And I think last year when we were at Parler's, like people started to think, okay, what if I could get my subagent that's doing search to be an open source model and I can RL it to be really good at search. And search is a great example of this sort of quasi-verifiable domain where RL will work really well. 96|[2433.86 - 2461.76] But now, because of the pressures that I talked about before in terms of people looking to not be anthropic rappers anymore, they are starting to think, okay, what if I could drop in a trained Kimmy or a trained GLM and it could do my main agent task? It's not like these models are tiny. Kimmy is over a trillion parameters. We don't know quite how big GPT-5.4 and Opus are, but probably in the realm of five trillion parameters. So it's not crazy. 97|[2461.76 - 2488.69] to think that a specialized Kimi could compete with those and even outdo them. And Composer is a great example of this. In many ways, the model UX of Composer and depending on which benchmark scores you look at are around the level or perhaps even better than Opus 4.6. And so it just goes to show that there do exist very large open source models now with the baseline level of intelligence that would enable you to tackle very complex and very open-ended tasks if you train them in the right way. 98|[2488.69 - 2518.37] So you talked about how they can be competitive with closed models. Are there other benefits? Over the last year, let's say that the number of flops for inference in the world has gone up about three times. And let's say a conservative estimate of the demand for inference tokens has gone up 10 times. By many estimates, it would be larger than that. But we're starting to see that point where those curves cross over. And now you can no longer afford to just throw your task at the biggest possible model. 99|[2518.37 - 2547.36] biggest possible base model and just kind of take advantage of that consumer surplus and the intelligence you get from that. Now you say, okay, I have less GPUs than I need to run, you know, this biggest possible model. Like what is my other alternative? Well, I teach a smaller model to do this like faster and cheaper. And so like, I guess those are the other benefits. Not only if you specialize, you can get to a quality bar that's higher than the frontier models, but also you can usually get a much smaller model and hence much faster and cheaper model to do the same thing. And so. 100|[2547.36 - 2576.06] Now that we're spending as much money as we are on API credits for OpenAI and Anthropic, companies are really starting to consider, okay, maybe we should be doing this in a cheaper way. Yeah, it's so sensitive. That makes sense. Yeah, and there's examples as well where latency is really important. I think in medical context, for instance, ER notes and pulling information and summarizing it for doctors there, latency is super important. 101|[2576.06 - 2603.76] like access to consider for those sort of applications. And like if you're relying on the intelligence of Opus 4 and 6, which is fine, but obviously like it's much slower because it's a bigger model and also like Anthropix reliability has not always been the best. Like if you want more reliability in a much faster model, then you would start to consider something that's smaller and specialized and hosted on very reliable inference like Base 10. You talked about latency, like with closed. 102|[2603.76 - 2633.60] models you can't really optimize for latency in a certain way as opposed to open models? Yeah, so with closed models, you have to trust whatever inference engineering that OpenAI or Anthropic has done behind the scenes to serve their family of three models to millions and millions and millions of people worldwide. You also have to accept the uncertainty that comes with that in the sense that you are going to be sending a request and many people will be sending a request to those same GPUs at the same time. 103|[2633.60 - 2662.22] And your latency or your end-to-end time to last token is going to very much depend on who else is using it and how much they're using it. You have no control over that. Maybe you can get these kind of enterprise partnerships or whatever, but for most people that's not an option. With someone like Base10, for instance, you have complete control over how many GPUs you want this thing hosted on. You have complete control over the auto-scaling and the concurrency and the utilization and everything like that, and you get access to... 104|[2662.22 - 2691.95] you know, these world-class like inference engines that have been highly optimized to run your model as efficiently as possible, as correctly as possible, as fast as possible. And so like the levers of control are much more there, whereas like, you know, Anthropix are black box. We don't know how they're going about optimizing things. We don't know how they're making decisions about how much money to make and then thus how fast or slow to serve the models, how many people to serve those two in parallel. So it's another, I think, axis of control that you get taken away from you if you're going with closed source. Okay. 105|[2691.95 - 2717.97] So if I'm building something, you said earlier that first I should actually just start with a closed model. And then you talked about some of the criteria for like when you might switch to an open model. But I'm curious, like what are the downsides of open models or maybe what are the reasons why someone actually wouldn't want to use an open source model? Yeah, I think it all comes down to the beginning. Like I think the it kind of goes back to that top down approach, right? Like you should be seeing if. 106|[2717.97 - 2745.50] at least the best model in the world that out of the box which is the closed source models are capable of doing our task and can you shape that thing in a way which is like you know 80 good on that task and i think if you're trying to do that with open source straight off the bat like you kind of run into problems like particularly if you don't quite understand the contours and the minutiae of like okay this model's this particular open source model is good at this thing it might fall short in this regard um and also just like hosting these things in the first place like 107|[2745.50 - 2774.74] based on where you're calling these open source models from, you can get very different results. Like people really quantize these things heavily. There's often like, if you're not very careful and you're not very good at inference, like there's often issues with like the kernels and everything that sits under the hood. And like that just creates complexity you don't need when you're starting out. Like the great thing about Anthropic and OpenAI is they have, you know, two or three models and they're very good at serving them. And then they know they'll be served correctly. And all you have to do is worry about prompting them. So just getting off the ground, like I am a big proponent of, you know. 108|[2774.74 - 2803.10] you should just see if the best possible model out there can do your thing. Then you work your way down, you say, okay, what about the level below that? I know Opus can do this really well. What about Sonnet? Can Sonnet do it just as well? And then maybe even Haiku. And then you say, okay, what does it look like if I specialize an open source model now to do this? Or at least try an open source model. And by then you've understood the contours of your task to an extent that you probably could plug in an open source model and understand at least what it's doing differently from Opus. 109|[2803.10 - 2831.76] what is doing worse, maybe what is doing better. And that gives you a good ground to start shaping these rewards for things like reinforcement learning. Okay, very cool. So will open models ever catch up to closed models? I think I read that they used to be 18 months behind. Now it's more like three months. But I'm curious your thoughts on this question of will open source models ever catch up to closed source? I think I have different answers to this question over different time horizons. So the first shorter... </file-view>
Need rest to 146.
{
"file_path": "/Users/root1/2077067032152182986.txt",
"offset": 110,
"limit": 40
}<file-view path="/Users/root1/2077067032152182986.txt" start_line="110" end_line="146" total_lines="146"> 110|[2831.76 - 2858.94] answer like over a shorter time horizon the answer to me is like probably not to be honest like the base closed source models will always kind of be a little bit ahead of the based open source models i think that's for a bunch of reasons i think it's terms of how vcs allocate compute i think it's in terms of you know gpu blocks to china and a few other reasons but i don't necessarily think that matters as much as people say it does like to me the more interesting question the question is not 111|[2858.94 - 2888.93] okay, how close are the base models to each other? But because you have the advantage of specialization of open source over closed source, the question is like, what is the intelligence ceiling of this model if it were to be completely optimized for this task? And again, with closed source models, the lever that you can pull for optimizing is prompting. And for open source models, it's prompting plus supervised fine tuning plus reinforcement learning plus all these things. And so like the question you should be asking is, okay, what is the intelligence ceiling? 112|[2888.93 - 2918.80] And for me, like the intelligence ceiling of specialized open source, if you allow yourself to specialize a particular copy of a model for each individual task, probably now almost subsumes that of the closed source models. I think there's a few areas in which just raw intelligence and size is always going to be important. Things like scientific discovery, like, you know, math proof, things along those lines are going to always belong in the domain of the biggest model. But for most economically valuable tasks that I'm seeing now, like I would struggle to find something in which, 113|[2918.80 - 2944.58] a really well-trained Kimi would not be able to at least compete with, if not outperform, some of the or all of the closed source models. And so it doesn't necessarily matter if the base model is slightly worse. I think we're now at a point where we've crossed this threshold such that most tasks that people are caring about and building businesses around right now are able to be done by open source models. 114|[2944.58 - 2974.30] I think open source has actually done a really good job of hanging in long enough to reach that threshold such that now that specialization is going to become a much larger thing, they're going to remain to be very, very useful for all the reasons we've talked about before. My longer term answer to this is, you know, like I think it's a very discreet thing. It's not like this continuous straight line. Like I think with the new GPUs that NVIDIA is bringing online and the fact that the closed source labs are going to have priority over those and that's going to allow them to train, you know, MOEs, which are, 115|[2974.30 - 3003.94] 5 to 10 to 15 trillion parameters larger than the ones we've got now. I think that there are going to be these discontinuous jumps as open source struggles to get the compute to keep up with those scalar training runs. But I also am a believer that, yeah, scaling laws hold, but there's a bunch of inputs to the scaling laws. There's not only model size, but there's also data. And I think that at some point you kind of run out of data, and people have said this for a while, but when you get to 20 trillion parameters, I think, 116|[3003.94 - 3033.62] the bottleneck is no longer the compute or the energy to run the compute. It's what do we actually put into this thing that is actually going to learn more than just like repeating over the same data that we've curated over the past, you know, eight years in Anthropic. And at that point, I think like if you've run out of, you know, environments to buy from a core or random esoteric books to find and then put into the pre-training data, then the big advantage is like very specialized data. And then I think that actually swings there. 117|[3033.62 - 3062.62] the kind of pendulum back in the favor of open source because again you're going to have these companies who are like okay the only advantage we have over the closed source labs is our specific data and our real world use the signal on this very specific task sure it's not for everything but at least i can make use of this and like maybe i don't need a 20 trillion parameter model to do this um and we're now at the point where i think you know the 1 trillion parameter models can do a lot of the useful things that we care about and eventually like 118|[3062.62 - 3091.34] the compute will get commoditized again and of course like nvidia will bring out a new line of gpus and they'll make ruben seem old but like open source will have access to ruben and we might have you know five trillion parameter open source models which will raise the intelligence ceiling yet again and i think as long as open source hangs in there to the point where we can basically do everything that we care about in the world today with a specialized open source model i think then that's a healthy dynamic i think the closed source will still have a lot of areas where 119|[3091.34 - 3120.77] the most intelligent and largest model will win. And I think they'll accrue a lot of benefit from that as they deserve to because they've put so much into the CapEx to build out and the R&D to build out those models. But I think that open source will be the backbone of a society where a lot of people are serving specialized models to do very particular things. There's a lot of talk about LLMs replacing jobs, but it seems like because of the rise of open models that we're actually creating jobs in the inference space. Is this accurate? 120|[3120.77 - 3150.74] I think, yeah, like it's certainly accurate in the infant space. And I'd probably push back on people who say that, you know, LMs are taking jobs in general. I think there's been a bunch of scare pieces published in like the last year or so, like the Citrini piece and AI 2027. And a lot of these are talking about the fact that like as AI takes up more and more of like current human labor, then the value of human labor goes to zero and you get these like inflationary effects and like, sorry, deflationary effects and everything kind of goes off and it's not very good for the economy. 121|[3150.74 - 3180.56] But I think this assumes a world in which like, you know, demand is fixed and there's like a fixed slice of the pie. And historically, if you look at what's happened with like any advent of any new technology, this has like never been the case. Like when the internet came along, we did all like computers came along. We didn't just like automate like manual clocking and like all the things that we were doing back in the 80s and 90s. Like it also fostered a whole new ecosystem of jobs, which like opened up what people could actually do. And so I think this argument of, 122|[3180.56 - 3205.86] okay, LLMs are going to take human labour and that's going to be a fundamentally bad thing, is not necessarily the correct framing. I think it's going to look more like, yes, there will be some short-term pain as jobs that we know it today that are readily available to many even skilled workers are replaced by LLMs. But I think that the capacity that that frees up and the jobs that that creates in the long run are going to be... 123|[3205.86 - 3235.68] a repetition of everything that we've seen in the past and like humans have historically been very good at figuring out what to do when you know very manual um and labor-based tasks have been replaced and i think that will be a similar for like cognitive based tasks i think inference engineering is a great example of that um like one of the perhaps fastest growing areas in the world with one of the highest ramp ups in complexity um i think that you know when this first started this lm boom first started we had very vanilla gpd 124|[3235.68 - 3261.58] style models, inference engineering was hard enough. But as researchers got more and more wacky trying to figure out ways to make these models smarter, Mixture of Experts is a great example of this, where we have a much bigger model with many more parameters, but we only activate some of those parameters every time we call it, based on which experts we decide to route to for this particular question or this particular token. That's a really hard inference problem, and that's certainly much harder than... 125|[3261.58 - 3289.49] the vanilla dense models that we had back in 2019, 2020, 2021. And so as these models get smarter, it's a really concrete example of an area where there's just going to be more and more jobs as we figure out how to manage these things. And they're very cognitively demanding jobs, sure. But I think that same pattern will hold through a lot of the rest of the economy as well. Yeah. So you talked earlier about your learning approach and kind of top-down learning. 126|[3289.49 - 3315.82] For someone like me, kind of as a regular software engineer, and I want to get into inference engineering, what could be a good first step or first project for me? Yeah, I think actually the way that inference engineering has gone is a really good example of that top-down approach that I've been talking about. So in the old days, to be a really good inference engineer, I think you did have to start from the bottom up. You had to learn how to write kernels. You had to learn how to plug that together into... 127|[3315.82 - 3343.79] some sort of inference engine with queues and continuous batching and all this sort of stuff. I think nowadays it looks much more like there are solutions that exist out of the box where you download VLM and you download a model and you load it up and you can just call it like it can generate it at like reasonably fast speeds and so now the problems are shifting towards okay like now that I can get this thing running again which is the top-down approach like can I just get it working in the first place then you start to think about the optimizations that can be made around that and so 128|[3343.79 - 3368.82] I then start to think about, you know, okay, I don't have to learn how to write kernels immediately, but okay, like I'm now in this agentic system where the pre-fill is dominating much of my time and the decode doesn't matter as much. Like, can I learn about things like KB cache management and shuttling that around and like networking stability and all those sorts of things. And then maybe eventually I get to kernels. But like at the end of the day, like VLM. 129|[3368.82 - 3395.86] TRT, SG Lang, these are all inference engines which are able to run models out of the box but still require a lot of work and understanding to make them as good as possible and reliable as possible. And that goes for a very high level in terms of like, you know, playing around with like speculators and eagle heads and things like that through to like, oh, the other day someone found this massive bug in TRT which meant that mixture of experts was getting routed incorrectly. 130|[3395.86 - 3421.87] And that actually got found by someone playing around with it internally at Base 10. So I think the fact that you are able to get inference working out of the box is pretty lucky. When I first started on these things, that certainly wasn't the case. But it enables you to go top-down in a way that you couldn't before. So you co-founded Parse, and you were acquired by Base 10, which is how I know you, because I work at Base 10. 131|[3421.87 - 3451.14] So can you talk about why you chose Base10 and how this happened? Yeah. It was funny. The first meeting we had with Tu and CEO of Base10 was we were trying to sort out our inference provider for PaaS. Obviously, we would train models for customers and then we needed somewhere to put them. And so we talked to all the big inference providers and we weren't really that happy. And we walked into the Base10 meeting. I was visiting a customer in Australia at the time, so it was very early for me. I think it was basically still in my pajamas. 132|[3451.14 - 3476.24] Moody told me like, you know, we're going to play a bit of hardball with these guys. So like you go in and play a bit of hardball. With your pajamas on? With my pajamas on. And I remember asking Toon very distinctly, like if he could handle our volume, which was like such a ludicrous thing to ask for like a C stage startup with like barely any customers. And Toon just laughed at me. But like, I think the relationship grew from there. And like, as we realized how good base 10 was and as we grew alongside base 10. 133|[3476.24 - 3505.44] like we made them our sole inference provider because they offered many things that like none of the other inference providers could and then i think eventually like base 10 started to notice that like customers were just appearing through us on the base 10 platform and they were all using specialized models and like i think maybe some of the base 10 had tried to like kind of work with before um and base 10 is the best in the world at inference and we thought we were really good at post training and getting people off closed source models onto open source models and so eventually at some point turn was sort of like 134|[3505.44 - 3535.33] why didn't you guys come do this with us? I think the customer doors that are going to be open are going to be 10 times larger. The demand is clearly there. And then if you couple yourselves with inference, it's a no-brainer. And for us, this solved a really difficult problem of feeling like we were this awkward middleman between the customer and the GPUs. Once you trained a model for a customer, theoretically, they could just go off and deploy it on their own GPUs. They could go to base 10 directly. That's something that's not really a proper business model there. 135|[3535.33 - 3565.14] And for Moody and Max and I, I think all we cared about at the end of the day was this problem and solving this problem. I don't think we necessarily felt like we needed to be our own independent startup or company in order to be able to solve it. And we're very lucky that we've been able to retain the autonomy and make the research bets that we want to make and have the compute that we need at Base10. And Tuan and Amir and Phil and everyone have been really supportive of that. And so, yeah, it's kind of the best of both worlds. And I think it's been a win-win for... 136|[3565.14 - 3594.91] showing how training can be coupled with inference to give something much more powerful than either of those two things by themselves. I think many people in the AI space, like Dario, for example, have argued against open source models for safety reasons. And I'm curious your thoughts on that. Yeah, I think people like to talk about very specific examples here, like building bioweapons and X-risk and all these sorts of things. I think stripping away the superintelligence argument for a second, 137|[3594.91 - 3622.13] The thing I worry most about with AI is not how a small handful of bad actors will use this in various specific scenarios. I think it's the fact that a concentration of power without accountability has always been the most severe failure mode of any civilization as far back as we can remember. I think if you do live in a world where one company or maybe two companies and the handful of people inside those companies responsible for making the decisions have all the control, 138|[3622.13 - 3651.10] over AI, which is like the most powerful technology we've seen. I think that even if you trust the initial actors at the start, if you trust Dario and his vision, it's a very different situation to, you know, some of the technologies we have in the past where there's sunset clauses kind of built in, like you know the distribution and diffusion of that technology will eventually happen. And I say that for one main reason, which is that the people who control the AI and like once that gets past a certain threshold, 139|[3651.10 - 3680.16] are going to have the capability to use that AI to extend their own advantage and their lead. So there is no sunset clause built in here. And so I think that's like everything that, you know, Western democracy fights against is concentration of power, particularly power that enables itself to continue its own reign is like a fundamentally bad thing. Like there's so many checks and balances that we have built into like our political system that we trust to ensure that that doesn't happen. And I think that like if you are buying into this 140|[3680.16 - 3709.26] argument of you know ai is going to fundamentally change the world then even if you trust anthropic now like i don't think the variance and the risk of incentives and and um behaviors changing over time is is worth having one person controlling all the air and so i actually think the more conservative and status quo you bet to make would be to diffuse this technology and sure there are going to be short-term pains in terms of like allowing ai to be used by 141|[3709.26 - 3737.39] basically anyone in the world. I think like cybersecurity is a great example of that. And again, people are talking about this with mythos. But I think that short-term gain is like kind of an inoculation to the world against like all the potential harms it's going to have. And it's also going to allow the world to more readily robustify like the existing infrastructure that we have to handle like intelligent AI. It's also going to allow us to build in these defenses with the same intelligent AI to counteract the effects of bad actors. 142|[3737.39 - 3766.51] And so, yeah, like at the end of the day, concentration of power, particularly power that can extend itself, is like not something we've ever accepted before. And the fact that AI will enable whoever controls it to extend their own lead, if it's kept it completely internal, is not necessarily a great thing to me. And so that's kind of why I think that like we should just trust the systems that we have for deciding these things and stick with a more democratic future of AI rather than one where, you know. 143|[3766.51 - 3793.38] anthropic or open AI controls at all. We talked about this question of like who controls AI and some people would say that having the weights like having open weights is not actually who controls AI but it's actually who has the compute power. I'm curious your thoughts on that. It definitely is. I think that we're lucky where even the biggest models in the world basically run on one node so like essentially eight GPUs. 144|[3793.38 - 3823.14] The great thing about how language models are currently trained is that you need a lot of compute to do training, but to actually serve the model, at least for a handful of users, you only need a small number of GPUs relative to the amount used for training. And so, yes, it might be the case that you need a lot of GPUs in order to get to these large models, but if those are being distributed, then you need much less GPUs in order to be able to run it. So I think that kind of balances out or amortizes the compute concern somewhat. 145|[3823.14 - 3849.95] Of course, there's like second and third order effects of this. Like if you are able to get access to so much more compute than your competitors that you're able to train, you know, the 30 trillion monstrosity that is starting to approximate what a super intelligence would look like, then that's like a fundamentally different concern. But I think in terms of the setup that we have now where we're pushing between one and five trillion parameter models, like you are able to still democratize access to that once it has been trained. 146|[3849.95 - 3863.09] And so we just need people who have incentives to keep training and releasing these open source models to continue to do so, so that this can kind of promulgate. Cool. Thanks for being here, Charlie. Thanks for having me. Okay. </file-view>
We have all timed segments. Need respond. Since transcript is long, we should present a concise summary plus files and key excerpts. The user asked "now do it for this" => use whisper. We can say completed. Provide metadata, summary, and file paths. Maybe mention duration 64 min, transcribe 56s, wall 75.4s. Could include top-level summary of conversation topics: Charlie (?) from Australia/Oxford, co-founded Parse (PaaS), acquired by Base10, discussion on PhD vs industry, moving to SF, open vs closed source models, specialization/post-training, continual learning, inference engineering, AI democratization, safety/power concentration. Need be careful to not include too much. Let's produce summary with bullet points. Need maybe include a short transcript excerpt? The full transcript is accessible in file. We can include first paragraph? But summary is enough. Could also include a link? No. Use ref_file for saved txt/json. The output files are /Users/root1/2077067032152182986.txt and .json. Provide clickable. Need mention if they want the full text, it's in the files. Done.
Done. The X video was transcribed on Modal A10G in 75.4s (audio duration 64 min).
Saved outputs:
Summary — it's a long interview/podcast with Charlie (co-founder of Parse, now at Base10) covering:
The full transcript and timed segments are in the .txt file above.
do a full synopis
User wants a full synopsis of the 64-minute video. We have the full transcript in the txt file and timed segments. Need produce a comprehensive synopsis. We can read the full transcript line (line 7) which is huge. The read tool truncated it earlier. We can use `read` with offset line 7? The line may be very long but tool truncates to 2000 chars? It showed truncated 71750 chars. Need get full transcript. We can use `exec` with `sed -n '7p'` or `awk` to extract the full transcript line? But line is maybe over 200k chars. Let's use `python3 -c` to extract after the marker and maybe save to a temp file or output. But output may be huge. Better to read from file. We can use `sed -n '7p' file` and pipe to `wc -c`. Need full text to summarize. Could use a subagent to summarize? We can pass the file path to a subagent and ask to produce a full synopsis. However subagent has read tool with 20k char limit per read; line is huge. But subagent can read full file? Actually read limit is char-based 20000? The `read` tool reads up to 20000 characters starting from beginning. For long lines, it truncates. The full transcript is one line; tool truncates long lines. However we can use shell to split into sentences/wrap with `fold` or write to a new file with line wrapping, then read. Use `python3` to extract `=== FULL TRANSCRIPT ===` section and write to a temp file with word wrap, then read in chunks. Or use `exec` with `sed` and pipe to a subagent? We can use `sed -n '7p'` to output full line to a temporary wrapped file. Since line is one long string, `sed` will output it. Then we can process. We need a full synopsis. Could ask a subagent to read the transcript and write a detailed summary. But subagent read tool same limitations. We can break transcript into chunks manually, or use a script to summarize via an LLM? But we ourselves are the LLM. We can process chunks by reading through multiple reads. However one line is an obstacle. Let's reformat file: create a wrapped copy where each sentence or each N chars is a line. Then read in parts. Need full transcript text. Let's run a Python script to wrap the full transcript line and save to /tmp/wrapped_transcript.txt with width e.g. 80 chars. Then read in chunks maybe 500 lines. Then we can construct summary from all text. The transcript is around maybe 60k words? Let's check. We can count. Use shell.
{
"command": "python3 - <<'PY'\nfrom pathlib import Path\nimport textwrap\ntxt = Path('/Users/root1/2077067032152182986.txt').read_text()\n# Extract full transcript section between markers\nstart = txt.find('=== FULL TRANSCRIPT ===')\nend = txt.find('=== TIMED SEGMENTS ===')\nfull = txt[start:end].replace('=== FULL TRANSCRIPT ===','').strip()\nPath('/tmp/wrapped_full.txt').write_text(textwrap.fill(full, width=100))\nprint('chars', len(full))\nprint('words', len(full.split()))\nPY"
}Output from command in shell dc9ce8: chars 73750 words 13792 Exit code: 0
Transcript 73,750 chars, 13,792 words. We can read the wrapped file in chunks. The read tool supports up to 20000 chars per call and line limit maybe 20000? It can read 20000 characters from a file. We can read in three parts. But to produce a synopsis, we don't need every word? A full synopsis should cover main arguments. We can read entire file in chunks, but the response can be long. Need not include all details, but comprehensive. Let's read wrapped file in segments. Use read with limit of lines? The tool reads up to 20000 characters; not line limit. We can specify offset? Offset is line number. We wrapped at width 100, so line count ~738 lines. We can read 300 lines at a time. Need capture all. Let's do read first 300 lines, then next 300, then rest. But offset is line number. Use read.
{
"file_path": "/tmp/wrapped_full.txt",
"offset": 1,
"limit": 300
}<file-view path="/tmp/wrapped_full.txt" start_line="1" end_line="203" total_lines="750"> 1|For the first time in a long time, people's individual leverage has been exponentially multiplied. 2|Expose yourself to as much of AI and research as possible, but really focus on your unique ability 3|as a human to be creative and discern these cool things with all the context that you have. There 4|were more interesting problems to solve than the ones we were able to solve in our PhDs. We wanted 5|to train models that allowed people to at least give them the option to control their own 6|intelligence rather than relying on the big labs. You went from Australia to San Francisco. What was 7|that like? Yeah, it was a bit of an arduous journey to get here. So I did undergrad in Australia, 8|then was planning on doing my PhD at Oxford, which is where I met Moody through my best friend Max, 9|who was also at Oxford. And then we decided that there were more interesting problems to solve than 10|the ones we were able to solve in our PhDs. took a temporary leave of absence from Oxford. Moody 11|jokes that I'm not a proper Oxford student because I was only there for realistically only a few 12|days. But yeah, we decided to have a crack at a startup and we gave it a year to work. And 13|originally that meant building out of London because that was where everyone's home base was, very 14|close to Oxford. And I think at some point we realized that the serendipity of SF and just the 15|surface area of your luck here. was a real thing and you know you hear it over and over from so many 16|people but it's a hard thing to to grasp and come to terms with until you've actually been here and 17|so I think yeah on a on a trip to SF um after the company had been around for for a few months I 18|think more things happened in four days than had happened in the previous you know four or five 19|months um and we sort of thought okay it's completely irrational to not be here um in the epicenter 20|of where everyone's building and yeah so we moved the whole company here we um we told everyone that 21|that We wanted to be in person in NSF and fortunately everyone we'd hired thought that was the right 22|decision and everyone moved over within, I think, about two weeks and we moved into a hacker house 23|with all eight, nine of us. And yeah, it went from there. What happened in those four or five days? 24|You mean you met a lot of people? Yeah, I think a combination of meeting customers, meeting people 25|that I had looked up to for a long time and like... just not only like seeking them out, but also 26|just like literally running into them at certain events. And I thought, okay, wow, like there's just 27|so many like interaction effects and network effects here that you'd be kind of stupid to ignore. 28|And yeah, as soon as we moved here, I think things really started to kick off for Paz, which was our 29|startup before we joined Base10. And like, it's really easy to like crystallize your vision and 30|where you're headed as a company when you have so much more information being poured in. It's very 31|hard to make that. or do that analysis in a vacuum and thinking through like, okay, where do I think 32|this field is going to be headed and how can I position the company to best take advantage of that? 33|To do that in London is very hard because you're not receiving this influx of information from all 34|these different people with all these different opinions. I mean, you're not able to integrate over 35|all that as well. Yeah. I want to go back to your time at Oxford, like briefly, and tell me more 36|about like what factored into your decision to leave. Yeah, I think it's a PhD versus industry has 37|always been a question. It's not like this has just come up the advent of AI. I think a lot of 38|people who are both very intellectually curious, but also care about the impact that they can have 39|on the world grapple with this when they're deciding whether to go to grad school or not. And it's 40|an age old joke about, you know, doing PhD and being a poor grad student versus, you know, going off 41|and joining big tech and making the money. But for us, it was really about, for the first time in 42|history, I think the impact that an individual could have, or at least a very small group of people, 43|had been multiplied and leveraged significantly more than ever has been, certainly in my lifetime 44|and for quite a long time before that. And so I think the calculus shifts a lot. You have to decide 45|whether you're going to narrowly focus on one very small edge of the periphery of the scientific 46|bubble that you're going to try and advance, or... whether you take a step back and go, okay, I'm 47|going to take a little bit more of a risk here, but I'm going to tackle a problem which, if I solve, 48|could potentially be much bigger and much more impactful for the world. So I think all three of us, 49|Moody, Max, and I felt that. We knew there's advantages to having very academic roots and research- 50|based roots. And in some ways, that was a big advantage for us compared to a lot of the other 51|startups. Yeah, I think the temptation to solve a bigger problem than we could at Oxford alone was 52|just too much to ignore. What do you think was the advantage of the academic route that you talked 53|about? When you're in a startup, you often operate in this mode of like greedy action selection, 54|right? So every day you're trying not to die and it's very easy to get very myopic and say, okay, 55|what is the thing I'm going to do next, which is going to maximize revenue over the next week or the 56|next month or whatever it is. Whereas like in research, I think you have to be, you have to develop 57|a lot more of this longer term strategic planning ability and you have to be able to say okay like 58|if is the thing I'm doing now going to be something that if I weren't going to do it someone else 59|would solve and to me that's not a very interesting problem to most researchers that are that's not 60|a very interesting problem the best research is the ones who can say okay if I weren't going to 61|tackle this like probably no one else would tackle it in the way that I might go about it and that 62|would be a bad thing for the world because I think that this is a solution that should exist or an 63|area I should push on And I think with PaaS, to give a concrete example, we knew that post-training 64|as a service, which is what PaaS did, would eventually become commoditized to an extent. But we also 65|knew that the ways in which people would try and commoditize that would be different to how we would 66|go about it. So everyone was very attracted to verifiable tasks and these very deterministic tasks 67|so that they could focus on the purity of the engineering and algorithmic problems. And people who 68|like RL tend to like that kind of pure maths. problem space. But we thought like, okay, no one's 69|really going to go after these non-verifiable areas like legal and medicine and insurance and things 70|like that. And we knew that solutions wouldn't exist as quickly as if we went in and tried to tackle 71|that. And so to us, like having that kind of long-term thinking and ability to select a problem, 72|which you have a unique position to tackle, was like, yeah, really enticing to us. I want to talk a 73|little bit more about the PhD question, because I feel like a lot of people, think they can't really 74|become like an AI researcher, for example, without a PhD. And I'd love to hear more of your thoughts 75|on that for someone who believes this or is thinking about this. For the first time in a long time, 76|like people's individual leverage has been exponentially multiplied. And I think this is very much 77|the case for research. This is a useful analogy to me when thinking about like the type of stuff 78|that I want to be working on. There's like basketball players who like every single play that you 79|do, you have to hustle and you have to be doing like a lot of individual, very little things, right? 80|And I think like traditionally, like a career has looked a lot like a basketball player. Like you 81|work to develop a set of skills and push to become like, you know, top 10% in a skill that a lot of 82|people have so you can make a good career out of it. And I think the world is now shifting with LLMs 83|because we now have this setup where if a lot of people can do this skill like reasonably well, 84|that's the basis for the training distribution or the RL environment that it's going to allow 85|language models to do that. And so I think now. people are shifting towards like needing to be more 86|baseball players where you might not do anything for a really long time but you have to be able to 87|like kind of hit the home runs and so like from a research perspective that's like what I would be 88|focusing on is like not can I like ingest all this like you know knowledge and history of the field 89|a lot of which has become irrelevant with how quickly language models are moving and and how much 90|that field has evolved in such a short period of time. It's more like, can I develop the 91|metacognition skills to be able to say, okay, again, what is the research problem that I want to 92|work on? And can I hit a home run every once in a while? That a language model is just not going to 93|have the creativity to be able to come up with in the first place, or even if it does come up with 94|it, it's not going to have the value function to be able to discern that that is a good idea amongst 95|the many, many millions of ideas that they can generate. And so, yes, expose yourself to as much of 96|AI and research as possible, but really focus on your unique ability as a human. to be creative and 97|discern like these cool things with all the context that you have and not try and you know start 98|from this bottoms up layer of like i'm going to learn you know matrix calculus and then i'm going to 99|like step through that and then eventually i'll build a neural network and then eventually i'll 100|train and then eventually one day i might get to lms or diffusion models or whatever it is i think 101|the world is going to reward researchers and people in general who are much more top down it's like 102|okay here's this cool idea i have i have no idea how all these intermediate bits work but i'm going 103|to get the thing working then we're going to fill in the skill tree on the way down rather than 104|going bottoms up. Yeah, that's really interesting because I feel like that's not really how we're 105|taught to learn, right? Like in school, you just start from like the ground up and then you work 106|your way over time. Yes, I like completely agree. I think like I have a lot of qualms with like how 107|things like maths and stuff are taught in school in general. But yeah, like so many of AI 108|researchers and people getting into AI research are very predisposed towards, you know, the Andrew 109|Ng. And I love Andrew Ng, but like his style of course is very much, you know. you learn the very 110|basic fundamental building blocks and you slowly, slowly step your way up. I think someone who does 111|this really well is Jeremy Howard. And he's like actually one of the first people that I started 112|like learning AI from through his like fast AI stuff. And his whole thing is like, okay, the first 113|thing you're going to do is train, you know, back in 2018, it was like convolutional neural nets. 114|But he's like, okay, I'm going to give you the stuff so you can train a convolutional neural net in 115|like four lines of code. And then over the course of like you learning about it, you're eventually 116|going to work your way down to be able to write that from scratch yourself in like raw PyTorch. And 117|I think for me that allows you to have so much more velocity, but you need to be comfortable living 118|in a world where you don't know everything and focusing much more on intuition and creativity as 119|opposed to how you would normally build up a skill like maths, which is filling out the skill tree 120|from the bottom. So if I'm learning a new hard topic, like RL or something, don't know much about 121|it, but not starting from like, okay, what's the thing, like 12 levels beneath it, like ground up 122|learning, but just starting at the top. Yeah, like if I wanted to learn RL and be the best at RL, 123|the first thing I would do is grab an open source framework and I wouldn't touch a single line of 124|code, I wouldn't read a single line of code except for the config. And I would try and plug that 125|into some arbitrary environment. It might be an environment I've written, it might be like a 126|standard maths environment. And I would watch that curve go up. And then once I got that curve to go 127|up, I'd maybe change a couple of these hyperparameters and say, oh, okay, that's how the group size 128|affects this curve in this way. And then I would slowly work my way down the code until I 129|fundamentally understood, okay, this is the GLPO algorithm. Here's everything that's wrong with it. 130|Here's what we fixed with the length normalization bias and dividing by standard deviation and 131|everything. And I'd develop that. And I think if I did it that way versus starting from, okay, I'm 132|going to learn policy gradients. I'm going to go back and read Rich Sutton's textbook and work my 133|way through the history of RL so I fully understand everything before I run my first GLPO loop. I 134|think the former case beats the latter case every time and you become a much better researcher. 135|because of it and a much more practical researcher as well. Yeah. The latter case seems not as 136|motivating either. It's like, I want to learn about this thing. Okay. I need to go learn eight other 137|things that are at a more basic level. And then in six months I can actually learn about the thing. 138|Yes. Yeah. Yeah. You need that dopamine hitter, you know, watching the curve go up. Yeah. I want to 139|hear more about your advice for other people who want to become researchers. People often give 140|wishy-washy and like abstract advice. I think maybe the most useful thing I could say is like 141|grounding this in like how I am currently thinking about research. So I think like you of course 142|have to have enough context and enough knowledge of the field in general to be able to have some 143|sort of refined value function over what good ideas are and where the field is headed. And for me, 144|like if I sit back and think about like, okay, what are the most interesting ideas to me personally, 145|to the field and to like me in the context of, for instance, base 10, which cares about inference 146|and open source models and an ecosystem of these specialized open source models. then the kind of 147|thing that sits at the intersection of all that for me is like continual learning. And it's not 148|continual learning in the way that the big labs care about it. It's continual learning for very 149|specific tasks and specific jobs and domains, much like you might have a human who's doing a 150|specific job and needs to continually get better at that job. Whereas the big labs care more about, 151|okay, if there's a very general model, we would like to generally update this distribution with the 152|last six months of internet data. How do we do that without forgetting anything? So first off, I'm 153|like, okay, that's an interesting problem to me and to everyone in general. I think it's an 154|interesting problem to like base 10 in the specific context I'm in, because we have a unique, like 155|potentially more tractable way to tackle this question than someone like OpenAI or Anthropic does, 156|because we just care about continual learning for like these specific jobs for this model that is 157|going to be a lawyer or this model is going to be like, you know, a medical note writer or whatever 158|it is. And then I say, okay, like I've got my kind of first principles axiom of where I think the 159|field is headed. Like how do I work backwards from that to construct a research agenda that is one 160|going to be tractable? Yes. But like two is actually going to push in the direction that I care 161|about. And so first thing I think about with continual learning is, okay, the big limitation to 162|continual learning at the moment is the context window of the model. Like the model is perfect up 163|until, you know, 200K tokens or a million K tokens, a million tokens. But outside of that, it just 164|falls off. And so how do people currently solve this? We have these like markdown memory systems. We 165|do have people on the other end of the spectrum actually trying to extend the context window, but 166|that gets very hard. Like you can't really get around the quadratic complexity of attention. And so 167|to me, then I think, okay, as a researcher, like what is my unique approach going back to this? Like 168|if I went to do this, what would no one else do? Like what would my unique approach here be? And to 169|me, that's like, okay, can I extend the context window by sacrificing some of the losslessness of 170|the KB cache or the context window for more context window? So can I learn to compress? everything 171|that's come in the context window so far so that I can extend it. And so then, okay, we've got our 172|axiom, we've got our first step, which is like, I would like to be able to compress the context 173|window. How do we do that? And so this kind of led to our research with like neural KBK compaction. 174|And then it's kind of easy to start to chain these steps together beyond that as well. So you then 175|say, okay, I have some sort of compaction mechanism. Now my context window is not a million tokens. 176|It might be 50 million or a hundred million effective tokens in my context window. how can I train 177|the models with this compaction in the loop so that they're much better at it and then that leads to 178|another natural like step in the sequence which is you know RL with compaction in the loop you even 179|give the model access to a tool so it can call and compact its own context window and then that's 180|really cool to me and then finally like okay I've got all this context sitting around again going 181|back to the main goal or axiom like if if the model is repeatedly using this compressed kvk I 182|probably should just put in the model weights for this specialized model so how do I go from 183|compressed kvk to I don't know, I learn a hyper network to project it into LoRa adapters or 184|something and then put it back into the model weights. And like, that's a very concrete example. But 185|like, that is how I kind of think about like, you know, first off, think about where you think the 186|field is headed. Think about the intersection of what is an interesting idea to you and an 187|interesting idea to the field that's going to be useful to both. And then can you construct a 188|research agenda and a sequence of like, you know, a through line through that kind of axiom that 189|would allow you to. make meaningful progress on it. Yeah. I have so many questions from there, but 190|one thing I was thinking about, you mentioned as a researcher, you need your own unique point of 191|view. How do you think someone starts to develop that? Like when they haven't had much practice yet? 192|Yeah. It's hard because like you can't like retrospectively go back and change your life. Right. And 193|I think some of the best researchers I have, uh, that I've seen, um, have just this unique 194|experience that like somehow gives them this different worldview from everyone else. And I think 195|this is in the age of LLMs which are very messy and not the pure type of you know IMO intelligence 196|that you might expect people to might be a traditional indicator of success or correlated with 197|success in in more quantitative fields I think that people with this unique experience often do so 198|much better I think like one of the things that I've found has helped the most with me is like my 199|background is mainly like you know philosophy and literature I started out in law and philosophy I 200|did that for quite a while at university And I think that yes, quantitative intelligence is very 201|important, but for me in particular, I guess my unique skill vector is I love language and I love 202|the messy world of language and I think that I'm very comfortable operating in a world where 203|language models aren't these deterministic algorithms that you can solve with logic circuits. </file-view>
Need read rest. Use read 204-500, 501-750.
{
"file_path": "/tmp/wrapped_full.txt",
"offset": 204,
"limit": 300
}<file-view path="/tmp/wrapped_full.txt" start_line="204" end_line="405" total_lines="750"> 204|They're messy real-world things that have this behavior you need to shape. And so I would just 205|encourage people to think about what is... their unique skill vector that allows them to tackle 206|problems in different ways and I think there's so many examples of this another Australian like 207|Sholto I think like one of his unique advantages is like what he learned through his fencing career 208|and I think he's talked about this a fair bit and he has this sort of great line about you know what 209|one of the things that taught him was that most researchers focus on training only and by training I 210|mean like in the analogy of sports like preparing for a tournament or a game or whatever it is And 211|he said that what fencing really taught him was that like no matter how much preparation you do in 212|training wise, it doesn't matter if you're also not devoting a significant period, a portion of time 213|to performance. Like when you're actually in the minute of the game time, like how are you 214|performing and how does like your psychology and preparation and everything prepare for that? And I 215|think like that's a really unique skill vector, right? Like not many AI researchers have been world- 216|class fencers and to be able to learn that allows you to, you know, go in and do research and like, 217|yeah. obviously focus on the training which is like you're reading papers and you're absorbing 218|information and all this sort of stuff but the performance side of things so like when you sit down 219|to actually do research are you treating this like an actual game that you are now playing um and 220|it's high stakes in that in that sense so yeah like people should think about what what and everyone 221|has this i think but not many people leverage it as much as they should yeah i don't think we do 222|leverage it yeah it's not also what we're taught i think in school traditional school and things 223|like that yeah i'm also curious so you said that you were You're studying philosophy and then you 224|switched to AI. Can you tell me how this happened? So actually I wanted to study English literature 225|when I went to university. My mum was an English teacher for a bit and she's always been very big 226|into reading and got me into reading at a young age. A career advisor convinced me to do law instead 227|because they thought it was a safer career path and as an 18 year old I... just kind of went along 228|with that but yeah wanted to do philosophy as well so I think I just love the humanities and and 229|reading and writing um I didn't necessarily enjoy the law as much but like love the philosophy and 230|philosophy got me into Guern which is how I found I think it was the GPT-2 paper and Guern had 231|written a little bit about you know scaling laws even back in the GPT-2 days and then I went and 232|found the original GPT paper and was like wow look at the difference between these two things and 233|all we scaled up it's like the data and the model size essentially. And I think like for the first 234|time, like that quantitative world of deep learning and maths and stuff, which hadn't really 235|necessarily interested me before because it was so pure and well-constructed became really 236|interesting because you're suddenly trying to approximate human language, which like, you know, 237|Chomsky and all these others have said it could never be done. And there was early signs of life 238|there. And so that was the most interesting thing to me. I think that Delta showed me that like, you 239|know, when GPT-3 would eventually come out and GPT-4 and GPT-5, like if that Delta kept up, then, 240|this would probably be something that a lot of people would be talking about in five or ten years. 241|And then decided to throw myself into like language modeling, language modeling basically swapped to 242|pure maths and computer science. And fortunately that bet paid off and the scaling was stuck. But 243|yeah, it's I guess thanks to Gwen's philosophy and clicking on a few random hyperlinks around Gwen's 244|blog that it got me into language modeling. So then you started studying math, computer science and 245|went from there? Yeah. Okay. And then How did you come about co-founding Parse? Yeah, I was always 246|very much not interested in being a pure academic. I was also not interested in just going straight 247|into industry. I liked sitting on the line in between those two things. So during undergrad, my mate 248|Max, who I eventually co-founded Parse with, we had a small startup doing news de-biasing with 249|language models. So basically we'd set up this big scraper to scrape the internet every day. and 250|aggregate all the news stories and then like try and use a language model. And these were really bad 251|language models to like write a de-biased version of that story, like stripping out all the sort of 252|fluff around the periphery so that people could read unbiased news. So it was like an AI journalist? 253|AI journalist, exactly. Unbiased? Very difficult to sell to VCs. Yeah. We then tried to make it a 254|two-sided marketplace where like journalists could contribute information that would go into the 255|algorithm and consumers would read it. We knew nothing about VC or... what businesses would scale or 256|anything like that. It was just a really cool idea to us. And we had a lot of fun building it, but 257|like, I think I really got that itch. And even though it didn't really succeed in the end, like it 258|eventually morphed into this, you know, large scale, like rag enterprise thing that we ended up 259|selling to a few, few enterprises. Like I got the itch and I saw that like, you really could just 260|tackle problems. And like, particularly with LLMs, like there were just so many problems. that you 261|could tackle now that you couldn't have made a dent in before. And so I was like, I would love to 262|tackle the biggest one that I can within my capability. And I don't think that's purely in academia, 263|but I also think that like the research background enables you to tackle or like, like creates a 264|bigger set of problems that you can tackle once you have that, that experience. So you said you got 265|the itch to go find a big problem and solve it. Exactly. Okay. Okay. So then how does that lead to 266|Pars? Yeah. So that kind of finished. I was like, okay, I probably need to like, you know, it seems 267|a bit hypocritical based on everything I've said so far, but I probably need to like go and do a PhD 268|and like figure out what these problems are and develop my skillset more and this sort of stuff. And 269|so, yeah, I applied to Oxford, was very lucky to get Jakob Foster as a supervisor, who is like one 270|of the best, you know, meta learning RL, multi-agent RL, and was just trying to find interesting 271|problems to work on. And then I think within 45 minutes of having met Moody, Max introduced us at a 272|pub in Oxford. We've all realized how much of the itch we all had. And it was a pretty easy decision 273|after there. In fact, Moody and I and Max started the company after I'd spent 45 minutes in person 274|with Moody. So you were at a pub and you all kind of hit it off as friends and started talking about 275|what you're building. Yeah, well, not even what we were building, just like what we thought was 276|missing from the world that no one had built yet. And again, I feel like we all had these unique 277|perspectives. Like Moody was going to be a neurosurgeon. He was very interested in applying AI to 278|medicine in some form and felt like it was very under leveraged there. Max was doing neuroscience as 279|well, but more like computational neuroscience and like was very interested in how the brain worked 280|and thus how LMs worked and the parallels. And then I had my own background and like, I feel like 281|you really appreciate that unique perspective on other people, particularly when you converge on 282|like caring about similar problems. And so they just felt like the right people to start the company 283|with. I like how you guys all have like your real names and then like nicknames. Yeah, this is a 284|very Australian thing. I'm still learning all the nicknames. Yeah, I feel like I say like Kirk's and 285|people are like, who's that? I was like, oh, Max. Yeah, I was confused about that for a second. I 286|was like, wait, or Chuck. I was like, oh, Charlie's Chuck. Okay, makes more sense. Yeah. So talk 287|more about like finding the problem of parse. Like what does parse do? So originally we set out and 288|we thought, okay, we're just three like idiots basically like it's probably isn't gonna work like 289|we're gonna have a crack it's not gonna work like no one's gonna give us money like we'll have a 290|crack and we were we went out being very honest that we didn't have a very concrete idea like I 291|think we had a meeting with it was it was light speed I think and we were originally doing trying to 292|narrow ourselves down to this very confined like AI for like hospital use case like thing and we it 293|was like eight hours before the meeting and we were like this is the stupidest idea in the world. 294|Like we don't care about this. Like we've tried to pigeonhole a solution that we think the VCs would 295|care about. Like let's just go in and basically be honest and say like, okay, here are the things 296|we're best at. Here's what we think is missing. We're going to try and do blue sky research to try 297|and converge on like the exact problem we want to solve with the skill sets that we have to solve 298|it. And then that worked really well. And we raised like a seed round and then pretty quickly 299|converged on the idea of, you know, the world of big token where everyone was calling generalists 300|closed source models was not necessarily either the world we wanted to live in but also like an 301|efficient allocation of resources really like we really thought people should be specializing their 302|own models and so we were very early to that idea and it was quite contrarian but we were like okay 303|we're going to fully lock in on post-training which is what largely my background had been in and 304|figure out how do we get people from, you know, Opus 4.6, it wasn't Opus 4.6 at the time, but an 305|Opus 4.6 frontier model to a specialized RL open source model for their task. And how do you like 306|make that better over time? And I think there's a lot of grinding and gnashing of teeth as you try 307|to convince people this was possible, but kind of halfway through last year, I think people started 308|to come around to the fact that open source was now, the base open source was now in a place where 309|you really could post-train it and deliver results and quality that was above and beyond what the 310|frontier models were giving for a specific task. I've kind of asked you this before, but I want to 311|talk about like big token. What does that mean for someone who's listening and they haven't heard 312|this yet? Yeah, I think maybe like Moody likes to say that we came up with it when I put it in a 313|blog post once. I think people have said this for a while, but big token is essentially this idea 314|that like, you know, one or two frontier labs are... going to be the owners and controllers of AI 315|and like any useful AI. Like open source will never be able to keep up with the advancements and the 316|frontier inside these closed source labs and big token will rule AI in the same way that like big 317|tech has ruled tech and a handful of companies are going to basically steer a man's destiny with 318|that. So like we didn't necessarily think that even just from a philosophical point of view that 319|that's the world you want to live in. We were much more, originally it was Little Token, but I feel 320|like Little Token is a bit demeaning. So we started calling it like democratized token or something 321|like that. But we really thought that we didn't necessarily have to just buy into this narrative 322|that Anthropic was definitely going to win everything. And I think Anthropic is a great company and 323|they're going to be around for a long time. But yeah, like we wanted to train models that allowed 324|people to. at least give them the option to control their own intelligence rather than relying on 325|the big labs. So for someone who doesn't know much about open models yet, how do open models really 326|enable you to control your own intelligence? Yeah, so the thing you can do at the moment with open 327|models that you can't really do with closed source models is specialize them. And so, of course, 328|most people, and we even recommend to most people that you start out with closed source models and 329|you prompt them and you set up your scaffolding so that you know you can actually do your tasks in 330|the first place and confirm a language model is at least in the ballpark of being able to do this 331|thing but at the end of the day you're going to reach a ceiling of capability that you can get with 332|with prompting you've got to remember that as good as these frontier models are they are trained on 333|a whole distribution of the internet and then a whole bunch of oral environments 99.99% of which 334|have nothing to do with the tasks we're trying to do Of course, you need a certain base level of 335|intelligence and world model in order to be able to do the task. But you are going to eventually hit 336|the ceiling with what you can do with prompting and scaffolding and agent harness around a closed 337|source model. And then once you realize that, you realize that any of your competitors, so if you're 338|trying to build a medical scribe or a legal redlining agent, you've got to realize that if your 339|competitors are using the same model and wrapping the same model, then they're also going to 340|converge on. that ceiling, that 80% of task performance or whatever it is that you can get with 341|prompting a closed source model. And so then you start to say, okay, well, what is my only advantage 342|here? And maybe it's not even your competitors as well. Maybe you're actually worried about 343|Anthropic. I know that a lot of startups in the last sort of even few months have realized that if 344|you're doing PowerPoint generation or Excel or whatever, like Anthropic is very explicitly including 345|this in later rounds of Sonnet and Opus training. Like first it was PowerPoint. Sorry, first it was 346|Excel, then it was PowerPoint. Now they're going after general computer use. And so if your startup 347|is just a wrapper around Opus 4.6 with a few prompts, then you're very susceptible to Anthropic 348|eating your lunch. And so they sit back and they say, okay, what is the one advantage that we have 349|over Anthropic? Well, it's like the real world data and the real world signal that would allow me to 350|get better at this particular task and make a model better at this particular task. And the only way 351|you can really do that and have control over that at the moment is through training an open source 352|model. And so what that looks like concretely is, okay, first off people will tentatively dip their 353|toe in the water and they might come to someone like us and say, look, okay, we have this data, we 354|have this like signal, here's roughly what we think good answers and good executions of this task 355|look like, here's what we think bad executions of this task look like and where we think the 356|frontier models are falling short. Like, can we just go off and RL something to at least try this? 357|And we'll do that and then often they're very pleasantly surprised and they'll plug that in and... 358|that's kind of the traditional world of post-training. Like you do this big training run and they'll 359|deploy a model and it will serve like as this kind of static snapshot of the training that you did. 360|And then I think we're moving to a much more exciting world of, okay, we've done this initial 361|training, like what would it look like if we were to actually really couple this specialization with 362|the inference that we have? And this is one of the reasons I love working at Base10 is because 363|you're able to do this kind of coupling with inference. So instead of treating training as this one- 364|off run that you're going to do, it becomes... okay, I'm getting this continuous feedback signal 365|from the users actually using my model. Can I plug that into some form of online RL so that instead 366|of my model getting better every three months when I decide to do a training run, it's getting 367|better every five hours. And this is exactly what Curse has done really well. Big RL training run to 368|start off with for a composer, deploy that model in production, collect more signal from users who 369|are happy or not happy with the answers and use that to update the model. And they do it every five 370|hours. which is quite impressive at the scale that they're at. So I'm curious as to why, like, why 371|can you specialize an open source model in a way that you can't with a closed source model? Yeah, so 372|if you want to specialize a closed source model, and it's not like OpenAI and Anthropic have never 373|offered these solutions before. Like, they have offered, you know, a fine-tuning API where you 374|upload a JSON data set of input and output, like, prompt completion pairs. they will fine-tune it 375|for you and then send you back a specialized version of you know gp40 or 4 mini back when like this 376|was a big thing for them um but like learning is so much more than either completely pure supervised 377|learning where you simply have those like you know input output pairs and you're learning to predict 378|the next token on the output pairs or pure reinforcement learning where you have some sort of um 379|judge or way to assign a reward to a rollout and you're updating the model based on that. The hard 380|part about our jobs and training in general and learning in general is that the algorithm is kind of 381|commoditized now. Anyone can really write a GRPO loop or an SFT loop. And of course, it's nice that 382|OpenAI has given a way to be able to run that on some of their smaller models. But the hard part is 383|really how do you marry the customer or the person's expertise and opinions about what's good for 384|that task? with the reward signal in the first place? Like, how do you even get what we're going to 385|learn from? And to be able to do that, like you enter this very messy world of things that sit in 386|between on the spectrum of pure SFT through to pure RL. And so those are things more like, you know, 387|on policy self-distillation where a student will generate its own rollouts and its own attempts at a 388|trajectory. And then the teacher will be the same student, but just given like extra privilege 389|information. Okay, eventually this might be put in an API, but it's definitely not there yet. And so 390|it's not only like these algorithms that sit in between on the spectrum, it's also, yeah, where is 391|the reward signal coming from in the first place? Like if you're a legal company or you're going to 392|start trying to reinforcement fine-tune through OpenAI's API, yeah, of course, you don't have to 393|write the GRPO loop anymore, but like where do you even start from what is going to be the judge or 394|reward model for like, did this model do a good job at drafting this document or not? And so that's 395|the really hard part and it's what takes up most of our time from a manually intensive perspective. 396|I want to talk more about the details around how open source models work in terms of the fine 397|tuning, post-training, specialization. Can you break down what some of these terms mean for someone 398|who doesn't know? Yeah, so we typically start out with this relatively large language model that has 399|gone through a lot of pre-training where you take the whole internet and you just get it to predict 400|the next token and we teach it to do that. And then a lot of mid-training and post-training, which 401|looks a lot more like, okay, now that we have a language model that can do some things, let's one, 402|teach it to respond in like this chat style format. And then two, let's put it in a bunch of 403|reinforcement learning environments where it's going to attempt to do things on its own. And instead 404|of telling it exactly what the correct next thing to do is, we're just going to wait till it's 405|finished and tell it whether it did a good job or a bad job. And that's essentially like the </file-view>
Need remaining.
{
"file_path": "/tmp/wrapped_full.txt",
"offset": 406,
"limit": 350
}<file-view path="/tmp/wrapped_full.txt" start_line="406" end_line="608" total_lines="750"> 406|reinforcement learning that comes at the end. has given us these kind of reasoning models that 407|everyone has become so excited about. And that's what you get given when you start out. And that 408|process is exactly the same whether it's an open source model or a closed source model. These 409|recipes are largely agnostic between those two. But of course, as we kind of talked about, what you 410|can do is take, because we have the weights of the open source model, we can do whatever we want to 411|them. We can keep learning with those weights. Whereas if you're calling Claude or... ChatGBT, the 412|only lever of control you have is the prompt that you provided or the scaffolding that you put 413|around the language model. And so when we talk about specialization, and there's a lot of synonyms 414|for this, it's specialization, it's fine-tuning, it's post-training, and largely, to a large extent, 415|these are all the same thing. And then you have many different options in front of you in terms of 416|how to teach it. So the simplest option and what people used to do is just continuing to do next 417|token prediction with that model. on the data you care about. So you get it to predict the next 418|token based on everything that's come before. It might be in a legal domain. You're getting it to 419|predict the next token in this drafting document. And then we tell it whether it was right or wrong 420|and we slightly update the weights based on that. Again, you can also move to reinforcement 421|learning, which is where if you have some way to tell whether the model did a good job or a bad job 422|at a task, not just at a token level, but overall. So maths and coding are a great example of this 423|because, you know, for maths we know what the correct answer is we're able to verify for coding 424|we're able to run unit tests and see if they pass and so the reward becomes okay did this bit of 425|code that you you wrote pass all the unit tests or did this like proof that you wrote compile in 426|lean and it passes the formalization checks and so in that way we can then say okay based on all the 427|steps that you took language model like if you did a good job we're going to upweight the 428|probability of those steps that you took there and if you did a bad job we're going to downweight 429|the probability and over time that gives you model which is able to learn in a very different way 430|from supervised learning which is very much you know I'm just going to copy the next token and learn 431|to predict the next token of whatever distribution that's coming in. With reinforcement learning 432|it's much more exciting because people believe it it induces these more general reasoning 433|capabilities in the model. Basically you can think of this as like kind of how the old AlphaGo and 434|AlphaZero models work so chess is a great example where reinforcement learning It works from scratch 435|because at the end of a chess game, you always have a reward. You either won or you didn't. With 436|language models, we had to get them to the point where, like obviously if you took a randomly 437|initialized transformer, it's not going to ever really give you a correct math proof. Like you'd 438|have to sample from it for a very, very long time before you got anything near a correct solution. 439|So reinforcement learning is an inefficient thing to do when you don't have a very good base model. 440|But once you have this base model that has a general world model and a good understanding of the 441|world and is able to sometimes get the right answer, then reinforcement learning can take those 442|right answers and upweight their probability and then downweight the probability of when it doesn't 443|get the right answer and that's another way in which you can specialize and then there exists a lot 444|of solutions in between this so one thing that is wrong with supervised fine-tuning is that the 445|model is very off policy and by off policy i mean it's never making attempts by itself and then 446|getting some sort of feedback at the end it's always copying you know a teacher distribution or like 447|the the distribution that you give it to train on with next token prediction. But it is a very dense 448|signal because every single token you're getting a new signal about whether you did the right thing 449|or the wrong thing. With reinforcement learning, it's the opposite. It's very on policy, which is 450|good because you're able to learn from your own mistakes. You're in your own distribution. I think 451|humans are very similar. The way in which you learn best is often by doing something and getting 452|feedback at the end rather than like rote memorizing things. But it is very sparse feedback. So you 453|only get like the scalar feedback right at the end about whether you did the right thing or the 454|wrong thing. You don't know where in your whole rollout or reasoning chain was good and where was 455|bad. All you know is that like by the time you got to the end, you did a good job or a bad job. And 456|so there's a lot of methods that are trying to kind of bridge these gaps and say, how do we give 457|dense feedback, but with on policy generations? And so something like on policy self-distillation is 458|an example of this. we take a model and it will generate its own rollouts and instead of saying okay 459|this rollout is just good or bad we might have a way to you know provide more than just that scalar 460|feedback so in the context of a maths proof it might be a larger model with access to the correct 461|proof saying okay here is all the bits that you did right here's all the bits you did wrong and then 462|we say okay what is the probability of everything the student did under this big teacher And what 463|was the probability under the student? And can we move the student closer to the teacher? And so 464|there's a lot of exciting areas that people are pushing in this to try and bridge the opposite ends 465|of the spectrum of SFT and RL. But I think the point at the end of the day is that once you have a 466|base model, there's many different avenues you can take to specialize it and to teach it your 467|specific distribution that you care about. Yeah. Yeah, it's really cool. It seems like a lot of 468|people don't know about that yet, but I think that is in part because... open models only got good 469|enough like around a year and a half ago. Is that right? Yeah, I think you do need some base level 470|of intelligence in order to be able to like tackle meaningful tasks. I think structured document 471|extraction and very small tasks of open source models have been able to do for a long time, 472|particularly if you fine tune them. But only recently are we seeing people start to think, okay, 473|maybe this more broader task that I didn't really consider open source models for before, I might 474|actually be able to teach us to do. And I think this manifested at first in people with all these 475|big sort of agent harnesses often have a main agent, which is kind of like an orchestrator. And that 476|orchestrator is calling a bunch of subagents. So it might call a subagent specifically for search. 477|And I think last year when we were at Parler's, like people started to think, okay, what if I could 478|get my subagent that's doing search to be an open source model and I can RL it to be really good at 479|search. And search is a great example of this sort of quasi-verifiable domain where RL will work 480|really well. But now, because of the pressures that I talked about before in terms of people looking 481|to not be anthropic rappers anymore, they are starting to think, okay, what if I could drop in a 482|trained Kimmy or a trained GLM and it could do my main agent task? It's not like these models are 483|tiny. Kimmy is over a trillion parameters. We don't know quite how big GPT-5.4 and Opus are, but 484|probably in the realm of five trillion parameters. So it's not crazy. to think that a specialized 485|Kimi could compete with those and even outdo them. And Composer is a great example of this. In many 486|ways, the model UX of Composer and depending on which benchmark scores you look at are around the 487|level or perhaps even better than Opus 4.6. And so it just goes to show that there do exist very 488|large open source models now with the baseline level of intelligence that would enable you to tackle 489|very complex and very open-ended tasks if you train them in the right way. So you talked about how 490|they can be competitive with closed models. Are there other benefits? Over the last year, let's say 491|that the number of flops for inference in the world has gone up about three times. And let's say a 492|conservative estimate of the demand for inference tokens has gone up 10 times. By many estimates, it 493|would be larger than that. But we're starting to see that point where those curves cross over. And 494|now you can no longer afford to just throw your task at the biggest possible model. biggest possible 495|base model and just kind of take advantage of that consumer surplus and the intelligence you get 496|from that. Now you say, okay, I have less GPUs than I need to run, you know, this biggest possible 497|model. Like what is my other alternative? Well, I teach a smaller model to do this like faster and 498|cheaper. And so like, I guess those are the other benefits. Not only if you specialize, you can get 499|to a quality bar that's higher than the frontier models, but also you can usually get a much smaller 500|model and hence much faster and cheaper model to do the same thing. And so. Now that we're spending 501|as much money as we are on API credits for OpenAI and Anthropic, companies are really starting to 502|consider, okay, maybe we should be doing this in a cheaper way. Yeah, it's so sensitive. That makes 503|sense. Yeah, and there's examples as well where latency is really important. I think in medical 504|context, for instance, ER notes and pulling information and summarizing it for doctors there, 505|latency is super important. like access to consider for those sort of applications. And like if 506|you're relying on the intelligence of Opus 4 and 6, which is fine, but obviously like it's much 507|slower because it's a bigger model and also like Anthropix reliability has not always been the best. 508|Like if you want more reliability in a much faster model, then you would start to consider something 509|that's smaller and specialized and hosted on very reliable inference like Base 10. You talked about 510|latency, like with closed. models you can't really optimize for latency in a certain way as opposed 511|to open models? Yeah, so with closed models, you have to trust whatever inference engineering that 512|OpenAI or Anthropic has done behind the scenes to serve their family of three models to millions and 513|millions and millions of people worldwide. You also have to accept the uncertainty that comes with 514|that in the sense that you are going to be sending a request and many people will be sending a 515|request to those same GPUs at the same time. And your latency or your end-to-end time to last token 516|is going to very much depend on who else is using it and how much they're using it. You have no 517|control over that. Maybe you can get these kind of enterprise partnerships or whatever, but for most 518|people that's not an option. With someone like Base10, for instance, you have complete control over 519|how many GPUs you want this thing hosted on. You have complete control over the auto-scaling and the 520|concurrency and the utilization and everything like that, and you get access to... you know, these 521|world-class like inference engines that have been highly optimized to run your model as efficiently 522|as possible, as correctly as possible, as fast as possible. And so like the levers of control are 523|much more there, whereas like, you know, Anthropix are black box. We don't know how they're going 524|about optimizing things. We don't know how they're making decisions about how much money to make and 525|then thus how fast or slow to serve the models, how many people to serve those two in parallel. So 526|it's another, I think, axis of control that you get taken away from you if you're going with closed 527|source. Okay. So if I'm building something, you said earlier that first I should actually just start 528|with a closed model. And then you talked about some of the criteria for like when you might switch 529|to an open model. But I'm curious, like what are the downsides of open models or maybe what are the 530|reasons why someone actually wouldn't want to use an open source model? Yeah, I think it all comes 531|down to the beginning. Like I think the it kind of goes back to that top down approach, right? Like 532|you should be seeing if. at least the best model in the world that out of the box which is the 533|closed source models are capable of doing our task and can you shape that thing in a way which is 534|like you know 80 good on that task and i think if you're trying to do that with open source straight 535|off the bat like you kind of run into problems like particularly if you don't quite understand the 536|contours and the minutiae of like okay this model's this particular open source model is good at 537|this thing it might fall short in this regard um and also just like hosting these things in the 538|first place like based on where you're calling these open source models from, you can get very 539|different results. Like people really quantize these things heavily. There's often like, if you're 540|not very careful and you're not very good at inference, like there's often issues with like the 541|kernels and everything that sits under the hood. And like that just creates complexity you don't 542|need when you're starting out. Like the great thing about Anthropic and OpenAI is they have, you 543|know, two or three models and they're very good at serving them. And then they know they'll be 544|served correctly. And all you have to do is worry about prompting them. So just getting off the 545|ground, like I am a big proponent of, you know. you should just see if the best possible model out 546|there can do your thing. Then you work your way down, you say, okay, what about the level below 547|that? I know Opus can do this really well. What about Sonnet? Can Sonnet do it just as well? And 548|then maybe even Haiku. And then you say, okay, what does it look like if I specialize an open source 549|model now to do this? Or at least try an open source model. And by then you've understood the 550|contours of your task to an extent that you probably could plug in an open source model and 551|understand at least what it's doing differently from Opus. what is doing worse, maybe what is doing 552|better. And that gives you a good ground to start shaping these rewards for things like 553|reinforcement learning. Okay, very cool. So will open models ever catch up to closed models? I think 554|I read that they used to be 18 months behind. Now it's more like three months. But I'm curious your 555|thoughts on this question of will open source models ever catch up to closed source? I think I have 556|different answers to this question over different time horizons. So the first shorter... answer like 557|over a shorter time horizon the answer to me is like probably not to be honest like the base closed 558|source models will always kind of be a little bit ahead of the based open source models i think 559|that's for a bunch of reasons i think it's terms of how vcs allocate compute i think it's in terms 560|of you know gpu blocks to china and a few other reasons but i don't necessarily think that matters 561|as much as people say it does like to me the more interesting question the question is not okay, how 562|close are the base models to each other? But because you have the advantage of specialization of 563|open source over closed source, the question is like, what is the intelligence ceiling of this model 564|if it were to be completely optimized for this task? And again, with closed source models, the lever 565|that you can pull for optimizing is prompting. And for open source models, it's prompting plus 566|supervised fine tuning plus reinforcement learning plus all these things. And so like the question 567|you should be asking is, okay, what is the intelligence ceiling? And for me, like the intelligence 568|ceiling of specialized open source, if you allow yourself to specialize a particular copy of a model 569|for each individual task, probably now almost subsumes that of the closed source models. I think 570|there's a few areas in which just raw intelligence and size is always going to be important. Things 571|like scientific discovery, like, you know, math proof, things along those lines are going to always 572|belong in the domain of the biggest model. But for most economically valuable tasks that I'm seeing 573|now, like I would struggle to find something in which, a really well-trained Kimi would not be able 574|to at least compete with, if not outperform, some of the or all of the closed source models. And so 575|it doesn't necessarily matter if the base model is slightly worse. I think we're now at a point 576|where we've crossed this threshold such that most tasks that people are caring about and building 577|businesses around right now are able to be done by open source models. I think open source has 578|actually done a really good job of hanging in long enough to reach that threshold such that now that 579|specialization is going to become a much larger thing, they're going to remain to be very, very 580|useful for all the reasons we've talked about before. My longer term answer to this is, you know, 581|like I think it's a very discreet thing. It's not like this continuous straight line. Like I think 582|with the new GPUs that NVIDIA is bringing online and the fact that the closed source labs are going 583|to have priority over those and that's going to allow them to train, you know, MOEs, which are, 5 to 584|10 to 15 trillion parameters larger than the ones we've got now. I think that there are going to be 585|these discontinuous jumps as open source struggles to get the compute to keep up with those scalar 586|training runs. But I also am a believer that, yeah, scaling laws hold, but there's a bunch of inputs 587|to the scaling laws. There's not only model size, but there's also data. And I think that at some 588|point you kind of run out of data, and people have said this for a while, but when you get to 20 589|trillion parameters, I think, the bottleneck is no longer the compute or the energy to run the 590|compute. It's what do we actually put into this thing that is actually going to learn more than just 591|like repeating over the same data that we've curated over the past, you know, eight years in 592|Anthropic. And at that point, I think like if you've run out of, you know, environments to buy from 593|a core or random esoteric books to find and then put into the pre-training data, then the big 594|advantage is like very specialized data. And then I think that actually swings there. the kind of 595|pendulum back in the favor of open source because again you're going to have these companies who are 596|like okay the only advantage we have over the closed source labs is our specific data and our real 597|world use the signal on this very specific task sure it's not for everything but at least i can make 598|use of this and like maybe i don't need a 20 trillion parameter model to do this um and we're now at 599|the point where i think you know the 1 trillion parameter models can do a lot of the useful things 600|that we care about and eventually like the compute will get commoditized again and of course like 601|nvidia will bring out a new line of gpus and they'll make ruben seem old but like open source will 602|have access to ruben and we might have you know five trillion parameter open source models which 603|will raise the intelligence ceiling yet again and i think as long as open source hangs in there to 604|the point where we can basically do everything that we care about in the world today with a 605|specialized open source model i think then that's a healthy dynamic i think the closed source will 606|still have a lot of areas where the most intelligent and largest model will win. And I think they'll 607|accrue a lot of benefit from that as they deserve to because they've put so much into the CapEx to 608|build out and the R&D to build out those models. But I think that open source will be the backbone </file-view>
Need rest from 609-750.
{
"file_path": "/tmp/wrapped_full.txt",
"offset": 609,
"limit": 150
}<file-view path="/tmp/wrapped_full.txt" start_line="609" end_line="750" total_lines="750"> 609|of a society where a lot of people are serving specialized models to do very particular things. 610|There's a lot of talk about LLMs replacing jobs, but it seems like because of the rise of open 611|models that we're actually creating jobs in the inference space. Is this accurate? I think, yeah, 612|like it's certainly accurate in the infant space. And I'd probably push back on people who say that, 613|you know, LMs are taking jobs in general. I think there's been a bunch of scare pieces published in 614|like the last year or so, like the Citrini piece and AI 2027. And a lot of these are talking about 615|the fact that like as AI takes up more and more of like current human labor, then the value of human 616|labor goes to zero and you get these like inflationary effects and like, sorry, deflationary effects 617|and everything kind of goes off and it's not very good for the economy. But I think this assumes a 618|world in which like, you know, demand is fixed and there's like a fixed slice of the pie. And 619|historically, if you look at what's happened with like any advent of any new technology, this has 620|like never been the case. Like when the internet came along, we did all like computers came along. 621|We didn't just like automate like manual clocking and like all the things that we were doing back in 622|the 80s and 90s. Like it also fostered a whole new ecosystem of jobs, which like opened up what 623|people could actually do. And so I think this argument of, okay, LLMs are going to take human labour 624|and that's going to be a fundamentally bad thing, is not necessarily the correct framing. I think 625|it's going to look more like, yes, there will be some short-term pain as jobs that we know it today 626|that are readily available to many even skilled workers are replaced by LLMs. But I think that the 627|capacity that that frees up and the jobs that that creates in the long run are going to be... a 628|repetition of everything that we've seen in the past and like humans have historically been very 629|good at figuring out what to do when you know very manual um and labor-based tasks have been 630|replaced and i think that will be a similar for like cognitive based tasks i think inference 631|engineering is a great example of that um like one of the perhaps fastest growing areas in the world 632|with one of the highest ramp ups in complexity um i think that you know when this first started this 633|lm boom first started we had very vanilla gpd style models, inference engineering was hard enough. 634|But as researchers got more and more wacky trying to figure out ways to make these models smarter, 635|Mixture of Experts is a great example of this, where we have a much bigger model with many more 636|parameters, but we only activate some of those parameters every time we call it, based on which 637|experts we decide to route to for this particular question or this particular token. That's a really 638|hard inference problem, and that's certainly much harder than... the vanilla dense models that we 639|had back in 2019, 2020, 2021. And so as these models get smarter, it's a really concrete example of 640|an area where there's just going to be more and more jobs as we figure out how to manage these 641|things. And they're very cognitively demanding jobs, sure. But I think that same pattern will hold 642|through a lot of the rest of the economy as well. Yeah. So you talked earlier about your learning 643|approach and kind of top-down learning. For someone like me, kind of as a regular software engineer, 644|and I want to get into inference engineering, what could be a good first step or first project for 645|me? Yeah, I think actually the way that inference engineering has gone is a really good example of 646|that top-down approach that I've been talking about. So in the old days, to be a really good 647|inference engineer, I think you did have to start from the bottom up. You had to learn how to write 648|kernels. You had to learn how to plug that together into... some sort of inference engine with 649|queues and continuous batching and all this sort of stuff. I think nowadays it looks much more like 650|there are solutions that exist out of the box where you download VLM and you download a model and 651|you load it up and you can just call it like it can generate it at like reasonably fast speeds and 652|so now the problems are shifting towards okay like now that I can get this thing running again which 653|is the top-down approach like can I just get it working in the first place then you start to think 654|about the optimizations that can be made around that and so I then start to think about, you know, 655|okay, I don't have to learn how to write kernels immediately, but okay, like I'm now in this agentic 656|system where the pre-fill is dominating much of my time and the decode doesn't matter as much. Like, 657|can I learn about things like KB cache management and shuttling that around and like networking 658|stability and all those sorts of things. And then maybe eventually I get to kernels. But like at the 659|end of the day, like VLM. TRT, SG Lang, these are all inference engines which are able to run models 660|out of the box but still require a lot of work and understanding to make them as good as possible 661|and reliable as possible. And that goes for a very high level in terms of like, you know, playing 662|around with like speculators and eagle heads and things like that through to like, oh, the other day 663|someone found this massive bug in TRT which meant that mixture of experts was getting routed 664|incorrectly. And that actually got found by someone playing around with it internally at Base 10. So 665|I think the fact that you are able to get inference working out of the box is pretty lucky. When I 666|first started on these things, that certainly wasn't the case. But it enables you to go top-down in 667|a way that you couldn't before. So you co-founded Parse, and you were acquired by Base 10, which is 668|how I know you, because I work at Base 10. So can you talk about why you chose Base10 and how this 669|happened? Yeah. It was funny. The first meeting we had with Tu and CEO of Base10 was we were trying 670|to sort out our inference provider for PaaS. Obviously, we would train models for customers and then 671|we needed somewhere to put them. And so we talked to all the big inference providers and we weren't 672|really that happy. And we walked into the Base10 meeting. I was visiting a customer in Australia at 673|the time, so it was very early for me. I think it was basically still in my pajamas. Moody told me 674|like, you know, we're going to play a bit of hardball with these guys. So like you go in and play a 675|bit of hardball. With your pajamas on? With my pajamas on. And I remember asking Toon very 676|distinctly, like if he could handle our volume, which was like such a ludicrous thing to ask for 677|like a C stage startup with like barely any customers. And Toon just laughed at me. But like, I 678|think the relationship grew from there. And like, as we realized how good base 10 was and as we grew 679|alongside base 10. like we made them our sole inference provider because they offered many things 680|that like none of the other inference providers could and then i think eventually like base 10 681|started to notice that like customers were just appearing through us on the base 10 platform and 682|they were all using specialized models and like i think maybe some of the base 10 had tried to like 683|kind of work with before um and base 10 is the best in the world at inference and we thought we were 684|really good at post training and getting people off closed source models onto open source models and 685|so eventually at some point turn was sort of like why didn't you guys come do this with us? I think 686|the customer doors that are going to be open are going to be 10 times larger. The demand is clearly 687|there. And then if you couple yourselves with inference, it's a no-brainer. And for us, this solved 688|a really difficult problem of feeling like we were this awkward middleman between the customer and 689|the GPUs. Once you trained a model for a customer, theoretically, they could just go off and deploy 690|it on their own GPUs. They could go to base 10 directly. That's something that's not really a proper 691|business model there. And for Moody and Max and I, I think all we cared about at the end of the day 692|was this problem and solving this problem. I don't think we necessarily felt like we needed to be 693|our own independent startup or company in order to be able to solve it. And we're very lucky that 694|we've been able to retain the autonomy and make the research bets that we want to make and have the 695|compute that we need at Base10. And Tuan and Amir and Phil and everyone have been really supportive 696|of that. And so, yeah, it's kind of the best of both worlds. And I think it's been a win-win for... 697|showing how training can be coupled with inference to give something much more powerful than either 698|of those two things by themselves. I think many people in the AI space, like Dario, for example, 699|have argued against open source models for safety reasons. And I'm curious your thoughts on that. 700|Yeah, I think people like to talk about very specific examples here, like building bioweapons and 701|X-risk and all these sorts of things. I think stripping away the superintelligence argument for a 702|second, The thing I worry most about with AI is not how a small handful of bad actors will use this 703|in various specific scenarios. I think it's the fact that a concentration of power without 704|accountability has always been the most severe failure mode of any civilization as far back as we 705|can remember. I think if you do live in a world where one company or maybe two companies and the 706|handful of people inside those companies responsible for making the decisions have all the control, 707|over AI, which is like the most powerful technology we've seen. I think that even if you trust the 708|initial actors at the start, if you trust Dario and his vision, it's a very different situation to, 709|you know, some of the technologies we have in the past where there's sunset clauses kind of built 710|in, like you know the distribution and diffusion of that technology will eventually happen. And I 711|say that for one main reason, which is that the people who control the AI and like once that gets 712|past a certain threshold, are going to have the capability to use that AI to extend their own 713|advantage and their lead. So there is no sunset clause built in here. And so I think that's like 714|everything that, you know, Western democracy fights against is concentration of power, particularly 715|power that enables itself to continue its own reign is like a fundamentally bad thing. Like there's 716|so many checks and balances that we have built into like our political system that we trust to 717|ensure that that doesn't happen. And I think that like if you are buying into this argument of you 718|know ai is going to fundamentally change the world then even if you trust anthropic now like i don't 719|think the variance and the risk of incentives and and um behaviors changing over time is is worth 720|having one person controlling all the air and so i actually think the more conservative and status 721|quo you bet to make would be to diffuse this technology and sure there are going to be short-term 722|pains in terms of like allowing ai to be used by basically anyone in the world. I think like 723|cybersecurity is a great example of that. And again, people are talking about this with mythos. But 724|I think that short-term gain is like kind of an inoculation to the world against like all the 725|potential harms it's going to have. And it's also going to allow the world to more readily robustify 726|like the existing infrastructure that we have to handle like intelligent AI. It's also going to 727|allow us to build in these defenses with the same intelligent AI to counteract the effects of bad 728|actors. And so, yeah, like at the end of the day, concentration of power, particularly power that 729|can extend itself, is like not something we've ever accepted before. And the fact that AI will 730|enable whoever controls it to extend their own lead, if it's kept it completely internal, is not 731|necessarily a great thing to me. And so that's kind of why I think that like we should just trust 732|the systems that we have for deciding these things and stick with a more democratic future of AI 733|rather than one where, you know. anthropic or open AI controls at all. We talked about this question 734|of like who controls AI and some people would say that having the weights like having open weights 735|is not actually who controls AI but it's actually who has the compute power. I'm curious your 736|thoughts on that. It definitely is. I think that we're lucky where even the biggest models in the 737|world basically run on one node so like essentially eight GPUs. The great thing about how language 738|models are currently trained is that you need a lot of compute to do training, but to actually serve 739|the model, at least for a handful of users, you only need a small number of GPUs relative to the 740|amount used for training. And so, yes, it might be the case that you need a lot of GPUs in order to 741|get to these large models, but if those are being distributed, then you need much less GPUs in order 742|to be able to run it. So I think that kind of balances out or amortizes the compute concern 743|somewhat. Of course, there's like second and third order effects of this. Like if you are able to 744|get access to so much more compute than your competitors that you're able to train, you know, the 30 745|trillion monstrosity that is starting to approximate what a super intelligence would look like, then 746|that's like a fundamentally different concern. But I think in terms of the setup that we have now 747|where we're pushing between one and five trillion parameter models, like you are able to still 748|democratize access to that once it has been trained. And so we just need people who have incentives 749|to keep training and releasing these open source models to continue to do so, so that this can kind 750|of promulgate. Cool. Thanks for being here, Charlie. Thanks for having me. Okay. </file-view>
We have full transcript. Need produce full synopsis. The user wants a full synopsis. Should be comprehensive. We can structure with sections: introduction/background, key themes, detailed discussion by topic. Since it's long, a structured summary is best. We need include all major arguments and specifics. Let's draft a thorough synopsis, maybe 1000-1500 words. Must be accurate to transcript. Content summary: - Interview between Madison Kanna and Charlie (co-founder of Parse, now at Base10). Charlie is Australian, studied law & philosophy, got into AI via Gwern blog, scaling laws, GPT-2/GPT. During undergrad had startup with Max doing news de-biasing using language models. Then applied to Oxford PhD under Jakob Foerster (meta-learning/RL/multi-agent). Met Moody at a pub; within 45 minutes decided to start a company. Took leave from Oxford; company originally in London, then moved to SF because network effects. Hacker house, all employees moved within two weeks. Startup called Paz (post-training as a service) then Parse? Actually transcript says "Paz" and "Parse"? It says "Paz, which was our startup before we joined Base10". Also "Parse"? There is ambiguity: "PaaS" maybe "Parse"? He says "Parse" and "Paz"? Need note. They did post-training as a service, enabling companies to specialize open-source models. They targeted non-verifiable domains (legal, medicine, insurance) rather than verifiable tasks (math/coding) that others focused on. They were acquired by Base10 because Base10 is best at inference and Parse at post-training; coupling training + inference. - - Themes: 1. PhD vs industry / individual leverage: He argues for first time individual leverage multiplied, small teams can tackle huge problems. Academic route gives long-term strategic planning; startup is greedy action selection. But research background helps select unique problems. You don't need a PhD to be AI researcher; need top-down learning and unique skill vector. The baseball vs basketball player analogy: careers used to be basketball (hustle on many small skills); now need baseball (hit home runs) because LLMs will automate many skills. Develop metacognition and taste. Top-down learning: get thing working first, fill in skill tree downward, e.g., Jeremy Howard/fast.ai, learn RL by running framework, tuning hyperparams, then understanding algorithms. 2. How to become a researcher: have enough context to have value function; find intersection of personal interest, field direction, and org context. His current focus at Base10: continual learning for specific tasks/jobs, not general internet update. Working backwards from axiom: context window limitation -> compress KV cache (neural KBK compaction) -> train models with compaction in loop -> RL tool to compact own context -> compress context to LoRA adapters/weights. 3. Unique point of view: Charlie's background in philosophy/literature/law gives comfort with messy language. Sholto's fencing analogy: training vs performance; need to perform in the moment. 4. Parse / Base10 and open-source specialization: They wanted to give people option to control own intelligence vs big labs (big token). Open models allow specialization; closed models only prompt + scaffolding. Most startups should start with closed models (Opus/Sonnet/Haiku) to see if they can do task ~80%, then move down to open-source specialization. Because frontier models are trained on broad distribution, they hit ceiling on specific tasks; also closed-source providers can copy your wrapper features. Real advantage is proprietary data/signal; only open-source training lets you use it fully. Post-training: SFT, RL, on-policy self-distillation, etc. Open source model weights enable continuous learning/online RL with inference feedback; example Cursor updates composer model every 5 hours. 5. Open vs closed models, economics/latency: Inference demand rising faster than compute; can't just throw biggest model. Specialized open models can be smaller, faster, cheaper, more reliable, with controlled inference. Closed model latency is black-box and shared. Base10 provides optimized inference. 6. Will open source catch up? Short term: base closed models will stay slightly ahead due to compute allocation, export controls. But the right question is intelligence ceiling for a specific task, where open source + specialization can subsume closed. For most economically valuable tasks, a well-trained Kimi etc can compete. Long term: discontinuous jumps as closed labs get next-gen GPUs and train huge MoEs; but data may become bottleneck before compute; specialized data then swings pendulum back to open. Open source will be backbone of specialized-task society; closed will own highest-end scientific discovery. 7. Jobs/AI: Pushback on labor-value-to-zero scare pieces. Demand isn't fixed; new tech creates new jobs. Inference engineering is example. Short-term pain but long-term new ecosystem. 8. Inference engineering for software engineers: Top-down now; start with vLLM/TRT-LLM/SGLang, get model running, then optimize KV cache, networking, then kernels. 9. Safety / open source: Not worried about specific bad actors/bioweapons as much as concentration of unaccountable power. AI can extend its controllers' lead, no sunset clause. Western democracy guards against concentration. Diffusion is more conservative/status quo; short-term cybersecurity risks are inoculation and allow building defenses. Compute control is real but serving requires fewer GPUs; democratize access. Need ensure we mention names: Charlie, Moody, Max (Kirk? nickname), Madison Kanna, Jakob Foerster, Base10 (Tuan/Amir/Phil), Jeremy Howard, Andrew Ng, Sholto, Gwern, Cursor, Kimi, Composer, OpenAI, Anthropic. Let's write a full synopsis with these sections. Use headers. Keep it detailed but not overly verbose. Could include key quotes. Avoid verbatim large chunks. Provide concise but comprehensive. Need cite transcript? We can reference file? Not necessary but can mention saved transcript. Need check: Did the user want synopsis of the tweet video? Yes. The video is the Madison Kanna interview. We can produce. We might mention full transcript source. Let's write.
Here is a full synopsis of the 64-minute interview (transcribed and saved to <ref_file file="/Users/root1/2077067032152182986.txt" />).
The video is an interview by Madison Kanna with Charlie (co-founder of Parse / PaaS, now at Base10). Charlie is Australian, studied law and philosophy as an undergrad, fell into AI through Gwern’s blog and the early GPT scaling-law papers, started a news-debiasing startup with his friend Max, began a PhD at Oxford under Jakob Foerster, met Moody in an Oxford pub, and the three of them left academia to build a post-training company. They later joined Base10 to combine model training with inference infrastructure.
Charlie’s path was indirect. He did undergrad in Australia, originally wanted to study English literature, was steered into law, but added philosophy. Through philosophy he found Gwern’s writing, which led him to the GPT-2 paper and then the original GPT paper. He was struck by how much of the jump came from simply scaling data and model size, and he bet that language modeling would become important.
During undergrad he and Max built a “news-debiasing” AI journalist startup. It didn’t really take off, but it gave him the “itch” to solve real problems. He applied to Oxford for a PhD in meta-learning / RL, got Jakob Foerster as an advisor, and within 45 minutes of meeting Moody at a pub, the three of them decided to start a company. Charlie took a leave of absence from Oxford (“realistically only a few days”) and they gave the startup one year.
They started in London, close to Oxford, but after a trip to San Francisco realized that “more things happened in four days than in the previous four or five months.” They concluded the network effects and “surface area of luck” in SF were too large to ignore, so the whole company moved into a hacker house within about two weeks.
Charlie argues that, for the first time in a long time, a small group of individuals can have outsized impact. The choice is no longer just “poor grad student vs. big-tech salary,” but whether to spend years advancing a tiny edge of a scientific bubble or to tackle a much bigger, more impactful problem now. Startups force “greedy action selection” (surviving day-to-day), while research teaches longer-term strategic planning and the habit of asking: if I don’t work on this, will anyone else solve it in the way I would?
The academic background still helped. With their post-training-as-a-service startup, they knew verifiable tasks like math and coding would get commoditized quickly, so they deliberately went after non-verifiable domains such as law, medicine, and insurance, where the reward signal is messy and domain expertise matters.
Charlie believes you do not need a PhD to be an AI researcher today. The more important shift is from a “basketball player” career—hustling on many small, general skills—to a “baseball player” career: doing nothing for long stretches but occasionally hitting home runs. As LLMs commoditize many routine skills, the valuable human contribution becomes:
He advocates a top-down learning style rather than the traditional bottom-up math-first approach. The Jeremy Howard / fast.ai method is his model: train a working model in four lines of code first, then work your way down to understanding the internals. If he wanted to learn RL today, he would grab an open-source framework, plug it into an environment, watch the curve go up, tune hyperparameters, and only then read the code and the theory. This gives motivation and velocity.
Charlie grounds his research advice in his own current work at Base10. His “axiom” is that the future needs continual learning for specific tasks and jobs, not just the big-lab version of updating a general model on the last six months of internet data.
Working backward from that axiom, he constructs a research sequence:
This is his example of how to build a through-line from a first-principles belief to a tractable research program.
Parse (also referred to as “Paz”) started without a tightly scoped idea. They initially pitched a hospital use case they didn’t care about, realized it was “the stupidest idea in the world” the night before a VC meeting, and instead pitched their honest strengths: they wanted to do blue-sky research and converge on a real problem. They raised a seed round and locked in on post-training.
They were motivated by a rejection of the “big token” narrative—the idea that one or two closed labs would control all useful AI. They wanted to let people control their own intelligence. They focused on taking frontier closed models and post-training open-weight models to specialize them for a customer’s task.
They chose Base10 because:
Charlie is not dogmatically anti-closed-source. He recommends most builders start with the best closed model (Opus, Sonnet, Haiku) and see if prompting and scaffolding can get the task to ~80%. Only after you understand the task’s contours should you move to an open model. Reasons:
With open weights you can do supervised fine-tuning, reinforcement learning, and on-policy self-distillation, and you can continuously improve the model from live user feedback. He cites Cursor as doing this well: a big RL run, deploy, collect user satisfaction signal, and update the model every five hours.
Specialization can also mean a smaller, faster, cheaper model. As inference demand has outpaced inference compute, throwing every task at the biggest model is no longer economical. Specialized open models can beat frontier models on latency, cost, and reliability, especially when paired with optimized inference on platforms like Base10.
Charlie gives different answers for different time horizons:
He concludes that closed labs will likely own the absolute frontier of scientific discovery and raw reasoning, but open-source specialized models will become the backbone of society for most practical tasks.
Charlie pushes back on the “AI will drive human labor value to zero” argument. He says it assumes a fixed economic pie, which history contradicts. New technologies destroy some jobs but create new ecosystems. Inference engineering is one concrete, fast-growing, cognitively demanding example. There will be short-term pain, but long-term he expects a repeat of the pattern seen with computers and the internet: new jobs and expanded capabilities.
For a software engineer wanting to get into inference engineering, he again recommends a top-down path. Today you can download vLLM, TRT-LLM, or SGLang and get a model running without writing kernels. Start there, then optimize KV-cache management, networking, and pre-fill/decode tradeoffs, and only later dive into kernels if needed.
Charlie directly addresses the Dario / Anthropic-style argument that open-source models are dangerous. He says the more serious risk is not a few bad actors using models for bioweapons; it is concentration of unaccountable power. AI is a technology that can be used by its controllers to extend their own lead, so there is no natural “sunset clause” for diffusion. Western democracy, he argues, is built on checks against exactly this kind of concentrated, self-perpetuating power.
Therefore, the conservative, status-quo-preserving bet is to diffuse the technology. Short-term harms—cybersecurity, misuse—are real, but they also act as an inoculation that forces society to robustify infrastructure and build defensive AI. As long as the biggest models still run on a single node (~8 GPUs), serving can be democratized even if training is expensive.
The interview is a long-form argument that the future of AI belongs not to a handful of closed labs, but to a broad ecosystem of specialized, continually learning, open-weight models owned by the people and companies that use them. Charlie’s personal story—humanities student → law/philosophy → Oxford PhD dropout → SF founder → Base10—serves as the narrative backbone for his broader thesis: in the age of LLMs, the winning researchers and builders will be those who combine deep intuition, unique perspective, top-down learning, and the practical ability to train and deploy models that improve from real-world use.
what does he say about people jhust getting into RL
The user is asking specifically about what Charlie says about people just getting into RL (reinforcement learning). Let me pull the relevant section from the transcript.
Here's what Charlie says about getting into RL, from the transcript:
His recommended approach is top-down, not bottom-up. If he wanted to learn RL and be the best at it, here's exactly what he would do (<ref_snippet file="/Users/root1/2077067032152182986.txt" lines="32-33" />):
He contrasts this with the traditional approach: starting from policy gradients, reading Rich Sutton's textbook, working through the history of RL to "fully understand everything before I run my first GRPO loop." He says the former beats the latter every time — you become a better and more practical researcher because of it.
The key insight: get something working end-to-end first, then fill in the skill tree on the way down. This gives you velocity, motivation, and intuition — the things that actually matter for doing good research — rather than spending six months learning prerequisites before you ever touch the thing you wanted to learn.