Agents, Prompts, and RAG

Augmenting LLMs: Challenges and Opportunities. Prompt Engineering: The First Line of Optimization Fine-Tuning: Proceed with Caution Retrieval-Augmented Generation (RAG): Enhancing Model Utility Agentic AI Workflows: Toward Autonomous and Specialized Systems Case Study: Evals Multi-Agent Workflows: Parallelism What's Next in AI? Personal Thoughts

Boris Cherny Setup

Source: https://x.com/laurenmhreeder/status/2051351487515902247?s=20 A bunch of sessions. 5 to 10 sessions at the same time. Each session has a bunch of Agents. A few hundred Agents going. A few ways to manage it:

Notion: What we learned during the Custom Agents beta

We launched Custom Agents in beta back in February. In two months, you’ve created more than a million Custom Agents. What surprised us most was how quickly teams went from experimenting with their first agent to workflows that whole companies run on. Our agents “hall of fame” captures some of the best with examples from teams like Ramp, Vercel, and Clay with step-by-step guides so you can build them too. Custom Agents are out of beta and usage-based pricing goes live today. Here’s what we learned building alongside you. The fastest teams treated Custom Agents like internal products At Ramp, that approach grew into over 300 Custom Agents. Ben Levick, their Head of Ops and Internal AI, describes what that momentum feels like: Agent-building became a new operational skill Heidi’s product team saves up to 60 hours each month by creating an internal Q&A agent to answers product-related questions. Equals Money's product marketing team built a Knowledge Base Amendment agent, so docs stay current after every release and website updates don’t get missed. Notion's Customer Support team built an agent that helps reps respond to customers faster and spend more time on complex questions. None of these agents were built by engineers. If you can describe what you’d tell a new hire to do, you can build a Custom Agent. Equals Money built a Custom Agent that catches every product release in Slack and turns it into structured, actionable knowledge in Notion. The biggest wins came from rewiring workflows At Braintrust, competitive intelligence used to require the marketing team to do manual research every morning. When it slipped, reps showed up to calls without fresh context and deals slowed down. So the marketing team built Competitive Intelligence Buddy. Every morning, it scans competitor change logs and Gong transcripts, updates the sales one-pager in Notion, and flags launches that matter for active deals. Now reps show up ready without doing any extra prep and the marketing team gets hours back every day. Cost efficiency comes from both small and big wins Scaling confidently is the hard part Starting today, you'll have new tools to set guardrails like per-agent credit limits, full visibility into every agent across your workspace on your dashboard, and ways to catch issues fast like proactive alerts as you approach your credit limit and the ability to pause agents. Our team put together a quick guide covering every control and best practices for getting started. Getting ready for usage-based pricing What’s next

Prompts

PI

@earendil-works/pi-agent-core Stateful agent with tool execution and event streaming. Built on @earendil-works/pi-ai. Installation npm install @earendil-works/pi-agent-core Quick Start import { Agent } from "@earendil-works/pi-agent-core"; import { getModel } from "@earendil-works/pi-ai"; const agent = new Agent({ initialState: { systemPrompt: "You are a helpful assistant.", model: getModel("anthropic", "claude-sonnet-4-20250514"), }, }); agent.subscribe((event) => { if (event.type === "message_update" && event.assistantMessageEvent.type === "text_delta") { // Stream just the new text chunk process.stdout.write(event.assistantMessageEvent.delta); } }); await agent.prompt("Hello!"); Core Concepts Event Flow Agent Options Agent State Methods Steering and Follow-up Custom Message Types Tools Error Handling Proxy Usage Low-Level API