Get early access
Back to episodes
#2over 1 year ago

Reasoning Models vs Reasoning Prompts

Models can reason but you can also reason within a prompt. Which technique wins out when and why? We'll find out by adding reasoning to an existing movie chat agent.

Project details

Open in GitHub

🦄 reasoning models vs reasoning prompts

models can reason but you can also reason within a prompt. which technique wins out when and why? we'll find out by adding reasoning to an existing movie chat agent.

Video

image

Running this code

# Install dependencies
pnpm install
# Convert BAML files -> TypeScript
pnpm run generate
# Run the code
pnpm run dev

Followup Exercises

What workflows do you have that you can add reasoning to?

What reasoning workflows can you replace with smaller cheaper models?

Session Notes

Key Takeaways

  • You can make a cheap model do reasoning just by prompting it well
  • Time management of your Engineering Team
    • o3 / reasoning model if you just wanna move fast
  • Cost management / speed corollary
    • if you need performance / speed / choice
    • if you can only run small models e.g. OSS or at the edge
  • better prompts / guided reasoning, better than generic <THINK> tokens in general-purpose models
    • you can make a good reasoning model even better with guided reasoning
  • actor / checker / llm-as-judge workflows may work but are exponential in cost / latency

image

image

image

image

image

image

All episodes