Experiment · Jun 29, 2026
Here's an experiment we ran while tuning our routing rules. We took one mid-complexity prompt — 'plan a 3-day Tokyo itinerary for someone who hates crowds but loves food' — and ran it, unmodified, through five general-purpose chat tools.
The differences were bigger than we expected. One tool front-loaded a beautiful but generic list of famous districts, exactly the crowded places the prompt asked to avoid. Another took the constraint seriously and built the itinerary around early-morning visits and residential neighborhoods, which is genuinely how you dodge crowds in Tokyo. A third asked two clarifying questions before answering, which is either helpful or annoying depending on your mood.
Speed varied by a factor of four. Tone varied from tour-brochure to knowledgeable-friend. One answer included a restaurant that closed in 2023, a good reminder that every one of these tools can be confidently wrong.
The takeaway isn't that one tool is best. It's that the same prompt has a different center of gravity in each tool, shaped by how that tool was trained and tuned. The constraint-respecting answer came from a tool known for careful instruction-following. The list-heavy one came from a tool optimized for fast, broad coverage.
This is the entire reason routing matters. Prompt-engineering advice usually focuses on wording, and wording does matter. But in our testing, choosing the right tool moved answer quality more than any rewording of the prompt did. Fix the destination before you polish the envelope.