Topic 5: Tool design

Granularity and the cost of too many tools

Granularity is how much each tool does. A thin server exposes one tool per operation of the underlying code (get_note, get_note_tags, add_tag...). A task-shaped server exposes one tool per thing a user wants done (search_notes, create_note). Every tool definition is sent to the model on every turn of the conversation, so granularity has a direct cost in context, and more choices also mean more chances to pick the wrong one (Part B).

This script measures the definitions as a host would send them to the model: the prototype's 2 tools against a 12-tool thin wrapper over the same store.