Tool Use And Function Calling With Anthropic Claude 3 API

Hello Learners…

Welcome to the blog…

Table Of Contents

• Introduction
• Tool Use And Function Calling With Anthropic Claude 3 API
• How tool use works in Anthropic Claude 3 API
• bind_tools() in Anthropic Claude 3 API
• Parsing tool calls
• Summary
• References

Introduction

In this blog post, we discuss about Tool Use And Function Calling With Anthropic Claude 3 API.

From defining the tools and user prompts to extracting inputs, running code, and formulating responses, Claude’s journey exemplifies the fusion of AI and external resources to deliver comprehensive solutions.

Tool Use And Function Calling With Anthropic Claude 3 API

Anthropic just added tool use to their Claude 3 APIs, which is extremely useful for structured outputs.

Here’s an example of how to provide tools to Claude using the Messages API:

import anthropic

client = anthropic.Anthropic()

response = client.beta.tools.messages.create(
    model="claude-3-opus-20240229",
    max_tokens=1024,
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather in a given location",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    }
                },
                "required": ["location"],
            },
        }
    ],
    messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
)
print(response)

Please note that during the beta period:

• Streaming (stream=true) is not yet supported. We plan to add streaming support in a future beta version.
• While the feature is production-ready, we may introduce multiple beta versions before the final release.
• Tool use is not yet available on third-party platforms like Vertex AI or AWS Bedrock, but is coming soon.

How tool use works in Anthropic Claude 3 API

Using tools with Claude involves the following steps:

1. Provide Claude with tools and a user prompt: (API request)
   • Define the set of tools we want Claude to have access to, including their names, descriptions, and input schemas.
   • Provide a user prompt that may require the use of one or more of these tools to answer, such as “What is the weather in San Francisco?”.
2. Claude uses a tool: (API response)
   • Claude assesses the user prompt and decides whether any of the available tools would help with the user’s query or task. If so, it also decides which tool(s) to use and with what inputs.
   • Claude constructs a properly formatted tool use request.
   • The API response will have a stop_reason of tool_use, indicating that Claude wants to use an external tool.
3. Extract tool input, run code, and return results: (API request)
   • On the client side, we should extract the tool name and input from Claude’s tool use request.
   • Run the actual tool code on the client side.
   • Return the results to Claude by continuing the conversation with a new user message containing a tool_result content block.
4. Claude uses tool result to formulate a response: (API response)
   • After receiving the tool results, Claude will use that information to formulate its final response to the original user prompt.

Steps (3) and (4) are optional — for some workflows, Claude using the tool is all the information we need, and we might not need to return tool results back to Claude.

With Anthropic’s tool-calling, or tool-use, API, we can define tools for the model to invoke. This is extremely useful for building tool-using chains and agents, as well as for getting structured outputs from a model.

NOTE:

Anthropic’s tool-calling functionality is still in beta.

bind_tools() in Anthropic Claude 3 API

With ChatAnthropic.bind_tools, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an Anthropic tool schemas, which looks like:

{
    "name": "...",
    "description": "...",
    "input_schema": {...}  # JSONSchema
}

and passed in every model invocation.

from langchain_core.pydantic_v1 import BaseModel, Field

llm = ChatAnthropic(
    model="claude-3-opus-20240229",
)


class GetWeather(BaseModel):
    """Get the current weather in a given location"""

    location: str = Field(..., description="The city and state, e.g. San Francisco, CA")


llm_with_tools = llm.bind_tools([GetWeather])

ai_msg = llm_with_tools.invoke(
    "what is the weather like in San Francisco",
)
ai_msg

Output of above code snippet,

AIMessage(content=[{'text': '<thinking>\nBased on the user\'s question, the relevant function to call is GetWeather, which requires the "location" parameter.\n\nThe user has directly specified the location as "San Francisco". Since San Francisco is a well known city, I can reasonably infer they mean San Francisco, CA without needing the state specified.\n\nAll the required parameters are provided, so I can proceed with the API call.\n</thinking>', 'type': 'text'}, {'text': None, 'type': 'tool_use', 'id': 'toolu_01SCgExKzQ7eqSkMHfygvYuu', 'name': 'GetWeather', 'input': {'location': 'San Francisco, CA'}}], response_metadata={'id': 'msg_01GM3zQtoFv8jGQMW7abLnhi', 'model': 'claude-3-opus-20240229', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 487, 'output_tokens': 145}}, id='run-87b1331e-9251-4a68-acef-f0a018b639cc-0')

Notice that the output message content is a list that contains a text block and then a tool_use block:

ai_msg.content[0]

Output:

{'text': '<thinking>\nBased on the user\'s question, the relevant function to call is GetWeather, which requires the "location" parameter.\n\nThe user has directly specified the location as "San Francisco". Since San Francisco is a well known city, I can reasonably infer they mean San Francisco, CA without needing the state specified.\n\nAll the required parameters are provided, so I can proceed with the API call.\n</thinking>',
 'type': 'text'}

ai_msg.content[1]

Output:

{'text': None,
 'type': 'tool_use',
 'id': 'toolu_01SCgExKzQ7eqSkMHfygvYuu',
 'name': 'GetWeather',
 'input': {'location': 'San Francisco, CA'}}

Parsing tool calls

The langchain_anthropic.output_parsers.ToolsOutputParser makes it easy to extract just the tool calls from an Anthropic AI message:

from langchain_anthropic.output_parsers import ToolsOutputParser

parser = ToolsOutputParser()
chain = llm_with_tools | parser
chain.invoke("What is the weather like in nyc, la, sf and cleveland")

Output:

[{'name': 'GetWeather',
  'args': {'location': 'New York City, NY'},
  'id': 'toolu_01UK2AEWa75PUGA3DpiaHfBN',
  'index': 1},
 {'name': 'GetWeather',
  'args': {'location': 'Los Angeles, CA'},
  'id': 'toolu_01M84DY7xWg9bLoX6JCArczx',
  'index': 2},
 {'name': 'GetWeather',
  'args': {'location': 'San Francisco, CA'},
  'id': 'toolu_01FEasmxGpxFPwf9SF3nCTeb',
  'index': 3},
 {'name': 'GetWeather',
  'args': {'location': 'Cleveland, OH'},
  'id': 'toolu_01B8fZdiyPbzWyj5cDCzGSTe',
  'index': 4}]

Summary

As Claude navigates through the user’s query, he dynamically assesses the applicability of available tools, constructs requests, and seamlessly executes them. The exchange continues as Claude extracts tool inputs, executes code, and returns results to formulate a final response, epitomizing the synergy between AI capabilities and external resources in problem-solving scenarios.

Also you can learn more about other LLMs,

• https://galaxyofai.com/tag/llms/

References

• https://docs.anthropic.com/claude/docs/tool-use
• https://python.langchain.com/docs/integrations/chat/anthropic/