Simple Tool Calling Agent

The Problem: Most "Tool Calling" Isn't Real

Many AI agent tutorials create the illusion of tool calling without actual function execution. Common issues include:

• Using models that lack function calling capabilities
• Generating text descriptions instead of structured tool calls
• Skipping critical tool binding steps
• Missing proper input validation with Pydantic schemas
• Inadequate docstrings that fail to guide LLM behavior
• Using outdated LangGraph APIs that AI coding assistants generate

Technical Implementation

Hybrid Architecture: Single Model, Dual Roles

The system uses a single LLM instance (Hermes-3-Llama-3.1-8B) running locally via vLLM, serving two distinct purposes:

• Agent Mode (llm_with_tools): LLM with tools bound, generates structured tool calls
• Evaluator Mode (llm): Raw LLM without tools, performs semantic assessment

This hybrid approach runs on a 24GB GPU by avoiding the memory overhead of loading two separate models. The same underlying model serves both functions - one configuration with tool schemas, one without.

Why Model Selection is Critical

Not all LLMs support structured function calling. General instruction models will generate text descriptions like "I would call the weather tool for Boston" instead of making actual function calls. This implementation uses Hermes-3-Llama-3.1-8B, which is specifically trained for function calling.

Function-calling capable models: Hermes-3-Llama-3.1-8B, GPT-4, Claude 3+, Mistral-Large

Lacks function calling: Mistral-7B-Instruct, Base Llama models, most general chat models

Pydantic Schemas for Input Validation

Each tool uses Pydantic models to define expected input schemas, ensuring the LLM generates valid, structured tool calls:

class WeatherInput(BaseModel):
    location: str = Field(
        description="The city name and optionally state/country (e.g., 'San Francisco, CA')",
        min_length=2,
        max_length=100
    )

@tool(args_schema=WeatherInput)
def get_weather(location: str) -> str:
    """
    Retrieves current weather information for a specified location.
    
    Args:
        location: The city name and optionally state/country
    
    Returns:
        A string containing the current temperature and weather conditions.
    """
    return f"Current weather in {location}: Temperature is 72°F, conditions are sunny"
								

The Critical Tool Binding Step

Simply defining tools is insufficient - they must be explicitly bound to the LLM. This step is commonly skipped by AI coding assistants:

# Base LLM without tools (for evaluation)
llm = ChatOpenAI(
    base_url="http://localhost:8082/v1",
    model="NousResearch/Hermes-3-Llama-3.1-8B"
)

# Bind tools to create agent LLM
tools = [get_weather, calculator]
llm_with_tools = llm.bind_tools(tools)
								

Without binding, the LLM has no knowledge of available tools and cannot generate structured function calls.

LangGraph State Management

The agent uses LangGraph to manage conversation flow with conditional routing:

graph_builder = StateGraph(MessagesState)
graph_builder.add_node("agent", call_model)
graph_builder.add_node("tools", ToolNode(tools))
graph_builder.add_conditional_edges(
    "agent",
    lambda x: "tools" if x["messages"][-1].tool_calls else END
)
graph_builder.add_edge("tools", "agent")
graph_builder.add_edge(START, "agent")
graph = graph_builder.compile()
								

Important: This uses the current LangGraph API with START and END keywords. Many AI coding assistants generate outdated code using set_entry_point() and "__end__".

Semantic Evaluation Without Keyword Matching

Rather than brittle keyword matching, the system uses LLM-based evaluation to assess:

• Tool Selection: Did the agent choose appropriate tools for the query?
• Response Quality: Is the final answer clear, complete, and well-formatted?
• Overall Success: Does the response address the user's question?

This approach handles semantic variations ("multiply" vs "multiplied" vs "times") and provides reasoned assessment with explanations.

Technologies Used

• Python 3.11+
• LangChain
• LangGraph
• vLLM
• Pydantic
• Hermes-3-Llama-3.1-8B