Local agent kept skipping required tool calls — fixed it with a

Been fighting this for weeks with a local-first assistant running Gemma. The model would cheerfully answer "I'll check that for you" then... nothing. No tool call, just hallucinated text. tool_choice="auto" sounds convenient until you realize it lets the model opt out of required actions.

The fix isn't a better prompt — it's a programmatic guardrail around the model's decision.

The pattern: two-pass with buffered recovery

# First pass: normal auto selection
response = client.chat.completions.create(
    model="gemma-3-27b",
    messages=messages,
    tools=local_tools,
    tool_choice="auto",
    stream=True
)

# Buffer the stream, check for tool_calls
buffered = []
tool_called = False
for chunk in response:
    buffered.append(chunk)
    if chunk.choices[0].delta.tool_calls:
        tool_called = True
        break

# Recovery: if user explicitly asked for local data but no tool fired
if needs_local_data(user_msg) and not tool_called:
    # Replay with forced tool choice — but ONLY before any tool executes
    recovery = client.chat.completions.create(
        model="gemma-3-27b",
        messages=messages,
        tools=local_tools,
        tool_choice="required",  # forces a call
        stream=False
    )
    # Validate the recovered call belongs to read/write group
    if is_valid_local_tool(recovery.choices[0].message.tool_calls[0]):
        yield from execute_tool_flow(recovery)
    else:
        yield from buffered  # fall back, log anomaly
else:
    yield from buffered

Key constraints that matter:

• Retry only once — prevents infinite loops on genuinely confused outputs
• Gate it behind needs_local_data() — don't force tools on chit-chat
• Validate group membership — a recovered write_note call when the user asked "what's in my notes?" is a bug, not a feature
• Buffer the first stream — user never sees the ungrounded "I'll check" hallucination

Tested against automated regressions + live Gemma. The flow traces clean:

auto -> required -> auto

Prompts describe intent. Reliable agents need enforcement at the orchestration layer. The model still decides which tool — the wrapper just guarantees it picks one when the contract demands it.