Best practices for reducing hallucinated API calls in GitHub Copilot Workspace

GitHub Copilot Workspace often trips up when the codebase has custom internal libraries or when the API surface area is massive, leading it to "invent" methods that sound plausible but don't exist. After running a series of benchmarks across a few complex repos, I've found that the underlying model's tendency to extrapolate patterns based on naming conventions is the primary culprit. It sees getUserById() and assumes getUserByEmail() exists, even if your backend doesn't support it.

The most effective way to curb this is by treating your codebase as the only source of truth through aggressive context steering. If you leave the "Plan" phase too vague, the agent fills in the gaps with hallucinations.

Model-specific behavior observations
While Workspace uses a blend of models, the behavior mirrors the differences between GPT-4o and Claude 3.5 Sonnet. GPT-4o is more prone to "optimistic" hallucinations—assuming a utility function exists because it's logically convenient. Sonnet tends to be more conservative, but can get stuck in a loop if the provided context is contradictory. To minimize the "imaginary API" problem, you have to force the agent into a "verification" mindset.

The "Definition-First" Prompting Strategy
Instead of telling the agent to "Implement the feature using the User API," I've had much better luck explicitly referencing the file where the API is defined in the initial prompt. This anchors the model to actual tokens in the repo rather than its training weights.

Try structuring your plan request like this:

Use the methods explicitly defined in src/api/userClient.ts. 
Do not assume any helper methods exist outside of this file. 
If a required operation is missing, flag it in the plan instead of implementing a guessed method.

Measured Performance Gains
In a test run involving 15 feature tickets across a TypeScript monorepo, I compared "Naive Planning" (standard prompts) vs. "Anchored Planning" (referencing specific definition files):

Naive Planning

• Hallucinated API calls: 6/15 tickets
• Build failures due to missing methods: 40%
• Manual correction time: ~20 mins per ticket

Anchored Planning

• Hallucinated API calls: 1/15 tickets
• Build failures due to missing methods: 7%
• Manual correction time: ~5 mins per ticket

The "Skeleton" Technique
Another pro tip: if you're about to implement a complex flow, create a "skeleton" interface or a type definition file first. Even if the implementation is empty, having the interface defined in the codebase gives Copilot a hard constraint to follow. It’s much harder for the model to hallucinate a method call when the TypeScript compiler (which Copilot is aware of) would immediately flag it as an error against the defined interface.

Pros and Cons of the Workspace Workflow

• Pros: The ability to iterate from Plan -> Code -> Test in one environment is unmatched for speed.
• Cons: The "agentic" nature means it tries to be helpful by filling gaps, which is exactly where the hallucinations creep in.

Ultimately, reducing hallucinations in Workspace isn't about the model being "smarter," but about reducing the search space. The more you constrain the agent to specific files and explicit interfaces, the less it relies on probabilistic guessing.