The Hitchhiker's Guide to Context Grounding Your Copilot

Don't panic. And don't forget your towel.

In The Hitchhiker's Guide to the Galaxy, a race of hyper-intelligent beings builds a supercomputer called Deep Thought to calculate the answer to the Ultimate Question of Life, the Universe and Everything. Deep Thought asks for a little time to think about it (7.5 million years, give or take) and eventually announces the answer: 42.

Nobody is impressed. And Deep Thought's response is one of my favourite lines about technology ever written: "I think the problem, to be quite honest with you, is that you've never actually known what the question is."

That, in a nutshell, is the problem with a lot of the copilots and agents being built today. We are very excited about the answers, and not nearly careful enough about the question or the context around it. I presented this talk, Hitchhikers Guide to Context Grounding Your Copilot, at AgentCon 2025 in Johannesburg, and again with Carike Botha at the Data & AI Community Day in Durban (Coastlands Umhlanga, 24 May 2025). This post is the written version, minus the Vogons.

What is context grounding, anyway?

A large language model knows a lot of things in general and nothing about your organisation in particular. Context grounding is the practice of anchoring your copilot's answers in a specific, trusted body of knowledge, so that when someone asks a question, the agent answers from your data and within your boundaries, instead of from whatever it vaguely remembers about the internet.

Here's the architecture I used to explain it:

Users bring the prompts, the data brings the context, and the agent brings them together.

Walking through it:

• Users send prompts and questions to the agent.
• The data source supplies the context and the answer(s). This is the "grounding" part. The agent isn't guessing, it's retrieving.
• The agent responds, and the user validates whether that response is actually useful. That validation loop matters more than people think.
• Actions and schedules are where things get interesting. A chatbot answers questions. An agent can do something with the answer, such as triggering a flow, running on a schedule or kicking off a process.
• Metrics come from both sides: performance-related metrics from the actions being executed, and data-related metrics from the knowledge source itself.

That dotted line between "answering" and "acting" is the difference between a chatbot and an agent. Grounding is what makes it safe to cross it. If your agent is going to take actions on your behalf, you really want it to be working from the right information.

Grounded vs ungrounded: the Google search test

To make this tangible, I recorded two very short Google searches. No fancy AI tooling, just a search box.

In the first one, I typed: "what are the characteristics of a strong relationship". Google, very reasonably, gave me relationship advice. Honesty, trust, respect, open communication, giving your spouse time and space. Lovely. Completely correct. Entirely useless if what I actually wanted to know about was databases.

Ask a question without context and you'll get an answer. Just maybe not to the question you meant.

In the second recording, I typed exactly the same question and added three words at the end: "… in databases". Suddenly the results were all about strong (identifying) relationships, where the primary key of the related entity contains the primary key of the parent entity. Same question, completely different (and now correct) answer.

Three extra words of context, and we're in the right universe.

That's context grounding in its simplest form. The model wasn't wrong in the first search. It just didn't know which universe we were in. Your copilot has the same problem, except it will confidently answer with your HR policy when someone asked about your IT policy, unless you tell it where to look.

Limiting the knowledge: creating healthy boundaries

Once you accept that context matters, the next question is how much context to give your agent. I like to think of it as three options:

Healthy boundaries apply to agents too.

1. Limit the data. Restrict which data sources and data sets the agent can draw from. If it's an IT helpdesk agent, it doesn't need the finance share.
2. Limit the access. Control access to data sources, data sets and, ideally, a curated subset of data that has been cleaned up for the purpose.
3. "The Unlimited". Go forth and just prosper. Anything goes! The disclaimer on my slide said it best: if you don't ground your data, you may need to speak to a legal team sooner than you think.

In practice, the first two work together. Narrow the knowledge so the answers are relevant, and respect access so the answers are appropriate. Platforms like Copilot Studio help here because they respect the permissions on knowledge sources like SharePoint and OneDrive, so users only get answers from content they're already allowed to see. But the platform can't decide which content belongs in scope. That part is on you.

Copilot Studio supports public websites, SharePoint/OneDrive, Dataverse, file uploads and Microsoft Graph as knowledge sources.

Where does a grounded copilot actually help?

"But WHY (and how)?" is usually the next question in the room. The honest answer is that a well-grounded agent is useful almost anywhere there is a body of knowledge people repeatedly need to query or act on.

A non-exhaustive list. Pick one that annoys you the most and start there.

A few that stood out:

• Productivity: email summarisation, task management, calendar and availability assistants.
• Learning: virtual tutors, course companions and learning path planners (yes, the lecturer in me got excited about this one).
• IT development: code assistance and review, infrastructure management, data analysis and reporting.
• Customer support: sentiment analysis, proactive support and product training.
• Business improvement: document drafting and review, budgeting assistance and an HR query agent that knows your actual leave policy.
• Creative industries, healthcare and sustainability: content generation, telemedicine admin support, energy savings monitoring and carbon footprint tracking.

Notice that every single one of these depends on the agent knowing your context. A generic "HR query agent" that answers from the internet is worse than no agent at all.

The demo: a thing that does stuff (a lot of stuff)

The live demo in both sessions was an agentic service desk built on the Power Platform: a Power Apps service desk solution with an L1 support agent built in Copilot Studio. The idea was to show the full picture from the architecture diagram above: a user asks a question, the agent grounds its answer in the service desk knowledge, and then it actually takes action instead of just chatting about it.

The Durban session also covered the broader agent lifecycle in Copilot Studio (create, ground with knowledge, add topics, actions, channels, and monitor) and autonomous agents that react to triggers like an incoming email. I've written about getting an agent from Studio to production separately, so I won't repeat it all here.

Practical takeaways

If you remember nothing else, remember these:

• Know your question before you chase the answer. Define what the agent is for before you connect it to anything.
• Ground it in a specific, trusted source. Fewer, better knowledge sources beat "connect everything and hope".
• Respect access. Use platforms that honour existing permissions, and curate what goes in scope.
• Keep a human in the validation loop, especially once the agent starts taking actions.
• Measure both sides: how the agent performs and how good the underlying data is.

Conclusion

Deep Thought spent 7.5 million years on an answer nobody could use, because nobody had defined the question or the context. Your copilot doesn't have to make the same mistake. Give it the right context, set healthy boundaries, and it will give you answers that are actually worth waiting for. Hopefully in a lot less than 7.5 million years.

So long, and thanks for all the prompts. 🐼