From this article you’ll know:
- what’s Azure Application Insights and how to use it in context of Copilot Studio,
- how to effectively query Log Workspace connected to Application Insights to monitor chatbots,
- how to setup a connection between chatbots and App Insights,
- other nice tips. 😁
Many times I’ve heard about a problem that it’s not possible to monitor multiple chatbots at once. If you’re just starting your journey with Copilot Studio – you probably don’t think about monitoring and troubleshooting, as it’s quite easy to just perform few clicks in Copilot Studio and check the analytics. But what if you’ll have more chatbots?
5? It’s manageable.
15? With more effort, but still manageable.
55? You’re not going to manage it. 😉
So what to do to monitor them all in one place? Let’s check out what Azure Application Insights can give us!
Azure, help!
First thing that you need to do is to create Azure Application Insights resource as it’s going to be the place, where all agents logs will land. I’m not going to explain it in detail as it’s a topic for a separate post. You can read more about how to create this resource here. After creation you should see something like this:
This is a view from the resource group. When you click on the violet bulb icon – it’s our app insights resource, you’ll see this screen:
You need to copy the connection string – it’ll be needed in Copilot Studio. What we’re going to use to check our chatbots is Logs tab. When we navigate to it, we’ll have this:
All our logs are going to land to customEvents table. From Azure perspective we have everything, let’s see what we need to do in Copilot Studio!
How to Copilot?
Do we have our Connection String from App Insights copied? If yes, we can navigate to Copilot Studio and to the chatbot settings here:
Then we just need to simply paste the string and save. We can also think about what logs we would like to send – thinking especially about Log sensitive activity properties. That’s something we need to decide on, depending on the solution requirements. After that we can engage in to a chat in our test window. Let’s see what happens then!
Logs love!
After few chats in the test window, we can navigate back to our Azure resource, to Logs tab. When we’re back, we can simply query the resource to check the logs. But wait! It’s not that simple, right? 🤔 Yes, so the thing is that for App Insights the special query language is used to query the dataset – it’s called KQL (Kusto Query Language). Don’t worry! It’s quite easy to grasp. What you could start is just a simple query to retrieve all records. In case of Copilot Studio logs, we need to query customEvents table. When we perform a query, we should see something like this:
In the logs you have almost everything – from the user data to what topic has been invoked, what message has been sent etc. With this resource you can do almost anything with this data:
- query for only certain bots,
- check for any errors,
- view how many conversations are for a given date range,
- debug any potential problems, especially with generative answers node
and many more! Just as an example: here’s a query where I’m checking only one specific chatbot:
customEvents
| where cloud_RoleInstance == "YOUR CHATBOT NAME"But there’s even more. With App Insights you can define:
- alerts for each query, so that you’ll get informed when something wrong happens,
- dashboards to monitor all the chatbots in one place,
etc.
Regarding alerts: you can read more about then here. There’s a lot of options here to monitor anything, so keep in mind to not get lost. 😉
Summary
Did you like the monitoring possibilities for chatbots? Personally, I really like them, because you can easily get everything in one place, without much effort. I’m going to use them frequently with my deployments in order to monitor all the chatbots easily!
Liked the post? Tell me in the comments section. 😉
Artur Stepniak
Links:
About alerts: https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-create-metric-alert-rule
Create App Insights: https://learn.microsoft.com/en-us/azure/azure-monitor/app/create-workspace-resource?tabs=bicep
KQL reference: https://learn.microsoft.com/en-us/kusto/query/kql-quick-reference?view=microsoft-fabric