From this article you’ll know:

• what’s Azure Application Insights and how to use it in context of Copilot Studio,
• how to effectively query Log Workspace connected to Application Insights to monitor chatbots,
• how to setup a connection between chatbots and App Insights,
• other nice tips. 😁

Many times I’ve heard about a problem that it’s not possible to monitor multiple chatbots at once. If you’re just starting your journey with Copilot Studio – you probably don’t think about monitoring and troubleshooting, as it’s quite easy to just perform few clicks in Copilot Studio and check the analytics. But what if you’ll have more chatbots?

5? It’s manageable.

15? With more effort, but still manageable.

55? You’re not going to manage it. 😉

So what to do to monitor them all in one place? Let’s check out what Azure Application Insights can give us!

Azure, help!

First thing that you need to do is to create Azure Application Insights resource as it’s going to be the place, where all agents logs will land. I’m not going to explain it in detail as it’s a topic for a separate post. You can read more about how to create this resource here. After creation you should see something like this:

This is a view from the resource group. When you click on the violet bulb icon – it’s our app insights resource, you’ll see this screen:

You need to copy the connection string – it’ll be needed in Copilot Studio. What we’re going to use to check our chatbots is Logs tab. When we navigate to it, we’ll have this:

All our logs are going to land to customEvents table. From Azure perspective we have everything, let’s see what we need to do in Copilot Studio!

How to Copilot?

Do we have our Connection String from App Insights copied? If yes, we can navigate to Copilot Studio and to the chatbot settings here:

Then we just need to simply paste the string and save. We can also think about what logs we would like to send – thinking especially about Log sensitive activity properties. That’s something we need to decide on, depending on the solution requirements. After that we can engage in to a chat in our test window. Let’s see what happens then!

Logs love!

After few chats in the test window, we can navigate back to our Azure resource, to Logs tab. When we’re back, we can simply query the resource to check the logs. But wait! It’s not that simple, right? 🤔 Yes, so the thing is that for App Insights the special query language is used to query the dataset – it’s called KQL (Kusto Query Language). Don’t worry! It’s quite easy to grasp. What you could start is just a simple query to retrieve all records. In case of Copilot Studio logs, we need to query customEvents table. When we perform a query, we should see something like this:

In the logs you have almost everything – from the user data to what topic has been invoked, what message has been sent etc. With this resource you can do almost anything with this data:

• query for only certain bots,
• check for any errors,
• view how many conversations are for a given date range,
• debug any potential problems, especially with generative answers node

and many more! Just as an example: here’s a query where I’m checking only one specific chatbot:

customEvents
| where cloud_RoleInstance == "YOUR CHATBOT NAME"

But there’s even more. With App Insights you can define:

• alerts for each query, so that you’ll get informed when something wrong happens,
• dashboards to monitor all the chatbots in one place,

etc.

Regarding alerts: you can read more about then here. There’s a lot of options here to monitor anything, so keep in mind to not get lost. 😉

Summary

Did you like the monitoring possibilities for chatbots? Personally, I really like them, because you can easily get everything in one place, without much effort. I’m going to use them frequently with my deployments in order to monitor all the chatbots easily!

Liked the post? Tell me in the comments section. 😉

Artur Stepniak

Links:

About alerts: https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-create-metric-alert-rule

Create App Insights: https://learn.microsoft.com/en-us/azure/azure-monitor/app/create-workspace-resource?tabs=bicep

KQL reference: https://learn.microsoft.com/en-us/kusto/query/kql-quick-reference?view=microsoft-fabric

In this article you’ll know:

• how to properly configure a Question action,
• how to use Generative answers action as a fallback to the Question action.

Hello! I’ve recently found out that quite a lot of people would like to configure a Question action with Generative answers as a fallback. The idea to do it is quite easy, although you need to perform some changes in the system topics. Want to know more? Let’s check it out!

1. Configure a Question action

First thing that you need to perform is to configure properly the Question action. By default it repeats the question 2 times, until it’ll receive an answer which’s available on the choices list. We need to disable it, so that it won’t repeat by choosing “Don’t repeat” option in the question behavior:

2. Configure the Escalate topic

Now when you try to input a different option than the ones that are defined, it should immediately redirect your chat to Escalate topic, which’s used when the chatbot is unable to answer the question. The idea here is quite simple: you can create a generative answers action here, in Escalate topic, and it’ll always use it, when someone would enter an unexpected input. Here I’ve configured it in my topic:

I’ve used following input in the action:

"Try to help the user based on the message from the user: " & System.LastMessage.Text

But you can try anything that’ll fulfill your needs.

Summary

In this short article, we’ve learned on how to use Generative answers action as a fallback for a question. I hope you liked it and it’ll help you create the best chatbot ever. 😉

Cheers!

Artur Stepniak

Ever wondering how to reorder the choices in your Question action? Here I’m providing 1-minute answer to your problem! Check it out:

Usually, when we’re chatting with the bot in Copilot Studio, we’re receiving just plain text messages. Did you know that there’s a possibility to tweak them a bit by using an adaptive card? Check out below, where I’m going to describe what is it.

Adaptive card – what’s that?

An adaptive card is a form of sending messages that can, as the name suggests, adapt to your needs. It’s based on the framework prepared by Microsoft. The design is based on the JSON syntax with a proper schema definition. An example adaptive card JSON looks like this:

{
    "type": "AdaptiveCard",
    "body": [
        {
            "type": "TextBlock",
            "size": "Medium",
            "weight": "Bolder",
            "text": "${title}"
        },
        {
            "type": "ColumnSet",
            "columns": [
                {
                    "type": "Column",
                    "items": [
                        {
                            "type": "Image",
                            "style": "Person",
                            "url": "${creator.profileImage}",
                            "altText": "${creator.name}",
                            "size": "Small"
                        }
                    ],
                    "width": "auto"
                },
                {
                    "type": "Column",
                    "items": [
                        {
                            "type": "TextBlock",
                            "weight": "Bolder",
                            "text": "${creator.name}",
                            "wrap": true
                        },
                        {
                            "type": "TextBlock",
                            "spacing": "None",
                            "text": "Created {{DATE(${createdUtc},SHORT)}}",
                            "isSubtle": true,
                            "wrap": true
                        }
                    ],
                    "width": "stretch"
                }
            ]
        },
        {
            "type": "TextBlock",
            "text": "${description}",
            "wrap": true
        },
        {
            "type": "FactSet",
            "facts": [
                {
                    "$data": "${properties}",
                    "title": "${key}:",
                    "value": "${value}"
                }
            ]
        }
    ],
    "actions": [
        {
            "type": "Action.ShowCard",
            "title": "Set due date",
            "card": {
                "type": "AdaptiveCard",
                "body": [
                    {
                        "type": "Input.Date",
                        "id": "dueDate"
                    },
                    {
                        "type": "Input.Text",
                        "id": "comment",
                        "placeholder": "Add a comment",
                        "isMultiline": true
                    }
                ],
                "actions": [
                    {
                        "type": "Action.Submit",
                        "title": "OK"
                    }
                ],
                "$schema": "http://adaptivecards.io/schemas/adaptive-card.json"
            }
        },
        {
            "type": "Action.OpenUrl",
            "title": "View",
            "url": "${viewUrl}"
        }
    ],
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.3"
}

And it produces following card in the designer (see: https://adaptivecards.io/designer/):

Doesn’t it look nice? But…

…there are some limitations!

What you need to consider when creating an adaptive card in Copilot Studio is:

• make sure that you’re using a correct schema version:

When you enter the adaptive card designer, you should quickly notice a warning message. Don’t ignore it:

It states that selected host app (see the top of the screenshot) doesn’t support the schema version that was choosen in the upper right corner. Make sure to choose both the correct host app and the target version to avoid any errors.

• make sure that you’ve choosen a proper host app:

In the dropdown you have, I assume, two options to choose, which apply to Copilot Studio – Bot Framework WebChat and Microsoft Teams:

Option that you’d choose depends on your needs – for a chatbot used in MS Teams the choice is obvious. What you’d need to make sure is that card you’ve designed is supported by all the channels that you’re going to publish to. The best way to check it is to verify what controls are not available for an app here: https://learn.microsoft.com/en-us/adaptive-cards/resources/partners

Unfortunately, Copilot Studio is not mentioned on the Host App list, so sometimes you’d need to try if it works on the test chat, before publishing to production. 😉

Formatting issues

Recently, a lot of people were asking about how to format an adaptive card, so that it can take dynamic values. It’s a very easy task! Let me explain it step by step:

1. Open an adaptive card action’s properties:

2. Select the button Edit JSON and choose Edit formula – that way the adaptive card JSON is being converted to a Power Fx Record type. This enables you to put dynamic values:

3. Type any expression or place a variable as a value of the field that you’d like to edit:

Here I’ve used string interpolation ($”{EXPRESSION}”), which’s a nice feature in Power Fx to make the string look more elegant. 😎 But you could also format it like: Today(). Always make sure though that the result from an expression is of correct type that’s expected by the field.

Summary

Isn’t the feature nice? It gives a lot more freedom in how your messages would look, when chatting with a chatbot. Of course, for simple cases I’d stick to the simple Question action, but if you need to prepare something more fancy, then an adaptive card is definitely a way to go!

Happy designing!

Artur Stepniak

Links:

Schema explorer: https://adaptivecards.io/explorer/

Schema designer: https://adaptivecards.io/designer/

MS Learn resources: https://learn.microsoft.com/en-us/adaptive-cards/

Host app docs: https://learn.microsoft.com/en-us/adaptive-cards/resources/partners

Nice tutorial on adaptive cards: 

There was a time in my career when I’ve worked extensively in Excel. Being an Excel geek is a hard job in my opinion – especially when your colleagues know about it. 😉 Many times, I was swarmed with questions like:

• Artur, how to parse the date values in format X?
• Artur, this formula is not working. Please, please, help me!
• Artur, you’re so good in Excel, please tell me how to reformat this text.

etc.

Now, when M365 Copilot was announced, I was really eager to test it out to see what’s possible. What I’ve found out? Check below!

Forget about formulas

Did you forget about how to use certain formula? Or maybe you’ve had a time, when you’ve wanted to do something in Excel, like performing a LOOKUP(), but man, you’ve been doing it so long ago and you don’t remember how to do it NOW. Fear no more with Copilot in Excel onboard 🫡:

Many times you don’t even need to do anything as Copilot can perform the task for you. Just click the Apply button:

Of course, it’s always good to understand how it was performed – just in case Copilot wouldn’t work sometimes. 😉 The tool shouldn’t make us lazy, but curious! After applying the changes, you can see a nice table with formatted dates:

Inconsistency reduces efficiency

How often did you see an Excel file with something like:

• duplicate values;
• the same values, but with a different formatting, e.g. Product A vs. Product a;
• typos;

etc.

Do you know that Copilot can find these errors and help you clean the data? Here I’ve instructed it to help me find any missing data:

For my little table it’s just a matter of one look. But imagine that you have a sheet with 100000 records. Isn’t it nice to simply prompt the Copilot instead of scrolling the workbook up and down? 😉

Of course, it’s not without flaws and sometimes it doesn’t understand what we’d like to achieve:

Nevertheless, it’s a matter of time and patience to get the answer that we’d like to have (see Prompt engineering guide for help).

House of charts

Many times I’ve wasted a lot of time, when I’ve wanted to generate a nice chart in Excel. With Copilot it’s really easy to prepare a simple chart:

You can then customize it the way you want. And there’s always an option to ask Copilot on how to do something, where to find the setting etc. You can even ask questions about the data that was analyzed:

Of course, you could say: “Really? I can easily use a formula like =MAX([Sales Amount]) and I have it!”. But if you have a big file with many rows and you’re not really experienced, e.g. you’re not using shortcuts like CTRL + UP/DOWN, would it be an easy task? Definitely not! So it’ll help a lot of people who are not tech-savvy and will make their work more pleasant. 😉

Summary

I’m really amazed on how M365 Copilot in Excel is being developed and can’t wait for more updates on it. It’ll definitely be the first tool to find the information related to Excel, before searching in Google. ⭐

What do you think about the tool? Share your thoughts in the comments section!

Artur Stepniak

Links:

https://support.microsoft.com/en-us/copilot-excel

Whenever you’ve searched for something in the documentation or just wanted to learn about various Microsoft products (especially related to Copilot stuff), you’ve surely stumbled upon something like this:

What does this preview thing mean? Why is it mentioned there? How to interpret this? Let’s check out!

Number 1: preview features have additional Terms of Use

It means that when you’re using these features, you should keep in mind that there are some additional rules that apply, apart from the ones that are applied for Generally Available (GA) features. Those features could have reduced or different security, no privacy protection implemented yet etc. You can read more about the supplemental terms of use for Microsoft Power Platform preview features here:

https://www.microsoft.com/en-us/business-applications/legal/supp-powerplatform-preview

Number 2: preview features are not meant for production use

In other words: there’s a quite high risk that if you’re going to use those features in a solution deployed to production environment, it could raise problems. It’s not recommended to use them for anything apart from testing.

Number 3: preview features aren’t supported by Microsoft Support for production use

You’re not going to receive any support from Microsoft if you’re using preview features on production. However, Microsoft is eager to get your feedback on the preview functionality. If you want to test and send the feedback, then you’re good to go! 😉 Otherwise: be careful!

Number 4: preview features may have limited or restricted functionality

If you’ve stumbled upon a problem that something is not working as expected and it’s a preview feature, then you should understand that it’s natural. You need to be patient and wait for the feature to be Generally Available (GA). Microsoft often restricts that behaviour of the preview features could be changed over time, development of the feature could be discontinued etc. To put it bluntly: it’s not stable yet.

Number 5: preview features may not be available in all geographical areas

Your colleague from another company in Europe has the feature, but you don’t? No worries! Many preview features are deployed only in certain geographical areas before they become available for everyone. You should keep that in mind before thinking that it’s a sort of bug. 😊

Number 6: preview features are not completed yet

Would you like to do your laundry in an incomplete washing machine? Or drive a car with everything except the front window? Surely not! If you’re using a preview feature, keep in mind that it’s in development yet, so you could get a really nice surprise after deploying to production… 😉

Summary

So what to do with the preview features? Should we use them or not? In general, it’s good to get familiar with them in order to keep yourself in the loop, but for creating a serious solution I’d recommend to avoid them until they become generally available. You can always check when features are going to be production-ready on the release plan here:

https://releaseplans.microsoft.com/en-US/?app=Microsoft+Copilot+Studio

What do you think about preview features? Is it good that they’re available? Share thoughts in the comments section!

Artur Stepniak

Links:

https://learn.microsoft.com/en-us/dynamics365/sales/sales-previews-in-trial

https://www.microsoft.com/en-us/microsoft-365/copilot/enterprise#Pricing

1. Using a classic approach so utilizing trigger phrases to invoke topics, adding actions to the topic and using entities to extract information from the user etc.
2. Using generative orchestration – the model is responsible for choosing what topic to trigger, what action to invoke, etc.

What are pros and cons of each one and how to decide which one to use? Let’s see!

Classic approach

In the classic approach in chatbot settings we need to choose the Classic option:

When applied, our chatbot will use the classic way of triggering topics. For each topic we need to define so-called trigger phrases – words which will invoke the topic, when the user writes about something:

Just as an example – we have defined a topic with trigger phrases: pizza, buy.

User says: I’d like to buy a pizza.

Bot invokes the topic and performs all the actions contained in it.

The NLP model behind the chat is also able to find synonyms. In the example below I’m using a synonym for the word buy – purchase:

As you can see it’s still able to invoke the topic, even though I’m not including the word in my message. Actions like invoking a Power Automate flow or connector action need to be added to the topic flow – otherwise they won’t be invoked.

What are pros and cons of this approach?

Generative orchestration

In the generative orchestration approach GPT model behind decides which topic or action to invoke and how to respond to the user. In the chatbot settings, we need to choose the Generative (preview):

Switching the settings changes a lot of things in chatbot design:

1. Invoking a topic is based on its description, not on trigger phrases.
2. You can define an action for the chatbot, such as get items from a Sharepoint list, for example, and inputs/outputs of this action can be filled by the model automatically.
3. When testing the chatbot, it shows a conversation map, which contains information about what decisions the model has taken to answer the user.

The conversation with the chatbot starts to be more natural and straightforward. With the help of AI, you can prepare the inputs of an action without the need to fill them manually – by combining Instructions with relevant input parameters descriptions the model can easily invoke an action and return appropriate response. Here you can see an example of a chatbot, which works on a Sharepoint list as its memory:

The site address and list name were provided manually. It was able to define the ODATA filter query by itself, it only needed a bit of tweaking in the Instructions like:

• you should only use the columns that are available to fill for your memory,

• you should use substringof() for the filter,

etc.

That way you can design a chatbot which can communicate with various services and respond to the user with the data that it retrieved. The conversation is also more straightforward and it feels more natural to the user.

What are the pros and cons for this approach?

Summary

Which way is better to prepare a chatbot? The answer depends on particular business case and requirements. Generative orchestration definitely has brighter future as it fully utilizes the GPT model’s capabilities, but it’s not always the best approach for every solution. If the outcome of a conversation needs to be always stable and predictable, without any exceptions, then it’s better to use the classic approach. However if you are able to accept some unpredictability in the responses and you would like to fully use the GPT model’s capabilities, then generative orchestration is the way to go. What we also need to consider is, as of today, generative orchestration is still a preview feature (check out what preview features are here), so it’s not recommended by Microsoft for production workloads. To sum it up, I’ve prepared a table to illustrate the differences between each approach:

Did you like the article? Feel free to share your thoughts in the comments section!

Artur Stepniak

Useful links:

https://learn.microsoft.com/en-us/microsoft-copilot-studio/advanced-generative-actions

https://learn.microsoft.com/en-us/microsoft-copilot-studio/nlu-boost-node

https://learn.microsoft.com/en-us/microsoft-copilot-studio/advanced-entities-slot-filling