Build a custom engine agent using Teams AI library and Teams Toolkit

Άρθρο
Developer (Advanced)
6 minutes to complete

You can create a custom engine agent for Microsoft Teams to gain precise control over light operations. This showcases the effectiveness of the language model in accurately understanding and executing user intent, aligning it with app actions.

In this tutorial, you'll learn:

How to set up a new project with Teams Toolkit.
How to import Teams AI library capabilities.
How to deploy your custom engine agent in Teams and control over light operations.

You can see the following output:

Screenshot shows an example of the chat window in the local machine.

Prerequisites

5 minutes remaining

Here's a list of tools you need for building and deploying your custom engine agent.

	Install	For using...
	Teams Toolkit	Microsoft Visual Studio Code extension that creates a project scaffolding for your app. Use the latest version.
	Microsoft Teams	To collaborate with everyone, you work with apps for chat, meetings, and call all in one place.
	Node.js	Back-end JavaScript runtime environment. For more information, see Node.js version compatibility table for project type.
	Microsoft Edge (recommended) or Google Chrome	A browser with developer tools.
	Visual Studio Code	JavaScript, TypeScript, and Python build environments. Use the latest version.
	OpenAI or Azure OpenAI	First create your OpenAI API key to use OpenAI's GPT. If you want to host your app or access resources in Microsoft Azure, you must create an Azure OpenAI service before you begin.
	Microsoft 365 developer account	Access to Teams account with the appropriate permissions to install an app.

Prepare development environment

After you've installed the required tools, set up the development environment.

Install the Teams Toolkit

Teams Toolkit helps simplify the development process with tools to provision and deploy cloud resources for your app, publish to the Teams Store, and more.

You can use the toolkit with Visual Studio Code. You can also use CLI (command-line interface), called teamsapp.

Visual Studio Code
Command line

Open Visual Studio Code and select the Extensions view (Ctrl+Shift+X / ⌘⇧-X or View > Extensions).
In the search box, enter Teams Toolkit.
Select Install next to the Teams Toolkit.

The Teams Toolkit icon appears in the Visual Studio Code Activity Bar after it's installed.

You can also find the Teams Toolkit on the Visual Studio Code Marketplace.

To install the Teamsapp CLI, use the npm package manager:

npm install -g @microsoft/teamsapp-cli

Depending on your configuration, you might need to use sudo to install the CLI:

sudo npm install -g --unsafe-perm @microsoft/teamsapp-cli

This condition is more common on Linux and macOS systems.

Ensure you add the npm global cache to your path. This step is normally done as part of the Node.js installer.

You can use the CLI with the teamsapp command. Verify that the command is working by running teamsapp -h.

Caution

Before you can run Teamsapp in PowerShell terminals, you must enable the 'remote signed' execution policy for PowerShell.

Set up your Teams development tenant

A tenant is like a space, or a container for your organization in Teams, where you chat, share files, and run meetings. This space is also where you upload and test your custom app. Let's verify if you're ready to develop with the tenant.

Check for custom app upload option

After creating the app, you must load your app in Teams without distributing it. This process is known as custom app upload. Sign in to your Microsoft 365 account to view this option.

Note

Custom app upload is necessary for previewing and testing apps in Teams local environment. If it isn't enabled, you can't preview and test your app in Teams local environment.

Do you already have a tenant, and do you have the admin access? Let's check if you really do!

Verify if you can upload a custom app in Teams:

In the Teams client, select the Apps icon.
Select Manage your apps.
Select Upload an app.
Look for the option to Upload a custom app. If you see the option, custom app upload is enabled.

Note

Contact your Teams administrator, if you don't find the option to upload a custom app.

Create a free Teams developer tenant (optional)

If you don't have a Teams developer account, you can get it free. Join the Microsoft 365 developer program!

Go to the Microsoft 365 developer program.
Select Join Now and follow the onscreen instructions.
In the welcome screen, select Set up E5 subscription.
Set up your administrator account. After you finish, the following screen appears.
Sign in to Teams using the administrator account you just set up. Verify that you have the Upload a custom app option in Teams.

Get a free Azure account

If you want to host your app or access resources in Azure, you must have an Azure subscription. Create a free account before you begin.

Now you’ve got all the tools to set up your account. The development environment is ready to start building the app project.

Create project workspace for your custom engine agent app

4 minutes remaining

Let's create your first custom engine agent app.

The custom engine agent capability of a Teams app creates an AI chatbot. You use it to run simple and automated tasks, for example, providing customer service. A custom engine agent talks with a web service and helps you use its offerings. You can get weather forecast, make reservations, or any other service offered using the custom engine agent.

As you've already prepared for creating the app or the custom engine agent, you can set up a new Teams project for creating the custom engine agent.

In this tutorial, you'll learn:

How to set up a new custom engine agent project with Teams Toolkit.
About the directory structure of your app project.

Create your custom engine agent project workspace

If the prerequisites are in place, let's begin!

Visual Studio Code
Command line

Open Visual Studio Code.
Select the Teams Toolkit icon in the Visual Studio Code Activity Bar.
Select Create a New App.
Select Custom Engine Agent.
Select Basic AI chatbot.
Select TypeScript as the programming language.
Select OpenAI or Azure OpenAI.
Enter OpenAI Key or Azure OpenAI key and Azure OpenAI Endpoint.
Select Default folder to store your project root folder in default location.

You can also change the default location by the following steps:
1. Select Browse.
2. Select the location for project workspace.
3. Select the Select Folder.
Enter a suitable name for your app and then select Enter.

A dialog appears. Select Yes, I trust the authors or No, I don’t trust the authors based on your requirement.

Your custom engine agent is created in a few seconds.

After your app is created, the Teams Toolkit displays the following message:

A quick recap of creating a Teams app.

Watch this short recap for creating a Teams app.

Create a Teams app

Use the teamsapp CLI to create your first project. Start from the folder where you want to create the project folder.

teamsapp new

You can use the CLI to create a new Teams app. The CLI leads you through a series of questions. Every question includes an instruction on answering it.

For example, use arrow keys to select an option. After you make the choice, select Enter to confirm it.

Select Create a new Teams app.
Select Bot and deselect Tab.
Select JavaScript as the programming language.
Select Enter to select the default workspace folder.
Enter a suitable name for your app, like HelloBot. The name of the app must consist only of alphanumeric characters.

After you've answered all questions, your project is created.

Take a tour of the app source code

After scaffolding is done, view the project directories and files in the Explorer area of Visual Studio Code.

Screenshot shows the Teams Toolkit sample bot folder Structure.

Folder name	Contents
`env/.env.local.user`	Configuration file for local environment used by `teamsapp.yml` to customize the provisioning and deployment rules.
`index.ts`	Main entry point for the app.
`teamsBot.ts`	Teams activity handler.
`appPackage`	App manifest template files and app icons (color.png and outline.png).
`appPackage/manifest.json`	App manifest for running the app in local and remote environment.
`config.ts`	`config.ts` wraps access to environment variables that is used to set up the bots adapter.
`teamsapp.yml`	This is the main Teams Toolkit project that defines properties and configuration stage definitions.
`teamsapp.local.yml`	This overrides teamsapp.yml with actions that enable local execution and debugging.

Build and run your first custom engine agent

3 minutes remaining

After you set up your project workspace with Teams Toolkit, build your project. Ensure that you've signed in to your Microsoft 365 account.

Use this account to sign in to Teams. If you're using a Microsoft 365 developer program tenant, the admin account you set up while registering is your Microsoft 365 account.

Visual Studio Code
Command line

Open Visual Studio Code.
Select the Teams Toolkit icon in the sidebar.
Select Sign in to M365.

Your default web browser opens to let you sign in to the account.
Sign in to your Microsoft 365 account using your credentials.
Close the browser when prompted and return to Visual Studio Code.
Return to Teams Toolkit within Visual Studio Code.

Use this account to sign in to Teams. If you're using a Microsoft 365 developer program tenant, the admin account you set up while registering is your Microsoft 365 account.

Now you're ready to build the app and run it locally!

Sign in to Microsoft 365 with the Teamsapp CLI:
```
teamsapp account login m365
```
Your default web browser opens to let you sign in to the account. Sign in to your Azure account using your credentials. Close the browser when you're prompted.
Sign in to Azure with the Teamsapp CLI:
```
teamsapp account login azure
```
Your default web browser opens to let you sign in to the account. Sign in to your Azure account using your credentials. Close the browser when you're prompted.

The account logins are shared between Visual Studio Code and the Teamsapp CLI.

Now that the development environment is configured, you can create, build, and deploy your first Teams app.

Update your custom engine agent with Teams AI library capabilities

Teams Toolkit is the easiest way to scaffold and get a custom engine agent running. Teams Toolkit creates the basic structure for you to get started with configuring your custom engine agent.

Let's get started to configure Teams AI library capabilities:

Go to Visual Studio Code and open a terminal window.
Run the following command in the terminal to install yarn:
```
npm install
npm install @microsoft/teams-ai
```
Note

If you’re creating the sample through Teams Toolkit, you’ll find the .env.local.user file in the setup that is created automatically. If the file isn't available, create the .env.local.user file and update the OpenAI key or Azure OpenAI key and Azure OpenAI Endpoint.

Go to the app.ts file:

Add TurnContext to the import statement as follows:


import { MemoryStorage, TurnContext} from 'botbuilder';

Add DefaultConversationState, Memory, TurnState to the "@microsoft/teams-ai" import statement as follows:

     // See https://aka.ms/teams-ai-library to learn more about the Teams AI library.
     import { Application, ActionPlanner, DefaultConversationState, Memory, TurnState, OpenAIModel, PromptManager } from "@microsoft/teams-ai";

After "@microsoft/teams-ai", add the following:

// eslint-disable-next-line @typescript-eslint/no-empty-interface
interface ConversationState extends DefaultConversationState {
  lightsOn: boolean;
}
type ApplicationTurnState = TurnState<ConversationState>;

if (!process.env.OPENAI_API_KEY) {
throw new Error('Missing environment variables - please check that OPENAI_KEY or AZURE_OPENAI_KEY is set.');
}

After Define storage and application code, add the following:

// Define a prompt function for getting the current status of the lights
planner.prompts.addFunction('getLightStatus', async (context: TurnContext, memory: Memory) => {
return memory.getValue('conversation.lightsOn') ? 'on' : 'off';
});

After return memory.getValue('conversation.lightsOn') ? 'on' : 'off';, add the following:

// Register action handlers
app.ai.action('LightsOn', async (context: TurnContext, state: ApplicationTurnState) => {
    state.conversation.lightsOn = true;
    await context.sendActivity(`[lights on]`);
    return `the lights are now on`;
});

app.ai.action('LightsOff', async (context: TurnContext, state: ApplicationTurnState) => {
    state.conversation.lightsOn = false;
    await context.sendActivity(`[lights off]`);
    return `the lights are now off`;
});

interface PauseParameters {
    time: number;
}

app.ai.action('Pause', async (context: TurnContext, state: ApplicationTurnState, parameters: PauseParameters) => {
    await context.sendActivity(`[pausing for ${parameters.time / 1000} seconds]`);
    await new Promise((resolve) => setTimeout(resolve, parameters.time));
    return `done pausing`;
});

Go to the adapter.ts file:

Under Import required bot services., remove the existing code and update the following:

// Import required bot services.
// See https://aka.ms/bot-services to learn more about the different parts of a bot.
import { TeamsAdapter } from '@microsoft/teams-ai';
import { ConfigurationServiceClientCredentialFactory } from 'botbuilder';

Remove the following code under This bot's main dialog.:

// This bot's main dialog.
import config from "./config";

const botFrameworkAuthentication = new ConfigurationBotFrameworkAuthentication(
{},
new ConfigurationServiceClientCredentialFactory(config)
);

After import { ConfigurationServiceClientCredentialFactory } from 'botbuilder';, replace the code under Create adapter. with the following:

// Create adapter.
// See https://aka.ms/about-bot-adapter to learn more about how bots work.
const adapter = new TeamsAdapter(
    {},
    new ConfigurationServiceClientCredentialFactory({
        MicrosoftAppId: process.env.BOT_ID,
        MicrosoftAppPassword: process.env.BOT_PASSWORD,
        MicrosoftAppType: 'MultiTenant'
    })
);

Go to the index.ts file:

In the Create HTTP server., after console.log(\nBot Started, ${server.name} listening to ${server.url}); add the following:

console.log('\nGet Bot Framework Emulator: https://aka.ms/botframework-emulator');
console.log('\nTo test your bot in Teams, sideload the app manifest.json within Teams Apps.');

In the root folder, create new folder structure: prompts > sequence.
Under sequence, create the following files:
- config.json
- skprompt.txt
- actions.json

Go to the config.json file and add the following code to create the prompt model settings:

{
    "schema": 1.1,
    "description": "A bot that can turn the lights on and off",
    "type": "completion",
    "completion": {
        "model": "gpt-3.5-turbo",
        "completion_type": "chat",
        "include_history": true,
        "include_input": true,
        "max_input_tokens": 2800,
        "max_tokens": 1000,
        "temperature": 0.2,
        "top_p": 0.0,
        "presence_penalty": 0.6,
        "frequency_penalty": 0.0,
        "stop_sequences": []
    },
    "augmentation": {
        "augmentation_type": "sequence"
    }
}

The custom engine agent uses OpenAI or Azure OpenAI for its natural language modelling. The config.json file handles the language processing for the custom engine agent. This file, initiated by the PromptManager from the index.ts file, oversees the agent's language processing. On initiating a prompt, the prompt manager searches for the prompt config file (sequence/config.json) and passes the prompt settings to OpenAI or Azure OpenAI.

Go to the skprompt.txt file. Add the following instructions for OpenAI or Azure OpenAI to conduct and facilitate conversation:
```
The following is a conversation with an AI assistant.
The assistant can turn a light on or off.

context:
The lights are currently {{getLightStatus}}.
```
The skprompt.txt file contains text prompts with which the users interact with OpenAI or Azure OpenAI. OpenAI or Azure OpenAI attempts to use these prompts to produce the next series of words that are most likely to follow from the previous text.

Go to the actions.json file. Add the following code to define the actions for the custom engine agent:

{

    {
        "name": "LightsOn",
        "description": "Turn the lights on",
        "parameters": []
    },
    {
        "name": "LightsOff",
        "description": "Turn the lights off",
        "parameters": []
    },
    {
        "name": "Pause",
        "description": "Delays for a period of time",
        "parameters": {
            "type": "object",
            "properties": {
                "time": {
                    "type": "number",
                    "description": "The amount of time to delay in milliseconds"
                }
            },
            "required": [
                "time"
            ]
        }
    }
}

The actions.json file contains the actions that the custom engine agent can perform. The ActionPlanner from the index.ts file uses the actions defined in the actions.json file to perform the actions.

This sample uses OpenAI or Azure OpenAI for all its natural language modeling, the user can talk to the custom engine agent in any language of their choosing. The custom engine agent understands and responds appropriately without extra code.

Build and run your custom engine agent locally

To build and run your app in the local environment:

Select the F5 key to run your app in debug mode.

Note

If Teams Toolkit is unable to check a particular prerequisite, it prompts you to check.
Learn what happens when you run your app locally in the debugger.

In case you're wondering, when you press the F5 key, the Teams Toolkit:
1. Checks all the following prerequisites:
- You're logged in with a Microsoft 365 account.
- Custom app upload is enabled for your Microsoft 365 account.
- Supported Node.js version is installed.
- Port required by bot app is available.
1. Install npm packages.
2. Starts Dev Tunnel to create an HTTP tunnel.
3. Registers the app in Microsoft Entra ID and configures the app.
4. Registers the app in Bot Framework SDK and configures the app.
5. Registers the app in Teams Developer Portal and configures the app.
6. Starts the app.
7. Starts Teams in a web browser and uploads the app.
Note

When you run the app for the first time, all dependencies are downloaded, and the app is built. A browser window automatically opens when the build is complete. This process can take three to five minutes to complete.

Teams runs your app in a web browser.
Sign in with your Microsoft 365 account, if prompted.
Select Add when prompted to upload your app onto Teams on your local machine.

Now the custom engine agent is successfully running on Teams! After the app is loaded, a chat session with the custom engine agent opens.

You can type LightsOn to start interacting with the custom engine agent.

Learn how to troubleshoot if your app doesn't run locally.

To successfully run your app in Teams, ensure that you've enabled custom app upload in your Teams account. You can learn more about custom app upload in the prerequisites section.

Important

Uploading custom apps are available in Government Community Cloud (GCC), GCC-High, and Department of Defense (DOD) environments.

Tip

Check for issues before you upload a custom app, using the app validation tool. This tool is included in the toolkit. Fix the errors to upload the app.

Deploy your custom engine agent

2 minutes remaining

You've learned to create, build, and run custom engine agent. The final step is to deploy your app on Azure.

Let's deploy the custom engine agent on Azure using Teams Toolkit.

Use this account to access the Microsoft Azure portal and to provision new cloud resources to support your app.

Visual Studio Code
Command line

Open Visual Studio Code.
Open the project folder in which you created the app.
Select the Teams Toolkit icon in the sidebar.
Select Sign in to Azure.

Tip

If you have the Azure Account extension installed and are using the same account, you can skip this step. Use the same account as you're using in other extensions.

Your default web browser opens to let you sign in to the account.
Sign in to your Azure account using your credentials.
Close the browser when prompted and return to Visual Studio Code.

The ACCOUNTS section in the sidebar shows the two accounts separately. It also lists the number of usable Azure subscriptions available to you. Ensure you have at least one usable Azure subscription available. If not, sign out and use a different account.

Now you're ready to deploy your app to Azure!

Congratulations, you've created a custom engine agent!

Sign in to Microsoft 365 with the Teamsapp CLI:
```
teamsapp account login m365
```
Your default web browser opens to let you sign in to the account. Sign in to your Azure account using your credentials. Close the browser when you're prompted.
Sign in to Azure with the Teamsapp CLI:
```
teamsapp account login azure
```
Your default web browser opens to let you sign in to the account. Sign in to your Azure account using your credentials. Close the browser when you're prompted.

The account logins are shared between Visual Studio Code and the Teamsapp CLI.

Congratulations, you've signed in your Azure account!

Deploy your app to Azure

Deployment consists of two steps. First, necessary cloud resources are created (also known as provisioning). Then, your app's code is copied into the created cloud resources. For this tutorial, you'll deploy the custom engine agent.

What's the difference between Provision and Deploy?

The Provision step creates resources in Azure and Microsoft 365 for your app, but no code (HTML, CSS, JavaScript) is copied to the resources. The Deploy step copies the code for your app to the resources you created during the provision step. It's common to deploy multiple times without provisioning new resources. Since the provision step can take some time to complete, it's separate from the deployment step.

Visual Studio Code
Command Line

Select the Teams Toolkit icon in the Visual Studio Code sidebar.

Select Provision.
Select a subscription to use for the Azure resources.

Your app is hosted using Azure resources.

A dialog warns you that costs might be incurred when running resources in Azure.
Select Provision.

The provisioning process creates resources in the Azure cloud. It might take some time. You can monitor the progress by watching the dialogs in the bottom-right corner. After a few minutes, you see the following notice:

If you want, you can view the provisioned resources. For this tutorial, you don't need to view resources.

The provisioned resource appears in the ENVIRONMENT section.
Select Deploy from the LIFECYCLE panel after provisioning is complete.

As with provisioning, deployment takes some time. You can monitor the process by watching the dialogs in the bottom-right corner. After a few minutes, you see a completion notice.

Now, you can use the same process to deploy your custom engine agent to Azure.

In your terminal window:

Run teamsapp provision.
```
teamsapp provision
```

When prompted, select an Azure subscription to use Azure resources.

Your app is hosted using Azure resources.

Run teamsapp deploy.
```
teamsapp deploy
```

Run the deployed app

After the provisioning and deployment steps are complete:

Open the debug panel (Ctrl+Shift+D / ⌘⇧-D or View > Run) from Visual Studio Code.
Select Launch Remote (Edge) from the launch configuration dropdown.
Select the Start debugging (F5) to launch your app from Azure.
Select Add.

The toolkit displays a message to indicate that the app is added to Teams.

Your app is loaded on Teams client.

Learn what happens when you deployed your app to Azure

Before deployment, the application has been running locally:

The backend runs using Azure Functions Core Tools.
The application HTTP endpoint, where Microsoft Teams loads the application, runs locally.

Deployment is a two-step process. You provision the resources on an active Azure subscription, and then deploy or upload the backend and frontend code for the application to Azure.

The backend, if configured, uses various Azure services, including Azure App Service and Azure Storage.
The frontend application is deployed to an Azure Storage account configured for static web hosting.

Before deployment, the application has been running locally:

The backend runs using Azure Functions Core Tools.
The application HTTP endpoint, where Microsoft Teams loads the application, runs locally.

Deployment is a two-step process. You provision the resources on an active Azure subscription, and then deploy or upload the backend and frontend code for the application to Azure.

The backend, if configured, uses various Azure services, including Azure App Service and Azure Storage.
The frontend application is deployed to an Azure Storage account configured for static web hosting.

Κοινή χρήση μέσω

Build a custom engine agent using Teams AI library and Teams Toolkit

Prerequisites

Prepare development environment

Install the Teams Toolkit

Set up your Teams development tenant

Check for custom app upload option

Create a free Teams developer tenant (optional)

Get a free Azure account

Create project workspace for your custom engine agent app

Create your custom engine agent project workspace

Take a tour of the app source code

Build and run your first custom engine agent

Update your custom engine agent with Teams AI library capabilities

Build and run your custom engine agent locally

Deploy your custom engine agent

Deploy your app to Azure

Run the deployed app

Congratulations!

Κοινή χρήση μέσω

Build a custom engine agent using Teams AI library and Teams Toolkit

Prerequisites

Prepare development environment

Install the Teams Toolkit

Set up your Teams development tenant

Check for custom app upload option

Create a free Teams developer tenant (optional)

Get a free Azure account

Create project workspace for your custom engine agent app

Create your custom engine agent project workspace

Take a tour of the app source code

Build and run your first custom engine agent

Sign in to your Microsoft 365 account

Update your custom engine agent with Teams AI library capabilities

Build and run your custom engine agent locally

Deploy your custom engine agent

Sign in to your Azure account

Deploy your app to Azure

Run the deployed app

Congratulations!