Skip to content

Commit

Permalink
Merge pull request #452 from MicrosoftDocs/main
Browse files Browse the repository at this point in the history
Merge to publish.
  • Loading branch information
Lauragra authored Dec 10, 2024
2 parents b4407c4 + c700e75 commit b708339
Show file tree
Hide file tree
Showing 4 changed files with 67 additions and 6 deletions.
47 changes: 45 additions & 2 deletions docs/add-agent-capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,49 @@ ms.date: 11/15/2024

You can enhance the user experience of your declarative agent by adding capabilities. The **capabilities** element in the manifest reference and the **Capabilities** section in the Copilot Studio agent builder provide several options for you to unlock features for your users. This article describes the user capabilities that you can add to your agents.

## Code interpreter

Code interpreter is an advanced tool designed to solve complex tasks via Python code. It uses the reasoning model to write and run code, enabling users to solve complex math problems, analyze data, generate visualizations, and more. After the code runs, code interpreter outputs the results and generated code. It can also produce images or files based on the scenario, and accepts files as input for modifications and analysis.

> [!NOTE]
> Support for in-context agents that have code interpreter enabled varies by host.
### Code interpreter examples

#### Data graphing

The user prompt "Graph the first 20 numbers in a Fibonacci sequence" generates both a line graph and, when the user clicks the `</> Code` button, provides the corresponding Python code.

The following images show examples of the results of a data graphing request.

:::image type="content" source="assets/images/code-interpreter-python-1.png" alt-text="Graph response to the user prompt":::

:::image type="content" source="assets/images/code-interpreter-python-2.png" alt-text="Python code response to the user prompt":::

#### Data visualization

The user prompt "Create a word cloud of top pet names" generates a word cloud that includes the top names, as shown in the following example.

:::image type="content" source="assets/images/code-interpreter-pet-word-cloud.png" alt-text="Word cloud response to the user prompt":::

### Enable code interpreter

If you're using [Teams Toolkit and Visual Studio Code](build-declarative-agents.yml) to create your agent, to enable code interpreter, add the `CodeInterpreter` value to the **capabilities** property in your manifest reference, as shown in the following example.

```json
{
"capabilities": [
{
"name": "CodeInterpreter"
}
]
}
```

If you're using [Copilot Studio agent builder](copilot-studio-agent-builder.md) to create your agent, on the **Configure** tab, under **Capabilities**, choose the toggle next to **Code interpreter**.

:::image type="content" source="assets/images/capabilities-toggle.png" alt-text="Screenshot of the Capabilities section of the agent builder":::

## Image generator

The image generator capability enables declarative agents to generate images based on user prompts. Image generator uses the existing [Designer](https://designer.microsoft.com/) functionality to create visually appealing and contextually relevant graphics, and includes the following features:
Expand All @@ -21,7 +64,7 @@ The image generator capability enables declarative agents to generate images bas
- **Feedback mechanism**: Users can provide feedback on the generated images by giving a thumbs up or thumbs down. This helps improve the quality of future image generations.
- **Clipboard and sharing**: Users can copy the generated images to their clipboard to paste into other applications, or they can share the generated images directly from the interface.

### Image generation examples
### Image generator examples

The following examples show what users can do with the image generation capability in your agent.

Expand Down Expand Up @@ -61,4 +104,4 @@ If you're using [Copilot Studio agent builder](copilot-studio-agent-builder.md)
## Related content

- [Declarative agents overview](overview-declarative-agent.md)
- [Declarative agent manifest reference](declarative-agent-manifest.md)
- [Declarative agent manifest reference](declarative-agent-manifest.md)
3 changes: 2 additions & 1 deletion docs/copilot-studio-agent-builder-build.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,8 @@ If at least one connector is enabled, you can add it to your agent. Your agent i

In the **Capabilities** section of the **Configure** tab, you can choose to add the following capabilities to your agent:

- Image generator - Generates images base on user prompts.
- [Code interpreter](add-agent-capabilities.md#code-interpreter) - Uses Python code to solve complex math problems, analyze data, and generate visualizations.
- [Image generator](add-agent-capabilities.md#image-generator) - Generates images base on user prompts.

To add one or more capabilities to your agent, choose the toggle next to the capability you want to enable.

Expand Down
11 changes: 11 additions & 0 deletions docs/declarative-agent-manifest-1.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ This schema version introduces the following changes from [version 1.0](declarat

- The `sites` property is added to the [web search object](#web-search-object). This allows the web search capability to be limited to allowed sites.
- The [graphic art](#graphic-art-object) capability was added. This capability allows the declarative agent to generate images and art based on user input.
- The [code interpreter](#code-interpreter-object) capability was added. This capability allows the declarative agent to generate and execute Python code.

## JSON schema

Expand Down Expand Up @@ -79,6 +80,7 @@ The capabilities object is the base type of objects in the `capabilities` proper
- [OneDrive and SharePoint object](#onedrive-and-sharepoint-object)
- [Microsoft Graph Connectors object](#microsoft-graph-connectors-object)
- [Graphic art object](#graphic-art-object)
- [Code interpreter object](#code-interpreter-object)

#### Example of capabilities

Expand Down Expand Up @@ -216,6 +218,15 @@ The graphic art object contains the following properties.
| -------- | ------ | ----------- |
| `name` | String | Required. Must be set to `GraphicArt`. |

#### Code interpreter object

Indicates that the declarative agent can generate and execute Python code to solve complex math problems, analyze data, generate visualizations, and more. For more information, see [Code interpreter](add-agent-capabilities.md#code-interpreter).

The web search object contains the following properties.

| Property | Type | Description |
| -------- | ------ | ----------- |
| `name` | String | Required. Must be set to `CodeInterpreter`. |

### Conversation starters object

Expand Down
12 changes: 9 additions & 3 deletions docs/whats-new.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,21 @@ ms.topic: overview

As a developer, you can extend, enrich, and customize [Microsoft 365 Copilot](/microsoft-365-copilot/microsoft-365-copilot-overview) for the unique way your customers work. This article provides the latest information about what's new in Microsoft 365 Copilot extensibility.

## December 2024

## Add code interpreter to your declarative agent

Add the code interpreter capability to your declarative agent by using the Copilot Studio agent builder or Teams Toolkit. To learn more, see [Add capabilities to your declarative agent](add-agent-capabilities.md).

## November 2024

### Add capabilities to your declarative agent
### Add image generator to your declarative agent

Add new capabilities to your declarative agents, including an image generator. To learn more, see [Add capabilities to your declarative agent](add-agent-capabilities.md).
Add the image generator capability to your declarative agent by using the Copilot Studio agent builder or Teams Toolkit.. To learn more, see [Add capabilities to your declarative agent](add-agent-capabilities.md).

### Declarative agent manifest version 1.2

A new version of the declarative agent manifest schema has been released. Version 1.2 adds support for scoped web search and the new image generator capabilities. To learn more, see [Declarative agent schema 1.2 for Microsoft 365 Copilot](declarative-agent-manifest-1.2.md).
A new version of the declarative agent manifest schema has been released. Version 1.2 adds support for scoped web search and the new code interpreter and image generator capabilities. To learn more, see [Declarative agent schema 1.2 for Microsoft 365 Copilot](declarative-agent-manifest-1.2.md).

### API plugin manifest version 2.2

Expand Down

0 comments on commit b708339

Please sign in to comment.