@@ -65,7 +65,7 @@ from either OpenAI or Azure OpenAI and to run one of the C#, Python, and Java co
### For C#:
1. Create a new console app.
-2. Add the semantic kernel nuget `Microsoft.SemanticKernel`.
+2. Add the semantic kernel nuget [Microsoft.SemanticKernel](https://www.nuget.org/packages/Microsoft.SemanticKernel/).
3. Copy the code from [here](dotnet/README.md) into the app `Program.cs` file.
4. Replace the configuration placeholders for API key and other params with your key and settings.
5. Run with `F5` or `dotnet run`
@@ -80,8 +80,9 @@ from either OpenAI or Azure OpenAI and to run one of the C#, Python, and Java co
### For Java:
-1. Clone and checkout the experimental Java branch: `git clone -b experimental-java https://github.com/microsoft/semantic-kernel.git`
-2. Follow the instructions [here](https://github.com/microsoft/semantic-kernel/blob/experimental-java/java/samples/sample-code/README.md)
+1. Clone the repository: `git clone https://github.com/microsoft/semantic-kernel.git`
+ 1. To access the latest Java code, clone and checkout the Java development branch: `git clone -b java-development https://github.com/microsoft/semantic-kernel.git`
+2. Follow the instructions [here](https://github.com/microsoft/semantic-kernel/blob/main/java/samples/sample-code/README.md)
## Learning how to use Semantic Kernel
@@ -164,6 +165,10 @@ To learn more and get started:
- Attend [regular office hours and SK community events](COMMUNITY.md)
- Follow the team on our [blog](https://aka.ms/sk/blog)
+## Contributor Wall of Fame
+
+[](https://github.com/microsoft/semantic-kernel/graphs/contributors)
+
## Code of Conduct
This project has adopted the
diff --git a/SECURITY.md b/SECURITY.md
index e138ec5d6a77..eed215e185a7 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -4,7 +4,7 @@
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
-If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below.
+If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://www.microsoft.com/en-us/msrc/definition-of-a-security-vulnerability?rtc=1), please report it to us as described below.
## Reporting Security Issues
@@ -12,9 +12,9 @@ If you believe you have found a security vulnerability in any Microsoft-owned re
Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report).
-If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
+If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/en-us/msrc/pgp-key-msrc?rtc=2).
-You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
+You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/en-us/msrc?rtc=2).
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
@@ -28,7 +28,7 @@ Please include the requested information listed below (as much as you can provid
This information will help us triage your report more quickly.
-If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs.
+If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://www.microsoft.com/en-us/msrc/bounty?rtc=2) page for more details about our active programs.
## Preferred Languages
@@ -36,6 +36,6 @@ We prefer all communications to be in English.
## Policy
-Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd).
+Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/en-us/msrc/cvd?rtc=2).
diff --git a/docs/FAQS.md b/docs/FAQS.md
new file mode 100644
index 000000000000..1e3f07386e1b
--- /dev/null
+++ b/docs/FAQS.md
@@ -0,0 +1,55 @@
+# Frequently Asked Questions
+
+### How do I get access to nightly builds?
+
+Nightly builds of the Semantic Kernel are available [here](https://github.com/orgs/microsoft/packages?repo_name=semantic-kernel).
+
+To download nightly builds follow the following steps:
+
+1. You will need a GitHub account to complete these steps.
+1. Create a GitHub Personal Access Token with the `read:packages` scope using these [instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic).
+1. If you account is part of the Microsoft organization then you must authorize the `Microsoft` organization as a single sign-on organization.
+ 1. Click the "Configure SSO" next to the Person Access Token you just created and then authorize `Microsoft`.
+1. Use the following command to add the Microsoft GitHub Packages source to your NuGet configuration:
+
+ ```powershell
+ dotnet nuget add source --username GITHUBUSERNAME --password GITHUBPERSONALACCESSTOKEN --store-password-in-clear-text --name GitHubMicrosoft "https://nuget.pkg.github.com/microsoft/index.json"
+ ```
+
+1. Or you can manually create a `NuGet.Config` file.
+
+ ```xml
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ```
+
+ * If you place this file in your project folder make sure to have Git (or whatever source control you use) ignore it.
+ * For more information on where to store this file go [here](https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file).
+ * You can also use the following command `he Microsoft GitHub Packages source can be added easier to NuGet:`
+1. You can now add packages from the nightly build to your project.
+ * E.g. use this command `dotnet add package Microsoft.SemanticKernel.Core --version 0.26.231003.1-nightly`
+1. And the latest package release can be referenced in the project like this:
+ * ``
+
+For more information see:
diff --git a/docs/decisions/0004-error-handling.md b/docs/decisions/0004-error-handling.md
index 74be52a04227..3c3f61c648d7 100644
--- a/docs/decisions/0004-error-handling.md
+++ b/docs/decisions/0004-error-handling.md
@@ -1,15 +1,17 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
-contact: semenshi
+contact: SergeyMenshykh
date: 2023-06-23
deciders: shawncal
consulted: stephentoub
informed:
---
+
# Error handling improvements
## Disclaimer
+
This ADR describes problems and their solutions for improving the error handling aspect of SK. It does not address logging, resiliency, or observability aspects.
## Context and Problem Statement
@@ -41,4 +43,4 @@ Currently, there are several aspects of error handling in SK that can be enhance
- Identify all cases where the original exception is not preserved as an inner exception of the rethrown SK exception, and address them.
- Create a new exception HttpOperationException, which includes a StatusCode property, and implement the necessary logic to map the exception from HttpStatusCode, HttpRequestException, or Azure.RequestFailedException. Update existing SK code that interacts with the HTTP stack to throw HttpOperationException in case of a failed HTTP request and assign the original exception as its inner exception.
- Modify all SK components that currently store exceptions to SK context to rethrow them instead.
-- Simplify the SK critical exception handling functionality by modifying the IsCriticalException extension method to exclude handling of StackOverflowException and OutOfMemoryException exceptions. This is because the former exception is not thrown, so the calling code won't be executed, while the latter exception doesn't necessarily prevent the execution of recovery code.
\ No newline at end of file
+- Simplify the SK critical exception handling functionality by modifying the IsCriticalException extension method to exclude handling of StackOverflowException and OutOfMemoryException exceptions. This is because the former exception is not thrown, so the calling code won't be executed, while the latter exception doesn't necessarily prevent the execution of recovery code.
diff --git a/docs/decisions/0006-open-api-dynamic-payload-and-namespaces.md b/docs/decisions/0006-open-api-dynamic-payload-and-namespaces.md
index f7faf5f2125f..5935d4ce3c8c 100644
--- a/docs/decisions/0006-open-api-dynamic-payload-and-namespaces.md
+++ b/docs/decisions/0006-open-api-dynamic-payload-and-namespaces.md
@@ -1,32 +1,39 @@
---
status: accepted
-contact: semenshi
+contact: SergeyMenshykh
date: 2023-08-15
deciders: shawncal
consulted:
informed:
---
+
# Dynamic payload building for PUT and POST RestAPI operations and parameter namespacing
## Context and Problem Statement
+
Currently, the SK OpenAPI does not allow the dynamic creation of payload/body for PUT and POST RestAPI operations, even though all the required metadata is available. One of the reasons the functionality was not fully developed originally, and eventually removed is that JSON payload/body content of PUT and POST RestAPI operations might contain properties with identical names at various levels. It was not clear how to unambiguously resolve their values from the flat list of context variables. Another reason the functionality has not been added yet is that the 'payload' context variable, along with RestAPI operation data contract schema(OpenAPI, JSON schema, Typings?) should have been sufficient for LLM to provide fully fleshed-out JSON payload/body content without the need to build it dynamically.
+
## Decision Drivers
-* Create a mechanism that enables the dynamic construction of the payload/body for PUT and POST RestAPI operations.
-* Develop a mechanism(namespacing) that allows differentiation of payload properties with identical names at various levels for PUT and POST RestAPI operations.
-* Aim to minimize breaking changes and maintain backward compatibility of the code as much as possible.
+
+- Create a mechanism that enables the dynamic construction of the payload/body for PUT and POST RestAPI operations.
+- Develop a mechanism(namespacing) that allows differentiation of payload properties with identical names at various levels for PUT and POST RestAPI operations.
+- Aim to minimize breaking changes and maintain backward compatibility of the code as much as possible.
## Considered Options
-* Enable the dynamic creation of payload and/or namespacing by default.
-* Enable the dynamic creation of payload and/or namespacing based on configuration.
+
+- Enable the dynamic creation of payload and/or namespacing by default.
+- Enable the dynamic creation of payload and/or namespacing based on configuration.
## Decision Outcome
+
Chosen option: "Enable the dynamic creation of payload and/or namespacing based on configuration". This option keeps things compatible, so the change won't affect any SK consumer code. Additionally, it lets SK consumer code easily control both mechanisms, turning them on or off based on the scenario.
## Additional details
### Enabling dynamic creation of payload
+
In order to enable the dynamic creation of payloads/bodies for PUT and POST RestAPI operations, please set the `EnableDynamicPayload` property of the `OpenApiSkillExecutionParameters` execution parameters to `true` when importing the AI plugin:
```csharp
@@ -34,12 +41,13 @@ var plugin = await kernel.ImportPluginFunctionsAsync("", new Uri("", new Uri(""), new OpenApiSkillExecutionParameters(httpClient) { EnablePayloadNamespacing = true });
```
+
Remember that the namespacing mechanism depends on prefixing parameter names with their parent parameter name, separated by dots. So, use the 'namespaced' parameter names when adding arguments for them to the context variables. Let's consider this JSON:
```json
-{
- "upn": "",
+{
+ "upn": "",
"receiver": {
"upn": ""
},
@@ -70,7 +80,9 @@ Remember that the namespacing mechanism depends on prefixing parameter names wit
}
}
```
+
It contains `upn` properties at different levels. The the argument registration for the parameters(property values) will look like:
+
```csharp
var contextVariables = new ContextVariables();
contextVariables.Set("upn", "");
diff --git a/docs/decisions/0008-support-generic-llm-request-settings.md b/docs/decisions/0008-support-generic-llm-request-settings.md
index 43dcafe1f1e6..6fae2fdf3ef5 100644
--- a/docs/decisions/0008-support-generic-llm-request-settings.md
+++ b/docs/decisions/0008-support-generic-llm-request-settings.md
@@ -2,11 +2,12 @@
# These are optional elements. Feel free to remove any of them.
status: accepted
contact: markwallace-microsoft
-date: 2023-=9-15
+date: 2023-9-15
deciders: shawncal
-consulted: stoub, lemiller, dmytrostruk
-informed:
+consulted: stephentoub, lemillermicrosoft, dmytrostruk
+informed:
---
+
# Refactor to support generic LLM request settings
## Context and Problem Statement
@@ -25,18 +26,18 @@ Link to issue raised by the implementer of the Oobabooga AI service:
-* Good, SK abstractions contain no references to OpenAI specific request settings
-* Neutral, because anonymous types can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., `temperature` or combine properties for different AI services e.g., `max_tokens` (OpenAI) and `max_new_tokens` (Oobabooga).
-* Bad, because it's not clear to developers what they should pass when creating a semantic function
-* Bad, because it's not clear to implementors of a chat/text completion service what they should accept or how to add service specific properties.
-* Bad, there is no compiler type checking for code paths where the dynamic argument has not been resolved which will impact code quality. Type issues manifest as `RuntimeBinderException`'s and may be difficult to troubleshoot. Special care needs to be taken with return types e.g., may be necessary to specify an explicit type rather than just `var` again to avoid errors such as `Microsoft.CSharp.RuntimeBinder.RuntimeBinderException : Cannot apply indexing with [] to an expression of type 'object'`
+- Good, SK abstractions contain no references to OpenAI specific request settings
+- Neutral, because anonymous types can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., `temperature` or combine properties for different AI services e.g., `max_tokens` (OpenAI) and `max_new_tokens` (Oobabooga).
+- Bad, because it's not clear to developers what they should pass when creating a semantic function
+- Bad, because it's not clear to implementors of a chat/text completion service what they should accept or how to add service specific properties.
+- Bad, there is no compiler type checking for code paths where the dynamic argument has not been resolved which will impact code quality. Type issues manifest as `RuntimeBinderException`'s and may be difficult to troubleshoot. Special care needs to be taken with return types e.g., may be necessary to specify an explicit type rather than just `var` again to avoid errors such as `Microsoft.CSharp.RuntimeBinder.RuntimeBinderException : Cannot apply indexing with [] to an expression of type 'object'`
### Use `object` to pass request settings
@@ -127,11 +128,11 @@ The calling pattern is the same as for the `dynamic` case i.e. use either an ano
PR:
-* Good, SK abstractions contain no references to OpenAI specific request settings
-* Neutral, because anonymous types can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., `temperature` or combine properties for different AI services e.g., `max_tokens` (OpenAI) and `max_new_tokens` (Oobabooga).
-* Bad, because it's not clear to developers what they should pass when creating a semantic function
-* Bad, because it's not clear to implementors of a chat/text completion service what they should accept or how to add service specific properties.
-* Bad, code is needed to perform type checks and explicit casts. The situation is slightly better than for the `dynamic` case.
+- Good, SK abstractions contain no references to OpenAI specific request settings
+- Neutral, because anonymous types can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., `temperature` or combine properties for different AI services e.g., `max_tokens` (OpenAI) and `max_new_tokens` (Oobabooga).
+- Bad, because it's not clear to developers what they should pass when creating a semantic function
+- Bad, because it's not clear to implementors of a chat/text completion service what they should accept or how to add service specific properties.
+- Bad, code is needed to perform type checks and explicit casts. The situation is slightly better than for the `dynamic` case.
### Define a base class for AI request settings which all implementations must extend
@@ -221,12 +222,12 @@ this._summarizeConversationFunction = kernel.CreateSemanticFunction(
The caveat with this pattern is, assuming a more specific implementation of `AIRequestSettings` uses JSON serialization/deserialization to hydrate an instance from the base `AIRequestSettings`, this will only work if all properties are supported by the default JsonConverter e.g.,
-* If we have `MyAIRequestSettings` which includes a `Uri` property. The implementation of `MyAIRequestSettings` would make sure to load a URI converter so that it can serialize/deserialize the settings correctly.
-* If the settings for `MyAIRequestSettings` are sent to an AI service which relies on the default JsonConverter then a `NotSupportedException` exception will be thrown.
+- If we have `MyAIRequestSettings` which includes a `Uri` property. The implementation of `MyAIRequestSettings` would make sure to load a URI converter so that it can serialize/deserialize the settings correctly.
+- If the settings for `MyAIRequestSettings` are sent to an AI service which relies on the default JsonConverter then a `NotSupportedException` exception will be thrown.
PR:
-* Good, SK abstractions contain no references to OpenAI specific request settings
-* Good, because it is clear to developers what they should pass when creating a semantic function and it is easy to discover what service specific request setting implementations exist.
-* Good, because it is clear to implementors of a chat/text completion service what they should accept and how to extend the base abstraction to add service specific properties.
-* Neutral, because `ExtensionData` can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., `temperature` or combine properties for different AI services e.g., `max_tokens` (OpenAI) and `max_new_tokens` (Oobabooga).
+- Good, SK abstractions contain no references to OpenAI specific request settings
+- Good, because it is clear to developers what they should pass when creating a semantic function and it is easy to discover what service specific request setting implementations exist.
+- Good, because it is clear to implementors of a chat/text completion service what they should accept and how to extend the base abstraction to add service specific properties.
+- Neutral, because `ExtensionData` can be used which allows a developer to pass in properties that may be supported by multiple AI services e.g., `temperature` or combine properties for different AI services e.g., `max_tokens` (OpenAI) and `max_new_tokens` (Oobabooga).
diff --git a/docs/decisions/0009-support-multiple-named-args-in-template-function-calls.md b/docs/decisions/0009-support-multiple-named-args-in-template-function-calls.md
index 40251ece1aca..fbf533aa4d77 100644
--- a/docs/decisions/0009-support-multiple-named-args-in-template-function-calls.md
+++ b/docs/decisions/0009-support-multiple-named-args-in-template-function-calls.md
@@ -4,9 +4,10 @@ status: accepted
contact: dmytrostruk
date: 2013-06-16
deciders: shawncal, hario90
-consulted: dmytrostruk, matthewbolanos
+consulted: dmytrostruk, matthewbolanos
informed: lemillermicrosoft
---
+
# Add support for multiple named arguments in template function calls
## Context and Problem Statement
@@ -15,30 +16,30 @@ Native functions now support multiple parameters, populated from context values
## Decision Drivers
-* Parity with Guidance
-* Readability
-* Similarity to languages familiar to SK developers
-* YAML compatibility
+- Parity with Guidance
+- Readability
+- Similarity to languages familiar to SK developers
+- YAML compatibility
## Considered Options
### Syntax idea 1: Using commas
-
+
```handlebars
{{Skill.MyFunction street: "123 Main St", zip: "98123", city:"Seattle", age: 25}}
```
Pros:
-* Commas could make longer function calls easier to read, especially if spaces before and after the arg separator (a colon in this case) are allowed.
+- Commas could make longer function calls easier to read, especially if spaces before and after the arg separator (a colon in this case) are allowed.
Cons:
-* Guidance doesn't use commas
-* Spaces are already used as delimiters elsewhere so the added complexity of supporting commas isn't necessary
+- Guidance doesn't use commas
+- Spaces are already used as delimiters elsewhere so the added complexity of supporting commas isn't necessary
### Syntax idea 2: JavaScript/C#-Style delimiter (colon)
-
+
```handlebars
{{MyFunction street:"123 Main St" zip:"98123" city:"Seattle" age: "25"}}
@@ -47,12 +48,12 @@ Cons:
Pros:
-* Resembles JavaScript Object syntax and C# named argument syntax
+- Resembles JavaScript Object syntax and C# named argument syntax
Cons:
-* Doesn't align with Guidance syntax which uses equal signs as arg part delimiters
-* Too similar to YAML key/value pairs if we support YAML prompts in the future. It's likely possible to support colons as delimiters but would be better to have a separator that is distinct from normal YAML syntax.
+- Doesn't align with Guidance syntax which uses equal signs as arg part delimiters
+- Too similar to YAML key/value pairs if we support YAML prompts in the future. It's likely possible to support colons as delimiters but would be better to have a separator that is distinct from normal YAML syntax.
### Syntax idea 3: Python/Guidance-Style delimiter
@@ -62,29 +63,29 @@ Cons:
Pros:
-* Resembles Python's keyword argument syntax
-* Resembles Guidance's named argument syntax
-* Not too similar to YAML key/value pairs if we support YAML prompts in the future.
+- Resembles Python's keyword argument syntax
+- Resembles Guidance's named argument syntax
+- Not too similar to YAML key/value pairs if we support YAML prompts in the future.
Cons:
-* Doesn't align with C# syntax
+- Doesn't align with C# syntax
### Syntax idea 4: Allow whitespace between arg name/value delimiter
```handlebars
-{{MyFunction street = "123 Main St" zip = "98123" city = "Seattle"}}
+{{MyFunction street="123 Main St" zip="98123" city="Seattle"}}
```
Pros:
-* Follows the convention followed by many programming languages of whitespace flexibility where spaces, tabs, and newlines within code don't impact a program's functionality
+- Follows the convention followed by many programming languages of whitespace flexibility where spaces, tabs, and newlines within code don't impact a program's functionality
Cons:
-* Promotes code that is harder to read unless commas can be used (see [Using Commas](#syntax-idea-1-using-commas))
-* More complexity to support
-* Doesn't align with Guidance which doesn't support spaces before and after the = sign.
+- Promotes code that is harder to read unless commas can be used (see [Using Commas](#syntax-idea-1-using-commas))
+- More complexity to support
+- Doesn't align with Guidance which doesn't support spaces before and after the = sign.
## Decision Outcome
@@ -92,20 +93,18 @@ Chosen options: "Syntax idea 3: Python/Guidance-Style keyword arguments", becaus
Additional decisions:
-* Continue supporting up to 1 positional argument for backward compatibility. Currently, the argument passed to a function is assumed to be the `$input` context variable.
+- Continue supporting up to 1 positional argument for backward compatibility. Currently, the argument passed to a function is assumed to be the `$input` context variable.
Example
```handlebars
-
{{MyFunction "inputVal" street="123 Main St" zip="98123" city="Seattle"}}
-
```
-* Allow arg values to be defined as strings or variables ONLY, e.g.
-
+- Allow arg values to be defined as strings or variables ONLY, e.g.
+
```handlebars
-{{MyFunction street=$street zip="98123" city='Seattle'}}
+{{MyFunction street=$street zip="98123" city="Seattle"}}
```
If function expects a value other than a string for an argument, the SDK will use the corresponding TypeConverter to parse the string provided when evaluating the expression.
diff --git a/docs/decisions/0010-dotnet-project-structure.md b/docs/decisions/0010-dotnet-project-structure.md
index 21ff81f6e962..7b21e0711647 100644
--- a/docs/decisions/0010-dotnet-project-structure.md
+++ b/docs/decisions/0010-dotnet-project-structure.md
@@ -1,13 +1,15 @@
---
-
# These are optional elements. Feel free to remove any of them
status: accepted
contact: markwallace-microsoft
date: 2023-09-29
-deciders: semenshi, dmytrostruk, rbarreto
-consulted: shawncal, stoub, lemiller
-informed: {list everyone who is kept up-to-date on progress; and with whom there is a one-way communication}
+deciders: SergeyMenshykh, dmytrostruk, RogerBarreto
+consulted: shawncal, stephentoub, lemillermicrosoft
+informed:
+ {
+ list everyone who is kept up-to-date on progress; and with whom there is a one-way communication,
+ }
---
# DotNet Project Structure for 1.0 Release
@@ -49,13 +51,13 @@ Chosen option: Option #2: Folder naming matches assembly name, because:
Main categories for the projects will be:
-1. `Connectors`: ***A connector project allows the Semantic Kernel to connect to AI and Memory services***. Some of the existing connector projects may move to other repositories.
-1. `Planners`: ***A planner project provides one or more planner implementations which take an ask and convert it into an executable plan to achieve that ask***. This category will include the current action, sequential and stepwise planners (these could be merged into a single project). Additional planning implementations e.g., planners that generate Powershell or Python code can be added as separate projects.
-1. `Functions`: ***A function project that enables the Semantic Kernel to access the functions it will orchestrate***. This category will include:
- 1. Semantic functions i.e., prompts executed against an LLM
- 1. GRPC remote procedures i.e., procedures executed remotely using the GRPC framework
- 1. Open API endpoints i.e., REST endpoints that have Open API definitions executed remotely using the HTTP protocol
-1. `Plugins`: ***A plugin project contains the implementation(s) of a Semantic Kernel plugin***. A Semantic Kernel plugin is contains a concrete implementation of a function e.g., a plugin may include code for basic text operations.
+1. `Connectors`: **_A connector project allows the Semantic Kernel to connect to AI and Memory services_**. Some of the existing connector projects may move to other repositories.
+1. `Planners`: **_A planner project provides one or more planner implementations which take an ask and convert it into an executable plan to achieve that ask_**. This category will include the current action, sequential and stepwise planners (these could be merged into a single project). Additional planning implementations e.g., planners that generate Powershell or Python code can be added as separate projects.
+1. `Functions`: **_A function project that enables the Semantic Kernel to access the functions it will orchestrate_**. This category will include:
+ 1. Semantic functions i.e., prompts executed against an LLM
+ 1. GRPC remote procedures i.e., procedures executed remotely using the GRPC framework
+ 1. Open API endpoints i.e., REST endpoints that have Open API definitions executed remotely using the HTTP protocol
+1. `Plugins`: **_A plugin project contains the implementation(s) of a Semantic Kernel plugin_**. A Semantic Kernel plugin is contains a concrete implementation of a function e.g., a plugin may include code for basic text operations.
### Option #1: New `planning`, `functions` and `plugins` project areas
@@ -97,17 +99,17 @@ SK-dotnet
### Changes
-| Project | Description |
-|-------------------------------------|-------------|
-| `Functions.Native` | Extract native functions from Semantic Kernel core and abstractions. |
-| `Functions.Semantic` | Extract semantic functions from Semantic Kernel core and abstractions. Include the prompt template engine. |
-| `Functions.Planning` | Extract planning from Semantic Kernel core and abstractions. |
-| `Functions.Grpc` | Old `Skills.Grpc` project |
-| `Functions.OpenAPI` | Old `Skills.OpenAPI` project |
-| `Plugins.Core` | Old `Skills.Core` project |
-| `Plugins.Document` | Old `Skills.Document` project |
-| `Plugins.MsGraph` | Old `Skills.MsGraph` project |
-| `Plugins.WebSearch` | Old `Skills.WebSearch` project |
+| Project | Description |
+| -------------------- | ---------------------------------------------------------------------------------------------------------- |
+| `Functions.Native` | Extract native functions from Semantic Kernel core and abstractions. |
+| `Functions.Semantic` | Extract semantic functions from Semantic Kernel core and abstractions. Include the prompt template engine. |
+| `Functions.Planning` | Extract planning from Semantic Kernel core and abstractions. |
+| `Functions.Grpc` | Old `Skills.Grpc` project |
+| `Functions.OpenAPI` | Old `Skills.OpenAPI` project |
+| `Plugins.Core` | Old `Skills.Core` project |
+| `Plugins.Document` | Old `Skills.Document` project |
+| `Plugins.MsGraph` | Old `Skills.MsGraph` project |
+| `Plugins.WebSearch` | Old `Skills.WebSearch` project |
### Semantic Kernel Skills and Functions
@@ -125,19 +127,19 @@ SK-dotnet
│
├── Microsoft.SemanticKernel.Connectors.AI.OpenAI*
│ ├── src
- │ └── tests
+ │ └── tests
│ (Not shown but all projects will have src and tests subfolders)
├── Microsoft.SemanticKernel.Connectors.AI.HuggingFace
├── Microsoft.SemanticKernel.Connectors.Memory.AzureCognitiveSearch
├── Microsoft.SemanticKernel.Connectors.Memory.Qdrant
│
├── Microsoft.SemanticKernel.Planners*
- │
+ │
├── Microsoft.SemanticKernel.Reliability.Basic*
├── Microsoft.SemanticKernel.Reliability.Polly
- │
+ │
├── Microsoft.SemanticKernel.TemplateEngines.Basic*
- │
+ │
├── Microsoft.SemanticKernel.Functions.Semantic*
├── Microsoft.SemanticKernel.Functions.Grpc
├── Microsoft.SemanticKernel.Functions.OpenAPI
@@ -156,7 +158,7 @@ SK-dotnet
└── Microsoft.SemanticKernel.MetaPackage
```
-***Notes:***
+**_Notes:_**
- There will only be a single solution file (initially).
- Projects will be grouped in the solution i.e., connectors, planners, plugins, functions, extensions, ...
@@ -197,33 +199,33 @@ SK-dotnet
└── SemanticKernel.UnitTests
```
-\\* - Means the project is part of the Semantic Kernel meta package
+\\\* - Means the project is part of the Semantic Kernel meta package
### Project Descriptions
-| Project | Description |
-|-------------------------------------|-------------|
-| Connectors.AI.OpenAI | Azure OpenAI and OpenAI service connectors |
-| Connectors... | Collection of other AI service connectors, some of which will move to another repository |
-| Connectors.UnitTests | Connector unit tests |
-| Planner.ActionPlanner | Semantic Kernel implementation of an action planner |
-| Planner.SequentialPlanner | Semantic Kernel implementation of a sequential planner |
-| Planner.StepwisePlanner | Semantic Kernel implementation of a stepwise planner |
-| TemplateEngine.Basic | Prompt template engine basic implementations which are used by Semantic Functions only |
-| Extensions.UnitTests | Extensions unit tests |
-| InternalUtilities | Internal utilities which are reused by multiple NuGet packages (all internal) |
-| Skills.Core | Core set of native functions which are provided to support Semantic Functions |
-| Skills.Document | Native functions for interacting with Microsoft documents |
-| Skills.Grpc | Semantic Kernel integration for GRPC based endpoints |
-| Skills.MsGraph | Native functions for interacting with Microsoft Graph endpoints |
-| Skills.OpenAPI | Semantic Kernel integration for OpenAI endpoints and reference Azure Key Vault implementation |
-| Skills.Web | Native functions for interacting with Web endpoints e.g., Bing, Google, File download |
-| Skills.UnitTests | Skills unit tests |
-| IntegrationTests | Semantic Kernel integration tests |
-| SemanticKernel | Semantic Kernel core implementation |
-| SemanticKernel.Abstractions | Semantic Kernel abstractions i.e., interface, abstract classes, supporting classes, ... |
-| SemanticKernel.MetaPackage | Semantic Kernel meta package i.e., a NuGet package that references other required Semantic Kernel NuGet packages |
-| SemanticKernel.UnitTests | Semantic Kernel unit tests |
+| Project | Description |
+| --------------------------- | ---------------------------------------------------------------------------------------------------------------- |
+| Connectors.AI.OpenAI | Azure OpenAI and OpenAI service connectors |
+| Connectors... | Collection of other AI service connectors, some of which will move to another repository |
+| Connectors.UnitTests | Connector unit tests |
+| Planner.ActionPlanner | Semantic Kernel implementation of an action planner |
+| Planner.SequentialPlanner | Semantic Kernel implementation of a sequential planner |
+| Planner.StepwisePlanner | Semantic Kernel implementation of a stepwise planner |
+| TemplateEngine.Basic | Prompt template engine basic implementations which are used by Semantic Functions only |
+| Extensions.UnitTests | Extensions unit tests |
+| InternalUtilities | Internal utilities which are reused by multiple NuGet packages (all internal) |
+| Skills.Core | Core set of native functions which are provided to support Semantic Functions |
+| Skills.Document | Native functions for interacting with Microsoft documents |
+| Skills.Grpc | Semantic Kernel integration for GRPC based endpoints |
+| Skills.MsGraph | Native functions for interacting with Microsoft Graph endpoints |
+| Skills.OpenAPI | Semantic Kernel integration for OpenAI endpoints and reference Azure Key Vault implementation |
+| Skills.Web | Native functions for interacting with Web endpoints e.g., Bing, Google, File download |
+| Skills.UnitTests | Skills unit tests |
+| IntegrationTests | Semantic Kernel integration tests |
+| SemanticKernel | Semantic Kernel core implementation |
+| SemanticKernel.Abstractions | Semantic Kernel abstractions i.e., interface, abstract classes, supporting classes, ... |
+| SemanticKernel.MetaPackage | Semantic Kernel meta package i.e., a NuGet package that references other required Semantic Kernel NuGet packages |
+| SemanticKernel.UnitTests | Semantic Kernel unit tests |
### Naming Patterns
@@ -283,10 +285,9 @@ dotnet/
This diagram show current skills are integrated with the Semantic Kernel core.
-***Note:***
+**_Note:_**
- This is not a true class hierarchy diagram. It show some class relationships and dependencies.
- Namespaces are abbreviated to remove Microsoft.SemanticKernel prefix. Namespaces use `_` rather than `.`.
-
diff --git a/docs/decisions/0010-openai-function-calling.md b/docs/decisions/0010-openai-function-calling.md
deleted file mode 100644
index 63c605916cdd..000000000000
--- a/docs/decisions/0010-openai-function-calling.md
+++ /dev/null
@@ -1,69 +0,0 @@
----
-status: accepted
-contact: gitri-ms
-date: 2023-09-21
-deciders: gitri-ms, shawncal
-consulted: lemillermicrosoft, awharrison-28, dmytrostruk, nacharya1
-informed: eavanvalkenburg, kevdome3000
----
-# OpenAI Function Calling Support
-
-## Context and Problem Statement
-
-The [function calling](https://platform.openai.com/docs/guides/gpt/function-calling) capability of OpenAI's Chat Completions API allows developers to describe functions to the model, and have the model decide whether to output a JSON object specifying a function and appropriate arguments to call in response to the given prompt. This capability is enabled by two new API parameters to the `/v1/chat/completions` endpoint:
-- `function_call` - auto (default), none, or a specific function to call
-- `functions` - JSON descriptions of the functions available to the model
-
-Functions provided to the model are injected as part of the system message and are billed/counted as input tokens.
-
-We have received several community requests to provide support for this capability when using SK with the OpenAI chat completion models that support it.
-
-## Decision Drivers
-
-* Minimize changes to the core kernel for OpenAI-specific functionality
-* Cost concerns with including a long list of function descriptions in the request
-* Security and cost concerns with automatically executing functions returned by the model
-
-## Considered Options
-
-* Support sending/receiving functions via chat completions endpoint _with_ modifications to interfaces
-* Support sending/receiving functions via chat completions endpoint _without_ modifications to interfaces
-* Implement a planner around the function calling capability
-
-## Decision Outcome
-
-Chosen option: "Support sending/receiving functions via chat completions endpoint _without_ modifications to interfaces"
-
-With this option, we utilize the existing request settings object to send functions to the model. The app developer controls what functions are included and is responsible for validating and executing the function result.
-
-### Consequences
-
-* Good, because avoids breaking changes to the core kernel
-* Good, because OpenAI-specific functionality is contained to the OpenAI connector package
-* Good, because allows app to control what functions are available to the model (including non-SK functions)
-* Good, because keeps the option open for integrating with planners in the future
-* Neutral, because requires app developer to validate and execute resulting function
-* Bad, because not as obvious how to use this capability and access the function results
-
-## Pros and Cons of the Options
-
-### Support sending/receiving functions _with_ modifications to chat completions interfaces
-
-This option would update the `IChatCompletion` and `IChatResult` interfaces to expose parameters/methods for providing and accessing function information.
-
-* Good, because provides a clear path for using the function calling capability
-* Good, because allows app to control what functions are available to the model (including non-SK functions)
-* Neutral, because requires app developer to validate and execute resulting function
-* Bad, because introduces breaking changes to core kernel abstractions
-* Bad, because OpenAI-specific functionality would be included in core kernel abstractions and would need to be ignored by other model providers
-
-### Implement a planner around the function calling capability
-
-Orchestrating external function calls fits within SK's concept of planning. With this approach, we would implement a planner that would take the function calling result and produce a plan that the app developer could execute (similar to SK's ActionPlanner).
-
-* Good, because producing a plan result makes it easy for the app developer to execute the chosen function
-* Bad, because functions would need to be registered with the kernel in order to be executed
-* Bad, because would create confusion about when to use which planner
-
-## Additional notes
-There has been much discussion and debate over the pros and cons of automatically invoking a function returned by the OpenAI model, if it is registered with the kernel. As there are still many open questions around this behavior and its implications, we have decided to not include this capability in the initial implementation. We will continue to explore this option and may include it in a future update.
\ No newline at end of file
diff --git a/docs/decisions/0012-kernel-service-registration.md b/docs/decisions/0012-kernel-service-registration.md
index db2e8ac14478..2e69907bcc45 100644
--- a/docs/decisions/0012-kernel-service-registration.md
+++ b/docs/decisions/0012-kernel-service-registration.md
@@ -4,9 +4,10 @@ status: accepted
contact: dmytrostruk
date: 2023-10-03
deciders: dmytrostruk
-consulted: semenshi, rbarreto, markwallace-microsoft
-informed:
+consulted: SergeyMenshykh, RogerBarreto, markwallace-microsoft
+informed:
---
+
# Kernel Service Registration
## Context and Problem Statement
@@ -82,12 +83,12 @@ Custom service collection and service provider on Kernel level to simplify depen
Interface `IKernel` will have its own service provider `KernelServiceProvider` with minimal functionality to get required service.
```csharp
-public interface IKernelServiceProvider
+public interface IKernelServiceProvider
{
T? GetService(string? name = null);
-}
+}
-public interface IKernel
+public interface IKernel
{
IKernelServiceProvider Services { get; }
}
@@ -174,7 +175,7 @@ Cons:
- Additional dependency for Semantic Kernel package - `Microsoft.Extensions.DependencyInjection`.
- No possibility to include specific list of services (lack of isolation from host application).
-- Possibility of `Microsoft.Extensions.DependencyInjection` version mismatch and runtime errors (e.g. users have `Microsoft.Extensions.DependencyInjection` `--version 2.0` while Semantic Kernel uses `--version 6.0`)
+- Possibility of `Microsoft.Extensions.DependencyInjection` version mismatch and runtime errors (e.g. users have `Microsoft.Extensions.DependencyInjection` `--version 2.0` while Semantic Kernel uses `--version 6.0`)
## Decision Outcome
diff --git a/docs/decisions/0014-chat-completion-roles-in-prompt.md b/docs/decisions/0014-chat-completion-roles-in-prompt.md
index 3f4b39b12fab..354367c27746 100644
--- a/docs/decisions/0014-chat-completion-roles-in-prompt.md
+++ b/docs/decisions/0014-chat-completion-roles-in-prompt.md
@@ -1,146 +1,165 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
-contact: semenshi
+contact: SergeyMenshykh
date: 2023-10-23
-deciders: markwallace-microsoft, mabolan
+deciders: markwallace-microsoft, matthewbolanos
consulted:
informed:
---
+
# SK prompt syntax for chat completion roles
## Context and Problem Statement
+
Today, SK does not have the ability to mark a block of text in a prompt as a message with a specific role, such as assistant, system, or user. As a result, SK can't chunk the prompt into the list of messages required by chat completion connectors.
Additionally, prompts can be defined using a range of template syntaxes supported by various template engines, such as Handlebars, Jinja, and others. Each of these syntaxes may represent chat messages or roles in a distinct way. Consequently, the template engine syntax may leak into SK's domain if no proper abstraction is put in place, coupling SK with the template engines and making it impossible to support new ones.
+
## Decision Drivers
-* It should be possible to mark a block of text in a prompt as a message with a role so that it can be converted into a list of chat messages for use by chat completion connectors.
-* The syntax specific to the template engine message/role should be mapped to the SK message/role syntax to abstract SK from a specific template engine syntax.
+
+- It should be possible to mark a block of text in a prompt as a message with a role so that it can be converted into a list of chat messages for use by chat completion connectors.
+- The syntax specific to the template engine message/role should be mapped to the SK message/role syntax to abstract SK from a specific template engine syntax.
## Considered Options
+
**1. Message/role tags are generated by functions specified in a prompt.** This option relies on the fact that many template engines can invoke functions specified in the template. Therefore, an internal function can be registered with a template engine, and the function will create a message/model tag based on the provided arguments. The prompt template engine will execute the function and emit the function result into the prompt template, and the rendered prompt will have a section for each message/role decorated with these tags. Here's an example of how this can be done using the SK basic template engine and Handlebars:
- Function:
- ```csharp
- internal class SystemFunctions
- {
- public string Message(string role)
- {
- return $"";
- }
- }
- ```
-
- Prompt:
-
- ```bash
- {{message role="system"}}
- You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
- {{message role="system"}}
-
- {{message role="user"}}
- I want to {{$input}}
- {{message role="user"}}
- ```
-
- Rendered prompt:
-
- ```xml
-
- You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
-
-
- I want to buy a house.
-
- ```
-
-**2. Message/role tags are generated by a prompt-specific mechanism.** This option utilizes template engine syntax constructions, helpers, and handlers other than functions to inject SK message/role tags into the final prompt.
- In the example below, to parse the prompt that uses the handlebars syntax we need to register a block helper (a callback that is invoked when the Handlebars engine encounters it) to emit the SK message/role tags in the resulting prompt.
-
- Block helpers:
- ```csharp
- this.handlebarsEngine.RegisterHelper("system", (EncodedTextWriter output, Context context, Arguments arguments) => {
- //Emit the tags
- });
- this.handlebarsEngine.RegisterHelper("user", (EncodedTextWriter output, Context context, Arguments arguments) => {
- //Emit the tags
- });
- ```
-
- Prompt:
- ```bash
- {{#system~}}
- You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
- {{~/system}}
- {{#user~}}
- I want to {{$input}}
- {{~/user}}
- ```
-
- Rendered prompt:
- ```xml
-
- You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
-
-
- I want to buy a house.
-
- ```
+Function:
+
+```csharp
+internal class SystemFunctions
+{
+ public string Message(string role)
+ {
+ return $"";
+ }
+}
+```
+
+Prompt:
+
+```bash
+{{message role="system"}}
+You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
+{{message role="system"}}
+
+{{message role="user"}}
+I want to {{$input}}
+{{message role="user"}}
+```
+
+Rendered prompt:
+
+```xml
+
+You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
+
+
+I want to buy a house.
+
+```
+
+**2. Message/role tags are generated by a prompt-specific mechanism.** This option utilizes template engine syntax constructions, helpers, and handlers other than functions to inject SK message/role tags into the final prompt.
+In the example below, to parse the prompt that uses the handlebars syntax we need to register a block helper (a callback that is invoked when the Handlebars engine encounters it) to emit the SK message/role tags in the resulting prompt.
+
+Block helpers:
+
+```csharp
+this.handlebarsEngine.RegisterHelper("system", (EncodedTextWriter output, Context context, Arguments arguments) => {
+ //Emit the tags
+});
+this.handlebarsEngine.RegisterHelper("user", (EncodedTextWriter output, Context context, Arguments arguments) => {
+ //Emit the tags
+});
+```
+
+Prompt:
+
+```bash
+{{#system~}}
+You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
+{{~/system}}
+{{#user~}}
+I want to {{$input}}
+{{~/user}}
+```
+
+Rendered prompt:
+
+```xml
+
+You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
+
+
+I want to buy a house.
+
+```
**3. Message/role tags are applied on top of prompt template engine**. This option presumes specifying the SK message/role tags directly in a prompt to denote message/role blocks in way that template engine does not parse/handle them and considers them as a regular text.
- In the example below, the prompt the `` tags are marking boundaries of the system and user messages and SK basic template engine consider them as regular text without processing them.
-
- Prompt:
- ```xml
-
- You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
-
-
- I want to {{$input}}
-
- ```
-
- Rendered prompt:
- ```xml
-
- You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
-
-
- I want to buy a house.
-
- ```
+In the example below, the prompt the `` tags are marking boundaries of the system and user messages and SK basic template engine consider them as regular text without processing them.
+
+Prompt:
+
+```xml
+
+You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
+
+
+I want to {{$input}}
+
+```
+
+Rendered prompt:
+
+```xml
+
+You are a bank manager. Be helpful, respectful, appreciate diverse language styles.
+
+
+I want to buy a house.
+
+```
## Pros and Cons
+
**1. Message/role tags are generated by functions specified in a prompt**
-
- Pros:
- * Functions can be defined once and reused in prompt templates that support function calling.
- Cons:
- * Functions might not be supported by some template engines.
- * The system/internal functions should be pre-registered by SK so users don't need to import them.
- * Each prompt template engine will have how to discover and call the system/internal functions.
+Pros:
+
+- Functions can be defined once and reused in prompt templates that support function calling.
+
+Cons:
+
+- Functions might not be supported by some template engines.
+- The system/internal functions should be pre-registered by SK so users don't need to import them.
+- Each prompt template engine will have how to discover and call the system/internal functions.
**2. Message/role tags are generated by prompt specific mechanism**
- Pros:
- * Enables message/role representation with the optimal template engine syntax constructions, aligning with other constructions for that specific engine.
-
- Cons:
- * Each prompt template engine will have to register callbacks/handlers to handle template syntax constructions rendering to emit SK message/role tags.
+Pros:
+
+- Enables message/role representation with the optimal template engine syntax constructions, aligning with other constructions for that specific engine.
+
+Cons:
+
+- Each prompt template engine will have to register callbacks/handlers to handle template syntax constructions rendering to emit SK message/role tags.
**3. Message/role tags are applied on top of prompt template engine**
- Pros:
- * No changes are required to prompt template engines.
+Pros:
- Cons:
- * The message/role tag syntax may not align with other syntax constructions for that template engine.
- * Syntax errors in message/role tags will be detected by components parsing the prompt and not by prompt template engines.
+- No changes are required to prompt template engines.
+
+Cons:
+
+- The message/role tag syntax may not align with other syntax constructions for that template engine.
+- Syntax errors in message/role tags will be detected by components parsing the prompt and not by prompt template engines.
## Decision Outcome
+
It was agreed not to limit ourselves to only one possible option because it may not be feasible to apply that option to new template engines we might need to support in the future. Instead, each time a new template engine is added, every option should be considered, and the optimal one should be preferred for that particular template engine.
-It was also agreed that, at the moment, we will go with the "3. Message/role tags are applied on top of the prompt template engine" option to support the message/role prompt syntax in SK, which currently uses the `BasicPromptTemplateEngine` engine.
\ No newline at end of file
+It was also agreed that, at the moment, we will go with the "3. Message/role tags are applied on top of the prompt template engine" option to support the message/role prompt syntax in SK, which currently uses the `BasicPromptTemplateEngine` engine.
diff --git a/docs/decisions/0015-completion-service-selection.md b/docs/decisions/0015-completion-service-selection.md
index c669d03798fa..624fcfd886b0 100644
--- a/docs/decisions/0015-completion-service-selection.md
+++ b/docs/decisions/0015-completion-service-selection.md
@@ -1,24 +1,31 @@
---
# These are optional elements. Feel free to remove any of them.
status: accepted
+contact: SergeyMenshykh
date: 2023-10-25
-deciders: markwallace, mabolan
+deciders: markwallace-microsoft, matthewbolanos
consulted:
informed:
---
+
# Completion service type selection strategy
## Context and Problem Statement
+
Today, SK runs all text prompts using the text completion service. With the addition of a new chat completion prompts and potentially other prompt types, such as image, on the horizon, we need a way to select a completion service type to run these prompts.
+
## Decision Drivers
-* Semantic function should be able to identify a completion service type to use when processing text, chat, or image prompts.
+
+- Semantic function should be able to identify a completion service type to use when processing text, chat, or image prompts.
## Considered Options
+
**1. Completion service type identified by the "prompt_type" property.** This option presumes adding the 'prompt_type' property to the prompt template config model class, 'PromptTemplateConfig.' The property will be specified once by a prompt developer and will be used by the 'SemanticFunction' class to decide which completion service type (not instance) to use when resolving an instance of that particular completion service type.
**Prompt template**
+
```json
{
"schema": "1",
@@ -29,12 +36,13 @@ Today, SK runs all text prompts using the text completion service. With the addi
```
**Semantic function pseudocode**
+
```csharp
if(string.IsNullOrEmpty(promptTemplateConfig.PromptType) || promptTemplateConfig.PromptType == "text")
{
var service = this._serviceSelector.SelectAIService(context.ServiceProvider, this._modelSettings);
//render the prompt, call the service, process and return result
-}
+}
else (promptTemplateConfig.PromptType == "chat")
{
var service = this._serviceSelector.SelectAIService(context.ServiceProvider, this._modelSettings);
@@ -68,12 +76,12 @@ config: {
```
Pros:
- - Deterministically specifies which completion service **type** to use, so image prompts won't be rendered by a text completion service, and vice versa.
-Cons:
- - Another property to specify by a prompt developer.
+- Deterministically specifies which completion service **type** to use, so image prompts won't be rendered by a text completion service, and vice versa.
+Cons:
+- Another property to specify by a prompt developer.
**2. Completion service type identified by prompt content.** The idea behind this option is to analyze the rendered prompt by using regex to check for the presence of specific markers associated with the prompt type. For example, the presence of the `` tag in the rendered prompt might indicate that the prompt is a chat prompt and should be handled by the chat completion service. This approach may work reliably when we have two completion service types - text and chat - since the logic would be straightforward: if the message tag is found in the rendered prompt, handle it with the chat completion service; otherwise, use the text completion service. However, this logic becomes unreliable when we start adding new prompt types, and those prompts lack markers specific to their prompt type. For example, if we add an image prompt, we won't be able to distinguish between a text prompt and an image prompt unless the image prompt has a unique marker identifying it as such.
@@ -107,11 +115,15 @@ config: {
...
}
```
+
Pros:
+
- No need for a new property to identify the prompt type.
Cons:
+
- Unreliable unless the prompt contains unique markers specifically identifying the prompt type.
## Decision Outcome
+
We decided to choose the '2. Completion service type identified by prompt content' option and will reconsider it when we encounter another completion service type that cannot be supported by this option or when we have a solid set of requirements for using a different mechanism for selecting the completion service type.
diff --git a/docs/decisions/0016-custom-prompt-template-formats.md b/docs/decisions/0016-custom-prompt-template-formats.md
new file mode 100644
index 000000000000..c5b39fdfa805
--- /dev/null
+++ b/docs/decisions/0016-custom-prompt-template-formats.md
@@ -0,0 +1,289 @@
+---
+status: approved
+contact: markwallace-microsoft
+date: 2023-10-26
+deciders: matthewbolanos, markwallace-microsoft, SergeyMenshykh, RogerBarreto
+consulted: dmytrostruk
+informed:
+---
+
+# Custom Prompt Template Formats
+
+## Context and Problem Statement
+
+Semantic Kernel currently supports a custom prompt template language that allows for variable interpolation and function execution.
+Semantic Kernel allows for custom prompt template formats to be integrated e.g., prompt templates using [Handlebars](https://handlebarsjs.com/) syntax.
+
+The purpose of this ADR is to describe how a custom prompt template formats will be supported in the Semantic Kernel.
+
+### Current Design
+
+By default the `Kernel` uses the `BasicPromptTemplateEngine` which supports the Semantic Kernel specific template format.
+
+#### Code Patterns
+
+Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
+
+```csharp
+IKernel kernel = Kernel.Builder
+ .WithPromptTemplateEngine(new BasicPromptTemplateEngine())
+ .WithOpenAIChatCompletionService(
+ modelId: openAIModelId,
+ apiKey: openAIApiKey)
+ .Build();
+
+kernel.ImportFunctions(new TimePlugin(), "time");
+
+string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
+var promptTemplateConfig = new PromptTemplateConfig();
+var promptTemplate = new PromptTemplate(templateString, promptTemplateConfig, kernel.PromptTemplateEngine);
+var kindOfDay = kernel.RegisterSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate);
+
+var result = await kernel.RunAsync(kindOfDay);
+Console.WriteLine(result.GetValue());
+```
+
+We have an extension method `var kindOfDay = kernel.CreateSemanticFunction(promptTemplate);` to simplify the process to create and register a semantic function but the expanded format is shown above to highlight the dependency on `kernel.PromptTemplateEngine`.
+Also the `BasicPromptTemplateEngine` is the default prompt template engine and will be loaded automatically if the package is available and not other prompt template engine is specified.
+
+Some issues with this:
+
+1. `Kernel` only supports a single `IPromptTemplateEngine` so we cannot support using multiple prompt templates at the same time.
+1. `IPromptTemplateEngine` is stateless and must perform a parse of the template for each render
+1. Our semantic function extension methods relay on our implementation of `IPromptTemplate` (i.e., `PromptTemplate`) which stores the template string and uses the `IPromptTemplateEngine` to render it every time. Note implementations of `IPromptTemplate` are currently stateful as they also store the parameters.
+
+#### Performance
+
+The `BasicPromptTemplateEngine` uses the `TemplateTokenizer` to parse the template i.e. extract the blocks.
+Then it renders the template i.e. inserts variables and executes functions. Some sample timings for these operations:
+
+| Operation | Ticks | Milliseconds |
+| ---------------- | ------- | ------------ |
+| Extract blocks | 1044427 | 103 |
+| Render variables | 168 | 0 |
+
+Sample template used was: `"{{variable1}} {{variable2}} {{variable3}} {{variable4}} {{variable5}}"`
+
+**Note: We will use the sample implementation to support the f-string template format.**
+
+Using `HandlebarsDotNet` for the same use case results in the following timings:
+
+| Operation | Ticks | Milliseconds |
+| ---------------- | ----- | ------------ |
+| Compile template | 66277 | 6 |
+| Render variables | 4173 | 0 |
+
+**By separating the extract blocks/compile from the render variables operation it will be possible to optimise performance by compiling templates just once.**
+
+#### Implementing a Custom Prompt Template Engine
+
+There are two interfaces provided:
+
+```csharp
+public interface IPromptTemplateEngine
+{
+ Task RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default);
+}
+
+public interface IPromptTemplate
+{
+ IReadOnlyList Parameters { get; }
+
+ public Task RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default);
+}
+```
+
+A prototype implementation of a handlebars prompt template engine could look something like this:
+
+```csharp
+public class HandlebarsTemplateEngine : IPromptTemplateEngine
+{
+ private readonly ILoggerFactory _loggerFactory;
+
+ public HandlebarsTemplateEngine(ILoggerFactory? loggerFactory = null)
+ {
+ this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
+ }
+
+ public async Task RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default)
+ {
+ var handlebars = HandlebarsDotNet.Handlebars.Create();
+
+ var functionViews = context.Functions.GetFunctionViews();
+ foreach (FunctionView functionView in functionViews)
+ {
+ var skfunction = context.Functions.GetFunction(functionView.PluginName, functionView.Name);
+ handlebars.RegisterHelper($"{functionView.PluginName}_{functionView.Name}", async (writer, hcontext, parameters) =>
+ {
+ var result = await skfunction.InvokeAsync(context).ConfigureAwait(true);
+ writer.WriteSafeString(result.GetValue());
+ });
+ }
+
+ var template = handlebars.Compile(templateText);
+
+ var prompt = template(context.Variables);
+
+ return await Task.FromResult(prompt).ConfigureAwait(true);
+ }
+}
+```
+
+**Note: This is just a prototype implementation for illustration purposes only.**
+
+Some issues:
+
+1. The `IPromptTemplate` interface is not used and causes confusion.
+1. There is no way to allow developers to support multiple prompt template formats at the same time.
+
+There is one implementation of `IPromptTemplate` provided in the Semantic Kernel core package.
+The `RenderAsync` implementation just delegates to the `IPromptTemplateEngine`.
+The `Parameters` list get's populated with the parameters defined in the `PromptTemplateConfig` and any missing variables defined in the template.
+
+#### Handlebars Considerations
+
+Handlebars does not support dynamic binding of helpers. Consider the following snippet:
+
+```csharp
+HandlebarsHelper link_to = (writer, context, parameters) =>
+{
+ writer.WriteSafeString($"{context["text"]}");
+};
+
+string source = @"Click here: {{link_to}}";
+
+var data = new
+{
+ url = "https://github.com/rexm/handlebars.net",
+ text = "Handlebars.Net"
+};
+
+// Act
+var handlebars = HandlebarsDotNet.Handlebars.Create();
+handlebars.RegisterHelper("link_to", link_to);
+var template = handlebars1.Compile(source);
+// handlebars.RegisterHelper("link_to", link_to); This also works
+var result = template1(data);
+```
+
+Handlebars allows the helpers to be registered with the `Handlebars` instance either before or after a template is compiled.
+The optimum would be to have a shared `Handlebars` instance for a specific collection of functions and register the helpers just once.
+For use cases where the Kernel function collection may have been mutated we will be forced to create a `Handlebars` instance at render time
+and then register the helpers. This means we cannot take advantage of the performance improvement provided by compiling the template.
+
+## Decision Drivers
+
+In no particular order:
+
+- Support creating a semantic function without a `IKernel`instance.
+- Support late binding of functions i.e., having functions resolved when the prompt is rendered.
+- Support allowing the prompt template to be parsed (compiled) just once to optimize performance if needed.
+- Support using multiple prompt template formats with a single `Kernel` instance.
+- Provide simple abstractions which allow third parties to implement support for custom prompt template formats.
+
+## Considered Options
+
+- Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`.
+-
+
+### Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`
+
+
+
+Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
+
+```csharp
+// Semantic function can be created once
+var promptTemplateFactory = new BasicPromptTemplateFactory();
+string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
+var promptTemplateConfig = new PromptTemplateConfig();
+// Line below will replace the commented out code
+var promptTemplate = promptTemplateFactory.CreatePromptTemplate(templateString, promptTemplateConfig);
+var kindOfDay = ISKFunction.CreateSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate)
+// var promptTemplate = new PromptTemplate(promptTemplate, promptTemplateConfig, kernel.PromptTemplateEngine);
+// var kindOfDay = kernel.RegisterSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate);
+
+// Create Kernel after creating the semantic function
+// Later we will support passing a function collection to the KernelBuilder
+IKernel kernel = Kernel.Builder
+ .WithOpenAIChatCompletionService(
+ modelId: openAIModelId,
+ apiKey: openAIApiKey)
+ .Build();
+
+kernel.ImportFunctions(new TimePlugin(), "time");
+// Optionally register the semantic function with the Kernel
+kernel.RegisterCustomFunction(kindOfDay);
+
+var result = await kernel.RunAsync(kindOfDay);
+Console.WriteLine(result.GetValue());
+```
+
+**Notes:**
+
+- `BasicPromptTemplateFactory` will be the default implementation and will be automatically provided in `KernelSemanticFunctionExtensions`. Developers will also be able to provide their own implementation.
+- The factory uses the new `PromptTemplateConfig.TemplateFormat` to create the appropriate `IPromptTemplate` instance.
+- We should look to remove `promptTemplateConfig` as a parameter to `CreateSemanticFunction`. That change is outside of the scope of this ADR.
+
+The `BasicPromptTemplateFactory` and `BasicPromptTemplate` implementations look as follows:
+
+```csharp
+public sealed class BasicPromptTemplateFactory : IPromptTemplateFactory
+{
+ private readonly IPromptTemplateFactory _promptTemplateFactory;
+ private readonly ILoggerFactory _loggerFactory;
+
+ public BasicPromptTemplateFactory(IPromptTemplateFactory promptTemplateFactory, ILoggerFactory? loggerFactory = null)
+ {
+ this._promptTemplateFactory = promptTemplateFactory;
+ this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
+ }
+
+ public IPromptTemplate? CreatePromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig)
+ {
+ if (promptTemplateConfig.TemplateFormat.Equals(PromptTemplateConfig.SEMANTICKERNEL, System.StringComparison.Ordinal))
+ {
+ return new BasicPromptTemplate(templateString, promptTemplateConfig, this._loggerFactory);
+ }
+ else if (this._promptTemplateFactory is not null)
+ {
+ return this._promptTemplateFactory.CreatePromptTemplate(templateString, promptTemplateConfig);
+ }
+
+ throw new SKException($"Invalid prompt template format {promptTemplateConfig.TemplateFormat}");
+ }
+}
+
+public sealed class BasicPromptTemplate : IPromptTemplate
+{
+ public BasicPromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig, ILoggerFactory? loggerFactory = null)
+ {
+ this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
+ this._logger = this._loggerFactory.CreateLogger(typeof(BasicPromptTemplate));
+ this._templateString = templateString;
+ this._promptTemplateConfig = promptTemplateConfig;
+ this._parameters = new(() => this.InitParameters());
+ this._blocks = new(() => this.ExtractBlocks(this._templateString));
+ this._tokenizer = new TemplateTokenizer(this._loggerFactory);
+ }
+
+ public IReadOnlyList Parameters => this._parameters.Value;
+
+ public async Task RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default)
+ {
+ return await this.RenderAsync(this._blocks.Value, executionContext, cancellationToken).ConfigureAwait(false);
+ }
+
+ // Not showing the implementation details
+}
+```
+
+**Note:**
+
+- The call to `ExtractBlocks` is called lazily once for each prompt template
+- The `RenderAsync` doesn't need to extract the blocks every time
+
+## Decision Outcome
+
+Chosen option: "Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`", because
+addresses the requirements and provides good flexibility for the future.
diff --git a/docs/decisions/0016-semantic-function-multiple-model-support.md b/docs/decisions/0016-semantic-function-multiple-model-support.md
deleted file mode 100644
index bb2a3a72ae33..000000000000
--- a/docs/decisions/0016-semantic-function-multiple-model-support.md
+++ /dev/null
@@ -1,173 +0,0 @@
----
-# These are optional elements. Feel free to remove any of them.
-status: approved
-contact: markwallace-microsoft
-date: 2023-10-26
-deciders: markwallace-microsoft, semenshi, rogerbarreto
-consulted: mabolan, dmytrostruk
-informed:
----
-# Multiple Model Support for Semantic Functions
-
-## Context and Problem Statement
-
-Developers need to be able to use multiple models simultaneously e.g., using GPT4 for certain prompts and GPT3.5 for others to reduce cost.
-
-## Use Cases
-
-In scope for Semantic Kernel V1.0 is the ability to select AI Service and Model Request Settings:
-
-1. By service id.
- * A Service id uniquely identifies a registered AI Service and is typically defined in the scope of an application.
-1. By developer defined strategy.
- * A _developer defined strategy_ is a code first approach where a developer provides the logic.
-1. By model id.
- * A model id uniquely identifies a Large Language Model. Multiple AI service providers can support the same LLM.
-1. By arbitrary AI service attributes
- * E.g. an AI service can define a provider id which uniquely identifies an AI provider e.g. "Azure OpenAI", "OpenAI", "Hugging Face"
-
-**This ADR focuses on items 1 & 2 in the above list. To implement 3 & 4 we need to provide the ability to store `AIService` metadata.**
-
-## Decision Outcome
-
-Support use cases 1 & 2 listed in this ADR and create separate ADR to add support for AI service metadata.
-
-## Descriptions of the Use Cases
-
-**Note: All code is pseudo code and does not accurately reflect what the final implementations will look like.**
-
-### Select Model Request Settings by Service Id
-
-_As a developer using the Semantic Kernel I can configure multiple request settings for a semantic function and associate each one with a service id so that the correct request settings are used when different services are used to execute my semantic function._
-
-The semantic function template configuration allows multiple model request settings to be configured. In this case the developer configures different settings based on the service id that is used to execute the semantic function.
-In the example below the semantic function is executed with "AzureText" using `max_tokens=60` because "AzureText" is the first service id in the list of models configured for the prompt.
-
-```csharp
-// Configure a Kernel with multiple LLM's
-IKernel kernel = new KernelBuilder()
- .WithLoggerFactory(ConsoleLogger.LoggerFactory)
- .WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
- endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
- .WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
- endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
- .WithOpenAITextCompletionService(modelId: oai.ModelId,
- serviceId: "OpenAIText", apiKey: oai.ApiKey, setAsDefault: true)
- .WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
- serviceId: "OpenAIChat", apiKey: oai.ApiKey, setAsDefault: true)
- .Build();
-
-// Configure semantic function with multiple LLM request settings
-var modelSettings = new List
-{
- new OpenAIRequestSettings() { ServiceId = "AzureText", MaxTokens = 60 },
- new OpenAIRequestSettings() { ServiceId = "AzureChat", MaxTokens = 120 },
- new OpenAIRequestSettings() { ServiceId = "OpenAIText", MaxTokens = 180 },
- new OpenAIRequestSettings() { ServiceId = "OpenAIChat", MaxTokens = 240 }
-};
-var prompt = "Hello AI, what can you do for me?";
-var promptTemplateConfig = new PromptTemplateConfig() { ModelSettings = modelSettings };
-var func = kernel.CreateSemanticFunction(prompt, config: promptTemplateConfig, "HelloAI");
-
-// Semantic function is executed with AzureText using max_tokens=60
-result = await kernel.RunAsync(func);
-```
-
-This works by using the `IAIServiceSelector` interface as the strategy for selecting the AI service and request settings to user when invoking a semantic function.
-The interface is defined as follows:
-
-```csharp
-public interface IAIServiceSelector
-{
- (T?, AIRequestSettings?) SelectAIService(
- string renderedPrompt,
- IAIServiceProvider serviceProvider,
- IReadOnlyList? modelSettings) where T : IAIService;
-}
-```
-
-A default `OrderedIAIServiceSelector` implementation is provided which selects the AI service based on the order of the model request settings defined for the semantic function.
-
-* The implementation checks if a service exists which the corresponding service id and if it does it and the associated model request settings will be used.
-* In no model request settings are defined then the default text completion service is used.
-* A default set of request settings can be specified by leaving the service id undefined or empty, the first such default will be used.
-* If no default if specified and none of the specified services are available the operation will fail.
-
-### Select AI Service and Model Request Settings By Developer Defined Strategy
-
-_As a developer using the Semantic Kernel I can provide an implementation which selects the AI service and request settings used to execute my function so that I can dynamically control which AI service and settings are used to execute my semantic function._
-
-In this case the developer configures different settings based on the service id and provides an AI Service Selector which determines which AI Service will be used when the semantic function is executed.
-In the example below the semantic function is executed with whatever AI Service and AI Request Settings `MyAIServiceSelector` returns e.g. it will be possible to create an AI Service Selector that computes the token count of the rendered prompt and uses that to determine which service to use.
-
-```csharp
-// Configure a Kernel with multiple LLM's
-IKernel kernel = new KernelBuilder()
- .WithLoggerFactory(ConsoleLogger.LoggerFactory)
- .WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
- endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
- .WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
- endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
- .WithOpenAITextCompletionService(modelId: oai.ModelId,
- serviceId: "OpenAIText", apiKey: oai.ApiKey, setAsDefault: true)
- .WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
- serviceId: "OpenAIChat", apiKey: oai.ApiKey, setAsDefault: true)
- .WithAIServiceSelector(new MyAIServiceSelector())
- .Build();
-
-// Configure semantic function with multiple LLM request settings
-var modelSettings = new List
-{
- new OpenAIRequestSettings() { ServiceId = "AzureText", MaxTokens = 60 },
- new OpenAIRequestSettings() { ServiceId = "AzureChat", MaxTokens = 120 },
- new OpenAIRequestSettings() { ServiceId = "OpenAIText", MaxTokens = 180 },
- new OpenAIRequestSettings() { ServiceId = "OpenAIChat", MaxTokens = 240 }
-};
-var prompt = "Hello AI, what can you do for me?";
-var promptTemplateConfig = new PromptTemplateConfig() { ModelSettings = modelSettings };
-var func = kernel.CreateSemanticFunction(prompt, config: promptTemplateConfig, "HelloAI");
-
-// Semantic function is executed with AI Service and AI request Settings dynamically determined
-result = await kernel.RunAsync(func, funcVariables);
-```
-
-## More Information
-
-### Select AI Service by Service Id
-
-The following use case is supported. Developers can create a `Kernel`` instance with multiple named AI services. When invoking a semantic function the service id (and optionally request settings to be used) can be specified. The named AI service will be used to execute the prompt.
-
-```csharp
-var aoai = TestConfiguration.AzureOpenAI;
-var oai = TestConfiguration.OpenAI;
-
-// Configure a Kernel with multiple LLM's
-IKernel kernel = Kernel.Builder
- .WithLoggerFactory(ConsoleLogger.LoggerFactory)
- .WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
- endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
- .WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
- endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
- .WithOpenAITextCompletionService(modelId: oai.ModelId,
- serviceId: "OpenAIText", apiKey: oai.ApiKey)
- .WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
- serviceId: "OpenAIChat", apiKey: oai.ApiKey)
- .Build();
-
-// Invoke the semantic function and service and request settings to use
-result = await kernel.InvokeSemanticFunctionAsync(prompt,
- requestSettings: new OpenAIRequestSettings()
- { ServiceId = "AzureText", MaxTokens = 60 });
-
-result = await kernel.InvokeSemanticFunctionAsync(prompt,
- requestSettings: new OpenAIRequestSettings()
- { ServiceId = "AzureChat", MaxTokens = 120 });
-
-result = await kernel.InvokeSemanticFunctionAsync(prompt,
- requestSettings: new OpenAIRequestSettings()
- { ServiceId = "OpenAIText", MaxTokens = 180 });
-
-result = await kernel.InvokeSemanticFunctionAsync(prompt,
- requestSettings: new OpenAIRequestSettings()
- { ServiceId = "OpenAIChat", MaxTokens = 240 });
-```
diff --git a/docs/decisions/0017-custom-prompt-template-formats.md b/docs/decisions/0017-custom-prompt-template-formats.md
deleted file mode 100644
index 755d19a75472..000000000000
--- a/docs/decisions/0017-custom-prompt-template-formats.md
+++ /dev/null
@@ -1,284 +0,0 @@
----
-status: approved
-contact: markwallace-microsoft
-date: 2023-10-26
-deciders: mabolan, markwallace-microsoft, semenshi, rbarreto
-consulted: dmytrostruk
-informed:
----
-# Custom Prompt Template Formats
-
-## Context and Problem Statement
-
-Semantic Kernel currently supports a custom prompt template language that allows for variable interpolation and function execution.
-Semantic Kernel allows for custom prompt template formats to be integrated e.g., prompt templates using [Handlebars](https://handlebarsjs.com/) syntax.
-
-The purpose of this ADR is to describe how a custom prompt template formats will be supported in the Semantic Kernel.
-
-### Current Design
-
-By default the `Kernel` uses the `BasicPromptTemplateEngine` which supports the Semantic Kernel specific template format.
-
-#### Code Patterns
-
-Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
-
-```csharp
-IKernel kernel = Kernel.Builder
- .WithPromptTemplateEngine(new BasicPromptTemplateEngine())
- .WithOpenAIChatCompletionService(
- modelId: openAIModelId,
- apiKey: openAIApiKey)
- .Build();
-
-kernel.ImportFunctions(new TimePlugin(), "time");
-
-string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
-var promptTemplateConfig = new PromptTemplateConfig();
-var promptTemplate = new PromptTemplate(templateString, promptTemplateConfig, kernel.PromptTemplateEngine);
-var kindOfDay = kernel.RegisterSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate);
-
-var result = await kernel.RunAsync(kindOfDay);
-Console.WriteLine(result.GetValue());
-```
-
-We have an extension method `var kindOfDay = kernel.CreateSemanticFunction(promptTemplate);` to simplify the process to create and register a semantic function but the expanded format is shown above to highlight the dependency on `kernel.PromptTemplateEngine`.
-Also the `BasicPromptTemplateEngine` is the default prompt template engine and will be loaded automatically if the package is available and no other prompt template engine is specified.
-
-Some issues with this:
-
-1. You need to have a `Kernel` instance to create a semantic function, which is contrary to one of the goals of allow semantic functions to be created once and reused across multiple `Kernel` instances.
-1. `Kernel` only supports a single `IPromptTemplateEngine` so we cannot support using multiple prompt templates at the same time.
-1. `IPromptTemplateEngine` is stateless and must perform a parse of the template for each render
-1. Our semantic function extension methods rely on our implementation of `IPromptTemplate` (i.e., `PromptTemplate`) which stores the template string and uses the `IPromptTemplateEngine` to render it every time. Note implementations of `IPromptTemplate` are currently stateful as they also store the parameters.
-
-#### Performance
-
-The `BasicPromptTemplateEngine` uses the `TemplateTokenizer` to parse the template i.e. extract the blocks.
-Then it renders the template i.e. inserts variables and executes functions. Some sample timings for these operations:
-
-| Operation | Ticks | Milliseconds |
-|------------------|---------|--------------|
-| Extract blocks | 1044427 | 103 |
-| Render variables | 168 | 0 |
-
-Sample template used was: `"{{variable1}} {{variable2}} {{variable3}} {{variable4}} {{variable5}}"`
-
-**Note: We will use the sample implementation to support the f-string template format.**
-
-Using `HandlebarsDotNet` for the same use case results in the following timings:
-
-| Operation | Ticks | Milliseconds |
-|------------------|---------|--------------|
-| Compile template | 66277 | 6 |
-| Render variables | 4173 | 0 |
-
-**By separating the extract blocks/compile from the render variables operation it will be possible to optimise performance by compiling templates just once.**
-
-#### Implementing a Custom Prompt Template Engine
-
-There are two interfaces provided:
-
-```csharp
-public interface IPromptTemplateEngine
-{
- Task RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default);
-}
-
-public interface IPromptTemplate
-{
- IReadOnlyList Parameters { get; }
-
- public Task RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default);
-}
-```
-
-A prototype implementation of a handlebars prompt template engine could look something like this:
-
-```csharp
-public class HandlebarsTemplateEngine : IPromptTemplateEngine
-{
- private readonly ILoggerFactory _loggerFactory;
-
- public HandlebarsTemplateEngine(ILoggerFactory? loggerFactory = null)
- {
- this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
- }
-
- public async Task RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default)
- {
- var handlebars = HandlebarsDotNet.Handlebars.Create();
-
- var functionViews = context.Functions.GetFunctionViews();
- foreach (FunctionView functionView in functionViews)
- {
- var skfunction = context.Functions.GetFunction(functionView.PluginName, functionView.Name);
- handlebars.RegisterHelper($"{functionView.PluginName}_{functionView.Name}", async (writer, hcontext, parameters) =>
- {
- var result = await skfunction.InvokeAsync(context).ConfigureAwait(true);
- writer.WriteSafeString(result.GetValue());
- });
- }
-
- var template = handlebars.Compile(templateText);
-
- var prompt = template(context.Variables);
-
- return await Task.FromResult(prompt).ConfigureAwait(true);
- }
-}
-```
-
-**Note: This is just a prototype implementation for illustration purposes only.**
-
-Some issues:
-
-1. The `IPromptTemplate` interface is not used and causes confusion.
-1. There is no way to allow developers to support multiple prompt template formats at the same time.
-
-There is one implementation of `IPromptTemplate` provided in the Semantic Kernel core package.
-The `RenderAsync` implementation just delegates to the `IPromptTemplateEngine`.
-The `Parameters` list get's populated with the parameters defined in the `PromptTemplateConfig` and any missing variables defined in the template.
-
-#### Handlebars Considerations
-
-Handlebars does not support dynamic binding of helpers. Consider the following snippet:
-
-```csharp
-HandlebarsHelper link_to = (writer, context, parameters) =>
-{
- writer.WriteSafeString($"{context["text"]}");
-};
-
-string source = @"Click here: {{link_to}}";
-
-var data = new
-{
- url = "https://github.com/rexm/handlebars.net",
- text = "Handlebars.Net"
-};
-
-// Act
-var handlebars = HandlebarsDotNet.Handlebars.Create();
-handlebars.RegisterHelper("link_to", link_to);
-var template = handlebars1.Compile(source);
-// handlebars.RegisterHelper("link_to", link_to); This also works
-var result = template1(data);
-```
-
-Handlebars allows the helpers to be registered with the `Handlebars` instance either before or after a template is compiled.
-The optimum would be to have a shared `Handlebars` instance for a specific collection of functions and register the helpers just once.
-For use cases where the Kernel function collection may have been mutated we will be forced to create a `Handlebars` instance at render time
-and then register the helpers. This means we cannot take advantage of the performance improvement provided by compiling the template.
-
-## Decision Drivers
-
-In no particular order:
-
-* Support creating a semantic function without a `IKernel`instance.
-* Support late binding of functions i.e., having functions resolved when the prompt is rendered.
-* Support allowing the prompt template to be parsed (compiled) just once to optimize performance if needed.
-* Support using multiple prompt template formats with a single `Kernel` instance.
-* Provide simple abstractions which allow third parties to implement support for custom prompt template formats.
-
-## Considered Options
-
-* Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`.
-
-### Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`
-
-
-
-Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
-
-```csharp
-// Semantic function can be created once
-var promptTemplateFactory = new BasicPromptTemplateFactory();
-string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
-var promptTemplate = promptTemplateFactory.CreatePromptTemplate(templateString, new PromptTemplateConfig());
-var kindOfDay = ISKFunction.CreateSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate)
-
-// Create Kernel after creating the semantic function
-// Later we will support passing a function collection to the KernelBuilder
-IKernel kernel = Kernel.Builder
- .WithOpenAIChatCompletionService(
- modelId: openAIModelId,
- apiKey: openAIApiKey)
- .Build();
-
-kernel.ImportFunctions(new TimePlugin(), "time");
-// Optionally register the semantic function with the Kernel
-// kernel.RegisterCustomFunction(kindOfDay);
-
-var result = await kernel.RunAsync(kindOfDay);
-Console.WriteLine(result.GetValue());
-```
-
-**Notes:**
-
-* `BasicPromptTemplateFactory` will be the default implementation and will be automatically provided in `KernelSemanticFunctionExtensions`. Developers will also be able to provide their own implementation.
-* The factory uses the new `PromptTemplateConfig.TemplateFormat` to create the appropriate `IPromptTemplate` instance.
-* We should look to remove `promptTemplateConfig` as a parameter to `CreateSemanticFunction`. That change is outside of the scope of this ADR.
-
-The `BasicPromptTemplateFactory` and `BasicPromptTemplate` implementations look as follows:
-
-```csharp
-public sealed class BasicPromptTemplateFactory : IPromptTemplateFactory
-{
- private readonly IPromptTemplateFactory _promptTemplateFactory;
- private readonly ILoggerFactory _loggerFactory;
-
- public BasicPromptTemplateFactory(IPromptTemplateFactory promptTemplateFactory, ILoggerFactory? loggerFactory = null)
- {
- this._promptTemplateFactory = promptTemplateFactory;
- this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
- }
-
- public IPromptTemplate? CreatePromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig)
- {
- if (promptTemplateConfig.TemplateFormat.Equals(PromptTemplateConfig.SEMANTICKERNEL, System.StringComparison.Ordinal))
- {
- return new BasicPromptTemplate(templateString, promptTemplateConfig, this._loggerFactory);
- }
- else if (this._promptTemplateFactory is not null)
- {
- return this._promptTemplateFactory.CreatePromptTemplate(templateString, promptTemplateConfig);
- }
-
- throw new SKException($"Invalid prompt template format {promptTemplateConfig.TemplateFormat}");
- }
-}
-
-public sealed class BasicPromptTemplate : IPromptTemplate
-{
- public BasicPromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig, ILoggerFactory? loggerFactory = null)
- {
- this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
- this._logger = this._loggerFactory.CreateLogger(typeof(BasicPromptTemplate));
- this._templateString = templateString;
- this._promptTemplateConfig = promptTemplateConfig;
- this._parameters = new(() => this.InitParameters());
- this._blocks = new(() => this.ExtractBlocks(this._templateString));
- this._tokenizer = new TemplateTokenizer(this._loggerFactory);
- }
-
- public IReadOnlyList Parameters => this._parameters.Value;
-
- public async Task RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default)
- {
- return await this.RenderAsync(this._blocks.Value, executionContext, cancellationToken).ConfigureAwait(false);
- }
-
- // Not showing the implementation details
-}
-```
-
-**Note:**
-
-* The call to `ExtractBlocks` is called lazily once for each prompt template
-* The `RenderAsync` doesn't need to extract the blocks every time
-
-## Decision Outcome
-
-Chosen option: "Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`", because
-addresses the requirements and provides good flexibility for the future.
diff --git a/docs/decisions/0017-openai-function-calling.md b/docs/decisions/0017-openai-function-calling.md
new file mode 100644
index 000000000000..f0035ce458c1
--- /dev/null
+++ b/docs/decisions/0017-openai-function-calling.md
@@ -0,0 +1,72 @@
+---
+status: accepted
+contact: gitri-ms
+date: 2023-09-21
+deciders: gitri-ms, shawncal
+consulted: lemillermicrosoft, awharrison-28, dmytrostruk, nacharya1
+informed: eavanvalkenburg, kevdome3000
+---
+
+# OpenAI Function Calling Support
+
+## Context and Problem Statement
+
+The [function calling](https://platform.openai.com/docs/guides/gpt/function-calling) capability of OpenAI's Chat Completions API allows developers to describe functions to the model, and have the model decide whether to output a JSON object specifying a function and appropriate arguments to call in response to the given prompt. This capability is enabled by two new API parameters to the `/v1/chat/completions` endpoint:
+
+- `function_call` - auto (default), none, or a specific function to call
+- `functions` - JSON descriptions of the functions available to the model
+
+Functions provided to the model are injected as part of the system message and are billed/counted as input tokens.
+
+We have received several community requests to provide support for this capability when using SK with the OpenAI chat completion models that support it.
+
+## Decision Drivers
+
+- Minimize changes to the core kernel for OpenAI-specific functionality
+- Cost concerns with including a long list of function descriptions in the request
+- Security and cost concerns with automatically executing functions returned by the model
+
+## Considered Options
+
+- Support sending/receiving functions via chat completions endpoint _with_ modifications to interfaces
+- Support sending/receiving functions via chat completions endpoint _without_ modifications to interfaces
+- Implement a planner around the function calling capability
+
+## Decision Outcome
+
+Chosen option: "Support sending/receiving functions via chat completions endpoint _without_ modifications to interfaces"
+
+With this option, we utilize the existing request settings object to send functions to the model. The app developer controls what functions are included and is responsible for validating and executing the function result.
+
+### Consequences
+
+- Good, because avoids breaking changes to the core kernel
+- Good, because OpenAI-specific functionality is contained to the OpenAI connector package
+- Good, because allows app to control what functions are available to the model (including non-SK functions)
+- Good, because keeps the option open for integrating with planners in the future
+- Neutral, because requires app developer to validate and execute resulting function
+- Bad, because not as obvious how to use this capability and access the function results
+
+## Pros and Cons of the Options
+
+### Support sending/receiving functions _with_ modifications to chat completions interfaces
+
+This option would update the `IChatCompletion` and `IChatResult` interfaces to expose parameters/methods for providing and accessing function information.
+
+- Good, because provides a clear path for using the function calling capability
+- Good, because allows app to control what functions are available to the model (including non-SK functions)
+- Neutral, because requires app developer to validate and execute resulting function
+- Bad, because introduces breaking changes to core kernel abstractions
+- Bad, because OpenAI-specific functionality would be included in core kernel abstractions and would need to be ignored by other model providers
+
+### Implement a planner around the function calling capability
+
+Orchestrating external function calls fits within SK's concept of planning. With this approach, we would implement a planner that would take the function calling result and produce a plan that the app developer could execute (similar to SK's ActionPlanner).
+
+- Good, because producing a plan result makes it easy for the app developer to execute the chosen function
+- Bad, because functions would need to be registered with the kernel in order to be executed
+- Bad, because would create confusion about when to use which planner
+
+## Additional notes
+
+There has been much discussion and debate over the pros and cons of automatically invoking a function returned by the OpenAI model, if it is registered with the kernel. As there are still many open questions around this behavior and its implications, we have decided to not include this capability in the initial implementation. We will continue to explore this option and may include it in a future update.
diff --git a/docs/decisions/0018-custom-prompt-template-formats.md b/docs/decisions/0018-custom-prompt-template-formats.md
new file mode 100644
index 000000000000..5cd1f7f90cb4
--- /dev/null
+++ b/docs/decisions/0018-custom-prompt-template-formats.md
@@ -0,0 +1,285 @@
+---
+status: approved
+contact: markwallace-microsoft
+date: 2023-10-26
+deciders: matthewbolanos, markwallace-microsoft, SergeyMenshykh, RogerBarreto
+consulted: dmytrostruk
+informed:
+---
+
+# Custom Prompt Template Formats
+
+## Context and Problem Statement
+
+Semantic Kernel currently supports a custom prompt template language that allows for variable interpolation and function execution.
+Semantic Kernel allows for custom prompt template formats to be integrated e.g., prompt templates using [Handlebars](https://handlebarsjs.com/) syntax.
+
+The purpose of this ADR is to describe how a custom prompt template formats will be supported in the Semantic Kernel.
+
+### Current Design
+
+By default the `Kernel` uses the `BasicPromptTemplateEngine` which supports the Semantic Kernel specific template format.
+
+#### Code Patterns
+
+Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
+
+```csharp
+IKernel kernel = Kernel.Builder
+ .WithPromptTemplateEngine(new BasicPromptTemplateEngine())
+ .WithOpenAIChatCompletionService(
+ modelId: openAIModelId,
+ apiKey: openAIApiKey)
+ .Build();
+
+kernel.ImportFunctions(new TimePlugin(), "time");
+
+string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
+var promptTemplateConfig = new PromptTemplateConfig();
+var promptTemplate = new PromptTemplate(templateString, promptTemplateConfig, kernel.PromptTemplateEngine);
+var kindOfDay = kernel.RegisterSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate);
+
+var result = await kernel.RunAsync(kindOfDay);
+Console.WriteLine(result.GetValue());
+```
+
+We have an extension method `var kindOfDay = kernel.CreateSemanticFunction(promptTemplate);` to simplify the process to create and register a semantic function but the expanded format is shown above to highlight the dependency on `kernel.PromptTemplateEngine`.
+Also the `BasicPromptTemplateEngine` is the default prompt template engine and will be loaded automatically if the package is available and no other prompt template engine is specified.
+
+Some issues with this:
+
+1. You need to have a `Kernel` instance to create a semantic function, which is contrary to one of the goals of allow semantic functions to be created once and reused across multiple `Kernel` instances.
+1. `Kernel` only supports a single `IPromptTemplateEngine` so we cannot support using multiple prompt templates at the same time.
+1. `IPromptTemplateEngine` is stateless and must perform a parse of the template for each render
+1. Our semantic function extension methods rely on our implementation of `IPromptTemplate` (i.e., `PromptTemplate`) which stores the template string and uses the `IPromptTemplateEngine` to render it every time. Note implementations of `IPromptTemplate` are currently stateful as they also store the parameters.
+
+#### Performance
+
+The `BasicPromptTemplateEngine` uses the `TemplateTokenizer` to parse the template i.e. extract the blocks.
+Then it renders the template i.e. inserts variables and executes functions. Some sample timings for these operations:
+
+| Operation | Ticks | Milliseconds |
+| ---------------- | ------- | ------------ |
+| Extract blocks | 1044427 | 103 |
+| Render variables | 168 | 0 |
+
+Sample template used was: `"{{variable1}} {{variable2}} {{variable3}} {{variable4}} {{variable5}}"`
+
+**Note: We will use the sample implementation to support the f-string template format.**
+
+Using `HandlebarsDotNet` for the same use case results in the following timings:
+
+| Operation | Ticks | Milliseconds |
+| ---------------- | ----- | ------------ |
+| Compile template | 66277 | 6 |
+| Render variables | 4173 | 0 |
+
+**By separating the extract blocks/compile from the render variables operation it will be possible to optimise performance by compiling templates just once.**
+
+#### Implementing a Custom Prompt Template Engine
+
+There are two interfaces provided:
+
+```csharp
+public interface IPromptTemplateEngine
+{
+ Task RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default);
+}
+
+public interface IPromptTemplate
+{
+ IReadOnlyList Parameters { get; }
+
+ public Task RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default);
+}
+```
+
+A prototype implementation of a handlebars prompt template engine could look something like this:
+
+```csharp
+public class HandlebarsTemplateEngine : IPromptTemplateEngine
+{
+ private readonly ILoggerFactory _loggerFactory;
+
+ public HandlebarsTemplateEngine(ILoggerFactory? loggerFactory = null)
+ {
+ this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
+ }
+
+ public async Task RenderAsync(string templateText, SKContext context, CancellationToken cancellationToken = default)
+ {
+ var handlebars = HandlebarsDotNet.Handlebars.Create();
+
+ var functionViews = context.Functions.GetFunctionViews();
+ foreach (FunctionView functionView in functionViews)
+ {
+ var skfunction = context.Functions.GetFunction(functionView.PluginName, functionView.Name);
+ handlebars.RegisterHelper($"{functionView.PluginName}_{functionView.Name}", async (writer, hcontext, parameters) =>
+ {
+ var result = await skfunction.InvokeAsync(context).ConfigureAwait(true);
+ writer.WriteSafeString(result.GetValue());
+ });
+ }
+
+ var template = handlebars.Compile(templateText);
+
+ var prompt = template(context.Variables);
+
+ return await Task.FromResult(prompt).ConfigureAwait(true);
+ }
+}
+```
+
+**Note: This is just a prototype implementation for illustration purposes only.**
+
+Some issues:
+
+1. The `IPromptTemplate` interface is not used and causes confusion.
+1. There is no way to allow developers to support multiple prompt template formats at the same time.
+
+There is one implementation of `IPromptTemplate` provided in the Semantic Kernel core package.
+The `RenderAsync` implementation just delegates to the `IPromptTemplateEngine`.
+The `Parameters` list get's populated with the parameters defined in the `PromptTemplateConfig` and any missing variables defined in the template.
+
+#### Handlebars Considerations
+
+Handlebars does not support dynamic binding of helpers. Consider the following snippet:
+
+```csharp
+HandlebarsHelper link_to = (writer, context, parameters) =>
+{
+ writer.WriteSafeString($"{context["text"]}");
+};
+
+string source = @"Click here: {{link_to}}";
+
+var data = new
+{
+ url = "https://github.com/rexm/handlebars.net",
+ text = "Handlebars.Net"
+};
+
+// Act
+var handlebars = HandlebarsDotNet.Handlebars.Create();
+handlebars.RegisterHelper("link_to", link_to);
+var template = handlebars1.Compile(source);
+// handlebars.RegisterHelper("link_to", link_to); This also works
+var result = template1(data);
+```
+
+Handlebars allows the helpers to be registered with the `Handlebars` instance either before or after a template is compiled.
+The optimum would be to have a shared `Handlebars` instance for a specific collection of functions and register the helpers just once.
+For use cases where the Kernel function collection may have been mutated we will be forced to create a `Handlebars` instance at render time
+and then register the helpers. This means we cannot take advantage of the performance improvement provided by compiling the template.
+
+## Decision Drivers
+
+In no particular order:
+
+- Support creating a semantic function without a `IKernel`instance.
+- Support late binding of functions i.e., having functions resolved when the prompt is rendered.
+- Support allowing the prompt template to be parsed (compiled) just once to optimize performance if needed.
+- Support using multiple prompt template formats with a single `Kernel` instance.
+- Provide simple abstractions which allow third parties to implement support for custom prompt template formats.
+
+## Considered Options
+
+- Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`.
+
+### Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`
+
+
+
+Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
+
+```csharp
+// Semantic function can be created once
+var promptTemplateFactory = new BasicPromptTemplateFactory();
+string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
+var promptTemplate = promptTemplateFactory.CreatePromptTemplate(templateString, new PromptTemplateConfig());
+var kindOfDay = ISKFunction.CreateSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate)
+
+// Create Kernel after creating the semantic function
+// Later we will support passing a function collection to the KernelBuilder
+IKernel kernel = Kernel.Builder
+ .WithOpenAIChatCompletionService(
+ modelId: openAIModelId,
+ apiKey: openAIApiKey)
+ .Build();
+
+kernel.ImportFunctions(new TimePlugin(), "time");
+// Optionally register the semantic function with the Kernel
+// kernel.RegisterCustomFunction(kindOfDay);
+
+var result = await kernel.RunAsync(kindOfDay);
+Console.WriteLine(result.GetValue());
+```
+
+**Notes:**
+
+- `BasicPromptTemplateFactory` will be the default implementation and will be automatically provided in `KernelSemanticFunctionExtensions`. Developers will also be able to provide their own implementation.
+- The factory uses the new `PromptTemplateConfig.TemplateFormat` to create the appropriate `IPromptTemplate` instance.
+- We should look to remove `promptTemplateConfig` as a parameter to `CreateSemanticFunction`. That change is outside of the scope of this ADR.
+
+The `BasicPromptTemplateFactory` and `BasicPromptTemplate` implementations look as follows:
+
+```csharp
+public sealed class BasicPromptTemplateFactory : IPromptTemplateFactory
+{
+ private readonly IPromptTemplateFactory _promptTemplateFactory;
+ private readonly ILoggerFactory _loggerFactory;
+
+ public BasicPromptTemplateFactory(IPromptTemplateFactory promptTemplateFactory, ILoggerFactory? loggerFactory = null)
+ {
+ this._promptTemplateFactory = promptTemplateFactory;
+ this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
+ }
+
+ public IPromptTemplate? CreatePromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig)
+ {
+ if (promptTemplateConfig.TemplateFormat.Equals(PromptTemplateConfig.SEMANTICKERNEL, System.StringComparison.Ordinal))
+ {
+ return new BasicPromptTemplate(templateString, promptTemplateConfig, this._loggerFactory);
+ }
+ else if (this._promptTemplateFactory is not null)
+ {
+ return this._promptTemplateFactory.CreatePromptTemplate(templateString, promptTemplateConfig);
+ }
+
+ throw new SKException($"Invalid prompt template format {promptTemplateConfig.TemplateFormat}");
+ }
+}
+
+public sealed class BasicPromptTemplate : IPromptTemplate
+{
+ public BasicPromptTemplate(string templateString, PromptTemplateConfig promptTemplateConfig, ILoggerFactory? loggerFactory = null)
+ {
+ this._loggerFactory = loggerFactory ?? NullLoggerFactory.Instance;
+ this._logger = this._loggerFactory.CreateLogger(typeof(BasicPromptTemplate));
+ this._templateString = templateString;
+ this._promptTemplateConfig = promptTemplateConfig;
+ this._parameters = new(() => this.InitParameters());
+ this._blocks = new(() => this.ExtractBlocks(this._templateString));
+ this._tokenizer = new TemplateTokenizer(this._loggerFactory);
+ }
+
+ public IReadOnlyList Parameters => this._parameters.Value;
+
+ public async Task RenderAsync(SKContext executionContext, CancellationToken cancellationToken = default)
+ {
+ return await this.RenderAsync(this._blocks.Value, executionContext, cancellationToken).ConfigureAwait(false);
+ }
+
+ // Not showing the implementation details
+}
+```
+
+**Note:**
+
+- The call to `ExtractBlocks` is called lazily once for each prompt template
+- The `RenderAsync` doesn't need to extract the blocks every time
+
+## Decision Outcome
+
+Chosen option: "Obsolete `IPromptTemplateEngine` and replace with `IPromptTemplateFactory`", because
+addresses the requirements and provides good flexibility for the future.
diff --git a/docs/decisions/0018-kernel-hooks-phase2.md b/docs/decisions/0018-kernel-hooks-phase2.md
new file mode 100644
index 000000000000..0b1de476b117
--- /dev/null
+++ b/docs/decisions/0018-kernel-hooks-phase2.md
@@ -0,0 +1,454 @@
+## Context and Problem Statement
+
+Currently Kernel invoking and invoked handlers don't expose the prompt to the handlers.
+
+The proposal is a way to expose the prompt to the handlers.
+
+- Pre-Execution / Invoking
+
+ - Get: Prompt generated by the current `SemanticFunction.TemplateEngine` before calling the LLM
+ - Set: Modify a prompt content before sending it to LLM
+
+- Post-Execution / Invoked
+
+ - Get: Generated Prompt
+
+## Decision Drivers
+
+- Prompt template should be generated just once per function execution within the Kernel.RunAsync execution.
+- Handlers should be able to see and modify the prompt before the LLM execution.
+- Handlers should be able to see prompt after the LLM execution.
+- Calling Kernel.RunAsync(function) or ISKFunction.InvokeAsync(kernel) should trigger the events.
+
+## Out of Scope
+
+- Skip plan steps using Pre-Hooks.
+- Get the used services (Template Engine, IAIServices, etc) in the Pre/Post Hooks.
+- Get the request settings in the Pre/Post Hooks.
+
+## Current State of Kernel for Pre/Post Hooks
+
+Current state of Kernel:
+
+```csharp
+class Kernel : IKernel
+
+RunAsync()
+{
+ var context = this.CreateNewContext(variables);
+ var functionDetails = skFunction.Describe();
+ var functionInvokingArgs = this.OnFunctionInvoking(functionDetails, context);
+
+ functionResult = await skFunction.InvokeAsync(context, cancellationToken: cancellationToken);
+ var functionInvokedArgs = this.OnFunctionInvoked(functionDetails, functionResult);
+}
+```
+
+## Developer Experience
+
+Below is the expected end user experience when coding using Pre/Post Hooks to get or modify prompts.
+
+```csharp
+const string FunctionPrompt = "Write a random paragraph about: {{$input}}.";
+
+var excuseFunction = kernel.CreateSemanticFunction(...);
+
+void MyPreHandler(object? sender, FunctionInvokingEventArgs e)
+{
+ Console.WriteLine($"{e.FunctionView.PluginName}.{e.FunctionView.Name} : Pre Execution Handler - Triggered");
+
+ // Will be false for non semantic functions
+ if (e.TryGetRenderedPrompt(out var prompt))
+ {
+ Console.WriteLine("Rendered Prompt:");
+ Console.WriteLine(prompt);
+
+ // Update the prompt if needed
+ e.TryUpdateRenderedPrompt("Write a random paragraph about: Overriding a prompt");
+ }
+}
+
+void MyPostHandler(object? sender, FunctionInvokedEventArgs e)
+{
+ Console.WriteLine($"{e.FunctionView.PluginName}.{e.FunctionView.Name} : Post Execution Handler - Triggered");
+ // Will be false for non semantic functions
+ if (e.TryGetRenderedPrompt(out var prompt))
+ {
+ Console.WriteLine("Used Prompt:");
+ Console.WriteLine(prompt);
+ }
+}
+
+kernel.FunctionInvoking += MyPreHandler;
+kernel.FunctionInvoked += MyPostHandler;
+
+const string Input = "I missed the F1 final race";
+var result = await kernel.RunAsync(Input, excuseFunction);
+Console.WriteLine($"Function Result: {result.GetValue()}");
+```
+
+Expected output:
+
+```
+MyPlugin.MyFunction : Pre Execution Handler - Triggered
+Rendered Prompt:
+Write a random paragraph about: I missed the F1 final race.
+
+MyPlugin.MyFunction : Post Execution Handler - Triggered
+Used Prompt:
+Write a random paragraph about: Overriding a prompt
+
+FunctionResult:
+```
+
+## Considered Options
+
+### Improvements Common to all options
+
+Move `Dictionary` property `Metadata` from `FunctionInvokedEventArgs` to `SKEventArgs` abstract class.
+
+Pro:
+
+- This will make all SKEventArgs extensible, allowing extra information to be passed to the EventArgs when `specialization` isn't possible.
+
+### Option 1: Kernel awareness of SemanticFunctions
+
+```csharp
+class Kernel : IKernel
+
+RunAsync()
+{
+
+ if (skFunction is SemanticFunction semanticFunction)
+ {
+ var prompt = await semanticFunction.TemplateEngine.RenderAsync(semanticFunction.Template, context);
+ var functionInvokingArgs = this.OnFunctionInvoking(functionDetails, context, prompt);
+ // InvokeWithPromptAsync internal
+ functionResult = await semanticFunction.InternalInvokeWithPromptAsync(prompt, context, cancellationToken: cancellationToken);
+ }
+ else
+ {
+ functionResult = await skFunction.InvokeAsync(context, cancellationToken: cancellationToken);
+ }
+}
+class SemanticFunction : ISKFunction
+
+public InvokeAsync(context, cancellationToken)
+{
+ var prompt = _templateEngine.RenderAsync();
+ return InternalInvokeWithPromptAsync(prompt, context, cancellationToken);
+}
+
+internal InternalInvokeWithPromptAsync(string prompt)
+{
+ ... current logic to call LLM
+}
+```
+
+### Pros and Cons
+
+Pros:
+
+- Simpler and quicker to implement
+- Small number of changes limited mostly to `Kernel` and `SemanticFunction` classes
+
+Cons:
+
+- `Kernel` is aware of `SemanticFunction` implementation details
+- Not extensible to show prompts of custom `ISKFunctions` implementations
+
+### Option 2: Delegate to the ISKFunction how to handle events (Interfaces approach)
+
+```csharp
+class Kernel : IKernel
+{
+ RunAsync() {
+ var functionInvokingArgs = await this.TriggerEvent(this.FunctionInvoking, skFunction, context);
+
+ var functionResult = await skFunction.InvokeAsync(context, cancellationToken: cancellationToken);
+
+ var functionInvokedArgs = await this.TriggerEvent(
+ this.FunctionInvoked,
+ skFunction,
+ context);
+ }
+
+ private TEventArgs? TriggerEvent(EventHandler? eventHandler, ISKFunction function, SKContext context) where TEventArgs : SKEventArgs
+ {
+ if (eventHandler is null)
+ {
+ return null;
+ }
+
+ if (function is ISKFunctionEventSupport supportedFunction)
+ {
+ var eventArgs = await supportedFunction.PrepareEventArgsAsync(context);
+ eventHandler.Invoke(this, eventArgs);
+ return eventArgs;
+ }
+
+ // Think about allowing to add data with the extra interface.
+
+ // If a function don't support the specific event we can:
+ return null; // Ignore or Throw.
+ throw new NotSupportedException($"The provided function \"{function.Name}\" does not supports and implements ISKFunctionHandles<{typeof(TEventArgs).Name}>");
+ }
+}
+
+public interface ISKFunctionEventSupport where TEventArgs : SKEventArgs
+{
+ Task PrepareEventArgsAsync(SKContext context, TEventArgs? eventArgs = null);
+}
+
+class SemanticFunction : ISKFunction,
+ ISKFunctionEventSupport,
+ ISKFunctionEventSupport
+{
+
+ public FunctionInvokingEventArgs PrepareEventArgsAsync(SKContext context, FunctionInvokingEventArgs? eventArgs = null)
+ {
+ var renderedPrompt = await this.RenderPromptTemplateAsync(context);
+ context.Variables.Set(SemanticFunction.RenderedPromptKey, renderedPrompt);
+
+ return new SemanticFunctionInvokingEventArgs(this.Describe(), context);
+ // OR Metadata Dictionary
+ return new FunctionInvokingEventArgs(this.Describe(), context, new Dictionary() { { RenderedPrompt, renderedPrompt } });
+ }
+
+ public FunctionInvokedEventArgs PrepareEventArgsAsync(SKContext context, FunctionInvokedEventArgs? eventArgs = null)
+ {
+ return Task.FromResult(new SemanticFunctionInvokedEventArgs(this.Describe(), context));
+ }
+}
+
+public sealed class SemanticFunctionInvokedEventArgs : FunctionInvokedEventArgs
+{
+ public SemanticFunctionInvokedEventArgs(FunctionDescription functionDescription, SKContext context)
+ : base(functionDescription, context)
+ {
+ _context = context;
+ Metadata[RenderedPromptKey] = this._context.Variables[RenderedPromptKey];
+ }
+
+ public string? RenderedPrompt => this.Metadata[RenderedPromptKey];
+
+}
+
+public sealed class SemanticFunctionInvokingEventArgs : FunctionInvokingEventArgs
+{
+ public SemanticFunctionInvokingEventArgs(FunctionDescription functionDescription, SKContext context)
+ : base(functionDescription, context)
+ {
+ _context = context;
+ }
+ public string? RenderedPrompt => this._context.Variables[RenderedPromptKey];
+}
+```
+
+### Pros and Cons
+
+Pros:
+
+- `Kernel` is not aware of `SemanticFunction` implementation details or any other `ISKFunction` implementation
+- Extensible to show dedicated EventArgs per custom `ISKFunctions` implementation, including prompts for semantic functions
+- Extensible to support future events on the Kernel thru the `ISKFunctionEventSupport` interface
+- Functions can have their own EventArgs specialization.
+- Interface is optional, so custom `ISKFunctions` can choose to implement it or not
+
+Cons:
+
+- Any custom functions now will have to responsibility implement the `ISKFunctionEventSupport` interface if they want to support events.
+- Handling events in another `ISKFunction` requires more complex approaches to manage the context and the prompt + any other data in different event handling methods.
+
+### Option 3: Delegate to the ISKFunction how to handle events (InvokeAsync Delegates approach)
+
+Add Kernel event handler delegate wrappers to `ISKFunction.InvokeAsync` interface.
+This approach shares the responsibility of handling the events between the `Kernel` and the `ISKFunction` implementation, flow control will be handled by the Kernel and the `ISKFunction` will be responsible for calling the delegate wrappers and adding data to the `SKEventArgs` that will be passed to the handlers.
+
+```csharp
+class Kernel : IKernel
+{
+ RunAsync() {
+ var functionInvokingDelegateWrapper = new(this.FunctionInvoking);
+ var functionInvokedDelegateWrapper = new(this.FunctionInvoked);
+
+ var functionResult = await skFunction.InvokeAsync(context, functionInvokingDelegateWrapper, functionInvokingDelegateWrapper, functionInvokedDelegateWrapper);
+
+ // Kernel will analyze the delegate results and make flow related decisions
+ if (functionInvokingDelegateWrapper.EventArgs.CancelRequested ... ) { ... }
+ if (functionInvokingDelegateWrapper.EventArgs.SkipRequested ... ) { ... }
+ if (functionInvokedDelegateWrapper.EventArgs.Repeat ... ) { ... }
+ }
+}
+
+class SemanticFunction : ISKFunction {
+ InvokeAsync(
+ SKContext context,
+ FunctionInvokingDelegateWrapper functionInvokingDelegateWrapper,
+ FunctionInvokedDelegateWrapper functionInvokedDelegateWrapper)
+ {
+ // The Semantic will have to call the delegate wrappers and share responsibility with the `Kernel`.
+ if (functionInvokingDelegateWrapper.Handler is not null)
+ {
+ var renderedPrompt = await this.RenderPromptTemplateAsync(context);
+ functionInvokingDelegateWrapper.EventArgs.RenderedPrompt = renderedPrompt;
+
+ functionInvokingDelegateWrapper.Handler.Invoke(this, functionInvokingDelegateWrapper.EventArgs);
+
+ if (functionInvokingDelegateWrapper.EventArgs?.CancelToken.IsCancellationRequested ?? false)
+ {
+ // Need to enforce an non processed result
+ return new SKFunctionResult(context);
+
+ //OR make InvokeAsync allow returning null FunctionResult?
+ return null;
+ }
+ }
+ }
+}
+
+// Wrapper for the EventHandler
+class FunctionDelegateWrapper where TEventArgs : SKEventArgs
+{
+ FunctionInvokingDelegateWrapper(EventHandler eventHandler) {}
+
+ // Set allows specialized eventargs to be set.
+ public TEventArgs EventArgs { get; set; }
+ public EventHandler Handler => _eventHandler;
+}
+```
+
+### Pros and Cons
+
+Pros:
+
+- `ISKFunction` has less code/complexity to handle and expose data (Rendered Prompt) and state in the EventArgs.
+- `Kernel` is not aware of `SemanticFunction` implementation details or any other `ISKFunction` implementation
+- `Kernel` has less code/complexity
+- Could be extensible to show dedicated EventArgs per custom `ISKFunctions` implementation, including prompts for semantic functions
+
+Cons:
+
+- Unable to add new events if needed (ISKFunction interface change needed)
+- Functions need to implement behavior related to dependency (Kernel) events
+- Since Kernel needs to interact with the result of an event handler, a wrapper strategy is needed to access results by reference at the kernel level (control of flow)
+- Passing Kernel event handlers full responsibility downstream to the functions don't sound quite right (Single Responsibility)
+
+### Option 4: Delegate to the ISKFunction how to handle events (SKContext Delegates approach)
+
+Add Kernel event handler delegate wrappers to `ISKFunction.InvokeAsync` interface.
+This approach shares the responsibility of handling the events between the `Kernel` and the `ISKFunction` implementation, flow control will be handled by the Kernel and the `ISKFunction` will be responsible for calling the delegate wrappers and adding data to the `SKEventArgs` that will be passed to the handlers.
+
+```csharp
+class Kernel : IKernel
+{
+ CreateNewContext() {
+ var context = new SKContext(...);
+ context.AddEventHandlers(this.FunctionInvoking, this.FunctionInvoked);
+ return context;
+ }
+ RunAsync() {
+ functionResult = await skFunction.InvokeAsync(context, ...);
+ if (this.IsCancelRequested(functionResult.Context)))
+ break;
+ if (this.IsSkipRequested(functionResult.Context))
+ continue;
+ if (this.IsRepeatRequested(...))
+ goto repeat;
+
+ ...
+ }
+}
+
+class SKContext {
+
+ internal EventHandlerWrapper? FunctionInvokingHandler { get; private set; }
+ internal EventHandlerWrapper? FunctionInvokedHandler { get; private set; }
+
+ internal SKContext(
+ ...
+ ICollection? eventHandlerWrappers = null
+ {
+ ...
+ this.InitializeEventWrappers(eventHandlerWrappers);
+ }
+
+ void InitializeEventWrappers(ICollection? eventHandlerWrappers)
+ {
+ if (eventHandlerWrappers is not null)
+ {
+ foreach (var handler in eventHandlerWrappers)
+ {
+ if (handler is EventHandlerWrapper invokingWrapper)
+ {
+ this.FunctionInvokingHandler = invokingWrapper;
+ continue;
+ }
+
+ if (handler is EventHandlerWrapper invokedWrapper)
+ {
+ this.FunctionInvokedHandler = invokedWrapper;
+ }
+ }
+ }
+ }
+}
+
+class SemanticFunction : ISKFunction {
+ InvokeAsync(
+ SKContext context
+ {
+ string renderedPrompt = await this._promptTemplate.RenderAsync(context, cancellationToken).ConfigureAwait(false);
+
+ this.CallFunctionInvoking(context, renderedPrompt);
+ if (this.IsInvokingCancelOrSkipRequested(context, out var stopReason))
+ {
+ return new StopFunctionResult(this.Name, this.PluginName, context, stopReason!.Value);
+ }
+
+ string completion = await GetCompletionsResultContentAsync(...);
+
+ var result = new FunctionResult(this.Name, this.PluginName, context, completion);
+ result.Metadata.Add(SemanticFunction.RenderedPromptMetadataKey, renderedPrompt);
+
+ this.CallFunctionInvoked(result, context, renderedPrompt);
+ if (this.IsInvokedCancelRequested(context, out stopReason))
+ {
+ return new StopFunctionResult(this.Name, this.PluginName, context, result.Value, stopReason!.Value);
+ }
+
+ return result;
+ }
+}
+```
+
+### Pros and Cons
+
+Pros:
+
+- `ISKFunction` has less code/complexity to handle and expose data (Rendered Prompt) and state in the EventArgs.
+- `Kernel` is not aware of `SemanticFunction` implementation details or any other `ISKFunction` implementation
+- `Kernel` has less code/complexity
+- Could be extensible to show dedicated EventArgs per custom `ISKFunctions` implementation, including prompts for semantic functions
+- More extensible as `ISKFunction` interface doesn't need to change to add new events.
+- `SKContext` can be extended to add new events without introducing breaking changes.
+
+Cons:
+
+- Functions now need to implement logic to handle in-context events
+- Since Kernel needs to interact with the result of an event handler, a wrapper strategy is needed to access results by reference at the kernel level (control of flow)
+- Passing Kernel event handlers full responsibility downstream to the functions don't sound quite right (Single Responsibility)
+
+## Decision outcome
+
+### Option 4: Delegate to the ISKFunction how to handle events (SKContext Delegates approach)
+
+This allow the functions to implement some of the kernel logic but has the big benefit of not splitting logic in different methods for the same Execution Context.
+
+Biggest benefit:
+**`ISKFunction` has less code/complexity to handle and expose data and state in the EventArgs.**
+**`ISKFunction` interface doesn't need to change to add new events.**
+
+This implementation allows to get the renderedPrompt in the InvokeAsync without having to manage the context and the prompt in different methods.
+
+The above also applies for any other data that is available in the invocation and can be added as a new EventArgs property.
diff --git a/docs/decisions/0019-semantic-function-multiple-model-support.md b/docs/decisions/0019-semantic-function-multiple-model-support.md
new file mode 100644
index 000000000000..569bc6b293a5
--- /dev/null
+++ b/docs/decisions/0019-semantic-function-multiple-model-support.md
@@ -0,0 +1,174 @@
+---
+# These are optional elements. Feel free to remove any of them.
+status: approved
+contact: markwallace-microsoft
+date: 2023-10-26
+deciders: markwallace-microsoft, SergeyMenshykh, rogerbarreto
+consulted: matthewbolanos, dmytrostruk
+informed:
+---
+
+# Multiple Model Support for Semantic Functions
+
+## Context and Problem Statement
+
+Developers need to be able to use multiple models simultaneously e.g., using GPT4 for certain prompts and GPT3.5 for others to reduce cost.
+
+## Use Cases
+
+In scope for Semantic Kernel V1.0 is the ability to select AI Service and Model Request Settings:
+
+1. By service id.
+ - A Service id uniquely identifies a registered AI Service and is typically defined in the scope of an application.
+1. By developer defined strategy.
+ - A _developer defined strategy_ is a code first approach where a developer provides the logic.
+1. By model id.
+ - A model id uniquely identifies a Large Language Model. Multiple AI service providers can support the same LLM.
+1. By arbitrary AI service attributes
+ - E.g. an AI service can define a provider id which uniquely identifies an AI provider e.g. "Azure OpenAI", "OpenAI", "Hugging Face"
+
+**This ADR focuses on items 1 & 2 in the above list. To implement 3 & 4 we need to provide the ability to store `AIService` metadata.**
+
+## Decision Outcome
+
+Support use cases 1 & 2 listed in this ADR and create separate ADR to add support for AI service metadata.
+
+## Descriptions of the Use Cases
+
+**Note: All code is pseudo code and does not accurately reflect what the final implementations will look like.**
+
+### Select Model Request Settings by Service Id
+
+_As a developer using the Semantic Kernel I can configure multiple request settings for a semantic function and associate each one with a service id so that the correct request settings are used when different services are used to execute my semantic function._
+
+The semantic function template configuration allows multiple model request settings to be configured. In this case the developer configures different settings based on the service id that is used to execute the semantic function.
+In the example below the semantic function is executed with "AzureText" using `max_tokens=60` because "AzureText" is the first service id in the list of models configured for the prompt.
+
+```csharp
+// Configure a Kernel with multiple LLM's
+IKernel kernel = new KernelBuilder()
+ .WithLoggerFactory(ConsoleLogger.LoggerFactory)
+ .WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
+ endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
+ .WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
+ endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
+ .WithOpenAITextCompletionService(modelId: oai.ModelId,
+ serviceId: "OpenAIText", apiKey: oai.ApiKey, setAsDefault: true)
+ .WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
+ serviceId: "OpenAIChat", apiKey: oai.ApiKey, setAsDefault: true)
+ .Build();
+
+// Configure semantic function with multiple LLM request settings
+var modelSettings = new List
+{
+ new OpenAIRequestSettings() { ServiceId = "AzureText", MaxTokens = 60 },
+ new OpenAIRequestSettings() { ServiceId = "AzureChat", MaxTokens = 120 },
+ new OpenAIRequestSettings() { ServiceId = "OpenAIText", MaxTokens = 180 },
+ new OpenAIRequestSettings() { ServiceId = "OpenAIChat", MaxTokens = 240 }
+};
+var prompt = "Hello AI, what can you do for me?";
+var promptTemplateConfig = new PromptTemplateConfig() { ModelSettings = modelSettings };
+var func = kernel.CreateSemanticFunction(prompt, config: promptTemplateConfig, "HelloAI");
+
+// Semantic function is executed with AzureText using max_tokens=60
+result = await kernel.RunAsync(func);
+```
+
+This works by using the `IAIServiceSelector` interface as the strategy for selecting the AI service and request settings to user when invoking a semantic function.
+The interface is defined as follows:
+
+```csharp
+public interface IAIServiceSelector
+{
+ (T?, AIRequestSettings?) SelectAIService(
+ string renderedPrompt,
+ IAIServiceProvider serviceProvider,
+ IReadOnlyList? modelSettings) where T : IAIService;
+}
+```
+
+A default `OrderedIAIServiceSelector` implementation is provided which selects the AI service based on the order of the model request settings defined for the semantic function.
+
+- The implementation checks if a service exists which the corresponding service id and if it does it and the associated model request settings will be used.
+- In no model request settings are defined then the default text completion service is used.
+- A default set of request settings can be specified by leaving the service id undefined or empty, the first such default will be used.
+- If no default if specified and none of the specified services are available the operation will fail.
+
+### Select AI Service and Model Request Settings By Developer Defined Strategy
+
+_As a developer using the Semantic Kernel I can provide an implementation which selects the AI service and request settings used to execute my function so that I can dynamically control which AI service and settings are used to execute my semantic function._
+
+In this case the developer configures different settings based on the service id and provides an AI Service Selector which determines which AI Service will be used when the semantic function is executed.
+In the example below the semantic function is executed with whatever AI Service and AI Request Settings `MyAIServiceSelector` returns e.g. it will be possible to create an AI Service Selector that computes the token count of the rendered prompt and uses that to determine which service to use.
+
+```csharp
+// Configure a Kernel with multiple LLM's
+IKernel kernel = new KernelBuilder()
+ .WithLoggerFactory(ConsoleLogger.LoggerFactory)
+ .WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
+ endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
+ .WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
+ endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
+ .WithOpenAITextCompletionService(modelId: oai.ModelId,
+ serviceId: "OpenAIText", apiKey: oai.ApiKey, setAsDefault: true)
+ .WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
+ serviceId: "OpenAIChat", apiKey: oai.ApiKey, setAsDefault: true)
+ .WithAIServiceSelector(new MyAIServiceSelector())
+ .Build();
+
+// Configure semantic function with multiple LLM request settings
+var modelSettings = new List
+{
+ new OpenAIRequestSettings() { ServiceId = "AzureText", MaxTokens = 60 },
+ new OpenAIRequestSettings() { ServiceId = "AzureChat", MaxTokens = 120 },
+ new OpenAIRequestSettings() { ServiceId = "OpenAIText", MaxTokens = 180 },
+ new OpenAIRequestSettings() { ServiceId = "OpenAIChat", MaxTokens = 240 }
+};
+var prompt = "Hello AI, what can you do for me?";
+var promptTemplateConfig = new PromptTemplateConfig() { ModelSettings = modelSettings };
+var func = kernel.CreateSemanticFunction(prompt, config: promptTemplateConfig, "HelloAI");
+
+// Semantic function is executed with AI Service and AI request Settings dynamically determined
+result = await kernel.RunAsync(func, funcVariables);
+```
+
+## More Information
+
+### Select AI Service by Service Id
+
+The following use case is supported. Developers can create a `Kernel`` instance with multiple named AI services. When invoking a semantic function the service id (and optionally request settings to be used) can be specified. The named AI service will be used to execute the prompt.
+
+```csharp
+var aoai = TestConfiguration.AzureOpenAI;
+var oai = TestConfiguration.OpenAI;
+
+// Configure a Kernel with multiple LLM's
+IKernel kernel = Kernel.Builder
+ .WithLoggerFactory(ConsoleLogger.LoggerFactory)
+ .WithAzureTextCompletionService(deploymentName: aoai.DeploymentName,
+ endpoint: aoai.Endpoint, serviceId: "AzureText", apiKey: aoai.ApiKey)
+ .WithAzureChatCompletionService(deploymentName: aoai.ChatDeploymentName,
+ endpoint: aoai.Endpoint, serviceId: "AzureChat", apiKey: aoai.ApiKey)
+ .WithOpenAITextCompletionService(modelId: oai.ModelId,
+ serviceId: "OpenAIText", apiKey: oai.ApiKey)
+ .WithOpenAIChatCompletionService(modelId: oai.ChatModelId,
+ serviceId: "OpenAIChat", apiKey: oai.ApiKey)
+ .Build();
+
+// Invoke the semantic function and service and request settings to use
+result = await kernel.InvokeSemanticFunctionAsync(prompt,
+ requestSettings: new OpenAIRequestSettings()
+ { ServiceId = "AzureText", MaxTokens = 60 });
+
+result = await kernel.InvokeSemanticFunctionAsync(prompt,
+ requestSettings: new OpenAIRequestSettings()
+ { ServiceId = "AzureChat", MaxTokens = 120 });
+
+result = await kernel.InvokeSemanticFunctionAsync(prompt,
+ requestSettings: new OpenAIRequestSettings()
+ { ServiceId = "OpenAIText", MaxTokens = 180 });
+
+result = await kernel.InvokeSemanticFunctionAsync(prompt,
+ requestSettings: new OpenAIRequestSettings()
+ { ServiceId = "OpenAIChat", MaxTokens = 240 });
+```
diff --git a/docs/decisions/0020-prompt-syntax-mapping-to-completion-service-model.md b/docs/decisions/0020-prompt-syntax-mapping-to-completion-service-model.md
new file mode 100644
index 000000000000..5c3b7f4b5a7a
--- /dev/null
+++ b/docs/decisions/0020-prompt-syntax-mapping-to-completion-service-model.md
@@ -0,0 +1,57 @@
+---
+# These are optional elements. Feel free to remove any of them.
+status: accepted
+date: 2023-10-27
+contact: SergeyMenshykh
+deciders: markwallace, mabolan
+consulted:
+informed:
+---
+# Mapping of prompt syntax to completion service model
+
+## Context and Problem Statement
+Today, SK runs all prompts using the text completion service by simply passing the rendered prompt as is, without any modifications, directly to a configured text completion service/connector. With the addition of new chat completion prompt and potentially other prompt types, such as image, on the horizon, we need a way to map completion-specific prompt syntax to the corresponding completion service data model.
+
+For example, [the chat completion syntax](https://github.com/microsoft/semantic-kernel/blob/main/docs/decisions/0014-chat-completion-roles-in-prompt.md) in chat completion prompts:
+```xml
+
+ You are a creative assistant helping individuals and businesses with their innovative projects.
+
+
+ I want to brainstorm the idea of {{$input}}
+
+```
+should be mapped to an instance of the [ChatHistory](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Abstractions/AI/ChatCompletion/ChatHistory.cs) class with two chat messages:
+
+```csharp
+var messages = new ChatHistory();
+messages.Add(new ChatMessage(new AuthorRole("system"), "You are a creative assistant helping individuals and businesses with their innovative projects."));
+messages.Add(new ChatMessage(new AuthorRole("user"), "I want to brainstorm the idea of {{$input}}"));
+```
+
+This ADR outlines potential options for the location of the prompt syntax mapping functionality.
+
+## Considered Options
+**1. Completion connector classes.** This option proposes to have the completion connector classes responsible for the `prompt syntax -> completion service data model` mapping. The decision regarding whether this mapping functionality will be implemented in the connector classes themselves or delegated to mapper classes should be made during the implementation phase and is out of the scope of this ADR.
+
+Pros:
+ - The `SemanticFunction` won't need to change to support the mapping of a new prompt syntax when new completion type connectors (audio, video, etc.) are added.
+
+ - Prompts can be run by
+ - Kernel.RunAsync
+ - Completion connectors
+
+Cons:
+ - Every new completion connector, whether of an existing type or a new type, will have to implement the mapping functionality
+
+**2. The SemanticFunction class.** This option proposes that the `SemanticFunction` class be responsible for the mapping. Similar to the previous option, the exact location of this functionality (whether in the `SemanticFunction` class or in the mapper classes) should be decided during the implementation phase.
+
+Pros:
+ - New connectors of a new type or existing ones don't have to implement the mapping functionality
+
+Cons:
+ - The `SemanticFunction` class has to be changed every time a new completion type needs to be supported by SK
+ - Prompts can be run by Kernel.RunAsync method only.
+
+## Decision Outcome
+It was agreed to go with the option 1 - `1. Completion connector classes` since it a more flexible solution and allows adding new connectors without modifying the `SemanticFunction` class.
\ No newline at end of file
diff --git a/docs/decisions/0021-aiservice-metadata.md b/docs/decisions/0021-aiservice-metadata.md
new file mode 100644
index 000000000000..70822b1e82c4
--- /dev/null
+++ b/docs/decisions/0021-aiservice-metadata.md
@@ -0,0 +1,157 @@
+---
+# These are optional elements. Feel free to remove any of them.
+status: {proposed}
+date: {2023-11-10}
+deciders: SergeyMenshykh, markwallace, rbarreto, dmytrostruk
+consulted:
+informed:
+---
+# Add AI Service Metadata
+
+## Context and Problem Statement
+
+Developers need to be able to know more information about the `IAIService` that will be used to execute a semantic function or a plan.
+Some examples of why they need this information:
+
+1. As an SK developer I want to write a `IAIServiceSelector` which allows me to select the OpenAI service to used based on the configured model id so that I can select the optimum (could eb cheapest) model to use based on the prompt I am executing.
+2. As an SK developer I want to write a pre-invocation hook which will compute the token size of a prompt before the prompt is sent to the LLM, so that I can determine the optimum `IAIService` to use. The library I am using to compute the token size of the prompt requires the model id.
+
+Current implementation of `IAIService` is empty.
+
+```csharp
+public interface IAIService
+{
+}
+```
+
+We can retrieve `IAIService` instances using `T IKernel.GetService(string? name = null) where T : IAIService;` i.e., by service type and name (aka service id).
+The concrete instance of an `IAIService` can have different attributes depending on the service provider e.g. Azure OpenAI has a deployment name and OpenAI services have a model id.
+
+Consider the following code snippet:
+
+```csharp
+IKernel kernel = new KernelBuilder()
+ .WithLoggerFactory(ConsoleLogger.LoggerFactory)
+ .WithAzureChatCompletionService(
+ deploymentName: chatDeploymentName,
+ endpoint: endpoint,
+ serviceId: "AzureOpenAIChat",
+ apiKey: apiKey)
+ .WithOpenAIChatCompletionService(
+ modelId: openAIModelId,
+ serviceId: "OpenAIChat",
+ apiKey: openAIApiKey)
+ .Build();
+
+var service = kernel.GetService("OpenAIChat");
+```
+
+For Azure OpenAI we create the service with a deployment name. This is an arbitrary name specified by the person who deployed the AI model e.g. it could be `eastus-gpt-4` or `foo-bar`.
+For OpenAI we create the service with a model id. This must match one of the deployed OpenAI models.
+
+From the perspective of a prompt creator using OpenAI, they will typically tune their prompts based on the model. So when the prompt is executed we need to be able to retrieve the service using the model id. As shown in the code snippet above the `IKernel` only supports retrieving an `IAService` instance by id. Additionally the `IChatCompletion` is a generic interface so it doesn't contain any properties which provide information about a specific connector instance.
+
+## Decision Drivers
+
+* We need a mechanism to store generic metadata for an `IAIService` instance.
+ * It will be the responsibility of the concrete `IAIService` instance to store the metadata that is relevant e.g., model id for OpenAI and HuggingFace AI services.
+* We need to be able to iterate over the available `IAIService` instances.
+
+## Considered Options
+
+* Option #1
+ * Extend `IAIService` to include the following properties:
+ * `string? ModelId { get; }` which returns the model id. It will be the responsibility of each `IAIService` implementation to populate this with the appropriate value.
+ * `IReadOnlyDictionary Attributes { get; }` which returns the attributes as a readonly dictionary. It will be the responsibility of each `IAIService` implementation to populate this with the appropriate metadata.
+ * Extend `INamedServiceProvider` to include this method `ICollection GetServices() where T : TService;`
+ * Extend `OpenAIKernelBuilderExtensions` so that `WithAzureXXX` methods will include a `modelId` property if a specific model can be targeted.
+* Option #2
+ * Extend `IAIService` to include the following method:
+ * `T? GetAttributes() where T : AIServiceAttributes;` which returns an instance of `AIServiceAttributes`. It will be the responsibility of each `IAIService` implementation to define it's own service attributes class and populate this with the appropriate values.
+ * Extend `INamedServiceProvider` to include this method `ICollection GetServices() where T : TService;`
+ * Extend `OpenAIKernelBuilderExtensions` so that `WithAzureXXX` methods will include a `modelId` property if a specific model can be targeted.
+* Option #3
+* Option #2
+ * Extend `IAIService` to include the following properties:
+ * `public IReadOnlyDictionary Attributes => this.InternalAttributes;` which returns a read only dictionary. It will be the responsibility of each `IAIService` implementation to define it's own service attributes class and populate this with the appropriate values.
+ * `ModelId`
+ * `Endpoint`
+ * `ApiVersion`
+ * Extend `INamedServiceProvider` to include this method `ICollection GetServices() where T : TService;`
+ * Extend `OpenAIKernelBuilderExtensions` so that `WithAzureXXX` methods will include a `modelId` property if a specific model can be targeted.
+
+These options would be used as follows:
+
+As an SK developer I want to write a custom `IAIServiceSelector` which will select an AI service based on the model id because I want to restrict which LLM is used.
+In the sample below the service selector implementation looks for the first service that is a GPT3 model.
+
+### Option 1
+
+``` csharp
+public class Gpt3xAIServiceSelector : IAIServiceSelector
+{
+ public (T?, AIRequestSettings?) SelectAIService(string renderedPrompt, IAIServiceProvider serviceProvider, IReadOnlyList? modelSettings) where T : IAIService
+ {
+ var services = serviceProvider.GetServices();
+ foreach (var service in services)
+ {
+ if (!string.IsNullOrEmpty(service.ModelId) && service.ModelId.StartsWith("gpt-3", StringComparison.OrdinalIgnoreCase))
+ {
+ Console.WriteLine($"Selected model: {service.ModelId}");
+ return (service, new OpenAIRequestSettings());
+ }
+ }
+
+ throw new SKException("Unable to find AI service for GPT 3.x.");
+ }
+}
+```
+
+## Option 2
+
+``` csharp
+public class Gpt3xAIServiceSelector : IAIServiceSelector
+{
+ public (T?, AIRequestSettings?) SelectAIService(string renderedPrompt, IAIServiceProvider serviceProvider, IReadOnlyList? modelSettings) where T : IAIService
+ {
+ var services = serviceProvider.GetServices();
+ foreach (var service in services)
+ {
+ var serviceModelId = service.GetAttributes()?.ModelId;
+ if (!string.IsNullOrEmpty(serviceModelId) && serviceModelId.StartsWith("gpt-3", StringComparison.OrdinalIgnoreCase))
+ {
+ Console.WriteLine($"Selected model: {serviceModelId}");
+ return (service, new OpenAIRequestSettings());
+ }
+ }
+
+ throw new SKException("Unable to find AI service for GPT 3.x.");
+ }
+}
+```
+
+## Option 3
+
+```csharp
+public (T?, AIRequestSettings?) SelectAIService(string renderedPrompt, IAIServiceProvider serviceProvider, IReadOnlyList? modelSettings) where T : IAIService
+{
+ var services = serviceProvider.GetServices();
+ foreach (var service in services)
+ {
+ var serviceModelId = service.GetModelId();
+ var serviceOrganization = service.GetAttribute(OpenAIServiceAttributes.OrganizationKey);
+ var serviceDeploymentName = service.GetAttribute(AzureOpenAIServiceAttributes.DeploymentNameKey);
+ if (!string.IsNullOrEmpty(serviceModelId) && serviceModelId.StartsWith("gpt-3", StringComparison.OrdinalIgnoreCase))
+ {
+ Console.WriteLine($"Selected model: {serviceModelId}");
+ return (service, new OpenAIRequestSettings());
+ }
+ }
+
+ throw new SKException("Unable to find AI service for GPT 3.x.");
+}
+```
+
+## Decision Outcome
+
+Chosen option: Option 1, because it's a simple implementation and allows easy iteration over all possible attributes.
diff --git a/docs/decisions/0021-json-serializable-custom-types.md b/docs/decisions/0021-json-serializable-custom-types.md
new file mode 100644
index 000000000000..d7a0072409a7
--- /dev/null
+++ b/docs/decisions/0021-json-serializable-custom-types.md
@@ -0,0 +1,124 @@
+---
+status: proposed
+contact: dehoward
+date: 2023-11-06
+deciders: alliscode, markwallace-microsoft
+consulted:
+informed:
+---
+
+# JSON Serializable Custom Types
+
+## Context and Problem Statement
+
+This ADR aims to simplify the usage of custom types by allowing developers to use any type that can be serialized using `System.Text.Json`.
+
+Standardizing on a JSON-serializable type is necessary to allow functions to be described using a JSON Schema within a planner's function manual. Using a JSON Schema to describe a function's input and output types will allow the planner to validate that the function is being used correctly.
+
+Today, use of custom types within Semantic Kernel requires developers to implement a custom `TypeConverter` to convert to/from the string representation of the type. This is demonstrated in [Example60_AdvancedNativeFunctions](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/KernelSyntaxExamples/Example60_AdvancedNativeFunctions.cs#L202C44-L202C44) as seen below:
+
+```csharp
+ [TypeConverter(typeof(MyCustomTypeConverter))]
+ private sealed class MyCustomType
+ {
+ public int Number { get; set; }
+
+ public string? Text { get; set; }
+ }
+
+ private sealed class MyCustomTypeConverter : TypeConverter
+ {
+ public override bool CanConvertFrom(ITypeDescriptorContext? context, Type sourceType) => true;
+
+ public override object? ConvertFrom(ITypeDescriptorContext? context, CultureInfo? culture, object value)
+ {
+ return JsonSerializer.Deserialize((string)value);
+ }
+
+ public override object? ConvertTo(ITypeDescriptorContext? context, CultureInfo? culture, object? value, Type destinationType)
+ {
+ return JsonSerializer.Serialize(value);
+ }
+ }
+```
+
+The above approach will now only be needed when a custom type cannot be serialized using `System.Text.Json`.
+
+## Considered Options
+
+**1. Fallback to serialization using `System.Text.Json` if a `TypeConverter` is not available for the given type**
+
+- Primitive types will be handled using their native `TypeConverter`s
+ - We preserve the use of the native `TypeConverter` for primitive types to prevent any lossy conversions.
+- Complex types will be handled by their registered `TypeConverter`, if provided.
+- If no `TypeConverter` is registered for a complex type, our own `JsonSerializationTypeConverter` will be used to attempt JSON serialization/deserialization using `System.Text.Json`.
+ - A detailed error message will be thrown if the type cannot be serialized/deserialized.
+
+This will change the `GetTypeConverter()` method in `NativeFunction.cs` to look like the following, where before `null` was returned if no `TypeConverter` was found for the type:
+
+```csharp
+private static TypeConverter GetTypeConverter(Type targetType)
+ {
+ if (targetType == typeof(byte)) { return new ByteConverter(); }
+ if (targetType == typeof(sbyte)) { return new SByteConverter(); }
+ if (targetType == typeof(bool)) { return new BooleanConverter(); }
+ if (targetType == typeof(ushort)) { return new UInt16Converter(); }
+ if (targetType == typeof(short)) { return new Int16Converter(); }
+ if (targetType == typeof(char)) { return new CharConverter(); }
+ if (targetType == typeof(uint)) { return new UInt32Converter(); }
+ if (targetType == typeof(int)) { return new Int32Converter(); }
+ if (targetType == typeof(ulong)) { return new UInt64Converter(); }
+ if (targetType == typeof(long)) { return new Int64Converter(); }
+ if (targetType == typeof(float)) { return new SingleConverter(); }
+ if (targetType == typeof(double)) { return new DoubleConverter(); }
+ if (targetType == typeof(decimal)) { return new DecimalConverter(); }
+ if (targetType == typeof(TimeSpan)) { return new TimeSpanConverter(); }
+ if (targetType == typeof(DateTime)) { return new DateTimeConverter(); }
+ if (targetType == typeof(DateTimeOffset)) { return new DateTimeOffsetConverter(); }
+ if (targetType == typeof(Uri)) { return new UriTypeConverter(); }
+ if (targetType == typeof(Guid)) { return new GuidConverter(); }
+
+ if (targetType.GetCustomAttribute() is TypeConverterAttribute tca &&
+ Type.GetType(tca.ConverterTypeName, throwOnError: false) is Type converterType &&
+ Activator.CreateInstance(converterType) is TypeConverter converter)
+ {
+ return converter;
+ }
+
+ // now returns a JSON-serializing TypeConverter by default, instead of returning null
+ return new JsonSerializationTypeConverter();
+ }
+
+ private sealed class JsonSerializationTypeConverter : TypeConverter
+ {
+ public override bool CanConvertFrom(ITypeDescriptorContext? context, Type sourceType) => true;
+
+ public override object? ConvertFrom(ITypeDescriptorContext? context, CultureInfo? culture, object value)
+ {
+ return JsonSerializer.Deserialize
-
diff --git a/docs/decisions/0010-openai-function-calling.md b/docs/decisions/0010-openai-function-calling.md
deleted file mode 100644
index 63c605916cdd..000000000000
--- a/docs/decisions/0010-openai-function-calling.md
+++ /dev/null
@@ -1,69 +0,0 @@
----
-status: accepted
-contact: gitri-ms
-date: 2023-09-21
-deciders: gitri-ms, shawncal
-consulted: lemillermicrosoft, awharrison-28, dmytrostruk, nacharya1
-informed: eavanvalkenburg, kevdome3000
----
-# OpenAI Function Calling Support
-
-## Context and Problem Statement
-
-The [function calling](https://platform.openai.com/docs/guides/gpt/function-calling) capability of OpenAI's Chat Completions API allows developers to describe functions to the model, and have the model decide whether to output a JSON object specifying a function and appropriate arguments to call in response to the given prompt. This capability is enabled by two new API parameters to the `/v1/chat/completions` endpoint:
-- `function_call` - auto (default), none, or a specific function to call
-- `functions` - JSON descriptions of the functions available to the model
-
-Functions provided to the model are injected as part of the system message and are billed/counted as input tokens.
-
-We have received several community requests to provide support for this capability when using SK with the OpenAI chat completion models that support it.
-
-## Decision Drivers
-
-* Minimize changes to the core kernel for OpenAI-specific functionality
-* Cost concerns with including a long list of function descriptions in the request
-* Security and cost concerns with automatically executing functions returned by the model
-
-## Considered Options
-
-* Support sending/receiving functions via chat completions endpoint _with_ modifications to interfaces
-* Support sending/receiving functions via chat completions endpoint _without_ modifications to interfaces
-* Implement a planner around the function calling capability
-
-## Decision Outcome
-
-Chosen option: "Support sending/receiving functions via chat completions endpoint _without_ modifications to interfaces"
-
-With this option, we utilize the existing request settings object to send functions to the model. The app developer controls what functions are included and is responsible for validating and executing the function result.
-
-### Consequences
-
-* Good, because avoids breaking changes to the core kernel
-* Good, because OpenAI-specific functionality is contained to the OpenAI connector package
-* Good, because allows app to control what functions are available to the model (including non-SK functions)
-* Good, because keeps the option open for integrating with planners in the future
-* Neutral, because requires app developer to validate and execute resulting function
-* Bad, because not as obvious how to use this capability and access the function results
-
-## Pros and Cons of the Options
-
-### Support sending/receiving functions _with_ modifications to chat completions interfaces
-
-This option would update the `IChatCompletion` and `IChatResult` interfaces to expose parameters/methods for providing and accessing function information.
-
-* Good, because provides a clear path for using the function calling capability
-* Good, because allows app to control what functions are available to the model (including non-SK functions)
-* Neutral, because requires app developer to validate and execute resulting function
-* Bad, because introduces breaking changes to core kernel abstractions
-* Bad, because OpenAI-specific functionality would be included in core kernel abstractions and would need to be ignored by other model providers
-
-### Implement a planner around the function calling capability
-
-Orchestrating external function calls fits within SK's concept of planning. With this approach, we would implement a planner that would take the function calling result and produce a plan that the app developer could execute (similar to SK's ActionPlanner).
-
-* Good, because producing a plan result makes it easy for the app developer to execute the chosen function
-* Bad, because functions would need to be registered with the kernel in order to be executed
-* Bad, because would create confusion about when to use which planner
-
-## Additional notes
-There has been much discussion and debate over the pros and cons of automatically invoking a function returned by the OpenAI model, if it is registered with the kernel. As there are still many open questions around this behavior and its implications, we have decided to not include this capability in the initial implementation. We will continue to explore this option and may include it in a future update.
\ No newline at end of file
diff --git a/docs/decisions/0012-kernel-service-registration.md b/docs/decisions/0012-kernel-service-registration.md
index db2e8ac14478..2e69907bcc45 100644
--- a/docs/decisions/0012-kernel-service-registration.md
+++ b/docs/decisions/0012-kernel-service-registration.md
@@ -4,9 +4,10 @@ status: accepted
contact: dmytrostruk
date: 2023-10-03
deciders: dmytrostruk
-consulted: semenshi, rbarreto, markwallace-microsoft
-informed:
+consulted: SergeyMenshykh, RogerBarreto, markwallace-microsoft
+informed:
---
+
# Kernel Service Registration
## Context and Problem Statement
@@ -82,12 +83,12 @@ Custom service collection and service provider on Kernel level to simplify depen
Interface `IKernel` will have its own service provider `KernelServiceProvider` with minimal functionality to get required service.
```csharp
-public interface IKernelServiceProvider
+public interface IKernelServiceProvider
{
T? GetService
+
+Below is an expanded example of how to create a semantic function from a prompt template string which uses the built-in Semantic Kernel format:
+
+```csharp
+// Semantic function can be created once
+var promptTemplateFactory = new BasicPromptTemplateFactory();
+string templateString = "Today is: {{time.Date}} Is it weekend time (weekend/not weekend)?";
+var promptTemplateConfig = new PromptTemplateConfig();
+// Line below will replace the commented out code
+var promptTemplate = promptTemplateFactory.CreatePromptTemplate(templateString, promptTemplateConfig);
+var kindOfDay = ISKFunction.CreateSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate)
+// var promptTemplate = new PromptTemplate(promptTemplate, promptTemplateConfig, kernel.PromptTemplateEngine);
+// var kindOfDay = kernel.RegisterSemanticFunction("KindOfDay", promptTemplateConfig, promptTemplate);
+
+// Create Kernel after creating the semantic function
+// Later we will support passing a function collection to the KernelBuilder
+IKernel kernel = Kernel.Builder
+ .WithOpenAIChatCompletionService(
+ modelId: openAIModelId,
+ apiKey: openAIApiKey)
+ .Build();
+
+kernel.ImportFunctions(new TimePlugin(), "time");
+// Optionally register the semantic function with the Kernel
+kernel.RegisterCustomFunction(kindOfDay);
+
+var result = await kernel.RunAsync(kindOfDay);
+Console.WriteLine(result.GetValue