.Net: Bug: Retrieved GUID from OpenApi plugin sometimes altered by Semantic Kernel

[Garym3]

Describe the bug
I'm using Azure OpenAI, its corresponding chat completion service, a ChatHistory, and an OpenApi plugin imported from the swagger endpoint of a local API.

I've stumbled upon this problem in an actual project, so I've created a simplified sample project to reproduce this issue.

Below's the execution flow until the problem arises, but you can find everything in the attached sample project:

1. First, we configure the prompt execution settings with the auto function choice behavior,
2. The user sends the following prompt to the LLM: "Company name: TEST | Legal form: Société à Actions Simplifiée" which is the needed information for creating a new company,
3. Then, before contacting the LLM, the user sends another prompt where he asks this: "Create a company based on the aforementioned data",
4. Next, we send the chat history to the chat completion service in non-streaming mode (the bug is also present in streaming mode),
5. Afterwards, the "SampleFilter" should inform you about whether the GUID is OK, illformated or even if the LLM chose the wrong GUID by mistake.

Semantic Kernel sometimes adds a '2' digit at the beginning of the corporate form GUID, making it illformated and invalid.
To be more precise, the "Société par actions simplifiée" corporate form's GUID is correctly retrieved from the API, but then is sometimes prepended with a '2' seemingly by Semantic Kernel at some point before reusing it for the CreateDossier endpoint.

1. Semantic Kernel (SK) queries the "get_corporate_forms" endpoint from the API plugin,
2. API plugin sends "58f2c998-f1e8-11d4-9c26-00a024878585" as a GUID,
3. Semantic Kernel retrieves the GUID correctly as "58f2c998-f1e8-11d4-9c26-00a024878585",
4. Semantic Kernel does its things,
5. Semantic Kernel wants to query the next endpoint ("create_company") but may send "258f2c998-f1e8-11d4-9c26-00a024878585" (altered GUID) instead.

To Reproduce
Download the sample projects

Steps to reproduce the behavior:

1. Set the Azure Open AI API key and the HTTPS endpoint at the beginning of the Main method of the "SK-Guid-Error" project,
2. (optional) Set breakpoints on lines 17, 21 and 25 in "SK-Guid-Error.SampleFilter.cs" for debugging,
3. Start the API by running the "SK-Guid-Error-OpenApi" project,
4. Start the "SK-Guid-Error" project then wait until the breakpoints are hit, or read the console,
5. If the GUID is still correct when hitting a breakpoint in the filter, restart the console app until the GUID is altered (illformated).

Expected behavior
I want "Semantic Kernel" to never alter the GUID that it retrieved beforehand from the OpenApi plugin.

Screenshots
N/A

Platform

• OS: Windows 11
• IDE: Visual Studio 2022
• Language: C# .NET 9
• Source:
• • Microsoft.SemanticKernel (1.33.0),
• • Microsoft.SemanticKernel.Abstractions (1.33.0),
• • Microsoft.SemanticKernel.Connectors.OpenAI (1.33.0),
• • Microsoft.SemanticKernel.Connectors.AzureOpenAI (1.33.0),
• • Microsoft.SemanticKernel.Core (1.33.0),
• • Microsoft.SemanticKernel.Plugins.OpenApi (1.33.0-preview).

Additional context
In my case, the problem only arises with this specific corporate form GUID, but I've not managed to reproduce the issue with another corporate form GUID, even though I didn't spend much time for testing other GUIDs.
Furthermore, the problem does never occur with the user GUID (allegedly).

[SergeyMenshykh]

@Garym3, I guess the reason the Guid is prefixed with the 2 character is that the AI model can't tell where the \u0022 double quote escape sequence ends and where the Guid starts when it's provided in escaped JSON like this:

{
   \u0022id\u0022:\u002258f2c998-f1e8-11d4-9c26-00a024878585\u0022,
   \u0022long_name\u0022:\u0022Soci\u00E9t\u00E9 par actions simplifi\u00E9e\u0022
}

SK OpenAPI functionality, by default, keeps a JSON response as a string, and when the string is serialized, all double quotes for property names and string values are escaped. Changing that legacy behavior at the moment would mean a breaking change, which we would like to avoid at the moment, but we may do it in the future.

At the moment, to get unblocked, consider using the custom content reader that will read the response JSON content as a JsonElement, which works well during serialization.

var uri = new Uri("https://localhost:7279/swagger/v1/swagger.json", UriKind.Absolute);
var apiPlugin = await kernel.CreatePluginFromOpenApiAsync("api", uri, executionParameters: new OpenApiFunctionExecutionParameters()
{
   // Register the custom response content reader
    HttpResponseContentReader = ReadResponseAsync
});
kernel.Plugins.Add(apiPlugin);

private async Task<object?> ReadResponseAsync(HttpResponseContentReaderContext context, CancellationToken cancellationToken)
{
    if (context.Response.Content.Headers.ContentType?.MediaType == "application/json")
    {
        return await context.Response.Content.ReadFromJsonAsync<JsonElement>(cancellationToken);
    }

    // Return null to indicate that any other HTTP content not handled above should be read by the default reader.
    return null;
}

A little more detail about the custom content reader can be found at: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/OpenApiPlugin_CustomHttpContentReader.cs"

Please try the solution and let us know if it works for you.

[SergeyMenshykh]

BTW, thanks for the working sample project. It was very easy to build and use it to reproduce this issue.

[Garym3]

@SergeyMenshykh Thank you for the explanation.

A bit before your reply, I ended up overwriting the imported OpenAPI kernel functions but I don't know if it's recommended to do so, or if I better use your solution? In any case, applying any of these 2 solutions 'solves' the problem, thanks.

public static KernelFunction UpdateFunction(KernelFunction function)
{
	async Task<FunctionResult> OverwrittenMethodAsync(Kernel kernel, KernelFunction currentFunction, KernelArguments arguments, CancellationToken cancellationToken)
	{
		foreach (KernelParameterMetadata parameter in currentFunction.Metadata.Parameters)
		{
			if (!arguments.TryGetValue(parameter.Name, out object? value)) continue;
			if (value == null) continue;

			if (IsString(parameter.ParameterType) && !Guid.TryParseExact(value.ToString(), "D", out Guid _))
			{
				string? tempGuid = value.ToString()?[1..];
				arguments[parameter.Name] = Guid.TryParseExact(tempGuid, "D", out Guid _) ? tempGuid : value;
			}
		}

		return await function.InvokeAsync(kernel, arguments, cancellationToken);
	}

	var options = new KernelFunctionFromMethodOptions()
	{
		FunctionName = function.Name,
		Description = function.Description,
		Parameters = function.Metadata.Parameters,
		ReturnParameter = function.Metadata.ReturnParameter,
		AdditionalMetadata = function.Metadata.AdditionalProperties,
	};
	return KernelFunctionFactory.CreateFromMethod(OverwrittenMethodAsync, options);
}

private static bool IsString(Type? parameterType) => parameterType == typeof(string);