Skip to content

OpenAPI Schema generation stackoverflow when converting Flags enum to a string array & is used in API models with default value #62023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 task done
wsloth opened this issue May 20, 2025 · 1 comment · Fixed by #62051
Closed
1 task done

OpenAPI Schema generation stackoverflow when converting Flags enum to a string array & is used in API models with default value #62023

wsloth opened this issue May 20, 2025 · 1 comment · Fixed by #62051
Labels
area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc feature-openapi
Milestone
10.0-preview6

Comments

@wsloth
Copy link

wsloth commented May 20, 2025 •
edited
Loading

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

Hi, I'm not 100% sure this is an aspnetcore or a user issue, but it might be a bit of both.

I have a flags enumeration defined like this:

[Flags]
[JsonConverter(typeof(FlagsEnumAsArrayConverter<FlagsEnum>))]
enum FlagsEnum
{
    None = 0,
    Flag1 = 1,
    Flag2 = 2,
    Flag3 = 4,
    Flag4 = 8,
    Flag5 = 16
}

And I want to use it in an API model similarly to this:

/// <summary>
/// Use the flags in any model and use a default value instead of a null value.
/// This is where things go wrong -- if it is a nullable property it works.
/// </summary>
record FlagsEnumModel(FlagsEnum Flags = FlagsEnum.Flag1);

To have a better developer experience when using this, I want the client to be able to send flags as arrays of strings, instead of an integer or concatenated string. So Example: ["Flag1", "Flag2"] instead of "Flag1, Flag2".

To do this, I created the following JsonConverter below. I've tested this in other places (Kafka (de)serialization, unit tests outputting JSON snapshots), and it works fine.

However, when I want to view my OpenAPI document, it fails to generate, crashing the process with a stack overflow.

/// <summary>
/// This converter is used to serialize and deserialize flags enums as arrays.
/// Note: the default enum value (Unknown/None/0) is not included in the output array.
/// Example: ["Flag1", "Flag2"] instead of "Flag1, Flag2".
/// </summary>
/// <typeparam name="T">An enum decorated with [Flags]</typeparam>
public class FlagsEnumAsArrayConverter<T> : JsonConverter<T>
    where T : struct, Enum
{
    public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        if (reader.TokenType != JsonTokenType.StartArray)
        {
            throw new JsonException($"Expected StartArray token, got {reader.TokenType}");
        }

        T value = default;
        reader.Read();
        while (reader.TokenType != JsonTokenType.EndArray)
        {
            if (reader.TokenType != JsonTokenType.String)
            {
                throw new JsonException($"Expected string in array, got {reader.TokenType}");
            }

            var flagName = reader.GetString();
            if (!Enum.TryParse<T>(flagName, out var flag))
            {
                throw new JsonException($"Invalid flag value: {flagName}");
            }

            value = (T)Enum.ToObject(typeof(T), Convert.ToInt64(value) | Convert.ToInt64(flag));
            reader.Read();
        }

        return value;
    }

    public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
    {
        var names = value.ToString().Split(", ", StringSplitOptions.RemoveEmptyEntries);
        writer.WriteStartArray();
        foreach (var name in names)
        {
            writer.WriteStringValue(name);
        }

        writer.WriteEndArray();
    }
}

Expected Behavior

I would expect the OpenAPI document to generate correctly as outputting an array of enum values. Or at the very least, an array of strings.

Steps To Reproduce

I created a reproducable sample repository here: https://github.com/wsloth/aspnetcore-flagsenum-arrayconverter-issue

It contains both a .NET 9 and a .NET 10 Preview 4 project. They both contain the issue, but it seems like the .NET 9 executable crashes entirely, while the .NET 10 executable keeps running (but does output "stack overflow" in the console).

I was diving into the internals on the .NET 9 version, and saw that it went wrong in OpenApiJsonSchema, specifically in ReadOpenApiAny. It would see the array start token, but it had no value at all in the schema (it looked like [ ]) before reaching the end token.

I'm not sure if there should be a value in this array, or if the issue is that if there is a value, it's another instance of the FlagsEnum, which enters another (de)serialization loop which generates yet another array.

Exceptions (if any)

... continues forever ...
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef, System.String ByRef)
   at OpenApiJsonSchema.ReadOpenApiAny(System.Text.Json.Utf8JsonReader ByRef)
   at OpenApiJsonSchema.ReadProperty(System.Text.Json.Utf8JsonReader ByRef, System.String, Microsoft.OpenApi.Models.OpenApiSchema, System.Text.Json.JsonSerializerOptions)
   at OpenApiJsonSchema+JsonConverter.Read(System.Text.Json.Utf8JsonReader ByRef, System.Type, System.Text.Json.JsonSerializerOptions)
   at System.Text.Json.Serialization.JsonConverter`1[[System.__Canon, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TryRead(System.Text.Json.Utf8JsonReader ByRef, System.Type, System.Text.Json.JsonSerializerOptions, System.Text.Json.ReadStack ByRef, System.__Canon ByRef, Boolean ByRef)

.NET Version

10.0.100-preview.4.25258.110

Anything else?

No response
@github-actions github-actions bot added the needs-area-label Used by the dotnet-issue-labeler to label those issues which couldn't be triaged automatically label May 20, 2025
@martincostello martincostello added feature-openapi area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc and removed needs-area-label Used by the dotnet-issue-labeler to label those issues which couldn't be triaged automatically labels May 20, 2025
@captainsafia captainsafia pinned this issue May 20, 2025
@captainsafia captainsafia unpinned this issue May 20, 2025
@BrennanConroy
Copy link
Member

BrennanConroy commented May 21, 2025 •
edited
Loading

Can you please share what request you tried?

Wasn't able to reproduce any issues with 9.0 or 10.0-preview4 using body: JSON.stringify({"Flags": ["Flag1", "Flag3"]}

Oh, when you access openapi/v1.json.

@BrennanConroy BrennanConroy mentioned this issue May 21, 2025
Merged
@BrennanConroy BrennanConroy added this to the 10.0-preview6 milestone May 27, 2025
@BrennanConroy BrennanConroy mentioned this issue May 27, 2025
Open
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc feature-openapi
Projects
None yet
Milestone
10.0-preview6
Development

Successfully merging a pull request may close this issue.

Fix OpenApiJsonSchema array parsing dotnet/aspnetcore
3 participants
@martincostello @wsloth @BrennanConroy