'AsyncEnumerableReader' 在枚举值时达到了配置的缓冲区的最大大小

Question

我正在使用返回 IAsyncEnumerable<> 的 async/await 方法从 SQL Server 数据库中检索行并通过 Web API .Net Core 3.0 接口提供它们。它工作正常，直到我超过 8192 行。对于超出该点的行，它会失败。我尝试了 3 个不同的表，每个表的行大小都非常不同。每次在产生 8192 行后发生故障。这是我看到的错误：

An unhandled exception occurred while processing the request.
InvalidOperationException: 'AsyncEnumerableReader' reached the configured maximum size of the buffer when enumerating a value of type 'Microsoft.EntityFrameworkCore.TestDatabase.TestDatabaseDbSet`1[TestNamespace.TestClass]'. This limit is in place to prevent infinite streams of 'IAsyncEnumerable<>' from continuing indefinitely. If this is not a programming mistake, consider ways to reduce the collection size, or consider manually converting 'Microsoft.EntityFrameworkCore.TestDatabase.TestDatabaseDbSet`1[TestNamespace.TestClass]' into a list rather than increasing the limit.
Microsoft.AspNetCore.Mvc.Infrastructure.AsyncEnumerableReader.ReadTestDatabase<T>(IAsyncEnumerable<object> value)

Stack Query Cookies Headers Routing
InvalidOperationException: 'AsyncEnumerableReader' reached the configured maximum size of the buffer when enumerating a value of type 'Microsoft.EntityFrameworkCore.TestDatabase.TestDatabaseDbSet`1[TestNamespace.TestClass]'. This limit is in place to prevent infinite streams of 'IAsyncEnumerable<>' from continuing indefinitely. If this is not a programming mistake, consider ways to reduce the collection size, or consider manually converting 'Microsoft.EntityFrameworkCore.TestDatabase.TestDatabaseDbSet`1[TestNamespace.TestClass]' into a list rather than increasing the limit.
Microsoft.AspNetCore.Mvc.Infrastructure.AsyncEnumerableReader.ReadTestDatabase<T>(IAsyncEnumerable<object> value)
Microsoft.AspNetCore.Mvc.Infrastructure.AsyncEnumerableReader.ReadTestDatabase<T>(IAsyncEnumerable<object> value)
Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor.ExecuteAsyncEnumerable(ActionContext context, ObjectResult result, IAsyncEnumerable<object> asyncEnumerable)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeResultFilters>g__Awaited|27_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Show raw exception details
System.InvalidOperationException: 'AsyncEnumerableReader' reached the configured maximum size of the buffer when enumerating a value of type 'Microsoft.EntityFrameworkCore.TestDatabase.TestDatabaseDbSet`1[TestNamespace.TestClass]'. This limit is in place to prevent infinite streams of 'IAsyncEnumerable<>' from continuing indefinitely. If this is not a programming mistake, consider ways to reduce the collection size, or consider manually converting 'Microsoft.EntityFrameworkCore.TestDatabase.TestDatabaseDbSet`1[TestNamespace.TestClass]' into a list rather than increasing the limit.
   at Microsoft.AspNetCore.Mvc.Infrastructure.AsyncEnumerableReader.ReadTestDatabase[T](IAsyncEnumerable`1 value)
   at Microsoft.AspNetCore.Mvc.Infrastructure.AsyncEnumerableReader.ReadTestDatabase[T](IAsyncEnumerable`1 value)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor.ExecuteAsyncEnumerable(ActionContext context, ObjectResult result, IAsyncEnumerable`1 asyncEnumerable)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeResultFilters>g__Awaited|27_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

reached the configured maximum size of the buffer when enumerating a value of type Microsoft.EntityFrameworkCore DbSet  This limit is in place to prevent infinite streams from continuing indefinitely. If this is not a programming mistake, consider ways to reduce the collection size, or consider manually converting Microsoft.EntityFrameworkCore DbSet  into a list rather than increasing the limit.

Microsoft.EntityFrameworkCore.DbSet AsyncEnumerable

Answer 1

发生了什么

限制来自MvcOptions.MaxIAsyncEnumerableBufferLimit 属性：

获取或设置 ObjectResultExecutor 将缓冲的 IAsyncEnumerable 的最多条目数。

当 Value 是 IAsyncEnumerable 的一个实例时，ObjectResultExecutor 将在调用选定的格式化程序之前急切地读取枚举并添加到同步集合中。此属性确定允许执行程序缓冲的最大条目数。

（……）

默认为 8192。

使固定

在我看来，从最好到最坏。

A.（首选）不要返回这么多记录

我建议不要从 web api 返回这么多记录。

分页是处理大量记录的一种方法。您可以查看教程：使用 EF Core 添加排序、过滤和分页 - ASP.NET MVC。

B. 调整限制

MvcOptions.MaxIAsyncEnumerableBufferLimit 可以设置为另一个值。

C、不要使用 IAsyncEnumerable

您可以返回IEnumerable- 它将能够处理更多的记录。