When an ASP.NET Core API returns a few records, it is simple to return JSON. When an endpoint must return hundreds of thousands of records, a huge export, or a collection with significant volumes of data, the scenario is different.
A typical solution delivers the response after loading the complete result into memory and serializing it into JSON. When several queries are made at once, this method may result in increased memory use, a delay in the initial response bytes, and strain on the garbage collector.
Streaming allows an application to send data progressively instead of building the complete response before transmission. However, streaming does not automatically solve every memory problem. If the application loads the entire database result into a list before writing the response, it may still consume substantial memory.
This article explains how to stream large JSON responses in ASP.NET Core, how to use System.Text.Json with IAsyncEnumerable<T>, and what to consider when streaming database results in production.
Why Buffering Large JSON Responses Causes Problems
Consider an API that returns a large collection of orders.
app.MapGet("/api/orders", async (OrdersDbContext db) =>
{
var orders = await db.Orders
.Select(order => new
{
order.Id,
order.OrderNumber,
order.Total
})
.ToListAsync();
return Results.Ok(orders);
});
The call to ToListAsync() materializes the complete result in memory. The framework then serializes the collection into the HTTP response.
For a small collection, this is usually acceptable. For a large export, the application may need to hold many objects in memory while preparing the response.
Under heavy traffic, several requests can perform this work simultaneously. Memory usage can increase quickly, leading to garbage collection activity, longer response times, or even an out-of-memory failure.
Streaming can reduce the amount of data that must be held in memory at once. It can also allow the server to begin sending a response before the entire dataset has been processed.
Stream JSON with IAsyncEnumerable<T>
ASP.NET Core supports asynchronous streaming through IAsyncEnumerable<T>. When an endpoint returns an asynchronous sequence, the framework can serialize the items progressively using System.Text.Json, rather than requiring the endpoint to first create a complete list.
For example, consider this endpoint:
using System.Runtime.CompilerServices;
app.MapGet("/api/orders", async (
OrdersDbContext db,
CancellationToken cancellationToken) =>
{
return db.Orders
.OrderBy(order => order.Id)
.Select(order => new OrderResponse(
order.Id,
order.OrderNumber,
order.Total))
.AsAsyncEnumerable();
});
public sealed record OrderResponse(
int Id,
string OrderNumber,
decimal Total);
This example returns an asynchronous sequence instead of calling ToListAsync().
The framework can enumerate the sequence as it writes the JSON response. The database provider can also retrieve results incrementally, depending on its implementation and query behavior.
Notice that the query projects database entities into a smaller response model. This avoids returning every database column and keeps the API contract separate from the database schema.
The CancellationToken parameter is included in the endpoint signature, but this version does not explicitly pass it to the query. For cancellation-aware database execution, use the token in the query operation as shown in the next example.
Pass Cancellation Tokens to the Database Query
Streaming can keep a database query active for as long as the response is being consumed. If the client disconnects, continuing to retrieve records wastes resources.
Pass the request cancellation token to the database operation and use it while enumerating the results.
using System.Runtime.CompilerServices;
app.MapGet("/api/orders", (
OrdersDbContext db,
CancellationToken cancellationToken) =>
{
return StreamOrders(
db,
cancellationToken);
});
static async IAsyncEnumerable<OrderResponse> StreamOrders(
OrdersDbContext db,
[EnumeratorCancellation]
CancellationToken cancellationToken)
{
await foreach (var order in db.Orders
.OrderBy(order => order.Id)
.Select(order => new OrderResponse(
order.Id,
order.OrderNumber,
order.Total))
.AsAsyncEnumerable()
.WithCancellation(cancellationToken))
{
yield return order;
}
}
public sealed record OrderResponse(
int Id,
string OrderNumber,
decimal Total);
The EnumeratorCancellation attribute allows the cancellation token supplied when the asynchronous iterator is enumerated to be applied to the iterator's token.
The WithCancellation() call passes the cancellation token to the asynchronous enumerator. Whether the database operation stops immediately depends on the database provider and its cancellation support.
This example illustrates the streaming pattern, but production code should also consider database command timeouts, connection limits, and how long clients are allowed to consume an export.
Understand the JSON Response Format
When ASP.NET Core serializes an IAsyncEnumerable<T> as a JSON response, it normally produces a JSON array.
For example:
[
{
"id": 1,
"orderNumber": "ORD-1001",
"total": 125.50
},
{
"id": 2,
"orderNumber": "ORD-1002",
"total": 240.00
}
]
The response is still one JSON document. The difference is that the server can write the array progressively as it enumerates the items.
This is useful for clients that download the response as a stream. However, many clients still deserialize the complete JSON array into memory before making the records available to application code.
If clients need to process each record as it arrives, verify that their HTTP and JSON libraries support incremental parsing. Server-side streaming alone does not guarantee that the client processes records incrementally.
Also remember that a streamed JSON array must remain valid JSON. If an exception occurs after the server has already written part of the response, the server generally cannot replace the partial response with a normal error object or a different HTTP status code.
Stream a Custom JSON Response with Utf8JsonWriter
Sometimes an API needs more control over the JSON structure or the serialization process. Utf8JsonWriter allows the application to write JSON directly to the response body.
The following example writes an array of values progressively:
using System.Text.Json;
app.MapGet("/api/numbers", async (
HttpContext context,
CancellationToken cancellationToken) =>
{
context.Response.ContentType = "application/json";
await using var writer = new Utf8JsonWriter(
context.Response.BodyWriter);
writer.WriteStartArray();
for (var i = 1; i <= 1000; i++)
{
cancellationToken.ThrowIfCancellationRequested();
writer.WriteNumberValue(i);
if (i % 100 == 0)
{
await writer.FlushAsync(cancellationToken);
}
}
writer.WriteEndArray();
await writer.FlushAsync(cancellationToken);
});
This endpoint produces a JSON array containing numbers from 1 through 1000.
The example uses HttpResponse.BodyWriter, which provides a PipeWriter for writing response data. Calling FlushAsync() makes buffered output available to the underlying response pipeline.
The periodic flushes demonstrate how an application can send data progressively. They do not guarantee that every group of 100 numbers arrives as a separate network packet. The server, operating system, network, and client can all affect when bytes are actually delivered.
For real-world exports, write data as it becomes available instead of generating an artificial sequence. Avoid flushing after every individual value because frequent flushes can add overhead.
Use Keyset Pagination for Very Large Exports
Streaming a database query does not eliminate the cost of executing that query. A long-running query can hold a connection for an extended period and may place substantial pressure on the database.
For very large exports, keyset pagination is another option. The application retrieves a limited number of records at a time, using a stable ordering key to find the next batch.
For example:
app.MapGet("/api/orders/export", async (
OrdersDbContext db,
CancellationToken cancellationToken) =>
{
const int batchSize = 500;
var lastId = 0;
while (true)
{
var batch = await db.Orders
.AsNoTracking()
.Where(order => order.Id > lastId)
.OrderBy(order => order.Id)
.Select(order => new OrderResponse(
order.Id,
order.OrderNumber,
order.Total))
.Take(batchSize)
.ToListAsync(cancellationToken);
if (batch.Count == 0)
{
break;
}
foreach (var order in batch)
{
// Process the current batch.
}
lastId = batch[^1].Id;
}
return Results.Ok();
});
This example illustrates fetching records in batches. It does not yet stream the records to the client; the response is only returned after the loop completes.
To build a real streaming export, write each batch to the response as it is retrieved. You must also choose a valid output format and define how the response should behave if a later batch fails.
Keyset pagination works best when the ordering key is stable and indexed. If the dataset changes during the export, records can be added or modified while the request is running. Decide whether the export needs a consistent snapshot or can tolerate changes during processing.
- Common Mistakes When Streaming JSON
- Loading everything before returning the response
Calling ToListAsync() on the entire dataset before returning a response can defeat the main memory benefit of streaming.
Retrieve records incrementally or in manageable batches when the dataset is large.
Assuming streaming removes all buffering
The framework, database provider, server, proxy, or client may still buffer some data. Streaming reduces application-level buffering, but it does not guarantee that every layer operates without buffers.
Ignoring client disconnections
A client may cancel a request before the export finishes. Propagate cancellation tokens to database operations and asynchronous enumerators where supported.
Returning an error after the response has started
Once the response body has been partially written, changing the status code or returning a new JSON error object may no longer be possible.
Validate parameters and perform essential checks before starting the response. For long-running exports, consider a background export job that generates a file and exposes its completion status separately.
Holding database connections for too long
Direct database streaming may keep a connection occupied throughout the download. For slow clients, this can reduce the number of concurrent database operations your application can support.
Batching, background generation, and temporary export files may be more appropriate when datasets are extremely large or downloads take a long time.
When Should You Use Streaming?
Streaming is a good fit when clients need large datasets and the application can produce records incrementally. It can reduce peak memory usage and allow the response to begin earlier.
However, not every endpoint needs streaming. For small responses, ordinary JSON serialization is simpler and usually easier to maintain.
For large exports, compare three approaches:
- A normal buffered response for small collections.
- Direct streaming when records can be produced and consumed progressively.
- A background export that generates a downloadable file for very large or long-running jobs.
- Choose based on response size, database behavior, client requirements, and operational limits. Monitor memory usage, database connection utilization, request duration, and failure rates before and after introducing streaming.
Conclusion
Large JSON replies can be streamed by ASP.NET Core without needing the application to first build the complete response in memory. For supported JSON endpoints, returning an IAsyncEnumerable is a simple choice, while Utf8JsonWriter offers greater control over the response format. Use asynchronous query execution, pass cancelation tokens, project just the necessary fields, and think about keyset pagination when a single long-running query is inappropriate for database-backed APIs.
When the entire pipeline is built for incremental processing, streaming works well. To make sure that the solution truly lowers memory load and enhances answer delivery, examine the database, server, network, and client behavior collectively.
HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.
