European ASP.NET 4.5 Hosting BLOG

BLOG about ASP.NET 4, ASP.NET 4.5 Hosting and Its Technology - Dedicated to European Windows Hosting Customer

European ASP.NET Core 10.0 Hosting - HostForLIFE :: Incrementally Reading CBOR data in .NET 11

clock November 3, 2026 11:20 by author Peter

Data is often received by applications via files, IoT devices, APIs, and other services. Loading the full payload into memory before processing it might result in more memory utilization and a slower response as the data is big. Concise Binary Object Representation, or CBOR, is a binary data format intended to compactly represent structured data. It is helpful when applications need to transmit data effectively, particularly when processing overhead and payload size are important considerations.

In .NET, developers can work with CBOR data using APIs provided by the System.Formats.Cbor package. When data arrives in smaller chunks, incremental processing can help an application avoid waiting for the complete payload before beginning its work.

This article explains how CBOR works, how to read CBOR data incrementally in .NET, and how to handle incomplete input safely. The examples focus on the core parsing concepts and the practical considerations involved in processing streamed binary data.

One important detail: the availability of specific APIs depends on the target framework and package version. Verify the APIs supported by the version used in your project rather than assuming that every CBOR feature is built into .NET 11.

What Is CBOR?

CBOR is a binary serialization format standardized for representing structured data. It supports values such as integers, strings, byte strings, arrays, maps, Boolean values, and null.

Unlike JSON, which represents data using readable text, CBOR encodes values in a binary format.

For example, a JSON message might look like this:
{
  "deviceId": 42,
  "temperature": 26.5,
  "active": true
}


CBOR can represent the same information in a compact binary structure.

The binary representation is not intended to be read directly by a person. An application uses a CBOR decoder to interpret its contents.

CBOR vs. JSON

Feature

CBOR

JSON

Representation

Binary

Text

Human readability

Requires decoding tools

Readable without special tools

Data types

Supports binary values and additional native data types

Primarily supports objects, arrays, strings, numbers, Booleans, and null

Payload size

Can be smaller for suitable data

Depends on the structure and textual representation

Debugging

Often requires a decoder or specialized tools

Easy to inspect in logs and text editors

Common use cases

Binary protocols, constrained devices, structured binary messages

Web APIs, configuration files, and general data exchange

CBOR is not automatically smaller or faster for every payload. The result depends on the data, encoding choices, and application implementation.

Why Read CBOR Incrementally?
Consider an application that receives sensor information from thousands of devices. Each device sends a CBOR message containing its identifier, timestamp, and measurements.
A straightforward implementation might collect the entire message into a byte array and then decode it.
For small messages, this is often sufficient. However, large messages or many concurrent requests can increase memory pressure.
Incremental reading allows an application to process data as it becomes available.

The main advantages include:

  • Processing data without requiring one large input buffer.
  • Starting work before the entire input has arrived.
  • Managing memory more carefully for large payloads.
  • Supporting network protocols that deliver data in multiple chunks.

Incremental reading does not mean that every individual CBOR value can be processed before its encoding is complete. The parser still needs enough bytes to recognize and decode the current value. It is also important to distinguish incremental parsing from network streaming. A CBOR decoder processes encoded values, while the network layer determines how bytes arrive.

Understanding CBOR Parsing in .NET

The System.Formats.Cbor package provides the CborReader class for reading CBOR data.
A reader can decode values from a byte buffer. For example, the following program reads a CBOR array containing a device identifier, a temperature, and a status flag.

First, install the package if it is not already available in your project:
dotnet add package System.Formats.Cbor

The package version should be selected according to your project's target framework and dependency requirements.

Reading a Complete CBOR Message
The following example creates a CBOR message and reads its values.
using System;
using System.Formats.Cbor;

byte[] payload;

var writer = new CborWriter();

writer.WriteStartArray(3);
writer.WriteInt32(42);
writer.WriteDouble(26.5);
writer.WriteBoolean(true);
writer.WriteEndArray();

payload = writer.Encode();

var reader = new CborReader(payload);

int arrayLength = reader.ReadStartArray()
    ?? throw new FormatException("An array length is required.");

if (arrayLength != 3)
{
    throw new FormatException("Unexpected CBOR array length.");
}

int deviceId = reader.ReadInt32();
double temperature = reader.ReadDouble();
bool active = reader.ReadBoolean();

reader.ReadEndArray();

if (reader.BytesRemaining != 0)
{
    throw new FormatException("Unexpected trailing CBOR data.");
}

Console.WriteLine($"Device ID: {deviceId}");
Console.WriteLine($"Temperature: {temperature}");
Console.WriteLine($"Active: {active}");


The writer is included to make the example self-contained. In a real application, the payload would usually come from a file, a network connection, or another service.

The reader processes the array in the order in which its values were encoded.

The length check helps reject unexpected structures. The final check ensures that the example does not silently ignore extra bytes after the expected message.

This example reads a complete payload. To process data incrementally, the application needs to handle incomplete input and preserve the bytes that have not yet been consumed.

Handling CBOR Data That Arrives in Chunks

A network stream does not guarantee that a complete CBOR value will arrive in a single read operation.

For example, a message may contain 100 bytes, but the first network read may return only 20 bytes. The remaining bytes may arrive in later reads.

The application must not assume that one read returns one complete message.

A common approach is to maintain a buffer, append newly received bytes, and parse complete values while retaining incomplete data.

For low-level incremental parsing, .NET provides System.Formats.Cbor reader APIs, but the exact approach depends on the package version and the required CBOR structure.

The following example demonstrates the buffering principle using a stream and a memory buffer. It collects one complete payload before decoding it, so it is useful for understanding partial reads but is not a fully incremental CBOR parser.
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;

static async Task<byte[]> ReadPayloadAsync(
    Stream stream,
    int maximumSize,
    CancellationToken cancellationToken = default)
{
    using var buffer = new MemoryStream();
    byte[] chunk = new byte[4096];

    while (true)
    {
        int bytesRead = await stream.ReadAsync(
            chunk.AsMemory(),
            cancellationToken);

        if (bytesRead == 0)
        {
            break;
        }

        if (buffer.Length + bytesRead > maximumSize)
        {
            throw new InvalidDataException(
                "The payload exceeds the permitted size.");
        }

        await buffer.WriteAsync(
            chunk.AsMemory(0, bytesRead),
            cancellationToken);
    }

    return buffer.ToArray();
}


This helper reads until the stream ends and rejects payloads larger than the configured limit. It is suitable only when the stream's end indicates the end of the payload. If the connection stays open or contains multiple messages, the application needs a framing protocol that identifies where each message ends.

The helper also retains the complete payload in memory. For large messages, a more advanced parser should process complete CBOR values as bytes arrive.

Using SequenceReader and Pipelines for Incremental Processing

For high-throughput applications, System.IO.Pipelines can help manage incoming bytes efficiently. A pipeline provides a PipeReader that exposes available input as a ReadOnlySequence<byte>. The application can examine the available data, process what it understands, and retain the remainder for the next read.

The general process is:

  • Read available bytes from the pipeline.
  • Attempt to decode the next CBOR value.
  • Record how many bytes were successfully consumed.
  • Preserve incomplete bytes for the next read.
  • Release consumed memory and continue processing.

SequenceReader<byte> can help navigate a sequence of bytes, but it does not decode CBOR values by itself. The CBOR parser must still understand the format's encoding rules.

A production implementation should use the incremental or sequence-based APIs supported by the selected CBOR library version. If those APIs are unavailable, use a well-tested parser or buffer complete framed messages before decoding them.

Avoid manually implementing the CBOR binary format unless you have a strong reason to do so. Incorrect handling of lengths, nesting, or special values can lead to parsing errors and security problems.

Handling Incomplete and Invalid Data
Incremental parsers must distinguish between incomplete input and invalid input.

Suppose a message begins with an array declaration but ends before all the array elements arrive. The data may be valid but incomplete.

By contrast, a message may contain a value that does not match the expected structure. That is a validation failure rather than a temporary lack of bytes.

A robust implementation should account for the following situations:

Situation

Recommended Handling

More bytes are needed

Preserve the incomplete input and read more data

Invalid CBOR encoding

Reject the message and record an appropriate error

Unexpected value type

Reject the message if it violates the expected schema

Payload exceeds the configured limit

Stop processing and reject the input

Unexpected trailing bytes

Reject the message or handle them according to the protocol

Network connection closes early

Treat the message as incomplete if the required data has not arrived

The parser's exception behavior depends on the specific API. Do not treat every parsing exception as proof that more bytes are needed. Some exceptions indicate malformed data and should not trigger endless retries.

Common Mistakes to Avoid
Assuming One Read Returns the Entire Message

A stream read may return fewer bytes than requested. Always account for partial reads.

Loading Unlimited Data Into Memory

An application that accepts unlimited payload sizes can consume excessive memory. Configure a maximum message size and enforce it while reading.

Ignoring Message Boundaries

CBOR describes data values, but a network protocol must still establish how messages are separated. Use an explicit length prefix, a protocol-defined boundary, or another reliable framing mechanism.

Confusing Incomplete Input With Invalid Input

Incomplete data may become valid when additional bytes arrive. Invalid data should be rejected rather than retried indefinitely.

Accepting Unexpected Data
Check that decoded values have the expected types, lengths, and structure. Reject trailing bytes when the protocol expects exactly one message.

Best Practices for Production Applications

Use these practices when building a CBOR-processing service:

  • Set a maximum payload size based on the application's requirements.
  • Use asynchronous stream operations for network-bound workloads.
  • Preserve unconsumed bytes between parsing operations.
  • Validate the decoded structure before using its values.
  • Handle cancellation and network failures explicitly.
  • Avoid logging complete payloads when they may contain sensitive information.
  • Test with fragmented input, malformed messages, and unexpected value types.
  • Use a maintained CBOR library rather than writing a custom decoder.
  • Measure memory usage and throughput under realistic workloads before choosing a buffering strategy.

These practices improve reliability and help protect the application from malformed or unexpectedly large inputs.

Advantages and Disadvantages
Advantages

Incremental CBOR processing can reduce the need to buffer an entire payload before starting useful work. It can improve memory management and support applications that receive data gradually.

It is particularly useful when processing large messages or handling many concurrent connections.

Disadvantages
Incremental parsing requires more careful buffer management than decoding a complete byte array. Developers must handle incomplete input, message boundaries, parser state, and error recovery correctly.

The implementation can also become more complex if the chosen CBOR library does not expose the required sequence-based or incremental APIs.

Summary

CBOR provides a compact binary format for structured data, and .NET applications can decode it with the System.Formats.Cbor package. For small payloads, reading the complete message into memory and using CborReader is often the simplest solution. For larger or continuously arriving payloads, incremental processing can reduce buffering overhead, but it requires careful handling of incomplete data and message boundaries.

The most important considerations are to choose APIs supported by your package version, enforce payload limits, validate the decoded structure, and distinguish incomplete input from malformed data. Start with a simple implementation that matches your message sizes and performance requirements. Introduce incremental parsing when the application's workload justifies the additional complexity.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



European ASP.NET Core 10.0 Hosting - HostForLIFE :: Why Determinism Matters for Protected Assemblies?

clock October 26, 2026 12:09 by author Peter

An unprotected Release build is already subject to non-deterministic factors (timestamps, path embedding, culture settings). Once an obfuscator is introduced, additional sources of variation appear:

  • Random name generation and control-flow seeds
  • Order of member processing
  • Timing of strong-name signing relative to IL rewriting
  • Whether the rename mapping file is retained and versioned

Without deliberate controls, teams frequently encounter two related problems:

  • Two CI runs from the identical commit produce different SHA-256 hashes for the protected output.
  • A production stack trace cannot be mapped back to source because the corresponding rename map was never archived or belongs to a different build.

Both problems are process issues, not fundamental limitations of modern obfuscation tools.

Enabling Deterministic Compilation
The first control point is the compiler itself. In the project file, enable deterministic mode for Release configurations:
<PropertyGroup Condition="'$(Configuration)' == 'Release'">
  <Deterministic>true</Deterministic>
</PropertyGroup>


With this setting, MSBuild produces assemblies whose content is stable when the inputs (source, references, and compiler options) are identical. Deterministic mode is a prerequisite; without it, later attempts to stabilize the protected output will still show hash drift.

Stabilizing the Protection Configuration
Most obfuscators that support command-line automation accept a project or settings file. For tools that use an XML configuration (for example an .opq file), two practices are essential:

  • Store the configuration file in source control next to the project.
  • Set an explicit, fixed random seed inside that configuration so that renaming and control-flow transformations are repeatable.

Treat the settings file as infrastructure code. Changes to protection rules should go through the same pull-request review process as changes to application code. This prevents accidental drift between developer machines and build agents.

Strong-Name Signing Order

Obfuscation rewrites IL and metadata. Any strong-name signature applied before protection is invalidated by those changes. The correct sequence is:

  • Compile the assembly (deterministic Release build).
  • Run the obfuscator / protector against the compiled output.
  • Re-sign the protected assembly with the strong-name key, either as part of the obfuscator’s own signing step or immediately afterward with sn.exe.

Never commit private key material (.snk or .pfx) to the repository. Inject the key at build time from a secret store (Azure Key Vault, GitHub Secrets, or an equivalent vault), write it to a temporary path, pass that path to the tool, and delete the file when the step completes.

Archiving the Rename Map
During protection, the tool typically emits a rename mapping file (commonly an XML document). That file is the only reliable way to translate a production stack of obfuscated names back into original method and type names.

The map must be treated as a first-class build artifact:

  • Publish it under the same build number or release tag as the protected binary.
  • Store it in a durable location (pipeline artifacts, symbol server, or internal package feed).
  • Ensure the incident-response process can locate the map that corresponds to the exact binary that was deployed.

If the map is missing or belongs to a different build, de-obfuscation of crash dumps becomes impossible.
Integrating into a CI/CD Pipeline

A typical Azure Pipelines fragment that respects the controls above looks like this (paths and tool name adjusted to the environment):
- task: MSBuild@1
  displayName: 'Build Release (Deterministic)'
  inputs:
    solution: '**/*.sln'
    configuration: 'Release'
    msbuildArguments: '/p:Deterministic=true'

- task: CmdLine@2
  displayName: 'Protect assemblies'
  inputs:
    script: |
      # Key is fetched from secure files / Key Vault into a temp path
      "C:\Tools\Opaquer\Opaquer.exe" ^
        -XMLFILE="$(Build.SourcesDirectory)\src\opaquer.opq" ^
        -SOURCE="$(Build.SourcesDirectory)\src\bin\Release\net8.0\MyApp.dll" ^
        -OUTPUT="$(Build.SourcesDirectory)\src\bin\Release\net8.0\" ^
        -KEY="$(Agent.TempDirectory)\signing.snk" ^
        -WRITELOG="$(Build.ArtifactStagingDirectory)\protect.log"
  condition: and(succeeded(), eq(variables['BuildConfiguration'], 'Release'))

- task: PublishBuildArtifacts@1
  displayName: 'Publish protected binary and rename map'
  inputs:
    pathToPublish: '$(Build.SourcesDirectory)\src\bin\Release\net8.0'
    artifactName: 'ProtectedRelease_$(Build.BuildNumber)'

The important properties of this flow are:

  • Deterministic compilation is explicit.
  • Protection uses a version-controlled settings file with a fixed seed.
  • Signing occurs as part of (or immediately after) protection.
  • Both the protected assembly and its rename map are published under the same build identifier.

Verification Before Release
Before promoting a protected package:

  • Signature check - Run sn -v against the output assembly and confirm a valid strong name.
  • Hash stability - Trigger two pipeline runs from the same commit and compare SHA-256 hashes of the protected assemblies. They should be identical.
  • Functional tests - Execute the automated test suite against the protected binary, not only against the unobfuscated build.
  • Map presence - Confirm the rename map for that build number was published and is retrievable.

Any failure in the hash comparison almost always points back to a missing fixed seed or Deterministic mode still disabled.

Common Failures and Remedies

Symptom Typical cause Remedy
FileLoadException / strong-name validation failed Assembly signed before obfuscation Re-sign inside the protection step or immediately after
Tool exits with “missing dependency” Search path does not include output or reference folders Supply explicit library search paths to the CLI
Production stack cannot be decoded Rename map never archived or belongs to another build Enforce map publishing as a required pipeline artifact
SHA-256 differs between identical commits Deterministic mode off or random seed not fixed Enable <Deterministic>true</Deterministic> and set a fixed seed in the settings file

Recommended Practices

  • Keep the protection settings file under source control and review changes through pull requests.
  • Inject signing keys at build time from a secret store; never store private keys in the repository.
  • Publish the rename map under the same release identifier as the binary.
  • Gate the protection step on successful Release compilation so Debug builds are never protected by accident.
  • Prefer self-hosted agents when the protection tool must be installed on the build machine.

Conclusion
Deterministic protected builds are achieved by controlling three layers: the compiler (Deterministic mode), the obfuscation configuration (fixed seed and version-controlled settings), and the post-processing pipeline (correct signing order and mandatory archiving of the rename map). When these controls are in place, the same commit always yields the same protected binary, and production crash stacks can be decoded reliably. Teams that treat protection configuration and symbol maps as first-class build artifacts eliminate an entire class of “we cannot read the crash” incidents without changing the underlying protection technology.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



European ASP.NET Core 10.0 Hosting - HostForLIFE :: How Can a NuGet Package Be Verified Before Including It in Your Project?

clock October 23, 2026 11:19 by author Peter

Hours of development labor can be saved by adding a NuGet package. You can leverage an existing library for database access, logging, validation, or API connectivity rather than creating everything yourself. However, there may be issues if a package is installed without first being checked. It could cease getting updates, provide features your program doesn't need, create incompatible dependencies, or have known vulnerabilities.

A package that builds successfully is not automatically safe or suitable for production. Before adding a dependency, you should understand who maintains it, what it depends on, how it is licensed, and whether it is appropriate for your project. In this article, we will look at practical ways to inspect a NuGet package, check its dependencies and security advisories, and decide whether it is ready to use in a .NET application.

Why You Should Review a NuGet Package First?

A NuGet package becomes part of your application's dependency chain. Even if you use only one method from a library, its dependencies may introduce additional code into your application.

For example, a package used for HTTP communication might depend on another library for serialization or authentication. If one of those dependencies has a vulnerability or becomes incompatible with your target framework, your application may be affected.

Before installing a package, consider these questions:

  • Does it solve a real problem in the application?
  • Is the package maintained and compatible with the target framework?
  • Does it have known security vulnerabilities?
  • Are its dependencies necessary and trustworthy?
  • Does its license permit your intended use?
  • Can the team maintain the package over time?

These checks do not need to take hours. A few simple commands and a review of the package metadata can catch many common problems.

Step 1: Check the Package Details
Start by reviewing the package information in the NuGet package manager or the package's metadata. Pay attention to the package ID, publisher information, supported frameworks, release history, license, and project documentation.

Do not select a package based only on its name. Two packages may have similar names but different maintainers, capabilities, and dependency requirements.

For example, if you need structured logging, compare packages based on their actual features and compatibility with your application rather than choosing the first result in a search.

Check whether the package has a clear purpose and whether its documentation explains how to configure it correctly.

Review the Package's Release History

A package with recent releases may be easier to maintain, but release frequency alone does not prove that it is safe.

Look for signs that the maintainers address bug reports, publish fixes, and document breaking changes. An older package may still be appropriate if it is stable and actively supported for your use case.

Be cautious when a package has unclear ownership, unexplained changes, or documentation that does not match its published behavior.

Step 2: Inspect the Package Using the .NET CLI

You can search for packages and inspect their metadata without immediately adding them to your project.

To search NuGet for a package, run:
dotnet package search Serilog

This searches for packages matching the supplied term. Review the results and identify the package that matches your requirements.

To inspect a package already installed in your project, run:
dotnet list package

This displays the project's package references.

For more detail about a particular dependency, you can use:
dotnet list package --include-transitive

This includes transitive dependencies: packages brought into your project by other packages.

The exact output depends on your .NET SDK version and project type. If a command is unavailable in your environment, check the syntax supported by the installed SDK.

Step 3: Check Package Compatibility

Before adding a package, confirm that it supports your project's target framework.

For example, a project might contain the following configuration:
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

</Project>


The project targets .NET 8 in this example. A NuGet package may support several target frameworks, such as .NET Standard and specific versions of .NET. However, package compatibility depends on the frameworks declared by the package and the compatibility rules used by NuGet.

Before installation, check the package's supported frameworks and any documented runtime requirements.

After adding a package, restore and build the project:
dotnet restore
dotnet build


A successful build confirms that the project can compile with the resolved dependencies. It does not prove that the package is secure or that its runtime behavior is correct.

For packages that perform important operations, such as database access or authentication, add tests that verify the expected behavior.

Step 4: Check for Known Vulnerabilities
Security should be part of package selection, not something you check only after deployment.

The .NET CLI can report known vulnerabilities affecting packages in your project.

Run:
dotnet list package --vulnerable

To include transitive dependencies in the report, use:
dotnet list package --vulnerable --include-transitive

These commands help identify package versions associated with known security advisories in the available advisory data.

If the output reports a vulnerability, review the affected package version, severity, available fixes, and the conditions required for exploitation.

For example, a vulnerability may affect only a specific feature or configuration. That information helps you assess the risk, but it does not mean you should ignore the finding when the feature is not obviously used.

If a fixed version is available, determine whether upgrading is compatible with your application and test the change before deployment.

You should also remember that an empty vulnerability report does not guarantee that a package is safe. Newly discovered issues, incomplete advisory data, or vulnerabilities that have not been publicly reported may not appear.

Step 5: Review Direct and Transitive Dependencies

A direct dependency is a package your project references explicitly. A transitive dependency is brought in by another package.

Suppose your application references a logging library, and that library depends on another package for configuration. Your application now has an additional dependency even though you did not install it directly.

Use this command to inspect the dependency list:
dotnet list package --include-transitive

Review packages that introduce many dependencies, especially when those dependencies are unrelated to the functionality you need.

A larger dependency tree is not automatically bad, but every additional dependency adds another component that may require updates, compatibility checks, and security review.

If a vulnerability appears in a transitive dependency, check whether an updated version of the parent package resolves it. Avoid forcing a different dependency version without checking compatibility, because the parent package may rely on particular behavior.

Step 6: Check the Package License

Package licensing determines how you may use, modify, and distribute the software. This matters when developing commercial applications, distributing software to customers, or incorporating packages into products with specific licensing requirements.

Check the license information provided by the package maintainers. Confirm that the license file or declared license matches the actual package and that your intended use complies with its terms.

Do not assume that a publicly available package can be used without restrictions.

For business-critical applications, record the licenses of direct and transitive dependencies and follow your organization's software approval process. If the licensing terms are unclear, request a review from the appropriate legal or compliance team.

Step 7: Install the Package Using an Explicit Version
Once you have reviewed the package, install it with a deliberate version selection.

For example:
dotnet add package Serilog --version 4.0.0

This command illustrates installing a specific package version. The version shown is an example, not a recommendation to use that version in a new application.
Select a version that is compatible with your project and meets your maintenance and security requirements.

After installation, inspect the project file. A package reference may look like this:
<ItemGroup>
  <PackageReference Include="Serilog" Version="4.0.0" />
</ItemGroup>


Keep the version consistent with the one you reviewed and tested.

For projects that use central package management, package versions may instead be declared in a Directory.Packages.props file. Check the configuration used by your repository before changing individual project files.

Commit the appropriate project configuration and lockfiles used by your build process so that other developers and CI/CD agents can restore consistent dependencies.

Step 8: Test the Package Before Production

Installing a package is only the beginning. Verify that it works correctly in the application.
Follow these steps:

  • Restore the project and confirm that dependency resolution succeeds.
  • Build the solution and review warnings or errors.
  • Write tests for the functionality that uses the package.
  • Test failure cases, invalid inputs, and unexpected responses where relevant.
  • Review runtime behavior and application logs.
  • Run the project's security and dependency checks.
  • Verify the result in a staging environment before deploying it.

If the package performs network operations, test timeouts and connection failures. If it handles sensitive information, verify that it does not expose secrets through logs or error messages.

For packages that affect performance, measure their impact under representative workloads instead of assuming they will improve performance.

Common Mistakes to Avoid

One mistake is selecting a package only because it has a high download count. Popularity can indicate adoption, but it does not establish security, maintenance quality, or compatibility.

Another mistake is installing the latest version without checking breaking changes. A newer release may require changes to your code or target framework.

Do not rely only on direct dependency checks. Vulnerabilities can exist in transitive packages that your project receives indirectly.

Also avoid suppressing security warnings without recording why the finding is not applicable or how the risk is being managed.

Finally, do not assume that a package is trustworthy simply because it is available through NuGet. Review the publisher, package identity, version history, license, and dependency tree.

Best Practices for Managing NuGet Dependencies

For a maintainable .NET project, follow these practices:

  • Prefer packages that have clear documentation and a defined purpose.
  • Review compatibility before installation.
  • Run vulnerability checks regularly, including transitive dependencies.
  • Keep package versions explicit and manage updates through source control.
  • Review licenses before using packages in commercial products.
  • Test package upgrades before releasing them.
  • Remove dependencies that are no longer needed.
  • Use automated dependency and security checks in CI/CD pipelines.
  • Investigate warnings instead of treating a successful build as proof of safety.

For larger teams, maintain a record of approved dependencies and establish a process for handling vulnerable or abandoned packages.

Advantages and Limitations of Package Review
Reviewing a package before installation can reduce avoidable security risks, prevent compatibility problems, and make dependency updates easier to manage.

It also helps teams understand which external libraries their applications rely on and why those libraries are necessary.

However, package metadata and automated security reports cannot prove that every line of code is safe. A package may contain an undiscovered vulnerability, and a well-maintained package can still be unsuitable for a particular application.

Use package review as one part of a broader development process that includes secure coding, testing, monitoring, and regular maintenance.

Summary

Before adding a NuGet package, check its purpose, publisher, compatibility, license, release history, and dependencies. Use the .NET CLI to inspect installed packages and identify known vulnerabilities, including issues in transitive dependencies.

Choose a version that fits your project, test it before production, and keep dependency configuration under source control. A few minutes spent reviewing a package can prevent unnecessary dependencies, difficult upgrades, and avoidable security problems later.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



European ASP.NET Core 10.0 Hosting - HostForLIFE :: How to Include a Rate Restricting to APIs for ASP.NET Core

clock October 19, 2026 11:03 by author Peter

When a client submits too many queries in a short amount of time, an API may become sluggish or unavailable. A poorly designed application, an automated script, an unexpected surge in traffic, or a deliberate attempt to overwhelm the service can all cause this. Rate restriction aids in regulating the frequency of client access to an API. Developers may create rules that restrict requests based on time windows, concurrent requests, or other permitted criteria using ASP.NET Core's built-in rate limitation middleware.

Setting a request restriction is simply one aspect of the approach, though. When restrictions are exceeded, the policy must reflect the usage patterns of the API, identify clients effectively, provide helpful replies, and function well across many application instances.

This article describes how to apply rules to endpoints, return HTTP 429 replies, establish rate limitation in ASP.NET Core, and steer clear of typical implementation errors.

What Is Rate Limiting?
Rate limiting controls the number of requests allowed within a defined period or the number of requests that can execute concurrently. For example, an API might allow a client to make 100 requests per minute. If the client exceeds that limit, the API can reject additional requests until the applicable limit permits more traffic.

Rate limiting can help protect APIs from excessive usage, but it does not replace authentication, authorization, input validation, or network-level protection.

ASP.NET Core provides several built-in rate-limiting algorithms.

Algorithm

How it works

Typical use

Fixed window

Allows a configured number of requests during each time window

Simple request quotas

Sliding window

Distributes the limit across overlapping time segments

Smoother request limits

Token bucket

Refills tokens over time and allows controlled bursts

APIs that need limited burst capacity

Concurrency limiter

Limits the number of requests executing simultaneously

Expensive or resource-intensive operations

The first three algorithms limit request frequency over time. A concurrency limiter controls simultaneous operations rather than enforcing a requests-per-minute quota.

Choosing the right algorithm depends on the endpoint and the resources it consumes.

Add Rate Limiting Middleware

ASP.NET Core includes rate limiting middleware through the Microsoft.AspNetCore.RateLimiting namespace. In modern ASP.NET Core applications, you can configure it in Program.cs.

The following example uses a fixed-window policy:
using System.Threading.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode =
        StatusCodes.Status429TooManyRequests;

    options.AddFixedWindowLimiter("api", limiter =>
    {
        limiter.PermitLimit = 100;
        limiter.Window = TimeSpan.FromMinutes(1);
        limiter.QueueLimit = 0;
        limiter.AutoReplenishment = true;
    });
});

var app = builder.Build();

app.UseRouting();

app.UseRateLimiter();

app.MapControllers();

app.Run();


This configuration creates a policy named api. It allows up to 100 requests per minute for each limiter partition. Requests that exceed the limit are rejected with HTTP 429 because the queue limit is zero.The example defines the policy but does not automatically apply it to every endpoint. You must attach the policy to the endpoints that should use it.

The middleware should run after routing when endpoint-specific rate-limiting policies are being applied. If authentication is part of the design, place authentication and authorization middleware in an order that allows the limiter to use the appropriate authenticated identity.

Apply a Rate-Limiting Policy to a Controller

For a controller-based API, use the [EnableRateLimiting] attribute to apply a named policy.

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.RateLimiting;

[ApiController]
[Route("api/products")]
[EnableRateLimiting("api")]
public class ProductsController : ControllerBase
{
    [HttpGet]
    public IActionResult GetProducts()
    {
        return Ok(new[]
        {
            new { Id = 1, Name = "Keyboard" },
            new { Id = 2, Name = "Mouse" }
        });
    }
}

Requests to this controller now use the api rate-limiting policy.

You can also apply a policy to a specific endpoint instead of an entire controller. For example, a product listing endpoint might allow a higher request rate than an expensive report-generation endpoint.

For minimal APIs, apply the policy with RequireRateLimiting:

app.MapGet("/api/products", () =>
{
    return Results.Ok(new[]
    {
        new { Id = 1, Name = "Keyboard" },
        new { Id = 2, Name = "Mouse" }
    });
})
.RequireRateLimiting("api");

Use endpoint-level policies when different parts of the API have different performance or business requirements.

Choose the Right Rate-Limiting Algorithm
Fixed-window limiting

A fixed-window limiter counts requests within a fixed period. Once the window expires, the counter resets.
options.AddFixedWindowLimiter("fixed", limiter =>
{
    limiter.PermitLimit = 60;
    limiter.Window = TimeSpan.FromMinutes(1);
    limiter.QueueLimit = 0;
    limiter.AutoReplenishment = true;
});

This configuration allows 60 requests per minute per partition.

Fixed windows are straightforward to configure and understand. However, they can allow bursts around window boundaries. A client might send 60 requests immediately before a window ends and another 60 just after the next window begins.

For workloads where boundary bursts are acceptable, a fixed window can be sufficient.

Sliding-window limiting

A sliding-window limiter divides a larger time window into smaller segments. When a new segment begins, older request counts gradually stop contributing to the active window.

options.AddSlidingWindowLimiter("sliding", limiter =>
{
    limiter.PermitLimit = 60;
    limiter.Window = TimeSpan.FromMinutes(1);
    limiter.SegmentsPerWindow = 6;
    limiter.QueueLimit = 0;
    limiter.AutoReplenishment = true;
});


This policy uses six segments in a one-minute window. It smooths the transition between windows compared with a fixed-window limiter.

Sliding windows are useful when you want to limit request frequency without allowing large bursts at window boundaries. They do not guarantee perfectly even spacing between every request.

Token-bucket limiting

A token bucket starts with a configured number of tokens and replenishes tokens over time. Each accepted request consumes a token.
options.AddTokenBucketLimiter("token-bucket", limiter =>
{
    limiter.TokenLimit = 20;
    limiter.TokensPerPeriod = 10;
    limiter.ReplenishmentPeriod =
        TimeSpan.FromSeconds(30);
    limiter.QueueLimit = 0;
    limiter.AutoReplenishment = true;
});


This configuration permits a burst of up to 20 requests when the bucket is full, then replenishes 10 tokens every 30 seconds.

Token buckets are useful when an API should tolerate short bursts while controlling sustained traffic. The precise behavior depends on the token capacity, replenishment rate, and current token count.

Concurrency limiting
A concurrency limiter restricts how many requests can execute at the same time.
options.AddConcurrencyLimiter("expensive", limiter =>
{
    limiter.PermitLimit = 10;
    limiter.QueueLimit = 0;
});


This policy allows up to 10 concurrent requests per partition. It does not limit the total number of requests a client can make over a minute if those requests complete quickly.

Concurrency limiting is useful for expensive operations such as report generation, large exports, or resource-intensive queries. It can protect a service from having too many costly operations running simultaneously.

Partition Requests by Client

The earlier examples use a named policy, but a limiter's partitioning strategy determines which requests share the same limit.

A global limiter can apply a common limit to all requests. A partitioned limiter can maintain separate limits for different users, API keys, tenants, or other client identifiers.

For example, the following policy uses an authenticated user's identity when available and otherwise falls back to the remote IP address:
using System.Threading.RateLimiting;
using Microsoft.AspNetCore.RateLimiting;

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode =
        StatusCodes.Status429TooManyRequests;

    options.AddPolicy("per-client", context =>
    {
        var userId = context.User.Identity?.IsAuthenticated == true
            ? context.User.Identity.Name
            : null;

        var clientKey = !string.IsNullOrWhiteSpace(userId)
            ? $"user:{userId}"
            : $"ip:{context.Connection.RemoteIpAddress}";

        return RateLimitPartition.GetFixedWindowLimiter(
            clientKey,
            _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 60,
                Window = TimeSpan.FromMinutes(1),
                QueueLimit = 0,
                AutoReplenishment = true
            });
    });
});

Apply this policy to the required endpoints:
app.MapGet("/api/orders", () =>
{
    return Results.Ok("Order list");
})
.RequireRateLimiting("per-client");


This is a starting point rather than a complete identity strategy. Identity.Name may be null or unsuitable for partitioning if the authentication configuration does not provide a stable unique value. In a production application, use a trusted, stable user identifier or a validated API-key identifier.

The partition key should also be normalized and bounded. Creating partitions from arbitrary attacker-controlled strings can consume memory as the number of distinct keys grows.

For anonymous clients, IP-based partitioning is only as reliable as the remote address presented to the application. When the application is behind a reverse proxy or load balancer, configure forwarded-header processing and trust only known proxies. Do not trust arbitrary client-supplied forwarding headers, because an attacker could manipulate the apparent IP address to bypass a limit.

Return a Useful HTTP 429 Response

When a client exceeds a rate limit, HTTP 429 Too Many Requests is the standard status code for communicating that the request was rejected because of rate limiting.

ASP.NET Core lets you customize the rejection response:
builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode =
        StatusCodes.Status429TooManyRequests;

    options.OnRejected = async (context, token) =>
    {
        context.HttpContext.Response.ContentType =
            "application/json";

        await context.HttpContext.Response.WriteAsJsonAsync(
            new
            {
                error = "rate_limit_exceeded",
                message =
                    "Too many requests. Please try again later."
            },
            cancellationToken: token);
    };
});

This produces a JSON response when the limiter rejects a request. The application can also include a suitable Retry-After header when it can calculate a reliable retry time for the selected policy. Do not promise a precise retry time unless the application can determine it accurately. For a token bucket or a partitioned policy, the correct retry interval may depend on the current state of the limiter.

Clients should treat HTTP 429 as a signal to slow down. They should avoid immediately retrying every rejected request, which can increase pressure on the service.

Understand In-Memory Limits in Multi-Instance Deployments
The built-in ASP.NET Core rate limiter maintains its limiter state in the application process. It is not automatically a shared, distributed rate limiter.
Suppose an application runs on three server instances behind a load balancer. Each instance has its own in-memory counter. If each instance allows 100 requests per minute for a given partition, a client distributed across all three instances may be able to make substantially more than 100 requests during that period.

This behavior is important when enforcing account quotas, paid API plans, or security-related limits.

For simple per-instance protection, the built-in limiter may be sufficient. For a strict limit shared across multiple instances, use a distributed rate-limiting solution or enforce the quota at a gateway or service that maintains shared state.

An API gateway can also provide an additional layer of protection before requests reach the application. However, gateway limits and application-level limits solve different problems and may be used together.

Configure Rate Limiting Alongside Authentication
A rate limiter should use an identity that matches the intended policy.

For public endpoints, limiting by IP address may be useful. For authenticated APIs, limiting by user or tenant can provide fairer quotas. For API-key-based services, a validated key identifier can be more appropriate than the source IP.

If a policy depends on HttpContext.User, ensure that authentication runs before rate limiting evaluates the endpoint policy.

A typical middleware sequence for a controller application is:
app.UseRouting();

app.UseAuthentication();
app.UseRateLimiter();
app.UseAuthorization();

app.MapControllers();


The exact ordering should match the application's routing, authentication, and authorization requirements. If authentication endpoints themselves need protection, consider a separate policy that can also protect anonymous requests.

Rate limiting does not replace authorization. A client that remains under its rate limit must still be prevented from accessing resources it is not permitted to use.

Test Rate-Limiting Behavior

Do not validate a rate limiter only by inspecting its configuration. Send enough requests to confirm that it accepts requests within the limit and rejects excess requests.

For a policy that allows five requests, a test can send requests sequentially and inspect their status codes. Keep the test isolated from other requests sharing the same partition, and account for the configured time window.

Verify that:

  • Requests within the limit succeed.
  • Requests beyond the limit receive HTTP 429.
  • The rejection response has the expected format.
  • Different users or tenants receive separate limits when intended.
  • Anonymous requests cannot bypass the policy by manipulating untrusted headers.
  • Cancellation and timeouts behave appropriately for queued requests.
  • The effective limit is correct when the application runs on multiple instances.

If the application uses a queue, test the queue behavior as well. Queuing can smooth short bursts, but it can also increase latency and consume resources. For APIs that should reject excess traffic immediately, a queue limit of zero is often the simpler choice.

Common Mistakes to Avoid

Several implementation mistakes can make rate limiting ineffective or unnecessarily disruptive:

  • Defining a policy but never attaching it to an endpoint.
  • Using one limit for every endpoint despite significant differences in cost.
  • Treating a concurrency limit as a requests-per-minute quota.
  • Trusting arbitrary forwarding headers for client IP addresses.
  • Using unstable or attacker-controlled partition keys.
  • Assuming in-memory limits are shared across application instances.
  • Returning HTTP 500 instead of HTTP 429 when a request is rate-limited.
  • Allowing clients to retry rejected requests immediately.
  • Relying on rate limiting instead of authentication and authorization.
  • Applying limits without testing legitimate burst traffic and expected usage.

The best configuration is one that protects the service without unnecessarily rejecting normal client activity.

Conclusion

Rate-limiting middleware for fixed-window, sliding-window, token-bucket, and concurrency-based rules is included into ASP.NET Core. These systems aid in regulating the frequency of requests and shielding application resources from overuse.

Apply the policy at the proper controller or route after first selecting one that corresponds to the endpoint's workload. Partition limitations using trusted client identities, test the policy in practical scenarios, and return HTTP 429 for denied requests.


For applications running across multiple instances, remember that the built-in limiter is process-local. If you need a strict shared quota, use an appropriate distributed or gateway-level solution.

Rate limiting works best as one part of an API protection strategy that also includes authentication, authorization, monitoring, and deliberate capacity planning.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



European ASP.NET Core 10.0 Hosting - HostForLIFE :: How to Use ASP.NET to Stream Big JSON Responses Core Without Buffering?

clock October 16, 2026 13:46 by author Peter

When an ASP.NET Core API returns a few records, it is simple to return JSON. When an endpoint must return hundreds of thousands of records, a huge export, or a collection with significant volumes of data, the scenario is different.

A typical solution delivers the response after loading the complete result into memory and serializing it into JSON. When several queries are made at once, this method may result in increased memory use, a delay in the initial response bytes, and strain on the garbage collector.

Streaming allows an application to send data progressively instead of building the complete response before transmission. However, streaming does not automatically solve every memory problem. If the application loads the entire database result into a list before writing the response, it may still consume substantial memory.

This article explains how to stream large JSON responses in ASP.NET Core, how to use System.Text.Json with IAsyncEnumerable<T>, and what to consider when streaming database results in production.

Why Buffering Large JSON Responses Causes Problems

Consider an API that returns a large collection of orders.

app.MapGet("/api/orders", async (OrdersDbContext db) =>
{
    var orders = await db.Orders
        .Select(order => new
        {
            order.Id,
            order.OrderNumber,
            order.Total
        })
        .ToListAsync();

    return Results.Ok(orders);
});


The call to ToListAsync() materializes the complete result in memory. The framework then serializes the collection into the HTTP response.

For a small collection, this is usually acceptable. For a large export, the application may need to hold many objects in memory while preparing the response.

Under heavy traffic, several requests can perform this work simultaneously. Memory usage can increase quickly, leading to garbage collection activity, longer response times, or even an out-of-memory failure.

Streaming can reduce the amount of data that must be held in memory at once. It can also allow the server to begin sending a response before the entire dataset has been processed.

Stream JSON with IAsyncEnumerable<T>

ASP.NET Core supports asynchronous streaming through IAsyncEnumerable<T>. When an endpoint returns an asynchronous sequence, the framework can serialize the items progressively using System.Text.Json, rather than requiring the endpoint to first create a complete list.

For example, consider this endpoint:
using System.Runtime.CompilerServices;

app.MapGet("/api/orders", async (
    OrdersDbContext db,
    CancellationToken cancellationToken) =>
{
    return db.Orders
        .OrderBy(order => order.Id)
        .Select(order => new OrderResponse(
            order.Id,
            order.OrderNumber,
            order.Total))
        .AsAsyncEnumerable();
});

public sealed record OrderResponse(
    int Id,
    string OrderNumber,
    decimal Total);


This example returns an asynchronous sequence instead of calling ToListAsync().

The framework can enumerate the sequence as it writes the JSON response. The database provider can also retrieve results incrementally, depending on its implementation and query behavior.

Notice that the query projects database entities into a smaller response model. This avoids returning every database column and keeps the API contract separate from the database schema.

The CancellationToken parameter is included in the endpoint signature, but this version does not explicitly pass it to the query. For cancellation-aware database execution, use the token in the query operation as shown in the next example.
Pass Cancellation Tokens to the Database Query

Streaming can keep a database query active for as long as the response is being consumed. If the client disconnects, continuing to retrieve records wastes resources.

Pass the request cancellation token to the database operation and use it while enumerating the results.
using System.Runtime.CompilerServices;

app.MapGet("/api/orders", (
    OrdersDbContext db,
    CancellationToken cancellationToken) =>
{
    return StreamOrders(
        db,
        cancellationToken);
});

static async IAsyncEnumerable<OrderResponse> StreamOrders(
    OrdersDbContext db,
    [EnumeratorCancellation]
    CancellationToken cancellationToken)
{
    await foreach (var order in db.Orders
        .OrderBy(order => order.Id)
        .Select(order => new OrderResponse(
            order.Id,
            order.OrderNumber,
            order.Total))
        .AsAsyncEnumerable()
        .WithCancellation(cancellationToken))
    {
        yield return order;
    }
}

public sealed record OrderResponse(
    int Id,
    string OrderNumber,
    decimal Total);


The EnumeratorCancellation attribute allows the cancellation token supplied when the asynchronous iterator is enumerated to be applied to the iterator's token.

The WithCancellation() call passes the cancellation token to the asynchronous enumerator. Whether the database operation stops immediately depends on the database provider and its cancellation support.

This example illustrates the streaming pattern, but production code should also consider database command timeouts, connection limits, and how long clients are allowed to consume an export.

Understand the JSON Response Format

When ASP.NET Core serializes an IAsyncEnumerable<T> as a JSON response, it normally produces a JSON array.

For example:
[
  {
    "id": 1,
    "orderNumber": "ORD-1001",
    "total": 125.50
  },
  {
    "id": 2,
    "orderNumber": "ORD-1002",
    "total": 240.00
  }
]


The response is still one JSON document. The difference is that the server can write the array progressively as it enumerates the items.

This is useful for clients that download the response as a stream. However, many clients still deserialize the complete JSON array into memory before making the records available to application code.

If clients need to process each record as it arrives, verify that their HTTP and JSON libraries support incremental parsing. Server-side streaming alone does not guarantee that the client processes records incrementally.

Also remember that a streamed JSON array must remain valid JSON. If an exception occurs after the server has already written part of the response, the server generally cannot replace the partial response with a normal error object or a different HTTP status code.

Stream a Custom JSON Response with Utf8JsonWriter

Sometimes an API needs more control over the JSON structure or the serialization process. Utf8JsonWriter allows the application to write JSON directly to the response body.

The following example writes an array of values progressively:
using System.Text.Json;

app.MapGet("/api/numbers", async (
    HttpContext context,
    CancellationToken cancellationToken) =>
{
    context.Response.ContentType = "application/json";

    await using var writer = new Utf8JsonWriter(
        context.Response.BodyWriter);

    writer.WriteStartArray();

    for (var i = 1; i <= 1000; i++)
    {
        cancellationToken.ThrowIfCancellationRequested();

        writer.WriteNumberValue(i);

        if (i % 100 == 0)
        {
            await writer.FlushAsync(cancellationToken);
        }
    }

    writer.WriteEndArray();

    await writer.FlushAsync(cancellationToken);
});


This endpoint produces a JSON array containing numbers from 1 through 1000.

The example uses HttpResponse.BodyWriter, which provides a PipeWriter for writing response data. Calling FlushAsync() makes buffered output available to the underlying response pipeline.

The periodic flushes demonstrate how an application can send data progressively. They do not guarantee that every group of 100 numbers arrives as a separate network packet. The server, operating system, network, and client can all affect when bytes are actually delivered.

For real-world exports, write data as it becomes available instead of generating an artificial sequence. Avoid flushing after every individual value because frequent flushes can add overhead.

Use Keyset Pagination for Very Large Exports

Streaming a database query does not eliminate the cost of executing that query. A long-running query can hold a connection for an extended period and may place substantial pressure on the database.

For very large exports, keyset pagination is another option. The application retrieves a limited number of records at a time, using a stable ordering key to find the next batch.

For example:
app.MapGet("/api/orders/export", async (
    OrdersDbContext db,
    CancellationToken cancellationToken) =>
{
    const int batchSize = 500;
    var lastId = 0;

    while (true)
    {
        var batch = await db.Orders
            .AsNoTracking()
            .Where(order => order.Id > lastId)
            .OrderBy(order => order.Id)
            .Select(order => new OrderResponse(
                order.Id,
                order.OrderNumber,
                order.Total))
            .Take(batchSize)
            .ToListAsync(cancellationToken);

        if (batch.Count == 0)
        {
            break;
        }

        foreach (var order in batch)
        {
            // Process the current batch.
        }

        lastId = batch[^1].Id;
    }

    return Results.Ok();
});


This example illustrates fetching records in batches. It does not yet stream the records to the client; the response is only returned after the loop completes.

To build a real streaming export, write each batch to the response as it is retrieved. You must also choose a valid output format and define how the response should behave if a later batch fails.

Keyset pagination works best when the ordering key is stable and indexed. If the dataset changes during the export, records can be added or modified while the request is running. Decide whether the export needs a consistent snapshot or can tolerate changes during processing.

  • Common Mistakes When Streaming JSON
  • Loading everything before returning the response

Calling ToListAsync() on the entire dataset before returning a response can defeat the main memory benefit of streaming.

Retrieve records incrementally or in manageable batches when the dataset is large.

Assuming streaming removes all buffering

The framework, database provider, server, proxy, or client may still buffer some data. Streaming reduces application-level buffering, but it does not guarantee that every layer operates without buffers.

Ignoring client disconnections

A client may cancel a request before the export finishes. Propagate cancellation tokens to database operations and asynchronous enumerators where supported.

Returning an error after the response has started

Once the response body has been partially written, changing the status code or returning a new JSON error object may no longer be possible.

Validate parameters and perform essential checks before starting the response. For long-running exports, consider a background export job that generates a file and exposes its completion status separately.

Holding database connections for too long
Direct database streaming may keep a connection occupied throughout the download. For slow clients, this can reduce the number of concurrent database operations your application can support.

Batching, background generation, and temporary export files may be more appropriate when datasets are extremely large or downloads take a long time.

When Should You Use Streaming?
Streaming is a good fit when clients need large datasets and the application can produce records incrementally. It can reduce peak memory usage and allow the response to begin earlier.
However, not every endpoint needs streaming. For small responses, ordinary JSON serialization is simpler and usually easier to maintain.

For large exports, compare three approaches:

  • A normal buffered response for small collections.
  • Direct streaming when records can be produced and consumed progressively.
  • A background export that generates a downloadable file for very large or long-running jobs.
  • Choose based on response size, database behavior, client requirements, and operational limits. Monitor memory usage, database connection utilization, request duration, and failure rates before and after introducing streaming.

Conclusion
Large JSON replies can be streamed by ASP.NET Core without needing the application to first build the complete response in memory. For supported JSON endpoints, returning an IAsyncEnumerable is a simple choice, while Utf8JsonWriter offers greater control over the response format. Use asynchronous query execution, pass cancelation tokens, project just the necessary fields, and think about keyset pagination when a single long-running query is inappropriate for database-backed APIs.

When the entire pipeline is built for incremental processing, streaming works well. To make sure that the solution truly lowers memory load and enhances answer delivery, examine the database, server, network, and client behavior collectively.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



European ASP.NET Core 10.0 Hosting - HostForLIFE :: Why the Database Query is Quick Yet Your C# API is Slow

clock October 13, 2026 12:31 by author Peter

Have you ever faced this problem?
Your SQL query runs in 100 milliseconds, but your C# API takes 2 or 3 seconds to give a response.
You check the database and think :

"The database is fast. So why is my API slow?"

This is a common problem in ASP.NET Core applications.

The database is only one part of an API request. There are many other things happening between the time a user sends a request and the time they get a response.

In this article, we will understand some simple and common reasons why a C# API can be slow even when the database query is fast.
First, understand what happens in an API

Suppose we have this API :
[HttpGet("orders")]
public async Task<IActionResult> GetOrders()
{
var orders = await _context.Orders
.ToListAsync();
return Ok(orders);
}


When a user calls this API, many things can happen:


The database is only one step in this process.
So even if the database takes only 100 ms, another part of the application may be taking 2 seconds.

Let's look at some common reasons :

1. You are returning too much data
This is one of the easiest mistakes to make. Imagine your Customers table has 100,000 records.

You write:
var customers = await _context.Customers.ToListAsync();
return Ok(customers);

The database query might be reasonably fast.

But now your application has to:

  • Read all the records.
  • Create C# objects.
  • Convert those objects into JSON.
  • Send all that data to the client.
  • That can take a lot of time.

A better approach , Only get the data you actually need.
var customers = await _context.Customers.Select(x => new
{
x.Id,
x.Name,
x.Email
})
.ToListAsync();
return Ok(customers);


Now you are not sending unnecessary columns.

2. Your API may be making many database calls

Your first query might be fast.

But what if your code runs that query 100 times?

For example:
foreach (var order in orders)
{
var customer = await _context.Customers
.FirstOrDefaultAsync(x => x.Id == order.CustomerId);
}


If you have 100 orders, you could end up making 100 extra database calls.

Each call may be fast. But together, they can make your API slow.

This problem is commonly called the N+1 query problem.

The simple lesson is:

  • One fast query does not always mean a fast API.
  • Always check how many database queries your API is actually making.

3. Entity Framework Core also needs time
Many C# applications use Entity Framework Core to work with databases.

When you run:
var users = await _context.Users.ToListAsync();

the work does not stop when SQL Server returns the data.

Entity Framework Core also needs to:

  • Read the returned data.
  • Create C# objects.
  • Track those objects.
  • Prepare them for your application.
  • For read-only operations, you can sometimes use AsNoTracking():

var users = await _context.Users.AsNoTracking().ToListAsync();

This tells Entity Framework Core that you don't need to track these objects for changes.

For large read-only queries, this can reduce some unnecessary work.

4. JSON conversion can take time

Your API normally sends JSON to the client.

For example, your C# object:
new Customer
{
Id = 1,
Name = "John"
}

may become:

{
"id": 1,
"name": "John"
}


If you return thousands of records with many properties, converting all those objects to JSON can take time.

It also creates a larger response.

So the problem may not be the database.

It could be:

  • Database → Fast
  • C# processing → Slow
  • JSON conversion → Slow

Network → Slow

5. Another API may be slowing you down
Sometimes your API calls another service.

For example:

Your database is fast. But your complete API request is slow because it depends on other services.

This is very common in real-world applications.

6. Business logic can also take time
Suppose your API gets data from the database and then performs a lot of processing :

var orders = await GetOrders();

foreach (var order in orders)
{
// Complex calculations
// Price calculation
// Discount calculation
// Tax calculation
}

The database may take only 100 ms. But your C# code could take another 1 or 2 seconds.

This is why you should measure the complete API, not only the SQL query.

7. How do you find the real problem?
Don't guess. Measure the different parts of your API.

For example :
var stopwatch = Stopwatch.StartNew();
var orders = await _context.Orders
.AsNoTracking().ToListAsync();
stopwatch.Stop();
_logger.LogInformation(
"Database call took {Time} ms",
stopwatch.ElapsedMilliseconds);


Now you can see how long the database operation took from the application's point of view.

You can also measure other parts of your code.

For example:

Process

Time

Database

100 ms

Business Logic

900 ms

JSON Conversion

600 ms

External API

800 ms

Total Time

2400 ms (2.4 seconds)

Now you have a much better idea of where the problem is.

A simple real-world example :

  • Imagine an e-commerce application.
  • A customer opens the My Orders page.
  • The API takes 3 seconds.
  • The developer checks SQL Server:
  • SQL Query = 80 ms

They think:
"The database is not the problem." Then they check the API.

They discover :

They discover :

Process

Time

Database

80 ms

Customer Lookup

400 ms

Product Lookup

700 ms

JSON Conversion

600 ms

Other Processing

500 ms

Network

300 ms

Total Time

~2580 ms (2.58 seconds)

Now the problem is much clearer.

The SQL query itself was fast. The API was slow because of the other work happening around the query.

What should you check when your C# API is slow?

When debugging a slow API, check these things:

  • How many database queries are being executed?
  • Are you loading more data than necessary?
  • Are you using AsNoTracking() for read-only queries where appropriate?
  • Are you making calls to other APIs?
  • Is your C# business logic taking too much time?
  • Is JSON serialization taking too much time?
  • Is the response very large?
  • Is the network slow?
  • Are middleware or authentication steps taking time?

Final thoughts , A slow API does not always mean a slow database.

Your API is a complete process:

The database is only one part of that process. So the next time someone says:
"The SQL query is fast, but the API is slow."

Don't immediately optimize the SQL query.

First, measure the complete request and find where the time is actually being spent. Measure first. Find the slow part. Then optimize it.

Conclusion

A slow API is not always caused by the database. Identify performance bottlenecks, optimize queries, and avoid unnecessary processing to make your API faster and more efficient.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.

 



European ASP.NET Core 10.0 Hosting - HostForLIFE :: Learning MediatR and CQRS in Clean Architecture for ASP.NET Core

clock October 7, 2026 14:26 by author Peter

Controllers frequently become victims of the "fat controller" anti-pattern as applications grow. They create an unmaintainable file with hundreds of lines of code that combines database queries, DTO mapping, business logic, and validation criteria. Additionally, even though reads typically need optimal projections and writes require stringent business rule validation, standard CRUD systems handle reads and writes in the same manner.

Modern corporate systems use Clean Architecture with MediatR and CQRS (Command Query Responsibility Segregation) to address this.

We will examine how CQRS and MediatR collaborate to decouple your application layers, maintain extremely thin controllers, and make your software incredibly scalable and testable in this thorough book.

What is CQRS and MediatR?
CQRS: Splits your data operations into two distinct categories:

  • Commands: Write operations that mutate state (Create, Update, Delete) and typically return nothing or an ID.
  • Queries: Read operations that fetch data without modifying state and return DTOs.

MediatR: An implementation of the Mediator pattern in .NET. It facilitates in-process messaging, decoupling your API controllers from your business logic handlers so that controllers simply act as thin dispatchers.

Step 1: Install MediatR in the Application Layer
Commands and queries belong in your Application layer because they represent business use cases.

Navigate to your YourSolution.Application project and install the core MediatR package:
dotnet add package MediatR

(Note: In modern versions of MediatR, dependency injection extensions are bundled directly into the core package).

Step 2: Organize by Feature Folders
Instead of grouping files by technical role (putting all services together or all classes together), organize your Application layer by features. This keeps everything related to a specific domain action localized:

YourSolution.Application/
├── Features/
│   â””── Products/
│       â”œâ”€â”€ Commands/
│       â”‚   â”œâ”€â”€ CreateProductCommand.cs
│       â”‚   â””── CreateProductCommandHandler.cs
│       â””── Queries/
│           â”œâ”€â”€ GetProductByIdQuery.cs
│           â””── GetProductByIdQueryHandler.cs
└── DependencyInjection.cs


Step 3: Implement a Command (Write Operation)

Commands represent actions that change state. We use C# record types for immutability, paired with a dedicated handler that executes the logic.

1. The Command Record

using MediatR;

namespace YourSolution.Application.Features.Products.Commands
{
    public record CreateProductCommand(string Name, decimal Price) : IRequest<Guid>;
}


2. The Command Handler
The handler contains the business rules and interacts with domain entities and repository interfaces.
using MediatR;
using YourSolution.Domain.Entities;
using YourSolution.Domain.Interfaces;

namespace YourSolution.Application.Features.Products.Commands
{
    public class CreateProductCommandHandler : IRequestHandler<CreateProductCommand, Guid>
    {
        private readonly IProductRepository _productRepository;

        public CreateProductCommandHandler(IProductRepository productRepository)
        {
            _productRepository = productRepository;
        }

        public async Task<Guid> Handle(CreateProductCommand request, CancellationToken cancellationToken)
        {
            // 1. Instantiate domain entity (encapsulating business rules)
            var product = new Product(request.Name, request.Price);

            // 2. Persist via repository interface
            await _productRepository.AddAsync(product);
            await _productRepository.UnitOfWork.SaveChangesAsync(cancellationToken);

            // 3. Return the generated ID
            return product.Id;
        }
    }
}


Step 4: Implement a Query (Read Operation)

Queries fetch data without mutating state and typically return lightweight DTOs.

1. The Query Record
using MediatR;
using YourSolution.Application.DTOs;

namespace YourSolution.Application.Features.Products.Queries
{
    public record GetProductByIdQuery(Guid Id) : IRequest<ProductDto>;
}


2. The Query Handler
using MediatR;
using YourSolution.Application.DTOs;
using YourSolution.Domain.Interfaces;

namespace YourSolution.Application.Features.Products.Queries
{
    public class GetProductByIdQueryHandler : IRequestHandler<GetProductByIdQuery, ProductDto>
    {
        private readonly IProductRepository _productRepository;

        public GetProductByIdQueryHandler(IProductRepository productRepository)
        {
            _productRepository = productRepository;
        }

        public async Task<ProductDto> Handle(GetProductByIdQuery request, CancellationToken cancellationToken)
        {
            var product = await _productRepository.GetByIdAsync(request.Id);
            
            if (product == null)
                throw new KeyNotFoundException($"Product with ID {request.Id} was not found.");

            // Map domain model to a lightweight DTO
            return new ProductDto(product.Id, product.Name, product.Price);
        }
    }
}

Step 5: Register MediatR in Dependency Injection
To make MediatR discover your command and query handlers automatically, add an extension method inside your Application layer:

using Microsoft.Extensions.DependencyInjection;

namespace YourSolution.Application
{
    public static class DependencyInjection
    {
        public static IServiceCollection AddApplication(this IServiceCollection services)
        {
            // Scans the assembly and registers all IRequestHandler implementations
            services.AddMediatR(cfg => cfg.RegisterServicesFromAssembly(typeof(DependencyInjection).Assembly));

            return services;
        }
    }
}


Ensure this is invoked in your API's Program.cs: builder.Services.AddApplication();.

Step 6: Keep API Controllers Thin
With CQRS and MediatR handling the heavy lifting, your API controllers become remarkably streamlined. They no longer contain business logic or database dependencies; instead, they simply inject IMediator and dispatch requests.
using MediatR;
using Microsoft.AspNetCore.Mvc;
using YourSolution.Application.Features.Products.Commands;
using YourSolution.Application.Features.Products.Queries;

namespace YourSolution.API.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class ProductsController : ControllerBase
    {
        private readonly IMediator _mediator;

        public ProductsController(IMediator mediator)
        {
            _mediator = mediator;
        }

        [HttpPost]
        public async Task<IActionResult> Create([FromBody] CreateProductCommand command)
        {
            var productId = await _mediator.Send(command);
            return CreatedAtAction(nameof(GetById), new { id = productId }, productId);
        }

        [HttpGet("{id}")]
        public async Task<IActionResult> GetById(Guid id)
        {
            var product = await _mediator.Send(new GetProductByIdQuery(id));
            return Ok(product);
        }
    }
}

Summary of Benefits
By combining Clean Architecture with CQRS and MediatR, you achieve:
Single Responsibility Principle: Every command and query handler has exactly one job and one reason to change.
Extreme Testability: You can unit test any command or query handler in isolation without spinning up an ASP.NET Core web server or database.

Scalability: Read models and write models can evolve independently as your application requirements grow.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



European ASP.NET Core 10.0 Hosting - HostForLIFE :: Redact Sensitive Data from AI Application Logs in .NET

clock October 5, 2026 13:48 by author Peter

Before invoking the logger, project each event into a specific set of allowed fields to prevent sensitive data from appearing in AI application logs. Instead of relying on a final text replacement to capture everything, do not feed raw prompts, nested request objects, response bodies, or error messages into regular operational logs.

An operation identification, a controlled route name, status, duration, and a fixed error category are all recorded in the C# sample that follows. It renders renders renders renders renders renders renders renders renders renders renders renders.

The goal is a compact, inspectable logging contract with practical diagnostic evidence. This sample addresses one logging border; it does not verify an application's whole telemetry infrastructure.

Why can structured logs still expose sensitive data?
Structured logging preserves named fields so tools can search and aggregate events. Those fields can contain sensitive values just as easily as a formatted message can.

Masking the visible message does not establish that the underlying state is safe. A sink can serialize structured values separately. An exception can carry sensitive text in its message, inner exception, or data dictionary. A nested payload can acquire a new field after the original redaction code was written. The OWASP logging guidance identifies data such as credentials and sensitive personal information as requiring exclusion or appropriate handling. The practical implication here is to decide which operational facts are needed before selecting what to record.

What should the event contain?

Start with questions the operations team needs to answer: which route failed, how long the attempt took, and which related events belong to the same operation.

Input

Logged form in this example

Reason

Application-generated operation ID

GUID

Correlate attempts without recording conversation text

Provider route

Controlled label

Group behavior without exposing a credential or URL

HTTP status

Validated number

Inspect response classes

Duration

Nonnegative milliseconds

Diagnose slow attempts

Exception

Fixed category

Identify a broad failure type without raw exception content

Prompt and nested payload

Constant marker

Prevent payload contents entering this event

The operation ID should be generated for the workflow, not copied from arbitrary user input. Correlation identifiers can still reveal activity patterns, so access and retention remain relevant even when content is omitted.

What environment does the example require?
The code was compiled with SDK 8.0.414 and C# 12 and executed on .NET 8.0.20 on Linux. It uses Microsoft.Extensions.Logging from the ASP.NET Core shared framework.
Use a .NET 8 console project with a FrameworkReference to Microsoft.AspNetCore.App. This complete project file supplies that reference:
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <LangVersion>12</LangVersion>
  </PropertyGroup>
  <ItemGroup>
    <FrameworkReference Include="Microsoft.AspNetCore.App" />
  </ItemGroup>
</Project>

Replace Program.cs with the C# code below and run the project. The framework reference makes the logging abstractions available; it does not turn this console example into a web server.

All email-like strings, credentials, and request values are synthetic. The example does not call an AI provider or send telemetry to an external logging service.
Create a narrow logging boundary
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Text.Json;
using System.Text.Json.Nodes;
using Microsoft.Extensions.Logging;

var sink = new CaptureLogger();
const string email = "[email protected]";
const string secret = "SECRET_47_SYNTHETIC";
var payload = new JsonObject
{
    ["prompt"] = $"Contact {email}",
    ["headers"] = new JsonObject { ["Authorization"] = secret },
    ["messages"] = new JsonArray(new JsonObject { ["content"] = secret })
};
var error = new HttpRequestException($"Failed with {secret}",
    new Exception($"Nested message: {email}"));
error.Data["raw_response"] = secret;
var attempt = new AiAttempt(Guid.NewGuid(), Route.Primary, 503, 120,
                            payload, error);
Audit.Write(sink, attempt);
payload["future_unknown_field"] = new JsonArray(secret, email);
Audit.Write(sink, attempt with { Status = 200, Error = null });
Audit.Write(sink, attempt with { Error = new OperationCanceledException(secret) });

foreach (var captured in sink.Events)
{
    string all = captured.Message + JsonSerializer.Serialize(captured.State)
                 + captured.Exception;
    if (all.Contains(secret) || all.Contains(email))
        throw new Exception("Sensitive sentinel reached a captured log surface");
    if (captured.Exception is not null)
        throw new Exception("Raw exception reached logger");
    string[] keys = captured.State.Keys.OrderBy(x => x).ToArray();
    string[] expected = ["{OriginalFormat}", "DurationMs", "ErrorKind",
        "OperationId", "Payload", "Route", "Status"];
    if (!keys.SequenceEqual(expected.OrderBy(x => x)))
        throw new Exception("Unexpected structured field");
    Console.WriteLine(captured.Message);
}
Console.WriteLine("PASS: 3 events; message, structured state, and exception checked");

enum Route { Primary, Backup }
record AiAttempt(Guid OperationId, Route Route, int Status, long DurationMs,
                 JsonNode Payload, Exception? Error);

static class Audit
{
    public static void Write(ILogger logger, AiAttempt a)
    {
        string route = a.Route switch
        {
            Route.Primary => "primary", Route.Backup => "backup", _ => "unknown"
        };
        string errorKind = a.Error switch
        {
            null => "none", OperationCanceledException => "canceled",
            HttpRequestException => "transport", _ => "internal"
        };
        int status = a.Status is >= 100 and <= 599 ? a.Status : 0;
        long duration = Math.Max(0, a.DurationMs);
        // Deliberately omit Payload and all exception content.
        logger.LogInformation(
            "AI attempt {OperationId} route={Route} status={Status} " +
            "duration_ms={DurationMs} error={ErrorKind} payload={Payload}",
            a.OperationId, route, status, duration, errorKind, "[REDACTED]");
    }
}

record Captured(string Message, Dictionary<string, object?> State,
                string? Exception);

sealed class CaptureLogger : ILogger
{
    public List<Captured> Events { get; } = [];
    public IDisposable? BeginScope<TState>(TState state) where TState : notnull
        => null;
    public bool IsEnabled(LogLevel level) => true;
    public void Log<TState>(LogLevel level, EventId eventId, TState state,
        Exception? exception, Func<TState, Exception?, string> formatter)
    {
        var fields = state is IEnumerable<KeyValuePair<string, object?>> values
            ? values.ToDictionary(x => x.Key, x => x.Value)
            : new Dictionary<string, object?>();
        Events.Add(new Captured(formatter(state, exception), fields,
                                exception?.ToString()));
    }
}


AiAttempt represents the broader application event. Audit.Write selects only the permitted operational fields. It receives the raw payload but never passes it to ILogger; the Payload logging field always contains the same marker. The exception category comes from a controlled type switch. Neither Message, InnerException, nor Data is serialized. The original exception is also omitted from the logging call, so a sink cannot receive it through that argument.

The custom logger captures both the formatted message and the structured state. This makes the test inspect the event's data, instead of checking only what happens to appear on a console.

Why omit a nested payload instead of walking every key?

A key-based redaction rule must recognize every sensitive location. A newly added messages, attachments, or diagnostic field can bypass an incomplete list. A projection that never includes the raw object has a smaller contract to inspect.

In the test, a new nested field is added after the first event. The later events still contain only the approved operational fields and the constant marker. The test does not depend on recognizing the new field's name. A technical walkthrough for Ranknod could use this distinction to show why redaction should be evaluated at the logging boundary. A clean-looking sample line alone does not establish that structured data and exception arguments are equally clean.

If selective content logging is necessary, design that as a separate, reviewed path with explicit purpose, access, and retention. Do not gradually expand an operational event into a transcript archive because each extra field is convenient.

What did the checks establish?

The executed sample captured three events: a transport failure, an event without an exception after the payload mutation, and a cancellation-category event. It checked each event for both sensitive sentinel strings, required the expected structured keys, and required the exception field to be null.

The final output was:
PASS: 3 events; message, structured state, and exception checked
The event messages retained status, duration, route, error category, and the redaction marker. The randomly generated operation GUID varies between runs.

These checks are intentionally scoped. They establish the behavior of Audit.Write with the supplied fixtures. They do not test background HTTP diagnostics, logging scopes, tracing spans, crash dumps, SDK diagnostics, or a production sink's enrichment rules.

How does this relate to .NET's redaction APIs?

Microsoft's .NET redaction documentation describes data classification, redactor configuration, and integration with logging. Those facilities can support applications that need classified fields and consistent redaction policies across many event types.

The example here uses explicit projection and a constant payload marker. It does not configure the compliance-redaction packages or claim that installing a package automatically classifies every field.

Choose the approach that fits the data contract, then inspect the actual emitted event. With either approach, an unclassified free-form field or a separate raw-exception call can create a path around the intended protection.

What should you check in the real application?

Exercise success, provider failure, timeout, cancellation, and retry paths with synthetic sentinels. Inspect every enabled sink, including structured exports. Add a new nested payload field and confirm that it does not enter an existing event unnoticed. Review scopes and automatic enrichers separately. This capture logger does not implement scope storage, so its success says nothing about a real application's scope contents. Also inspect SDK logging that occurs before your application wrapper receives the error.

Keep the fixed template and permitted-field list under review when the event changes. A useful operational log should explain what happened without copying the conversation that happened to trigger it.

HostForLIFE.eu ASP.NET Core 11.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.

 



European ASP.NET Core 10.0 Hosting - HostForLIFE :: System Design's Availability: What Takes Place When Something Fails?

clock September 28, 2026 11:42 by author Peter

Phase 01 — System Design Fundamentals | Topic 11
Imagine your application is working perfectly.

  • Customers can log in.
  • Orders can be placed.
  • Payments are processed.

Then suddenly, one application server stops working.

What happens next?
If the entire application becomes unavailable, we have a design problem.
This is where Availability becomes an important part of System Design.

What Is Availability?
Availability describes the ability of a system to remain accessible and operational when users need it.

In simple words:
Availability means the system should continue serving users when possible, even when some components fail.

A system does not need to be available 100% of the time in every situation.
Instead, the business usually defines an expected availability level based on the importance of the application.

For example, a payment system may require much higher availability than an internal reporting application.

Why Does Availability Matter?

Users expect applications to be available when they need them.

Imagine a customer trying to place an order and receiving:
    “Service unavailable.”

Even if the problem lasts only a few minutes, it can affect customer experience and business operations.

For critical systems, downtime can also cause:

  • Lost transactions
  • Lost revenue
  • Operational delays
  • Customer dissatisfaction

This is why availability should be considered during system design rather than only after deployment.

What Happens When a Component Goes Down?
Consider a very simple architecture:
Users
   ↓
Web Server
   ↓
Application Server
   ↓
Database

Now imagine the web server fails.

Users cannot reach the application even if the application server and database are still working.

This web server has become a Single Point of Failure.

What Is a Single Point of Failure?

A Single Point of Failure, or SPOF, is a component whose failure can cause the entire system or a critical part of it to become unavailable.

For example:
Users
   ↓
One Server
   ↓
Application
   ↓
Database


If that one server fails, the system may become unavailable.

The problem is not that the server can fail.

Servers can fail.

The problem is that the architecture has no alternative path.

This leads to an important System Design principle:

Do not depend on a single component when its failure can stop the entire service.

How Do We Improve Availability?

One common approach is redundancy.

Instead of running one application server, we can run multiple instances.

For example:

             â”Œâ”€â”€ Application Server 1
Users → Load Balancer
             â””── Application Server 2

Now, if one server fails, the other server may continue handling requests.

This is the basic idea behind designing systems that can tolerate individual component failures.
What Is Redundancy?

Redundancy means having additional instances or components available so that the system does not depend on only one resource.

For example, instead of:

    One application server

we might have:

    Two or more application servers.

Instead of relying on one critical component, we create alternatives.

Redundancy can be used at different levels of a system, including application servers, databases, network components, and other infrastructure.

The exact level of redundancy depends on the business requirements and the expected failure scenarios.
What Does a Load Balancer Do?

When multiple application servers exist, we need some way to distribute incoming requests.

A load balancer can sit in front of those servers and distribute traffic between them.

A simplified design looks like:

Users
   ↓
Load Balancer
   ↓
---------------------
↓                   ↓
Server 1          Server 2

If both servers are healthy, requests can be distributed between them.

If one server becomes unhealthy, the load balancer can stop sending new requests to that server, depending on the configured health-check and routing behavior.

This improves availability because the application does not depend on a single server.
What Is Failover?

Failover means moving processing to another available component when the primary component fails.

For example:

Primary Server
      ↓
    Fails
      ↓
Backup Server
      ↓
Continues Service

Failover can be automatic or manual depending on the system.

For important production systems, automatic failover is often preferred because it can reduce the time users are affected by a failure.
Health Checks Are Important

How does the system know that a component is unhealthy?

One common approach is a health check.

The application can expose an endpoint such as:

GET /health

The platform or load balancer can periodically check that endpoint.

If the service is not healthy, traffic can be redirected away from that instance.

Health checks are therefore an important part of availability design.
Availability Is Not Only About Servers

Developers sometimes think:

    “Just add multiple servers and the system is highly available.”

Not necessarily.

Imagine you have two application servers, but both depend on one database.

        Load Balancer
        /           \
   Server 1       Server 2
        \           /
          Database

If the database becomes unavailable, both application servers may still be unable to process requests.

The database has now become a potential Single Point of Failure.

This is why availability needs to be considered across the whole system, not just one layer.
A Practical Example

Suppose you are designing an e-commerce application.

A basic design may look like:

Users
   ↓
Single Application Server
   ↓
Single Database

This is simple, but there are clear failure points.

A more resilient design could use:

                 â”Œâ”€â”€ Application Server 1
Users → Load Balancer
                 â””── Application Server 2
                         ↓
                   Highly Available
                      Database

Now the system has more than one application instance, and the database can also be designed with appropriate redundancy depending on the chosen database platform.

The architecture is more complex, but that complexity exists for a reason: reducing the impact of failures.
Availability vs “Nothing Will Fail”

A common misunderstanding is:

    “A highly available system never fails.”

That is not realistic.

Components can fail.

Networks can fail.

Databases can become unavailable.

External services can stop responding.

The goal of availability is not to prevent every failure.

The goal is to reduce the impact of failures and keep the service available whenever possible.

This is a much more practical way to think about production systems.
How Should You Think About Availability?

When designing a system, ask:

    What happens if this component stops working?

Then continue:

    Is there another component that can take over?

And:

    How will the system detect the failure?

And finally:

    How quickly can the system recover?

These questions naturally lead to design decisions involving:

    Redundancy

    Load balancing

    Health checks

    Failover

    Monitoring

    Alerts

    Backup strategies

The exact solution depends on the system's requirements and constraints.
IMPORTANT Takeaway

A production system should be designed with the possibility of failure in mind.

Do not assume:

    “This server will always be available.”

Instead, ask:

    “What happens when this server goes down?”

That one question can reveal Single Points of Failure and lead to better architecture.

A simple way to remember it is:

  • Availability is not about preventing every failure.
  • It is about keeping the system usable when failures happen.

Final Thoughts
When you design an application, it is easy to focus on features and happy-path scenarios.
System Design asks us to think about something else:

What happens when something goes wrong?
If one server fails, can another handle the traffic?
If a database becomes unavailable, what happens to the application?
If an instance becomes unhealthy, can the system detect it?

The answers to these questions shape the availability of the system.

That is why availability should be considered from the beginning of the architecture, not after the first production failure.

HostForLIFE.eu ASP.NET Core 10.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



European ASP.NET Core 10.0 Hosting - HostForLIFE :: Ten Bottlenecks in ASP.NET Core API Performance and How to Identify Them

clock September 25, 2026 12:17 by author Peter

Even with typical CPU and memory use, an ASP.NET Core API may nevertheless feel sluggish. Troubleshooting that is one of the more annoying issues with API performance. You examine the server. CPU appears to be in good condition. Memory appears to be in good condition. The database is accessible. Infrastructure breakdowns are not readily apparent.

However, an endpoint that ought to react within 100–200 ms is taking 1-2 seconds.

The explanation is that CPU and memory are not the only factors that affect API latency.

A request can spend most of its time waiting for a database query, another API, a thread, a lock, serialization, or some application code that is doing more work than expected.

The first question I ask when troubleshooting a slow ASP.NET Core API is:

Where is the request actually spending its time?
This article walks through 10 common ASP.NET Core performance bottlenecks and, more importantly, how to identify them before changing the code.

1. Start With Measurement, Not Optimization
Before changing a LINQ query, adding caching, or increasing server resources, establish where the latency comes from.

A useful way to think about an API request is:
Total Request Time
        |
        +-- Application code
        +-- Database
        +-- External APIs
        +-- Serialization
        +-- Network
        +-- Middleware
        +-- Thread scheduling / blocking


For example, suppose an endpoint takes 1.4 seconds:
Application code       120 ms
SQL queries            180 ms
External API           900 ms
JSON serialization      80 ms
Other                   120 ms
--------------------------------
Total                 1400 ms


Optimizing the 180 ms database operation will not solve the main problem.

The external API is responsible for most of the latency.
This is why performance troubleshooting should generally follow:

Measure → Trace → Identify → Fix → Measure again

Useful metrics include:

  • Average response time
  • P95 latency
  • P99 latency
  • Requests per second
  • Database query duration
  • Number of database calls
  • External dependency latency
  • Exception rate
  • Thread-pool behavior
  • Response payload size

Average latency is useful, but P95 and P99 can reveal problems that average measurements hide.

2. Slow SQL Queries

The database is one of the first places to investigate when an ASP.NET Core API becomes slow.

A query that performs well with 10,000 rows in development may behave very differently when the production database contains millions of records.

Common causes include:

  • Missing indexes
  • Inefficient joins
  • Full table scans
  • Large result sets
  • Selecting unnecessary columns
  • Poorly written LINQ queries
  • Repeated database calls
  • Expensive execution plans

For example:
var customers = await db.Customers
    .Where(x => x.Status == "Active")
    .ToListAsync();

The LINQ itself doesn't tell you whether the generated SQL is efficient.

You need to inspect the SQL and execution plan.

With EF Core, you can inspect generated SQL during development:
var query = db.Customers
    .Where(x => x.Status == "Active");

Console.WriteLine(query.ToQueryString());


Then check the actual database behavior.

Look at:

  • Execution plan
  • Index usage
  • Query duration
  • Number of rows scanned
  • Number of rows returned
  • Database round trips

A useful rule is:
Don't optimize the C# code around a slow query until you know how much time the database is actually consuming.

3. EF Core N+1 Queries

The N+1 query problem is particularly easy to introduce when working with Entity Framework Core.

Imagine an endpoint retrieves 100 customers:
1 query → Get 100 customers
100 queries → Get details for each customer
Total = 101 database queries


The API may work correctly, but the number of database round trips can quickly become a performance problem.

A simplified example looks like this:
var customers = await db.Customers
    .ToListAsync();

foreach (var customer in customers)
{
    var orders = await db.Orders
        .Where(x => x.CustomerId == customer.Id)
        .ToListAsync();
}


With 100 customers, this can result in 101 database calls.

One approach is to project only the data required by the API:
var customers = await db.Customers
    .Select(c => new CustomerDto
    {
        Id = c.Id,
        Name = c.Name,
        Email = c.Email
    })
    .ToListAsync();

Depending on the relationship and use case, eager loading, projections, joins, or specifically designed queries may be appropriate.
The important part is not simply changing the LINQ.
Measure the generated SQL and verify how many database calls the endpoint actually makes.

4. Blocking Calls and Thread Pool Starvation

ASP.NET Core is designed around asynchronous I/O, but blocking calls can still appear in application code.

For example:
var result = service.GetDataAsync().Result;

or:

service.GetDataAsync().Wait();

These calls can block a thread while waiting for I/O.
Under light traffic, the problem may not be obvious.
Under higher concurrency, blocked threads can accumulate and contribute to thread-pool starvation.

Possible symptoms include:

  • Increasing response times
  • Requests waiting in queues
  • Reduced throughput
  • Timeouts
  • Increasing thread counts
  • CPU usage that does not necessarily look high

Prefer asynchronous code throughout the request path:

var result = await service.GetDataAsync();

Also look for:

.Result
.Wait()
Blocking locks
Synchronous I/O
Blocking third-party libraries

For runtime investigation, tools such as dotnet-counters and dotnet-trace can help identify thread-pool and runtime behavior.

The important point is that low CPU utilization does not automatically mean the application is healthy.

An application can be waiting rather than computing.

5. Slow External API Calls

Your ASP.NET Core application may be fast while one of its dependencies is slow.

Consider an endpoint that calls three services:
Customer API      150 ms
Payment API       300 ms
Shipping API      800 ms
--------------------------------
Total           1,250 ms

Your own application code may execute in only a small fraction of that time.

The user still waits 1.25 seconds.

This is why distributed tracing becomes important in applications with multiple dependencies.

For outbound HTTP calls, consider:

IHttpClientFactory

  • Async HTTP calls
  • Connection reuse
  • Appropriate timeouts
  • Limited retries
  • Circuit breakers
  • Caching where appropriate
  • Distributed tracing

For example:
public async Task<Customer> GetCustomerAsync(
    int customerId,
    CancellationToken cancellationToken)
{
    return await httpClient.GetFromJsonAsync<Customer>(
        $"customers/{customerId}",
        cancellationToken);
}


Also be careful with retries.

A retry strategy that blindly retries every failure can increase traffic against an already unhealthy dependency.

6. Large API Responses and JSON Serialization

Sometimes the API isn't slow because it is doing too much work.

It is slow because it is returning too much data.

An endpoint that originally returned a 20 KB response may eventually return hundreds of KB—or several MB—as new fields and relationships are added.

Large payloads affect:

  • JSON serialization
  • Network transfer
  • Memory usage
  • Client-side processing
  • Mobile application performance

Instead of returning an entire entity:
return Ok(customer);

consider using a DTO designed specifically for the endpoint:
var result = await db.Customers
    .Select(c => new CustomerResponse
    {
        Id = c.Id,
        Name = c.Name,
        Email = c.Email
    })
    .ToListAsync();

Other useful techniques include:

  • Pagination
  • Filtering
  • Sorting
  • Field selection
  • Compression where appropriate
  • Avoiding unnecessary nested objects

A smaller response is often easier to process at every layer.

7. Poor Caching Strategy

Caching can reduce database calls and repeated computation, but adding a cache does not automatically make an API faster.
First identify whether the data is a good candidate for caching.

For example:
Frequently requested
+
Expensive to calculate
+
Doesn't change frequently
=
Good caching candidate


Depending on the application, you may use:

  • IMemoryCache
  • Distributed caching
  • Response caching
  • Redis
  • Application-level caching

But caching introduces its own questions:

  • How long should the data remain cached?
  • When should it be invalidated?
  • Is stale data acceptable?
  • Is the data sensitive?
  • Does every application instance need the same cached value?

A poorly designed cache can create consistency problems or add unnecessary infrastructure.
Cache deliberately, not automatically.

8. Database and HTTP Connection Management

Connection management problems can remain hidden during development and become visible under production traffic.
One common HTTP mistake is repeatedly creating HttpClient instances instead of using IHttpClientFactory.

For example, avoid creating clients unnecessarily for every request:
using var client = new HttpClient();


For ASP.NET Core applications, IHttpClientFactory provides a better approach to managing HTTP clients and handlers.

Example:
builder.Services.AddHttpClient<PaymentClient>(client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
    client.Timeout = TimeSpan.FromSeconds(10);
});


Connection management can affect:

  • Connection reuse
  • Socket usage
  • Connection establishment time
  • API latency
  • Reliability under load

The same principle applies to database connections.

Use the connection pooling and lifecycle management provided by your database provider and ORM rather than manually creating unnecessary connections.

9. Excessive Middleware and Logging
Logging is essential for production troubleshooting.
But logging everything on every request can become expensive.
For example, logging large request and response objects can involve:

Build log message
      ↓
Serialize object
      ↓
Write log
      ↓
Send to logging platform
      ↓
Store and process

When an API handles thousands of requests per minute, that additional work can become noticeable.

Review:

  • Log levels
  • Request/response logging
  • Large object logging
  • Duplicate logs
  • Middleware that runs on every request

The goal isn't to remove logging.

The goal is to make logging useful without adding unnecessary work to the request path.

The same applies to middleware.

If every request passes through many expensive middleware components, the overhead can accumulate.

Measure the middleware pipeline before removing components blindly.

10. Inefficient Application Logic

Not every slow ASP.NET Core API has a database problem.
Sometimes the bottleneck is simply application code.

Common examples include:

  • Expensive loops
  • Repeated LINQ operations
  • Processing large collections in memory
  • Repeated calculations
  • Excessive object allocation
  • Unnecessary transformations
  • Performing the same operation multiple times

Consider this example:
Database query       50 ms
Application logic   600 ms
Serialization        50 ms
---------------------------
Total               700 ms


Optimizing the SQL query from 50 ms to 30 ms will barely change the overall response time.

The application logic is where the majority of the time is being spent.

This is where profiling becomes useful.

Instead of guessing which method is slow, use profiling and tracing to identify the actual hot path.

How to Troubleshoot a Slow ASP.NET Core API?

When an endpoint becomes slow, I recommend following a structured process rather than changing multiple things at once.

Step 1: Establish the baseline

Record:

  • Average latency
  • P95 latency
  • P99 latency
  • Requests/sec
  • Error rate

Step 2: Break Down the Request
Measure:

  • API processing
  • Database
  • External APIs
  • Serialization
  • Network

Step 3: Check Database Activity
Look for:

  • Slow queries
  • Missing indexes
  • Large result sets
  • N+1 queries
  • Repeated database calls

Step 4: Check Asynchronous Code
Search the codebase for:
.Result
.Wait()
Blocking locks
Synchronous I/O


Step 5: Trace External Dependencies
Find out whether the API is waiting on:

Payment services
Authentication providers
CRMs
Third-party APIs
Internal microservices


Step 6: Inspect Response Size

Check whether the endpoint is returning more data than the client actually needs.

Step 7: Profile Application Code
If the database and dependencies look healthy, profile the application itself.

Step 8: Fix One Bottleneck

Don't make five performance changes at once.

Change one thing, measure the result, and continue from there.

A Simple Performance Investigation Example

Suppose an endpoint has this profile:
Request latency: 1,850 ms

SQL queries:       620 ms
External API:      900 ms
Application code:  180 ms
Serialization:      80 ms
Other:               70 ms


It would be tempting to start optimizing the C# application code.

But the measurements tell us something different.

The two largest contributors are:
External API     900 ms
Database         620 ms

Those are the first areas worth investigating.

Now imagine the database profile reveals:
1 query
+
100 related queries
=
101 database calls


Fixing the N+1 problem could have a much larger impact than micro-optimizing a LINQ expression elsewhere in the application.

This is why profiling beats guessing.

Final Takeaway

A slow ASP.NET Core API does not necessarily mean you need a larger server or more CPU.

The bottleneck may be:

  • SQL queries
  • EF Core N+1 queries
  • Blocking code
  • Thread pool starvation
  • External APIs
  • Large JSON responses
  • Poor caching
  • Connection management
  • Middleware and logging
  • Application logic

The most effective starting point is simple:
Measure where the request spends its time before changing the implementation.
Once you know whether the latency is coming from the database, application code, external dependencies, serialization, or infrastructure, the optimization path becomes much clearer.
For production ASP.NET Core applications, especially APIs supporting SaaS and enterprise workloads, monitoring P95/P99 latency and tracing dependencies can help catch performance regressions before they become user-facing problems.

Summary

CPU and memory use are not necessarily the only indicators of ASP.NET Core API performance issues. Latency can be caused by a variety of factors, including slow database queries, N+1 queries, blocking activities, external dependencies, big answers, cache choices, connection management, middleware, logging, and application logic. A useful method of troubleshooting performance without depending on conjecture is to measure the request, trace its dependencies, identify the real bottleneck, and measure again after each modification.

HostForLIFE.eu ASP.NET Core 10.0 Hosting
European best, cheap and reliable ASP.NET hosting with instant activation. HostForLIFE.eu is #1 Recommended Windows and ASP.NET hosting in European Continent. With 99.99% Uptime Guaranteed of Relibility, Stability and Performace. HostForLIFE.eu security team is constantly monitoring the entire network for unusual behaviour. We deliver hosting solution including Shared hosting, Cloud hosting, Reseller hosting, Dedicated Servers, and IT as Service for companies of all size.



About HostForLIFE.eu

HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

We have offered the latest Windows 2016 Hosting, ASP.NET Core 2.2.1 Hosting, ASP.NET MVC 6 Hosting and SQL 2017 Hosting.


Month List

Tag cloud

Sign in