A circuit breaker is a software development design pattern intended to improve the resilience and fault tolerance of programs that connect with external services or resources. It takes its name from its electrical counterpart, which is intended to safeguard an electrical circuit from harm caused by high current. A circuit breaker in software performs a similar function by preventing the application from continually attempting to call a malfunctioning service or resource, which might lead to performance deterioration and other issues.
Circuit Breaker Diagram
Here's a breakdown of the circuit breaker pattern.
1. Normal Operation (Closed State)
When the circuit breaker is closed, requests can travel through normally.
It tracks the success and failure of these queries during this condition.
When a particular number of failures are reached, the circuit breaker switches to the open position.
2. Unavailable State (Service)
When the circuit breaker is open, it prevents requests from reaching the failing service or resource.
This condition is activated when a preset number of failures is exceeded.
No requests are sent to the service/resource while it is open, which might minimize the stress on the resource.
The circuit breaker verifies if the service/resource has recovered on a regular basis by allowing a restricted number of test requests to get through.
3. Half-Open State (Service Availability Testing)
After a predetermined period of time, the circuit breaker switches to the half-open position.
In this condition, only a limited number of test requests can be sent to the service/resource to see if it has recovered.
If these test requests are successful, the circuit breaker returns to the closed position.
If the test requests continue to fail, the circuit breaker returns to the open position, protecting the system from additional failures.
Important Concepts
- Circuit breakers have thresholds that determine when they switch between states. The number of successive failures, the failure rate, or a combination of factors could be used to set these criteria.
- Timers: Timers regulate how long the circuit breaker stays in each state. For example, it may remain in the open state for a predetermined period of time before shifting to the half-open state.
- Monitoring and request tracking are required to determine the health of the service or resource and to determine when to open or close the circuit.
- Fallback Mechanism: While in the open state, apps can design fallback mechanisms to provide alternative functionality or smooth service degradation.
- Automated Recovery: When the service/resource appears to have recovered, circuit breakers automatically return to the closed state, decreasing the impact on the system.
By isolating problematic services or resources, the circuit breaker design helps to prevent cascading failures in distributed systems, enhances system stability, and provides a level of fault tolerance. It is a basic idea in the development of strong and resilient software systems.
Circuit Breaker Pattern in .Net Core
Let's implement the same in .Net Core using two different approaches.
- Approach 1 - Without using any specific Nuget package
- Approach 2 - Using Polly Nuget package
Common steps for both approaches( will also upload the solution with this article). I am using Visual Studio for mac but you can use Visual Studio code as well.
Create a new .NET Core API Project
Add a new interface for service and the concrete service class with HttpClient as a dependency
In the concrete service, add code to make an HttpRequest to a random api and get request. I am using an open source webapi for getting random jokes
Add a controller and call the above created service in it
Joke Interface
namespace CircuitBreaker
{
public interface IJokeService
{
public Task<string> GetRandomJokes();
}
}
Joke Service
namespace CircuitBreaker
{
public class JokeService : IJokeService
{
private readonly HttpClient _httpClient;
public JokeService(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async Task<string> GetRandomJokes()
{
try
{
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri("https://official-joke-api.appspot.com/random_joke")
};
var response = await _httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
var responseBody = await response.Content.ReadAsStringAsync();
return responseBody;
}
catch (Exception ex)
{
throw ex;
}
}
}
}
Joke Controller
using Microsoft.AspNetCore.Mvc;
namespace CircuitBreaker.Controllers
{
[ApiController]
[Route("[controller]")]
public class JokeController : ControllerBase
{
public readonly IJokeService _jokeService;
public JokeController(IJokeService jokeService)
{
_jokeService = jokeService;
}
[HttpGet(Name ="GetRandomeJokes")]
public async Task<IActionResult> Get()
{
try
{
var response = await _jokeService.GetRandomJokes();
return Ok(response);
}
catch (Exception ex)
{
throw ex;
}
}
}
}
Program.cs/StartUp.cs
public static void ConfigureService(IServiceCollection services)
{
services.AddHttpClient<IJokeService, JokeService>();
}
Sample output
{"type":"general","setup":"What's brown and sticky?","punchline":"A stick.","id":271}
Now, the circuit breaker strategy usually allows an n number of failed tries before not-allowing any futher requests and breaking the circuite for some time. These n tries are with state, let me explain in more clarity.
Let's suppose the n count is 3 and 3 three different users tries to access the api one after the other(irrespective of the time difference between these requests) and if all three of these fails, then the 4th atempt should not even go through. The code should not further try to hit the other api untill a specifc time internal. Uptill this time interval, all requests attempts will just by-pass the api call and provide a message to end-user to try after some time. This can go in a loop if the other api issue is not resolved. This helps in making sure that the other service that we are calling is not overwhelmed with failing requests.
Let's try to implement this in Approach 1.
Create some static variables. We are creating them static as we want to persist the value of them between different calls. Please read comments for more understanding of each of it's use
//count of failed attempts that have been made
private static int attemptCount = 0;
//count of failed attempts allowed, post which the circiut will break
private static int maxAttemptCount = 3;
//flag to represent if the circuit is open or close
private static bool isCircuitOpen = false;
//field to keep a track of the utc time when the circuit was opened/broken
private static DateTime circuitOpenStartTime = DateTime.MinValue;
//the timestamp (in millisecond) for which the circuit should remain open and api call attempts should not be made
private static int circuitBreakerTimeSpanMilliseconds = 120000;
//method to start the circuit breaker
//if sets the isCircuitOpen to true and sets the time when the circuit was broken in utc
private void RecordCircuitBreakerStart()
{
circuitOpenStartTime = DateTime.UtcNow;
isCircuitOpen = true;
}
//method to end the circuit breaker
private void RecordCircuitBreakerEnd()
{
circuitOpenStartTime = DateTime.MinValue;
isCircuitOpen = false;
attemptCount = 0;
}
//check if currently the circuit is broken or not
private void CheckIfCircuitBreakerTimeStampIsComplete()
{
if(isCircuitOpen == true && circuitOpenStartTime.AddMilliseconds(circuitBreakerTimeSpanMilliseconds) < DateTime.UtcNow)
{
RecordCircuitBreakerEnd();
}
}
Now, let's call the above method in our controller method
public async Task<string> GetRandomJokes()
{
try
{
//check if the circuit was earlier open can can be closed now
CheckIfCircuitBreakerTimeStampIsComplete();
if (isCircuitOpen == false)
{
#region MakeAPICall
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri("https://official-joke-api.appspot.com/random_joke")
};
var response = await _httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
var responseBody = await response.Content.ReadAsStringAsync();
return responseBody;
#endregion
}
return "Service is not available. Please try after some time";
}
catch (Exception ex)
{
//in case of exception, if the max attempt of failure is not yet reached, then increase the counter
if(isCircuitOpen == false && attemptCount < maxAttemptCount)
{
attemptCount++;
}
//if the count of max attempt if reached, then open the circuits and retuen message that the service is not available
if(attemptCount == maxAttemptCount)
{
if (isCircuitOpen == false)
{
RecordCircuitBreakerStart();
}
return "Service is not reachable. Please try after some time";
}
return ex.Message;
}
}
I agree with what you all must be thinking, the above approach is indeed messy and prone to more errors as we are managing multiple things at the same time. In order to make this code more readable and less error prone, we move to Approach 2
Approach 2 - In this we use an existing Nuget package, called Polly to achieve the same.
In the above code, add the nuget package Polly as in the screenshot below:
Add another nuget package - Microsoft.Extensions.Http.Polly to the solution
These packages enable us to define different policies to come into effect in case of an error. One of such policy is CircuitBreaker. Other policies to look at are RetryPolicy and WaitAndRetryPolicy.
For this, we will be adding another interface method and another service method called GetRandomJokesV2().
namespace CircuitBreaker
{
public interface IJokeService
{
public Task<string> GetRandomJokes();
public Task<string> GetRandomJokesV2();
}
}
public async Task<string> GetRandomJokesV2()
{
try
{
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri("https://official-joke-api.appspot.com/random_joke")
};
var response = await _httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
var responseBody = await response.Content.ReadAsStringAsync();
return responseBody;
}
catch (BrokenCircuitException ex)
{
return $"Request failed due to opened circuit: {ex.Message}";
}
catch (HttpRequestException httpEx)
{
return $"Request failed. StatusCode={httpEx.StatusCode} Message={httpEx.Message}";
}
}
We then updated the ConfigureService method in Program.cs to add this policy. See code below:
public static void ConfigureService(IServiceCollection services)
{
services.AddHttpClient<IJokeService, JokeService>(client =>
{
client.BaseAddress = new Uri("https://official-joke-api.appspot.com/random_joke");
}).AddTransientHttpErrorPolicy(policy => policy.CircuitBreakerAsync(3, TimeSpan.FromMilliseconds(120000)));
}
In the above code, we are telling the middleware that in case this api call fails after 3 attempts, then by-pass all future api calls for 120000 miliseconds(2 mins). Post that there will be attempts to hit the api.
The way we call it in the Controller or servide does not change much
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
// For more information on enabling MVC for empty projects, visit https://go.microsoft.com/fwlink/?LinkID=397860
namespace CircuitBreaker.Controllers
{
[ApiController]
[Route("[controller]")]
public class JokeV2Controller : ControllerBase
{
public readonly IJokeService _jokeService;
public JokeV2Controller(IJokeService jokeService)
{
_jokeService = jokeService;
}
[HttpGet(Name = "GetRandomeJokesV2")]
public async Task<IActionResult> Get()
{
try
{
var response = await _jokeService.GetRandomJokesV2();
return Ok(response);
}
catch (Exception ex)
{
throw ex;
}
}
}
}
We can clearly see that the Approach 2 is a more advisable and enterprise approach which makes our code more readable and less prone to errors. We can also make all the policies inside another folders as classes and inject them wherever required. This will make our code more abstract.
Please let me know your thoughts on this. I am also attaching my solutions in this post.