Leveraging Consul for Service Discovery in Microservices with .NET Core

Introduction:

In a microservices architecture, service discovery is pivotal in enabling seamless communication between services. Imagine having a multitude of microservices running across different ports and instances and the challenge of locating and accessing them dynamically. This is where the Consul comes into play.

Introduction to Consul:
Consul, a distributed service mesh solution, offers robust service discovery, health checking, and key-value storage features. In this tutorial, we’ll explore leveraging Consul for service discovery in a .NET Core environment. We’ll set up Consul, create a .NET Core API for service registration, and develop a console application to discover the API using Consul.

Step 1: Installing Consul:
Before integrating Consul into our .NET Core applications, we need to install Consul. Follow these steps to install Consul:

  1. Navigate to the Consul downloads page: Consul Downloads.
  2. Download the appropriate version of Consul for your operating system.
  3. Extract the downloaded archive to a location of your choice. 
  4. Add the Consul executable to your system’s PATH environment variable to run it from anywhere in the terminal or command prompt. 
  5. Open a terminal or command prompt and verify the Consul installation by running the command consul --version.
  6. Run the Consul server by running the command consul agent -dev

Step 2: Setting Up the Catalog API:

Now, let’s create a .NET Core API project named ServiceDiscoveryTutorials.CatalogApi. This API will act as a service that needs to be discovered by other applications. Use the following command to create the project:

dotnet new webapi -n ServiceDiscoveryTutorials.CatalogApi

Next, configure the API to register with the Consul upon startup. Add the Consul client package to the project:

dotnet add package Consul

In the Startup.cs file, configure Consul service registration in the ConfigureServices method:

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();

    services.AddSingleton<IConsulClient>(p => new ConsulClient(consulConfig =>
    {
        var consulHost = builder.Configuration["Consul:Host"];
        var consulPort = Convert.ToInt32(builder.Configuration["Consul:Port"]);
        consulConfig.Address = new Uri($"http://{consulHost}:{consulPort}");
    }));
    
    services.AddSingleton<IServiceDiscovery, ConsulServiceDiscovery>();

}

Create a class named ConsulServiceDiscovery that implements the IServiceDiscovery interface to handle service registration:

public interface IServiceDiscovery
{
    Task RegisterServiceAsync(string serviceName, string serviceId, string serviceAddress, int servicePort);
    Task RegisterServiceAsync(AgentServiceRegistration serviceRegistration);
    
    Task DeRegisterServiceAsync(string serviceId);
}

public class ConsulServiceDiscovery : IServiceDiscovery
{
    private readonly IConsulClient _consulClient;

    public ConsulServiceDiscovery(IConsulClient consulClient)
    {
        _consulClient = consulClient;
    }

    public async Task RegisterServiceAsync(string serviceName, string serviceId, string serviceAddress, int servicePort)
    {
        var registration = new AgentServiceRegistration
        {
            ID = serviceId,
            Name = serviceName,
            Address = serviceAddress,
            Port = servicePort
        };
        await _consulClient.Agent.ServiceDeregister(serviceId);
        await _consulClient.Agent.ServiceRegister(registration);
    }

    public async Task DeRegisterServiceAsync(string serviceId)
    {
        await _consulClient.Agent.ServiceDeregister(serviceId);
    }

    public async Task RegisterServiceAsync(AgentServiceRegistration registration)
    {
        await _consulClient.Agent.ServiceDeregister(registration.ID);
        await _consulClient.Agent.ServiceRegister(registration);
    }
}

In the Configure method of Startup.cs, add the service registration logic:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env, IConsulClient consulClient)
{
    // Configure the HTTP request pipeline.
    if (app.Environment.IsDevelopment())
    {
        app.UseSwagger();
        app.UseSwaggerUI();
    }
    
    //app.UseHttpsRedirection();
    
    app.UseAuthorization();
    
    
    app.MapControllers();
    
    var discovery = app.Services.GetRequiredService<IServiceDiscovery>();
    var lifetime = app.Services.GetRequiredService<IHostApplicationLifetime>();
    var serviceName = "CatalogApi";
    var serviceId = Guid.NewGuid().ToString();
    var serviceAddress = "localhost";
    var servicePort = 7269;
    
    lifetime.ApplicationStarted.Register(async () =>
    {
        var registration = new AgentServiceRegistration
        {
            ID = serviceId,
            Name = serviceName,
            Address = serviceAddress,
            Port = servicePort,
            Check = new AgentServiceCheck
            {
                HTTP = $"https://{serviceAddress}:{servicePort}/Health",
                Interval = TimeSpan.FromSeconds(10),
                Timeout = TimeSpan.FromSeconds(5)
            }
        };
        await discovery.RegisterServiceAsync(registration);
    });
    
    lifetime.ApplicationStopping.Register(async () =>
    {
        await discovery.DeRegisterServiceAsync(serviceId);
    });

}

With these configurations, the Catalog API will register itself with the Consul upon startup and deregister upon shutdown.

Step 3: Creating the Client Application:

Next, create a console application named ServiceDiscoveryTutorials.ClientApp. Use the following command to create the project:

dotnet new console -n ServiceDiscoveryTutorials.ClientApp

Add the Consul client package to the project:

dotnet add package Consul

In the Program.cs file, configure the Consul client to discover services:

class Program
{
    static async Task Main(string[] args)
    {
        using (var client = new ConsulClient(consulConfig =>
        {
            consulConfig.Address = new Uri("http://localhost:8500");
        }))
        {
            var services = await client.Catalog.Service("CatalogApi");
            foreach (var service in services.Response)
            {
                Console.WriteLine($"Service ID: {service.ServiceID}, Address: {service.ServiceAddress}, Port: {service.ServicePort}");
            }
        }
        //var consulClient = new ConsulClient();
        //// Specify the service name to discover
        //string serviceName = "CatalogApi";
        //// Query Consul for healthy instances of the service
        //var services = (await consulClient.Health.Service(serviceName, tag: null, passingOnly: true)).Response;
        //// Iterate through the discovered services
        //foreach (var service in services)
        //{
        //    var serviceAddress = service.Service.Address;
        //    var servicePort = service.Service.Port;
        //    Console.WriteLine($"Found service at {serviceAddress}:{servicePort}");
        //    // You can now use the serviceAddress and servicePort to communicate with the discovered service.
        //}

    }
}

This code snippet retrieves all instances of the CatalogApi service registered with the Consul.

Step 3: Testing the API and Client Application:

Below is the project structure in the Visual Studio. 

Next, let’s run both applications using the command dotnet run. When this application starts, the Consul portal will display the registered service. 

Below is the final results of the application.

Conclusion:
In this tutorial, we’ve learned how to set up Consul for service discovery and register a .NET Core API with Consul. Additionally, we’ve developed a console application to discover services using Consul’s API. By leveraging Consul, you can enhance the scalability and reliability of your microservices architecture.

Source Code

Building Resilient Microservices: Implementing Resiliency Patterns with Polly Framework

Resiliency is critical to building distributed systems, especially in microservices architectures where failures are inevitable. In this comprehensive guide, we'll explore how to implement resiliency patterns using the Polly framework in .NET Core. We'll cover the retry, circuit breaker, and fallback patterns, each with detailed examples to help you understand their implementation and benefits.

Introduction to Polly Framework

Polly is a robust resilience and transient-fault-handling library for .NET designed to help developers quickly implement resiliency patterns. It provides a fluent interface for defining policies for retry, circuit breaker, and fallback strategies.

Retry Pattern

The retry pattern allows you to automatically retry an operation that has failed due to transient faults, such as network errors or temporary unavailability of resources. Let's dive into a step-by-step implementation of the retry pattern using Polly.

  1. Install Polly NuGet Package: First, install the Polly NuGet package in your .NET Core application.

    Install-Package Polly
     
  2. Create a Retry Policy: Use Polly's fluent syntax to define a retry policy. Specify the number of retry attempts and the duration between retries.

    var retryPolicy = Policy
        .Handle<Exception>()
        .WaitAndRetry(5, retryAttempt => TimeSpan.FromSeconds(5));
     
  3. Execute the Operation with Retry: Use the retry policy to execute the operation you want to retry.

    retryPolicy.Execute(() =>
    {
        // Perform the operation that may fail
        YourOperation();
    });
     
  4. Handle Exceptions: Polly will handle exceptions thrown by the operation and retry it according to the retry policy.

Circuit Breaker Pattern

The circuit breaker pattern is used to prevent repeated execution of an operation that is likely to fail, thereby reducing the load on the system. Let's see how to implement the circuit breaker pattern with Polly.

  1. Create a Circuit Breaker Policy: Define a circuit breaker policy specifying the number of consecutive failures before the circuit is opened and the duration of the open state.

    var circuitBreakerPolicy = Policy
        .Handle<Exception>()
        .CircuitBreaker(3, TimeSpan.FromSeconds(30));
     
  2. Execute the Operation with Circuit Breaker: Use the circuit breaker policy to execute the operation.

    circuitBreakerPolicy.Execute(() =>
    {
        // Perform the operation that may fail
        YourOperation();
    });
     
  3. Handle Circuit Breaker State: Polly will manage the circuit breaker state internally, transitioning between closed, open, and half-open states based on the defined thresholds.

Fallback Pattern

The fallback pattern provides an alternative behaviour or value when an operation fails. It helps gracefully handle failures by providing a fallback mechanism. Let's implement the fallback pattern using Polly.

  1. Define a Fallback Policy: Create a fallback policy specifying the fallback action to be executed when the primary operation fails.

    var fallbackPolicy = Policy
        .Handle<Exception>()
        .Fallback(() =>
        {
            // Perform fallback operation
            FallbackOperation();
        });
     
  2. Execute the Operation with Fallback: Use the fallback policy to execute the primary operation, with fallback behaviour defined.

    fallbackPolicy.Execute(() =>
    {
        // Perform the primary operation
        YourOperation();
    });
     
  3. Handle Fallback: Polly will execute the fallback action when the primary operation fails, ensuring graceful functionality degradation.

Conclusion

Implementing resiliency patterns like retry, circuit breaker, and fallback using the Polly framework can significantly enhance the reliability and robustness of your microservices architecture. By intelligently handling transient faults and failures, you can ensure that your application remains responsive and available under challenging conditions. You can just experiment with these patterns in your microservices projects to build more resilient, fault-tolerant systems.

Source Code

Securing Your Microservices: Azure B2C Authentication in ASP.NET Core API with Ocelot API Gateway

Introduction:
Microservices architecture offers flexibility and scalability but also presents challenges in managing authentication and authorization across multiple services. In this blog post, we will explore how to secure your microservices using Azure B2C authentication in ASP.NET Core API with Ocelot API Gateway. We’ll start by configuring Azure B2C for authentication and then integrate it with our ASP.NET Core API through Ocelot.

Prerequisites:

  1. Azure Subscription: You’ll need an Azure subscription to create and configure Azure B2C resources.
  2. Create one now if you haven’t already created your own Azure AD B2C Tenant. You can use an existing Azure AD B2C tenant.
  3. Visual Studio or Visual Studio Code: We’ll use Visual Studio or Visual Studio Code to create and run the ASP.NET Core API project.
  4. .NET Core SDK: Ensure that the .NET Core SDK is installed on your development machine.
  5. Azure CLI (Optional): Azure CLI provides a command-line interface for interacting with Azure resources. It’s optional but can help manage Azure resources.

Step 1: App registrations

  1. Sign in to the Azure portal (https://portal.azure.com) using your Azure account credentials.
  2. Navigate to the Azure Active Directory service and select App registrations.
  3. Click on “+ New registration” to create a new application registration.
  4. Provide a name for your application, select the appropriate account type, and specify the redirect URI for authentication callbacks.
  5. After creating the application registration, note down the Application (client) ID and Directory (tenant) ID. 

Step 2: Create a client secret

  1. Once the application is registered, note the Application (client) ID and Directory (tenant) ID.
  2. If you are not on the application management screen, go to the Azure AD B2C—App registrations page and select the application you created.
  3. To access the Certificates & secrets settings, navigate to the Manage option and select it. The Certificates & secrets option can be found in the left menu.
  4. Under “Certificates & secrets”, generate a new client secret by clicking on New client secret.
  5. Enter a description of the client’s secret in the Description box. For example, Ocelotsecret.
  6. Under Expires, select a duration for which the secret is valid, and then click Add.
  7. Copy the secret’s Value for use in your client application code and save it securely.

Step 3: Configure scopes

  1. In the Azure AD B2C - App registrations page, select the application you created if you are not on the application management screen.
  2. Select App registrations. Select the OcelotTutorials application to open its Overview page.
  3. Under Manage, select Expose an API.
  4. Next to the Application ID URI, select the Add link.
  5. I have not changed the default GUID with my API, but you can replace the default value (a GUID) with an API and then select Save. The full URI is shown and should be in the format https://your-tenant-name.onmicrosoft.com/api. When your web application requests an access token for the API, it should add this URI as the prefix for each scope you define for the API.
  6. Under Scopes defined by this API, select Add a scope.

  7. Enter the following values to create a scope that defines read access to the API, then select Add scope:

    Scope name: ocelottutorial.read
    Admin consent display name: Read access to API Gateway API
    Admin consent description: Allows read access to the API Gateway API

Step 4: Grant permissions

  1. Select App registrations and then the web application that should have access to the API, such as OcelotTutorials.
  2. Under Manage, select API permissions.
  3. Under Configured permissions, select Add a permission.
  4. Select the My APIs tab.
  5. Select the API to which the web application should be granted access. For example, webapi1.
  6. Under Permission, expand API Name, and then select the scopes that you defined earlier. For example, ocelottutorial.read and ocelottutorial.write.
  7. Select Add permissions.
  8. Select Grant admin consent for (your tenant name).
  9. If you’re prompted to select an account, select your currently signed-in administrator account, or sign in with an account in your Azure AD B2C tenant that’s been assigned at least the Cloud application administrator role.
  10. Select Yes. 
  11. Select Refresh, and then verify that “Granted for …” appears under Status for both scopes.

Step 5: Enable ID token implicit grant

If you register this app and configure it with https://jwt.ms/ app for testing a user flow or custom policy, you need to enable the implicit grant flow in the app registration:

  1. In the left menu, under Manage, select Authentication.
  2. Under Implicit grant and hybrid flows, select both the Access tokens (used for implicit flows) and ID tokens (used for implicit and hybrid flows) checkboxes.
  3. Select Save. 

Step 6: Set Up Azure B2C Authentication in ASP.NET Core API

  1. Create 3 new ASP.NET Core Web API projects in Visual Studio or Visual Studio Code.
    Accounting.API
    Inventory.API
    ApiGateway

  2. Assign the ports to the API. ApiGateay 9000, Accounting.API 9001, Inventory.API 9002
     {
       "Urls": "http://localhost:9001",
       "Logging": {
         "LogLevel": {
           "Default": "Information",
           "Microsoft.AspNetCore": "Warning"
         }
       },
       "AllowedHosts": "*"
     }
    
  3. Install the necessary NuGet packages for Azure B2C authentication. Install the below packages in the ApiGateway project

     dotnet add package Microsoft.Identity.Web
     dotnet add package Ocelot
    
  4. Configure Azure B2C authentication in your Startup.cs file:

     builder.Services.AddMicrosoftIdentityWebApiAuthentication(builder.Configuration);
    
  5. Add the Azure B2C settings to your appsettings.json file:

     {
       "Urls": "http://localhost:9000",
       "Logging": {
         "LogLevel": {
           "Default": "Information",
           "Microsoft.AspNetCore": "Warning"
         }
       },
       "AllowedHosts": "*",
       "AzureAd": {
         "Instance": "https://login.microsoftonline.com/",
         "Domain": "http://localhost:9000/",
         "TenantId": "",
         "ClientId": ""
       }
     }
    
  6. Ensure that the authentication middleware is added to the request processing pipeline in the Configure method of Startup.cs:

     app.UseAuthentication(); // Place UseAuthentication before UseOcelot
     app.UseAuthorization(); // Place UseAuthorization before UseAuthentication
    
  7. Add the ocelot.json file to the the ApiGateway with the below configuration
     {
       "Routes": [
         {
           "DownstreamPathTemplate": "/api/values",
           "DownstreamScheme": "http",
           "DownstreamHostAndPorts": [
             {
               "Host": "localhost",
               "Port": 9001
             }
           ],
           "UpstreamPathTemplate": "/accounting",
           "UpstreamHttpMethod": [ "GET" ],
           "AuthenticationOptions": {
             "AuthenticationProviderKey": "Bearer",
             "AllowedScopes": []
           }
         },
        
         {
           "DownstreamPathTemplate": "/api/values",
           "DownstreamScheme": "http",
           "DownstreamHostAndPorts": [
             {
               "Host": "localhost",
               "Port": 9002
             }
           ],
           "UpstreamPathTemplate": "/inventory",
           "UpstreamHttpMethod": [ "GET" ],
           "AuthenticationOptions": {
             "AuthenticationProviderKey": "Bearer",
             "AllowedScopes": []
           }
         }    
       ],
       "GlobalConfiguration": {
         "BaseUrl": "http://localhost:9000"
       }
     } 
    
  8. Added ocelot configuration to the services
    // Ocelot configuration
    builder.Configuration.AddJsonFile("ocelot.json", optional: false, reloadOnChange: true);
    builder.Services.AddOcelot(builder.Configuration);
    
  9. Added Ocelot to the middleware pipeline in the end.
    app.UseAuthentication(); // Place UseAuthentication before UseOcelot
    app.UseAuthorization(); // Place UseAuthorization before UseAuthentication
    app.MapControllers();
    app.UseOcelot().Wait();
    app.Run();
    

Step 7: Testing authentication

To Test this, refer to this tutorial OAuth 2.0 authorization code flow in Azure Active Directory B2C

  1. Replace the required fields and use the below URL in the browser to get the code to fetch the token. https://login.microsoftonline.com/{tenant}/oauth2/v2.0/authorize?client_id={client id}&response_type=code&response_mode=query&scope={scope uri}&state=007 

  2. Open Postman and use the returned code to generate the token. See the image below to check the URL and the required fields to get the token.
    https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token

  3. Now we are ready to call our API Gateway with the token. 

Conclusion:
In this blog post, we’ve covered the first part of securing your microservices architecture using Azure B2C authentication. We walked through the process of configuring Azure B2C for authentication, including creating a tenant, setting up user flows (policies), and integrating Azure B2C authentication into an ASP.NET Core API project. In the next part of this series, we’ll explore how to integrate Azure B2C authentication with Ocelot API Gateway for centralized authentication and authorization management across microservices.

References:
Tutorial: Register a web application in Azure Active Directory B2C
Add a web API application to your Azure Active Directory B2C tenant

Source Code

Understanding Microservices: A Comprehensive Guide