Introducing the AWS Message Processing Framework for .NET

We are thrilled to announce the developer preview release of the AWS Message Processing Framework for .NET. This framework is designed to simplify the development of .NET applications for message processing using AWS services like Amazon Simple Queue Service (SQS), Amazon Simple Notification Service (SNS), and Amazon EventBridge. It provides a streamlined, AWS-native experience, allowing developers to leverage AWS messaging features without getting bogged down by boilerplate code commonly associated with these services.

What Makes It Different?

Unlike cloud-agnostic frameworks that may lack the tailored functionality needed for AWS services, this framework is optimized for AWS. It exposes advanced features such as message attributes, FIFO (First-In-First-Out) queues, and message visibility management in a way that’s intuitive for .NET developers.

Grasping the Basics

When working with SQS in a long-running application, several responsibilities fall onto the developer:

1. Receiving messages from the SQS queue.
2. Deserializing messages into .NET types.
3. Processing each message.
4. Deleting each message from the queue after successful processing.

To illustrate, let’s consider a simplified approach using the AWS SDK for .NET directly:

csharp
var client = new AmazonSQSClient();
var queueUrl = “https://sqs.us-west-2.amazonaws.com/012345678910/MyQueue“;

while (true) // process messages continually
{
var request = new ReceiveMessageRequest()
{
QueueUrl = queueUrl,
VisibilityTimeout = 30
};

var response = await client.ReceiveMessageAsync(request);

foreach (var message in response.Messages)
{
    GreetingMessage? greetingMessage = null;
    try
    {
        greetingMessage = JsonSerializer.Deserialize<GreetingMessage>(message.Body);
    }
    catch (JsonException ex)
    {
        // handle malformed JSON
    }

    Console.WriteLine($"Received message {greetingMessage?.Greeting} from {greetingMessage?.SenderName}");

    var deleteRequest = new DeleteMessageRequest()
    {
        QueueUrl = queueUrl,
        ReceiptHandle = message.ReceiptHandle
    };

    await client.DeleteMessageAsync(deleteRequest);
}

}

With the AWS Message Processing Framework for .NET, you only focus on implementing the business logic to handle each message. Let’s see how you can define a handler for the GreetingMessage.

csharp
///

public class GreetingMessageHandler : IMessageHandler
{
public Task HandleAsync(MessageEnvelope messageEnvelope, CancellationToken token = default)
{
Console.WriteLine($”Received message {messageEnvelope.Message.Greeting} from {messageEnvelope.Message.SenderName}”);

    return Task.FromResult(MessageProcessStatus.Success());
}

}

Seamless Configuration

During the startup of your application, you can configure the queue URL for polling and map message types to their respective handlers effortlessly. The framework takes care of everything from polling SQS to deserializing messages into the appropriate .NET types and dispatching them to the correct handler. Here’s how you configure it:

csharp
var builder = Host.CreateDefaultBuilder(args);

builder.ConfigureServices(services =>
{
services.AddAWSMessageBus(builder =>
{
builder.AddMessageHandler();
builder.AddSQSPoller(“https://sqs.us-west-2.amazonaws.com/012345678910/MyQueue“);
});
});

var host = builder.Build();
await host.RunAsync();

Noteworthy Features

The framework supports not only sending messages to SQS but also publishing events to SNS and EventBridge. When sending messages, the .NET message types are serialized to JSON and wrapped in an envelope that adheres to the CloudEvents standard, enabling easy interoperability with other programming languages and frameworks.

Here’s an example of the message envelope for the GreetingMessage:

json
{
“id”:”b02f156b-0f02-48cf-ae54-4fbbe05cffba”,
“source”:”/aws/messaging”,
“specversion”:”1.0″,
“type”:”GreeterMessage”,
“time”:”2024-03-21T16:36:02.8957126+00:00″,
“data”:”{\”SenderName\”:\”DemoUser\”,\”Greeting\”:\”Hello!\”}”
}

The framework also efficiently handles message receipt from SQS through a long-running poller, designed for background services and ideal for deployment on environments like EC2 or ECS. It manages the message visibility timeout during processing to prevent multiple clients from handling the same message simultaneously.

Lambda Integration

The framework extends its functionality for use with AWS Lambda functions via the AWS.Messaging.Lambda package, building on the existing integration with SQS. This ease of use ensures you can comfortably shift your message processing tasks to serverless architectures.

FIFO Queue Support

For applications that require ordered message processing, this framework fully supports FIFO queues and respects message ordering, making it ideal for scenarios where the order of operations is crucial.

OpenTelemetry Compatibility

If observability matters to your application, the framework includes support for OpenTelemetry via the AWS.Messaging.Telemetry.OpenTelemetry package. Simply call AddAWSMessagingInstrumentation during your OpenTelemetry configuration.

csharp
public void ConfigureServices(IServiceCollection services)
{
services.AddAWSMessageBus(builder =>
{
builder.AddMessageHandler();
builder.AddSQSPoller(“https://sqs.us-west-2.amazonaws.com/012345678910/MyQueue“);
});

services.AddOpenTelemetry()
   .ConfigureResource(resource => resource.AddService("myApplication"))
   .WithTracing(tracing => tracing
       .AddAWSMessagingInstrumentation()
       .AddConsoleExporter());

}

Getting Started

To effectively leverage this framework, let’s walk through a simple example that consists of an ASP.NET Core Minimal API to publish messages to SQS upon API requests, and a long-running console application to handle these messages.

Prerequisites

You’ll need an SQS queue ready for this process, as it may incur charges if you exceed SQS’s Free Tier. You can create a queue using either the AWS CLI or AWS Tools for PowerShell:

bash

AWS CLI

aws sqs create-queue –queue-name DemoQueue

AWS Tools for PowerShell

New-SQSQueue -QueueName DemoQueue

Publishing Messages

1. Create a new .NET project for the publisher:
   bash
   dotnet new webapi –name Publisher

2. Navigate to the project folder and add a dependency on the AWS Message Processing Framework for .NET:
   bash
   cd Publisher
   dotnet add package AWS.Messaging

3. Replace the contents of Program.cs with your configured queue details:
   csharp
   using AWS.Messaging;
   using Microsoft.AspNetCore.Mvc;

   var builder = WebApplication.CreateBuilder(args);
   builder.Services.AddEndpointsApiExplorer();
   builder.Services.AddSwaggerGen();

   builder.Services.AddAWSMessageBus(builder =>
   {
   builder.AddSQSPublisher(““, “greetingMessage”);
   });

   var app = builder.Build();

   if (app.Environment.IsDevelopment())
   {
   app.UseSwagger();
   app.UseSwaggerUI();
   }

   app.UseHttpsRedirection();

   app.MapPost(“/greeting”, async ([FromServices] IMessagePublisher publisher, GreetingMessage message) =>
   {
   if (message.SenderName == null || message.Greeting == null)
   {
   return Results.BadRequest();
   }

   await publisher.PublishAsync(message);
   return Results.Ok();

   })
   .WithName(“SendGreeting”)
   .WithOpenApi();

   app.Run();

   public class GreetingMessage
   {
   public string? SenderName { get; set; }
   public string? Greeting { get; set; }
   }

4. Run the project:
   bash
   dotnet watch run

5. Test your API through the Swagger UI.

Handling Messages

1. Create another .NET project for the message handler:
   bash
   dotnet new console –name Handler

2. Add dependencies:
   bash
   cd Handler
   dotnet add package AWS.Messaging
   dotnet add package Microsoft.Extensions.Hosting

3. Modify Program.cs to poll for messages from the queue:
   csharp
   using AWS.Messaging;
   using Microsoft.Extensions.DependencyInjection;
   using Microsoft.Extensions.Hosting;

   var builder = Host.CreateDefaultBuilder(args);
   builder.ConfigureServices(services =>
   {
   services.AddAWSMessageBus(builder =>
   {
   builder.AddSQSPoller(““);
   builder.AddMessageHandler(“greetingMessage”);
   });
   });

   var host = builder.Build();
   await host.RunAsync();

   public class GreetingMessage
   {
   public string? SenderName { get; set; }
   public string? Greeting { get; set; }
   }

   public class GreetingMessageHandler : IMessageHandler
   {
   public Task HandleAsync(MessageEnvelope messageEnvelope, CancellationToken token = default)
   {
   Console.WriteLine($”Received message {messageEnvelope.Message.Greeting} from {messageEnvelope.Message.SenderName}”);
   return Task.FromResult(MessageProcessStatus.Success());
   }
   }

4. Run the handler to initiate message processing:
   bash
   dotnet run

Cleaning Up

When you’re done, remember to delete the SQS queue using:

bash

AWS CLI

aws sqs delete-queue –queue-url ““

Or using AWS Tools for PowerShell

Remove-SQSQueue -QueueUrl ““

The AWS Message Processing Framework for .NET opens the door to efficient, AWS-optimized message handling in your applications. For more information, the framework is available as the AWS.Messaging package on NuGet.org, alongside comprehensive documentation and sample templates to kickstart your development journey. You can find the source code and contribute on GitHub. If you have ideas for enhancements, feel free to submit an issue or pull request!