Microsoft CRM + Azure Service Bus, part 4 (two-way relaying and using Azure Event Hubs)

Integrating Microsoft Dynamics CRM with Microsoft Azure Service Bus, using queues, topics, relays and the Event Hub

In the previous posts we've looked at using endpoints in CRM, how we can create a simply workflow to push the Activity Context to the Azure Service Bus, using queues and creating a simple one-way listener. In this post we'll focus on extending the code to enable two-way relaying, which will allow us to return data back to CRM. We'll also look at how we can integrate CRM with Azure Event Hubs, a service based on ASB which allows for a lot of throughput.

Click here for part1

Click here for part2

Click here for part3

Allow return value in the CRM Workflow step

The first thing we'll do is to write some more logic for our workflow, to enable us to use a return value. The relay service only returns a string, which means that we could easily push JSON or base64 through the bus to have even more meaningful feedback from the service. Be sure to remember the maximum message size for your service bus, and the timeout values. I do not recommend keeping a connection for a long time to transfer end-to-end data, but there are loads of scenarios where you'd like to get something back.

[Output("Return value")]
public OutArgument<string> ReturnValue { get; set; }

OK, so what we've done here is to add an output variable to our workflow step. This allows us to take the return value and use it inside our workflow in later steps.

protected override void Execute(CodeActivityContext executionContext) 
{
    var context = executionContext.GetExtension<IWorkflowContext>();
    var serviceEndpointNotifySvc = executionContext
        .GetExtension<IServiceEndpointNotificationService>();
    var result = serviceEndpointNotifySvc.Execute(
        ServiceEndpoint.Get(executionContext), context
    );
    ReturnValue.Set(executionContext, result);
}

The code looks mostly the same as it did in part2 of this series, with a couple of additions. Firstly, we assign the return value from the notification service's Execute method to a variable "result". Next we set the value of the new output variable "ReturnValue" to the result, using the execution context of the workflow.
Now just build and deploy the updated workflow, and we'll head into MSCRM. Navigate to the Settings -> Processes area, and open up the workflow created earlier. Deactivate the workflow to enable editing, and then add an update step after the custom workflow step.
I'm going to update the Description field on the Account to the ReturnValue from our custom workflow step.

Finally, go into the plugin registration tool to update the relay service we added in part3. Make sure it's changed to a two-way relay, and eventually update the name and path if you want that. Just make sure to update the same values inside the service.

Writing a two-way relay service

To enable two way communication we first have to change our service behavior class. We have to change the interface it inherits from and add a return value.

[ServiceBehavior]
public class RemoteService : ITwoWayServiceEndpointPlugin
{
    public string Execute(RemoteExecutionContext c)
    {
        Console.WriteLine(
            $"Entity: {c.PrimaryEntityName}, id: {c.PrimaryEntityId}");
        return "Message processed in two-way listener";
    }
}

As you can see we're now inheriting from the ITwoWayServiceEndpointPlugin interface, modified the Execute method to expect a return value, and we added a return statement after printing out to the console. This means that whenever our relay service is triggered it will print out the message as before, but will also return our magic string.

The only other change we have to make is to change the typeof implementation. Earlier we specified IServiceEndpointPlugin, so we have to change it to ITwoWayServiceEndpointPlugin. I've done a little dirty hack, because I've been changing back and forth between relay types while testing, so mine looks like this:

sh

    .AddServiceEndpoint(

        typeof(RemoteService).GetInterfaces().First(), 

        new WS2007HttpRelayBinding(), 

        AppSettings["SBusEndpoint"]

    ).Behaviors.Add(tcEndpointBehavior);

sh.Open();
Console.WriteLine("TwoWay-listener, press ENTER to close");
Console.ReadLine();

sh.Close();

While this may seem like reflection and a good idea, it will only work until you've built yourself a framework to reuse across different projects/customers, and all of a sudden you're implementing several interfaces and everything stops working. For this demo, it's OK, but I'd rather specify the interface type in production code.

Testing two-way relaying

Now that we've got that out of the way, it's time to test our new implementation. Start your new and improved service, and go to CRM and trigger the plugin. If everything goes as planned you'll be looking at the following window for your relay service

 And if we go into CRM and check out the description on our contact entity we'll see the following

So that's all it takes to have two-way communication working in a relay service. We've got CRM on one side, Azure Service Bus as the communication channel, and a simple console project on the other side.

What's a REST listener?

One option I haven't mentioned so far is the REST listener which can be specified in the plugin registration tool. This is simply a two-way relay which uses REST endpoints instead of .Net binaries. This would allow you to create and run a relay service in Node.js, apache tomcat, powershell, IIS, or whichever web server you'd want. Just to trigger all the popular buzwords, this can enable you to use MSCRM, Azure Service Bus, and deploy node.js relay services in docker containers.

Azure Event Hubs

Azure Event Hubs is a special kind of service bus which is designed to handle huge amounts of messages, we're talking millions of messages per second. It's probably not what you're looking at for general integration purposes, but there are several scenarios where it could benefit your company in a big way.
The first thing I thought of was using this for auditing. Just write a simple workflow or plugin which pushes any creates, updates and deletes as messages to the event hub. Then you can use stream analytics or some other technology to populate an auditing system with actions performed by your users and integration systems. Anyone who's used the out-of-the-box auditing functions in MSCRM knows that processing the information is tedious, at best, and more often than not close to impossible to get any useful data from. But if you start pushing it into an external auditing system based on Azure services then you could use clever stuff like temporal tables to design a robust, maintainable system.

The second thing I thought of was predictive analysis. Pushing raw data to the event hub, which allows for transfer to an Azure Data Warehouse for real-time data crunching and you have a great additional source of data that can be used for predictive analysis or (buzzword warning) machine learning.

There are probably a lot of cool things you can do with this, but I won't elaborate all my ideas in this blog post. What I want to stress is the price tag. It is incredibly cheap compared to legacy systems based on clusters of servers running different messages queues with some expensive bus technology on top. And the performance is great, no matter which size you pick. It doesn't matter if you're running hundres, thousands, or hundreds of millions of messages per day, the performance is always great but there's no entry level cost, the price scales with usage.


That's all for this blog series (at least for now). I might come back to visit later on when I've done some more in-the-field deployments with the ASB.