Tag: routing

Posted on

Azure APIM Content Based Routing Policy

Content based routing is where the message routing endpoint is determined by the contents of the message at runtime. Instead of using an Azure Logic App workflow to determine where route the message, I decided to use Azure APIM and a custom policy. The main objective of using an APIM policy was to allow me to add or update the routing information through configuration rather than code and to add a proxy API for each of the providers API endpoints.

Below is the high level design of the solution based on Azure APIM and custom policies. Here the backend ERP system sends a generic PO message to the PO Routing API which will then forward the message onto the suppliers proxy API based on the SupplierId contained in the message. The proxy API will then transform the generic PO to the correct format before being sent to the supplier’s API.

The main component of this solution is the internal routing API that accepts the PO message and uses a custom APIM Policy to read the SupplierId attribute in the message. The outbound URL of this API is then set to the internal supplier proxy API by using a lookup list to find the matching reroute URL for the SupplierId. This lookup list is a JSON array stored as a Name-Value pair in APIM. This allows us to add or update the routing information through configuration rather than code.

Here is the structure of the URL lookup list which is an array of JSON objects stored as a Name-value pair in APIM.

[
   {
     "Name": "Supplier A",
     "SupplierId": "AB123",
     "Url": https://myapim.azure-api.net/dev/purchaseorder/suppliera
    },
    {
     "Name": "Supplier B",
     "SupplierId": "XYZ111",
     "Url": https://myapim.azure-api.net/dev/purchaseorder/supplierb
    },
    {
     "Name": "SupplierC",
     "SupplierId": "WAK345",
     "Url": https://myapim.azure-api.net/dev/purchaseorder/supplierc
    }
 ]

PO Routing API

The code below is the custom policy for the PO Routing API. This loads the above JSON URL Lookup list into a variable called “POList”.  The PO request body is then loaded into a JObject  type and the SupplierId attribute value found. Here you can use more complex queries if the required routing values are scattered within the message.

Next the URL Lookup list is converted into a JArray type and then searched for the matching SupplierId which was found in the request message. The  variable “ContentBasedUrl” is then set to the URL attribute of the redirection object. If no errors are found then the backend url is set to the  internal supplier’s proxy API and the PO message forwarded on.


  // Get the URL Lookup list from the Name-value pair in APIM 
    <set-variable name="POList" value="{{dev-posuppliers-lookup}}" />


    <set-variable name="ContentBasedUrl" value="@{ 

        JObject body = context.Request.Body.As<JObject>(true);

        // Create an JSON object collection
        try
        {

          // Get the AccountId from the inbound PO message
            var supplierId = body["Purchase"]?["SupplierId"];

            var jsonList = JArray.Parse((string)context.Variables.GetValueOrDefault<string>("POList"));

          // Find the AccountId in the json array 
            var arr = jsonList.Where(m => m["SupplierId"].Value<string>() == (string)supplierId);

            if(arr.Count() == 0 )
            {
                return "ERROR - No matching account found for :- " + supplierId;
            }

            var url = ((JObject)arr.First())["Url"];
            if( url == null )
            {
                return "ERROR - Cannot find key 'Url'";
            }          

            return url.ToString();
        }
        catch( Exception ex)
        {
            return "ERROR - Invalid message received.";
        }

    }" />
    <choose>
        <when condition="@(((string)context.Variables["ContentBasedUrl"]).StartsWith("ERROR"))">
            <return-response>
                <set-status code="400" />
                <set-header name="X-Error-Description" exists-action="override">
                    <value>@((string)context.Variables["ContentBasedUrl"])</value>
                </set-header>
            </return-response>
        </when>
    </choose>
    <set-backend-service base-url="@((string)context.Variables["ContentBasedUrl"])" />
</inbound>
<backend>
    <base />
</backend>
<outbound>
    <base />
</outbound>
<on-error>
    <base />
</on-error>

Supplier Proxy API

A proxy API is created for each of the suppliers in APIM where the backend is set to the suppliers API endpoint.  The proxy will typically have the following policies.

The primary purpose of this proxy API is to map the generic PO message to the required suppliers format and to manage any authentication requirements. See my other blog about using a mapping function within a policy https://connectedcircuits.blog/2019/08/29/using-an-azure-apim-scatter-gather-policy-with-a-mapping-function/ There is no reason why you cannot use any other mapping process in this section.

The map in the Outbound section of the policy takes the response message from the Suppliers API and maps it to the generic PO response message which is then passed back to the PO Routing API and then finally to the ERP system as the response message.

Also by adding a proxy for each of the supplier’s API provides decoupling from the main PO Routing API,  allowing us to test the suppliers API directly using the proxy.

The last step is to add all these API’s to a new Product in APIM so the subscription key can be shared and passed down from the Routing API to the supplier’s proxy APIs.

Enjoy…

Posted on

Using an Azure APIM Scatter-Gather policy with a Mapping Function

This is a follow up from a previous blog “Azure APIM Scatter-Gather Pattern Policy” where I wrote about using the Wait policy to create a scatter-gather pattern.  A few colleges were asking about being able to map the inbound request to the different schemas required by each of the Microservices.

A high level design is shown below using two “Wait” polices and an Azure Function for the mapping. The first policy ‘Translation’ sends the request to the mapping function and when completed the second policy ‘Scatter’ is executed to send the mapped request to each of the Microservices.

image

The internals of the Azure Function that maps the incoming request is shown below as an example. Here I am using a route on the supplier name and a simple If-Then statement to select which static translator method to call.

public static class MappingService
   {
       [FunctionName("DataMapping")]
       public static async Task<IActionResult> Run(
           [HttpTrigger(AuthorizationLevel.Function,  "post", Route = "Pricing/{suppliername}")]
           HttpRequest req,
           string suppliername,
           ILogger log)
       {
           log.LogInformation("C# HTTP trigger function processed a request.");

           string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
           dynamic response = null;
           if (suppliername.Equals("Supplier1", StringComparison.OrdinalIgnoreCase))
               response = Translator.TranslateSupplier1(requestBody);
           else if (suppliername.Equals("Supplier2", StringComparison.OrdinalIgnoreCase))
               response = Translator.TranslateSupplier2(requestBody);            

           return response != null
               ? (ActionResult)new OkObjectResult(response)
               : new BadRequestObjectResult("Invalid message.");
       }
   }

Below is the code for the Translation Policy which calls the two Azure function resources in parallel that accepts the inbound price request message. The function access code is stored as a named-value pair in APIM called “functionapkey”.

<!-- Call the mapping service -->
<wait for="all">
   <send-request mode="copy" response-variable-name="res_SupplierMap1" timeout="120" ignore-error="false">
       <set-url>https://mapsvc.azurewebsites.net/api/Pricing/supplier1?code={{funcmapkey}}</set-url>
       <set-method>POST</set-method>
       <set-body>@(context.Variables.GetValueOrDefault<string>("requestPayload"))</set-body>
   </send-request>
   <send-request mode="copy" response-variable-name="res_SupplierMap2" timeout="120" ignore-error="false">
       <set-url>https://mapsvc.azurewebsites.net/api/Pricing/supplier2?code={{funcmapkey}}</set-url>
       <set-method>POST</set-method>
       <set-body>@(context.Variables.GetValueOrDefault<string>("requestPayload"))</set-body>
   </send-request>
</wait>
 

The code for the Scatter Policy is shown below which is similar to the original blog post. However it uses the mapped outputs from the Translation policy that are stored in the res_SupplerMap1 and res_SupplerMap2 context variables instead.

<!-- Call the pricing services -->
<wait for="all">
   <send-request mode="copy" response-variable-name="response_1" timeout="120" ignore-error="false">
       <set-url>{{serviceendpoint_1}}</set-url>
       <set-method>POST</set-method>
       <set-body>@(((IResponse)context.Variables["res_SupplierMap1"]).Body.As<string>())</set-body>
   </send-request>
   <send-request mode="copy" response-variable-name="response_2" timeout="120" ignore-error="false">
       <set-url>{{serviceendpoint_2}}</set-url>
       <set-method>POST</set-method>
       <set-body>@(((IResponse)context.Variables["res_SupplierMap2"]).Body.As<string>())</set-body>
   </send-request>
</wait>

The last policy checks the status of each of the pricing services and returns the results as a composite message if there were no errors encountered. This is similar to the original blog post but instead of returning a JObject I am now returning a JArray collection.

<choose>
   <when condition="@((int)((IResponse)context.Variables["response_1"]).StatusCode != 200)">
       <return-response response-variable-name="response_1" />
   </when>
   <when condition="@((int)((IResponse)context.Variables["response_2"]).StatusCode != 200)">
       <return-response response-variable-name="response_2" />
   </when>
   <otherwise>
       <return-response>
           <set-status code="200" reason="OK" />
           <set-header name="Content-Type" exists-action="override">
               <value>application/json</value>
           </set-header>
           <set-body>@{
                       JArray suppliers = new JArray();
                       suppliers.Add(((IResponse)context.Variables["response_1"]).Body.As<JObject>());
                       suppliers.Add(((IResponse)context.Variables["response_2"]).Body.As<JObject>());                               
                       return suppliers.ToString();
           }</set-body>
       </return-response>
   </otherwise>
</choose>

The completed policy for the method looks like this below. Take note the request payload is stored in the variable named “requestPayload” initially to avoid locking body context in the <Wait> policies.

<policies>
    <inbound>
        <set-variable name="requestPayload" value="@(context.Request.Body.As&lt;string>(true))" />
        <!-- Call the mapping service -->
        <wait for="all">
            <send-request mode="copy" response-variable-name="res_SupplierMap1" timeout="120" ignore-error="false">
                <set-url>https://mapsvc.azurewebsites.net/api/Pricing/supplier1?code={{funcmapkey}}</set-url>
                <set-method>POST</set-method>
                <set-body>@(context.Variables.GetValueOrDefault<string>("requestPayload"))</set-body>
            </send-request>
            <send-request mode="copy" response-variable-name="res_SupplierMap2" timeout="120" ignore-error="false">
                <set-url>https://mapsvc.azurewebsites.net/api/Pricing/supplier2?code={{funcmapkey}}</set-url>
                <set-method>POST</set-method>
                <set-body>@(context.Variables.GetValueOrDefault<string>("requestPayload"))</set-body>
            </send-request>
        </wait>
        <!-- Call the pricing services -->
        <wait for="all">
            <send-request mode="copy" response-variable-name="response_1" timeout="120" ignore-error="false">
                <set-url>{{serviceendpoint_1}}</set-url>
                <set-method>POST</set-method>
                <set-body>@(((IResponse)context.Variables["res_SupplierMap1"]).Body.As<string>())</set-body>
            </send-request>
            <send-request mode="copy" response-variable-name="response_2" timeout="120" ignore-error="false">
                <set-url>{{serviceendpoint_2}}</set-url>
                <set-method>POST</set-method>
                <set-body>@(((IResponse)context.Variables["res_SupplierMap2"]).Body.As<string>())</set-body>
            </send-request>
        </wait>
        <choose>
            <when condition="@((int)((IResponse)context.Variables["response_1"]).StatusCode != 200)">
                <return-response response-variable-name="response_1" />
            </when>
            <when condition="@((int)((IResponse)context.Variables["response_2"]).StatusCode != 200)">
                <return-response response-variable-name="response_2" />
            </when>
            <otherwise>
                <return-response>
                    <set-status code="200" reason="OK" />
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/json</value>
                    </set-header>
                    <set-body>@{
                                JArray suppliers = new JArray();
                                suppliers.Add(((IResponse)context.Variables["response_1"]).Body.As<JObject>());
                                suppliers.Add(((IResponse)context.Variables["response_2"]).Body.As<JObject>());                               
                                return suppliers.ToString();
                    }</set-body>
                </return-response>
            </otherwise>
        </choose>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

Using the tracing feature in APIM, you can see the initial price request message below. This will be sent to the mapping function Pricing method.

 image

Below is the trace output from APIM showing the two different messages returned from mapping function and assigned to the res_Supplier variables.

image

Below is the composite message returned from APIM as an JSON array containing each supplier.

image

In conclusion, using the two <Wait> polices to send multiple requests in parallel yields a request/response latency of around 200ms on average in this scenario. Also instead of using an Azure Function to map the incoming request, you could replace it with a couple of “Transformation” policies.

Enjoy…