How to Automatically Translate Content Using Webhooks
On multilingual sites, it has always been a challenge to keep the content translated to all languages. How does the recent update of webhooks help solve that challenge automatically and let you take a coffee break instead?
It’s nice when things just work automatically, isn’t it? Sure, you could translate that new Kontent article into Spanish on your own, but why waste your time if it can be done for you? This is where our powerful webhook feature comes in—when you change a content item, the system sends a notification to the URL provided. This is generally a serverless function hosted in Azure or Amazon, or a third-party endpoint that performs the translation. In this example, we will be using Microsoft’s Translator Text Cognitive Service for translation.
Setting up the project
If you run the application with `npm start`, it should be accessible at http://localhost:3000/. Now, we need to set up the endpoint that Kontent will POST the webhook to. In app.js, add a new "/webhook" route and simply return a 200 response:
Note: You can also access all mentioned code files on this GitHub repository.
Accessing a local server from the Internet
We should test the /webhook route to ensure that Kontent can reach your endpoint, before adding any more code. To make our local application publicly accessible, we can use ngrok. It’s free to register; then follow the steps here to get it running on your PC. In step 4, use the port your local application is running on:
If successful, you will see a message displaying information including your public URL:
We’ll use this URL in the webhook configuration in Kontent.
First, we need to set up our workflow. We want to automatically translate a language variant when it reaches a particular workflow step, so let’s add the Translation step:
Now go to Project settings > Webhooks to create a new webhook and call it “Automatic translation”. Delete the existing triggers and select your Translation step from the drop-down menu under “Workflow steps of content items to watch”. Copy the URL from ngrok and add it to the URL address field with our /webook route at the end:
We’re ready to test! Make sure both your Express application and ngrok are running and move a content item from the Review step to Translation. Within seconds, you should see ngrok report a successful POST to /webhook, and the dot next to your webhook in Kontent should turn green, indicating that it’s working.
Validating the Webhook
As the sample code notes, we need to use body-parser to parse the raw JSON data. Install body-parser:
Then tell Express to use it when parsing JSON data in app.js:
We can now get the signature from the header of the POST request and confirm whether the signature is valid. You will need to save the webhook secret found in Kontent as a const in your app.js file, or use an environment variable. Create a new function to validate the signature:
In your /webhook route, call your hasValidSignature function by passing the body and signature of the req.headers object:
Test your webhook again to ensure that you receive a successful 200 response. Once the webhook is properly validated, we can move on to reading the data that Kontent sent in the body of the webhook notification.
Processing the Webhook
Let’s take a look at the information given to us in the webhook notification and determine what we need to translate our variants. The body of the workflow webhook notification differs from previous non-workflow webhooks. It will look something like this:
Item is the ID of the language variant that moved into the Translate workflow step. Since this webhook triggers only for that workflow step, we don’t need to worry about the transition_from or transition_to values. To completely translate this language variant into other languages, we need a few different pieces of information:
- The elements of the content type that should be translated
- The text of those elements from the updated language variant
- The languages that we need to translate into
We can break this down into 4 steps:
- Get the language variant using the ID in the webhook notification.
- Get the content item of the language variant using the ID from step 1.
- Get the content type of the content item using the ID from step 2.
- Get the project languages to translate content into.
To begin this process, create a new function in app.js to contain the logic and call it from the /webhook route:
We need the updated variant’s ID from the JSON data to use in the Content Management API call we will make in step 1 of the process. Let’s also get the language ID, since we only want to translate English variants (the default language in this example) into other languages:
In the app.js file, declare some global variables that will store the results of our CM API calls:
Install rxjs and load the mergeMap() function, as we’re going to use this function to create a single Observable for our 4 requests, along with map() which we’ll use later:
Step 1 - Get the language variant using the ID in the webhook notification
Now we’re ready to get the data we need in our four-step process. The first step is to get the full language variant using viewLanguageVariant(), since we only have the ID at this point:
This will provide us with the variant’s element IDs and their values, and some system data about the variant itself. To determine which elements should be translated, we need the content item.
Step 2 - Get the content item of the language variant
We can use the result of the first Observable to get the content item’s ID and request the content item using viewContentItem():
Step 3 - Get the content type of the content item
With the content item definition, we can now request the content type. This will allow us to identify which elements of the updatedVariant object are text or rich_text elements, and translate their values. We’ll save the results of this Observable, then call viewContentType():
Step 4 - Get the project languages to translate content into
We now have all of the data except the project languages. We can store the results of the above Observable and then use listLanguages() to get the languages:
Now, we can roll up these 4 Observables into one:
The result of this final Observable will be the list of languages. It contains a lot of information about the language, but we only want the codename to send to our external service. If you are not using standard codenames like “en-us” in your project, go change them now! This allows us to easily send the codename to the external service, which is expecting the four-letter format. We’ll use rxjs’ map() function to create an array of codenames:
With the results of the fourth CM API call, we now have all the information we need to translate the content!
Translating the content
Our next plan of action is to get all of the translatable elements in the content type, loop through the language code names, and send the values from the English language variant to an external service for translation. We’ll use filter() to find the text and rich_text elements, then create an array of their IDs using the map() function:
With this list of elements that need translating, we can loop through all of the language code names and call a function which we’ll create next to perform the actual translation and update the variant:
Our code is now calling upsertLanguageVariant() for each non-English culture and passing the array of element IDs that need to be translated. We’re going to use rxjs’ zip() to send multiple requests to Microsoft’s Translator Text API—one request per element. Create the new upsertLanguageVariant function and an array to hold our Observables, and register the zip() function:
Now, we’ll loop through all of the elements in the updated English variant. If the element’s ID is in our list of translatable IDs, we create an Observable for translation and add it to our array:
To create an Observable for a REST request to Microsoft, we’ll use axios-observable. Install it and then register it in app.js
You will also need a key from Microsoft to use their service. Create an Azure Cognitive Services account for the Translator Text service, and after that, save “Key 1” from the Keys tab as an environment variable or const in app.js.
We’re ready to create the REST request to translate each individual element’s value. According to Microsoft’s API reference, we should provide the from parameter for the source language, the to parameter for the target language, and the textType as rich text elements store HTML in them. The key is sent in the “Ocp-Apim-Subscription-Key” header, and the body will conform to their model. Here is the function so far:
You could technically translate all of the elements in one request, but there would be no way to tell which text belongs to which element due to the way Microsoft responds. This is what our pipe(map()) function is for after the request is created—the result of each individual request will be stored as an array with the element ID and translated text, like this: ['4e9acd7a-2db8-4c33-a13a-0c368ec2f108', 'Ahoj']. All of these results are then stored in a single object returned by our zipped Observable.
Let’s zip() this array of Observables using the spread operator and get the result. With the result, we want to loop through each element of the English variant and see if there is a matching element ID. If so, we can set the value in updatedVariant to the new translated value:
You’ll notice we needed to do a small replace(). Microsoft’s service does not close line breaks, which the Kontent API requires, so we need to fix that. The updatedVariants.data.elements array is now populated with the default English values and some translated values, and we can use these elements to upsert a new language variant.
To accomplish this, we’ll use createNewVersionOfLanguageVariant() to create a new version of the target variant. After that, we’ll call upsertLanguageVariant() and pass the content item’s ID, the target language, and the elements array which we’ve updated:
Note that this automatic translation will only work for variants that are published. In order for this to work for all variants, you will need to check whether the variant is published, and move the variant to the Draft step if it is not. You can view the full source code for the app.js file on GitHub using the link at the end of this article.
What We’ve Learned
Whew! It took several CM API calls to get the data we needed, but with the help of rxjs our code was organized and efficient. This is just one small example of the freedom that webhooks provide for your projects. They can be used to keep external integrations up to date, perform automatic functions with your content, and so much more!
If you’re interested in seeing other examples of using webhooks, you can check out this great article: Clearing Obsolete Cache Entries with Webhooks. You can view the full code for my implementation of automatic translation on GitHub, including instructions to get the sample application with automatic translation running on your machine!