Avoid vendor lock-in: Migrating content from Contentful to Kontent
In the times of monolithic systems, it was crucial to pick the right system at the beginning. That decision inevitably locked you with a single vendor for years. Nowadays, in the microservices era, vendors claim itʼs easy to switch between their systems. Letʼs take a look at what “easy” means when switching your headless CMS and whether that decision could cost you your job.
Ondrej PolesnyPublished on Jul 30, 2020
In this article, Iʼll first explain what migration actually means and what are the general steps youʼll need to take. In the second part, I will migrate a real Contentful project to Kentico Kontent and point out the tricky parts.
A few years ago, a friend of mine asked me if I could migrate her website—she owns a food delivery service—to Prestashop. The website was built on top of a custom CMS built by an agency that no longer existed at that time. So I downloaded the website backups and started working on it.
With every migration, or at least with a web-centric one, weʼre looking at two things: data and front-end migrations.
The first thing was the product data. Her e-shop had almost 500 sandwiches, salads, desserts, and other items. Every item featured a set of pictures, nutrition tables, descriptions, tags, and so on. As you can imagine, it would take a week to manually copy-paste them into the new solution. So I created a migration script. Then another one for categories. And another one for content pages. And another one... You get the idea. Thatʼs still doable, you cherry-pick the data you need and map it to the new data structure.
The worst part of the migration is unstructured data. In my case, those were the content pages. Over 50 percent of all content was stored in components, widgets, hardcoded in templates, and so on, making it almost impossible to extract the data programmatically. The migration script implementation handling all edge cases would have likely taken longer than manual copy-pasting.
The second part of the migration was the front end. It gets easier there as you can use the same front-end assets and donʼt spend days on optimizing the look and feel. Unless youʼre doing a look refresh as part of the migration. But I still needed to find all the code blocks and translate them from XSLT to Smarty as thatʼs what Prestashop uses for templating.
The era of microservices
You see that handling that migration from one monolith to another was tricky. We had to draw a line between what was a meaningful migration effort and what was going to be manually copied over. In the end, it took me over 5 weeksʼ worth of work anyway.
Nowadays, we tend to use microservices architecture, which changes one key aspect of all migrations. Content structure.
To be able to cover all use cases editors might have for their pages, monoliths allowed their users and developers to save data outside of any structure. Did your components need to store key-value pairs? Or just a little HTML? No problem.
The headless CMSs that represent content storage in microservices space solved the “unstructured data” problem. They had to. While monoliths were web-centric, headless CMSs deliver the content to multiple channels. So they canʼt allow you to store anything anywhere, but rather lead you towards always structuring your content. For example, what would a smartwatch app do with rich HTML code anyway?
They also simplified the front-end migrations by being decoupled from the implementation. In reality, you still need to solve different data schemes in your implementation, but weʼre already seeing data normalization tools, such as Sourcebit, that make this process easier. If youʼre interested, read more about Sourcebit.
Content migration between headless CMSs
But letʼs take a look at the real thing. I used Contentful to store seven blog posts and the content of About me page from my personal site.
They actually cover a good portion of the CMSʼs features, as blog posts need an image, URL slug, and rich text, while the About me page contains nested items (outlined below).
All that is connected via Sourcebit to a Next.js website. I want to migrate everything to another headless CMS, Kentico Kontent.
Pure API migration
The first option is to use the management APIs of these headless CMSs and create migration scripts. This is somewhat similar to what I did with my friendʼs website. I created a few SQL queries to adjust the data and store it in another MySQL database. With management APIs, it gets much simpler. You get access to all of the tools for the platform youʼre using to implement the migration scripts. Often, the CMSs also give you an SDK that handles the communication so you can focus on the actual data migration.
But surely youʼre not the first one trying to move content from one CMS to another. Just like I wasnʼt the first one trying to stream videos from Windows to Apple TV, except in this case, the first problem actually has a solution. Open source is very popular these days, and a lot of developers make their implementations available to the community, so I always check if there is a tool for migrating between the source and the target CMS.
For migration from Contentful to Kontent, there is an open-source tool developed by Matthew Castrillon-Madrigal. He used it to migrate a large and complex project between these two CMSs, so itʼs not really an experimental tool. The implementation contains a set of shell scripts that download content models and content items from Contentful, store it locally, and run respective API calls towards Kontent to create the same structure and data there.
So, in other words, the same code youʼd have to write for the migration.
Letʼs take a look at how it works. First, we need to set up a connection to both CMSs:
yarn program --contentful_setup
yarn program --kontent_setup
The tool asks for project IDs and API keys and stores them in local configuration files .environments.json and export/config.json.
Then, we can start fetching data from Contentful.
Note: If youʼre running on Windows, youʼll need to use git bash client to run the migration. You may also need to tweak the shell execution in ./src/index.ts as described here.
yarn program --fetch
This triggers the download of the content models, content items, and assets into a local folder structure.
The data is then mapped to Kontent structures and fed into Kontent by:
yarn program --migrate
Once we execute this command, the migration tool starts uploading the data into the CMS. It shows progress on screen and logs failed operations:
I stumbled upon two issues I had to solve:
Before you start uploading the data into Kontent, change the codename of your default language in Kontent to the codename you were using in Contentful. I was using en-US. If you leave the default “default” codename of your default language in Kontent, you will end up with one extra language.
- Clearing the project
Things go wrong. Especially in website development. We tend not to trust our code if it works right away :-). If the import fails or you misconfigured the tool, you can repeat it. To make sure your project is clean and prepared for another import, use the Kontent Template Managerʼs cleaner. Sometimes you also need to re-fetch the data from Contentful.
I ended up with these blog posts:
And these nested items correctly positioned within the About me page content item:
The import also correctly handled all assets. Letʼs take a look now at how we can adjust the front end.
Switching headless CMSs on the front end
My website was built using Next.js and Sourcebit , which should minimize the effort needed for switching between data providers. If youʼre not familiar with how Sourcebit works, take a look at this article or watch this video. Given that I imported the data into a blank Kontent project, I expected rather easy configuration changes than having to dig into templates and the componentsʼ code.
This is what the website looked like when connected to Contentful:
We can change the data source by reinitializing Sourcebit and selecting the new CMS by running the following command at the project root:
npm init sourcebit
Iʼm prompted to add the new project ID and the language codenames. In my case, there is only one language with the newly changed codename en-US. All other steps of the wizard will be exactly the same as with the old CMS.
Running the project right after the switch shows some errors that need to be fixed:
It looks like the blog posts are not rendered. However, the problem is that the images are missing. If the blog post content item contains an image, its URL is available under
post.page.image.urlwhen using Contentful.
If the blog post uses a remote image, the codename of the field is
- Rich text resolving
Every CMS handles rich text resolving slightly differently. The implemented resolver for data coming from Contentful is therefore obsolete and we need to add a new one for Kontent.
These little differences were the only changes I had to make. Now the website is successfully migrated and uses data from a different CMS.
Just an experiment?
So, would you consider that process easy?
In my case, the content migration required only access configuration. There will be cases when you need to adjust the scripts as 1:1 conversion of data between different CMSs is not always possible due to different feature sets. And that is possible because the implementation is open-sourced.
The front-end migration worked after a few small tweaks, mainly thanks to the data normalization done by Sourcebit.
You see, nowadays, migration is not something to be overly scared of. We should still do proper research before making a decision about a tool we want to use for our next projects. But if we make a mistake or find a better tool later, as tools for microservices are evolving surprisingly quickly, it should not cost us our jobs.
Take a look yourself at what the implementation looks like on my GitHub.