Full Migration For Our Environments
I had a great project recently that required interacting with several different pieces on the stack.
The project was to create a script for helping the team migrate full environemnts from one another.
That means if they are changing production content, they could easily migrate everything to the staging environemnt or any other developer environment.
Environment Stack Explained
The site itself is built with "Gatsby.js". That means we use javascript, react, styled-components, hooks and more.
For Gatsby we use data sources from "Contentful" and "Shopify".
"Contentful" is used as a cms, as we set the pages structure, content, landing pages, blog posts, product bundles and more.
Shopify is used as the "source of truth" for the product data. Pricing, stock, discounts, product details and images.
The checkout process using Shopify as the prodvider as well.
Requirements
These are the rough requirements:
- easy command to run and pass variables
- copy the data from one Contentful env to another
- copy the connected Shopify store to the destination Shopify store
- make sure all plugins are set properly with right credentials
Problems and Performence
As I gave a quick brief on the tech stack and the requirements, there are some limits that pop up. API limits from Shopify and Contenful. Time constraints, we want the migration to happen as fast as possible. We want feedback that it happened successfully or failed.
The Script
After digging into some packages I found some great tools that will reduce the work.
Contenful cli tool, has a nice command that migrates full environments. That means we saved a ton of work on that side. The only missing piece is updating the credentials for the extra plugins we have there.
Next package is for easy integration with Shopify, that was quite easy. Just creating a store with the needed tokens and we are good to good with easy api calls.
Now the full flow of the script:
- validating parameters
- asking the client if he is sure on running the command with the above parameters
- run contenful migration script
- now doing Shopify updates:
- delete all destination store products if exists
- fetch origin store products
- transform the products for insertion on the destination store
- insert the adapted products to the destination store
- on caviat here, is that Shopify as api limits we can easily reach with many products
- the easy solution here is making one call every 0.5 sec
- that's because Shopify api refills api limit count with 2 calls per second
- store the created products locally
- now update Contenful products models with the newly created product IDs
- update Shopify store and add all products to the proper sales channel
- update Contenful plugin credentials
- done