Search

The Great Migration (to Google Cloud)

Updated: May 25

This entry was imported from Group's former blog on Blogger written in 2020



All things must come to an end and Groups' current hosting is no exception. Until now, we have enjoyed a pretty good setup from Corvid. But their platform is not stable

Connections have been randomly refused and false DB timeouts have been seen on a daily basis. Furthermore, their service became half broken the month they renewed our SSL certificates


I tried to engage their support and see if purchasing a higher tier subscription would fix these issues. Their reply has been that "they are working on improving their service" and that all their subscription tiers offer the same performance. We simply can't wait for them since Groups' userbase is growing and if we don't act now I fear we run the risk of dying of success (or of poor planning).


Our destination of choice is Google's new Cloud Datastore and App Engine. Google's cloud platform offers automatic scalability that will give Groups great stability for years to come.


My preliminary analysis shows that it's economical and that it works. Response times are good. Its ecosystem is very rich which will undoubtedly be a huge bonus in the long term


The migration will take place on the following date: Saturday, 4th of July at 10AM UTC time. A "maintenance" popup will be shown during one hour. The migration is not a simple copy/paste process since Corvid uses its own flavour of Node.js. The backend data also needs to pass through an ETL and the new backend service needs to use a new DB adapter. So there is a chance for small failures here and there. If you see any annomaly, please do get in touch at javierantonf@hotmail.com

A few things to consider after the migration is complete:

If you joined Groups before March, your user's new last name will be "null". Feel free to change it via the user settings menu. I did not start saving this field until March, but it now must contain something so empty last names have become "null". I have taken this opportunity to redesign the database schema as it wasn't completely efficient before. Nothing like a full migration to change these things. The downside is that issues might arise from both the migration and the new schema at the same time. Let's hope this doesn't happen

In order to run the migration in a way that users don't feel any differences, a client build has been released (1.83) that is compatible with the new and old worlds. During the migration (1 hour) the app won't be available. After the migration, the app will detect that it has to communicate with a new server and its core functionalities will change seamlessly

I'm really excited about moving things over to Google Cloud. I always was a bit anxious about our current hosting. Its low availability was a huge reason for concern. I also feared the service might just die at any point in time

The lesson learned is that one should always build things with scalability in mind from the beginning. Otherwise, too much work is spent on moving to a new platform and redesigning things