🔗 Database Migrations

I consider database migrations one of the most annoying problems to deal with during a software engineer's life. Not only that, if it goes wrong, as it often does, people tend to develop anxiety related to any schema changes.

So why is it so annoying? It looks simple at first glance but is deceptively tricky when you start thinking about it.

My primary framework of choice is the Python-based Django, which released schema migrations over a decade ago and which I still consider one of the best migration engines that I've seen. (and I worked with many different frameworks across many languages).

But even with an excellent framework that automatically generates migration files, stores the migration references in the database, and allows for easy rollbacks — there are a ton of things to consider when you do database migrations.

Imagine a novel approach for splitting up the -'name' field into 'first_name' and -'last_name' in the user Table. All is going fine. You do the schema migration, run the migration script of splitting old data into two fields, and deploy the latest API that works with these changes. Still, something goes wrong with the new fields. Your monitoring shows many unique users with failed saved due to some validation error, e.g., not all characters are being accepted. This issue is deemed critical, so you and the team decide to do a rollback. Sounds good. You revert the schema migration and revert the app to the previous state, but then you notice that writes were happening to the new fields during that small window of deployment, and now some users have missing data in the name field, which leads to data inconsistency.

And now add more complexity with the requirement of doing all that with zero downtime.

continue reading on vadimkravcenko.com

⚠️ This post links to an external website. ⚠️