Our AWS RDS Blue/Green Migration Avoids MySQL 8.4 Validation Pitfalls

My team is caught in the MySQL 8.0 end-of-life panic. When AWS began charging us Extended Support fees, we had to migrate everything to 8.4 LTS. Our setup is complicated, though: one giant RDS instance serves about seven separate projects. Older Laravel apps, several Node.js services, and a handful of legacy sites all share a single cluster through different logical databases.

The obvious choice is an RDS Blue/Green deployment. It offers zero-downtime, but we encountered a major obstacle during validation. Anyone planning a migration needs to understand that the Green environment is essentially read-only for testing purposes. A staging app cannot simply connect to the Green instance and run a full integration suite involving writes, because that could disrupt the replication stream or cause something unexpected before the actual switchover.

We tried running several write-heavy migration scripts on the Green instance to check whether the 8.4 engine supported our specific data types correctly, but the restriction became clear. In a real-world AI workflow or complex app deployment, you cannot merely “hope” that the read queries work; you need to verify the full write cycle.

Instead of forcing the Green instance to serve as our testbed, we began adding snapshots to the process. Here is the practical tutorial for the workflow we used:

The “Snapshot-Parallel” Testing Workflow

Rather than testing directly on Green, we treated the Green instance as a “golden image” for compatibility testing.

1. Spin up the Blue/Green Deployment: Create the Green environment and let it upgrade to MySQL 8.4.
2. Take a Snapshot of Green: Once the Green instance is upgraded and synchronized, take a manual snapshot.
3. Restore to a Temporary Test Instance: Restore that snapshot to a completely separate, isolated RDS instance.
4. Run Destructive Tests: Give each of the seven project teams its own restored snapshot instance. Since this is a standalone copy, they can run UPDATE, DELETE, and schema migrations without affecting Blue/Green replication.
5. Verify and Sign-off: Once a project confirms 8.4 compatibility on the snapshot, it receives the green light.
6. Execute Switchover: Trigger the actual Blue/Green switchover only after every single project has signed off.

This approach reduces the “big bang” risk. If a Laravel app breaks on 8.4, it breaks on a temporary snapshot instance rather than the environment being promoted to production.

For anyone following this process, monitor the parameter groups closely. Many of the 8.4 compatibility issues we encountered were not SQL syntax errors; they resulted from default parameter changes in the new engine version. With the snapshot method, we could adjust the parameter group on the test instance and document exactly which changes were needed on the Green instance before the final cutover.

The process adds a few extra steps to the deployment, but it is the only way to sleep soundly when managing a shared database for multiple teams.