5 things to consider when migrating your data warehouse to AWS Redshift
How to reduce risks
during large-scale data warehouse migration
Feb. 19, 2019 | By Ram Palaniappan
The benefits of migrating your on-premises data warehouse (e.g., Oracle, Teradata, Netezza, etc.) to cloud-based AWS Redshift are numerous. Agility, scale, security and the cost of unused capacity can all be improved by a well-thought-out and executed move to the cloud. But a hasty migration can quickly become frustrating—and expensive. Before making your move, here is a short list of things to consider:
1. Migrate your data before your process
Take a step-by-step approach to moving your data and associated Extract Transform Load (ETL) processes to Redshift. Move all your existing data first. Check your reporting and fine tune as needed to ensure that Redshift is optimized for your needs. After that, begin moving incremental data to Redshift periodically. Only after you’re satisfied with your new database’s performance should you migrate your processes.
2. Develop design patterns
An on-premises database is a one-size-fits-all solution: all your data lives in a single database regardless of type, frequency or volume. But the AWS cloud offers a specialized ecosystem of data management platforms and tools for different design patterns. Think about your needs before you move. For instance, unstructured data performs better on S3; structured does best in Redshift; semi-structured is at home in Dynamo. To get the full benefit of the cloud, you may need more than one type of database.
3. Use cloud-native tools
The best tools are attuned to their native environment. So evaluate legacy tools versus cloud-native components. Convert legacy processes, like Informatica, to AWS Glue, which was designed to operate seamlessly in the AWS ecosystem. Also, consider migrating your ETL processes in an automated fashion rather than doing it manually.
4. Keep your data model the same
Trying to rewrite your data model while migrating is a little like building an airplane while you’re flying it. At the very least it’s complicated; at worst, it’s a disaster. Avoid the risk of additional complications and costs by maintaining your original data model while moving to the cloud. Once you’ve completed your migration, you’ll have numerous new, more efficient methods for optimizing your reporting structure to choose from.
5. Tune, tune, tune to optimize
Once you’ve migrated to Redshift, the opportunities to tune your database are almost limitless. Keep in mind, though, that Redshift is different from on-premises data warehouses like Oracle, Teradata or Netezza, so it’s best to look at best practices for tuning Redshift.
By investing some time and thought into your move at the start, you’ll avoid many common pitfalls—and realize the full benefits of your data warehouse move more quickly.
Ram Palaniappan leads data analytics and insights for TEKsystems.