Amazon Redshift DC2 migration strategy with a buyer case research

It is a visitor put up by Satoru Ishikawa, Options Architect at Classmethod in partnership with AWS.

In April 2025, AWS introduced the deprecation of Amazon Redshift DC2 cases, guiding customers emigrate to both Redshift RA3 cases or Redshift Serverless. Redshift RA3 cases and Serverless undertake a design that separates storage and compute, provides new options resembling knowledge sharing, concurrency scaling for writes, zero-ETL , and cluster relocation.

On this put up, we share insights from considered one of our clients’ migration from DC2 to RA3 cases. The shopper, a big enterprise within the retail business, operated a 16-node dc2.8xlarge cluster for enterprise intelligence (BI) and ETL workloads. Going through rising knowledge volumes and disk capability limitations, they efficiently migrated to RA3 cases utilizing a Blue-Inexperienced deployment strategy, reaching improved ETL question efficiency and expanded storage capability whereas sustaining value effectivity.

Amazon Redshift structure varieties

Amazon Redshift provides two deployment choices: Provisioned mode, the place you select the occasion kind and variety of nodes and handle resizing as wanted, and Redshift Serverless, which mechanically provisions knowledge warehouse capability and intelligently scales the underlying assets. The next diagram compares these two structure varieties.

Provisioned clusters require you to find out cluster measurement prematurely, however you may optimize prices by buying Reserved Cases (RI) or scheduling pause and resume actions. Serverless mechanically provisions assets as wanted, with a pay-per-use mannequin the place you solely pay for compute assets consumed. Each companies help migration between one another and provide the identical options together with SQL, zero-ETL, and Federated Question capabilities. For particular pricing particulars, see Amazon Redshift pricing.

Provisioned clusters are appropriate for large-scale, predictable workloads and provide computerized scaling primarily based on queuing. Serverless gives management-free computerized scaling for variable workloads with AI-driven optimization that scales primarily based on workload complexity and knowledge volumes. For extra particulars, seek advice from Evaluating Amazon Redshift Serverless to an Amazon Redshift provisioned knowledge warehouse.

Buyer case research: Migration from DC2 cases

This part describes the shopper’s migration from Amazon Redshift DC2 to RA3 occasion varieties. The migration used a Blue-Inexperienced deployment strategy that minimized downtime whereas reaching each value optimization and efficiency enchancment.

The shopper’s workload had the next traits:

Use instances

The shopper had the next key use instances for his or her Amazon Redshift deployment:

1. Question by way of BI software throughout enterprise hours
   1. Excessive quantity of learn queries
   2. Peak entry throughout Mondays and starting of months
2. Knowledge processing in early morning
   1. Concentrated write queries for knowledge loading and transformation
3. Regular-state workload traits
   1. Run queries greater than 16 hours day by day

Necessities

The shopper had the next key necessities for his or her Amazon Redshift migration:

1. Efficiency
   1. Use auto-scaling (resembling concurrency scaling) throughout peak entry durations
2. Knowledge measurement
   1. Disk capability enlargement wanted
3. Price Administration
   1. Simple price range prediction and administration
   2. Make the most of low cost companies for long-term utilization
4. Compatibility
   1. Preserve compatibility with present purposes and BI instruments
   2. Keep away from endpoint adjustments
5. Availability
   1. Most downtime of 8 hours acceptable throughout migration
6. Community
   1. Don’t modify the prevailing 2-Availability Zone (AZ) subnet configuration
7. When emigrate
   1. To be carried out throughout low-load days and hours
   2. Deliberate downtime doable inside 8 hours

Key issues in system design, implementation, and operation included prolonged operation hours, ease of price range prediction and administration, value optimization via Reserved Cases (RI), and sustaining compatibility with present programs (avoiding endpoint adjustments). The shopper evaluated Amazon Redshift Serverless, which supplied enticing options resembling a pay-per-use mannequin, computerized scaling capabilities, and the potential for higher value efficiency for variable workloads. Whereas each Redshift Serverless and provisioned clusters might successfully help their workload patterns, the shopper selected the provisioned mannequin with RA3 nodes, leveraging their years of operational expertise with provisioned environments, present RI technique, and established capability planning strategy.

Options of RA3 occasion kind

Constructed on the AWS Nitro System, RA3 cases with managed storage undertake an structure that separates computing and storage, permitting impartial scaling and separate billing for every element. These cases use high-performance SSDs for decent knowledge and Amazon S3 for chilly knowledge, offering ease of use, cost-effective storage, and quick question efficiency. For extra particulars, seek advice from Amazon Redshift RA3 cases with managed storage.

Migration conditions

The shopper had the next migration conditions in place:

• The shopper used a Redshift cluster with 16 nodes of dc2.8xlarge configuration.
• The shopper selected a Blue-Inexperienced deployment strategy for migration, the place they’d restore from a snapshot to RA3 occasion kind, enabling fast rollback if needed.
• The shopper applied cluster switching and rollback via endpoint switching utilizing cluster identifier rotation.
• Moreover, to enhance efficiency with excessive concurrency, they transitioned the transaction isolation degree from SERIALIZABLE ISOLATION to SNAPSHOT ISOLATION.

Cluster migration strategies

There have been two migration choices accessible: Elastic Resize and Traditional Resize.

Amazon Redshift’s Traditional Resize performance had been enhanced, for resizing to RA3 occasion varieties, considerably lowering the write-unavailable interval. Primarily based on PoC testing, after initiating the resize, the cluster’s standing was modifying for 16 minutes earlier than it grew to become accessible. Primarily based on these outcomes, the shopper proceeded with the Traditional Resize strategy.

Cluster sizing

Sizing concerned figuring out the occasion kind and variety of nodes for the migration goal. Sizing factors thought of workload traits resembling CPU-intensive (queries utilizing excessive CPU), I/O-intensive (queries with excessive knowledge learn/write), or each.When migrating from DC2 occasion varieties, extra nodes could be required relying on workload necessities. Nodes had been added or eliminated primarily based on the computing necessities for needed question efficiency.

Evaluating configurations with comparable cluster prices by way of occasion measurement and rely, for a dc2.8xlarge 16-node cluster, the really helpful configuration was 8 nodes of ra3.16xlarge. The next was the fee comparability within the Tokyo Area:

1. Really useful: dc2.8xlarge 16-node cluster => ra3.16xlarge * 8-node cluster
   1. $97.52/h (6.095/h * 16 nodes) => $122.776/h (15.347/h * 8 nodes)
2. Price-focused: dc2.8xlarge 16-node cluster => ra3.16xlarge * 6-node cluster
   1. $97.52/h (6.095/h * 16 nodes) => $92.082/h (15.347/h * 6 nodes)

For this migration, the shopper proceeded with a cost-efficient 6-node ra3.16xlarge cluster to remain inside present price range constraints. Nonetheless, since this node rely might face throughput limitations throughout sure instances, they enabled concurrent scaling for the RA3 occasion kind to deal with spike entry.

Concurrency scaling gives as much as 1 hour of free credit per day for every energetic cluster, accumulating as much as 30 hours. On-demand utilization charges apply when exceeding this free tier.Whereas the shopper selected to implement concurrency scaling, Elastic Resize to briefly enhance nodes throughout peak hundreds was additionally thought of however rejected because of on-demand prices for extra nodes and the temporary disconnection interval throughout switching.

Managed storage value

RA3 cases use Redshift Managed Storage (RMS), which is charged at a set GB-month charge. The shopper’s roughly 2 TB of information required together with storage prices within the estimates. For pricing particulars, see Amazon Redshift pricing.

Migration step from DC2 to RA3

After creating an RA3 cluster from the DC2 cluster’s snapshot, the shopper swapped the cluster identifiers. The next diagram reveals this course of.

1. Take a snapshot of the present DC2 cluster.
2. Restore RA3 cluster from the snapshot with a distinct cluster identifier (Traditional Resize)
3. Swap the cluster identifiers between the present DC2 cluster and the brand new RA3 cluster.

If any points come up after the cluster swap, you may shortly roll again by returning the unique DC2 cluster to its unique cluster identifier.

Notice: Restore from a snapshot

Working the restore operation utilizing CLI instructions is really helpful to reduce operational errors and guarantee reproducibility. The next is a pattern command.

aws redshift restore-from-cluster-snapshot 
--cluster-identifier for-ra3-20250207 
--snapshot-identifier cm-cluster-for-ra3-20250207 
--cluster-subnet-group-name cm-cluster 
--vpc-security-group-ids sg-1234567a sg-2345678b sg-3456789c 
--cluster-parameter-group-name cm-cluster 
--node-type ra3.16xlarge 
--number-of-nodes 6 
--port 5439 
--no-publicly-accessible 
--enhanced-vpc-routing 
--availability-zone ap-northeast-1a 
--preferred-maintenance-window sat:17:00-sat:17:30 
--automated-snapshot-retention-period 14 
--iam-roles 'arn:aws:iam::123456789012:position/AmazonRedshift-CommandsAccessRole' 'arn:aws:iam::123456789012:position/AmazonRedshift-Spectrum' 
--maintenance-track-name present

Manufacturing migration period

The time required for the restore and traditional resize steps can fluctuate considerably relying on knowledge quantity and goal cluster specs. The shopper carried out a rehearsal beforehand to measure the precise required time.

Take a look at outcomes

Earlier than the manufacturing migration, the shopper created a check cluster by restoring a snapshot to the RA3 occasion kind. Whereas Redshift Take a look at Drive is usually helpful for workload testing, this buyer confronted distinctive constraints: enabling audit logging of their manufacturing cluster would require configuration adjustments, cluster restarts, and complicated approval processes below their strict change administration insurance policies. To deal with this, they developed a customized load testing software that captured workload patterns utilizing Amazon Redshift system views (SYS_QUERY_HISTORY and SYS_QUERY_TEXT), which keep 7 days of question historical past. The software replayed 55,755 historic queries with 50-way parallelism towards each DC2 and RA3 clusters, evaluating metrics together with question execution time, CPU utilization, and disk I/O. Question outcome caching was disabled throughout testing to make sure correct comparisons.

BI question efficiency

BI queries had been examined utilizing the customized load testing software. The outcomes signify the common execution time from 15 check runs of 55,755 queries executed with 50-way parallelism. With out concurrency scaling, the dc2.8xlarge 16-node cluster averaged 45.82 seconds per question, whereas the ra3.16xlarge 6-node cluster averaged 91.30 seconds. This indicated that RA3 cases confirmed longer execution instances for brief and medium queries in a direct migration with out optimizations. Nonetheless, enabling concurrency scaling improved RA3 efficiency progressively. With concurrency scaling enabled at most 2 clusters, the ra3.16xlarge 6-node cluster achieved a mean of 72.48 seconds per question, a 21% enchancment over the non-scaled configuration.

ETL question efficiency comparability

For long-running ETL queries (execution time larger than 10 minutes), the RA3 cluster demonstrated higher efficiency than DC2. These outcomes represented a direct migration of the shopper’s workload with no optimizations utilized.

• For the Giant-scale knowledge load workload 1, the ra3.16xlarge cluster accomplished the question 28% sooner than the dc2.8xlarge cluster (41 minutes vs. 57 minutes).
• For the Complicated transformation workload 1, the ra3.16xlarge cluster was 23% sooner (1 hour 1 minute vs. 1 hour 20 minutes).

These outcomes indicated that the RA3 node kind was extra performant for time-intensive knowledge loading and transformation duties. The upper CPU utilization values for RA3 recommended simpler compute useful resource utilization.

Efficiency tuning

Primarily based on the check outcomes, the shopper recognized that RA3 confirmed longer execution instances for brief and medium BI queries however sooner efficiency for long-running ETL queries in comparison with DC2. To optimize total efficiency, they centered on figuring out sluggish queries and ceaselessly referenced tables, prioritizing optimizations with the very best affect.

Efficiency tuning technique

The shopper thought of a number of optimization methods to leverage RA3’s architectural benefits. One key technique concerned pre-processing ad-hoc brief and medium question workloads throughout low-load durations, creating pre-processed tables or materialized views for queries that repeatedly carried out joins, aggregations, filters, and projections. RA3’s separated compute and storage structure, with cost-effective large-scale storage, supported this strategy.

Changing common views to materialized views

Evaluation of sluggish queries revealed the usage of joins in views, and ceaselessly referenced tables had been being accessed a number of instances via these views. As a countermeasure, the shopper changed ceaselessly used common views with materialized views, eradicating pointless knowledge ranges and redundant columns.

Amazon Redshift helps incremental updates of materialized view contents by way of the REFRESH MATERIALIZED VIEW command, enabling environment friendly knowledge updates.

Materialized views and question rewrite

By changing common views to materialized views, present queries could also be mechanically optimized via the “question rewrite” function offered by the question planner. For extra particulars, seek advice from “Automated question rewriting to make use of materialized views“.

Automated tuning with AutoMV

On the DC2 cluster, disk utilization persistently exceeded 80%, which disabled the AutoMV function because of inadequate disk area. With RA3’s expanded storage, computerized tuning via AutoMV grew to become doable, resulting in additional efficiency enhancements. For extra particulars about AutoMV, seek advice from Automated materialized views.

Efficiency tuning outcomes

After making use of these optimizations, the shopper achieved the next outcomes:

• Maintained present efficiency whereas controlling value will increase
• Achieved larger CPU utilization whereas sustaining throughput
• Enhanced dynamic throughput throughout peak load durations utilizing concurrency scaling’s computerized scaling

Conclusion

On this put up, you discovered how a big retail enterprise efficiently migrated from Amazon Redshift DC2 to RA3 cases. The Blue-Inexperienced deployment strategy enabled a protected migration with fast rollback functionality, whereas the separated compute and storage structure of RA3 offered flexibility to deal with rising knowledge volumes. Though RA3 confirmed completely different efficiency traits for brief BI queries in comparison with DC2, the shopper achieved vital enhancements in long-running ETL question efficiency (as much as 28% sooner for knowledge hundreds and 23% sooner for complicated transformations). By leveraging RA3-specific options resembling materialized views and AutoMV, they optimized total question efficiency whereas sustaining value effectivity via Reserved Cases and concurrency scaling.

To proceed your RA3 migration journey, see Greatest practices for upgrading from Amazon Redshift DC2 to RA3 and Amazon Redshift Serverless and Resize Amazon Redshift from DC2 to RA3 with minimal or no downtime for extra steering and finest practices.

Concerning the authors