Development Notes
  • Introduction
  • Programming Langauges
    • Java
      • Cache
      • Java Fundamentals
      • Multithreading & Concurrency
      • Spring Boot
        • Spring Security
        • Development tips
      • ORM
        • Mybatis
      • Implementation & Testing
    • Node.js
      • Asynchronous Execution
      • Node.js Notes
    • Python
      • Memo
  • Data Structure & Algorithm
  • Database
  • Design Pattern
  • AWS Notes
    • Services
      • API Gateway
      • CloudHSM
      • Compute & Load Balancing
        • Auto Scaling Group
        • EC2
        • ECS
        • ELB
        • Lambda
      • Data Engineering
        • Athena
        • Batch
        • EMR
        • IoT
        • Kinesis
        • Video Streaming
        • Quicksight
      • Deployment
        • CloudFormation
        • Code Deploy
        • Elastic Beanstalk
        • OpsWorks
        • SAM
        • SSM
      • ElasticSearch
      • Identity & Federation
        • Directory Service
        • IAM
        • Organizations
        • Resource Access Manager (RAM)
        • SSO
        • STS
      • KMS
      • Management Tools
        • Catalog
        • CloudTrail
        • CloudWatch
        • Config
        • Cost Allocation Tags
        • GuardDuty
        • Savings Plans
        • Trusted Advisor
        • X-Ray
      • Migration
        • Cloud Migration: The 6R
        • Disaster Recovery
        • DMS
        • VM Migrations
      • Networking
        • ACM
        • CloudFront
        • Direct Connect
        • EIP & ENI
        • Network Security
        • PrivateLink
        • Route53
        • VPC
        • VPN
      • Service Commnucation
        • Amazon MQ
        • SNS
        • SQS
        • Step Functions
        • SWF
      • Storage
        • Aurora
        • DynamoDB
        • EBS
        • EFS
        • ElastiCache
        • RDS
        • Redshift
        • S3
        • Storage Gateway
      • Other Services
        • Alexa for Business, Lex, Connect
        • AppStream 2.0
        • CloudSearch
        • Comprehend
        • Data Tools
        • Elastic Transcoder
        • Mechanical Turk
        • Rekognition
        • WorkDocs
        • WorkSpaces
    • Well Architect Framework
      • Security
      • Reliability
      • Performance Effeciency
      • Cost Optimization
      • Operational Excellence
    • Labs
      • Webserver Implementation
      • ELB Implementation
      • Auto-scaling Implementation
      • A 3-tier Architecture In VPC
  • Architecture
    • Security
  • Spark
    • Memo
  • Conference Notes
    • Notes of JCConf 2017
  • AI Notes
Powered by GitBook
On this page

Was this helpful?

  1. AWS Notes
  2. Services
  3. Migration

DMS

PreviousDisaster RecoveryNextVM Migrations

Last updated 4 years ago

Was this helpful?

Introduction

  • Migrates databases to AWS and remains available during migration

  • Supports:

    • Homogeneous migrations: ex. Oracle to Oracle

    • Heterogeneous migrations: ex. Microsofr SQL Server to Aurora

  • Continuous Data Replication using

  • Must create an EC2 instance to perform the replication tasks

  • Works over VPC Peering / VPN / Direct Connect

  • Supports loads: full load / full load + CDC / CDC only

Feature

  • Sources:

    • On-premise and EC2 instances databases: Oracle, MS SQL Server, MySQL, MariaDB, PostgreSQL, SAP, DB2, Mongo DB.

    • Azure: Azure SQL Database

    • RDS: all including Aurora

    • S3 (csv files as input)

  • Targets:

    • On-premise and EC2 instances databases: Oracle, MS SQL Server, MySQL, MariaDB, PostgreSQL, SAP

    • RDS

    • Redshift

    • DynamoDB

    • DocumentDB

    • S3

    • ElasticSearch

    • Kinesis Data Steams

  • Schema Conversion Tool (SCT)

    • Converts Database schema from one engine to another

    • OLTP: SQL Server / Oracle to MySQL / PostgreSQL / Aurora

    • OLAP: Teradata / Oracle to Redshift

    • can filter data by column value

    • can filter columns according by prefix

    • can add prefix to table name

  • For Oracle:

    • As source: supports TDE for the source using "BinaryReader"

    • As target: supports BLOBs in tables that have a primary key, and TDE

  • For ElasticSearch:

    • As source: NOT SUPPORTED

    • As target: possible to migrate through DMS from a relational DB

      • Calculate the number of queue slots required for the ES bulk request as a product of the number of indexes, shards, and replicas.

      • Adjust DMS subtask and thread parameters accordingly.

  • Can combine with Snowball Edge

    • Steps:

      • Use AWS Schema Conversion Tool to extract the data locally and move it into Snowball Edge

      • Ship Snowball edge back to AWS

      • Data is loaded into S3 bucket

      • DMS takes the S3 files and migrates data to the target data store. With CDC, updates are written to S3 bucket and then applied to the target data store

CDC
Customing tasks
Exception with code 429 "Too many requests,"