Live Migrate Your Sharded Cluster to Atlas¶

Migration Path¶

Atlas live migration supports the following migration paths:

Pre-Migration Validation¶

Atlas performs a number of validation checks on the source and target cluster before starting the Live Migration procedure.

• The source cluster must be a sharded cluster.

  If the source is replica set cluster, use Live Migration to migrate the cluster to a Atlas replica set first, then scale your cluster to a sharded cluster.

  If the source is a standalone, convert the standalone to a replica set first before using Live Migration. Then scale your cluster to a sharded cluster.

• The source cluster must use CSRS (Config Server Replica Sets). See Replica Set Config Servers
• Atlas must have connectivity to the hostname and port of each node in the source cluster.
• Atlas must be able to stop and start the Sharded Cluster Balancer on the source cluster.

• The source cluster has the same feature compatibility version and major MongoDB version as the target cluster. The major MongoDB verison is the first two digits of the full version, such as 4.0.x, 4.2.x, or 4.4.x.

  To check the feature compatibility version of a host in the source cluster, run the following command from mongosh:

  db.runCommand( { getParameter : 1, "featureCompatibilityVersion" : 1 } )

  Use the setFeatureCompatibilityVersion database command to set the featureCompatibilityVersion flag as needed.

• The target Atlas cluster must be a sharded cluster with the same number of shards as the source sharded cluster.

Source Cluster Security¶

Atlas only supports SCRAM for connecting to source clusters enforcing authentication.

If the source cluster enforces authentication, create a user with the following privileges:

• Read all databases and collections. The readAnyDatabase role on the admin database covers this requirement.
• Read the oplog.

Various built-in roles provide sufficient privileges. For example:

• For source clusters running MongoDB version 3.4 or later a user must have, at a minimum, both clusterMonitor and readAnyDatabase roles. For example:

  use admin
  db.createUser(
    {
      user: "mySourceUser",
      pwd: "mySourceP@$$word",
      roles: [ "clusterMonitor", "readAnyDatabase" ]
    }
  )

• For source clusters running MongoDB version 3.2 a user must have, at a minimum, both clusterManager and readAnyDatabase roles, as well as read access on the local database. This requires a custom role, which you can create with the following commands:

  use admin
  db.createRole(
       {
         role: "migrate",
         privileges: [
           { resource: { db: "local", collection: "" }, actions: [ "find" ] }
         ],
         roles: ["readAnyDatabase", "clusterManager"]
       }
     )
   db.createUser(
       {
         user: "mySourceUser",
         pwd: "mySourceP@$$word",
         roles: [ "migrate" ]
       }
     )

• For source clusters running MongoDB version 2.6 or 3.0 a user must have, at a minimum, the readAnyDatabase role. For example:

  use admin
     db.createUser(
       {
         user: "mySourceUser",
         pwd: "mySourceP@$$word",
         roles: [ "readAnyDatabase" ]
       }
     )

Specify the user name and password to Atlas when prompted by the Live Migration procedure.

Source Cluster Balancer¶

Atlas Live Migration stops the sharded cluster balancer on the source cluster at the start of the procedure, and starts the balancer at the end of the procedure.

If you cancel Live Migration, Atlas restarts the balancer on the source cluster.

Target Cluster Configuration¶

When configuring the target Atlas cluster, consider the following:

• The live migration process streams data through a MongoDB-managed application server. Each server runs on infrastructure hosted in the nearest region to the source cluster. The following regions are available:

  Europe

  • Frankfurt
  • Ireland
  • London

  Americas

  • Eastern US
  • Western US

  APAC

  • Mumbai
  • Singapore
  • Sydney
  • Tokyo

• Due to network latency, the live migration process may not be able to keep up with a source cluster that has an extremely heavy write load. In this situation, you can still migrate directly from the source cluster by pointing the mongomirror tool to the destination Atlas cluster.
• The live migration process may not be able to keep up with a source cluster whose write workload is greater than what can be transferred and applied to the destination cluster. You may need to scale the destination cluster up to a tier with more processing power, bandwidth or disk IO.

• You cannot target a Global Cluster as the target for Live Migration.

  Important

  You cannot modify the target Atlas cluster once you start the Live Migration procedure. If you need to scale up the target cluster, first cancel the Live Migration procedure, then scale up the cluster and restart the Live Migration procedure.

Database Users and Roles¶

Atlas does not migrate any user or role data to the target cluster.

If the source cluster enforced authentication, you must re-create the credentials used by your applications on the target Atlas cluster. Atlas uses SCRAM for user authentication. See Configure Database Users for a tutorial on creating database users in Atlas.

Canceling Live Migration¶

You can cancel the process at any time by clicking Cancel. Atlas displays the Sharded Cluster Live Import in Progress message for the target cluster until the cluster is ready for normal access.

If you cancel the Live Migration procedure before completion, Atlas does not remove any data migrated up to that point. If you restart the Live Migration procedure using the same Atlas cluster as the target cluster, Atlas wipes all data from the cluster.

Testing the Target Cluster¶

You may want to migrate data to your target cluster, then stop the migration process and test your target cluster while leaving the source cluster running and serving data to your applications.

To test your target cluster with production data, follow the migration procedure as far as step 2. When you're ready to perform the complete migration process, skip step 2 and proceed to step 3.

Avoid Namespace Changes¶

Don't make any namespace changes during the migration process, such as using the renameCollection command or executing an aggregation pipeline that includes the $out aggregation stage.

Avoid Elections¶

The Live Migration process makes a best attempt to continue a migration during temporary network interruptions and elections on the source or target clusters. However, these events might cause the Live Migration process to fail. If the Live Migration process can't recover automatically, restart it from the beginning.

Target Cluster Network Access¶

During Live Migration, the mongos processes on the target cluster are shut down and cluster connectivity via the mongos servers is suspended. The mongos processes restart automatically once migration is complete.

Rolling Restarts¶

After the migration process completes, your cluster restarts each of its members one at a time. This is called a rolling restart, and as a consequence, a failover will occur on the primary. To ensure a smooth migration, consider running a Test Failover procedure before migrating your data to the target cluster.

Pre-Migration Checklist¶

Before starting the import process:

• If you don't already have a destination cluster, create a new Atlas deployment and configure it as needed. For complete documentation on creating an Atlas cluster, see Create a New Cluster.

• After your Atlas cluster is deployed, ensure that you can connect to it from all client hardware where your applications run. Testing your connection string helps ensure that your data migration process can complete with minimal downtime.

  1. Download and install mongosh on a representative client machine, if you don't already have it.
  2. Connect to your destination cluster using the connection string from the Atlas UI. For more information, see Connect via mongosh.

  Once you have verified your connectivity to your target cluster, start the import procedure.

Procedure¶