Live Migrate (Pull) a Replica Set into Atlas

Migration Path

Atlas live migration (pull) supports the following migration paths:

Pre-Migration Validation

Before starting the pull live migration procedure, Atlas runs validation checks on the source and target clusters.

• The source cluster is a replica set.

  If the source cluster is a standalone, convert the standalone to a replica set first before using the pull-type live migration.

• The target Atlas cluster is a replica set.

Source Cluster Security

Various built-in roles provide sufficient privileges. For example:

• For source clusters running MongoDB 3.6 or later a user must have the readAnyDatabase, the clusterMonitor, and the backup roles.

  To verify that the database user that will run the live migration process has these roles, run the db.getUser() command on the admin database.

  use admin
  db.getUser("admin")
  {
    "_id" : "admin.admin",
    "user" : "admin",
    "db" : "admin",
    "roles" : [
      {
        "role" : "backup",
        "db" : "admin"
      },
      {
        "role" : "clusterMonitor",
        "db" : "admin"
      }
      {
        "role" : "readAnyDatabase",
        "db" : "admin"
      }
    ]
  } ...

  In addition, the database user from your source cluster must have the role to read the oplog on your admin database. To learn more, see Oplog Access.

• For source clusters running MongoDB 3.4 a user must have, at a minimum, both clusterMonitor and readAnyDatabase roles. For example:

   use admin
   db.createUser(
     {
       user: "mySourceUser",
       pwd: "mySourceP@$$word",
       roles: [ "clusterMonitor", "readAnyDatabase" ]
     }
   )

• For source clusters running MongoDB 3.2 a user must have, at a minimum, both clusterManager and readAnyDatabase roles, as well as read access on the local database. This requires a custom role, which you can create with the following commands:

    use admin
    db.createRole(
         {
           role: "migrate",
           privileges: [
             { resource: { db: "local", collection: "" }, actions: [ "find" ] }
           ],
           roles: ["readAnyDatabase", "clusterManager"]
         }
       )
     db.createUser(
         {
           user: "mySourceUser",
           pwd: "mySourceP@$$word",
           roles: [ "migrate" ]
         }
       )

• For source clusters running MongoDB 2.6 or 3.0 a user must have, at a minimum, the readAnyDatabase role. For example:

    use admin
       db.createUser(
         {
           user: "mySourceUser",
           pwd: "mySourceP@$$word",
           roles: [ "readAnyDatabase" ]
         }
       )

Specify the user name and password to Atlas when prompted by the live migration procedure.

Atlas only supports SCRAM for connecting to source clusters that enforce authentication.

If the source cluster uses a different authentication mechanism to connect, you can use mongomirror to migrate data from the source cluster to the target Atlas cluster.

Network Encryption

During pull live migrations, if the source cluster does not use TLS encryption for its data, the traffic from the source cluster to Atlas is not encrypted. Determine if this is acceptable before you start a pull live migration procedure.

Target Cluster Configuration

When configuring the target cluster, consider the following:

• The live migration process streams data through a MongoDB-managed application server. Each server runs on infrastructure hosted in the nearest region to the source cluster. The following regions are available:

  Europe

  • Frankfurt
  • Ireland
  • London

  Americas

  • Eastern US
  • Western US

  APAC

  • Mumbai
  • Singapore
  • Sydney
  • Tokyo

• Due to network latency, the live migration process may not be able to keep up with a source cluster that has an extremely heavy write load. In this situation, you can still migrate directly from the source cluster by pointing the mongomirror tool to the destination Atlas cluster.
• The live migration process may not be able to keep up with a source cluster whose write workload is greater than what can be transferred and applied to the destination cluster. You may need to scale the destination cluster up to a tier with more processing power, bandwidth or disk IO.

• The target Atlas cluster must be a replica set.
• You can't select an M0 (Free Tier) or M2/M5 shared-tier cluster as the target cluster for live migration.
• Do not change the featureCompatibilityVersion flag while Atlas live migration is running.

Database Users and Roles

Atlas doesn't migrate any user or role data to the target cluster.

If the source cluster enforced authentication, you must re-create the credentials used by your applications on the target Atlas cluster. Atlas uses SCRAM for user authentication. To learn more, see Configure Database Users.

Avoid Namespace Changes

Don't make any namespace changes during the migration process, such as using the renameCollection command or executing an aggregation pipeline that includes the $out aggregation stage.

Avoid Elections

The live migration process makes a best attempt to continue a migration during temporary network interruptions and elections on the source or target clusters. However, these events might cause the live migration process to fail. If the live migration process can't recover automatically, restart it from the beginning.

Rolling Restarts

After the migration process completes, your cluster restarts each of its members one at a time. This is called a rolling restart, and as a consequence, a failover will occur on the primary. To ensure a smooth migration, consider running a Test Failover procedure before migrating your data to the target cluster.

Pre-Migration Checklist

Before starting the import process:

• If you don't already have a destination cluster, create a new Atlas deployment and configure it as needed. For complete documentation on creating an Atlas cluster, see Create a Cluster.

• After your Atlas cluster is deployed, ensure that you can connect to it from all client hardware where your applications run. Testing your connection string helps ensure that your data migration process can complete with minimal downtime.

  1. Download and install mongosh on a representative client machine, if you don't already have it.
  2. Connect to your destination cluster using the connection string from the Atlas UI. For more information, see Connect via mongosh.

  Once you have verified your connectivity to your target cluster, start the import procedure.

Procedure