Return Stored Source Fields

On this page

Syntax

Behavior
Sample Use
Examples

Note

The Atlas Search index storedSource option and $search returnStoredSource option are in preview, but can be used in production applications. If there are any syntax or behavior changes between the preview stage and general availability (GA), we will proactively communicate before introducing any breaking changes. The MongoDB Cloud Support team will help troubleshoot any issues related to using this feature as part of your contract.

If you configured storedSource option in the index definition for a collection, you can retrieve the stored fields only using the returnStoredSource boolean option in your Atlas Search queries.

By default, Atlas Search performs full document lookup implicitly on the backend database after Atlas Search matches documents for the query. This lookup could significantly degrade performance for subsequent aggregation pipeline stages that fetch all matched dataset from $search stage (such as $sort, $group) or filter a large part of it (such as $match, $skip). If you configured the storedSource option, you can avoid full document lookup on the database until further down the pipeline and use the returnStoredSource option to retrieve only parts of the documents directly stored on Atlas Search. This allows you to perform most database-side filtering operations on documents with a minimum number of fields and avoid full document lookup until further down the aggregation pipeline.

Syntax

returnStoredSource has the following syntax in your queries:

{
  $search: {
    "<operator>": {
      <operator-specification>
    },
    "returnStoredSource": true | false // optional, defaults to "false"
  }
}

To learn more about query syntax, see $search.

Behavior

The returnStoredSource boolean option specifies whether Atlas Search must perform a full document lookup on the database or return the stored fields directly from Atlas Search. You can use the returnStoredSource option only if your index definition includes the configuration for storing fields on Atlas Search. To learn more about storing fields on Atlas Search, see Review Atlas Search Index Syntax and Define Stored Source Fields in Your Atlas Search Index.

You can set one of the following values for the returnStoredSource option:

true - to return only stored source fields directly from Atlas Search
false - to do an implicit full document lookup on the backend database (default)

If you run Atlas Search queries with the returnStoredSource boolean option set to true:

Atlas Search returns an empty document if the document doesn't include the fields configured for storing.
Atlas Search returns errors if the index definition doesn't include the Stored Source configuration.
Atlas Search might return stale data due to a replication lag.
Atlas Search might return duplicate data on sharded clusters.

If you perform a high volume and rate of data insert and update operations for your collection on the backend database, Atlas Search might return stale data because the data stored on mongot might not be current due to a replication lag. You can view the approximate number of milliseconds that Atlas Search is behind in replicating changes from the oplog of mongod in the Atlas UI. To learn more, see Review Atlas Search Metrics.

If there are orphaned documents during chunk migration, Atlas Search might return duplicate documents for queries against sharded cluster.

Sample Use

If your $search stage discards a lot of the results and you need to perform implicit document lookup on your database, we recommend using the returnStoredSource option. You can store fields required for sorting or filtering and use returnStoredSource option at query time to perform the following actions:

Intermediate operations on partial documents returned by Atlas Search
$lookup at the end of the pipeline if full documents are needed.

Important

For efficiency, configure only a minimum number of fields for storage on Atlas Search. Use this option if your documents are large enough to cause issues during lookup.

Examples

The examples in this section use the sample_mflix.movies collection. The examples show how to do a sort or match on the documents that Atlas Search returns after the $search stage and then lookup documents on the database.

← Explain Timing Breakdown Count Search Results →