MongoDB Aggregation $out

MongoDB’s Aggregation Pipeline is a powerful tool for data processing that allows users to perform a series of operations to filter, transform, and analyze data. One key stage in the aggregation pipeline is the $out stage, which enables users to write the output of the pipeline to a new collection.

In this article, we’ll explore the $out stage in the MongoDB aggregation pipeline along with its syntax, use cases, and examples of how it can be used efficiently to manage and analyze data.

$out Stage (aggregation)

The $out stage is a crucial component of the Aggregation Pipeline, it takes the documents returned by the aggregation pipeline and writes them to a specified collection. The returned documents can be written into a new collection by specifying the desired name of the new collection in the $out stage

$out stage must be the last stage in the aggregation pipeline. This stage is beneficial when we want to persist the aggregated data for further analysis, and reporting or to serve as a basis for subsequent queries.

$out Stage Syntax

The syntax for the $out stage is given below:

{ $out: { db: "<output-db>", coll: "<output-collection>" } }

This syntax allows users to write the results of an aggregation pipeline to a specified collection in a specified database.

$out Example

Below we have provided a step-by-step guide, on how to use $out stage in the aggregation pipeline.

Step 1: In this step, we have used the sample_supplies database. The aggregation pipeline that is shown below will group documents by the storeLocation field and calculate the total items sold at each location. Then the queried results are written to a new collection named “store_items_totals“.

Step1: SHOWING THE AGGREGATION PIPELINE

Step 2: After executing the query we come to know that a new collection has been created in the database. This can be confirmed by writing the command “show collections“. From the below picture, we can infer that “store_items_totals” is created as a new collection in the sample_suppies database.

Step 2: LIST OF COLLECTIONS IN THE DATABASE

Step 3: To check whether the query is written successfully in the new collection we run the command find( ) to display the items in the collections.

Step 3: DISPLAYING THE STORE_ITEMS_TOTALS COLLECTION

Step 4: Finally we can infer from the above image that our query is written successfully on a new collection.

Use Cases of $out Stage

The $out stage is employed in various scenarios, including:

1. Aggregating and summarizing large datasets to create compact and summary collections for improved query performance.
2. Storing pre-aggregated data in separate collections to build a data warehouse for analytical purposes.
3. Achieving historical data by creating aggregated collections and helping to manage the size of the operational dataset.
4. Generating periodic reports by running aggregation pipelines and storing the results in dedicated collections for quick retrieval.

MongoDB $out vs $merge

There is another useful stage that can be used in the MongoDB that works similarly to the $out stage, But the only difference in the usage of both is that $merge does not form any new collection, but stores the query in the existing collection itself.

$out	$merge
It writes the query to the new collection	It writes the query to the existing collection
It overwrites the existing data	It merges with the existing data
It is useful for storing results	It is useful for updating or merging the data

Important Takeaways About $out Stage

• Using the $out stage for querying database efficiently developers can write/store query results in a new collection.
• If the specified collection does not exist, the $out operator creates a new collection.
• If the collection specified by the $out operation already exists, then after aggregation completion, the $out stage replaces the original collection with a new one.
• It can not write results into a capped collection.
• Sharded collection can not be specified as output collection in the $out stage.
• Allows deeper exploration and analysis of data while enabling the performance of operations and insightful conclusions.