How to Create a Search Index with Dynamic Field Mapping in MongoDB

Dynamic mapping in MongoDB Atlas Search is a powerful feature that automatically indexes all supported field types in a collection and making it ideal for applications with evolving data models. This type of mapping is particularly useful when the schema is unknown or changes regularly. Dynamic mappings assign fields automatically when new data is inserted, simplifying the setup for diverse data structures.

In this article, We will learn about, What is Dynamic Mapping, Need of Dynamic Field Mapping for Search Indexes, When to Use Dynamic Mapping and how to create Search Index for Dynamic Mapping Using Mongosh or MongoDB Driver detail.

What is Dynamic Mapping?

• Dynamic mappings in MongoDB Atlas Search can automatically index all supported field types in a collection. This can be done by defining the collections (field mapping) as dynamic. 
• Dynamic mappings can be used when the schema is unknown or changes regularly, or when experimenting with Atlas Search.
• Dynamic mappings assign fields automatically when new data is inserted. we can use dynamic mappings to configure an entire index or specify individual fields, such as fields of type document, to be dynamically mapped.
• It is the fastest and easiest way to get started with searching data. We can use dynamic mapping if the data schema constantly changes, when the exact structure is unknown, or when the collection contains a mix of document types with various field names.

Note: All the fields in a MongoDB collection are Indexed (except boolean, objectID and timestamps).

Why Use Dynamic Field Mapping for Search Indexes?

• Dynamic field mapping allows the index to adapt to changes in the document structure without manual intervention.
• It solves the indexing process by automatically incorporating new fields as they appear in documents.
• Dynamic field mapping is great for apps that have changing data structures..

When to Use Dynamic Mapping?

• When data schema is constantly changing as dynamic mapping adapts automatically to new fields, saving us to do manual updates.
• When there is a mix of document types because it has a simplified setup for diverse data, automatically handles different field structures.
• When we need a quick search setup where we can get started quickly without defining every field mapping.

Dynamic Field Mapping Strategies

1. Default Dynamic Field Mapping: MongoDB’s default dynamic field mapping automatically indexes all fields in the document.

2. Custom Dynamic Field Mapping: Define specific fields for dynamic mapping using the $text index type. We can create a custom dynamic index on `authors` collection and view the search index created using the command “db.authors.getIndexes()“.

Prerequisites

1. We should know how to connect MongoDB using a connection String when working with MongoDB driver (MongoDB version 4.2 or higher).
2. A Collection to create a Atlas Search Index.

Steps for creating search index with dynamic field mapping In MongoDB ATLAS:

1. Connect to MongoDB By logging into our MongoDB Atlas Database.
2. Select project your desired project from the Projects menu in the navigation bar.
3. Click your cluster’s name.
4. Click the Atlas Search tab.
5. Click Create Search Index. As shown in Fig: 1.1.

Fig: 1.1

6. Select an Atlas Search Configuration Method and click Next.

a) For a guided experience, select the Atlas Search Visual Editor.

b) To edit the raw index definition, select the Atlas Search JSON Editor.

7. Enter the Index Name, In the Index Name field, enter default. (Note: If you name your index default, we don’t need to specify an index parameter when using the $search pipeline stage. Otherwise, we must specify the index name using the index parameter.) and then Click Next.

a) Visual Editor

Click Next. Review the `default` index definition for the books collection.

b) JSON Editor

Click Next. Review the index definition. Your index definition should look similar to the following (Fig 1.2) and Click Next.

8. Set the Database and Collection. As per here, we are considering “Aggregation_MDB” database and “books” collection within it.

9. Specify an index definition by use the Atlas Search Visual Editor or the Atlas Search JSON Editor in the Atlas user interface to create the index. For Example: We define index definition as dynamic for the books collection.

Fig: 1.2

10. Click Next

11. Review the index definition. Our index definition should look similar to the following (Fig: 1.3).

Fig: 1.3

12. Click Save Changes.

13. Click Create Search Index.

14. Close the You’re All Set! Modal Window.

15. Click the Close button. Wait for the index to finish building. The index should take about one minute to build. While it is building, the Status column reads Build in Progress. When it is finished building, the Status column reads Active.

Search Index with dynamic mapping for books collection

Creating Search Index for Dynamic Mapping Using Mongosh or MongoDB Driver:

1. Create a file named `create-index.js`.
2. Copy and paste the following code into the `create-index.js` file.

const { MongoClient } = require("mongodb");

// connect to your Atlas deployment
const uri =  
  "<connection-string>";

const client = new MongoClient(uri);

async function run() {
  try {

    // set namespace
    const database = client.db("Aggregation_MDB");
    const collection = database.collection("books");
    
    // define your Atlas Search index
    const index = {
        name: "default",
        definition: {
            /* search index definition fields */
            "mappings": {
                "dynamic": true
            }
        }
    }

    // run the helper method
    const result = await collection.createSearchIndex(index);
    console.log(result);
  } finally {
    await client.close();
  }
}

run().catch(console.dir);

This code creates a search index named “default” in the “books” collection of a MongoDB Atlas database. The key point is `mappings: { dynamic: true }`, which enables dynamic field mapping for the search index. This lets Atlas Search automatically detect and index supported fields in your documents. The `collection.createSearchIndex(index)` method is called. This method takes the defined search index object (`index`) as an argument and creates the search index within the specified collection.

3. Replace the `<connection-string>` in the query and then save the file. Ensure that your connection string includes your database user’s credentials.

4. Create the index by run the following command and the output will be name of index field, which is `default` for this particular example.

Output

Advantages of Using Dynamic Mapping

• Dynamic field mapping allows indexes to adapt to the evolving structure of the document without direct involvement, making it suitable for application with dynamic schema.
• It simple the indexing process by automatically detecting and indexing the new fields that appear in the document, making it easy to get started with running search queries.
• Since dynamic mapping adapts automatically to the changing schema, it reduces the need for constantly updating the search index definition.

Disadvantages of Using Dynamic Mapping

• Limited Control: Dynamic mapping in MongoDB offers less control over the indexing process compared to static mapping.
  
• Irrelevant indexes: Large number fields are indexed in the document regardless of whether they are relevant or not.
• Performance Considerations: It added extra processing time by determining the data types during indexing and querying.
• Extra Space: It takes more disk space and might negatively impacts the cluster performance.
• Potential for Inaccuracy: Since dynamic mapping automatically detects field types, there’s a chance it might misinterpret certain data types, leading to less accurate search results.

Best Practices for Using Dynamic Field Mapping

• Monitor Performance: Use MongoDB profile to monitor and optimize the index performance.
• Analyze Schema Changes: Even with dynamic mapping, analyze any major schema changes to ensure they’re being indexed appropriately.
• Thoughtfully Choose Index Fields: To avoid necessary overheads, select relevant fields for indexing.
• Levages compound indexes: Consider creating compound indexes to leverage the search performance.
• Optimize query performance: Optimize query performance by denormalizing data across multiple documents and creating indexes to support queries.

Comparsion between Dyanmic and Static Mapping

	Dynamic Mapping	Static mapping
1	It is suitable for evolving schema.	It Is less adaptable to dynamic schema.
2	Less control over the indexing process by automatically indexing all the fields in the documents.	It has Granular control over the indexing process by allowing to explicitly define which fields to be indexed.
3	It automatically adapts to the changes in the documents structure.	It requires manually indexing the relevant fields of the document.
4	It requires more disk space, as it indexes all the fields regardless of whether there are relevant or not.	It can be optimized for better performance by improving the size and structure of the search index for the relevant fields in the doscuments.

Conclusion

Overall, Dynamic mapping in MongoDB Atlas Search provides a flexible and efficient approach to indexing data, particularly for applications with evolving schemas. It eliminates the need for manual updates and fast the indexing process by automatically adapting to changes in the document structure. Along with its advantages, dynamic mapping offers less control over the indexing process compared to static mapping.