How to Generate Sitemap in a MERN Application ?
Last Updated :
10 Nov, 2023
Generate Sitemap in a MERN Application refers to creating a file that lists the pages, videos, and other files on your website, as well as their relationships. Search engines (such as Google, Duckduckgo, Bing, and others) use this file to help them crawl your site more efficiently.
Prerequisites:
We can generate the sitemaps in the MERN application for these cases:
Steps to Generate Static Sitemap in MERN Application :
If the website you are working on has a fixed number of URLs, for example, your portfolio. You can generate a sitemap.xml file with the help of any online tool and place the folder in the src folder.
Step 1: Visit https://www.xml-sitemaps.com/ and generate the sitemap.xml file.
Step 2: Move the file into the public folder of your react app.
Project Structure:
Step 3: Verify the changes by visiting https://ahampriyanshu.com/sitemap.xml.
Step 4: Finally, Add the Sitemap to your robots.txt file
User-agent: *
Allow: /
Sitemap: https://baseurl/sitemap.xml
Steps to Generate Dynamic Sitemap :
So far we have discussed creating a sitemap with static URLs. But what if the number of URLs and the content of existing URLs change from time to time. Suppose we are creating a GFG clone. So, our sitemap should contain URLs of all the articles and the important pages.
For this, we will send a sitemap file from our backend by first looping through all the required records from our database. After this, we will manually add the URLs of other important pages like about, contact, etc.
Step 1: For demonstration purposes, We are considering a basic project with three files.
Javascript
const express = require( "express" ),
mongoose = require( "mongoose" ),
todoRouter = require( "./routers/todoRouter" ),
app = express();
app.use( "/" , todoRouter);
const port = 3000,
mongoose
.connect(db)
.then(conn => {
console.log(`${db} connected`);
});
app.listen(port, () => console.log(
`Server listening on ${port}`));
|
Javascript
const mongoose = require( "mongoose" ),
todoSchema = new mongoose.Schema(
{
title: { type: String, unique: true },
},
{ timestamps: true }
);
module.exports = mongoose.model( "Todo" , todoSchema);
|
Javascript
const express = require( "express" ),
router = express.Router();
module.exports = router;
|
Step 2: Install the’ sitemap’ package to stream the sitemap buffer and write its data.
npm i sitemap
Step 3: Create the sitemapRouter.js file in your router’s directory. At this point, your folder structure will look like this.
The updated dependencies in package.json file will look like.
"dependencies": {
"express": "^4.18.2",
"mongoose": "^8.0.0",
"sitemap": "^7.1.1"
}
Example: This example implements sitemap with rewuired haders and including required URLs.
Javascript
const express = require( "express" ),
{ SitemapStream, streamToPromise } = require( 'sitemap' ),
Todo = require( "../models/todoModel" ),
date = new Date().toISOString(),
zlib = require( "zlib" ),
router = express.Router();
let sitemap;
router.get( '/' , async function (req, res) {
res.header( 'Content-Type' , 'application/xml' );
res.header( 'Content-Encoding' , 'gzip' );
if (sitemap) return res.send(sitemap)
try {
const data = await Todo.find(),
todos = data.map(({ title }) => `/todo/${title}`),
smStream = new SitemapStream({
pipeline = smStream.pipe(zlib.createGzip());
todos.forEach(
item => smStream.write({
url: item, lastmod: date,
changefreq: 'daily' , priority: 0.7
}));
smStream.write({
url: '/about' , lastmod: date,
changefreq: 'monthly' , priority: 0.9
})
smStream.write({
url: '/contact' , lastmod: date,
changefreq: 'monthly' , priority: 0.9
})
streamToPromise(pipeline).then(sm => sitemap = sm);
smStream.end()
pipeline.pipe(res).on( 'error' , e => { throw e });
} catch (err) {
console.error(err)
res.status(500).end()
}
});
module.exports = router;
|
Output: Verify the changes in output by visiting {basename}/sitemap.xml.
Add the Sitemap to your robots.txt file.
User-agent: *
Allow: /
Sitemap: https://baseurl/sitemap.xml
Share your thoughts in the comments
Please Login to comment...