Detailed guide to automatically generate a sitemap for your Nextjs website.
Sitemap is an xml file often available at /sitemap.xml path that either links to all pages on your website to gives link to other sitemap urls which might list all those pages. Thsi helps search engines discover pages on your site.
Search engines like Google visit your sitemaps to discover new links published on your website. If you do not have a sitemap Google will still be able to discover them when they crawl your website but not all links could be discoverable in this manner.
NextJs is a very popular frontend framework these days which is used by react developers to build their frontends. It is capable generating websites statically as well as dynamically. I have written about this in detail in my post building higher performance websites using nexts.
Becuse NextJs can generate pages statically during build time, it can also know the paths during build time and hence it can generate sitemaps very easily. Not all paths can be generated during build time as some of them will be added dynamically in the backend, but these paths can be very easily added to the sitemap during the next build cycle.
Sitemaps are only used by search engines and search engines only occasionally crawl your website depending on how important they think your web pages are. Hence there is no need to generate sitemaps every time there is an update to your site, rather you can generate them at a periodic cadence.
If you write your own code that generates the sitemap dynamically everytime search engine crawler visits /sitemap.xml that code might waste too many compute cycles, also this generation can be slow and hence search engines might think your site suffers from low performance issues.
It makes sense to generate sitemaps once and then cache them until the next generation cycle. It is upto you to decide what cadence to follow but for most, a simpel post build generation would workd just fine.
I recommend using "next-sitemap" module to make this process of sitemap generation incredibely simple.
Install the module using following command.
Please refer to next-sitemap documentation if you want more details.
Search engines often check a file /robots.txt before it can crawl your website. It makes sense to add this file as well to your sitemap generation process.
We first add sitemap file and robots.txt file to gitignore becuase they will be generated during build time. Add following lines to your .gitignore file.
Add a file "next-sitemap.config.js" to the root path of your nextjs app.
Note that in our case we store the base path of the website in an environment variable. You may have a different setup. An example value here could be "https://www.tanvi.dev"
After this we add sitemap generation to the post build process.
Modify your package.json file and add the following line under "scripts" object.
An example package.json file will look like this.
You can test if the sitemap was generated or not by running the following command.
Generation of sitemaps in nextjs apps is relatively simple process and should not take more than 10 minutes of your time.
I want to point out that sitemaps can also get complex. Video sitemaps and Google news sitemaps are different too but nearly all those complex cases can be handled easily by next-sitemap with merely configuration changes.