# Sitemaps

A sitemap provides search engines with information about the pages on your site and their relationships. Search engines like Google read sitemaps to crawl your site more efficiently, helping to improve search engine optimization. By default, search engines look for the sitemap at `{your-website}/sitemap.xml`.

## Generate the sitemap

To generate the sitemaps, we need to fetch the list of all available paths, create a standard sitemap structure by adding necessary fields such as `loc` and `lastmod` to the sitemap response, and return the response in the XML format.

We use the helper methods from the `frontastic` package to get the list of paths and generate 3 sitemaps one for each: [static pages](/frontend-development/sitemaps.md#sitemap-for-static-pages), [product pages](/frontend-development/sitemaps.md#sitemap-for-products), and [category pages](/frontend-development/sitemaps.md#sitemap-for-categories) for easier maintenance.

### Sitemap for static pages

To generate the sitemap for all the static pages created using the Studio, we create a new file, `sitemap-static.xml/route.tsx`, in the `frontend/app/` directory. In this file, we implement the `GET` route handler method of Next.js that returns the sitemap response.

In the following code, we get the static pages structure using the `sdk.page.getPages()` method and convert the received data into `SitemapField` structure required by the sitemap utility. Then we pass the sitemap data `fields` to the `generateSiteMap` method from our sitemap utility and return the response.

The generated sitemap is made available on `{your-site}/sitemap-static.xml`.

```ts title="/src/app/[locale]/sitemap-static.xml/route.tsx" highlightLines="14-21, 41"
import { NextRequest } from 'next/server';
import { SiteMapField, generateSiteMap } from '@/utils/sitemap';
import { i18nConfig } from '@/project.config';
import { sdk } from '@/sdk';

export async function GET(
  request: NextRequest,
  { params }: { params: { locale: string } }
) {
  const locale = params.locale ?? i18nConfig.defaultLocale;

  const siteUrl = process.env.SITE_URL;

  sdk.defaultConfigure(locale);

  const fields = [] as SiteMapField[];

  const path = '/';
  const depth = 1;

  const res = await sdk.page.getPages({ path, depth });

  if (res.isError) return new Response(null, { status: 500 });

  const data = res.data;

  if (data?.pageFolderStructure) {
    fields.push(
      ...data.pageFolderStructure?.map((pageFolderStructureValue) => ({
        loc: `${siteUrl}/${locale}${(pageFolderStructureValue as any)._url}`, //eslint-disable-line
        lastmod: new Date().toISOString(),
        changefreq: 'daily' as const,
      }))
    );
  }

  return new Response(generateSiteMap(fields), {
    status: 200,
    headers: {
      'Cache-Control': 'public, s-maxage=86400, stale-while-revalidate',
      'Content-Type': 'application/xml',
    },
  });
}
```

### Sitemap for products

To generate the sitemap for all product pages created using the Studio, we create a new file `sitemap-products.xml/route.tsx` in the `frontend/app/` directory. In this file, we implement the `GET` route handler method of Next.js that returns the sitemap response.

In the following code, we get a list of all products by calling the `extensions.product.query` method and convert the received data into the `SitemapField` structure required by the sitemap utility. Then we pass the sitemap data `fields` to the `generateSiteMap` method from our sitemap utility and return the response.

The generated sitemap is made available on `{your-site}/sitemap-products.xml`.

```ts title="/src/app/[locale]/sitemap-products.xml/route.tsx" highlightLines="17-25, 43"
import { NextRequest } from 'next/server';
import { Product } from '@shared/types/product/Product';
import { generateSiteMap, SiteMapField } from '@/utils/sitemap';
import { i18nConfig } from '@/project.config';
import { sdk } from '@/sdk';

export async function GET(
  request: NextRequest,
  { params }: { params: { locale: string } }
) {
  const locale = params.locale ?? i18nConfig.defaultLocale;

  const siteUrl = process.env.SITE_URL;

  sdk.defaultConfigure(locale);

  const fields = [] as SiteMapField[];

  let nextCursor: string | undefined;

  do {
    const extensions = sdk.composableCommerce;

    const response = await extensions.product.query({
      cursor: nextCursor,
      limit: 500,
    });

    const items = [] as Product[];

    if (!response.isError && response.data.items != null) {
      items.push(...response.data.items);
    }

    fields.push(
      ...items?.map((product) => ({
        loc: `${siteUrl}/${locale}${product._url}`,
        lastmod: new Date().toISOString(),
        changefreq: 'daily' as const,
      }))
    );

    nextCursor = !response.isError ? response.data.nextCursor : undefined;
  } while (nextCursor);

  return new Response(generateSiteMap(fields), {
    headers: {
      'Cache-Control': 'public, s-maxage=86400, stale-while-revalidate',
      'Content-Type': 'application/xml',
    },
  });
}
```

### Sitemap for categories

To generate the sitemap for all category pages created using the Studio, we create a new file `sitemap-categories.xml/route.tsx` in the `frontend/app/` directory. In this file, we implement the `GET` route handler method of Next.js that returns the sitemap response.

In the following code, we get a list of all categories by calling the `extensions.product.queryCategories` method and convert the received data into the `SitemapField` structure required by the sitemap utility. Then we pass the sitemap data `fields` to the `generateSiteMap` method from our sitemap utility and return the response.

The generated sitemap is made available on `{your-site}/sitemap-categories.xml`.

```ts title="/src/app/[locale]/sitemap-categories.xml/route.tsx" highlightLines="17-25, 43"
import { NextRequest } from 'next/server';
import { Category } from '@shared/types/product/Category';
import { generateSiteMap, SiteMapField } from '@/utils/sitemap';
import { i18nConfig } from '@/project.config';
import { sdk } from '@/sdk';

export async function GET(
  request: NextRequest,
  { params }: { params: { locale: string } }
) {
  const locale = params.locale ?? i18nConfig.defaultLocale;

  const siteUrl = process.env.SITE_URL;

  sdk.defaultConfigure(locale);

  const fields = [] as SiteMapField[];

  let nextCursor: string | undefined;

  const extensions = sdk.composableCommerce;

  do {
    const response = await extensions.product.queryCategories({
      cursor: nextCursor,
      limit: 500,
    });

    const items = [] as Category[];

    if (!response.isError && response.data.items != null) {
      items.push(...response.data.items);
    }

    fields.push(
      ...items?.map((category) => ({
        loc: `${siteUrl}/${locale}${category._url}`,
        lastmod: new Date().toISOString(),
        changefreq: 'daily' as const,
      }))
    );

    nextCursor = (!response.isError && response.data.nextCursor) as string;
  } while (nextCursor);

  return new Response(generateSiteMap(fields), {
    headers: {
      'Cache-Control': 'public, s-maxage=86400, stale-while-revalidate',
      'Content-Type': 'application/xml',
    },
  });
}
```

### Final sitemap

Now that we've set up sitemap generation for all our pages, we can use a utility to generate a [sitemap index](https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps) that lists all the different sitemaps, such as the following:

- `{your-site}/sitemap-static.xml`
- `{your-site}/sitemap-products.xml`
- `{your-site}/sitemap-categories.xml`

```ts title="/src/utils/sitemap/index.ts"
export interface SiteMapField {
  loc?: string;
  lastmod?: string;
  changefreq?:
    | 'always'
    | 'hourly'
    | 'daily'
    | 'weekly'
    | 'monthly'
    | 'yearly'
    | 'never';
  priority?: string;
}

export const generateSiteMap = (fields: SiteMapField[]) => {
  const sitemap = `
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    ${fields
      .filter(
        ({ loc, lastmod, changefreq, priority }) =>
          !!loc || !!lastmod || !!changefreq || !priority
      )
      .map(
        ({ loc, lastmod, changefreq, priority }) =>
          `
                <url>
                    ${loc ? `<loc>${loc}</loc>` : ''}
                    ${lastmod ? `<lastmod>${lastmod}</lastmod>` : ''}
                    ${
                      changefreq ? `<changefreq>${changefreq}</changefreq>` : ''
                    }
                    ${priority ? `<priority>${priority}</priority>` : ''}
                </url>
            `
      )
      .join('\n')}
    </urlset>
    `;

  return sitemap;
};
```

This utility can be run as a `postbuild` script that is specified in the `frontend/scripts/generate-sitemaps-index.ts` file and generates the sitemap after the `next build` has finished.

```ts title="/scripts/generate-sitemaps-index.ts"
import fs from 'fs';
import dotenv from 'dotenv';
import { i18nConfig } from 'src/project.config';

dotenv.config();

const host = process.env.SITE_URL;

const sitemaps = [
  'sitemap-categories.xml',
  'sitemap-products.xml',
  'sitemap-static.xml',
];

const sitemapLocations = sitemaps
  .map((sitemap) =>
    i18nConfig.locales.map((locale) => `${host}/${locale}/${sitemap}`)
  )
  .flat();

const sitemapIndexContent = (() => {
  const now = new Date().toISOString();

  let xml = `<?xml version="1.0" encoding="UTF-8"?>\n<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n`;

  sitemapLocations.forEach((location) => {
    xml += `  <sitemap>\n    <loc>${location}</loc>\n    <lastmod>${now}</lastmod>\n  </sitemap>\n`;
  });

  xml += `</sitemapindex>`;

  return xml;
})();

fs.writeFileSync('./public/sitemap.xml', sitemapIndexContent);
```

After the build, the generated sitemap index is made available on `{your-site}/sitemap.xml`. The sitemap index provides the URL to all three sitemaps we created, allowing search engines and web crawlers to index your website.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap><loc>{your-site}/sitemap-static.xml</loc></sitemap>
  <sitemap><loc>{your-site}/sitemap-categories.xml</loc></sitemap>
  <sitemap><loc>{your-site}/sitemap-products.xml</loc></sitemap>
</sitemapindex>
```

## Related pages

- [Area overview page with navigation](/frontend-development.md)
- [Previous page: Styling your site](/frontend-development/styling-your-site.md)
- [Next page: Client-side routing for dynamic pages](/frontend-development/client-side-routing-for-dynamic-pages.md)