# Project sync

Synchronize resources between commercetools Projects using the project-sync CLI tool.

[commercetools-project-sync](https://github.com/commercetools/commercetools-project-sync) is a Dockerized command-line application that automatically synchronizes resources between two commercetools Projects.

## Use cases

Project sync is suitable for the following use cases:

- Migrating data from a staging Project to a production Project.
- Keeping a secondary Project up to date with additions and updates from a primary Project.
- Copying data into a new Project for local development or testing.

Project sync only creates and updates supported resources in the target Project. It doesn't delete resources in the target Project when they're removed from the source Project, so the two Projects can diverge over time. Project sync isn't a full backup or disaster-recovery solution.

Project sync operates independently of the [Import API](/api/import-export/overview.md). It uses a different mechanism and doesn't rely on the Import API for its functionality.

## Supported resources

Project sync supports the following resources. Pass the listed value to the `-s` option to select which resources to sync:

| Resource | `-s` value |
| --- | --- |
| [Cart Discounts](/api/projects/cartDiscounts.md) | `cartDiscounts` |
| [Categories](/api/projects/categories.md) | `categories` |
| [Custom Objects](/api/projects/custom-objects.md) | `customObjects` |
| [Customers](/api/projects/customers.md) | `customers` |
| [Inventory](/api/projects/inventory.md) | `inventoryEntries` |
| [Products](/api/projects/products.md) | `products` |
| [Product Types](/api/projects/productTypes.md) | `productTypes` |
| [Shopping Lists](/api/projects/shoppingLists.md) | `shoppingLists` |
| [States](/api/projects/states.md) | `states` |
| [Tax Categories](/api/projects/taxCategories.md) | `taxCategories` |
| [Types](/api/projects/types.md) | `types` |

By default, project sync runs a delta sync, meaning it only synchronizes resources that changed since the last run. It tracks the last sync timestamp using a Custom Object in the target Project.

The first time you sync a given resource, no last sync timestamp exists yet. That run synchronizes the entire selected data set instead, the same as a full sync, so expect it to take longer than later runs.

Later runs then synchronize only the changes made since the previous run. You can also use the `--full` option to sync the entire data set at any time.

## Get started

### Prerequisites

Before running project sync, complete the following steps:

- Install [Docker](https://www.docker.com/) to run the project-sync Docker image.
- [Create an API Client](/api/getting-started/create-api-client.md) for both the source and target Project.
- Make sure the required identifier fields are set on each resource and subresource you intend to sync.

#### API Client scopes

Grant the `manage_project:{projectKey}` scope for both API Clients, or narrow the scopes down to only the resources you intend to sync, as shown in the following table. When narrowing scopes, always include one of the Custom Objects scopes for the target Project, because project sync stores its sync timestamps there. For more information about scopes, see [Scopes](/api/scopes.md).

| Resource | Scope |
| --- | --- |
| [Cart Discounts](/api/projects/cartDiscounts.md) | `manage_cart_discounts:{projectKey}` |
| [Categories](/api/projects/categories.md) | `manage_categories:{projectKey}` |
| [Custom Objects](/api/projects/custom-objects.md) | `manage_products:{projectKey}`, `manage_orders:{projectKey}`, `manage_customers:{projectKey}`, or `manage_key_value_documents:{projectKey}` |
| [Customers](/api/projects/customers.md) | `manage_customers:{projectKey}` |
| [Inventory](/api/projects/inventory.md) | `manage_products:{projectKey}` |
| [Products](/api/projects/products.md) | `manage_products:{projectKey}` |
| [Product Types](/api/projects/productTypes.md) | `manage_products:{projectKey}` |
| [Shopping Lists](/api/projects/shoppingLists.md) | `manage_shopping_lists:{projectKey}` |
| [States](/api/projects/states.md) | `manage_states:{projectKey}` |
| [Tax Categories](/api/projects/taxCategories.md) | `manage_tax_categories:{projectKey}` |
| [Types](/api/projects/types.md) | `manage_types:{projectKey}` |

#### Required identifier fields

The following resources and subresources must have these identifier fields set before you sync them:

| Resource | Required fields |
| --- | --- |
| [Product](/search.md?urn=ctp:api:type:Product) | `key` |
| [Product Variant](/search.md?urn=ctp:api:type:ProductVariant) | `key` and `sku` |
| Product Variant [Asset](/search.md?urn=ctp:api:type:Asset) | `key` |
| [Product Type](/search.md?urn=ctp:api:type:ProductType) | `key` |
| [Type](/search.md?urn=ctp:api:type:Type) | `key` |
| [Category](/search.md?urn=ctp:api:type:Category) | `key` |
| Category [Asset](/search.md?urn=ctp:api:type:Asset) | `key` |
| [Cart Discount](/search.md?urn=ctp:api:type:CartDiscount) | `key` |
| [Inventory Entry](/search.md?urn=ctp:api:type:InventoryEntry) | `sku` |
| [State](/search.md?urn=ctp:api:type:State) | `key` |
| [Tax Category](/search.md?urn=ctp:api:type:TaxCategory) | `key` |
| [Custom Object](/search.md?urn=ctp:api:type:CustomObject) | `container` and `key` |
| [Customer](/search.md?urn=ctp:api:type:Customer) | `key` |
| Customer [Address](/search.md?urn=ctp:api:type:Address) | `key` |
| [Shopping List](/search.md?urn=ctp:api:type:ShoppingList) | `key` |
| [Shopping List Line Item](/search.md?urn=ctp:api:type:ShoppingListLineItem) (Product Variant) | `sku` |
| Shopping List [Text Line Item](/search.md?urn=ctp:api:type:TextLineItem) | `name` |

### Set environment variables

Double-check the `SOURCE_PROJECT_KEY` and `TARGET_PROJECT_KEY` values before running project sync. Project sync creates and updates resources in the target Project to match the source Project, so running it with the wrong Project keys can overwrite existing data in that Project.

Set the following environment variables with the credentials and URLs of your source and target Projects:

```bash title="Set source and target environment variables"
export SOURCE_PROJECT_KEY="source-project-key"
export SOURCE_CLIENT_ID="sourceClientId"
export SOURCE_CLIENT_SECRET="sourceClientSecret"
export SOURCE_AUTH_URL="https://auth.eu-central-1.aws.commercetools.com/oauth/token"
export SOURCE_API_URL="https://api.eu-central-1.aws.commercetools.com"
export SOURCE_SCOPES="manage_project:source-project-key"

export TARGET_PROJECT_KEY="target-project-key"
export TARGET_CLIENT_ID="targetClientId"
export TARGET_CLIENT_SECRET="targetClientSecret"
export TARGET_AUTH_URL="https://auth.eu-central-1.aws.commercetools.com/oauth/token"
export TARGET_API_URL="https://api.eu-central-1.aws.commercetools.com"
export TARGET_SCOPES="manage_project:target-project-key"
```

### Run the Docker image

Pull the project-sync Docker image, replacing `5.9.1` with the [latest available version](https://hub.docker.com/r/commercetools/commercetools-project-sync/tags):

```bash title="Pull the project-sync Docker image"
docker pull commercetools/commercetools-project-sync:5.9.1
```

Run the image, using the `-s` option to choose which resources to sync. For example, to sync all supported resources:

```bash title="Run project sync for all supported resources"
docker run 
  -e SOURCE_PROJECT_KEY 
  -e SOURCE_CLIENT_ID 
  -e SOURCE_CLIENT_SECRET 
  -e SOURCE_AUTH_URL 
  -e SOURCE_API_URL 
  -e SOURCE_SCOPES 
  -e TARGET_PROJECT_KEY 
  -e TARGET_CLIENT_ID 
  -e TARGET_CLIENT_SECRET 
  -e TARGET_AUTH_URL 
  -e TARGET_API_URL 
  -e TARGET_SCOPES 
  commercetools/commercetools-project-sync:5.9.1 -s all
```

You can also sync individual resources, or a combination of resources, by passing their names to the `-s` option. For example, the following command syncs only Types and Product Types:

```bash title="Run project sync for selected resources"
docker run 
  -e SOURCE_PROJECT_KEY 
  -e SOURCE_CLIENT_ID 
  -e SOURCE_CLIENT_SECRET 
  -e SOURCE_AUTH_URL 
  -e SOURCE_API_URL 
  -e SOURCE_SCOPES 
  -e TARGET_PROJECT_KEY 
  -e TARGET_CLIENT_ID 
  -e TARGET_CLIENT_SECRET 
  -e TARGET_AUTH_URL 
  -e TARGET_API_URL 
  -e TARGET_SCOPES 
  commercetools/commercetools-project-sync:5.9.1 -s types productTypes
```

After the sync completes, the application prints a summary report for each synced resource, including how many resources were created, updated, or failed to sync.

For a full list of command-line options, supported Docker image tags, and troubleshooting information, see the [commercetools-project-sync repository](https://github.com/commercetools/commercetools-project-sync) on GitHub.

## Related pages

- [Area overview page with navigation](/dev-tooling.md)
- [Next page: Connect CLI](/dev-tooling/connect/cli.md)
- [Search documentation and API specs](/search.md)