Learning PathModule

Use case: Integrate with external product and inventory data

Learn to import and synchronize data from external systems into your commercetools Project.

Ask about this Page
Copy for LLM
View as Markdown

After completing this page, you should be able to:

  • Understand the key considerations and best practices for using both the Import API and the HTTP API.

  • Analyze different strategies for importing and synchronizing data into commercetools.

  • Choose the optimal integration tool based on specific Project requirements and data characteristics.

In many enterprise architectures, Product data originates from multiple external systems of record. Zen Electron's setup is a classic example: they need to source core Product data from a Product Information Management (PIM) system and inventory levels from an Enterprise Resource Planning (ERP).

The synchronous HTTP API and the asynchronous Import API are the two primary ways to send data to the Project.

For a concise overview of planning your product data integration and choosing between the Import API and HTTP API, see Integrate product data.

A key best practice for any data import is to define unique keys for all resources. Keys make your import operations idempotent (safe to re-run) and provide a stable way to reference resources across different import jobs. Notice how each CSV file below includes a unique key for the primary resource it defines.

Assume the data is formatted into the following CSV files:

inventory_entries_from_erp.csvcsv
sku,quantityOnStock,restockableInDays
WM-001,150, 3
WM-001,85, 5
BK-001,200, 2
UCH-001,92, 3
products_from_pim.csvcsv
key,name_en_US,slug_en_US,product_type_key,variant_sku,variant_key,price_key,price_cent_amount,price_currency,tax_category_key
product-001,Wireless Mouse,wireless-mouse,electronics,WM-001,wm-001,wm-001-us-price,2999,USD,standard-tax
product-002,Bluetooth Keyboard,bluetooth-keyboard,electronics,BK-001,bk-001,bk-001-us-price,1232,USD,standard-tax
product-003,USB-C Hub,usb-c-hub,electronics,UCH-001,uch-001,uch-001-us-price,3999,USD,standard-tax

Solution A: The Import API

This approach leverages the Import API, which is designed for bulk, asynchronous data ingestion. A service extracts data from the PIM and ERP, transforms it into the required Import API format, and uploads it in bulk.
The Import API processes data asynchronously, making it an ideal choice for initial catalog imports and scheduled batch updates. The general Import API workflow for Zen Electron's scenario involves:
  1. Create an Import Container that acts as a staging area for your import operations. Do not specify the resourceType so the container can handle both inventory and product data.
  2. Submit inventory data from the ERP: You can import Inventory Entries before the associated Products exist because Inventory Entries are independent resources. The sku field determines which Product Variants receive the inventory availability data after matching variants exist.
  3. Submit product data from the PIM: Reuse the same Import Container. The Import API resolves KeyReference values that target Product Types, Tax Categories, and other resources.
  4. Poll the Import Container status to monitor progress. The Import API holds operations with unresolved KeyReference values for up to 48 hours while waiting for the referenced resources.
For step-by-step implementation examples with TypeScript code, see Bulk import with the Import API.
Once the Product Variant and Inventory Entry imports are complete, commercetools asynchronously updates each ProductVariant's availability field with the isOnStock, restockableInDays, and availableQuantity values from the corresponding InventoryEntry resources. For more details, read Product Variant availability: Strong vs Eventual Consistency.

Key considerations for the Import API

  • Idempotency and keys: provide a unique key for each imported resource, including Products, Product Variants, Prices, and Inventory Entries. The Import API uses the key to create a resource or update an existing resource, which makes repeated imports idempotent. Image data uses url, and Asset Sources use uri in the Product Variant import payload.
  • Asynchronous operation: the Import API is asynchronous. You submit a request and then poll for status updates.
  • Data orchestration: the Import API resolves resource relationships represented by KeyReferences.
For example, supplyChannel can reference a Channel key that doesn't exist yet. The API holds the operation in unresolved until the Channel is imported or created. Create the Channel within 48 hours.

The Import API uses the same request structure for create and update scenarios. Its KeyReference dependency management simplifies integrations with multiple external systems.

Solution B: The HTTP API

The HTTP API is synchronous and provides immediate feedback on whether an operation succeeded or failed. While this approach offers more control, it also requires managing backoff responses, resource dependencies, and error handling within your client.

The core pattern is an "upsert" (update or insert) operation:

  1. Attempt to fetch each resource (InventoryEntry or Product) by its key.
  2. If the resource is not found (404 response), create it with a POST request.
  3. If the resource exists, get its current version and update it with the appropriate update actions.

Results are nearly instantaneous, provided all dependencies (like Product Types) are in place. However, you are responsible for building a resilient client that ramps up request load gradually, retries transient errors with exponential backoff, and manages concurrency to avoid triggering back-pressure responses.

For step-by-step implementation examples with TypeScript code and guidance on building a resilient client, see Incremental sync with the HTTP API.

Choosing the right API

The Import API is designed for asynchronous bulk operations and resolves KeyReference values for resources imported from multiple systems. It is suitable for initial migrations and scheduled batch syncs. The HTTP API is designed for synchronous, real-time operations where immediate feedback and precise control over individual requests are required.

In practice, many projects use both: the Import API for the initial catalog load and periodic bulk refreshes, and the HTTP API for real-time incremental updates triggered by events in the PIM or ERP. For a detailed comparison, see Choose the right API.

Test your knowledge