To best utilize the Import API, we recommend the following best practices.
Effective use of Import Containers
Organize Import Containers
- When importing full data sets, creating a new dedicated Import Container for the task can help in distinguishing imports performed on different occasions.
- When performing routine data imports, reusing a dedicated Import Container may be better.
- When importing data from multiple sources, using an Import Container for each source can help in organizing and monitoring progress.
- When performing high-volume, temporary imports, using ephemeral Import Containers BETA can help manage resources effectively.
Use Case | Possible Import Container organization |
---|---|
Import Product and Category | Create separate Import Containers for Product and Category |
Import Price changes daily at 5 PM | Create a reusable Import Container. If there are more than 200 000 imports per day, this may be broken down by some other business logic or temporary Import Container for excess counts. |
Import Product changes from multiple sources | Create one Import Container per source for Product imports. |
Optimize performance
200 000
Import Operations per Import Container. This way, monitoring activities at the container level will not be costly.48
hours after they are created, you can reuse Import Containers over time. An example schedule is as follows:Day | Import Operation total count | Import Containers (Import Operation count) |
---|---|---|
Day 1 | 100,000 | container-a (100,000) |
Day 2 | 500,000 | container-a (100,000), container-b (200,000), container-c (200,000) |
Day 3 | 400,000 | container-a (0), container-b (200,000), container-c (200,000) |
Day 4 | 200,000 | container-a (200,000), container-b (0), container-c (0) |
48
hours. container-a is now ready to be reused. Similarly container-b and container-c can be reused from Day 4.Clean up data from Import Containers and remove unused Import Containers
48
hours after they are created.Automatically delete Import Containers BETA
retentionPolicy
in the ImportContainerDraft when creating the ImportContainer.expiresAt
field based on your defined timeToLive
value. Once the expiresAt
datetime is reached, the ImportContainer and its Import Operations are permanently deleted. This setting is especially useful for workflows that generate many temporary Import Containers which you do not intend to reuse.1h
) and the highest is 30 days (30d
).Import large data sets
20
resources per Import Request. If you have a huge number of resources to import, we recommend using thread optimization to send your data as fast as possible to an Import Container.Rate limits
The Import API does not have rate limits, but to ensure the best performance we recommend that you send 300 API calls per second, per Project, to the Import API.
20
CategoryImport items, this means that up to 360,000 Categories can be sent to the Import API every minute.Choose the right Product import endpoint
The Import API has multiple endpoints for importing Product data. The following table summarizes what can be imported by each endpoint.
Endpoint | What can be imported |
---|---|
Product Drafts | Product data including Product Variants and Prices. |
Products | Product data without Product Variants and Prices. |
Product Variants | Product Variant data without Prices. |
Product Variant Patches | Product Variant Attribute data. |
Embedded Prices | Price data for a specific Product Variant. |
The following information explains common use cases for each endpoint.
Product Drafts
Common use cases:
- when you want to update a Product with 10+ new Product Variants, each with 20+ Embedded Prices.
- when you want to update a large number of Product Variants with new Attribute values and Prices.
Products
Product Variants
Product Variant Patches
Embedded Prices
Manage published state of Products
publish
which accepts a boolean value. The result of using this field varies based on whether you are importing data to create or update Products.When importing data to create a Product
true
, the Product is created and published immediately to the current projection. If false
, the Product is created but not published.When importing data to update an existing Product
The result of updating existing Products depends on if you are including changes in your import request, and if the Product currently has staged changes.
Value of publish | Does the import request have changes? | Does the Product have staged changes before importing? | Result |
---|---|---|---|
true | No | Yes | The staged changes are applied to the current projection and the Product is published. |
true | No | No | If the Product is currently unpublished, it is published to the current projection. |
true | Yes | N/A | The changes are applied to both the current and staged projection.
|
false | No | N/A | The Product is unpublished. |
false | Yes | N/A | The changes are applied to the staged projection, the Product is unpublished, and hasStagedChanges becomes true . |
Utilize the lifetime of Import Operations
48
hours to allow you to send other resources, including those referenced by other resources (unresolved references), during this time period.48
hours and waits for the Product to be imported.Handle retries
rejected
status. In other cases, the Import API will handle the retry internally without you needing to do anything.What not to do
Do not send duplicate import requests concurrently. Since the Import API imports data asynchronously, the order is not guaranteed. It may also lead to a concurrent modification error.
errors
field.Avoid concurrency errors
product
field in the ProductVariantPatch minimizes concurrency errors during the import process.product
field on one ProductVariantPatch, you have to set it for every ProductVariantPatch in the same ProductVariantPatchRequest.
Otherwise, the API returns an InvalidField error.