HTTP API
This document gives an overview of the characteristics and common features of the API.
The commercetools platform HTTP API provides an interface for programmatic access to the data stored on the platform and associated functionality. While the API can be used directly, for experimentation or custom integration solutions, we recommend taking a look at the available client libraries and SDKs, which provide a more comfortable development experience.
Regions
To provide the best possible Service Level Agreement (SLA) and minimal latency, the commercetools platform API is provided in three different Regions: North America, Europe, and Australia. These Regions are completely isolated from each other and no data is transferred between them.
Platform accounts
Platform accounts created for one Region are not valid for the other Regions. A signup is required for each Region individually.
In case you need advice in which Region your project should be located, please contact commercetools support.
Hosts
The commercetools HTTP API is hosted at the following URLs:
Region | API Host |
---|---|
North America (Google Cloud, Iowa) | https://api.us-central1.gcp.commercetools.com/ |
North America (AWS, Ohio) | https://api.us-east-2.aws.commercetools.com/ |
Europe (Google Cloud, Belgium) | https://api.europe-west1.gcp.commercetools.com/ |
Europe (AWS, Frankfurt) | https://api.eu-central-1.aws.commercetools.com/ |
Australia (Google Cloud, Sydney) | https://api.australia-southeast1.gcp.commercetools.com/ |
All requests to endpoints have to be prepended with these URLs.
If documentation refers to https://api.{region}.{cloudProvider}.commercetools.com
, the {region}
and {cloudProvider}
placeholders have to be replaced with the respective values.
The Machine Learning endpoints use different hosts, which are listed with each endpoint.
Although the deprecated hostnames like api.sphere.io
(Google Cloud, Belgium), api.commercetools.com
(Google Cloud, Belgium) and api.commercetools.co
(Google Cloud, Iowa) remain intact as aliases we strongly encourage changing your configuration the new host name structure.
Disallowed requests
Requests to the HTTP API are blocked if any of the following conditions are true:
- The combination of request URL and headers is longer than about 15KB.
- The request method does not allow a body, but the request has one.
- The request contains an upgrade header.
- The HTTP version is unknown.
It is strongly advised to use the host and domain names in your applications instead of hardcoding any IP address of the API hosts. IP addresses can change any time without prior notice.
JSON
The majority of the HTTP API endpoints consume and produce standard application/json
payloads. To ensure that payloads are encoded correctly, use Content-type: application/json
in your HTTP header when sending POST
requests.
When consuming a JSON response, ignore any unrecognized fields encountered in JSON objects of responses, or deal with them in a way that does not cause the client application to crash. The API assumes clients behave properly and considers the addition of new fields to be backwards-compatible with respect to existing clients. In addition, client applications should not rely on the presence of undocumented fields. Fields returned by an endpoint that are undocumented can be renamed or removed at any time without prior notice.
Identifier
Some resources contain an id
field which uniquely identifies a resource. The ID is generated by commercetools when creating a resource.
Some resources have a string field key
which also uniquely identifies a resource. The key
identifier is always
assigned to a resource by the client.
Update Guarantees
The API provides different update guarantees for different kinds of requests.
Strong Consistency
Each resource provides read-after-write consistency. Read-after-write consistency means that when you update an entity, wait for the successful response, and then read the same entity, you see the changes you just made.
Eventual Consistency
Some updates are not visible immediately after the update. For example, if you update a discount, updating the discount itself provides read-after-write consistency, but the prices on all affected products are updated after a delay. When you read a price immediately after the discount, you may read an the old value and not the updated one. If you wait a bit, and retrieve the price again, you will read the new value.
Similarly, after updating a product the Product Projection Search index is updated after a delay. If you retrieve a search result immediately after an update, the result may not reflect the updated product. When trying later, the search index will be updated.
These delayed updates provide an Eventual Consistency
guarantee. It means that after a delay, what you can read reflects all past updates.
These delays can vary depending on the amount of data to update. We try to keep these delays as small as possible.
Optimistic Concurrency Control
Many API resources use optimistic concurrency
control to prevent lost updates when making changes to existing data concurrently.
These resources have a version
attribute.
When sending (partial) updates to these resources, the expected
version of a resource is sent as a part of the request.
After making an update, the HTTP response body contains the new version of the resource. An API client must wait for the new version before sending follow-up requests.
Background processes and other events can also change a resource's version number without any requests sent from an API client directly to the resource. Do not rely on a consistently incremented version number.
The API does not use ETag
and If-Match
HTTP headers
for the purpose of optimistic concurrency control.
If a version mismatch occurs, the API returns a 409 HTTP status (Conflict) error.
Security
All API access is over HTTPS. API authorization uses OAuth 2.0 bearer tokens. See the section on authorization for details.
All HTTPS traffic is secured through Transport Layer Security (TLS). The relevant certificates are signed by trusted authorities. At any point in time, we will rely on a trusted root certificate. It is the API user's obligation to keep their systems updated regarding SSL/TLS capabilities as well as trusted root certificates. The server certificate as well as intermediary certificates of the certificate chain are transmitted as part of the TLS handshake.
Please note that it is actually possible to connect to the API via HTTP (without SSL/TLS). You will, however, receive a 404
error including an HTML body response on every attempt.
It is not necessary to actually support the full set of SSL/TLS features listed here (or stop supporting other features), especially the cipher suites, as long as one valid combination exists. The TLS handshake includes a "negotiation phase" that chooses the best combination that both client and server support.
Be aware that the certificates may at times support more features in transition periods. However, only the specifications listed below are reliable. You must not rely on other SSL/TLS features even if they currently work.
- Ciphersuites:
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
- Versions: TLSv1.2
Note: With this setup, we are following recommendations for modern compatibility by the Mozilla Foundation. Our API is, however, only bound to what is detailed out above.
Query Features
Common query features include sorting, paging, and reference expansion. Certain aspects of these features are standardized across all parts of the API as described in the following sections.
To filter results, Query Predicates allow to express complex queries on many resources.
Single Resource Query Response
Query responses for single resources only return a single, top-level JSON object.
PagedQueryResult
For query responses of requests supporting paging via limit
and offset
, the following common
top-level fields are returned:
offset
- Number
The offset supplied by the client or the server default. It is the number of elements skipped, not a page number.limit
- Numbercount
- Number
The actual number of results returned inresults
.total
- Number - Optional
The total number of results matching the query. This number is an estimation that is not strongly consistent.
This field is returned by default.
For improved performance, calculating this field can be deactivated by using the query parameterwithTotal=false
.
When the results are filtered with a Query Predicate, thetotal
is limited to the maximumoffset
of 10000.results
- Array of resultsfacets
- Object containing FacetResults - Optionalmeta
- Object containing supplementary information about the results - Optional
Sorting
A query endpoint that supports sorting does so through the sort
query parameter.
The provided value must be a valid sort expression.
The default sort direction is ASC
. The allowed sort paths are typically listed on the
specific query endpoints.
Here are some examples of sort expressions:
name desc
dog.age desc
If multiple sort expressions are specified via multiple sort
parameters,
they are combined into a composed sort where the results are first sorted
by the first expression, followed by equal values being sorted according to
the second expression, and so on.
Note: The sorting is case sensitive.
Paging
Queries on collections of resources can optionally provide paging functionality
through limit
and offset
query parameters, allowing clients to request
a certain page of the complete query result.
Limit
The maximum number of results to return from a query can be set using the limit
query parameter. Allowed is a value between 1 and 500. The default limit
on most query endpoints is 20.
Offset
The offset into the results matching the query can be set using the offset
query parameter. The default value is 0, indicating that no results should be skipped. The default maximum offset is 10000.
Total
The value of total
is limited to the maximum offset
when the results are filtered with a Query Predicate.
To improve query performance, the calculation of the total
field in the PagedQueryResult should be deactivated by passing false
in the withTotal
query parameter whenever possible.
Iterating over all elements
When iterating over all elements from an endpoint (for example to modify or export all products), using offset is not suitable to page through results:
offset
has an upper limit (see above), so exports can later become incomplete when the number of resources grows.- When an element is inserted while iterating over pages, a request with the next offset value might return already processed elements.
- Using large offsets can lead to slower response times.
To iterate over all elements while avoiding offset
you should follow this alternative approach instead:
- Query for a limited number of results in which the elements are sorted by their ID. Do no request the
total
field by passingwithTotal=false
. - Iterate with the same query, but additionally filter for IDs that are greater than the last ID from the previous step to get the next chunk of elements.
- Repeat step 2 as long as the number of returned results equals the requested limit.
The following pseudo-code demonstrates the approach using chunks of 100 elements at each iteration:
lastId = nullcontinue = truewhile (continue) {if (lastId == null)response = client.get(endpoint + "?withTotal=false&limit=100&sort=id+asc")elseresponse = client.get(endpoint + "?withTotal=false&limit=100&sort=id+asc&where=id%3E" + lastId) // "id%3E" is "id >" encodedresults = response.resultscontinue = (results.size == 100)lastId = results.last.id// do something with result}
Resource Timestamps
Many resources exposed on the API contain the createdAt
and the lastModifiedAt
timestamps.
Those timestamps are automatically set by the server on creation or on modification of the resource. They can be used as predicates and for sorting in queries.
Reference Expansion
Reference expansion is a feature of the endpoints listed in the section below that enables clients to request server-side expansion of Reference resources, thereby reducing the number of required client-server roundtrips to obtain the data that a client needs for a specific use-case.
Reference expansion can be used when creating, updating, querying, and deleting these resources.
Endpoints supporting Reference Expansion
Expansion paths
All query endpoints that support reference expansion provide an expand
query parameter, which can be used to specify one or more expansion paths
.
Expansion paths define the full path to the reference in a resource. Expansion path format:
refFieldName
- expands a reference field namedrefFieldName
located at the root of the resourcerefArrayFieldName[*]
- expands all references contained in the array of references namedrefArrayFieldName
refArrayFieldName[0]
- expands the first reference in the array of references namedrefArrayFieldName
objFieldName.refFieldName
- expands the referencerefFieldName
contained in the objectobjFieldName
objArrayFieldName[*].refFieldName
- expands the referencerefFieldName
in all objects contained in the arrayobjArrayFieldName
objNestedArrayFieldName[*][*].refFieldName
- expands the referencerefFieldName
in all objects contained in the nested arrayobjNestedArrayFieldName
The expansion path can also point to references in expanded objects.
Examples for Reference Expansion
Product Type and Tax Category on a Product
The JSON snippet below is taken from an Example-Product
representation that contains two references: one to an Example-Product-Type
and one to an Example-TaxCategory
. Without Reference Expansion you will get such a representation as response from the endpoints:
{"id": "<example-product-id>","version": 4,"productType": {"typeId": "product-type","id": "<example-product-type-id>"},..."taxCategory": {"typeId": "tax-category","id": "<example-tax-category-id>"},...}
With Reference Expansion you will get the expanded objects embedded into the product by an additional obj
field as shown in the following JSON snippet:
{"id": "<example-product-id>","version": 4,"productType": {"typeId": "product-type","id": "<example-product-type-id>""obj": {"id": "<example-product-type-id>","version": 4,"name": "Example Product Type","description": "example Product Type to showcase Reference Expansion","classifier": "Complex","attributes": [....],},..."taxCategory": {"typeId": "tax-category","id": "<tax-category-id>""obj": {"id": "<tax-category-id>","version": 2,"name": "Standard tax category","rates": [{"name": "5% US","amount": 0.05,"includedInPrice": false,"country": "US","id": "zJJ5KIGH"},{"name": "19% MwSt","amount": 0.19,"includedInPrice": true,"country": "DE","id": "893f_nXJ"}],...}
The query for the Example Product
introduced before - including both expansion paths: productType
and taxCategory
- would look like this:
GET https://api.{region}.{cloudProvider}.commercetools.com/<project-key>/products/<example-product-id>?expand=productType&expand=taxCategory
Categories on a Product
Expanding all Categories within a Product is an example for an expansion path for References in arrays: masterData.current.categories[*]
.
The query would thus look like this:
GET https://api.{region}.{cloudProvider}.commercetools.com/<project-key>/products/<example-product-id>?expand=masterData.current.categories%5B*%5D
To get the parent Category of the product's categories expanded, use masterData.current.categories[*].parent
as expansion path.
ReferenceType Attributes on a Product
In case you want to have the resources expanded that are the values on the ReferenceType Attributes you need to specify two expansion paths (one for the masterVariant
and one for the other variants
on the Product) to get the expanded objects on all the product variants. The example below specifies the expansion path for the current
ProductCatalogData, but it works similar for the staged
catalog data.
masterData.current.masterVariant.attributes[*].value
masterData.current.variants[*].attributes[*].value
GET https://api.{region}.{cloudProvider}.commercetools.com/<project-key>/products/<example-product-id>?expand=masterData.current.masterVariant.attributes%5B*%5D.value&expand=masterData.current.variants%5B*%5D.attributes%5B*%5D.value
For SetType of ReferenceType Attributes, you will get all the attribute values expanded when appending the value
field in the expansion path with a wildcard [*]
, like so:
masterData.current.masterVariant.attributes[*].value[*]
masterData.current.variants[*].attributes[*].value[*]
Line Item States on an Order
The ItemStates of LineItems in an Order can be expanded with the expansion path: lineItems[*].state[*].state
.
GET https://api.{region}.{cloudProvider}.commercetools.com/<project-key>/orders/<example-order-id>?expand=lineItems%5B*%5D.state%5B*%5D.state
Cleanup of old data
Some types of resources such as Carts, Orders, Payments, Shopping Lists and refresh tokens are created continuously. Keeping them around forever can negatively influence the performance of your project and the time it takes to restore it from a backup. Deleting unused data or moving old data to a cheaper storage for archival (for example a blob storage such as AWS S3 or Google Cloud Storage) ensures the best performance for your project.
Automatic Cleanup
Carts, Shopping Lists and refresh tokens can be automatically cleaned up after they haven't been used for some time. Carts and Shopping Lists allow you to set a number of days after which they are deleted while refresh tokens automatically expire. In both cases, the time is reset whenever the resource is being modified. A default can be set for Carts and Shopping Lists, and can be overwritten for individual Carts and Shopping Lists. For example, you could set the default to 90 days to have anonymous carts be deleted and then overwrite the default for all individual carts of logged-in customers to keep them for longer, for example 360 days. You could also let Shopping Lists be deleted after 360 days by default but turn off the automatic cleanup for Shopping Lists that have been publicly shared.
Additionally, if your project contains more than the maximum allowed amount of Carts, Shopping Lists or refresh tokens (see limits, the commercetools platform will automatically delete the resources that have been least recently modified.
Create carts and anonymous sessions only once necessary
Creating a new cart or an anonymous session on the first page visit can result in a very high number of resources, for example if a stateless web crawler acts like a new customer on every request or if a lot of visitors drop off after one or two page views. To avoid creating unused resources, only create them when needed (for example when the visitor is adding a product to the cart).
Data Erasure of Personal Data
In some countries, especially in the EU, you are required by a law such as the GDPR to erase personal data of a customer on request, also known as Right to be Forgotten.
A default DELETE
request may not clean up all data, both visible at the HTTP API layer (for example personal data may be part of messages) and invisible (the commercetools platform internally keeps logs for some time, to reconstruct data in case of faulty system behavior).
Endpoints that store personal data therefore offer a parameter for DELETE
requests called dataErasure
. If set to true
, the commercetools platform guarantees that all personal data related to the particular object, including invisible data, is erased, in compliance with the GDPR. You are, however, responsible for identifying and deleting all objects that belong to a customer, and deleting them.
The parameter is available for:
Personal data must not be stored in objects other than the ones listed above.
Partial Updates
For partial updates to existing resources, a common approach used by the HTTP API is
to accept a custom patch format in the form of a list of resource-specific actions that
need to be processed. The request method for partial update requests is usually POST
,
but PATCH
might also be supported in the future.
The general request representation (patch format) of a partial update request is as follows:
id
- String - Optional if part of the URL.
The ID of the targeted resource.version
- Number - Required if the targeted resource is versioned.
The expected current version of the resource on which to apply the changes.actions
- Array of resource-specific actions - RequiredMisspelt optional fields within the
actions
array are treated as additional fields and ignored by the API. In particular, no error is sent back.
Example Request:
{"id": "a136fd9e-bc29-4d05-813f-350b248aefd8","version": 1,"actions": [{"action": "setTitle","title": "My title"},{"action": "setText","text": "My text"}]}
The concrete actions that can be used on a specific resource are documented together with the resource.
All actions provided in a specific resource's patch format are generally freely combinable in a single request. Furthermore, such update requests only succeed or fail as a whole, that means unless explicitly stated otherwise, the result of all actions sent in the request is applied to a resource or none at all, if any of the actions fail.
Not all updates or state transitions of a resource may be freely composable with one another in a single HTTP request. For those updates, other endpoints may be provided to handle only a specific update or state transition in a single HTTP request.
Correlation ID
A Correlation ID is a unique identifier of a single action or event (such as checking out a cart), allowing to trace the action across multiple systems. For example, a mobile app makes a request to its backend, that backend makes another request to the commercetools platform, and the commercetools platform makes yet another request to an API Extension. If the mobile app sets a Correlation ID, and the backend forwards it to the commercetools platform, it'll also be forwarded to the API Extension. Therefore, the action can be followed through the logs of all four systems.
Correlation IDs are strings that may only contain alphanumeric characters, underscores, and hyphens and have a length of 8 to 256 characters. UUIDs can be used to ensure uniqueness.
The HTTP header X-Correlation-ID
is used to propagate the Correlation ID on HTTP requests and responses. If a client does not provide a Correlation ID for an HTTP request, the commercetools platform generates one.
When contacting support regarding a specific request, please also include the Correlation ID as it allows us to look up the request more easily.