Description
Integrate commercetools with an external system through Connect — payment (Stripe, Adyen, Mollie, PayPal), tax (Avalara, Vertex, TaxJar), PIM (Akeneo), CRM (Salesforce, HubSpot), order management/OMS (Fluent Commerce, fulfillmenttools, NewStore), gift cards (Voucherify), transactional email (SendGrid, Mailgun), marketplace (Mirakl, Marketplacer), promotion and loyalty (Talon.One, Voucherify), analytics export to a warehouse/CDP (BigQuery, Snowflake, Segment), search and product discovery (Algolia, Constructor, Bloomreach), and shipping (carriers, rate engines, label platforms). Each sub-area rules out the native capability first, then picks a rung — use a public connector, close the gap with config, fork, or build. Use when syncing commercetools to or from an external system, or configuring, forking, or debugging a connector. Type-agnostic build contracts live in commercetools-connect.
Installation
In any Claude Code session:
/plugin marketplace add commercetools/commercetools-ai-plugins
/plugin install commercetools@commercetools
If you've updated the plugin or installed it in another window and need the current session to pick up the latest version:
/reload-plugins
commercetools/commercetools-ai-plugins. Then, click on the plugin and click Install.Instructions Included
commercetools integrations
connect.yaml mapping, the runtime contract, and the verification steps.service / event / job application is written, the connect.yaml contract, least-privilege scopes, lifecycle scripts, sync-vs-async idempotency and ack semantics, testing, deployment, and the production-readiness gate are type-agnostic and live in commercetools-connect. Start here to decide what to build for a given vendor; go there for how to build and ship it.Route to the sub-area first
overview.md — it owns the workflow, the decision ladder, and the traps for that domain.| Domain | Sub-area |
|---|---|
| Payment (Stripe, Adyen, Mollie, PayPal, …) | references/payment/overview.md |
| Tax (Avalara, Vertex, TaxJar, …) | references/tax/overview.md |
| CRM (Salesforce, HubSpot, Dynamics 365, Zoho, …) | references/crm/overview.md |
| PIM (Akeneo, inriver, Bluestone, Pimcore, …) | references/pim/overview.md |
| Order management / OMS (Fluent Commerce, fulfillmenttools, kbrw, OneStock, NewStore, Pipe17) | references/order-management/overview.md |
| Gift card (Voucherify, in-house store credit, …) | references/giftcard/overview.md |
| Transactional email (SendGrid, Mailgun, AWS SES, Postmark, …) | references/email/overview.md |
| Marketplace (Marketplacer, Mirakl, Convictional, channel managers) | references/marketplace/overview.md |
| Promotion / loyalty (Talon.One, Voucherify, Dovetech, Eagle Eye, …) | references/promotion/overview.md |
| Analytics — warehouse / CDP / product analytics (BigQuery, Snowflake, Redshift, Databricks, Segment, mParticle) | references/analytics/overview.md |
| Search / product discovery (Algolia, Constructor, Bloomreach, Coveo, Elasticsearch, Typesense) | references/search/overview.md |
| Shipping (carriers, rate-shopping engines, label/shipping-execution platforms) | references/shipping/overview.md |
The ladder every sub-area walks
- Native commercetools capability — does this need an integration at all? Real for promotion (Product Discounts / Cart Discounts / Discount Codes / Discount Groups), search (Product Search), and shipping (Zones, Shipping Methods, tiered rates, predicates). Rule it out explicitly, with the reason stated.
- Use a public connector — one exists and covers the requirements → install and configure. Check live, never from memory; name the connector and version you checked.
- The gap is config, not code — enabled features, credentials, markup, field mappings and fallback behavior are typically
connect.yamlvalues → back to rung 1. - Customise / fork — a real gap config can't close, and an open-source connector exists → fork, add only the delta, publish as an Organization connector.
- Build a new one — nothing exists for this vendor → build from the closest application template.
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<terms from the user's request>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
scripts/openApi-schemata.mjs and scripts/graphql-schemata.mjs are here too, for confirming request/response shapes from the OAS or GraphQL schema instead of from memory.What a sub-area contains
overview.md and connector-selection.md exist everywhere. The rest is the common shape, not a guarantee — list the sub-area's directory rather than assuming a file exists:| File | Owns |
|---|---|
overview.md | Start here. Present in all twelve. Orientation, the rung-0 gate, requirements extraction, the workflow, and routing to the rest |
connector-selection.md | Present in all twelve. The decision ladder for this domain: what exists, how to check live, which template to scaffold from |
config-from-requirements.md | Requirements → connect.yaml: applications, credentials, config keys, least-privilege scopes, worked example |
*-contract.md | The runtime contract: what each application must do, and the pitfall catalog |
verification.md | Proving the round trip works end to end, plus the traps that look like bugs |
pim/ uses build-connector.md + data-mapping.md + testing.md, order-management/ uses build-oms-connector.md + sync-architecture.md, and analytics/ uses pipeline-architecture.md — those last two carry the runtime contract in place of a *-contract.md. Others add provider specifics, test harnesses, or public-connector assessments.Rules that hold across all twelve
- Ask, don't assume. Requirements come before config, and config before code. Direction, source of truth, and what happens when the external system is down are business decisions — get them from the user and record them.
- Check the registry live. Connector availability changes. One you remember may not exist; one you don't may.
- Vendor facts come from the vendor. Their auth, payloads, field names, limits, and sandbox behavior are theirs to document and change — read their current API docs, and for a public connector its repo's
connect.yamland README. Do not write vendor field names from memory. - Present the ladder; let the user choose the rung. Give a recommendation and its reasoning, then record the decision.
- Hand back for the build. Once the rung is chosen and the applications are designed, the lifecycle, testing, and production-readiness gate are commercetools-connect.
References
Requirements → analytics connector config
connect.yaml values. For a public connector these are its documented keys; for a build (the common case) these are the keys and apps you define.The requirement → config map
| Requirement (Step 1) | Config / decision | Why |
|---|---|---|
| Destination + credentials | securedConfiguration: destination API key / service-account JSON / connection string | Secrets never in standardConfiguration, never hardcoded — often PII-adjacent |
| Latency (stream / batch / both) | App composition (see below) | Direction is fixed (egress); latency decides which apps exist |
| Which data domains | The Messages the streamer subscribes to and the read scopes | Only subscribe to / read what you export |
| Historical backfill | A separate job with a schedule (or on-demand) | Backfill and delta need different tooling — keep them separate |
| Destination schema / grain | Transform module + dedup/merge key | Event-row vs upserted current-state decides the transform |
| Region + project | standardConfiguration: CTP_REGION; scopes via inheritAs | Host + client provisioning are region/project specific |
| Volume / throughput | Batch page size + backoff toggles; Subscription budget | The 50-Subscription soft limit and destination rate limits constrain design |
Latency → app composition
- Streaming (near-real-time): an
eventapp. Register a MessageSubscription to the specific Messages you export (e.g.OrderCreated,OrderStateChanged,CustomerCreated), or a ChangeSubscription on a resource to catch every change. Subscribe to the minimum set — the 50-Subscription-per-Project limit is a soft limit, so don't burn it with one Subscription per message type when a ChangeSubscription covers the resource. - Batch / backfill: a
job(properties.schedule) that queries the API with alastModifiedAtwindow + cursor pagination and loads the delta. Also the vehicle for the one-time historical load. - Optional full-export service: a
serviceendpoint that triggers an on-demand full export (the template's full-export app). Mention it; build it only if the user needs on-demand full loads.
Scopes and secrets
- Least-privilege, read-only. Egress reads commercetools and writes the destination — so the commercetools scopes are read scopes for the domains you export plus
manage_subscriptionsfor the streamer's registration. Never an admin/manage_projectclient. → commercetools-connect security.md. - Destination credentials in
securedConfiguration— API key / service-account JSON / connection string — neverstandardConfiguration, never hardcoded. Analytics data includes customer PII; treat destination creds accordingly.
The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / properties / configuration; inheritAs), and place the file at the repository root — a nested connect.yaml silently fails to deploy.Native client provisioning (prefer this)
CTP_CLIENT_ID/SECRET:inheritAs:
apiClient:
scopes:
- manage_subscriptions # streamer: postDeploy registers the Subscriptions
- view_orders # export orders / order Messages
- view_customers # export customers (PII — only if in scope)
- view_published_products # export catalog (use the read scope your domains need)
# add view_payments / view_stock etc. only for the domains you actually export
configuration:
standardConfiguration:
- key: DESTINATION_DATASET
description: Warehouse dataset / table (or CDP source id)
securedConfiguration:
- key: DESTINATION_CREDENTIALS
description: Destination API key / service-account JSON / connection string
Note:view_subscriptionsis not a valid standalone scope —manage_subscriptionscovers read + write. Declaring non-existent view scopes fails client creation. Grant only theview_*scopes for the domains you export — nothing more.
Per-app config
deployAs:
- name: analytics-streamer # near-real-time
applicationType: event
endpoint: /analyticsStreamer
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the Subscriptions
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "Injected by Connect (GoogleCloudPubSub or SNS)"
- name: analytics-backfill # scheduled batch + one-time history
applicationType: job
endpoint: /analyticsBackfill
properties:
schedule: "0 2 * * *" # 02:00 daily; or run on-demand for the one-time load
Worked example (Snowflake, build, orders + customers, stream + nightly backfill)
europe-west1.gcp.Derived config:
inheritAs:
apiClient:
scopes: [manage_subscriptions, view_orders, view_customers]
configuration:
standardConfiguration:
- key: SNOWFLAKE_ACCOUNT
description: "Snowflake account + database/schema/table"
securedConfiguration:
- key: SNOWFLAKE_CREDENTIALS
description: Snowflake key-pair / PAT for the loader role
deployAs:
- name: analytics-streamer
applicationType: event
endpoint: /analyticsStreamer
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub (injected on Connect)"
- name: analytics-backfill
applicationType: job
endpoint: /analyticsBackfill
properties:
schedule: "0 2 * * *"
order (OrderCreated, OrderStateChanged) and a ChangeSubscription on customer — re-fetch by id, transform to the Snowflake row, deliver, and emit resource.id + sequenceNumber as the dedup key so a MERGE on the warehouse side is idempotent; one job windowing on lastModifiedAt + cursor pagination for the nightly gap-repair and the one-time history load; scopes are exactly the two read scopes + manage_subscriptions the apps need; Snowflake creds are securedConfiguration; customer PII is minimized to the columns analytics needs and never logged (pipeline-architecture.md).Is a public connector enough? (analytics)
Do the live check anyway — don't answer from memory
- Search the Connect marketplace and the integration docs (via
docs-search/ the Knowledge MCP) for the user's destination (e.g. "Segment", "Snowflake", "BigQuery") — not for "analytics". - Apply the marketplace-listing rule. Analytics/CDP listings are especially likely to be partner services, SaaS products, or iPaaS/ELT middleware (Fivetran/Airbyte-style loaders, a CDP's own commercetools source) rather than a deployable Connect connector. Confirm a Connect affordance (public repo /
connect.yaml/ a Connect deploy action) before calling anything install/configure — full rule: Marketplace listings are not all Connect connectors. - Name what you checked (connector/product + version) or record "none exists", and confirm the path with the user before building.
The landscape (verify, but this is the shape)
| What you may find on/around the marketplace | What it actually is | Default rung |
|---|---|---|
| A CDP's own commercetools source/integration (Segment, mParticle, RudderStack, Tealium) | Often the CDP's product, configured on their side — may not be a Connect connector | Use it if it covers the domains — but verify it's Connect-deployable; else it's out of this skill's scope |
| An ELT/data-loader (Fivetran/Airbyte-style) reading the commercetools API | Third-party pipeline tooling, not a Connect connector | Valid alternative — but not built/deployed via Connect (say so) |
| A warehouse listing (BigQuery/Snowflake/Redshift) | Almost never a turnkey commercetools connector | 4 (build from template) |
| Nothing for the destination | The common case | 4 (build from template) |
connect.yaml, the Connect CLI, lifecycle scripts) apply only if they choose a Connect connector.The ladder (stop at the first rung that fits)
Rung 1 — Configure a public connector / native destination integration
deployment create) is the commercetools-connect skill's deployment-installation.md. Hand it the config from config-from-requirements.md.Rung 2 — A gap that config can close
Which Messages/domains flow, field-to-column mapping, and destination table/dataset are usually configuration, not code. Re-check the apparent gap against the connector's configuration surface before forking.
Rung 3 — Fork/extend a public connector (only if open source)
Rung 4 — Build from the Product export template (the common case)
- a full-export application (an API endpoint that exports all resources of a Store to an external system) — your backfill/full-load base;
- an incremental updater — an
eventapp that subscribes to Messages and pushes each change to the external system — your streamer base.
commercetools connect init, template product-export), then adapt: change which resources/Messages you subscribe to (orders/customers/payments, not just products), rewrite the transform to your destination's schema, and replace the delivery call with your destination's ingestion API. What you write is the transform + the destination client + the dedup key; the Connect plumbing (envelope handling, subscription registration, lifecycle) is scaffolded. Contract and gotchas: pipeline-architecture.md.Recording the decision
Destination: Snowflake · rung 4 (build) · checked marketplace 2026-08 — no Connect-deployable Snowflake connector; found only ELT loaders (out of Connect scope) · building an event streamer + a nightly backfill job from the product-export template, deduped on resource.id+sequenceNumber.
Analytics destinations — pick the mechanism, don't catalog vendors
The transport underneath (same for every destination)
CONNECT_SUBSCRIPTION_DESTINATION and build the destination from the matching injected vars (CONNECT_GCP_* or CONNECT_AWS_TOPIC_ARN), never assuming Pub/Sub (event-applications.md, Pattern 7). Batch loads bypass the broker entirely and query the HTTP/GraphQL API (pipeline-architecture.md).Categories (route by these, not by brand)
Data warehouses — BigQuery, Snowflake, Redshift, Databricks
- Decision: the default and best-fit analytics destination. Both mechanisms apply — stream events for freshness, batch for history/gap-repair. Land raw event rows into staging keyed on
resource.id+sequenceNumber, then model/MERGEdownstream. - What flows: all transactional/state truth — orders, line items, customers, payments, inventory, catalog.
- PII: the warehouse becomes a PII store — minimize columns, and propagate erasure (pipeline-architecture.md → PII).
CDPs — Segment, mParticle, RudderStack, Tealium
- Decision: stream server-side commerce events into the CDP so it can unify profiles and fan out to downstream tools. Check first whether the CDP has its own commercetools source / integration — if it does and it fits, that may be a configure path (rung 1), not a Connect build (connector-selection.md).
- What flows: customer + order events keyed to a stable user id; usually a curated event set, not the full catalog.
- PII: CDPs are identity-centric — consent and identifier mapping matter; carry only permitted fields.
Product / behavioral analytics — Amplitude, Mixpanel, GA4, Snowplow
- Decision: server-side ingestion of commerce events only (e.g. purchase/refund from
OrderCreated/order-state Messages). Honesty caveat: GA4 and Mixpanel are client-side-first — their server-side ingestion (e.g. GA4 Measurement Protocol) is limited and event-shaped, not a natural full-data-export target. Don't present them as a warehouse substitute; if the user wants complete history and modeling, route to a warehouse. Snowplow/Amplitude have more first-class server-side ingestion. - What flows: a small, well-defined set of conversion/behavioral events — not orders-as-rows.
- PII: keep to hashed/consented identifiers per the tool's model.
BI tools — Looker, Tableau, Power BI
- Decision: do not integrate BI with commercetools directly. BI sits on the warehouse, not on commercetools — point the connector at a warehouse (above) and let BI read from there. If the user asks to "connect Tableau to commercetools," redirect: build the warehouse egress, then BI reads the warehouse.
- What flows: nothing directly from commercetools — it reads modeled tables in the warehouse.
Restating the boundaries (so the design doesn't drift)
- Server-side egress only. Client-side behavioral/pixel tracking (GA4 gtag, Google Tag Manager, Segment.js) belongs in the storefront (e.g. commercetools Frontend) with a tag manager — not a Connect connector. A connector can feed server-side ingestion of the same tools, but page-view/click tracking is a storefront concern.
- Not Platform Insights. If the "analytics" the user wants is API latency / error rates / request logs, that's Platform Insights (an Add-On forwarding telemetry to New Relic / Datadog / OpenTelemetry / Dynatrace) — operational APM, not commerce data. Route there, not to a connector.
- Not Change History. The Audit Log / Change History is a governance change log on a separate, rate-limited host — never an analytics feed.
Choosing, in one line
Analytics connector — export commercetools data to an analytics destination
- Streaming (near-real-time): a Connect
eventapp on Subscriptions / Messages — the resource changes, a Message is delivered, you transform and deliver a row. - Batch (scheduled backfill / periodic load): a Connect
jobapp that queries the HTTP/GraphQL API with alastModifiedAtwindow + cursor pagination and loads the delta.
The mistake to internalize first: delivery is at-least-once, so dedup on the destination side — and never build analytics off Change History. ASubscriptiondelivers each Message at least once with no ordering (delivery guarantees); without a dedup key on the warehouse side (resource.id+sequenceNumber) you get duplicate rows. And the Change History / Audit Log is not an analytics feed: it lives on a separate host, is token-rate-limited (429 +Retry-After), and the docs explicitly say to "avoid making API calls in response to an event stream or message subscription" — it is a governance/compliance log, not a high-throughput data source.
Disambiguate "analytics" before designing (three different things)
Fix which one the user means — they route completely differently:
- Commerce/business analytics (this sub-area). Transactional/state truth (orders, customers, inventory, catalog) flowing out to a warehouse/CDP/analytics tool, server-side. This is what you build here.
- Platform Insights — an Add-On that forwards commercetools API metrics and server-side logs to an APM (New Relic, Datadog, OpenTelemetry, Dynatrace). This is operational/APM telemetry, not commerce data — if the user wants request latency / error rates, route them here, not to a connector.
- Change History / Audit Log — a governance/compliance change log (who changed what). Separate host, rate-limited — not an analytics feed (see the blockquote).
Server-side egress only — client-side tracking is out of scope
Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<analytics terms from the user's request, e.g. 'export orders data warehouse subscription messages product export template lastModifiedAt query pagination'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root, where scripts/docs-search.mjs lives.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or https://docs.commercetools.com for deeper follow-up.Step 1 — Extract requirements (before any config or code)
- Which destination, and what kind? A data warehouse (BigQuery, Snowflake, Redshift, Databricks), a CDP (Segment, mParticle, RudderStack, Tealium), a product/behavioral-analytics tool (Amplitude, Mixpanel, GA4, Snowplow), or a BI tool (Looker/Tableau/Power BI — which sits on the warehouse, not on commercetools). The category decides the mechanism — see destinations.md.
- Which data domains? Orders, carts, customers, payments, inventory, catalog (products/prices) — and which fields. Customer data is PII: name it explicitly so GDPR handling is designed in, not bolted on.
- Latency — streaming, batch, or both? Near-real-time (event app on Messages) vs periodic load (job querying the API). Most real pipelines need both: a stream for freshness + a batch backfill for history and gap-repair.
- Historical backfill needed? A one-time (or periodic full) load of existing data is a separate
jobfrom the ongoing stream — like a migration. - Destination schema / grain. One row per event, or an upserted current-state table? This decides the transform and the dedup/merge key.
- Volume & throughput. Order/event volume shapes batch page size, backoff, and whether the ~50-Subscription budget is a constraint.
- Anything special or non-standard? (always ask — open-ended) Multi-project/multi-region consolidation, data residency, real-time personalization needs, existing warehouse loader/ELT tooling (Fivetran/Airbyte-style) the user already runs, retention/erasure policy. Capture each as its own line; don't force it into a slot above.
event streamer on the relevant Messages for freshness plus a job backfill for history, delivering to a warehouse, deduped on resource.id + sequenceNumber, read-only least-privilege scopes, destination creds in securedConfiguration — and say so explicitly.Step 1.5 — Is a public connector enough? (a hard, ordered gate — do the live check anyway)
- Check live data. Search the Connect marketplace and the integration docs (via
docs-search/ Knowledge MCP) for anything targeting the user's destination. Don't answer from memory — the marketplace changes. - Apply the marketplace-listing rule. A listing may be a partner/iPaaS/SaaS product, not a deployable Connect connector — see Marketplace listings are not all Connect connectors.
- Confirm with the user, and only then conclude the rung.
- A public connector / native destination integration covers it → install + configure. Installation is the commercetools-connect skill's deployment-installation.md.
- A gap looks like config → prove it (which Messages, field mapping, destination table) before forking → back to rung 1.
- An open-source connector with a real gap → fork/extend it; hand off to commercetools-connect for the lifecycle.
- No connector for the destination (the common case) → build from the Product export template and adapt it to your destination. This is the default landing rung. → connector-selection.md, pipeline-architecture.md.
Step 2 — Design the pipeline (the core deliverable)
payloadNotIncluded) live. Read pipeline-architecture.md and pick the destination mechanism from destinations.md; produce, for the user: the app list, the message-subscription list, the transform/schema mapping, and the dedup strategy.Step 3 — Build (rungs 3–4), test-first
- Event streamer = an
eventapp subscribing to the relevant Messages → event-applications.md. At-least-once, no ordering: decode the Pub/Sub envelope, re-fetch by id (required onpayloadNotIncluded), ack correctly, and emit a stable dedup key. - Batch/backfill = a
jobquerying the API withlastModifiedAt+ cursor pagination → job-applications.md. Checkpoint the window; each unit idempotent. - Optional full-export service (like the template's full export, or an in-Merchant-Center dashboard via a custom application) = a
serviceapp → service-applications.md. Mentioned as an extension, not required. - Registration of Subscriptions in idempotent
postDeploy/preUndeploy→ lifecycle-scripts.md.
payloadNotIncluded, idempotent batch window — are invisible at the call site. Mock the destination and the commercetools API and assert on what your code decided to write. → testing.md.Step 4 — Deploy
securedConfiguration, never in code.Step 5 — Verify the round trip
payloadNotIncluded; Messages query API off by default).References
| Need | Reference |
|---|---|
| Is a connector enough?: the live registry/marketplace check even though we expect build; CDP/ELT-changes-the-answer; the configure/fork/build-from-template ladder | connector-selection.md |
Requirements → config: which Messages / job schedule, least-privilege read scopes, destination creds in securedConfiguration, worked example | config-from-requirements.md |
Pipeline architecture (the substance): event streamer + batch/backfill job, event→row transform, warehouse-side dedup keys, payloadNotIncluded re-fetch, PII/GDPR, the limits; full pitfall catalog | pipeline-architecture.md |
| Destinations: warehouse vs CDP vs product-analytics vs BI — the stream-vs-batch decision, what data flows, PII implication, the client-side honesty caveat | destinations.md |
| Verify the round trip: one change → one row, idempotent batch window; the no-subscription / duplicate-row / re-fetch / query-off-by-default traps | verification.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
Adding another destination later reuses this same tree — the two primitives, the dedup model, and the flow don't change; only the destination's delivery API does.
Checklist
Requirements
- Destination named + categorized (warehouse / CDP / product-analytics / BI); data domains + fields listed
- "analytics" disambiguated (commerce data vs Platform Insights vs Change History); client-side tracking flagged out of scope
- Latency decided (stream / batch / both); historical backfill in or out of scope
- Destination schema/grain (event rows vs upserted current-state) and dedup/merge key decided
- PII data domains flagged; retention/erasure policy captured
- Open-ended "anything special?" asked; each special requirement its own line
- Requirements block written and confirmed
Connector fit (decide before building)
- Ran the live marketplace/registry check even though build is expected; applied the listing-isn't-a-connector rule; confirmed with the user
- Ladder rung presented to the user and chosen by them, and recorded: configure (1) · config-closes-gap (2) · fork (3) · build-from-template (4, the default)
Pipeline design (the deliverable)
- Apps chosen: event streamer (
event) and/or batch job (job); optional full-export service noted - Streamer subscribes to only the needed Messages; transform is a pure, tested function
- Dedup key on the destination side (
resource.id+sequenceNumber);payloadNotIncludedre-fetch handled - Batch job windows on
lastModifiedAt+ cursor pagination + checkpoint - PII minimized; not logged; least-privilege read scopes; destination creds in
securedConfiguration
Build & verify (test-first)
- Built test-first on the commercetools-connect event/job/service references and their checklists
- Subscriptions registered idempotently in postDeploy; cleaned up in preUndeploy
- Round trip verified: one change → one row (no duplicate); batch window loads idempotently
The analytics egress pipeline
The one rule that spans the pipeline: dedup on the destination side
- For
notificationType: "Message"→resource.id+sequenceNumber(monotonic per resource; higher wins). - For Change payloads (
ResourceCreated/Updated/Deleted) →resource.id+version(noteversionis not sequential, but is comparable per resource).
MERGE/upsert current-state on resource.id keeping the row with the highest sequenceNumber/version. This is the analytics analogue of CRM's upsert-by-externalId and OMS's orderNumber idempotency — same principle, warehouse side.App 1 — the event streamer (event, near-real-time)
- Subscribe to the minimum Messages for the domains you export — register in idempotent
postDeploy(lifecycle-scripts.md). A MessageSubscription for specific message types (OrderCreated,OrderStateChanged,CustomerCreated, …) when you want typed, targeted events; a ChangeSubscription on a resource to capture all changes in one Subscription (spends less of the 50-Subscription budget). Message catalogs: Cart & Order, Customer, all Messages. On Connect the broker is injected and follows the deployment region — branch onCONNECT_SUBSCRIPTION_DESTINATIONand build the destination from the matching injected vars (CONNECT_GCP_*orCONNECT_AWS_TOPIC_ARN); don't hardcode a broker (event-applications.md, Pattern 7). - Decode + validate the envelope (base64
message.data→ JSON → resource ref → notificationType), and ack correctly (2xxfor handled/irrelevant; non-2xx only for a transient delivery/destination failure you want redelivered). Don't 4xx a subscribed-but-unhandled type into a redelivery loop; don't swallow a transient destination outage into silent data loss. → event-applications.md, Patterns 1–3. - Re-fetch the resource by
resource.id— never transform from the Message payload. It can be stale (no ordering) or absent: if a Message exceeds the queue size limit it is delivered withpayloadNotIncludedset and no resource data, so re-fetch is mandatory, not optional (event-applications.md, Pattern 5). Fetch current state, then transform. - Transform to the destination schema (see below) — a pure function, no network, unit-tested without a deployment.
- Deliver + emit the dedup key. Send to the destination's ingestion API with
resource.id+sequenceNumber(orversion) attached so the destination deduplicates.
App 2 — the batch / backfill job (job, scheduled + one-time history)
- Window on
lastModifiedAt. Query only resources changed since the last checkpoint: awherepredicate likelastModifiedAt >= :cursor(the documented integration best practice for "query for changes using timestamps"). Far cheaper than re-scanning the whole dataset. - Cursor-based pagination, not
offset. Offset degrades with depth and is capped at 10,000 records; cursor pagination (a stable sort +lastId/timestamp cursor) is consistent at any depth. Place the most restrictive predicate first (performance considerations). - Checkpoint the window (e.g. the last processed
lastModifiedAt+ id in a Custom Object) so a restart resumes and a re-run loads only the delta. Each unit idempotent via the same destination dedup key as the stream. - Two jobs, one shape: a one-time historical backfill (wide window, from the beginning) and an ongoing gap-repair run (narrow window since the last checkpoint). Keep the backfill separate from the stream.
- Not for heavy bulk. Job containers are capped (2 CPU / 4 GB) and the docs advise against jobs for memory-intensive bulk work (job-applications.md). For a large one-time history load, stream page-by-page to the destination (don't buffer the whole dataset), or orchestrate an external batch loader from the job rather than doing the heavy lift in-container.
The event→row transform (the real work)
Mapping a commercetools resource onto a destination schema is where the value is; keep it pure and tested:
- Localized strings, money, nested arrays. commercetools localized strings (
{ "en": … }),centAmount/currencyCodemoney, and nested line-item/address arrays rarely map 1:1 to a flat warehouse column. Decide per field: flatten to columns, keep as a JSON/VARIANT column, or explode line items into a child table. - Grain. One row per event (append-only fact table, dedup on the key) or one current-state row per resource (
MERGE/upsert onresource.id)? This is a destination-schema decision — pin it in Step 1. - Carry the dedup key as columns (
resource_id,sequence_number/version,last_modified_at) so warehouse-side dedup/merge and late-arrival handling are possible.
PII / GDPR (customer data)
- Minimize — export only the fields analytics needs; don't ship full customer profiles by default.
- Never log PII or destination credentials; structured logs carry only identifiers/correlation keys (security.md).
- Propagate erasure. If a
CustomerDeleted/ anonymization must reach the warehouse (right-to-be-forgotten), subscribe to it and delete/anonymize the destination rows — an analytics warehouse is a common place orphaned PII hides. - Destination creds in
securedConfiguration; least-privilege read scopes only (config-from-requirements.md).
Platform limits & realities to design around
- No Export API — you build the pipeline from Subscriptions + API queries; there is no bulk "dump" endpoint (Import and export).
- ~50 Subscriptions per Project — a soft limit, increasable on request. Prefer a ChangeSubscription per resource over many per-message-type Subscriptions to stay within budget.
payloadNotIncluded(payload omitted above the queue size limit, often around 256 KB) — re-fetch by id always; never rely on the payload being present (event-applications.md).- Messages query API is off by default — Messages are not persisted for querying unless you enable the feature in Merchant Center Developer Settings (enable querying Messages). Subscriptions deliver regardless; don't design a poller against the Messages API assuming it's queryable.
- Change History is not the source — separate host, token-rate-limited (429), explicitly not for event-driven/high-throughput use (see overview.md, and Audit Log overview).
Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
| No destination dedup key | Duplicate rows after redelivery / batch-stream overlap | Emit resource.id + sequenceNumber (or version); dedup/MERGE on the warehouse side |
| Transforming from the payload | Wrong/missing data; empty rows on payloadNotIncluded | Re-fetch the resource by resource.id; transform from current state |
| 4xx/5xx on an unhandled type | Redelivery loop flooding the destination | Ack (2xx) irrelevant messages; subscribe narrowly |
| Swallowing a destination outage | Silent gaps (events acked but never landed) | Non-2xx on transient destination failure so it redelivers; DLQ terminal failures |
offset pagination for backfill | Batch stalls / caps at 10,000 records | Cursor pagination + lastModifiedAt window |
| No checkpoint on the batch window | Re-run reloads everything / restart loses progress | Checkpoint the window; resume from it |
| One Subscription per message type | Burns the ~50-Subscription budget | ChangeSubscription per resource where you need all changes |
| Polling the Messages API | Empty results — querying is off by default | Use Subscriptions; only query Messages if the feature is enabled |
| Building off Change History | 429s; not event-driven; missing API-origin changes on Basic | Use Subscriptions + API queries, not the Audit Log |
| PII in the warehouse / logs | Compliance exposure; erasure gaps | Minimize fields; never log PII; propagate deletion/anonymization |
Route ≠ connect.yaml endpoint | Platform traffic 404s | Mount the router at the app's endpoint base path |
| Legacy SDK | Fails the commercetools-connect skill's pinned-version gate | @commercetools/platform-sdk@^8 + @commercetools/ts-client@^4 |
Test-first checklist (mirror in the suite)
Event streamer
- Decodes the base64 envelope; validates type; acks irrelevant messages (no loop)
- Re-fetches the resource by id (never transforms from the payload); handles
payloadNotIncluded - Transform is a pure, unit-tested function (localized strings / money / arrays handled)
- Emits the dedup key (
resource.id+sequenceNumber/version); duplicate delivery lands one row - Transient destination failure → non-2xx (redelivered); terminal → ack + DLQ/alert
- No PII in logged output; least-privilege read scopes
Batch / backfill job
- Windows on
lastModifiedAt; cursor pagination (notoffset) - Checkpoints the window; a re-run loads only the delta and is idempotent (same dedup key)
- Streams pages to the destination (no whole-dataset buffering); overlap lock + timeout headroom
- Boundary mocked (commercetools API + destination); suite runs with no deployment/secrets
Verify the analytics round trip
Check 1 — one change produces exactly one row (the stream)
- The row appears in the destination with the mapped fields correct (localized strings, money, addresses).
- The dedup key is present (
resource.id+sequenceNumber, orversion) on the row — this is what makes redelivery a no-op. - Redeliver the same envelope (or force a redelivery) and confirm the destination still has one row, not two. A second row is the tell that the warehouse-side dedup/merge key is missing (pipeline-architecture.md).
Check 2 — a batch window loads idempotently
job over a known lastModifiedAt window, then run it again over the same window:- The first run loads the delta; the second run adds no new rows (same dedup key → upsert/no-op).
- The checkpoint advanced, and a run started from the checkpoint fetches only changes since it — not the whole dataset.
- Cursor pagination reached the end of the window (no silent truncation at 10,000 rows — the sign someone used
offset).
The traps (behavior that looks like a bug — or hides one)
Trap 1 — no data at all → the Subscription isn't registered
postDeploy Subscription registration didn't run or failed, so no Messages are delivered. Confirm the Subscription exists (query Subscriptions), that it targets the right resource/message types, and that its destination matches the injected broker (Pub/Sub or SNS). A Subscription change takes up to a minute to take effect (Subscriptions).Trap 2 — duplicate rows are expected without a dedup key
Trap 3 — empty/partial rows → payloadNotIncluded, re-fetch missing
payloadNotIncluded). Fix: always re-fetch by resource.id and transform from current state (event-applications.md).Trap 4 — the Messages query API returns nothing → it's off by default
lastModifiedAt, rather than polling Messages.Trap 5 — acked but never landed → silent gap
2xx even when the destination delivery failed acks the Message away with no retry — events vanish. Confirm transient destination failures return non-2xx (redelivered) and terminal ones go to a DLQ/alert, not a blanket 200 (event-applications.md).Verification checklist
- One source change → exactly one destination row; dedup key present
- Redelivering the same envelope adds no second row (idempotent)
- Batch job over a window is idempotent on re-run; checkpoint advances; cursor pagination (no 10,000 cap)
- Subscription confirmed registered (right resource/types, destination matching the injected broker) when nothing arrives
-
payloadNotIncludedhandled by re-fetch (no empty/partial rows) - Transient destination failure redelivers; terminal failure DLQ'd — not acked into a gap
- No PII or destination credentials in logs; erasure propagates to the destination (if in scope)
Requirements → CRM connector config
connect.yaml values and a linking model. For a public connector these are its documented keys; for a build these are the keys and apps you define.The requirement → config map
| Requirement (Step 1) | Config / decision | Why |
|---|---|---|
| Which CRM + credentials | securedConfiguration: CRM API token / OAuth client id+secret | Secrets never in standardConfiguration, never hardcoded; this is PII-adjacent |
| Direction (out / in / both) | App composition (see below) | Direction is the architecture — it decides which apps exist |
| Source of truth | Field-level read/write ownership; read-only Custom Fields on the mastered side | Prevents the losing side from overwriting the master |
| Which entities/objects | Mapping module: Customer→Contact, Order→Deal | The core of the build; keep it a pure function |
| Which events sync (out) | Subscription message types registered in postDeploy | Only subscribe to what you sync |
| Deletion / consent | CustomerDeleted subscription (out) and/or erasure endpoint; consent field mapping | GDPR: deletion must propagate; consent must not be lost |
| Region + project | standardConfiguration: CTP_REGION; scopes via inheritAs | Host + client provisioning are region/project specific |
| Volume / latency | event/webhook (real-time) vs job (batch) + page size / backoff toggles | Batch vs broadcast is the documented trade-off |
Direction → app composition
- commercetools → CRM (outbound): one or more
eventapps. To catch every customer change, register a ChangeSubscription on thecustomerresource (deliversResourceCreated/ResourceUpdated/ResourceDeleted); to sync only specific changes, register MessageSubscriptions to the Customer messages you care about (CustomerCreated,CustomerEmailChanged,CustomerAddressAdded,CustomerDeleted, …). AddOrderCreatedif syncing orders. This is the broadcasting events pattern. - CRM → commercetools (inbound): a
serviceinbound webhook (CRM pushes changes; 5-min timeout, you authenticate the caller) or ajobthat polls the CRM for deltas on a schedule. Pick webhook when the CRM can push and you need low latency; poll when it can't or when batch is fine. - Initial migration: a
jobfor the one-time bulk backfill — kept separate from the ongoing sync, as the docs recommend, because backfill and delta need different pagination/throughput handling.
Source of truth and the linking model
- Link every synced pair by a stable key. Store the CRM record id in the Customer's
externalId(the field commercetools provides for exactly this — external-system references), or in a Custom Field ifexternalIdis already used. This key is your upsert / idempotency key in both directions — never blind-create. - When the CRM masters customer data, keep a Customer in commercetools anyway (it owns permissions, Cart/Order ownership, and promotions), and hold CRM-only attributes in Custom Fields marked read-only so storefront/MC edits can't diverge from the master.
- When commercetools masters, the CRM record is downstream; write to it, don't read authoritative fields back.
- Bi-directional is discouraged. If unavoidable, you must assign field-level ownership (which side wins per field) and add self-change filtering to break loops — see crm-contract.md.
The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / configuration; inheritAs), and place the file at the repository root — a nested connect.yaml silently fails to deploy.Native client provisioning (prefer this)
CTP_CLIENT_ID/SECRET. Scopes depend on direction:inheritAs:
apiClient:
scopes:
- manage_subscriptions # outbound: postDeploy registers the Subscriptions
- view_customers # outbound: re-fetch the Customer to build the CRM payload
- view_orders # outbound: re-fetch the Order (if syncing orders)
# inbound instead needs:
# - manage_customers # upsert Customers coming from the CRM
# - manage_types # only if postDeploy creates the Custom Type for CRM fields
configuration:
standardConfiguration:
- key: CRM_BASE_URL
description: CRM API base URL (or sandbox vs prod toggle)
securedConfiguration:
- key: CRM_API_TOKEN
description: CRM API token / OAuth client secret
Note:view_subscriptionsis not a valid standalone scope —manage_subscriptionscovers read + write. Declaring non-existent view scopes fails client creation. Grantmanage_customersonly on the inbound app that actually writes Customers.
Per-app config
deployAs:
- name: customer-syncer # outbound example
applicationType: event
endpoint: /customerSyncer
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the Subscriptions
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub or SNS"
- name: crm-migration # one-time backfill
applicationType: job
endpoint: /crmMigration
properties:
schedule: "0 3 * * *" # or run on-demand
Worked example (HubSpot, build, commercetools → CRM one-way)
OrderCreated out; one-time migration of existing customers; propagate deletion; near-real-time; europe-west1.gcp.Derived config:
inheritAs:
apiClient:
scopes: [manage_subscriptions, view_customers, view_orders]
configuration:
standardConfiguration:
- key: CRM_BASE_URL
description: "HubSpot API base (sandbox vs prod)"
securedConfiguration:
- key: CRM_API_TOKEN
description: HubSpot private-app token
deployAs:
- name: customer-syncer
applicationType: event
endpoint: /customerSyncer
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub or SNS"
- name: crm-migration
applicationType: job
endpoint: /crmMigration
customer (covering create/update/delete in one) plus an OrderCreated MessageSubscription; a job for the one-time backfill; scopes are exactly what the postDeploy registration and the resource re-fetches need — nothing more; the HubSpot token is securedConfiguration; each contact is upserted by externalId = HubSpot contact id, written back to the Customer on first sync. Marketing attributes that HubSpot masters are held in read-only Custom Fields so they aren't clobbered from commercetools (crm-contract.md).Is a public CRM connector enough?
Check live data first — don't answer from memory
The marketplace changes. Before deciding:
- Search the Connect marketplace (
marketplace.commercetools.com/connectors) and the integration docs via thedocs-searchscript or the Knowledge MCP. - Compare the requirements CRM-by-capability (which entities/objects, direction, field mapping, deletion/consent, real-time vs batch).
- Name the connector and version you checked — or record that none exists — in the requirements block.
The CRM landscape (verify, but this is the shape)
| Category | Examples on the marketplace | Typical default rung |
|---|---|---|
| Marketing / CDP / personalization | Klaviyo, Bloomreach, Mailchimp, Dynamic Yield, Relewise (verify live) | 1 (configure) if it covers the use case |
| Classic CRM (Salesforce, HubSpot, Dynamics 365, Zoho) | Typically no certified connector | 4 (build) |
| Anything else the user defines | Check the marketplace | Likely 4 unless a listing exists |
crm-integration Connect template (templates list: payment-integration, product-export, tax-integration, transactional-emails). So a build starts from plain apps, not a CRM-specific scaffold — see rung 4.The ladder (stop at the first rung that fits)
Rung 1 — Configure a public connector
deployment create --connector-key, or Merchant Center install) is the commercetools-connect skill's deployment-installation.md. Hand it the config you derive in config-from-requirements.md. Most CRM/CDP "customization" is field mapping and event selection — configuration, not code.Rung 2 — A gap that config can close
Rung 3 — Fork/extend a public connector (only if open source)
Rung 4 — Build for the CRM the user defines (the common case)
commercetools connect init then connect application add --type event|service|job), or start from the closest template and adapt. The nearest outbound shapes (react to a commercetools event → call an external API) are the transactional-emails and product-export templates; there is no inbound template, so build the CRM → commercetools direction as a plain service (webhook) or job (poll). Either way the Connect plumbing (lifecycle scripts, subscription/extension registration, envelope handling) is scaffolded; the CRM API calls and the mapping are what you write.- Outbound syncer (
event): commercetools message → CRM object upsert, idempotent byexternalId(see crm-contract.md). - Inbound app (
servicewebhook orjobpoll): CRM record → Customer upsert byexternalId, read-only mapped fields. - Migration job (
job): one-time bulk backfill, checkpointed. - Config + scopes (config-from-requirements.md).
Recording the decision
CRM: HubSpot · rung 4 (build) · checked marketplace 2026-07 — no public HubSpot connector; marketplace has marketing/CDP tools only · building an outbound event syncer + a migration job, CRM-as-master one-way, linked by externalId.
The CRM sync contract
The one rule that spans every app: upsert by externalId, never blind-create
externalId holds the CRM record id (and/or the CRM holds the commercetools id). At-least-once delivery means every message can arrive twice; a create-on-every-message design produces duplicate contacts and duplicate Customers. Look up by the key, update if present, create (and write the key back) if absent. This is the CRM analogue of tax's stable transaction_id.App 1 — the outbound syncer (commercetools → CRM, event)
What triggers it
event application: Connect provisions the queue/destination and delivers each message as an HTTP POST to the app's endpoint (port 8080). Register the Subscription in postDeploy (idempotent get-then-create):- ChangeSubscription on
customer— fires onResourceCreated/ResourceUpdated/ResourceDeletedfor any customer change. Simplest way to keep a full profile in sync. - MessageSubscriptions to specific Customer messages (
CustomerCreated,CustomerEmailChanged,CustomerAddressAdded,CustomerDeleted, …) when only certain changes should sync, or when you need the message's typed fields. - Add
OrderCreated(a MessageSubscription) if orders map to CRM deals/sales.
The delivery envelope
The payload shape depends on config, so don't hardcode one form (same as any event app):
- Transport wrapper (GCP):
{ "message": { "data": "<base64>", ... } }—message.datais base64-encoded JSON; decode first. - Message format: PlatformFormat (
{ notificationType, type, resource: { typeId, id }, ... }) or CloudEventsFormat ({ type: "com.commercetools.…", data: { … } }). Readtypeandresource.idfrom whichever you get, and validate the type before acting (ack-and-ignore the platform's test/probe messages). See Test an event application locally.
What it must do
- Re-fetch the Customer/Order by id from
resource.id— don't trust the payload. At-least-once delivery with no ordering means an olderResourceUpdatedcan arrive after a newer one; re-fetching the current resource makes the sync converge to the latest state instead of replaying stale deltas. - Map the resource to the CRM's object model (Customer→Contact/Lead, Order→Deal). Keep the mapping a pure function — no network — so it's unit-testable without a deployment or token.
- Upsert by
externalId. If the Customer has no CRM id yet, create the CRM record and write its id back to the Customer'sexternalId(or Custom Field) withsetExternalId/setCustomField. If it has one, update that CRM record. Idempotent on redelivery. - Ack correctly. Reply with a positive ack (
200/2xx; the Connect event contract treats102/200/201/202/204as "don't redeliver") for handled and irrelevant-but-acked messages. Return non-2xx only for transient failures you want redelivered.
Self-change filtering (only if bi-directional)
ResourceUpdated that this syncer would push straight back to the CRM — an infinite loop. Break it: mark connector-originated writes (e.g. a syncSource Custom Field, or compare against the last-synced hash/version) and skip re-syncing your own changes. This is the single nastiest CRM bug; a one-way design avoids it entirely, which is why the integration-patterns guidance discourages bi-directional sync.Deletion & PII (GDPR)
- If deletion is in scope, handle
CustomerDeleted/ResourceDeletedby deleting or anonymizing the CRM record — an erasure request must propagate, not leave orphaned PII downstream. - Customer data is PII: sync only the fields you need, keep credentials in
securedConfiguration, and never log PII or the CRM token (commercetools-connect's security.md). Carry marketing-consent flags through the mapping so a "do not contact" preference isn't lost.
App 2 — the inbound app (CRM → commercetools)
Two forms; pick per the CRM's capability and your latency need.
Form A — service inbound webhook (CRM pushes)
- Not an API Extension — no Extension is registered; the CRM calls your endpoint directly. The 5-min service timeout applies (not the 2 s extension limit).
- Authenticate the caller — the CRM calls you, so validate its proof (webhook signature / shared secret / JWT) in-app before writing (commercetools-connect's security.md). Never trust an unauthenticated inbound write to Customers. (
AuthorizationHeaderauthentication on an Extension'sHTTPdestination is the separate mechanism for the reverse direction — commercetools calling your endpoint as an Extension destination — not inbound-caller auth.) - Upsert the Customer by
externalId— look up by the CRM id, update or create. Usemanage_customersscope. - Idempotent — the same webhook may arrive twice; the upsert must be a no-op the second time.
- Read-only mapped fields — when the CRM masters these fields, store them in Custom Fields you treat as read-only elsewhere, so storefront/MC edits don't fight the master.
Form B — job poll (you pull deltas)
- A scheduled
job(properties.schedule) that queries the CRM for records changed since the last run, pages through them, and upserts each byexternalId. - Checkpoint the last-synced timestamp/cursor (e.g. in a CustomObject) so a restart resumes and you fetch only deltas, not the whole CRM each run.
- Owns its own overlap locking and 30-min timeout headroom (commercetools-connect job-applications.md).
App 3 — the migration job (one-time backfill)
job that pages the source (CRM or commercetools) in batches, upserts by externalId, and checkpoints progress so a failure resumes mid-run rather than restarting. Cleanse/validate records on the way (the migration guidance calls out cleansing Customer data). Respect CRM rate limits — batch and back off; a naive tight loop will get throttled or banned.Cross-cutting: mapping and rate limits
- Mapping is the real work. commercetools localized strings, addresses (array), customer groups, and Custom Fields rarely map 1:1 to a CRM's flat contact schema. Decide per field; keep it pure and tested.
- Rate limits & backoff. CRMs rate-limit hard. Give outbound calls a timeout, retry transient
429/5xxwith exponential backoff, and prefer the CRM's batch endpoints for migration.
Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
| Create-on-every-message | Duplicate contacts / duplicate Customers after redelivery | Upsert by externalId; write the id back on first sync |
| Trusting the payload | Stale/missing data synced; deltas replayed out of order | Re-fetch the resource by resource.id |
| No self-change filter (bi-directional) | Infinite sync loop, runaway API calls | Mark connector writes; skip your own changes — or go one-way |
| Envelope not decoded | Handler sees base64 garbage / crashes | Decode message.data (base64→JSON) before use |
| No message-type filter | Acting on unrelated/test messages | Validate type; ack-and-ignore the rest |
| Wrong ack | Handled message redelivered forever, or failures silently dropped | 2xx for handled/irrelevant; non-2xx only for retryable failures |
| Deletion not propagated | Orphaned PII in the CRM after erasure | Handle CustomerDeleted/ResourceDeleted → delete/anonymize |
| PII / token in logs | Compliance incident | Structured logs without PII; token in securedConfiguration |
| Unauthenticated inbound webhook | Anyone can write Customers | Validate signature/secret/JWT; least-privilege manage_customers |
| Migration mixed into ongoing sync | Throttling, restarts reload everything | Separate migration job; checkpoint; batch + backoff |
Route ≠ connect.yaml endpoint | Platform traffic 404s | Mount the router at the app's endpoint base path |
| Legacy SDK | Fails the commercetools-connect skill's pinned-version gate | @commercetools/platform-sdk@^8 + @commercetools/ts-client@^4 |
Test-first checklist (mirror in the suite)
Outbound syncer
- Decodes the base64 envelope; validates type; acks irrelevant messages
- Re-fetches the resource by id (doesn't map from the payload)
- Upserts by
externalId; creates-then-writes-back when absent; update when present - Duplicate delivery is a no-op (idempotent)
- Self-change filtering asserted if bi-directional
- Deletion → delete/anonymize (if in scope); no PII in logged output
Inbound (webhook or job)
- Webhook: rejects unauthenticated/invalid-signature calls (auth matrix)
- Upserts the Customer by
externalId; idempotent on repeat - CRM-mastered fields written as read-only Custom Fields
- Job: checkpoint advances; a re-run fetches only deltas
Migration job
- Pages + checkpoints; resumes mid-run after a simulated failure
- Upsert (not create) so a re-run doesn't duplicate
- Boundary mocked; suite runs with no deployment/secrets
CRM connector — integrate an external CRM (customer-sync-focused)
The mistake to internalize first: pick direction and source of truth before anything else. Almost every CRM-integration failure — duplicated contacts, overwritten edits, infinite sync loops — traces back to not having decided who masters customer data and which way it flows. commercetools' own integration guidance is explicit: pick a single source of truth per data domain, and avoid bi-directional syncs — they carry real conflict and loop risk.
The three shapes (by direction)
| Direction | Source of truth | Connect app(s) | Trigger |
|---|---|---|---|
| commercetools → CRM (push customers/orders out) | commercetools masters | event app(s) — the broadcasting events pattern | a ChangeSubscription on customer (all changes) or specific Customer messages; OrderCreated; … |
| CRM → commercetools (pull profiles/segments in) | CRM masters | service inbound webhook or job poll | CRM pushes a webhook, or a schedule polls the CRM for deltas |
| Initial migration (one-time bulk load) | either | job | On-demand / scheduled; separate from the ongoing sync |
Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<CRM terms from the user's request, e.g. 'CRM customer sync subscription CustomerCreated externalId integration patterns'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or the integration planning and patterns guide for deeper follow-up.Step 1 — Extract requirements (before any config or code)
- Which CRM, and do they have API access? Salesforce, HubSpot, Dynamics 365, Zoho, or another. Account, API credentials/OAuth app, sandbox vs production, and its rate limits.
- Direction and source of truth? commercetools → CRM, CRM → commercetools, or (discouraged) both. Who masters customer data? This is the single most consequential answer — it decides the app shape and which side's write wins on conflict.
- Which entities, and mapped to which CRM objects? commercetools Customer → CRM Contact/Lead/Person; Order → CRM Deal/Opportunity/Sales record; Cart → (rarely). Which fields on each side, and how localized names/addresses map.
- Ongoing sync, initial migration, or both? A one-time backfill of existing customers is a
job; ongoing delta sync is aneventor webhook/poll — usually both, built separately. - Which events trigger an outbound sync? Creation only, or every customer change and
OrderCreatedtoo? This maps to the two Subscription flavors: a ChangeSubscription on thecustomerresource fires on all changes (ResourceCreated/ResourceUpdated/ResourceDeleted); MessageSubscriptions target specific Customer messages (CustomerCreated,CustomerEmailChanged,CustomerAddressAdded,CustomerFirstNameSet,CustomerDeleted, …) when only certain changes matter. → decides which Subscriptions the connector registers. - Deletion / GDPR / consent? Must a
CustomerDeleted(or anonymize/erasure request) propagate to delete or anonymize the CRM record? Are there marketing-consent flags to carry? Customer data is PII — this is not optional to think about. - Volume and latency? Near-real-time (event/webhook) vs batch (nightly job); expected record counts (drives rate-limit and pagination handling). The docs frame the batch-vs-broadcast choice on exactly these axes.
- Anything special or non-standard? (always ask — open-ended) Multi-brand/multi-store contact separation, B2B accounts/company hierarchies, loyalty tiers or segments flowing in, double-opt-in, region/data-residency. Capture each as its own requirement line; don't force it into a slot above.
externalId, ongoing delta via events (or webhook), a separate migration job, deletion propagated — and say so explicitly.Step 1.5 — Is a public connector enough? (decide before wiring or building)
docs-search script / Knowledge MCP), and name the connector + version you checked.crm-integration template — you scaffold plain apps and adapt. See connector-selection.md.- Public connector covers everything → install + configure (Step 2). Don't build. Installing it (CLI auth, scopes,
deployment create) is the commercetools-connect skill's deployment-installation.md; it is not theconnectorstagedflow. - A public connector exists, gap looks like a capability → prove it isn't config first. Field mapping, which events sync, and consent handling are often connector settings → back to rung 1. See config-from-requirements.md.
- A public connector exists with a genuine gap config can't close, and it's open source → fork/extend it; add only the delta and deploy as an Organization connector. Don't rebuild a working connector. Hand off to commercetools-connect for the build/publish lifecycle.
- No public connector for the CRM (the common case — Salesforce, HubSpot, Dynamics, Zoho, or any CRM the user defines) → build it. There is no CRM template, so scaffold plain
event/service/jobapps (or start from the closest outbound template —transactional-emailsorproduct-export— and adapt; there is no inbound template). You implement the CRM API calls and the mapping. The exact contract and gotchas are in crm-contract.md.
Step 2 — Derive the config from the requirements
connect.yaml values (for the chosen connector or your own), with a one-line why for each. The full mapping is in config-from-requirements.md. Key decisions that live here:- App composition from direction — which
event/service/jobapps you deploy, per the table above. - Least-privilege API-client scopes via
inheritAs.apiClient.scopes— outbound needs read scopes (view_customers,view_orders) +manage_subscriptions; inbound needsmanage_customers. Don't hand-supply amanage_projectadmin client. - Secrets in
securedConfiguration— the CRM API token / OAuth client secret issecuredConfiguration, neverstandardConfiguration, never hardcoded. This is customer-PII-adjacent; treat it accordingly. - The linking model — store the CRM record id in the Customer's
externalId(or a Custom Field), and hold CRM-only attributes in Custom Fields marked read-only when the CRM is master.
Step 3 — Price the async contract (reference)
externalId, never blind-create), at-least-once with no ordering (re-fetch the resource by id; don't apply deltas from a possibly-stale payload), and loop avoidance if bi-directional (a CRM-originated write must not re-trigger an outbound sync). Full contract: crm-contract.md.Step 4 — Build/verify the sync apps (the main body of work), test-first
externalId, re-fetch by id, ack semantics on the event endpoint, self-change filtering, deletion propagation — are invisible at the call site and tedious to reproduce by hand. Each is one cheap assertion. Write the test first.- Outbound syncer(s) (
event) — on a customer change (ChangeSubscriptionResourceUpdated/ResourceCreated, or specific Customer messages) orOrderCreated, re-fetch the resource by id, map it to the CRM's object model, upsert byexternalId(idempotent), write the CRM id back to the Customer, ack correctly. - Inbound app (
servicewebhook orjobpoll) — authenticate the caller (webhook) or page the CRM (job); upsert the Customer byexternalId; set CRM-mastered fields read-only; be idempotent. - Migration job (
job) — page the source in bulk, upsert deltas, checkpoint so a restart resumes; keep each unit idempotent.
Step 5 — Verify the round trip
externalId, update a field and confirm the delta propagates once (no loop), and — if in scope — delete/anonymize and confirm it propagates. See verification.md, which also covers the traps that look like bugs but aren't (a sync loop from missing self-change filtering, CRM rate-limit throttling, sandbox data quirks).References
| Need | Reference |
|---|---|
| Is a public connector enough?: live-marketplace check; why classic CRMs are usually build-from-scratch; the ladder | connector-selection.md |
Requirements → config mapping: direction → app composition, source of truth, externalId/Custom Fields linking, scopes, secured config; the connect.yaml envelope; worked example | config-from-requirements.md |
The sync contract: outbound syncer, inbound webhook/poll, migration job; idempotent upsert by externalId, re-fetch by id, ack semantics, self-change/loop filtering, deletion/PII, mapping; full pitfall catalog | crm-contract.md |
Verify the round trip: record linked by externalId, delta propagates once, deletion propagates; the loop / rate-limit / sandbox traps | verification.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
Adding another CRM later means reusing this same tree — the direction-driven app shapes, the linking model, and the flow do not change; only the CRM's object model and API calls do.
Checklist
Requirements
- CRM chosen + API access/credentials (sandbox vs prod, rate limits) known
- Direction and source of truth decided (one-way preferred; bi-directional only with a stated reason)
- Entity → CRM-object mapping (Customer→Contact, Order→Deal) and field mapping identified
- Ongoing sync vs initial migration (usually both) decided; trigger events listed
- Deletion/GDPR/consent handling decided; PII scope minimized
- Asked the open-ended "anything special?" question; each special requirement its own line
- Requirements block written and confirmed; specials fed into the Step 1.5 fit-check
Connector fit (decide before wiring/building)
- Checked live marketplace + integration docs (not memory); named the connector + version (or confirmed none exists)
- Ladder rung presented to the user and chosen by them: configure (1) · config-closes-gap (2) · fork/extend (3) · build (4)
- For a classic CRM with no connector, recognized this is a build, not a marketplace install
Config (the deliverable)
- App composition matches the direction (event / webhook / poll / migration job)
- Only documented
connect.yamlenvelope fields; file at the repo root -
inheritAs.apiClient.scopesleast-privilege (read +manage_subscriptionsoutbound;manage_customersinbound) - CRM credentials in
securedConfiguration; toggles/region instandardConfiguration - Linking model chosen:
externalId(or Custom Field) as the stable key; CRM-only fields read-only when CRM is master
The sync apps (build test-first — do not write a function body before its red test)
- Outbound syncer re-fetches by id, upserts by
externalId, writes the CRM id back, acks with200/2xx - Inbound app authenticates the caller (webhook) / pages the CRM (job); upserts the Customer by
externalId; idempotent - Self-change filtering in place if bi-directional (no loop)
- Deletion/anonymization propagated if in scope
- Boundary mocked; suite runs with no deployment/secrets
Verification
- Counterpart record appears, linked by
externalId - A field update propagates exactly once (no loop)
- Deletion/anonymization propagates (if in scope); no PII in logs
Verify the CRM round trip
Check 1 — the counterpart record appears, linked by externalId
Create a Customer in commercetools (outbound) or in the CRM (inbound), let the sync run (or, locally without Pub/Sub, POST the base64 message envelope to the syncer directly), then:
- The counterpart record exists in the destination (a Contact in the CRM, or a Customer in commercetools).
- The link is set: the commercetools Customer's
externalId(or Custom Field) holds the CRM record id — this is what makes the next change an update, not a duplicate. A record that appears with noexternalIdwritten back is the tell that the upsert-and-link step is missing; the next sync will create a duplicate. - The mapped fields match (localized names, address, consent flags).
Check 2 — a delta propagates exactly once (no loop)
Update one field (e.g. last name) on the mastering side and watch the other side:
- The change appears on the counterpart — once.
- No duplicate record is created (proves upsert-by-
externalId, not create). - Watch for a loop. In a bi-directional setup, a missing self-change filter shows up as a burst of writes ping-ponging between the systems (rising API call counts, version numbers climbing on their own). One update should produce one write per direction and then stop. If it doesn't, the self-change filter is missing (crm-contract.md).
Check 3 — deletion / anonymization propagates (if in scope)
CustomerDeleted/ResourceDeleted event.The traps (behavior that looks like a bug — or hides one)
Trap 1 — the sync loop (looks fine at first, then floods)
Trap 2 — rate-limit throttling looks like "sync stopped"
429 throttling, not a logic bug — check for backoff/retry and batch endpoints (crm-contract.md), and confirm the job resumes from its checkpoint rather than restarting.Trap 3 — sandbox data quirks
Verification checklist
- Counterpart record created and
externalIdwritten back (link established) - A field update propagates once; no duplicate record created
- One change settles to one write per direction (no loop) — asserted, not just observed
- Deletion/anonymization propagates; no orphaned PII (if in scope)
- No PII or CRM token in logs
- Migration resumes from checkpoint; respects rate limits (batch + backoff)
Requirements → email connector config
connect.yaml values. For a ready-made connector these are its documented keys; for a from-template build these are the keys you define. The template's own key names are called out below.The requirement → config map
| Requirement (Step 1) | Config / decision | Why |
|---|---|---|
| Which ESP + credentials | securedConfiguration: EMAIL_PROVIDER_API_KEY (or the ESP's user/pass/region) | Secrets never in standardConfiguration, never hardcoded |
| Sender identity | securedConfiguration: SENDER_EMAIL_ADDRESS (must be a verified domain/sender in the ESP) | Unverified senders are rejected or land in spam |
| Which emails | The Subscription message types (in code/postDeploy) and one template id per email | The handler routes by message type to a template |
| ESP-hosted templates | securedConfiguration: one *_TEMPLATE_ID per email type | Points each email at its ESP template |
| Localization | Language source (customer.locale / order / store) → per-locale template id or a locale field passed to the ESP | Right language per recipient (template hardcodes en-US — a gap) |
| Order-state target states | Config or code list of the states that trigger a send | OrderStateChanged fires on every transition; gate it |
| Region + project | standardConfiguration: CTP_REGION; scopes via inheritAs | Host + client provisioning are region/project specific |
| Token emails in scope | manage_customers scope (mint token is a write); token-validity ≤ 60 min if you want the value in the Message | See email-contract.md |
Scopes — least-privilege depends on which emails you send
postDeploy and handlers use — no more. Build the set from the emails in scope:| Capability | Scope | Needed when |
|---|---|---|
Register the Subscription in postDeploy | manage_subscriptions | always |
| Re-fetch the Order to build order emails | view_orders | any order email (confirmation, state/shipment, refund) |
| Re-fetch the Customer to build customer emails | view_customers | registration / any email that reads customer data |
| Mint an email/password token in the handler | manage_customers | verification / password-reset emails (supersedes view_customers) |
Why token emails need write access. The token value is only present in theCustomerEmailTokenCreated/CustomerPasswordTokenCreatedMessage when the token's validity is ≤ 60 minutes (customer password reset). For longer-lived tokens the value is omitted, so the connector must create the token itself (aPOST .../password-tokenwrite) — which is what the official template does. If your reset tokens are short-lived and you read the value straight from the Message,view_customersis enough; if you mint in the handler, you needmanage_customers.
The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys, and place the file at the repository root — a nested connect.yaml silently fails to deploy.Native client provisioning (prefer this)
CTP_CLIENT_ID/SECRET:inheritAs:
apiClient:
scopes:
- manage_subscriptions # postDeploy registers the email Subscription
- view_orders # handlers re-fetch the Order for order emails
- manage_customers # only if minting verification/password tokens; else view_customers
CTP_CLIENT_ID/SECRET/SCOPE as secured config; migrating to inheritAs.apiClient.scopes is the more native, lower-maintenance form and is worth doing on a from-template build.The event app
deployAs:
- name: mail-sender
applicationType: event
endpoint: /mailSender
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the Subscription
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy # deletes it
configuration:
standardConfiguration:
- key: CTP_REGION
description: commercetools Composable Commerce API region
securedConfiguration:
- key: EMAIL_PROVIDER_API_KEY
description: API key for the email service provider
- key: SENDER_EMAIL_ADDRESS
description: Verified sender address shown in the email
- key: ORDER_CONFIRMATION_TEMPLATE_ID
description: ESP template id for order confirmation
# …one template id per email type in scope
Subscription destination is injected, not declared. Aneventapp's queue/topic is provisioned by Connect, which injectsCONNECT_SUBSCRIPTION_DESTINATIONandCONNECT_GCP_TOPIC_NAME/CONNECT_GCP_PROJECT_ID(orCONNECT_AWS_TOPIC_ARNfor SNS) at deploy time. Build the Subscription destination from those inpostDeploy— don't add them toconnect.yamland don't hardcode a broker (event-applications.md).
Worked example (SendGrid, from-template build)
customer.locale; europe-west1.gcp; short-lived reset tokens minted in the connector.Derived config:
inheritAs:
apiClient:
scopes: [manage_subscriptions, view_orders, manage_customers] # manage_customers: mints the reset token
configuration:
standardConfiguration:
- key: CTP_REGION
description: commercetools Composable Commerce API region
securedConfiguration:
- key: EMAIL_PROVIDER_API_KEY
description: SendGrid API key
- key: SENDER_EMAIL_ADDRESS
description: Verified sender (e.g. no-reply@shop.example)
- key: ORDER_CONFIRMATION_TEMPLATE_ID_EN
description: SendGrid dynamic template id — order confirmation (en)
- key: ORDER_CONFIRMATION_TEMPLATE_ID_DE
description: SendGrid dynamic template id — order confirmation (de)
- key: ORDER_SHIPMENT_TEMPLATE_ID_EN
description: SendGrid dynamic template id — shipment (en)
- key: ORDER_SHIPMENT_TEMPLATE_ID_DE
description: SendGrid dynamic template id — shipment (de)
- key: PASSWORD_RESET_TEMPLATE_ID_EN
description: SendGrid dynamic template id — password reset (en)
- key: PASSWORD_RESET_TEMPLATE_ID_DE
description: SendGrid dynamic template id — password reset (de)
deployAs:
- name: mail-sender
applicationType: event
endpoint: /mailSender
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
manage_subscriptions for the postDeploy registration, view_orders to re-fetch orders for confirmation/shipment, manage_customers because the reset email mints a short-lived token in the handler — nothing more. Two template ids per email keep localization explicit; the handler picks _EN/_DE from customer.locale with EN as the fallback. Shipment emails must be gated on the shipment reaching Shipped (not fired on every OrderStateChanged) — see email-contract.md. Provider-exact send-call shape (dynamic templates, dynamic_template_data, idempotency header): providers.md.Is a ready-made email connector enough?
Do this in order — don't skip, don't answer from memory
- List the connectors from the live Connect marketplace (
marketplace.commercetools.com/connectors) + the email docs via thedocs-searchscript or the Knowledge MCP — the email / messaging / marketing listings. - Present them to the user: name · vendor · service · certification/status, and flag whether any is a transactional email connector or only marketing/CRM platforms.
- Confirm the approach with the user — use as-is (rung 1) · config-closes-gap (rung 2) · modify/fork (rung 3) · create from template (rung 4). Do not presume the rung.
- Record platform/ESP · rung · connector + version checked · why.
The email landscape (verify, but this is the shape)
commercetools/connect-email-integration-template). It wires the Connect plumbing (the event app, the Subscription registration, message routing to per-type handlers, config) and leaves one thing stubbed: the actual call to the ESP (GenericHandler.sendMail). It is a template, not a marketplace install — you deploy your own customization of it.| Situation | Default rung |
|---|---|
| A marketplace connector exists for the exact ESP and covers the emails needed | 1 (configure) |
| A marketplace connector exists but source-available and has a real gap | 3 (fork/customize) |
| No marketplace connector for the ESP (the common case) | 4 (build from the official template) |
sendMail.The ladder (stop at the first rung that fits)
Rung 1 — Configure a ready-made connector
deployment create --connector-key, or Merchant Center install) is the commercetools-connect skill's deployment-installation.md. Hand it the config you derive in config-from-requirements.md (API key, sender, template IDs).Rung 2 — A gap that config can close
Rung 3 — Fork/customize (the "customize the code" path)
OrderShipmentStateChanged, a custom message), localize by customer.locale, attach a PDF invoice, gate order-state emails on specific target states, or swap the ESP — and the source is available (the official template always is). Fork it, add only the delta, deploy as an Organization connector. Don't rebuild the plumbing. Hand off to commercetools-connect for the fork's build/stage/publish lifecycle. The per-email contract and pitfalls to preserve are in email-contract.md.Rung 4 — Build from the template (the "create a new one" path)
mail-sender event app with the Connect plumbing done — lifecycle scripts, Subscription registration, envelope decode, message→handler routing, per-email personalization mapping — but the ESP call is a stub you implement (sendMail), plus retry/recovery is yours.What you actually write on rung 4:
- The ESP send call in
sendMail: template id +to/from+ personalization data → the provider's transactional-send API (providers.md). - Your delivery-semantics choice (ack-first vs ack-after-success + dedupe) and any retry (email-contract.md).
- Localization, state-filtering on order-state emails, and token-email handling if in scope.
- Config + least-privilege scopes (config-from-requirements.md).
Recording the decision
Email: SendGrid · rung 4 (build from template) · checked marketplace 2026-07 — no dedicated SendGrid email connector listed; using the official transactional email template and implementing the SendGrid dynamic-templates send call · emails: order confirmation + shipment + password reset.
The one-app email contract
mail-sender event app must do, and the pitfalls that silently break each. Grounded in the official transactional email template. This is a pure event app, so read it on top of event-applications.md (envelope decode, ack table, idempotency, re-fetch, self-change filtering) — this file adds only the email-specific layer. Provider send-call shapes are in providers.md.What triggers it — one Subscription, several message types
postDeploy (idempotently — the template deletes-by-key then recreates), keyed on a stable subscription key, with the destination built from the injected vars for whichever broker CONNECT_SUBSCRIPTION_DESTINATION reports (event-applications.md). Subscribe to only the message types you send email for — the broker shouldn't deliver noise you'll just ack-and-ignore.The canonical message set (grounded in the template) and what each email is:
resourceTypeId | Message type | |
|---|---|---|
| Registration / welcome | customer | CustomerCreated |
| Email verification (double opt-in) | customer-email-token | CustomerEmailTokenCreated |
| Password reset | customer-password-token | CustomerPasswordTokenCreated |
| Order confirmation | order | OrderCreated (and OrderImported if you email on imports) |
| Order state / cancellation | order | OrderStateChanged |
| Shipment | order | OrderShipmentStateChanged |
| Refund / returns | order | ReturnInfoAdded, ReturnInfoSet |
messages: [{ resourceTypeId, types: [...] }]. Message reference: customer messages, cart & order messages.What the handler must do
- Decode & validate the envelope, then branch on Message type to the right email (the template uses a handler factory). Ack anything you don't handle (see the delivery-semantics section, and event-applications.md).
- Re-fetch the resource by id —
getOrderById(message.resource.id),getCustomerById(order.customerId). Don't trust the payload: it can be stale (no ordering) or omitted (payloadNotIncluded). The template does this correctly. Token emails are the exception (below). - Build the personalization data (recipient, name, order lines, totals) and pick the template id for the email type (and locale).
- Send via the ESP (providers.md) behind a tight timeout — the event ack timeout is 10 s; a hung ESP call must abort, not stall the handler.
The central decision: delivery semantics for a non-idempotent send
2xx ack for an event app means "don't redeliver" — including 202. (This is the opposite of an API Extension, where 202 fails the operation. Same number, different contract, because event ack semantics differ from extension response semantics. The tax calculator is an Extension; this email app is an event — don't carry the 202 rule across.)Option A — ack first, then send (at-most-once; the template's default)
202 at the top of the handler, before validation and before the ESP call:response.status(HTTP_STATUS_SUCCESS_ACCEPTED).send(); // 202, immediately
// …then decode, route, re-fetch, sendMail — errors only get logged
- Guarantees: never double-sends on redelivery (the message is already acked).
- Cost: a transient ESP failure (or a throw) silently drops the email — the platform will not redeliver. Fire-and-forget.
- Use when a duplicate is worse than a miss, or you add your own retry/DLQ around the send.
Option B — send, then ack on success (at-least-once + dedupe)
Ack only after the ESP confirms; return non-2xx on a transient failure so the broker redelivers:
try {
await sendMail(...); // confirmed accepted by the ESP
res.status(204).send(); // ack — safe to stop
} catch (err) {
if (isTransient(err)) { res.status(503).send(); return; } // redeliver
res.status(200).send(); // permanent: ack + alert, don't loop
}
- Guarantees: transient failures retry — the email eventually goes out.
- Cost: redelivery will re-send unless you dedupe. Email sends aren't idempotent at the platform, so make them so:
- ESP idempotency key — pass a stable key (e.g.
resource.id+sequenceNumber, or the message id) so the ESP collapses duplicates (providers.md — SendGrid, others support this). - or a sent-marker — record "sent" on a stable key the target can check before re-sending (a Custom Field/Custom Object), re-checking live state — never an in-process set (event-applications.md).
- ESP idempotency key — pass a stable key (e.g.
Token emails (verification & password reset) — the value isn't always in the Message
CustomerEmailTokenCreated / CustomerPasswordTokenCreated Message only when the token's validity is ≤ 60 minutes (customer password reset). Otherwise it's omitted. Two designs:- Read from the Message — create tokens with ≤ 60-min validity so the value is present;
view_customersis enough. Simplest, and the emailed token is the one the user's action created. - Mint in the handler — call
POST .../password-token(or email-token) yourself and email that value (what the template does). Works for any validity, but needsmanage_customers(a write), and the emailed token differs from the triggering one. Under at-least-once this also means a redelivery mints another token — dedupe, or accept that older tokens stay valid until used (creating a token doesn't invalidate older ones by default).
Never log the token value (PII/secret) — see hygiene below.
Order-state emails must be gated on the target state
OrderStateChanged and OrderShipmentStateChanged fire on every transition. The template routes all of them to one handler and emails unconditionally — so a shopper gets an email on every internal state change. After re-fetching, gate on the specific target state you mean:const order = await getOrderById(id);
if (order.shipmentState !== "Shipped") return ack(); // only the shipment email
// or: if (order.orderState !== 'Cancelled') return ack();
Localization
DEFAULT_LOCALE = 'en-US' for line-item names and picks one template id per email — so every email is English. For multi-language:- Read the language from
customer.locale(fallback: order/store locale, then a default). - Resolve localized strings from
LocalizedStringfields (lineItem.name[locale]) with a fallback, and pick a locale-specific template id (or pass the locale to the ESP if the template branches internally).
Hygiene: PII, consent, deliverability
- Don't log PII or tokens. The template logs full message bodies and email addresses; scrub recipient addresses, names, and any token value from logs (log the
resource.id/sequenceNumbercorrelation key instead). → security.md, observability-operations.md. - Keep it transactional. Transactional emails (order/account/token) generally don't require marketing opt-in; marketing/promotional email does and belongs in a marketing platform, not this connector. Don't quietly turn a transactional connector into a marketing sender.
- Sender must be verified.
SENDER_EMAIL_ADDRESSmust be a verified sender/domain in the ESP or mail is rejected or spam-filed. - Bounces/complaints are the ESP's to report. If you need them reflected back into commercetools, that's a separate inbound-webhook
serviceapp consuming the ESP's event webhook — out of scope for the sender.
Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
| Ack-first + failed send | Email silently never arrives; no retry | Option B with dedupe for drop-intolerant emails, or add your own retry/DLQ |
| At-least-once without dedupe | Customer gets 2+ copies | ESP idempotency key or a sent-marker on a stable key |
Emailing on every OrderStateChanged | Shopper spammed on internal transitions | Gate on the target state after re-fetch |
| Trusting the payload | Wrong/missing data; throws on payloadNotIncluded | Re-fetch the Order/Customer by resource.id |
| Token value read from a >60-min Message | Empty reset link | Use ≤60-min validity, or mint the token in the handler (manage_customers) |
Hardcoded en-US | Wrong-language emails | Localize by customer.locale + locale-specific template id |
| Subscribing to whole resources | Broker delivers noise; every message hits a handler | Register only the exact message types |
Non-idempotent postDeploy | Duplicate/failed Subscription on redeploy | Delete-by-key then create, or get-then-skip |
| Logging recipient/token | PII & secret leakage | Log the correlation id only; scrub addresses and token values |
| Unverified sender | Sends rejected / spam-filed | Verify the sender domain in the ESP |
| Legacy SDK | Fails the commercetools-connect skill's pinned-version gate | @commercetools/platform-sdk@^8 + @commercetools/ts-client@^4 |
Test-first checklist (mirror in the suite)
- Decodes the base64 envelope; validates & branches on message type; acks unhandled types
- Delivery semantics asserted — ack-first or ack-after-success + dedupe; the failure path proven (no silent drop / no double-send for the chosen mode)
- Re-fetches Order/Customer by id; handles
payloadNotIncluded - Order-state/shipment emails gated on the target state (asserted)
- Correct template id + recipient + personalization data per email type; money/date formatting
- Localization picks the right template/strings from
customer.localewith fallback - Token email: value sourced correctly (Message ≤60 min, or minted) and never logged
-
postDeployregisters only the needed message types, idempotently; boundary mocked; suite runs with no deployment/secrets
Email connector — integrate a transactional email service (event-driven)
- mail-sender (an
eventapp driven by a Subscription on Customer/Order Messages) — commercetools delivers a Message to the app's queue; the handler picks the email type from the Message type, re-fetches the resource, builds the personalization data, and calls the ESP to send. Email is always asynchronous: sending must never block or fail a checkout, so there is no API Extension here.
mail-sender, applicationType: event, endpoint: /mailSender) and what the email integration tutorial documents. Because it's a pure event app, everything in event-applications.md applies directly — this sub-area layers the email-specific decisions (which Messages, delivery semantics for un-idempotent sends, templating, localization, PII) on top.The mistake to internalize first: delivery semantics. An ESP send is not idempotent — call it twice and the customer gets two emails. Event delivery is at-least-once, so the same Message will sometimes arrive twice. How you acknowledge decides everything: ack before sending (at-most-once — never double-sends, but a transient ESP failure silently drops the email) vs. ack after a confirmed send and dedupe on redelivery (at-least-once — retries failures, but you must dedupe or customers get duplicates). The official template acks first (fire-and-forget). Pick deliberately per email type — see email-contract.md.
Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<email terms from the user's request, e.g. 'transactional email connector subscription messages order confirmation customer registration'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or https://docs.commercetools.com/tutorials/connect-email-integration for deeper follow-up.Step 1 — List the publicly available connectors (required — do this first, before ESP or requirements)
marketplace.commercetools.com/connectors + the email docs, via the docs-search script / Knowledge MCP) and present the user a concrete list of the available email / messaging / marketing connectors — each with its name, vendor, the service it integrates, and its certification/status. Call out explicitly whether any is a transactional email connector or whether the listings are only marketing/CRM platforms (as of writing they are marketing-oriented; the classic transactional ESPs — SendGrid, Mailgun, AWS SES, Postmark — have no dedicated connector and are build-from-template). How to check and the current landscape: connector-selection.md.Step 2 — Confirm the approach: use, modify, or create (required — do not skip, do not assume)
- Use a public connector as-is (rung 1) → install + configure it; the emails are authored and sent inside that platform. Installation (CLI auth, scopes,
deployment create) is the commercetools-connect skill's deployment-installation.md. - Modify / fork an existing connector (rung 3) → the "customize the code" path: fork a source-available connector or the official template, add only the delta (message types, localization, attachments, a different ESP), deploy as an Organization connector. (First rule out rung 2 — a gap that config can close.)
- Create a new one from scratch (rung 4) → build from the transactional email template, implementing the stubbed ESP call for the service they define.
Email is template-first: unlike tax (where Avalara/Vertex ship certified connectors), most ESPs have no dedicated transactional connector, so "create new" or "modify the template" is the common outcome — but you still run Steps 1–2 and let the user decide; never skip the fit-check or presume the rung.
Step 3 — Extract requirements (after the approach is chosen)
Which emails, on which events, in which language, is downstream of business facts. Each maps to a config key in Step 4 or a decision in the contract. Ask the user (don't assume):
- Which ESP, and why? SendGrid, Mailgun, AWS SES, Postmark, Brevo, Mailchimp/Mandrill, … Do they already have an account + API key + a verified sender domain? (Deliverability, template model, and pricing differ; see providers.md.)
- Which emails do they need? Each maps to a commercetools Message — the canonical set the template covers: registration (
CustomerCreated), email verification (CustomerEmailTokenCreated), password reset (CustomerPasswordTokenCreated), order confirmation (OrderCreated), order state / shipment (OrderStateChanged,OrderShipmentStateChanged), refund/returns (ReturnInfoAdded,ReturnInfoSet). See email-contract.md. - For order-state emails, which target states trigger a send?
OrderStateChangedfires on every transition — you only want to email on specific ones (e.g.Confirmed,Cancelled, shipmentStateShipped). Without a state gate you spam customers on every internal state change. - Delivery guarantee per email: is a duplicate email acceptable, or is a dropped email worse? Token/reset emails are high-stakes (a dropped reset email blocks the user); marketing-ish confirmations tolerate at-most-once. → drives the ack strategy (email-contract.md).
- Templating & localization. Are templates authored in the ESP (dynamic/stored templates, referenced by ID — the template's model) or rendered in the connector? Multiple languages? What's the language source —
customer.locale, the order/store locale, or a single default? (The template hardcodesen-US— a gap to close.) - Region and project? e.g.
europe-west1.gcp, projectmy-project. - Token-email validity. For verification/reset emails: the token value only rides the Message when the token's validity is ≤ 60 minutes; otherwise the connector must mint the token itself (a write). → scope + design impact, email-contract.md.
- Anything special? (always ask — open-ended) Multi-store/brand (different sender/template per store), B2B/business-unit recipients, attachments (PDF invoice), unsubscribe/consent handling, bounce/complaint feedback back into commercetools, a batch/digest email (a separate
jobapp). Capture each as its own requirement line; don't force it into a slot above.
customer.locale with an en fallback → at-most-once for confirmations, and prioritized retry for token emails → and say so explicitly.connect.yaml and app.Step 4 — Derive the config from the requirements
connect.yaml values, with a one-line why each. Full mapping and provider key names: config-from-requirements.md. Key decisions here:- Least-privilege scopes via
inheritAs.apiClient.scopes(not hand-suppliedCTP_CLIENT_ID/SECRET). Which scopes depends on which emails: alwaysmanage_subscriptions(postDeploy registers the Subscription);view_orders/view_customersto re-fetch for order/registration emails;manage_customersif token emails mint a token. - Secrets in
securedConfiguration: ESP API key, and (per template) the per-email template IDs; region and toggles instandardConfiguration.
Step 5 — The Subscription & message routing (reference)
postDeploy on the exact resourceTypeId + message types you send email for — nothing more (the broker shouldn't deliver noise). Then the handler branches on the Message type to the right email. Full registration shape and routing: email-contract.md.Step 6 — Build the one app (the main body of work), test-first
- Subscription registration (
postDeploy) — idempotent (delete-then-create by a stable key, or get-then-skip); the exact message types; destination from the injected vars for the brokerCONNECT_SUBSCRIPTION_DESTINATIONreports. - The handler — decode the base64 envelope; validate & branch on Message type; ack per your chosen delivery semantics; re-fetch the Order/Customer by id; map to the ESP's send request (template id + personalization data); call the ESP behind a tight timeout.
Step 7 — Verify the round trip
References
| Need | Reference |
|---|---|
| Is a ready-made connector enough?: configure vs fork vs build-from-template; the template-first reality; live-marketplace check | connector-selection.md |
Requirements → config mapping: which messages, ESP + template IDs, sender, least-privilege scopes; the connect.yaml envelope; worked example | config-from-requirements.md |
| The one-app contract: subscription registration, message→email routing, the at-most-once vs at-least-once decision, token-email gotcha, state filtering, localization, PII; full pitfall catalog | email-contract.md |
| ESP specifics: SendGrid / Mailgun / AWS SES / Postmark send-call shape, ESP-hosted templates, idempotency keys; provider comparison | providers.md |
| Verify the round trip: per-event checks; the no-subscription and sandbox-doesn't-deliver traps; duplicate/silent-drop symptoms | verification.md |
| Generic event-app contract (envelope, ack table, idempotency, re-fetch) — this sub-area builds on it | event-applications.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
Checklist
Connector fit-check (do this FIRST — do not skip or reorder)
- Listed the live marketplace connectors (not from memory) and presented them to the user with name · vendor · service · status
- Flagged whether any is a transactional email connector or only marketing/CRM platforms
- Confirmed the approach with the user: use as-is (1) · config-closes-gap (2) · modify/fork (3) · create from template (4) — before gathering ESP/build details
- Recorded platform/ESP · rung · connector + version checked · why
Requirements (after the approach is chosen)
- ESP/platform chosen + API key + verified sender domain; region + project
- The exact emails/Messages listed; for order-state emails, the target states that trigger a send named
- Delivery guarantee decided per email (duplicate-tolerant vs drop-intolerant) — for use-as-is, owned by the platform
- Templating model (ESP-hosted by ID vs in-connector) and localization source identified
- Token-email validity (≤ 60 min → value in Message; else mint in connector) understood
- Asked the open-ended "anything special?" question; each special its own line
- Requirements block written and confirmed
Config (the deliverable)
-
inheritAs.apiClient.scopesleast-privilege for the emails in scope (manage_subscriptions+ the read/write the handlers need) - ESP key + template IDs in
securedConfiguration; region/toggles instandardConfiguration -
connect.yamlat the repo root; only documented envelope fields
The one app (build test-first)
- Subscription registered idempotently on only the needed message types; destination from the injected vars for the broker
CONNECT_SUBSCRIPTION_DESTINATIONreports - Ack strategy implemented and asserted (no silent drop; no double-send)
- Order-state emails gated on the target state; handlers re-fetch by id
- Boundary mocked; suite runs with no deployment/secrets
Verification
- Each event produces an email to the right recipient with the right template + data
- Understood: no Subscription → no email; ESP sandbox/test mode may not deliver
Email service provider specifics
GenericHandler.sendMail(sender, recipient, templateId, data) — the call to the ESP. This file is the shape of that call per provider. The commercetools side is identical regardless of ESP; only this outbound call changes. Verify each provider's exact API against its own docs (linked) — ESP APIs evolve and are outside commercetools' docs.The shape (ESP-agnostic)
Every transactional ESP send is the same four things:
- Auth — the API key from
EMAIL_PROVIDER_API_KEY(secured config), typically aBearerheader. - From/To —
SENDER_EMAIL_ADDRESS(a verified sender) → the recipient (order.customerEmail/customer.email). - A template reference — the ESP-hosted template id for this email type (+ locale), from secured config.
- Personalization data — the key/value object your handler built (order number, name, line items, totals, token/link) merged into the template by the ESP.
resource.id + sequenceNumber, or the message id.Providers
SendGrid (dynamic templates)
- Send:
POST https://api.sendgrid.com/v3/mail/send,Authorization: Bearer <key>. - Template:
template_id(a dynamic templated-…); personalization goes inpersonalizations[].dynamic_template_data. - Idempotency: SendGrid supports a batch/idempotency mechanism; at minimum set a stable custom arg / batch id to help dedupe.
- Docs: SendGrid Mail Send.
// sendMail body sketch
{
from: { email: senderEmailAddress },
personalizations: [{ to: [{ email: recipient }], dynamic_template_data: data }],
template_id: templateId,
}
Mailgun (stored templates)
- Send:
POST https://api.mailgun.net/v3/<domain>/messages, HTTP basic auth (api:<key>), form-encoded. - Template:
template= the stored template name; variables viah:X-Mailgun-Variables(JSON) orv:params. - Docs: Mailgun sending.
AWS SES (templated email)
- Send:
SendTemplatedEmail/SendBulkTemplatedEmail(SDK v3) or the SESv2SendEmailwith aTemplate. - Template:
Templatename +TemplateData(JSON string); auth via the app's AWS credentials (secured config). - Docs: SES send templated email.
Postmark (templated, transactional-first)
- Send:
POST https://api.postmarkapp.com/email/withTemplate,X-Postmark-Server-Token: <key>. - Template:
TemplateIdorTemplateAlias+TemplateModel; separate message streams for transactional vs broadcast. - Docs: Postmark templated email.
Cross-provider summary
| Dimension | SendGrid | Mailgun | AWS SES | Postmark |
|---|---|---|---|---|
| Template ref | template_id (d-…) | template name | Template name | TemplateId/TemplateAlias |
| Data field | dynamic_template_data | Mailgun variables | TemplateData | TemplateModel |
| Auth | Bearer key | basic api:<key> | AWS creds | server token header |
| Payload | JSON | form-encoded | SDK | JSON |
| Localization | one template id per locale, or a locale in the data | same | same | same |
sendMail(sender, recipient, templateId, data) seam — swapping ESP is a change to this one function, not the connector. Keep the mapping (resource → data) provider-independent and unit-tested; keep only the HTTP/SDK call provider-specific.Checklist
-
sendMailimplemented against the chosen ESP's transactional-send API; key fromEMAIL_PROVIDER_API_KEY - Sender is a verified domain/sender in the ESP
- ESP-hosted template referenced by id (per email type, per locale) unless rendering in-connector is justified
- Personalization data mapping is a pure, unit-tested function; only the HTTP/SDK call is provider-specific
- Idempotency key passed on the send when using at-least-once delivery (dedupe — email-contract.md)
- Outbound call has a tight timeout under the 10 s event ack budget
Verify the email round trip
Check 1 — the Subscription exists and points at the connector
- Query Subscriptions and confirm one exists for your key with the expected
messages(resourceTypeId+types) and a destination pointing at the deployed app. - If it's missing,
postDeploydidn't run or failed — check the deployment logs. This is the number-one "nothing happens" cause.
Check 2 — each event produces the right email
| Trigger | Confirm | |
|---|---|---|
| Registration | Create a Customer | ESP shows a send to the customer's email with the registration template |
| Email verification | Create an email token (≤60 min to get the value in the Message) | Send contains a working verification link/token |
| Password reset | Create a password token | Send contains a working reset link/token |
| Order confirmation | Place an order (convert a cart) | Send with the order number, line items, totals |
| Shipment | Transition the order's shipmentState to Shipped | Send fires only on the target state, not other transitions |
| Refund/return | Add/set return info | Send fires; other order changes don't |
OrderCreated/CustomerCreated envelope straight to the app's endpoint and assert the ESP call — see test an event application locally.The traps (correct-looking behavior that is a bug, or vice-versa)
Trap 1 — ESP sandbox / test mode accepts but doesn't deliver
202/200, your connector logs success — but nothing is delivered. An empty inbox after a "successful" send is expected in sandbox. Verify the contract (accepted, right payload) in sandbox; verify delivery on a live key sending to a real inbox (then clean up).Trap 2 — duplicate emails (at-least-once without dedupe)
Trap 3 — silent drops (ack-first + a failing send)
Trap 4 — an email on every state change
orderState/shipmentState before sending; ack the rest (email-contract.md).Trap 5 — empty reset/verification links
manage_customers) — email-contract.md.Verification checklist
- Subscription registered with the expected message types and destination (else: no email at all)
- Each in-scope event produces a send visible in the ESP feed to the right recipient
- Right template, right language, real data rendered (no empty placeholders)
- Order-state/shipment emails fire only on the target state
- Reset/verification links actually work (token present and valid)
- Delivery confirmed on a live key to a real inbox (sandbox/test mode may not deliver)
- No duplicates under redelivery; no silent drops on transient ESP failure
- No PII/token values in logs
Requirements → gift card connector config
connect.yaml values. For a public connector these are its documented keys; for a from-template build these are the keys you define. Grounded in the gift card integration template and the Voucherify connector.The requirement → config map
| Requirement (Step 1) | Config / decision | Why |
|---|---|---|
| Which gift card system + credentials | securedConfiguration: system API secret/token (+ any standardConfiguration application/program id, base URL) | Secrets never in standardConfiguration, never hardcoded |
| Region + project | standardConfiguration: CTP_PROJECT_KEY, CTP_AUTH_URL, CTP_API_URL, CTP_SESSION_URL, CTP_JWKS_URL, CTP_JWT_ISSUER | Hosts + token validation are region/project specific |
| Currency scope | standardConfiguration: a currency key (template: MOCK_CONNECTOR_CURRENCY; Voucherify: VOUCHERIFY_CURRENCY) | One deployment is typically scoped to one currency; multi-currency ⇒ multiple deployments or a converting system |
| Fallback payment method | (Merchant Center Payment Integration config, not connect.yaml) | The gift card integration is configured alongside a PSP integration — a Checkout Application setting |
| Balance / redeem | Both are core processor routes (always built) | The minimum gift-card contract |
| Refund / reverse on cancel-return | Implement the Payment Intents modifyPayment operations | Post-order lifecycle goes through the Payment Intents API, not the enabler |
| Partial + multiple cards | Redeem logic + remainder handling (code, not a single toggle) | The card may not cover the total; the remainder goes to the fallback method |
The commercetools connection block
standardConfiguration:
- key: CTP_PROJECT_KEY
description: commercetools project key
required: true
- key: CTP_AUTH_URL
description: commercetools Auth URL
required: true
default: https://auth.europe-west1.gcp.commercetools.com
- key: CTP_API_URL
description: commercetools API URL
required: true
default: https://api.europe-west1.gcp.commercetools.com
- key: CTP_SESSION_URL
description: Session API URL
required: true
default: https://session.europe-west1.gcp.commercetools.com
- key: CTP_CLIENT_ID
description: commercetools client ID (scopes below)
required: true
- key: CTP_JWKS_URL
description: JWKs URL for JWT validation
required: true
default: https://mc-api.europe-west1.gcp.commercetools.com/.well-known/jwks.json
- key: CTP_JWT_ISSUER
description: JWT issuer for JWT validation
required: true
default: https://mc-api.europe-west1.gcp.commercetools.com
securedConfiguration:
- key: CTP_CLIENT_SECRET
description: commercetools client secret
required: true
europe-west1.gcp; a project in another region needs the matching auth/api/session/mc-api hosts, or session validation and JWKS lookup fail. This is the most common misconfiguration.Scopes
CTP_CLIENT_ID description):manage_payments manage_orders view_sessions view_api_clients manage_checkout_payment_intents introspect_oauth_tokens
manage_payments— the processor creates and updates the Payment (redeem transactions).manage_orders— associate the Payment with the Order / read order context.view_sessions+introspect_oauth_tokens— validate the Checkout Session on/balanceand/redeem.view_api_clients— resolve the calling client during session/JWT validation.manage_checkout_payment_intents— acceptPOST /payment-intents/:idcalls from the Payment Intents API (refund/reverse). Automated reversals additionally require the connector to support thereversePaymentaction.
CTP_CLIENT_ID/SECRET. Grant only the scopes the routes above use — nothing broader like manage_project.The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / configuration), and place the file at the repository root — a nested connect.yaml silently fails to deploy. The two apps:deployAs:
- name: enabler
applicationType: assets
- name: processor
applicationType: service
endpoint: /
configuration:
standardConfiguration: [ ... CT block + currency + system config ... ]
securedConfiguration: [ ... CT client secret + system secret ... ]
assets (a static bundle, no endpoint); the processor is service with endpoint: / (routes are mounted at the root — /status, /balance, /redeem, /payment-intents/:id). Keep the router mounted at / to match, or Checkout's calls 404.Worked example (build from template, in-house gift card ledger)
europe-west1.gcp.Derived processor config (CT block from above, plus):
standardConfiguration:
- key: GIFTCARD_CURRENCY
description: Currency this deployment handles (EUR)
required: true
- key: GIFTCARD_API_URL
description: Base URL of the store-credit ledger API
required: true
securedConfiguration:
- key: GIFTCARD_API_KEY
description: Store-credit ledger API key
required: true
securedConfiguration, its base URL in standardConfiguration; scopes exactly the six the routes need. The fallback pairing (Stripe) is configured in the Checkout Application's Payment Integrations, not here — flag that the gift card integration must not be shipped alone (overview.md). For Voucherify's exact keys instead (VOUCHERIFY_APPLICATION_ID, VOUCHERIFY_API_URL, VOUCHERIFY_CURRENCY, VOUCHERIFY_SECRET_KEY), see its connect.yaml.Use it, customize it, or build it?
Check live data first — don't answer from memory
Supported systems and connectors change. Before deciding:
- Search the Connect marketplace (via the Merchant Center Connect view) and the gift-card docs via the
docs-searchscript or the Knowledge MCP. Filter for Public Connectors of type Gift Cards. - Compare the requirements system-by-capability (balance, redeem, partial redemption, multiple cards, refund/reverse, currency, region).
- Name the connector and version you checked, and record it in the requirements block.
The gift card landscape (verify, but this is the shape)
| System | Public connector? | Source available? | Default rung |
|---|---|---|---|
| Sample / mock (commercetools) | ✅ Yes — for test/PoC only | n/a (simulation) | Use for PoC; never production |
| Voucherify | ✅ Yes (commercetools/connect-giftcard-integration-voucherify) | ✅ Open source | 1 (use) — or 3 (fork) since the source is open |
| In-house / store-credit / other platform | ❌ Usually none | — (only the generic template) | 4 (build from template) |
The ladder (stop at the first rung that fits)
Rung 1 — Use a public connector directly (Voucherify; sample for PoC)
deployment create) are the commercetools-connect skill's deployment-installation.md; it is not the connectorstaged flow.Valid-10000-EUR (success), Valid-0010000-EUR (forced failure), Valid-0-EUR (no balance) drive the outcome and no payment is made (docs). Never ship it as the production integration.Rung 2 — A gap that config can close
Rung 3 — Customize/fork the public connector (Voucherify)
Rung 4 — Build a new one from the gift card template (the common case)
@commercetools/connect-payment-sdk) ships both apps with the Connect plumbing done — session/JWT authentication, the commercetools client, the route skeleton, the Payment lifecycle wiring — but the calls to your gift card service are stubs you implement (the template ships a mock in their place).What you actually write on rung 4:
- The processor balance/redeem logic:
code→ your system's balance/redeem API, response → the commercetools Payment transaction (see giftcard-contract.md). - The processor post-order operations (
modifyPayment): refund/reverse against your system, if in scope. - The enabler UI for capturing the gift card code (and PIN, if the system needs one).
- Config + scopes (config-from-requirements.md).
Recording the decision
Gift card: Voucherify · rung 1 (use) · checked marketplace 2026-07 — Voucherify public connector present and covers balance/redeem/refund for our single-currency (EUR) store · configuring it, paired with the existing Adyen PSP integration.
Gift card: in-house store-credit ledger · rung 4 (build) · checked marketplace 2026-07 — no public connector for our ledger · building both apps from the gift card template, paired with the existing Stripe integration.
The two-app gift card contract
@commercetools/connect-payment-sdk (TypeScript, Fastify).The rule that frames everything: never ship alone
App 1 — the processor (service, endpoint /)
endpoint: /).Routes and their auth (the auth split is the thing to get right)
| Route | Auth | Purpose |
|---|---|---|
GET /status | JWT | Health / liveness |
POST /balance | Session (SessionHeaderAuthenticationHook) | Body { code } → check the card's balance against the gift card system; report the amount and whether it covers the cart |
POST /redeem | Session | Body { code, redeemAmount } → redeem value against the system and record it on the Payment |
POST /payment-intents/:id | JWT / OAuth2 (manage_checkout_payment_intents) | modifyPayment({ paymentId, data }) → post-order operations (refund, reverse/rollback) driven by the Payment Intents API |
sessionId, not CT credentials); payment-intents is server/Checkout-driven and authenticated with a JWT/OAuth token carrying manage_checkout_payment_intents. Wiring session auth on the payment-intents route (or vice versa) breaks the corresponding flow. Use the SDK's session/JWT hooks as preHandler per route — don't hand-roll validation.Balance
- Take
{ code }(and PIN/security code if the system requires one), call the gift card system's balance API, and return the balance plus whether it's sufficient for the current cart. Checkout surfaces this to the shopper via thegift_card_balance_successMessage (amount,isBalanceSufficient). - Balance is a read — it must not redeem or reserve value. A common bug is redeeming on the balance call.
- Handle zero/invalid/expired codes cleanly →
gift_card_balance_error, so the shopper can try another card or method.
Redeem
- Take
{ code, redeemAmount }, redeemredeemAmountagainst the system, and record it on the commercetools Payment as a transaction (the processor owns the Payment). Checkout emitsgift_card_redeem_successon success. - Partial redemption is the norm. If the balance is less than the cart total, redeem the available amount and leave a remainder — the fallback PSP integration covers it. Redeeming a card must not assume it settles the whole cart.
- Multiple cards: a cart may redeem several cards in sequence, each reducing the outstanding amount. Each redeem is its own transaction on the Payment.
- Be idempotent. A retried redeem (network hiccup, double-submit) must not double-charge the card. Key redemption on a stable identifier (the code + amount + payment/cart context, or the system's own idempotency key) so a replay is a no-op, and reconcile against the Payment's existing transactions before adding another.
Post-order operations (/payment-intents/:id)
- Refund and reverse/rollback happen after the Order exists, through the Payment Intents API →
modifyPayment. This returns redeemed value to the card (refund) or unwinds a redemption (reverse). - Automated reversals require the connector to declare support for the
reversePaymentaction; implement it only if the requirements include automatic unwinding of authorized-but-not-completed payments. - These operations update the Payment's transactions to reflect the new state; keep them idempotent on the intent/operation id.
Keep the mapping pure and testable
App 2 — the enabler (assets)
/balance and /redeem with the session. Checkout loads it based on the Payment Integration configuration; it can also be embedded in a custom frontend. It is a thin slice — it holds no CT credentials and no gift-card-system secrets; it only carries the sessionId and talks to the processor. Sensitive operations stay server-side in the processor. Keep the enabler's job to: render, capture the code, call balance, call redeem, and surface the result.Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
| Gift card integration shipped alone | Shopper stuck when balance < total; "checkout is broken" | Configure a fallback PSP Payment Integration alongside it (Checkout Application config) |
| Redeem rejects when balance < total | Partial payments impossible; valid cards refused | Redeem the available amount, leave a remainder for the fallback method |
Session auth on /payment-intents (or JWT on /balance) | The corresponding flow 401s | Session hook on balance/redeem; JWT/OAuth (manage_checkout_payment_intents) on payment-intents |
| Balance call redeems/reserves value | Balance shrinks just from checking | Balance is a read; never mutate the card on /balance |
| Non-idempotent redeem | Double-submit or retry double-charges the card | Idempotency key on redeem; reconcile against existing Payment transactions |
| Wrong-region CT hosts / JWKS / issuer | Session validation or JWKS lookup fails; every call 401s | Match CTP_AUTH/API/SESSION_URL, CTP_JWKS_URL, CTP_JWT_ISSUER to the project region |
| Currency mismatch | Redeem fails or applies the wrong amount | One deployment per currency; validate the cart currency against the deployment's currency |
Router not mounted at / | Checkout's calls 404 | Processor endpoint: /; mount routes at the root |
| Using the sample connector in production | No real redemption happens; Valid-… codes "work" but nothing settles | Sample is PoC-only; build/use a real connector for production |
| Legacy SDK / no connect-payment-sdk hooks | Hand-rolled auth drifts from the platform contract | Use @commercetools/connect-payment-sdk session/JWT hooks; pin current CT SDK versions (commercetools-connect skill gate) |
Test-first checklist (mirror in the suite)
Processor
-
/balanceis read-only, session-authenticated; reports amount + sufficiency; handles zero/invalid/expired codes -
/redeemsession-authenticated; records the redeem transaction on the Payment - Partial redemption leaves the correct remainder; multiple cards accumulate transactions
- Redeem is idempotent — a replayed request is a no-op (asserted)
-
/payment-intents/:idrefund/reverse JWT/OAuth-authenticated; produces the right transaction (if in scope) - Boundary (gift card system, CT APIs) mocked; suite runs with no deployment/secrets
Enabler
- Renders code (and PIN) input; carries only the session; holds no secrets
- Calls
/balancethen/redeem; surfaces balance/redeem errors to the shopper
Gift card connector — integrate a gift card management system
- processor (a
service) — the backend middleware to the gift card system. It checks balances, redeems value, and owns the commercetools Payment object (creates it, adds/updates transactions). Its behavior is driven by itsconnect.yamlconfig; it authenticates callers with a Checkout Session (balance/redeem) or a JWT/OAuth token (post-order operations via the Payment Intents API). - enabler (an
assetsbundle) — a browser JS library that renders the gift-card input UI and calls the processor. Checkout loads it based on your Payment Integration configuration; it can also be embedded directly in a custom frontend.
The rule to internalize first: never ship a gift card integration alone. A gift card Payment Integration must always be configured alongside at least one other Payment Integration (docs). A gift card often can't cover the full cart total; without a fallback method the shopper is stuck when the balance falls short. This is a configuration requirement, not a nice-to-have.
Gift card connectors are consumed by Checkout
gift_card_balance_*, gift_card_redeem_*) you subscribe to via the Browser SDK, and drives post-order operations (refund/reverse) through the Payment Intents API. If your team is wiring the storefront side of that (rendering the integration, reacting to gift card messages), that's the commercetools-checkout skill; this sub-area is the connector behind it.Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<gift card terms from the user's request, e.g. 'gift card connector checkout balance redeem payment method'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or https://docs.commercetools.com/checkout/connectors-and-applications for deeper follow-up.Step 1 — Extract requirements (before any config or code)
Gift card behavior is downstream of business facts, and the wrong default silently produces a broken checkout. Extract these first; each maps to a config key in Step 2 or a rung in Step 1.5. Ask the user (don't assume):
- Which gift card system, and why? A dedicated gift-card/loyalty platform (e.g. Voucherify), an in-house ledger, or a store-credit service. Do they already have an account + API credentials?
- Is there a public connector for it? Voucherify has one; most systems don't. This decides configure-vs-build (Step 1.5) and changes the effort estimate — say it early.
- Region and project? e.g.
europe-west1.gcp, projectmy-project— the CT API/Auth/Session hosts and JWKS/issuer config are region-specific. - Which fallback payment method(s)? The gift card integration is configured alongside another Payment Integration (PSP). Which one, and is it already deployed? (Non-negotiable — see the rule above.)
- Currency handling? A single connector deployment is typically scoped to one currency; a multi-currency storefront may need multiple deployments or a system that handles conversion. Confirm the currencies in scope.
- Partial + multiple cards? Should one cart accept multiple gift cards, and combine a card with a PSP payment for the remainder? (Usually yes — confirm the system supports partial redemption.)
- Post-order operations? On cancellation/return, should redeemed value be refunded/reversed back to the card? → drives whether you implement the Payment Intents
refundPayment/reversePaymentoperations, not just balance+redeem. - Anything special or non-standard? (always ask — open-ended) Expiry rules, per-transaction caps, PIN/security-code entry, fraud checks, combining with discount codes, B2B store credit, or a specific gift-card account/program id. Capture each as its own requirement line; don't force it into a slot above.
Step 1.5 — Use a public connector, customize one, or build a new one? (decide before wiring or building)
docs-search script / Knowledge MCP), and name the connector + version you checked.- Use a public connector directly → if a Public Connector of type Gift Cards covers the system (e.g. Voucherify) or you just need a proof of concept (the sample gift card connector — see below), install + configure it (Step 2). Don't build.
- Public connector, gap looks like a capability → prove it isn't config first. Many "missing" behaviors (currency, which operations are enabled, fallback pairing) are
connect.yamlvalues or Merchant Center Payment Integration settings → back to rung 1. - Customize/fork a connector's code → genuine gap config can't close and an open-source connector exists for the system → fork it, add only the delta, deploy as an Organization connector. Don't rebuild a working one.
- Build a new one from the template → no connector for the system → build from the gift card integration template. The template ships both apps with the Connect + session/JWT plumbing done; you implement the calls to your gift card service and the mapping. This is the common case.
Valid-10000-EUR and makes no real payment (docs). Use it to validate the checkout wiring before a real system exists; it is not a production integration.Step 2 — Derive the config from the requirements
connect.yaml values for the chosen connector (or your own), with a one-line why for each. The mapping, the CT envelope keys, least-privilege scopes, and a worked example are in config-from-requirements.md. Key decisions that live here:- The commercetools connection block (
CTP_PROJECT_KEY,CTP_AUTH_URL,CTP_API_URL,CTP_SESSION_URL,CTP_JWKS_URL,CTP_JWT_ISSUER) — region-specific; the session/JWKS/issuer values are what let the processor validate Checkout sessions and Merchant Center JWTs. - Currency config (one deployment ≈ one currency for the template/Voucherify) and the gift-card-system credentials.
- Secured vs standard config — the gift card system API secret and the CT client secret are
securedConfiguration; URLs, currency, and behavioral toggles arestandardConfiguration. - The API-client scopes the connector needs (
manage_payments,manage_orders,view_sessions,view_api_clients,manage_checkout_payment_intents,introspect_oauth_tokens).
Step 3 — Build/verify the two apps (the main body of work), test-first
- Processor — balance + redeem (session-authenticated):
POST /balance({ code }) checks the gift card system and reports the balance and whether it covers the cart;POST /redeem({ code, redeemAmount }) redeems value against the system and records it on the commercetools Payment. Handle insufficient balance (partial redemption, remainder to the fallback method) and zero balance. - Processor — post-order operations (
POST /payment-intents/:id, JWT/OAuth,manage_checkout_payment_intents): implementmodifyPaymentfor the operations in scope (refund, reverse/rollback). Driven by the Payment Intents API, not by the enabler. - Enabler — the frontend touchpoint that renders the gift-card input and calls the processor with the session. Thin slice; contract is in giftcard-contract.md.
Step 4 — Verify the round trip
References
| Need | Reference |
|---|---|
| Use / customize / build?: the ladder (public connector · fork · build-from-template), the sample connector, live-marketplace check, landscape table | connector-selection.md |
Requirements → config mapping: the CT connection block, currency, gift-card-system credentials, least-privilege scopes; the connect.yaml envelope; worked example | config-from-requirements.md |
| The two-app contract: enabler (session-driven UI) + processor (balance/redeem session-auth, payment-intents modifyPayment for refund/reverse); partial/multiple cards; idempotency; full pitfall catalog | giftcard-contract.md |
| Verify the round trip: balance → redeem → Payment transaction → fallback remainder → refund/reverse; the sample-only-simulates and no-fallback traps | verification.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
| Storefront side: rendering the gift-card Payment Integration, reacting to gift card Messages | commercetools-checkout |
Adding another gift card system later means adding a sibling provider note and extending the selection table — the two-app architecture, the contract, and the flow do not change.
Checklist
Requirements
- Gift card system chosen + account/credentials; region + project
- Fallback Payment Integration identified (gift card is never shipped alone); currency scope confirmed
- Partial + multiple cards decided; post-order refund/reverse decided
- Asked the open-ended "anything special?" question; each special requirement its own line
- Requirements block written and confirmed; specials fed into the Step 1.5 fit-check
Use / customize / build (decide before wiring/building)
- Checked live marketplace + gift-card docs (not memory); named the connector + version
- Ladder rung presented to the user and chosen by them: use public (1) · config-closes-gap (2) · fork/customize (3) · build from template (4)
- For a real gap on a system with a public connector, chose fork over rebuild
- Used the sample connector only for PoC, not production
Config (the deliverable)
- Only documented
connect.yamlenvelope fields; file at the repo root - CT connection block + JWKS/issuer set for the region; currency configured
- Scopes =
manage_payments,manage_orders,view_sessions,view_api_clients,manage_checkout_payment_intents,introspect_oauth_tokens - Gift-card-system secret + CT client secret in
securedConfiguration; URLs/currency instandardConfiguration
The two apps (build test-first — do not write a function body before its red test)
-
/balanceand/redeemsession-authenticated; redeem creates/updates the Payment idempotently - Partial redemption leaves a remainder for the fallback method; zero balance handled
-
/payment-intents/:idrefund/reverse implemented (if in scope), JWT/OAuth-authenticated - Boundary mocked; suite runs with no deployment/secrets
Verification
- Balance check returns the correct amount; redeem creates a Payment transaction
- Remainder covered by the fallback method; refund/reverse returns value (if in scope)
- Understood: the sample connector only simulates; a gift card shown with no fallback is a config error, not a bug
Verify the gift card round trip
Check 1 — balance reads correctly (and doesn't redeem)
{ code } to the processor's /balance with a valid session):- The response reports the correct balance and whether it covers the cart. In Checkout this surfaces as the
gift_card_balance_successMessage withamountandisBalanceSufficient. - The balance did not change from checking it. A balance call that redeems or reserves value is a bug — check again and confirm the amount is unchanged.
Check 2 — redeem records a Payment transaction, remainder to fallback
{ code, redeemAmount } to /redeem with a session), then inspect the cart's Payment:- A commercetools Payment exists with a transaction for the redeemed amount (
gift_card_redeem_successin Checkout). - If the balance was less than the cart total, the outstanding amount is left for the fallback Payment Integration (the PSP), and completing the order requires paying that remainder. A short balance should never block checkout — it should route the rest to the fallback method.
- Multiple cards, if used, each add their own transaction.
Check 3 — refund / reverse (if in scope)
- A refund returns value to the card and records a refund transaction on the Payment.
- A reverse/rollback unwinds a redemption; automated reversals require the connector to support
reversePayment.
The two traps (correct behavior that looks like a bug)
Trap 1 — the sample connector only simulates
Valid-10000-EUR simulates success, Valid-0010000-EUR forces a failure, Valid-0-EUR simulates a zero-balance card (docs). Amounts are in the currency's minor units (e.g. 500 = 5 CHF). So "it works with the sample but nothing settles in our gift card system" is expected — the sample never calls a real system. Use it to prove the checkout wiring, then verify real redemption against the actual connector.Trap 2 — a gift card shown with no fallback looks broken
Verification checklist
- Balance check returns the correct amount and does not change the balance
- Redeem records a transaction on a commercetools Payment
- Short balance leaves a remainder covered by the fallback PSP integration (checkout not blocked)
- Multiple cards each record a transaction (if in scope)
- Refund/reverse via the Payment Intents API returns/unwinds value (if in scope)
- Understood: the sample connector only simulates — verify real redemption against the real connector
- Understood: a gift card with no fallback method is a config error, not a connector bug
Requirements → seller model → marketplace connector config
connect.yaml. For a public connector these are its documented keys; for a fork or a build these are the keys and apps you define.Model the marketplace domain onto commercetools
commercetools has no "seller" resource. Sellers are modeled with Channels, Stores, and Custom Objects — pick per requirement, not all of them by default.
| Marketplace concept | Model as | Why / the trap |
|---|---|---|
| Seller / vendor | a Channel keyed seller-<marketplaceSellerId>, with role InventorySupply (+ ProductDistribution if the seller prices independently) | Channels are the scoping primitive for stock and price. The key is your idempotency key. A Channel can't be deleted while referenced by an InventoryEntry, Line Item, Store, or Price — so offboarding means removing it from Stores and stopping sync, not deleting it |
| Seller profile data (business name, rating, logo, opening hours, address) | a CustomObject (container per entity type, key = marketplace seller id), and/or Custom Fields on the seller Channel | POST /custom-objects is create-or-update on container+key, so it is idempotent for free — the right home for arbitrary seller payloads. Put anything the storefront filters or scopes on the Channel instead |
| Seller storefront / isolated assortment / seller-scoped MC access | a Store per seller (+ Product Selections) | Only when isolation is a requirement — it also gives per-seller Merchant Center team permissions. Limits allow it at scale (300,000 Stores per Project), but a Store is capped at 100 Product Selections, so don't model one Product Selection per seller inside a shared Store |
| Offer / listing (a seller's sellable item) | a Product/Variant keyed on the marketplace's stable listing id — or, when several sellers sell the same SKU, one Product with per-seller Prices and InventoryEntries | Duplicating the Product per seller is the classic marketplace modeling mistake: it splinters search, ratings, and reporting. One Product + N seller offers is the default |
| Offer stock | an InventoryEntry per sku + supplyChannel (the seller's Channel) | Stock is tracked per SKU and optionally per supply channel — that pair is the per-seller stock record. A Cart bound to a Store only sees stock from that Store's supply channels |
| Offer price | a Price / StandalonePrice with channel = the seller's distribution Channel | A price with no channel is visible in every Store — the leak that shows one seller's price on another seller's storefront. Always set the channel on seller prices |
| Marketplace order coming in | Order Import with orderNumber = the marketplace order id, store set, and per-line supplyChannel (+ line custom fields for the marketplace line id) | orderNumber is your dedupe key — Order has no top-level externalId. Store-referenced import also filters languages, prices, and inventory to that Store's channels |
| Order handed off to a seller / marketplace | the Order's syncInfo via updateSyncInfo — channel (a Channel with role OrderExport, or OrderImport for inbound), externalId = the marketplace id, syncedAt | This is the platform-native "already exported" marker. Use it instead of inventing a custom field, and read it back to skip re-exporting |
| Per-seller fulfilment progress | Line Item state (ItemStates) per line, plus Deliveries/Parcels per shipment | One multi-seller Order has many independent fulfilment tracks; a single order-level state can't express "seller A shipped, seller B cancelled" |
| Commission, payout, settlement | not in commercetools — the marketplace/PSP owns them | commercetools tracks Payment status only; it has no payout ledger. Sync commission values onto the Order/line as Custom Fields if reporting needs them, but don't build payouts here |
Two limits that kill naive designs
- 50 Subscriptions and 25 Extensions per Project. Never one per seller. Register one Subscription per message type and fan out to sellers inside your handler.
- Store-scoped connectors don't scale per-seller either. The
product-exporttemplate deploys one Deployment per Store; with many sellers, one Deployment per seller is an operational trap — build a single app that resolves the seller from the resource instead.
Role + direction → app composition
serviceinbound webhook — the marketplace pushes seller/offer/inventory/price changes; you authenticate the caller and upsert. 5-min service timeout applies (not the extension limit).jobpoll — when the marketplace can't push, or for large periodic feeds.eventapp onOrderCreated— group the Order's lines by seller (their supply channel) and push each group to the marketplace; recordsyncInfo.eventapp on order/state changes — fulfilment, cancellation, and return status both ways.jobreconciliation — full sweep for drift (missed offers, stock divergence, orders the event path dropped), checkpointed.
eventapp on Product/Product Selection/Store/price/inventory messages — export listing, price, and stock deltas (theproduct-exporttemplate is the closest starting shape).job— full/batch feed export when the marketplace wants scheduled files instead of deltas.servicewebhook orjob— import marketplace orders (Order Import, keyed onorderNumber).eventapp — push shipment/tracking/cancellation back to the marketplace.
The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / configuration; inheritAs), and keep the file at the repository root — a nested connect.yaml silently fails to deploy.Native client provisioning (prefer this)
CTP_CLIENT_ID/CTP_CLIENT_SECRET as secured config (a pattern you will see in existing marketplace connectors and should not copy — check which form a fork candidate uses, per connector-selection.md):inheritAs:
apiClient:
scopes:
# operator, inbound seller + offer sync
- manage_products # Products/Variants — and Channels + Inventory Entries
- manage_standalone_prices # only if seller offers are Standalone Prices
- manage_key_value_documents # only if seller profiles are Custom Objects
- manage_orders # Order Import (seller role) / updateSyncInfo (operator)
- manage_types # only if postDeploy creates Custom Types
# plus, per app, the narrowest of:
# - view_products / view_orders (read-only apps)
# - manage_stores, manage_product_selections (only with Store-per-seller)
# - manage_subscriptions (apps whose postDeploy registers Subscriptions)
configuration:
standardConfiguration:
- key: MARKETPLACE_BASE_URL
description: Marketplace API base URL (sandbox vs production)
- key: SELLER_CHANNEL_KEY_PREFIX
description: Prefix for seller Channel keys, e.g. "seller-"
securedConfiguration:
- key: MARKETPLACE_API_TOKEN
description: Marketplace API token / OAuth client secret
- key: MARKETPLACE_WEBHOOK_SECRET
description: Shared secret or signing key used to authenticate inbound webhooks
Scope notes.manage_productscovers Channels and Inventory Entries too — there is no separate channel or inventory scope, so seller Channels and per-seller stock need no extra grant. Standalone Prices, Stores, Product Selections, and Custom Objects each need their own scope (manage_standalone_prices,manage_stores,manage_product_selections,manage_key_value_documents) — a frequent cause of a working-locally-but-403-in-Connect connector. Check the current list in API scopes rather than guessing, and grant per app, not per connector.view_subscriptionsis not a valid standalone scope;manage_subscriptionscovers read + write. Givemanage_ordersonly to the app that writes orders.
Per-app config
deployAs:
- name: seller-offer-sync # operator: marketplace pushes sellers + offers
applicationType: service
endpoint: /sellerOfferSync # the Express router must mount at this same base path
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
- name: order-router # operator: route each seller's lines out
applicationType: event
endpoint: /orderRouter
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the OrderCreated Subscription
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub or SNS"
- name: marketplace-reconcile # drift sweep
applicationType: job
endpoint: /marketplaceReconcile
properties:
schedule: "0 3 * * *"
Worked example (operator, Marketplacer-style service, build/fork)
europe-west1.gcp.seller-<id>, roles InventorySupply + ProductDistribution); seller profile payload in a CustomObject (container: seller, key: <marketplaceSellerId>); no Stores (no isolation requirement); one Product per listing id, with a StandalonePrice per seller (channel = seller channel) and an InventoryEntry per sku + seller supply channel; a Channel with role OrderExport per seller for syncInfo; per-line custom field holding the marketplace line id.inheritAs:
apiClient:
scopes:
[
manage_products, # Products/Variants + seller Channels + Inventory Entries
manage_standalone_prices, # per-seller offer prices
manage_key_value_documents, # seller profile Custom Objects
manage_orders, # updateSyncInfo on routed Orders
manage_subscriptions, # postDeploy registers the OrderCreated Subscription
manage_types, # postDeploy creates the line-item Custom Type
]
configuration:
standardConfiguration:
- key: MARKETPLACE_BASE_URL
description: "Marketplace API base (sandbox vs prod)"
securedConfiguration:
- key: MARKETPLACE_API_TOKEN
description: Marketplace API token
- key: MARKETPLACE_WEBHOOK_SECRET
description: Signing secret for inbound webhooks
deployAs:
- name: seller-offer-sync
applicationType: service
endpoint: /sellerOfferSync
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
- name: order-router
applicationType: event
endpoint: /orderRouter
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
- name: marketplace-reconcile
applicationType: job
endpoint: /marketplaceReconcile
service webhook app upserting sellers (Channel + CustomObject) and offers (Product by listing id, price by seller channel, inventory by sku+channel), authenticating every call with MARKETPLACE_WEBHOOK_SECRET; one event app whose postDeploy registers a single OrderCreated Subscription and which groups lines by supply channel, pushes one payload per seller, and records updateSyncInfo per seller channel so a redelivery doesn't double-push; one job reconciling offers and stock nightly with a checkpoint. Scopes are exactly what those three do; the marketplace token and webhook secret are securedConfiguration. Correctness rules per app: marketplace-contract.md.Which path: use as-is, customise, or build?
Check live data first — don't answer from memory
The marketplace changes. Before recommending anything:
- Browse the live Marketplaces category and the connector list; run this skill's
docs-searchscript / the Knowledge MCP for the service name. - For each candidate, capture: name, vendor, is it a Connect connector, direction, and what it syncs (sellers / offers / inventory / prices / orders / shipments).
- Name the connector and version you checked — or record that none exists — in the requirements block. Don't quote a listing's badge wording as a capability; badges describe the listing relationship, not what the code does.
The marketplace landscape (verify live — this is only the shape)
Two structural facts shape almost every marketplace engagement:
- Most marketplace listings are partner integrations, not Connect connectors. A vendor-operated integration, an iPaaS pipeline, or a cloud-function accelerator can be an excellent functional match and still have nothing Connect can deploy.
- There is no marketplace Connect template. The templates are
payment-integration,product-export,tax-integration, andtransactional-emails. A build therefore starts from plain apps — though for the seller role (pushing your catalog out to a marketplace) theproduct-exporttemplate is a genuinely close starting shape: it already does Store-scoped full export plus an incremental updater driven by Product/Product Selection/Store messages.
Verify it's an actual Connect connector — then ask the user
Marketplace-specific checks before treating any listing as path 1 or 2:
- Look for a connector repo with a root
connect.yamlanddeployAsapps. Noconnect.yaml, no Connect deployment. - A "connector" that deploys as the vendor's own cloud function, iPaaS flow, or hosted service is not a Connect application, whatever the listing or repo name says — a common shape for marketplace accelerators specifically.
- The marketplace platform's own commercetools integration may be operated by the platform, with nothing for you to deploy at all; that's a vendor onboarding task, not a connector build.
Present the three paths and let the user choose
Show the live findings, then ask. Don't skip straight to building.
Path 1 — Use a public connector directly (configure, no code)
deployment create against the published connector — not the connectorstaged flow). Hand it the config you derive in config-from-requirements.md.Path 2 — Customise it (fork an open-source connector)
Assess the candidate before you fork — from the repo, not from memory
connect.yamlat the repo root — thedeployAsapps and theirapplicationTypetell you which directions it covers (inboundservice, outboundevent, batchjob) and therefore which of the user's requirements it can't meet at all. Also read whether it usesinheritAs.apiClient.scopesor hand-suppliesCTP_CLIENT_ID/CTP_CLIENT_SECRET, and what its config keys are.- The handler entry points — how it identifies resources (upsert by key vs blind create), whether it authenticates inbound callers, and whether it re-fetches by id.
- The mapping code — what it maps sellers and offers onto, which is what you'll be rewriting per config-from-requirements.md.
- Language and framework — Java/Spring and TS/Node connectors both exist. Don't port; the commercetools-connect skill's contracts are language-agnostic, and a rewrite discards the mapping you forked for.
- README and repo framing — many marketplace connectors are published as accelerators or reference implementations, documented in the same spirit as the Connect templates: starting points that require customization before production use. Tell the user which grade they're forking.
- the commercetools-connect skill's production-readiness checklist — inbound authentication, native client provisioning, secrets in
securedConfiguration, no stack traces in responses, idempotent lifecycle scripts, structured logs, health endpoint, tests that assert behavior, README; - this sub-area's contract and pitfall catalog — upsert by marketplace id, seller Channel actually created, offers linked to a seller, one Product for a shared SKU, channel on every seller price, supply channel on every InventoryEntry, integer minor-unit money conversion, no hardcoded currency/locale/region, delisting handled,
syncInfobefore order export, and the directions the original omits.
Known landmarks (verify live — these change)
Pointers so you know a fork is even possible, not a substitute for reading the repo:
- Marketplacer has publicly available, open-source Connect connector code under the commercetools GitHub organization — inbound seller/listing sync, Java/Spring, published as an accelerator. It's the concrete path-2 candidate today.
- A separately branded Marketplacer accelerator also exists that deploys as a cloud function, not a Connect application — the listing-is-not-a-connector trap in its purest form.
- Mirakl appears in the category through vendor and partner listings; check live whether any is Connect-deployable before assuming path 1 or 2.
Path 3 — Build a new connector for the marketplace service they define
commercetools connect init, then connect application add --type service|event|job) — or start from product-export for the seller-role outbound direction and adapt. What you write is the marketplace API client, the mapping, and the keying; Connect scaffolds the plumbing.- Operator: inbound seller sync + inbound offer/inventory/price sync (
servicewebhook and/orjobpoll), outbound order routing (eventonOrderCreated), fulfilment status sync, reconciliationjob. - Seller role: outbound catalog/price/stock export (
event, orjobfor batch feeds), inbound marketplace order import (servicewebhook orjob), outbound shipment/tracking.
The ladder (stop at the first rung that fits)
- Is the listing a deployable Connect connector at all? If not → outside this skill: surface it with the not-a-Connect-solution warning, then offer forking an open-source alternative or building.
- Connect-deployable connector covers the requirements → install + configure (path 1).
- Right service, gap looks like a capability → prove it isn't config/mapping first → back to rung 1.
- Right service, genuine gap config can't close, and it's open source → fork (path 2). Don't rebuild a working sync engine.
- No usable connector for the service → build (path 3). No marketplace template;
product-exportis the closest shape for outbound.
Only rungs 3–4 leave this sub-area (hand off to the commercetools-connect skill for build/publish); the flow resumes here once the connector is deployed.
Recording the decision
Marketplacer · operator role · path 2 (fork) · checked the Marketplaces category and the open-source Marketplacer connector repo — Connect-deployable (rootconnect.yaml, twoserviceapps) but inbound catalog/seller only, accelerator-grade · forking to add order routing, native client provisioning, webhook auth, and idempotent seller upserts (backlog scored against the production gate + contract).
Checklist
- Checked the live Marketplaces category + connector list (not memory); cited each candidate + version
- Verified Connect-deployability per candidate (root
connect.yaml/ CLI registry — not the listing page) - A good-match listing that isn't a Connect connector was surfaced with the not-a-Connect-solution warning, not treated as path 1
- Presented all three paths to the user (use as-is · customise/fork · build for their service) and let them choose
- Apparent gaps re-checked as config/mapping before proposing code
- Fork chosen over from-scratch whenever an open-source connector for the service exists
- Told the user there is no marketplace template (and that
product-exportis the closest shape for outbound) - Decision + rung + version recorded in the requirements block
The marketplace sync contract
The rule that spans every app: upsert by the marketplace's id, never blind-create
| Entity | Key | Upsert mechanics |
|---|---|---|
| Seller | Channel key = seller-<marketplaceSellerId> | get-by-key → create if 404, else update |
| Seller profile blob | CustomObject container + key | POST /custom-objects is create-or-update — idempotent for free |
| Offer / listing | Product key = marketplace listing id (Variant key/sku per variant) | get-by-key → create or update actions |
| Offer price | StandalonePrice key = <sku>-<sellerId>-<currency>, or the embedded Price with the seller's channel | update the seller's price only — never rewrite prices of other sellers |
| Offer stock | InventoryEntry key = <sku>-<sellerId>, or query by sku + supplyChannel | one entry per seller per SKU |
| Inbound marketplace order | Order orderNumber = marketplace order id | query by orderNumber first; import only if absent |
| Outbound order hand-off | the Order's syncInfo entry for that seller's Channel | read syncInfo before pushing; skip if already recorded |
App 1 — seller sync (inbound, service webhook or job poll)
- Authenticate the caller. The marketplace calls you, so validate its proof — signature, shared secret, or JWT — before any write (security.md). An unauthenticated seller endpoint lets anyone create Channels and Products in the Project. (
AuthorizationHeaderauthentication on an Extension'sHTTPdestination is the reverse mechanism, for commercetools calling your endpoint; it does not authenticate inbound marketplace traffic.) - Upsert the seller Channel by key, with the roles the model requires (
InventorySupply, andProductDistributionwhen the seller prices independently). Store the profile payload in a CustomObject and/or Channel Custom Fields. - Only create a Store per seller if isolation was a requirement — and remember a Store is capped at 100 Product Selections.
- Offboard by deactivating, not deleting. Remove the Channel from Stores, deactivate the seller's Product Selection / delist their offers, and stop syncing. A Channel cannot be deleted while it is referenced by an InventoryEntry, LineItem, Store, Price, StandalonePrice, or CartDiscountValueGiftLineItem — the documented order is Carts → Orders → Stores → Channel. Note
StandalonePriceis on that list and is the offer-price model recommended above, so a seller synced that way is blocked by their own prices, not just by orders. Deleting a departed seller's historical Orders to satisfy that is technically the documented path, but it destroys order history; deactivation is the right offboarding move. - Idempotent — the same seller webhook twice must be a no-op.
App 2 — offer / inventory / price sync
Inbound (operator): marketplace → commercetools
- Upsert the Product by listing key; when several sellers sell the same SKU, resolve to the one shared Product and add the seller's price and stock, not a second Product.
- Every price carries the seller's distribution channel. A channel-less price is visible in every Store — the cross-seller price leak.
- Every InventoryEntry carries the seller's supply channel (
sku+supplyChannel). Writing stock without a channel makes it global stock for all sellers. - Map deliberately. Localized names/descriptions, currency, and money precision are where feeds go wrong: build the LocalizedString from the marketplace's locale rather than hardcoding one, derive the currency per seller/country rather than hardcoding it, and convert to
centAmountin integer minor units — multiply then round, never cast a float first (a(long) price * 100style conversion silently drops the cents). High-precision cases: money types. - Keep the mapping a pure function — no network calls — so it is unit-testable without a deployment or token.
- Don't publish blind. Decide whether an imported offer is published immediately or staged for review, and make it explicit in config.
- Bulk belongs in the Import API. Initial and periodic full loads go through the Import API (asynchronous, dependency-resolving, keyed → idempotent); the webhook path handles deltas. Keep them separate apps.
Outbound (seller role): commercetools → marketplace
- Driven by Subscription messages on products, prices, inventory, Product Selections, and Stores — the
product-exporttemplate is the closest existing shape (full export endpoint + incremental updater). - Re-fetch the resource by id from
resource.id; don't map from a possibly-stale or truncated payload. With no ordering guarantee, re-fetching makes the export converge on current state instead of replaying old deltas. - Scope what you export — which Store / Product Selection / channel defines "listed on this marketplace". Exporting the whole catalog to a marketplace that only sells a subset is a compliance and delisting problem.
- Delist explicitly. Unpublish, removal from a Product Selection, and stock hitting zero each need a defined outbound action; otherwise you keep selling items you no longer carry.
- Respect the marketplace's feed contract — batch sizes, schedules, and rate limits. Retry
429/5xxwith exponential backoff.
App 3 — orders
Inbound (seller role): import a marketplace order
- Dedupe on
orderNumber= the marketplace order id: query first, import only if absent. Order has no top-levelexternalId, soorderNumber(or a Custom Field) is the link. - Use Order Import — it creates an Order without a Cart. Set
store, per-linesupplyChannel/distributionChannel, and per-linecustomfields for the marketplace line id. NotetotalPricemust be set explicitly (it is not derived from the line items) and negative prices/quantities are not rejected — validate the payload yourself. - Record the inbound sync with
updateSyncInfoagainst a Channel with roleOrderImport. - Decide the inventory mode deliberately: stock was already committed on the marketplace side, so double-decrementing local stock is a real risk.
Outbound (operator): route lines to sellers
- Triggered by an
OrderCreatedMessageSubscription (registered idempotently inpostDeploy— get-then-create, never delete-then-recreate). - Decode the envelope before use: the GCP transport wrapper is
{ "message": { "data": "<base64>" } }, and the message body is PlatformFormat or CloudEventsFormat depending on config. Validate the type, then ack-and-ignore anything you don't handle (including the platform's test messages). - Re-fetch the Order by id, then group Line Items by seller (their
supplyChannel, or the seller reference you set at add-to-cart). Push one payload per seller — a multi-seller order is N seller orders downstream. - Record
updateSyncInfoper seller Channel (roleOrderExport) with the marketplace's id andsyncedAt, and readsyncInfofirst so a redelivered message doesn't double-submit. That's your idempotency mechanism; combine it with the marketplace's own idempotency key if it has one. - Ack correctly —
2xx(the event contract treats102/200/201/202/204as "don't redeliver") for handled and deliberately-ignored messages; non-2xx only for transient failures you want redelivered. - Partial failure is normal. If seller A's push succeeds and seller B's fails, don't re-push A on retry — per-seller
syncInfomakes the retry converge instead of duplicating.
App 4 — fulfilment, cancellation, and return status
- One Order, many sellers → track per line, not per order. Use Line Item
state(ItemStates) transitions for per-seller progress, and Deliveries/Parcels for split shipments; an order-levelshipmentStatealone can't express "seller A shipped, seller B cancelled". - Tracking numbers, carrier, and shipment events flow back per seller shipment; cancellations and returns must map to the marketplace's own state machine, not just a local status field.
- Self-change filtering wherever a domain syncs both ways: a status you write inbound raises a message your outbound app would push straight back. Mark connector-originated writes (a
syncSourceCustom Field, or compare againstsyncInfo) and skip them. One-way per domain avoids this entirely.
App 5 — reconciliation job
job that pages the marketplace (and commercetools) and repairs differences: missing offers, stock divergence, orders never imported, orders never exported (empty syncInfo). Checkpoint progress (e.g. in a CustomObject) so a restart resumes mid-run rather than restarting, respect the 30-min job timeout, own your overlap locking, and keep every unit of work an upsert so a re-run can't double-write. Keep the initial bulk migration a separate job from ongoing reconciliation.Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
| Create-on-every-payload | Duplicate sellers / Products / Orders after redelivery | Upsert by the marketplace id (table above) |
| One Product per seller for the same SKU | Splintered catalog, duplicate PDPs, unusable search and reporting | One Product; per-seller Prices + InventoryEntries |
| Price without a distribution channel | One seller's price shows in every Store | Always set channel on seller prices |
| InventoryEntry without a supply channel | Seller stock becomes global stock; overselling | sku + supplyChannel per seller |
| Availability read as a single number | Storefront shows aggregated stock across sellers | Read per-channel availability; a Store-bound Cart filters by its supply channels |
| Trusting the payload | Stale offers overwrite newer ones; deltas replayed out of order | Re-fetch by resource.id |
| Envelope not decoded | Handler sees base64 garbage / crashes | Decode message.data (base64 → JSON), then validate the type |
| Wrong ack | Handled message redelivered forever, or failures silently dropped | 2xx for handled/ignored; non-2xx only for retryable |
No syncInfo check before export | Multi-seller order pushed twice on redelivery | Read syncInfo, write updateSyncInfo per seller channel |
Order imported without orderNumber dedupe | Duplicate Orders for one marketplace order | Query by orderNumber first |
totalPrice assumed to be calculated on import | Wrong order totals | Set totalPrice explicitly; validate the draft |
| One Subscription (or Extension) per seller | Hits the 50-Subscription / 25-Extension Project limit | One Subscription per message type; fan out in the handler |
| Float → cents conversion | Cents dropped or inflated on every offer | Multiply then round in integer minor units |
| Hardcoded currency/locale/region | Works for one seller/market, breaks the rest | Derive from the payload/config; region from CTP_REGION |
| Seller offboarded by deleting the Channel | Delete fails; sync half-broken | Deactivate: unassign from Stores, delist offers, stop syncing |
| No self-change filter on a two-way domain | Status ping-pong, runaway API calls | Mark connector writes and skip; prefer one-way per domain |
| Unauthenticated inbound webhook | Anyone can write Products/Orders | Validate signature/secret/JWT in-app |
| Secrets or PII in logs / stack traces in responses | Compliance incident | Generic error responses; structured logs without payload dumps |
Route ≠ connect.yaml endpoint | Platform traffic 404s | Mount the router at the app's endpoint base path |
| Legacy SDK | Fails the commercetools-connect skill's pinned-version gate | @commercetools/platform-sdk@^8 + @commercetools/ts-client@^4 |
Test-first checklist (mirror in the suite)
Seller sync
- Rejects unauthenticated / bad-signature calls (parameterized auth matrix)
- Upserts the Channel by key; second delivery is a no-op
- Roles and profile storage asserted; offboarding deactivates rather than deletes
Offer / inventory / price sync
- Same-SKU second seller adds a price + inventory entry, does not create a second Product
- Every written price has a channel; every InventoryEntry has a supply channel
- Money conversion asserted on a value with non-zero cents; locale/currency taken from input
- Delist path asserted (unpublish / removed from selection / zero stock)
- Outbound: re-fetches by id, exports only in-scope products
Orders
- Inbound: duplicate marketplace order id imports once;
totalPriceset;syncInforecorded - Outbound: multi-seller order produces one payload per seller with only that seller's lines
- Redelivered
OrderCreatedpushes nothing (syncInfoshort-circuit) - Partial failure retries only the failed seller
- Envelope/ack matrix covered
Fulfilment + reconciliation
- Per-line state transitions asserted; split shipment maps per seller
- Self-change filter asserted on any two-way domain
- Reconciliation resumes from checkpoint after a simulated failure; repairs are upserts
- Boundary mocked; suite runs with no deployment and no secrets
Marketplace connector — integrate a marketplace service
First, disambiguate the word "marketplace" — ask if it isn't obvious. Two unrelated meanings collide here:
- The commercetools Connect marketplace — marketplace.commercetools.com, the catalog where connectors and integrations are listed. Every connector task touches this.
- A marketplace business model — selling third-party sellers' assortments, or selling your assortment on someone else's marketplace. That is what this sub-area is about.
"Build a marketplace connector" almost always means the second. If the user actually meant "publish my connector on the Connect marketplace", that's the commercetools-connect skill's deployment-installation.md, not this file.
Step 1 — Fix the role, then the direction
Everything else follows from these two answers. Get them before proposing an architecture.
| Role | The user is… | Direction(s) | Connect app(s) |
|---|---|---|---|
| Operator | running the marketplace: third-party sellers' offers sell through their commercetools-powered storefront | sellers/offers/inventory/prices in; order lines and fulfilment status out | service inbound webhook and/or job poll for seller + offer sync; event app on OrderCreated to route each seller's lines to the marketplace; optional job for reconciliation/backfill |
| Seller (channel) | selling their own catalog on an external marketplace (Amazon, eBay, a Mirakl operator) usually via a channel manager | catalog/price/stock out; marketplace orders in; shipment/tracking out | event app exporting catalog/price/stock changes (or a job for batch feeds); service webhook or job to import marketplace orders; event app pushing shipment/tracking back |
| Both | hybrid (operates a marketplace and lists on others) | both | both sets — as separate apps, never one app with a mode flag |
Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<marketplace terms from the request, e.g. 'marketplace seller supply channel distribution channel store product selection order import syncInfo'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root.) You may additionally use the commercetools Knowledge MCP for follow-up. There is no marketplace module in the public docs — the load-bearing references are Channels, Stores, Product Selections, Inventory, and Order Import. Read the ones your role needs.Step 1 — Extract requirements (before any config or code)
Ask the user — don't assume:
- Which marketplace service, and what API access? Marketplacer, Mirakl, Convictional, a channel manager, an operator's own portal, or a service they define. Webhooks vs polling, credentials, sandbox, rate limits.
- Role and direction (the table above). Operator, seller, or both.
- Source of truth per domain — offer content, inventory, price, order, seller record.
- How many sellers, and do they need isolation? Isolated storefront/catalog/permissions per seller → Store-per-seller; a shared catalog with per-seller offers → Channel-per-seller only. Drives Step 2.
- Do multiple sellers sell the same SKU? If yes, one Product with per-seller prices and stock — not one Product per seller. This is the single most consequential modeling answer.
- Which entities sync? Sellers, offers/listings, inventory, prices, orders, shipments/tracking, returns/cancellations, invoices.
- Order flow. Does commercetools capture the order and route lines to sellers (operator), or does the marketplace capture it and you import it (seller)? Can one cart span sellers → split shipments and per-line fulfilment states?
- Commission, payout, and settlement. Confirm explicitly that these stay in the marketplace/PSP: commercetools only tracks Payment status and has no payout ledger. Don't model seller payouts as commercetools resources.
- Seller onboarding/offboarding. Approval flow, and what happens on offboarding — note up front that a Channel can't be deleted while it's referenced by inventory, a Line Item, a Store, or a Price, so offboarding is deactivation, not deletion.
- Anything special or non-standard? (always ask — open-ended) Seller-specific shipping rules or lead times, per-seller tax, drop-ship vs consignment, marketplace-imposed feed formats/schedules, returns arbitration, multi-currency or multi-country sellers, seller-facing UI in the Merchant Center. Capture each as its own requirement line; don't force it into a slot above.
Step 1.5 — Ask the user which path: use as-is, customise, or build
- Use a public connector directly — install and configure it, no code. Only valid if the listing is an actually deployable Connect connector.
- Customise it — fork an open-source connector and add only the delta (the realistic path for marketplace work). Assess the candidate by reading its current repo —
connect.yaml, handlers, mapping — and score it against the production gate and this sub-area's contract; don't work from a remembered gap list. - Build a new one for the marketplace service they define — no listing fits, or the service is bespoke. There is no marketplace Connect template, so you scaffold plain
service/event/jobapps.
connect.yaml before calling it path 1; if a listing matches functionally but isn't a Connect connector, surface it and say plainly that this skill doesn't cover non-Connect integrations, then offer path 2 or 3. Marketplace-specific method, how to assess a fork candidate, and the ladder: connector-selection.md.Record the chosen path, the connector name + version you checked (or "none exists"), and why, in the requirements block.
Step 2 — Model sellers and offers, then derive the config
connect.yaml. The mapping table, the limits that constrain it, scopes, and a worked example are in config-from-requirements.md.Step 3 — Price the async contract (reference)
Step 4 — Build/verify the sync apps (the main body of work), test-first
orderNumber, per-line fulfilment state — are invisible at the call site and expensive to reproduce by hand. Each is one cheap assertion.- Seller sync (inbound) — upsert a Channel (and Store/CustomObject) per seller, keyed on the marketplace seller id.
- Offer/listing sync — inbound (operator): upsert Products/prices/inventory per seller; outbound (seller role): export catalog/price/stock changes to the marketplace.
- Order app — inbound (seller role): import marketplace orders via Order Import keyed on
orderNumber; outbound (operator): route each seller's lines onOrderCreatedand record the hand-off in the Order'ssyncInfo. - Fulfilment/status app — shipment, tracking, cancellation and return states back to the other side, per line/per seller.
- Reconciliation
job— periodic full sweep that catches what events dropped (offers, stock drift, missed orders), checkpointed.
Step 5 — Verify the round trip
References
| Need | Reference |
|---|---|
| Which path — use a public connector as-is, customise/fork one, or build for a defined service; live check, listing-is-not-a-connector verification, how to assess a fork candidate from its repo, the ladder | connector-selection.md |
Seller + offer modeling and config — Channel/Store/CustomObject per seller, offer keying, price and stock scoping, the limits that constrain it, scopes, connect.yaml, worked example | config-from-requirements.md |
The sync contract — per-app rules for seller sync, offer sync, order import/export, fulfilment, reconciliation; idempotency keys, syncInfo, split shipments; full pitfall catalog | marketplace-contract.md |
| Verify the round trip — seller, offer, order, split order; the channel-less-price, aggregated-availability, and throttling traps | verification.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
Adding another marketplace service later reuses this tree unchanged — the role/direction split, the seller model, and the keying rules don't change; only the service's API and payloads do.
Checklist
Requirements
- "Marketplace" disambiguated (business model vs the Connect marketplace listing catalog)
- Role fixed (operator / seller / both) and direction per domain decided
- Source of truth named per domain (offer, inventory, price, order, seller)
- Seller count and isolation needs known; same-SKU-multiple-sellers answered
- Entities in scope listed; order flow (route vs import) decided
- Commission/payout confirmed as out of scope for commercetools
- Offboarding path decided (deactivate, not delete — Channel delete constraints)
- Asked the open-ended "anything special?" question; each special requirement its own line
- Requirements block written and confirmed
Path (asked, not assumed)
- Checked live marketplace listings; named connector + version (or "none exists")
- Verified any candidate is a deployable Connect connector, not a partner/SaaS listing
- Presented all three paths — use as-is · customise/fork · build for their service — and let the user choose
- Chosen path + rung recorded
Modeling and config
- Seller modeling decided (Channel per seller; Store/Product Selection only if isolation is needed; CustomObject for profile data)
- Offer keying decided; same-SKU sellers modeled as one Product with per-seller prices/stock
- Every seller price carries a distribution channel; every InventoryEntry a supply channel
-
inheritAs.apiClient.scopesleast-privilege; marketplace credentials insecuredConfiguration - No per-seller Subscriptions or Extensions (Project limits)
The sync apps (build test-first)
- Every write is an upsert keyed on a stable marketplace id
- Inbound orders deduped on
orderNumber; outbound hand-off recorded insyncInfo - Per-seller fulfilment tracked per line item, not per order
- Reconciliation job checkpointed; rate limits respected (batch + backoff)
- Boundary mocked; suite runs with no deployment/secrets
Verification
- Seller, offer, and order round trips proven; multi-seller order splits correctly
- Re-delivery of the same webhook/message creates no duplicate
Verify the marketplace round trip
Check 1 — the seller exists and is usable, not just present
Create or change a seller on the marketplace side, then confirm:
- The seller Channel exists with the expected
key(seller-<marketplaceSellerId>) and the roles the model needs (InventorySupply,ProductDistributionwhere the seller prices independently). - Profile data landed where the model says (CustomObject / Channel Custom Fields).
- If Store-per-seller is in scope: the Store exists, references that Channel, and its Product Selection is assigned.
- Re-send the same seller webhook: nothing duplicates and no version conflict is raised.
A seller that exists only as a CustomObject with no Channel is the tell that the model is incomplete — stock and price have nothing to scope to.
Check 2 — the offer is sellable, per seller
Create or update a listing, then confirm end to end (not just "the Product appeared"):
- The Product/Variant exists, keyed on the marketplace listing id, published according to the configured publish/staging decision.
- A Price carrying the seller's distribution channel exists — and the amount is exact, cents included (this is where float→cent bugs surface).
- An InventoryEntry for
sku+ the seller's supply channel exists with the expected quantity. - Add it to a Cart in the seller's context (Store-bound if Store-per-seller, otherwise with the seller's channels on the Line Item) and confirm the correct price and availability are selected. A Product that exists but can't be added at the seller's price is not a working offer.
- Second seller, same SKU: sync a second seller's offer for the same SKU and confirm no second Product is created — only an additional price and inventory entry.
- Delist: unpublish / remove from the selection / drop stock to zero on the marketplace side and confirm it stops being sellable.
Check 3 — the order flows once
orderNumber = the marketplace order id, the right store, per-line supplyChannel, correct totalPrice, and a syncInfo entry against the OrderImport channel. Re-deliver the same payload and confirm no second Order.- Each seller received one payload containing only their own lines — with correct quantities and prices.
- The Order carries a
syncInfoentry per seller Channel with the marketplace'sexternalId. - Redeliver the
OrderCreatedmessage: nothing is pushed again (thesyncInfoshort-circuit works). This is the single most valuable assertion in the suite. - Force one seller's push to fail: on retry, only the failed seller is re-pushed.
Check 4 — fulfilment splits per seller
The traps (behavior that looks like a bug — or hides one)
Trap 1 — the channel-less price leak
Trap 2 — availability looks wrong because it's aggregated
ProductVariant.availability summarizes stock and lags real-time by seconds; with several sellers per SKU it reads as one blended number, and Order-driven stock changes are eventually consistent (up to ~10 s). Verify per-seller stock by querying the InventoryEntry for sku + supplyChannel, and verify storefront behavior through a Store-bound Cart — not by eyeballing the aggregate.Trap 3 — "sync stopped" is usually throttling
429 backpressure or a batch-size violation, not a logic bug. Confirm backoff/retry and that the reconciliation job resumes from its checkpoint rather than restarting the whole catalog.Trap 4 — sandbox marketplaces don't behave like production
Trap 5 — the seller you can't remove
Verification checklist
- Seller Channel (and Store, if modeled) created with the right key and roles; re-delivery is a no-op
- Offer sellable: price with channel and exact amount, InventoryEntry with supply channel, add-to-cart proven in the seller's context
- Second seller of the same SKU adds price + stock, no duplicate Product
- Delist path proven
- Inbound order: one Order per marketplace order id,
totalPricecorrect,syncInforecorded, redelivery creates nothing - Outbound order: one payload per seller with only their lines,
syncInfoper seller channel, redelivery pushes nothing, partial failure retries only the failed seller - Fulfilment states tracked per line; split shipment and mixed ship/cancel proven
- Reconciliation job resumes from checkpoint; backoff verified under throttling
- No secrets or payload dumps in logs; error responses carry no stack traces
- Test sellers/listings/orders cleaned up on both sides
Build a new OMS connector
Start from the fulfilment-integration template
fulfilment-integration template (connect-cli.md template list; repo connect-fulfilment-integration-template). Note it is not on the public Application templates overview page (which documents only payment, product-export, tax, and email) — it's exposed through the CLI, so scaffold from it rather than composing from scratch:commercetools connect init my-oms-connector --template fulfilment-integration
| Template app | Type | Trigger | Sync flow it implements |
|---|---|---|---|
| order-export | event | Subscription on OrderCreated / ReturnInfoAdded | Export placed Orders → OMS |
| order-updates | service (REST) | inbound endpoint: /order-updates | Inbound status/shipping/packaging/parcel/tracking OMS → commercetools |
| inventory-import | service (REST) | inbound endpoint: /inventory | Inbound stock/status updates → InventoryEntry |
| product-export | event | Subscription on ProductPublished | (product sync — keep only if you need it) |
job (nightly full sync — the template doesn't ship one), add it with commercetools connect application add --type job. The tax-integration template order-syncer is a secondary reference for the OrderCreated subscriber shape.event app — the template's order-export is deployAs: event, and Subscription Messages are delivered to event applications through the Connect message broker; service apps are for API Extensions or inbound webhooks (here, order-updates / inventory-import). See the decision framework in commercetools-connect and event-applications.md.connect.yaml envelope keys (deployAs/applicationType/configuration/inheritAs) and keep the file at the repository root — a nested file silently fails to deploy (project-structure.md).Connecting to the user-defined OMS
The OMS is an arbitrary external system — treat its API as the untrusted outbound/inbound boundary:
- Config, not code. OMS base URL, tenant/account id, and non-secret toggles →
standardConfiguration. OMS API key/client secret, webhook signing secret →securedConfiguration, never hardcoded (security.md). - Deploy-time validation.
postDeployshould test-connect to the OMS and surface bad credentials immediately, and register the Subscription + any custom State machine / Custom Types idempotently (get-then-create, never blind delete-recreate) → lifecycle-scripts.md. - Map at the boundary. Convert between the OMS's order/status model and commercetools' at the edge; keep SDK types end to end internally, no
anyescapes (project-structure.md). The concrete field/action mapping is sync-architecture.md. - Least-privilege scopes.
inheritAs.apiClient.scopeswith only what the flows need — typicallymanage_orders,view_orders,manage_subscriptions, andmanage_inventoryif syncing stock — notmanage_project. - Fail-open vs fail-closed. Since the export is async and the inbound is a webhook (neither on a synchronous checkout path), a transient OMS outage should fail closed with retry (return non-ack / non-2xx to trigger redelivery), not silently drop. Document the stance in the README.
Build test-first, then deploy
orderNumber/OMS ref, inbound idempotent (redelivery no-op, no stale overwrite), self-change filtering prevents loops, inbound webhook rejects unauthenticated callers.connectorstaged create → publish → deployment create (the publish-time production-readiness scan applies) → deployment-installation.md.Checklist
- Scaffolded from the
fulfilment-integrationtemplate (connect init --template fulfilment-integration); kept only the needed apps (order-export/order-updates/inventory-import), added a reconcilejobif required - Applications declared in a root
connect.yamlusing only documented envelope keys; router mounts matchendpoint; order-export isdeployAs: event - OMS URL/tenant in standardConfiguration; OMS + webhook secrets in securedConfiguration
-
postDeployvalidates OMS connectivity and idempotently registers Subscription + custom States/Types;preUndeploycleans up - Least-privilege scopes (
manage_orders/view_orders/manage_subscriptions/manage_inventoryas needed) - Fail-open/closed stance documented; inbound webhook authenticated
- Built test-first; sync invariants pinned as tests; deployed via
connectorstaged → publish → deployment create
Is a public OMS connector enough?
Two things that are easy to get wrong
- Installable Connect connector — published to Connect, deployed into your project via the Connect CLI / UI. This is the "install + configure" case.
- Vendor-hosted / partner integration — the OMS vendor operates the integration on their side (their connector calls the commercetools API, or you configure it in the vendor's console). You don't deploy anything in Connect; setup follows the vendor's docs.
Discover public connectors programmatically — don't hardcode a list
Search Connectors endpoint, which filters published connectors by integration type:GET {connect-host}/connectors/search?integrationTypes=oms
# add &integrationTypes=shipping for fulfillment/shipping connectors; &text=<keyword> to narrow
- Filter by the right type(s).
IntegrationTypevalues (verified against the Connect API):tax,marketplace,oms,psp,pim,promotion,search,erp,crm,email,analytics,shipping,giftcard. There is no separatefulfillmentvalue — fulfillment/OMS connectors are taggedomsand/orshipping, so query both for an order-management use case. → schema:openApi-schemata.mjs --resource-name connect-Connector; host/auth: Connect hosts & authorization. - Read the result. Each returned
Connectorcarriesname,key,integrationTypes,creator,repository,configurations,supportedRegions,certified,private, anddocumentationUrl. Usecertified: true/private: falseto identify public certified connectors;repositorytells you whether the source is available to fork;configurationsis the config surface you'd fill at install. Name the version you found. - Equivalent surfaces: the same search is available in the Merchant Center (Connect) and the Connect CLI; the marketplace is the human-browse view (note connectors span both the order-management and fulfillment marketplace categories — the API
oms+shippingquery covers both). - For a specific candidate, its
repository/documentationUrlis authoritative for capabilities, install shape, and config keys.
event order-export app (on Order Confirmed) + a service app for fulfillment status back to CT + Channel↔Facility inventory sync; deployable via the Merchant Center or Connect API. Others surface under the oms/shipping search (e.g. Fluent Commerce, kbrw, OneStock, NewStore, Pipe17) — verify each live rather than trusting this list.Turn the search result into the decision
Run the search first, then branch on what it returns — this is how the ladder below is driven:
- A published connector matches the OMS and covers the flows → install it (rung 1) —
deployment createwith the connector'sconfigurations. - A published connector matches but a behavior is missing → try config first (rung 2); if genuinely missing and its
repositoryis available → modify/fork it (rung 3). - No published connector matches the OMS → build one (rung 4): from scratch or, preferably, the
fulfilment-integrationtemplate → build-oms-connector.md.
State explicitly that you're checking current data, and cite the connector key + version you found.
The fit check
Compare the Step 1 requirements against what a candidate connector actually supports. Check each dimension:
| Dimension | Question | If not covered → which rung |
|---|---|---|
| OMS coverage | Is the user's OMS available as a connector at all? | No connector → rung 4 (build new). |
| Install shape | Installable Connect connector or vendor-hosted integration? | Vendor-hosted → follow vendor docs (still rung 1, but not a Connect deploy). |
| Flows | Does it cover the flows needed — order export, status/shipment inbound, fulfillment, inventory, returns? | Missing flow → config first (rung 2), else fork/customize (rung 3). |
| Direction / source of truth | Does its direction model match yours (who masters status, inventory)? | Mismatch → fork (rung 3) or build (rung 4). |
| Data mapping | Can statuses, SKUs, locations/channels, and OMS ids be mapped as needed? | Fixed/unsuitable mapping → config (rung 2) then fork (rung 3). |
| Split/partial fulfillment, BOPIS, returns | Does it handle split shipments, partial fulfillment, store pickup, RMA? | Missing → fork (rung 3) or build (rung 4). |
| Region/compliance | Available + supported for the region, volume, and data-residency needs? | Not available → different connector or build. |
| Special requirements | Each open-ended requirement from Step 1 (B2B/approvals, marketplace split, subscriptions, custom workflow states, existing OMS account/tenant) | Config (rung 2); bespoke logic → fork (rung 3); no connector at all → rung 4. |
The decision ladder
- A connector covers everything → install + configure (Connect connector) or follow the vendor's setup (vendor-hosted). Don't build. The common, recommended case.
- A connector exists, gap looks like a capability → prove it isn't config first (mappings, message selection, enabled flows). If config closes the gap, back to rung 1.
- A connector exists, genuine gap config can't close → fork/customize it if its source is available: add only the delta and deploy as an Organization connector. You keep the working sync scaffolding and only change what's different. A commercetools-connect build-side task; the sync design is sync-architecture.md.
- No connector fits, or the OMS is bespoke/home-grown → build a new connector connecting to the OMS the user defines. → build-oms-connector.md, then the commercetools-connect build-side workflow.
Checklist
- Ran
GET /connectors/search?integrationTypes=oms(andshipping) — not memory; cited the connector key + version, and itscertified/privateflags - Confirmed install shape: installable Connect connector vs vendor-hosted integration
- Flows, direction, mapping, split/partial fulfillment, region, and each special requirement compared to the requirements
- Apparent gaps re-checked as config (rung 2) before considering any build
- When a connector exists but has a real gap, chose fork/customize (rung 3) over building from scratch
- Decision + rung + version recorded: use (1), config (2), fork (3 → commercetools-connect), or build (4 → build-oms-connector)
Order-management connector — build & integration
event / service / job applications. For building from scratch, the Connect CLI ships a fulfilment-integration template whose apps (order-export, order-updates, inventory-import) map onto these flows — see build-oms-connector.md.Direction & source of truth (settle this first — it decides everything)
- Export (commercetools → OMS): a placed Order is pushed to the OMS for routing/fulfillment. Triggered by the
OrderCreatedMessage. - Inbound (OMS → commercetools): the OMS pushes status, shipment/tracking, fulfillment, and inventory back so the storefront and Merchant Center stay current.
Customer has an externalId; the Order does not — use its SyncInfo (the updateSyncInfo action) or a Custom Field (see sync-architecture.md). For architecture patterns on order replication, see the ERP integration tutorial.Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<OMS terms from the user's request, e.g. 'order export subscription OrderCreated shipment fulfillment inventory sync external system'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root, where scripts/docs-search.mjs lives.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or https://docs.commercetools.com for deeper follow-up.Step 1 — Extract requirements (do this before any config or code)
The connector choice and the sync design are both downstream of these. Ask the user (don't assume):
- Which OMS, and is there an existing connector? Name the OMS (Fluent Commerce, kbrw, OneStock, NewStore, Pipe17, a home-grown service, …). Is a public/partner connector already deployed, or is this greenfield?
- Region and project? e.g.
europe-west1.gcp, projectmy-project— drives theCTP_*_URLconfig and the deploy region. - Source of truth per domain? Which system masters order status, shipment/tracking, inventory, and customer data? (Usually the OMS masters status/shipment/inventory once the Order is placed.)
- What must be exported, and when? All Orders on
OrderCreated, or only after payment/approval? Do split shipments / partial fulfillment / store pickup (BOPIS) apply? - What comes back inbound? Order/line-item status transitions, shipment + tracking, delivery/parcel data, cancellations, returns, inventory levels — and how does the OMS deliver them (webhook, polling, batch file)?
- Latency & volume? Real-time (event + webhook) vs near-real-time vs nightly batch (job). Order and inventory volume shape the design.
- Data mapping? How OMS statuses map to commercetools Order/line-item/shipment states; how SKUs/locations/channels map; where the OMS id is stored on the Order (
SyncInfoviaupdateSyncInfo, or a Custom Field — Order has noexternalId). - Anything special or non-standard? (always ask — open-ended) The list above covers the common shape but not everything. Prompt to jog memory: B2B (PO numbers, approvals, business units), marketplaces/split fulfillment across many vendors, returns/RMA flows, multi-currency or per-market, subscriptions/recurring orders, existing OMS contract or a specific account/tenant, custom order workflows/states, compliance or data-residency constraints. Capture each as its own requirement line; don't force it into a slot above.
Step 1.5 — Is a public connector enough? (decide before wiring or building)
GET /connectors/search?integrationTypes=oms (add &integrationTypes=shipping — there's no separate fulfillment type). Filter results to public/certified (certified: true, private: false), compare requirement-by-requirement, and name the connector key + version. Full method (query params, IntegrationType values, reading the result, install-vs-vendor-hosted): connector-selection.md.- A connector covers everything → install + configure. Don't build. Note the important distinction (covered in connector-selection.md): some OMS marketplace listings are installable Connect connectors (deploy via the Connect CLI / UI), others are vendor-hosted integrations the OMS provider operates. Confirm which before promising a Connect deploy.
- A connector exists but a gap looks like a capability → prove it isn't config first (mappings, which messages, which flows are often configurable). If config closes the gap, back to rung 1.
- A connector exists but has a genuine gap config can't close → fork/customize it (if its source is available) — add only the delta and deploy as an Organization connector. Don't rebuild a working, maintained connector. → connector-selection.md, then the commercetools-connect build-side.
- No connector fits, or the OMS is bespoke/home-grown → build a new connector, scaffolding from the
fulfilment-integrationCLI template (order-exportevent+ order-updates/inventory-importservice), adding a reconcilejobif needed. → build-oms-connector.md.
Step 2 — Design the sync architecture (the core deliverable)
Step 3 — Build (rungs 3–4), test-first
- Export = an
eventapplication subscribing toOrderCreated(and status Messages) → event-applications.md. At-least-once, no ordering: idempotent onorderNumber/OMS id, re-fetch the Order by id, filter self-changes. - Inbound = a
serviceapplication as an inbound webhook the OMS calls → service-applications.md. Authenticate the caller, validate the payload, apply updates idempotently (upsert by key / re-check state), never blind-create. - Reconcile = a
jobfor nightly full sync / drift repair → job-applications.md. Owns its own locking and checkpointing. - Registration of the Subscription and any custom types/statuses = idempotent
postDeploy/preUndeploy→ lifecycle-scripts.md.
Step 4 — Deploy
deployment create --connector-key, or the Connect UI); a forked/built Organization connector goes through connectorstaged create → publish → deployment create. Pass the config you derived; secrets go in securedConfiguration, never in code.Step 5 — Verify the round trip
References
| Need | Reference |
|---|---|
| Is a connector enough? live fit-check against marketplace OMS connectors; installable-vs-vendor-hosted distinction; the use/configure/fork/build ladder | connector-selection.md |
| Sync design: direction & source of truth, export/inbound/reconcile flows, which Messages to subscribe to, OMS-status → CT-state mapping, idempotency per flow | sync-architecture.md |
Build a new connector for a user-defined OMS (rung 4): scaffold from the fulfilment-integration template, which applications to declare, connecting to the OMS API | build-oms-connector.md |
| Event app (export): envelope, ack, idempotency, re-fetch, injected destination | event-applications.md |
| Service app (inbound webhook): authenticated inbound, idempotent upsert, timeout | service-applications.md |
| Job app (reconcile): schedule, timeout, concurrency, checkpointing | job-applications.md |
| Idempotent Subscription/custom-type registration in postDeploy/preUndeploy | lifecycle-scripts.md |
| Deploy/install (public vs forked/built), regions, redeploy | deployment-installation.md |
| Testing (auth matrix, idempotency, ack edge cases), test-first loop | testing.md |
| Logs + correlation IDs, health, poison-message/replay runbook | observability-operations.md |
Checklist
Requirements
- OMS named; existing connector checked (and its type: installable Connect connector vs vendor-hosted integration)
- Region + project; source of truth fixed per domain (status, shipment, inventory, customer)
- Export scope + trigger; inbound scope + delivery mechanism; latency/volume; data mapping; OMS-id storage
- Open-ended "anything special?" asked; each special requirement captured as its own line
- Requirements block written and confirmed; special requirements flagged into Step 1.5
Connector fit (decide before wiring/building)
- Checked live marketplace + docs (not memory); named connector + version
- Confirmed installable-vs-vendor-hosted before promising a Connect deploy
- Apparent gaps re-checked as config before considering fork/build
- Ladder rung presented to the user and chosen by them: use (1) · configure (2) · fork/customize (3) · build new (4); decision recorded
Sync design (the deliverable)
- Direction fixed; no bidirectional sync of the same field
- Flows enumerated: export (event), inbound (service webhook), reconcile (job) as needed
- Export subscribes to the right Messages (
OrderCreated, status transitions); inbound applies idempotently - OMS-status → CT Order/line-item/shipment/delivery state mapping table produced
- Idempotency strategy stated per flow (orderNumber/OMS id; upsert by key; re-fetch by id)
Build & ship (rungs 3–4)
- Built test-first on the commercetools-connect event/service/job references and their checklists
- Subscription + custom types registered idempotently in postDeploy; cleaned up in preUndeploy
- Deployed via the type-agnostic deploy flow; secrets in securedConfiguration
- Round trip verified (Order → OMS → status back to CT); integration test asserts the CT trace
OMS sync architecture
node scripts/openApi-schemata.mjs --resource-name api-Order-write (update actions), --resource-name api-Order-read, --resource-name api-InventoryEntry-write; and node scripts/graphql-schemata.mjs --resource-name Order. Message shapes: Cart and Order Messages.The three flows
flowchart LR
subgraph CT[commercetools]
Order[Order]
Sub[Subscription]
Inv[Inventory]
Order -- OrderCreated / status Messages --> Sub
end
subgraph Conn[Connect connector]
Export[event: export]
Inbound[service: inbound webhook]
Reconcile[job: reconcile]
end
OMS[Order-management system]
Sub -- receives Messages --> Export
Export -- create/update order --> OMS
OMS -- status / shipment / fulfillment / inventory --> Inbound
Inbound -- update actions --> Order
Inbound -- adjust quantity --> Inv
Reconcile -- poll / full sync --> OMS
Reconcile -- repair drift --> Order
1. Export — event application (commercetools → OMS)
- Subscribe to the right Messages.
OrderCreatedfor the initial export; add order-lifecycle Messages only if the OMS must also learn about commercetools-side changes (OrderStateChanged,OrderCustomerSet, edits). Subscribe to the minimum set and ack-and-ignore the rest. Message catalog: Cart and Order Messages. Thefulfilment-integrationtemplate'sorder-exportapp is the canonical working example — it subscribes toOrderCreated/ReturnInfoAdded(connect-fulfilment-integration-template); the tax template'sorder-synceris a secondaryOrderCreated-subscriber reference. - Re-fetch the Order by
resource.id— don't trust the Message payload (it may be omitted whenpayloadNotIncluded). Fetch the full Order, map it, then push. - Idempotent export. At-least-once delivery means the same
OrderCreatedcan arrive twice. Make the OMS create idempotent: prefer the OMS's own idempotency key (send the commercetoolsorderNumberor Orderidas the OMS external reference and upsert), or check the OMS for an existing record before creating. Never keep a local dedup store. - Record the OMS id back on the Order so inbound updates and reconciliation can correlate — and so the export can detect "already exported". Order has no top-level
externalId; use the purpose-builtSyncInfovia theupdateSyncInfoaction (it carriesexternalId+channeland is exactly "synchronization activity information of the Order like export or import"), or a Custom Field. Query it back with thesyncInfo(externalId="…")predicate (or the custom-field predicate). - Timing. If Orders should export only after payment/approval, either subscribe to the state-change Message instead of
OrderCreated, or gate inside the handler on the Order's payment/approval state.
2. Inbound — service application as an inbound webhook (OMS → commercetools)
- Authenticate the caller and validate the payload before touching commercetools (security.md).
- Correlate the inbound event to the commercetools Order by the stored OMS id — query by the
syncInfo(externalId="…")predicate or a Custom Field predicate — not by position. - Apply as Order update actions, then persist. Common mappings (fetch exact action names via
openApi-schemata.mjs --resource-name api-Order-write):- order-level status →
changeOrderState(Open/Confirmed/Complete/Cancelled) and/or a customStatemachine viatransitionState - line-item fulfillment status →
transitionLineItemState(custom line-itemState) - shipment status →
changeShipmentState(Shipped,Delayed,Ready, …) - shipment/tracking →
addDelivery,addParcelToDelivery,setParcelTrackingData(andDelivery/Parcelcustom fields for extra data) - returns/RMA →
addReturnInfo,setReturnShipmentState - inventory → adjust the relevant
InventoryEntryquantityOnStockfor the SKU + supply channel (api-InventoryEntry-write)
- order-level status →
- Idempotent apply. Re-check current state before transitioning — a redelivered "Shipped" must be a no-op, and an out-of-order older event must not overwrite a newer state. Use the version/sequence the OMS provides (or the Order
versionfor optimistic concurrency) and guard the transition. Decide what a failed write returns so the OMS can retry safely.
3. Reconcile — job application (optional but recommended)
State mapping (produce this table for the user)
orderState, shipmentState, paymentState, a custom order State machine, and per-line-item State — the OMS usually has its own status vocabulary. Produce an explicit mapping table, e.g.:| OMS status | commercetools target | Action |
|---|---|---|
RECEIVED | order custom State = "Received" | transitionState |
ALLOCATED / PICKING | line-item State | transitionLineItemState |
SHIPPED (+ tracking) | shipmentState = Shipped; add delivery/parcel | changeShipmentState, addDelivery, addParcelToDelivery, setParcelTrackingData |
DELIVERED | orderState = Complete | changeOrderState |
CANCELLED | orderState = Cancelled | changeOrderState |
RETURN_INITIATED | return info | addReturnInfo, setReturnShipmentState |
orderState values, model them with a custom State machine and register the States + transitions idempotently in postDeploy (lifecycle-scripts.md). Decide up front which side wins on conflict for each field (source of truth), and make the other side read-only for that field.Idempotency & anti-loop (the invariants to pin as tests)
- Export idempotent on
orderNumber/OMS external ref (upsert or check-first) — redeliveredOrderCreateddoesn't create a duplicate OMS order. - Inbound idempotent — a redelivered status webhook is a no-op; an older event never overwrites a newer state (guard on version/sequence).
- No loops. If both an export subscription and an inbound webhook can touch order status, they must not master the same field. When the connector writes to the Order, that write emits its own Message — filter self-changes so the export doesn't re-push what the inbound flow just applied (event-applications.md self-change filtering).
- Correlation stored, not inferred — OMS id on the Order via
syncInfo(updateSyncInfo) or a Custom Field, never a bareexternalId(Order has none).
Checklist
- Flows chosen: export (
event), inbound (servicewebhook), reconcile (job) as needed - Export subscribes to the minimum Messages (
OrderCreated+ only the lifecycle Messages actually needed); re-fetches Order by id - Export idempotent on
orderNumber/OMS ref; OMS id recorded back on the Order - Inbound authenticates the caller, correlates by stored OMS id, applies via Order update actions, and is idempotent (redelivery no-op, no stale overwrite)
- State-mapping table produced; custom
Statemachine + transitions registered idempotently in postDeploy if needed - Single source of truth per field; no bidirectional sync of the same field; self-change filtering prevents loops
- Inventory sync direction fixed;
InventoryEntryupdated per SKU + supply channel - Reconcile job (if used) locks against overlap and checkpoints
Backend integration
Table of contents
- Server-side session creation (BFF)
- Creating the Order after payment
- Post-purchase: capture, refund, cancel
- Webhook reconciliation
- Who creates the Payment, revisited
Server-side session creation (BFF)
sessionId, the processor URL, and the enabler URL — never CT_CLIENT_SECRET or a manage_sessions token. The test harness (test-harness.md) cuts this corner for speed; the real integration must not.A single BFF endpoint does the three server steps and returns the session:
// POST /api/checkout/session — returns { sessionId, processorUrl, enablerUrl }
export async function createCheckoutSession(req, res) {
// 1. Verify the cart belongs to this user (IDOR guard) — fetch it and compare
// customerId / anonymousId to the authenticated caller before trusting cartId.
const cartId = req.session.cartId;
const token = await getManageSessionsToken(); // client_credentials, manage_sessions:{projectKey}
// 2. Ensure the cart is payable: recalculate and confirm a non-zero total
// (the processor rejects a €0 cart — see contract pitfall 3).
// 3. Create the Checkout Session (cartRef + processor-matching metadata)
const r = await fetch(`https://session.${region}.commercetools.com/${projectKey}/sessions`, {
method: 'POST',
headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' },
body: JSON.stringify({
cart: { cartRef: { id: cartId } },
metadata: { applicationKey: APP_KEY }, // or { processorUrl } — what the connector expects
}),
});
const session = await r.json();
res.json({ sessionId: session.id, processorUrl: PROCESSOR_URL, enablerUrl: ENABLER_URL });
}
Notes that matter:
- Create the session as late as possible — when the user reaches the payment step — because sessions expire. Don't mint it at cart creation.
- Verify cart ownership before creating a session (IDOR): fetch the cart and compare its
customerId/anonymousIdto the authenticated user. See the BFF responsibilities. - The browser never needs
projectKey/regionas public env vars — return them from this endpoint alongsidesessionId.
Creating the Order after payment
POST /orders:- The cart has shipping address, shipping method, and billing address (if required) — and the Payment is linked to the cart (the processor does the link via
addPayment; confirmcart.paymentInfo.paymentsis populated). - Payment authorization is complete for synchronous flows. For async PSPs, wait for the webhook to move the transaction to
Successbefore committing (see reconciliation). - You're using the latest cart version.
- Business validations (stock, min order value) pass.
// POST /api/checkout/place-order
const order = await apiRoot.orders().post({
body: {
cart: { typeId: 'cart', id: cartId },
version: cartVersion, // must be current
orderNumber, // unique, pre-generated → idempotency
},
}).execute();
orderNumber (a duplicate is rejected) and reuse the same value on retry; cart versioning gives you a second guard (a stale version fails). Creating the Order snapshots prices/payments and flips cartState to Ordered. An OrderCreated Message lets you trigger confirmation email / ERP sync via a Subscription — keep that work out of the request path.cartVersion for order creation. The processor bumps the cart version when it links the Payment to the cart via addPayment — this happens inside submit(), after the browser captured the version. Any version stored on the client (sessionStorage, URL param, hidden field) will be stale by the time the return URL fires. Always refetch the current cart version server-side immediately before calling POST /orders:// server-side, inside the order-creation route
const { body: cart } = await apiRoot.carts().withId({ ID: cartId }).get().execute()
// cart.version is now current — use it, not the client-supplied version
const order = await apiRoot.orders().post({ body: {
cart: { typeId: 'cart', id: cartId },
version: cart.version, // ← always freshly fetched
orderNumber,
}}).execute()
ConcurrentModification errors entirely on this path.Order-creation timing — pick one and be consistent:
- Authorize → create Order → capture on fulfillment (common for physical goods): the connector authorizes during
submit(); you create the Order on a successful authorization, then capture later. - Immediate capture → create Order (digital goods): the connector captures during
submit()(STRIPE_CAPTURE_METHOD=automatic); you create the Order once theChargeisSuccess.
If the cart total changed between authorization and order creation (discount expired, tax shift), don't silently proceed — cancel the authorization and re-authorize for the new amount, or surface the new total for confirmation. The processor/PSP handles the money; you orchestrate the decision.
Scopes for your BFF's own API client — a different client from the connector's
submit() under the connector's own runtime client (deploy-public-connector.md, "Two clients"). But the order-creation code above — refetch the cart, read paymentInfo.payments[0], GET that Payment to confirm a Success transaction, POST /orders — runs under your BFF's client, which needs its own scopes:view_payments:{projectKey}— to read the Payment and confirm it succeeded before creating the Order.manage_orders:{projectKey}— to create the Order (plus the cart scopes the rest of your BFF already uses).manage_payments:{projectKey}— only if your BFF writes back to a Payment outside the processor's operation routes (e.g. a reconciliation flag). Skip it otherwise.
CTP_CLIENT_ID/CTP_CLIENT_SECRET, and restarting the app (env vars aren't hot-reloaded). Request the full set up front, and cover this path in the integration test (integration-test.md) so a scope gap fails there rather than after a live charge — assert on the status code, not on a message string.Post-purchase: capture, refund, cancel
- Direct-connector path (this skill): the processor exposes its own operation routes for capture/refund/cancel. The template states "The processor application exposes additional API endpoints for initiating the capture, refund, and cancellation transactions." Call those processor routes (session- or service-authenticated per the connector) from your back-office/fulfillment backend. The processor talks to the PSP and writes the resulting
Charge/Refund/CancelAuthorizationtransaction onto the Payment. - Checkout-product path (not this skill): if the Payment was created by the hosted Checkout product, use the Checkout Payment Intents API (
manage_checkout_payment_intentsscope) to capture/refund/reverse/cancel. The Payment Intents API works only for payments created by Checkout — do not reach for it on the direct-connector path.
| Operation | Transaction added | When |
|---|---|---|
| Capture | Charge | funds taken (auto, or manual at fulfillment) |
| Cancel authorization | CancelAuthorization | void an auth before capture (order canceled/unfulfillable) |
| Refund | Refund | return captured funds; partial refunds allowed up to the captured amount, repeatable |
Charge per PSP interactionId) so a retried capture can't double-charge.Webhook reconciliation
onComplete. The processor verifies the webhook (e.g. signing secret / HMAC) and updates the transaction state. Your backend should:- Treat a UI "success" as provisional; gate Order creation (or order confirmation) on the transaction reaching
Successwhen the PSP is async. - If a transaction is stuck
Pending, suspect the webhook (endpoint registered, points at the processor, secret matches — see the provider reference). - Make handling idempotent: the same webhook may arrive twice.
The return URL race condition
Success. The browser redirect is nearly instant; the webhook delivery takes 1–5 seconds even in a healthy setup.Pending and fails with "no successful payment found."// return URL page — order creation with webhook-wait polling
const MAX_ATTEMPTS = 10
const GAP_MS = 1500 // 10 × 1.5s = 15s total — enough for any healthy webhook delivery
for (let i = 0; i < MAX_ATTEMPTS; i++) {
const res = await fetch('/api/orders/create', { method: 'POST', body: ... })
if (res.ok) return await res.json() // gate opened, order created
if (res.status !== 422) throw new Error(...) // hard error — don't retry
if (i < MAX_ATTEMPTS - 1) await sleep(GAP_MS) // 422 = still Pending, wait for webhook
}
throw new Error('Payment not confirmed after webhook timeout — check Stripe webhook delivery')
orderNumber must be pre-generated and stable across retries (idempotency — see above), so a retry that races a concurrent success doesn't double-create. A DuplicateField error on orderNumber means the first attempt already succeeded — fetch and return the existing Order.Who creates the Payment, revisited
Authorization/Charge and links it to the cart during submit()). Your backend does not create Payment objects — doing so produces duplicates. Creating Payments yourself is the raw BFF / custom-checkout model (no connector), documented separately at custom checkout → payment. Your backend's job is sessions, the Order, and post-purchase operations — not the Payment itself.Checklist
- Session/cart/token creation is server-side; browser gets only
sessionId+ processor/enabler URLs - Cart ownership verified before session creation (IDOR guard)
- Order created from cart with server-refetched version (never the client-supplied version — the processor bumps the cart via
addPaymentand makes any client-held version stale) + unique pre-generatedorderNumber(idempotent) - Order creation gated on authorization complete (and on webhook
Successfor async PSPs) - Return URL handler polls for Order creation (retries on 422 with a gap) rather than firing once — avoids the return-URL/webhook race condition
-
orderNumberis pre-generated and reused across retries so polling can't double-create - Capture/refund/cancel go through the processor's operation routes (not the Payment Intents API, which is Checkout-only)
- Post-order side effects (email, ERP) driven off the
OrderCreatedSubscription, not the request path - Backend does not create Payment objects (the processor owns them)
- The BFF's own API client has
view_payments+manage_orders(scopes can't be edited after client creation, and a gap only surfaces after a real charge)
Test-driving the backend
Hard rule: no implementation code before its test. Install Vitest and write the first failing test before writing any function body. If you find yourself with working code and no test, you have skipped this step — stop, write the test (it may already pass, which means it's now a regression guard rather than a design tool, but it still must exist), and confirm it would fail if the behavior were removed before continuing.
npm test runs (even with zero test files). This takes two minutes and means every subsequent test-first cycle has a working harness to run against. Do not defer this to "after the backend is done."npm install --save-dev vitest @vitest/coverage-v8
# add to package.json scripts: "test": "vitest run"
npm test # should exit 0 with "no test files found" — harness is live
This"test": "vitest run"is right for the BFF/storefront (which you run yourself). But for a custom connector, prefer Jest — the Connect platform validatesnpm testat publish and its examples (and the connector templates) use Jest. If you do use Vitest, run each app'stestthrough a wrapper that calls Vitest with a fixed arg list (Vitest aborts on unknown CLI options), and give every app — including theassetsenabler — atestscript. See stripe.md → "Prefer Jest for connector apps".
The loop
For each behavior, smallest first:
- Red — write one test that names the behavior and asserts the outcome. Run it. It must fail because the behavior is missing, not because the import is wrong or the mock isn't wired — a test that passes before you've written anything, or errors for a boring reason, is testing nothing. Read the failure and confirm it's the failure you expected.
- Green — write the least code that makes it pass. Resist generalizing; the next test will tell you what to generalize.
- Refactor — clean up with the test as a safety net.
orderNumber doesn't double-create" is a behavior worth a test; "the function calls fetch with these exact headers" is usually too brittle to be worth pinning unless the header is the behavior (the X-Session-Id auth header is — see below).Where to draw the test boundary
- Mock the outbound boundary (the processor's operation routes, the Sessions API, the PSP, the CT Orders/Payments API client) and assert on what your code decided to do: which endpoint it called, with what body, in what order, and what it did with the response. These tests are fast, deterministic, and run with no deployment and no secrets — so they run on every commit.
- Don't mock your own orchestration logic — that's the thing under test.
- Don't try to assert the PSP actually charged a card here. That's the job of the full-flow integration test (integration-test.md), which runs against a real deployed connector with test cards. Unit tests prove your decisions; the integration test proves the wiring.
processorClient with capture()/refund()/cancel(), a sessionsApi.create(), a ctOrders.create(). Tests inject a fake; production injects the real one. If you find a behavior hard to test, it's usually because the decision and the I/O are tangled — separating them is the refactor the test is asking for.vi.fn() → jest.fn() and they read identically under Jest or node:test.What to test, per backend piece
Success transaction → exactly one Order, Payment linked, cartState: Ordered." Write it first: it's the cheapest to get green, it forces the function's shape into existence, and without it a suite can drift into asserting every way the flow breaks while never asserting it actually works (a green build where the success path silently regressed). Then add the error and edge deviations below, which is where the real defects hide.BFF session creation
- Happy path: an owned, non-zero cart yields a
sessionIdand the processor/enabler URLs. This is the baseline the guards below deviate from. - IDOR guard: a session is created only when the cart belongs to the caller. Test the rejection path — a cart whose
customerIddiffers from the authenticated user must not produce a session. This is the test that matters most and the one most likely to be missing. - Secrets stay server-side: the object returned to the browser contains
sessionId,processorUrl,enablerUrland nothing else — assert the response has noaccess_token, no client secret. A snapshot or explicit key-set assertion catches a carelessres.json(session)that leaks the whole token response. - Non-zero cart: a €0 cart is refused before a session is minted (the processor would reject it anyway — contract pitfall 3).
import { describe, it, expect, vi } from 'vitest';
import { createCheckoutSession } from '../bff/session';
describe('BFF session creation', () => {
it('refuses to create a session for a cart the caller does not own (IDOR)', async () => {
const ctCarts = { get: vi.fn().mockResolvedValue({ id: 'cart-1', customerId: 'someone-else' }) };
const sessionsApi = { create: vi.fn() };
await expect(
createCheckoutSession({ cartId: 'cart-1', user: { customerId: 'me' }, ctCarts, sessionsApi }),
).rejects.toThrow(/forbidden|ownership/i);
expect(sessionsApi.create).not.toHaveBeenCalled(); // the real assertion: no session was minted
});
it('returns only sessionId + processor/enabler URLs to the browser', async () => {
const ctCarts = { get: vi.fn().mockResolvedValue({ id: 'cart-1', customerId: 'me', totalPrice: { centAmount: 1999 } }) };
const sessionsApi = { create: vi.fn().mockResolvedValue({ id: 'sess-1', accessToken: 'SECRET' }) };
const out = await createCheckoutSession({ cartId: 'cart-1', user: { customerId: 'me' }, ctCarts, sessionsApi });
expect(out).toEqual({ sessionId: 'sess-1', processorUrl: expect.any(String), enablerUrl: expect.any(String) });
expect(JSON.stringify(out)).not.toContain('SECRET'); // no token leaks to the client
});
});
Order creation
- Happy path: an owned cart whose linked Payment has a
Successtransaction creates exactly one Order at the current cart version and flipscartStatetoOrdered. This is the contract; the gates below are when it must not fire. - Gated on authorization: with no
Successtransaction on the linked Payment,placeOrdermust not callctOrders.create. For an async PSP, "authorization complete" means the webhook moved it toSuccess— so the gate is the same test with the transaction stillPending. - Declined payment never commits: a
Failuretransaction (card declined, insufficient funds — the most common real-world error path) must block Order creation just likePendingdoes, and the caller should get a clear decline back, not a generic 500. This is distinct fromPending:Pendingis "not yet,"Failureis "no" — and an Order built on a declined Payment is the worst outcome, an unpaid fulfilled order. - Idempotent on
orderNumber: two calls with the same pre-generatedorderNumbercreate at most one Order. Simulate the CT "duplicate orderNumber" rejection on the second call and assert your code treats it as success (returns the existing Order), not as an error to retry into a third attempt. - Uses the current cart version: a stale version is rejected; assert you refetch/propagate the version rather than reusing a cached one.
it('creates exactly one Order from an authorized cart and marks it Ordered (happy path)', async () => {
const created = { id: 'order-1', orderNumber: 'ord-1', cartState: 'Ordered' };
const ctOrders = { create: vi.fn().mockResolvedValue(created) };
const payment = { transactions: [{ type: 'Authorization', state: 'Success' }] }; // authorized
const order = await placeOrder({ cartId: 'c1', cartVersion: 3, orderNumber: 'ord-1', payment, ctOrders });
expect(ctOrders.create).toHaveBeenCalledOnce();
expect(ctOrders.create).toHaveBeenCalledWith(expect.objectContaining({ orderNumber: 'ord-1', version: 3 }));
expect(order.cartState).toBe('Ordered');
});
it('does not create an Order until a Success transaction exists', async () => {
const ctOrders = { create: vi.fn() };
const payment = { transactions: [{ type: 'Authorization', state: 'Pending' }] }; // async PSP, not settled
await expect(placeOrder({ cartId: 'c1', cartVersion: 3, orderNumber: 'ord-1', payment, ctOrders }))
.rejects.toThrow(/not authorized|pending/i);
expect(ctOrders.create).not.toHaveBeenCalled();
});
it('refuses to create an Order on a declined payment, surfacing the decline (error path)', async () => {
const ctOrders = { create: vi.fn() };
const payment = { transactions: [{ type: 'Authorization', state: 'Failure' }] }; // card declined
await expect(placeOrder({ cartId: 'c1', cartVersion: 3, orderNumber: 'ord-1', payment, ctOrders }))
.rejects.toMatchObject({ code: 'PaymentDeclined' }); // a clear decline, not a generic 500
expect(ctOrders.create).not.toHaveBeenCalled(); // never an unpaid Order
});
it('is idempotent: a duplicate orderNumber returns the existing Order, not an error', async () => {
const existing = { id: 'order-1', orderNumber: 'ord-1' };
const ctOrders = {
create: vi.fn().mockRejectedValueOnce({ statusCode: 400, code: 'DuplicateField', field: 'orderNumber' }),
getByOrderNumber: vi.fn().mockResolvedValue(existing),
};
const payment = { transactions: [{ type: 'Authorization', state: 'Success' }] };
const order = await placeOrder({ cartId: 'c1', cartVersion: 3, orderNumber: 'ord-1', payment, ctOrders });
expect(order).toEqual(existing); // a retry converges on the one Order, never a second
});
it('is idempotent: "cart not in active state" (cartState=Ordered) also returns the existing Order', async () => {
// After the first order creation succeeds, CT flips cartState → Ordered.
// A second POST /orders then fails with InvalidOperation "not in active state"
// *before* CT checks the orderNumber, so DuplicateField is never raised.
// The handler must also catch this case and return the existing Order.
const existing = { id: 'order-1', orderNumber: 'ord-1' };
const ctOrders = {
create: vi.fn().mockRejectedValueOnce({
statusCode: 400,
body: { errors: [{ code: 'InvalidOperation', message: 'The cart is not in active state.' }] },
}),
getByOrderNumber: vi.fn().mockResolvedValue(existing),
};
const payment = { transactions: [{ type: 'Authorization', state: 'Success' }] };
const order = await placeOrder({ cartId: 'c1', cartVersion: 4, orderNumber: 'ord-1', payment, ctOrders });
expect(order).toEqual(existing);
});
Post-purchase capture / refund / cancel
- Routes through the processor, never the Payment Intents API: assert the processor's operation route was called and that no Payment Intents endpoint (
/checkout/payment-intents, themanage_checkout_payment_intentspath) was touched. This is a guardrail test — its job is to fail loudly the day someone "simplifies" it to the wrong API. - Capture idempotency: one
Chargeper PSPinteractionId; a retried capture with the same interaction id doesn't double-charge. - Partial refund only when configured/allowed: a partial refund above the captured amount is rejected; multiple partial refunds sum correctly up to the captured total.
it('routes capture through the processor, not the Payment Intents API', async () => {
const processor = { capture: vi.fn().mockResolvedValue({ ok: true }) };
const paymentIntents = { capture: vi.fn() }; // the wrong API — must stay untouched
await capturePayment({ paymentId: 'pay-1', amount: { centAmount: 1999 }, processor, paymentIntents });
expect(processor.capture).toHaveBeenCalledOnce();
expect(paymentIntents.capture).not.toHaveBeenCalled(); // guardrail against the Checkout-only API
});
it('does not double-charge on a retried capture (idempotent by interactionId)', async () => {
const processor = { capture: vi.fn().mockResolvedValue({ interactionId: 'pi_123' }) };
const seen = new Set<string>();
await capturePayment({ paymentId: 'pay-1', interactionId: 'pi_123', processor, seen });
await capturePayment({ paymentId: 'pay-1', interactionId: 'pi_123', processor, seen }); // retry
expect(processor.capture).toHaveBeenCalledOnce();
});
Webhook reconciliation
This is where async PSPs live, and it's the piece most painful to exercise by hand because it depends on a signed event arriving — possibly twice. Tests pay off the most here.
- Idempotent on redelivery: the same webhook event id applied twice leaves the Payment in the same state and creates at most one transaction. PSPs will redeliver; assert it.
- Signature/verification is enforced: a tampered or unsigned payload is rejected before any state change. (For a custom processor you own this; for the public connector, test your own handler's gate if you have one in front.)
- Drives the gate the Order waits on: after the webhook moves the transaction to
Success, the Order-creation gate that was closed in the Order test now opens. A test that asserts "stuckPending→ no Order; webhook arrives → Order proceeds."
it('is idempotent when the PSP redelivers the same event', async () => {
const ctPayments = { addTransaction: vi.fn().mockResolvedValue({}) };
const processed = new Set<string>();
const event = { id: 'evt_1', type: 'payment_intent.succeeded', paymentId: 'pay-1' };
await handleWebhook({ event, ctPayments, processed });
await handleWebhook({ event, ctPayments, processed }); // redelivery
expect(ctPayments.addTransaction).toHaveBeenCalledOnce();
});
A note on not over-testing
Ordered, capture/refund recorded) plus the deviations that bite — IDOR, premature/declined Order, double-create, double-charge, wrong-API, webhook redelivery. That's roughly a dozen tests, and they're worth keeping forever. The happy path earns its place precisely because it's load-bearing: it's the one a broad refactor is most likely to break without any error test noticing. Resist mirroring every line of orchestration into an assertion; tests that pin implementation details (exact header order, internal call counts that aren't about idempotency) make refactoring miserable and tend to get deleted in frustration, taking the valuable tests with them. When in doubt, ask: "what production bug does this test catch?" If you can't name one, don't write it.Checklist
Gate: do not proceed to Step 5 (integration test / verification) until every box below is checked andnpm testexits 0 with no secrets in the environment.
- Vitest installed and
npm testruns before the first line of implementation — not after - Each backend behavior was written test-first: a failing test, confirmed to fail for the right reason, then the code; no function body existed before its test
- Outbound boundary (processor, Sessions/Orders API, PSP) is mocked behind a port; orchestration logic is not mocked
- Happy path pinned per piece: owned cart → session;
Success→ exactly one Order markedOrdered; capture/refund recorded - BFF: IDOR rejection tested; response asserted to carry no secrets; €0 cart refused
- Order: gated on a
Successtransaction (async = webhook); declined (Failure) payment refused with a clear decline (not a generic 500); idempotent onorderNumber; current cart version used - Capture/refund/cancel: routed through the processor with the Payment Intents API asserted untouched; capture idempotent by
interactionId - Webhook: idempotent on redelivery; signature verification enforced before any state change; opens the Order gate
-
npm testexits 0 with no deployment/secrets in the environment (those belong to the integration test)
From requirements to config
connect.yaml configuration, and most of it has a default that quietly bakes in a decision. So the job is: take the requirements gathered in Step 1, decide each value deliberately, and hand the user a filled config block with a one-line why per non-obvious key. This reference gives the provider-agnostic mapping; exact key names, defaults, and the secured-vs-standard split are in the provider reference (e.g. stripe.md).Table of contents
- The connect.yaml envelope
- The mapping
- How to present the result
- Worked example (Stripe)
- Pitfalls in the config itself
The connect.yaml envelope
connect.yaml — the authoritative spec is the documentation, not a linter — so the envelope is easy to get subtly wrong. Two rules close the common gaps.https://docs.commercetools.com/connect/development.md to confirm against the current spec) — read it rather than reconstructing the structure from memory.deployAs: # required — array of the connector's applications
- name: processor # required — must match the application's folder name in the repo
applicationType: service # service | event | job | merchant-center-custom-application | merchant-center-custom-view | assets
endpoint: /processor # required for service/event/job; omit for assets and the MC types
properties:
schedule: '*/5 * * * *' # job type only — cron expression
scripts: # optional — only if the app installs Extensions/Subscriptions
postDeploy: npm run connector:post-deploy
preUndeploy: npm run connector:pre-undeploy
configuration: # optional (omit for assets)
standardConfiguration: # non-secret env vars; each: key, description, required, default?
- key: CTP_PROJECT_KEY
description: ...
required: true
default: 'default-key' # default? is allowed here only
securedConfiguration: # secrets; each: key, description, required — NO default
- key: CTP_CLIENT_SECRET
description: ...
required: true
inheritAs: # optional — config/scopes shared across all applications
configuration:
standardConfiguration: [...]
securedConfiguration: [...]
apiClient:
scopes: # for auto-generated API Client credentials
- manage_payments
inheritAs.apiClientand self-suppliedCTP_CLIENT_ID/CTP_CLIENT_SECRETare mutually exclusive — declaring both is a deploy/install-time conflict. Pick one credential model, not both:
- Auto-generated (recommended): declare
inheritAs.apiClient.scopesand let Connect mint the credentials and inject them. Then remove the CT-client keys from your config —CTP_CLIENT_ID,CTP_CLIENT_SECRET, andCTP_SCOPEfromsecuredConfiguration, andCTP_PROJECT_KEYfromstandardConfiguration— Connect injects all of these at runtime, and leaving them declared causes a deploy conflict.- Self-supplied: declare
CTP_CLIENT_ID/CTP_CLIENT_SECRETinsecuredConfigurationand drop theinheritAs.apiClientblock; the deployer provides the values.ThesecuredConfigurationexample below shows the self-supplied half; if you keepinheritAs.apiClient, remove those CT-client keys.
name, applicationType, endpoint, properties (with schedule), scripts (postDeploy/preUndeploy), and configuration. Each config item is exactly { key, description, required } plus default for standardConfiguration only. Anything else — a type, value, env, secret, validation field on a config item, or a top-level key other than deployAs/inheritAs — is hallucinated. (Note: the connector author writes connect.yaml with these key declarations; the deployer supplies the actual value for each at deployment create time. The YAML itself carries no value field — don't add one.)connect.yaml lives at the repository root. It is the entry point Connect looks for, and it must sit at the top level of the connector repo — not inside processor/, enabler/, src/, or any nested folder. A nested connect.yaml is not discovered and the connector fails to stage/deploy with no obvious cause. The application folders (processor/, enabler/) are siblings below the root, and each app's name in deployAs points at its folder; the single connect.yaml at the root describes them all.my-stripe-connector/
├── connect.yaml ← here, and only here
├── processor/ ← name: processor, applicationType: service
└── enabler/ ← name: enabler, applicationType: assets
The mapping
| Requirement (Step 1) | Config concept it drives | Decision guidance |
|---|---|---|
| Region + project | the CTP_*_URL hosts (CTP_API_URL, CTP_AUTH_URL, CTP_SESSION_URL, CTP_CHECKOUT_URL), CTP_JWKS_URL, CTP_JWT_ISSUER, CTP_PROJECT_KEY | All must point at the user's region; defaults usually point at one region (often europe-west1.gcp) — change them or auth/session calls fail. |
| Capture mode (charge now vs. authorize→capture later) | capture-method key (e.g. automatic vs manual) | manual = authorize at submit(), you capture later via the processor on fulfillment → also delays when you create the Order (see backend). automatic = charged at submit(). |
| Saved payment methods / returning customers | saved-cards config + "setup future usage" | Enabling it requires the cart to carry a customerId (stored methods bind to a Customer). Off by default — only enable if the business wants reuse. |
| Partial refunds / split captures | multi-operations toggle | Off by default; enabling partial/multiple captures or refunds often also requires the capability enabled in the PSP account. Don't enable speculatively — it changes transaction handling. |
| Payment methods + drop-in vs components | layout / appearance / express-element config; the integration type chosen in the Merchant Center | Drop-in (one element) is simplest; web components give per-method control. Layout/appearance keys are cosmetic and safe to leave default. |
| Storefront origin(s) | allowed-origins (CORS) | Must list every exact origin the browser calls the processor from (scheme + host + port). Missing origin → processor CORS-rejects the browser. |
| Post-payment return URL | merchant-return-URL | Must be an absolute URL with a scheme — the enabler calls new URL() on it; a bare host throws and silently breaks the flow. |
| Payment interface naming | payment-interface value | The paymentMethodInfo.paymentInterface written on the Payment; pick a stable identifier so you can query payments by interface later. |
| Sync vs. async settlement | webhook id + signing secret (secured) | Required whenever final state arrives via webhook. Without it the transaction never finalizes. Drives whether Order creation waits on the webhook. |
| (always) PSP credentials, CT client | secured: PSP secret key, webhook signing secret, CTP_CLIENT_ID, CTP_CLIENT_SECRET | Always securedConfiguration, never standard, never hardcoded, never invented — the user supplies the real values. |
required flags, defaults, and runtime scope list can have moved. GET /connectors/key={connector-key} returns the authoritative configurations and apiClient.scopes — see deployment-installation.md, Pattern 4. This does not apply when you're authoring connect.yaml for a connector you're building, where this file is the source.How to present the result
Give the user four things, not a vague pointer:
- A filled
standardConfigurationblock with the chosen values inline. - The
securedConfigurationkeys they must set themselves (names only — never fabricate secret values). - The API-client scopes the connector needs (at minimum:
manage_payments,view_sessions; addmanage_ordersif the connector creates/links Carts or Orders). Two traps here:- Don't request
manage_projectas a shortcut. It's a broad superset that masks which scopes you actually need and over-privileges the connector; it also won't survive a least-privilege review or certification. List the specific scopes. - The runtime token scopes must match the declared scopes. Whatever the SDK client requests at token time (e.g.
withClientCredentialsFlow({ scopes: [...] })) must be covered by the API client's granted scopes. With auto-generated credentials (inheritAs.apiClient.scopes), requesting a scope you didn't declare — e.g.manage_project— fails token acquisition withinvalid_scope(400). Either omit the explicitscopesarray (inherit the client's scopes) or request exactly the declared set.
- Don't request
- A short rationale list: for each non-default or non-obvious key, one line tying it to the requirement it came from. This is what lets the user catch a wrong assumption.
Worked example (Stripe)
europe-west1.gcp, project acme, authorize now and capture on shipment, save cards for logged-in customers, partial refunds expected, drop-in, storefront at https://shop.acme.com (+ http://localhost:5173 for dev), return to https://shop.acme.com/order-complete.value: lines below are the deployment-time inputs the deployer supplies (e.g. via --configuration) — they are not part of connect.yaml itself, which only declares the keys (see the per-entry-fields note above):# processor — standardConfiguration — values shown are deployment inputs, NOT connect.yaml fields
- key: CTP_PROJECT_KEY
value: acme
- key: CTP_API_URL
value: https://api.europe-west1.gcp.commercetools.com # region
- key: CTP_AUTH_URL
value: https://auth.europe-west1.gcp.commercetools.com # region
- key: CTP_SESSION_URL
value: https://session.europe-west1.gcp.commercetools.com # region
- key: STRIPE_CAPTURE_METHOD
value: manual # authorize now, capture on shipment → also: create Order on auth, capture later
- key: STRIPE_SAVED_PAYMENT_METHODS_CONFIG
value: '{"payment_method_save":"enabled"}' # save cards → requires customerId on the cart
- key: STRIPE_ENABLE_MULTI_OPERATIONS
value: 'true' # partial refunds expected (also enable multicapture in the Stripe account)
- key: STRIPE_COLLECT_BILLING_ADDRESS
value: auto
- key: MERCHANT_RETURN_URL
value: https://shop.acme.com/order-complete # absolute URL w/ scheme
- key: ALLOWED_ORIGINS
value: https://shop.acme.com,http://localhost:5173 # every browser origin that calls the processor
- key: PAYMENT_INTERFACE
value: checkout-stripe # written on the Payment; query payments by this later
# processor — securedConfiguration (user supplies the values)
- key: STRIPE_SECRET_KEY # Stripe test/live secret key
- key: STRIPE_WEBHOOK_SIGNING_SECRET # verifies inbound Stripe webhooks
- key: CTP_CLIENT_ID
- key: CTP_CLIENT_SECRET
- key: CTP_SCOPE # required alongside ID/SECRET in the self-supplied model
# plus STRIPE_WEBHOOK_ID (standard) once the webhook endpoint exists
Rationale to hand back:
STRIPE_CAPTURE_METHOD: manual— they capture on shipment, so authorize at pay time and capture later via the processor; this is also why the Order is created on a successful authorization, not on charge.STRIPE_SAVED_PAYMENT_METHODS_CONFIG: enabled— saving cards binds methods to a Customer, so the cart must carry acustomerId; anonymous carts won't save.STRIPE_ENABLE_MULTI_OPERATIONS: true— partial refunds were required; this also needs multicapture enabled in the Stripe account.ALLOWED_ORIGINS/MERCHANT_RETURN_URL— the two values that silently break the browser flow if wrong; both pinned to the real storefront.
Pitfalls in the config itself
- Leaving region URLs at their defaults when the project is in another region → auth/session failures.
- Enabling saved cards without ensuring a
customerIdon the cart → no methods saved, confusing "why didn't it save" reports. - Enabling multi-operations in the connector but not in the PSP account → partial capture/refund calls fail at the PSP.
- A bare-host
MERCHANT_RETURN_URLor a missing origin inALLOWED_ORIGINS→ the frontend breaks at runtime, not at deploy.
Checklist
- every requirement from Step 1 mapped to a concrete value (no silent defaults left on behavior-changing keys)
- standardConfiguration filled; securedConfiguration listed by name only
- scopes stated; region URLs match the project's region
- rationale line per non-obvious key, tied to its requirement
- capture-mode decision reflected in the Order-creation timing (→ backend-integration.md)
Payment connector contract (provider-agnostic)
Table of contents
- Two URLs you need
- The 8-step flow
- Sessions API: the request body
- Loading the enabler
- Processor routes and auth
- Who owns the Payment object
- Pitfall catalog
- Configuration that breaks the frontend
Two URLs you need
A deployed connector exposes two public URLs (visible in the Merchant Center deployment view, or via the Connect deployments API):
- processor URL — the
serviceapp, e.g.https://service-….{region}.commercetools.app. Your frontend points the enabler at it; the enabler calls it; you can callGET /operations/statusdirectly. - enabler URL — the
assetsapp, e.g.https://assets-….{region}.commercetools.app. You load the enabler JS bundle from here.
deployment create gets a fresh URL. Reading them from config keeps you correct either way.The 8-step flow
1. OAuth token POST {auth}/oauth/token (client_credentials, manage_sessions[+])
2. Cart (non-zero total) POST {api}/{projectKey}/carts
3. Checkout Session POST https://session.{region}.commercetools.com/{projectKey}/sessions
4. Warm processor GET {processorUrl}/operations/status (cold-start guard)
5. Load enabler <script src="{enablerUrl}/connector-enabler.umd.js"> → window.<Global>.Enabler
6. Construct + build new Enabler({processorUrl, sessionId, onComplete, onError}) → createDropinBuilder('embedded') → build()
7. Mount + wait ready dropin.mount('#container'); wait for `ready` before enabling Pay
8. Submit dropin.submit() → processor authorizes/charges via PSP, writes the CT Payment
GET /payments itself (see pitfall 8).Sessions API: the request body
manage_sessions:{projectKey} (docs).POST https://session.{region}.commercetools.com/{projectKey}/sessions
Authorization: Bearer <token with manage_sessions>
{
"cart": { "cartRef": { "id": "<cartId>" } },
"metadata": { "applicationKey": "<checkout-application-key>" }
}
Two things the docs make non-obvious for the direct-connector path:
cart.cartRef.id— a reference to an existing cart, not an inline cart (see pitfall 1).metadatamust identify the processor the session is for. With a Checkout Application configured in the Merchant Center, that ismetadata.applicationKey. Some connector deployments instead validatemetadata.processorUrl(the processor checks the session's metadata matches its own deployed URL and otherwise returns 401 "Session is not active"). Use whichever the connector expects — if you get a 401 from the processor with a freshly created session, this metadata mismatch is the first thing to check. The provider reference notes which one a given connector wants.
id is the sessionId you hand to the enabler.activeCart.cartRef.id, not cart.cartRef.id. When the processor validates the session and reads the cart ID, use:const cartId = session.activeCart?.cartRef?.id;
session.cart?.cartRef?.id will always get undefined and return "Session has no cart reference".Loading the enabler
…enabler.es.js) and a UMD build (…enabler.umd.js) that attaches a global (e.g. window.Connector). The exact filename and global name are per-provider — see the provider reference.<script> tag. See pitfall 5 for why dynamic import() of the ES bundle fails in browsers.<script src="https://assets-….commercetools.app/connector-enabler.umd.js"></script>
<script>
const Enabler = window.Connector.Enabler; // global name is provider-specific
</script>
Then:
const enabler = new Enabler({
processorUrl,
sessionId,
locale: 'en-US', // pass the real locale; don't hardcode in prod
onComplete: (result) => { /* success → redirect to return URL */ },
onError: (err) => { /* surface err.message / err.code */ },
});
const builder = await enabler.createDropinBuilder('embedded');
const dropin = await builder.build({ showPayButton: false }); // own your Pay button
dropin.mount('#dropin-container');
Processor routes and auth
/operations + payment routes):GET /operations/config— public-ish config the enabler reads (publishable key, capture method, billing address setting, merchant return URL, etc.). On a custom connector you own this endpoint; ensure it returns at least the public key andmerchantReturnUrlso the enabler can configure the PSP's JS SDK and the redirect. Some PSPs need additional session-authenticated config routes beyond/operations/config(e.g. one that returns the real cartamount/currencyto initialize the payment element) — the provider reference documents any extra routes a given connector requires.GET /operations/status— health/readiness. Ping it right after session creation to warm a cold container (pitfall 10).- the payment route (the enabler calls this for you) — see pitfall 8.
- additional operation routes for capture/refund/cancel — not part of the storefront pay flow, but you call them from your backend for post-purchase money movements (→ backend-integration.md).
X-Session-Id: <sessionId> (the processor's session-authentication hook from @commercetools/connect-payments-sdk validates it). If you ever call a processor route directly, use X-Session-Id, not Authorization: Bearer (see pitfall 9).Who owns the Payment object
Authorization/Charge transaction, and records PSP interactions. Your frontend does not create Payment objects (that is the raw BFF model from custom checkout, which applies only when you integrate a PSP without a connector). Confusing the two leads to duplicate Payments. Verifying the round trip therefore means finding the Payment the processor wrote — see verification.md.Pitfall catalog
Each pitfall below cost real debugging time. Treat them as pre-flight checks.
1. Session body requires cartRef, not an inline cart
{ "cart": { "cartRef": { "id": "<cartId>" } } }.2. Session metadata must match what the processor expects
metadata (e.g. applicationKey or processorUrl) → processor returns 401 "Session is not active". First thing to check on a processor 401 with a fresh session.3. Cart must have a non-zero total
paidAmount >= cartAmount; a €0 cart is rejected ("already paid in full"). Easiest non-zero cart without needing a tax category: taxMode: ExternalAmount with a customLineItem carrying an externalTotalPrice. Example:{
"currency": "EUR",
"taxMode": "ExternalAmount",
"customLineItems": [{
"name": { "en": "Test item" },
"quantity": 1,
"money": { "currencyCode": "EUR", "centAmount": 1999 },
"slug": "test-item",
"externalTaxRate": { "name": "test", "amount": 0.0, "country": "DE" }
}]
}
4. Stale CT API Extension returns 502 on cart updates
addPayment — surfacing as a generic processor failure (500→502). Diagnose with GET {api}/{projectKey}/extensions and inspect each destination. API Extensions are project-global and may belong to tax, pricing, fraud, or another integration — deleting a live one silently breaks the project with no error. So do not delete one automatically: identify the suspect (destination URL matching the dead/old deployment), report it to the user with its key and destination, and remove it only on explicit confirmation — DELETE {api}/{projectKey}/extensions/key={key}?version=N. Modern templates do not register such an extension for the basic pay flow, but "likely legacy" is not proof — confirm the destination is actually dead before removing.5. Load the enabler via UMD script tag, not dynamic ES import()
import('…enabler.es.js') can fail with ERR_CONNECTION_CLOSED because the enabler internally imports the PSP's JS (e.g. @stripe/stripe-js), which injects its own script tag and trips up the ES-module loader in some browsers. Load the UMD bundle with a <script> tag and read the global (window.<Global>.Enabler).6. MERCHANT_RETURN_URL must be an absolute URL with a scheme
new URL(merchantReturnUrl), which throws on a bare host (e.g. the default 127.0.0.1/processor/callback/...). Set it to a real absolute URL like http://localhost:5173/payment-complete in the connector config.7. Wait for the enabler ready event before submit()
dropin.mount() returns before the PSP's payment iframe is actually ready. Calling submit() too early throws "could not retrieve data from the specified Element". Enable your Pay button only after the enabler signals ready (listen on the container, with a fallback timeout).8. The payment-creation route is GET, not POST
GET /payments. The enabler calls it for you — don't call it directly. If you're tempted to, you're probably reimplementing the enabler; don't.9. Processor auth header is X-Session-Id, not Bearer
X-Session-Id: <sessionId>. A GET /operations/status warm-up needs no auth; data routes need the session header. Authorization: Bearer will not authenticate you to the processor.10. Processor cold-start 504
GET {processorUrl}/operations/status right after creating the session to warm it before the enabler runs.11. Raw-body webhook parsing can reject empty POST bodies on all routes
POST with Content-Type: application/json and an empty/missing body fails, including the POST /payments call from the enabler. Defensive fix that works regardless of the plugin's quirks: always send body: "{}" (a valid empty JSON object) from the enabler, never an empty string or no body. The exact plugin behavior is provider-specific — see the provider reference (e.g. stripe.md for the fastify-raw-body v5 case).12. Deferred-intent: create the PSP intent inside submit(), not at mount time
submit() (your POST /payments call), then confirm with whatever token the create returns. Creating the intent at mount time instead — before the user has confirmed — leaves abandoned intents accumulating at the PSP, and confirming without the token the create returns fails. The provider-specific API names and the exact confirm sequence live in the provider reference — see stripe.md for the Stripe (stripe.elements() / clientSecret / confirmPayment) version.13. ConcurrentModification on Order creation — cart version is always stale from the client
addPayment on the cart inside submit() to link the newly created CT Payment. This bumps the cart version. Any cartVersion the browser captured before submit() (from the checkout page, sessionStorage, a URL param) is therefore stale by the time the return URL fires and Order creation runs. Passing it to POST /orders produces:"Object <cartId> has a different version than expected. Expected: 1 - Actual: 3."
POST /orders. The extra GET is cheap and eliminates this error entirely:const { body: cart } = await apiRoot.carts().withId({ ID: cartId }).get().execute()
// use cart.version — never the client-supplied value
Do not try to work around this by passing the version from the return URL query string or sessionStorage — those are just as stale. The only reliable source is a fresh GET.
14. Return URL fires before the webhook — Order creation gets a 422
MERCHANT_RETURN_URL (your payment-complete page) in under a second. The Stripe webhook that moves the CT Payment transaction from Pending to Success arrives 1–5 seconds later, even in a healthy setup. If your return URL handler calls Order creation immediately on page load, it hits the payment gate while the transaction is still Pending and gets back "no successful payment found" (422).orderNumber before the first attempt so retries reuse the same value and can't double-create:const MAX_ATTEMPTS = 10
const GAP_MS = 1500
for (let i = 0; i < MAX_ATTEMPTS; i++) {
const res = await fetch('/api/orders/create', { method: 'POST', body: JSON.stringify({ cartId, cartVersion, orderNumber }) })
if (res.ok) return await res.json()
if (res.status !== 422) throw new Error(await res.text()) // hard error — stop
if (i < MAX_ATTEMPTS - 1) await new Promise(r => setTimeout(r, GAP_MS))
}
throw new Error('Webhook timeout — check processor webhook secret and Stripe dashboard delivery log')
DuplicateField 400 on orderNumber means a concurrent retry already succeeded — fetch and return the existing Order. Do not use a fixed sleep: too short = still flaky; too long = bad UX. See backend-integration.md → Return URL race condition.15. Order creation returns 400 "cart is not in active state" on idempotent retry
POST /orders succeeds, CT flips cartState to Ordered. A second call with the same cartId then fails with InvalidOperation: The cart is not in active state before CT can check whether orderNumber is a duplicate. If your retry logic only catches DuplicateField 400, the second attempt throws instead of returning the existing Order.InvalidOperation with "not in active state" and fetch by orderNumber in that branch:const isDuplicate = err?.statusCode === 400 &&
err?.body?.errors?.some((e: any) => e.code === 'DuplicateField' && e.field === 'orderNumber');
const isCartOrdered = err?.statusCode === 400 &&
err?.body?.errors?.some((e: any) => e.code === 'InvalidOperation' && e.message?.includes('not in active state'));
if (isDuplicate || isCartOrdered) {
const { body: existing } = await apiRoot.orders().withOrderNumber({ orderNumber }).get().execute();
return existing;
}
16. The id the webhook records may not be the id the refund route needs
interactionId your webhook writes on the Success transaction is the PSP's authorization/intent id, but the PSP's refund API operates on a different object (the charge/capture), so passing the recorded id to refund returns a "not found" error. Two fixes: resolve the correct id from the PSP before refunding, or have the webhook handler record the refundable id on the Charge transaction in the first place. The provider-specific id types and lookup are in the provider reference — see stripe.md for the Stripe pi_xxx → ch_xxx case.17. /operations/status returns 401 during redeployment
Deploying), the old container is torn down before the new one is ready. During this window GET /operations/status — normally public/unauthenticated — returns 401. This is transient: wait for the deployment to reach Deployed, then the endpoint returns 200 as normal. Don't confuse this with an auth misconfiguration; if the 401 appears immediately after triggering a redeploy, it's the restart window.Configuration that breaks the frontend
| Config | Why it breaks the frontend | Fix |
|---|---|---|
MERCHANT_RETURN_URL | enabler new URL() throws on a bare host | absolute URL with scheme |
ALLOWED_ORIGINS | processor CORS-rejects the browser | include the frontend's exact origin |
| connector API-client scopes | session/payment calls 403 | manage payments + read sessions (provider reference lists exact set) |
| webhook id/secret (async PSPs) | transaction state never finalizes | register the PSP webhook, store its id/secret in secured config |
Webhook events — look up, then select for the use case
- Look it up. Consult the chosen PSP's official webhook-events documentation for the catalog of event types it emits (each PSP names them differently).
- Map to the lifecycle this skill cares about. The reconciliation only needs the events that move a commercetools transaction or open the Order gate: authorization succeeded, amount became capturable (authorize-now/capture-later), payment failed/declined, refund settled, and — for production — dispute/chargeback opened. Ignore events that don't change payment state.
- Select the minimal set for this user's flow. The capture mode, refund policy, and whether disputes must be handled (all gathered in Step 1 / config-from-requirements.md) decide which of the above apply. Example: a charge-now flow with no partial refunds needs the "succeeded" and "refunded" events but not "amount capturable"; a manual-capture flow does need the capturable event. Subscribe to what the use case requires, nothing more.
Register exactly that set when setting up the PSP webhook endpoint (the mechanics of registering are in the provider reference and the deploy guide). If a needed event isn't subscribed, the corresponding transaction silently never finalizes.
Checklist
- processor URL and enabler URL read from config (not hardcoded)
- session created with
cartRef+ processor-matchingmetadata; got asessionId - cart total is non-zero
- processor warmed via
GET /operations/status - enabler loaded from the UMD bundle; global resolved
- Pay button gated on the
readyevent;submit()only after - no stale API Extension pointing at a dead URL
- Payment object found after submit (→ verification.md)
- webhook events selected by looking up the PSP's docs and matching the user's use case (not a hardcoded list) — see Webhook events
- (Custom processor)
POST /paymentssendsbody: "{}"—fastify-raw-bodyv5 rejects empty bodies on all routes
Is a certified connector enough?
- Public connectors — listed in the Connect marketplace, ready to install. Some are built by commercetools (e.g. Adyen, PayPal), some by third parties (e.g. Stripe). If one covers the use case, this is almost always the right choice: install + configure, don't build.
- Organization (custom/private) connectors — deployed for your organization only. These come in two flavors that matter a lot here: a fork of an existing public connector's open-source repo (you extend it), or a connector built from scratch off the payment integration template. Both are commercetools-connect tasks.
Don't hardcode "what's supported" — check it live
- Run the skill's
docs-searchstep and/or query the commercetools Knowledge MCP for "supported PSPs payment methods payment connectors". - Read the live Supported PSPs, Payment Integration Types, and payment methods table: connectors-and-applications.md.
- Browse the live Connect marketplace for installable connectors and their versions: merchant-center/connect.md. For a third-party connector (e.g. Stripe), its own repo/README is the source of truth for capabilities and config keys.
connect.yaml; the Connect CLI registry is authoritative over the listing) before treating it as rung 1/2. If a good-match option turns out to be a non–Connect (partner/SaaS) integration, surface it but warn that this skill does not cover using non–Connect connectors and offer the build/fork path instead.The fit check
Compare the requirements gathered in Step 1 against what a candidate public connector actually supports. Check each dimension:
| Dimension | Question | If not covered → which rung |
|---|---|---|
| PSP | Is the user's PSP available as a public connector? | No public connector → rung 4 (build from template), or pick a different PSP. |
| Payment methods | Does it support the methods they need (cards, wallets, BNPL, local methods)? | Method missing → fork to add it (rung 3), or another connector/PSP. |
| Integration type | Drop-in vs. web components — does the connector offer what the storefront needs? | Type missing → may force the other type, else fork (rung 3). |
| Capabilities | Capture mode (manual/auto), saved payment methods, partial/multi capture & refund, regions/currencies | Re-check as config (rung 2) first; if genuinely missing → fork (rung 3). |
| Compliance/region | Is it available + certified for the user's region and currencies? | Not available in region → fork/build, or different PSP. |
| Special requirements | Each open-ended requirement from Step 1 (B2B PO numbers, subscriptions, split payments, custom fraud/risk hooks, PSP metadata/descriptors, surcharging, stored-credential mandates…) — does the public connector do it? | Re-check as config (rung 2); if it's bespoke processor logic → fork (rung 3); if it implies a PSP with no connector → rung 4. |
The decision ladder
- Public connector covers everything → install + configure (Step 2). Don't build anything. The common, recommended case.
- Supported PSP, gap looks like a capability → first prove it isn't config. Most "missing" behaviors on a supported PSP (partial refunds, manual capture, saved cards, layout) are
connect.yamltoggles, sometimes paired with a PSP-account setting. If a config closes the gap, you're back at rung 1. → config-from-requirements.md. - Supported PSP, genuine gap that config can't close → fork/extend the public connector. Its repo is open source (e.g.
stripe/stripe-commercetools-checkout-app); add the missing behavior to your fork and deploy it as an Organization connector. You keep the working processor/enabler contract, the session auth, the Payment-ownership model — and only change the delta. This is far cheaper and safer than rebuilding, and it's a commercetools-connect task (extending an existing connector). - No public connector for the PSP at all → build from the payment integration template → commercetools-connect. The from-scratch path, justified only when there's nothing to fork.
Only rungs 3–4 leave this skill (hand off to build/extend); the skill resumes once the resulting connector is deployed. Record the decision, the rung, and the connector version checked in the requirements block — so the rest of the work is grounded in a real, confirmed connector, not an assumed one.
Checklist
- Checked live marketplace + supported-PSPs docs (not memory); cited the connector + version
- Verified the candidate is a deployable Connect connector (not a partner/SaaS listing); asked the user, and warned if they chose a non–Connect integration this skill doesn't cover
- PSP, methods, integration type, capabilities, region each compared to the requirements
- Apparent capability gaps re-checked as config (rung 2) before considering any build
- When a public connector exists but has a real gap, chose fork/extend (rung 3) over build-from-scratch
- Decision + rung + connector version recorded: configure (1), config (2), fork (3 → commercetools-connect), or build (4 → commercetools-connect)
Deploy a custom (Organization) connector
The flow
0. connect validate → run the platform's checks LOCALLY, before staging (fast feedback)
1. connectorstaged create → registers your repo + tag, returns a connector id
2. connectorstaged publish → server-side validation (SAST/SCA + connect.yaml), makes it deployable (async)
3. deployment create → actually runs the connector in your project
manage_connectors + manage_connectors_deployments). No separate auth step.commercetools connect validate runs the same class of checks the platform runs at publish/preview (connect.yaml validation, image security analysis, SAST, SCA) — so running it before you stage turns a slow, async, server-side publish rejection into a local failure you fix in seconds. Do this before connectorstaged create/publish, not at deployment create: by the time you deploy, the code has already cleared validation at the publish gate, and deployment create fails on different things (missing config values, wrong scopes). For installing a public connector there's nothing for you to validate — it's already certified — so connect validate only applies to a connector you built or forked.Step 0 — Authenticate
Same as the public connector path:
commercetools auth login \
--client-credentials \
--client-id <CLIENT_ID> \
--client-secret <CLIENT_SECRET> \
--region <region e.g. europe-west1.gcp> \
--project-key <projectKey>
manage_connectors + manage_connectors_deployments (or manage_project).Step 1 — Stage the connector
commercetools connect connectorstaged create \
--name "my-connector" \
--description "Custom Stripe payment connector" \
--repository-url https://github.com/<org>/<repo>.git \
--repository-tag <git-tag> \
--creator-email <your-email> \
--supported-regions europe-west1.gcp \
--integration-types psp
| Pitfall | Detail |
|---|---|
| Wrong command path | The command is commercetools connect connectorstaged create — not bare connectorstaged create. The CLI binary is commercetools, not ct. |
No --region flag | connectorstaged create does not accept --region. Omit it — region is set via auth login. |
URL must end in .git | https://github.com/org/repo → error "not a valid Git repository URL". Use https://github.com/org/repo.git. |
--creator-email is required | Omitting it causes a flag validation error. Pass your email. |
| Private repo → "not reachable" | Connect clones the repo server-side. A private GitHub repo returns GitRepositoryNotReachable. Either make the repo public, or use the repo's SSH URL (git@github.com:org/repo.git) and grant read access to the connect-mu machine user — the documented way to give Connect access to a private repo. |
id in the response — you need it for step 2.Step 2 — Publish
commercetools connect connectorstaged publish --id <id-from-step-1>
- Only
--idor--key— there is no--forceflag. - Runs async — Connect clones your repo, validates
connect.yaml, and registers the connector. It can take a minute or two. You can check status withconnectorstaged describe --id <id>. - Once
statusshowspublished, proceed to step 3.
Publish runs a production-readiness scan — for private connectors too
connect.yaml. The validation process also runs image security analysis, SAST, and software composition analysis (SCA) over the code whenever you request a preview build or publish — for any connector, including private/Organization ones, not only for public marketplace certification. A connector that fails them won't publish, so clean the repo to the production bar before you publish, not after a rejected report. Catch most of it locally first:commercetools connect validate # connect.yaml + the same class of checks, locally
- No logs or any code/configuration that isn't meant for production. Strip leftover
console.log/debug logging, dev-only mocks or fixtures, test scaffolding, commented-out blocks, and local-only config (.envsamples,NODE_ENV=developmentdefaults baked into the build). If you forked a public connector (rung 3), this is where forks most often fail — leftover demo/sample code from the template. - No hardcoded URLs, tokens, credentials, or passwords in code or config — everything sensitive belongs in
securedConfigurationand is supplied at deploy time (see config-from-requirements.md). - No outdated/insecure dependencies, and stateless apps (no in-memory session state — the runtime scales and restarts).
commercetools-connect → security.md and observability-operations.md.The three scans fail for different reasons — read which one failed
Image security analysis, SAST and SCA analysis, Connector specification file validation, Application Build). Which one fails tells you where to look — they are not interchangeable:- Image security analysis failed (but SAST/SCA passed) → the finding is in the container base image's OS packages, not your code or your declared dependencies. The base image is chosen by the buildpack from your Node version, so the lever you control is pinning it. Add
engines.nodeto every app'spackage.json(e.g."engines": { "node": "20.x" }) so the buildpack selects a maintained, scanned-clean base image instead of a default. Astdlib-style CVE (e.g. a Go stdlib advisory) in this scan is the classic base-image symptom — it is never something in yourpackage.json. - Runtime-version vs. framework-version trap. Pinning the runtime can collide with a dependency's own engine requirement, and the two fixes can be mutually exclusive. Real example hit in the field: Fastify v5 requires Node 20+, but Fastify v4's transitive deps (
fast-uri,fast-json-stringify) carry HIGH-severity CVEs whose only fix is Fastify v5. So "pin Node 18" (image scan) and "downgrade Fastify to v4" (avoid a different finding) cancel out — the working combination was Fastify v5 + Node 20. When the image scan and the SCA scan seem to pull in opposite directions, check the framework's supported-Node matrix before downgrading anything; the fix for a dependency CVE is almost always to upgrade, not downgrade (downgrading lands you on the vulnerable version). - SCA failed → a declared dependency (in some
package-lock.jsonin the repo) has a known CVE. Note theFilefield in each finding — it tells you which lockfile. If it points at a folder that isn't a connector app (see "Keep the repo to connector apps only" below), the fix is removing that folder, not upgrading.
commercetools connect validate reproduces all of this locally (its buildpack is version-synced to the platform) — but the image scan step needs Docker running, and the buildpack pulls several GB of images, so ensure Docker has disk headroom or the build fails with opaque input/output errors that look like connector problems but are local-environment problems.Keep the repo to connector apps only — sibling folders poison SCA
connect.yaml. If you keep a storefront, BFF, or test harness in the same repo (e.g. a backend/ Next.js app alongside processor/ and enabler/), its dependencies get scanned too — and a stale storefront dep (old next, vite, vitest) will fail the connector's publish even though it ships none of that code. Keep the connector repo to the connector applications only; move any storefront/harness to its own repo (or, as a stopgap, .gitignore + git rm --cached it so it leaves the published git tag — the platform builds from the tag, though connect validate still scans the on-disk working tree).commercetools connect connectorstaged describe --id <id>
Step 3 — Deploy
Once published, deploy it into your project with your config:
commercetools connect deployment create \
--region <region> \
--connector-id <id-from-step-1> \
--key <your-deployment-key> \
--type sandbox \
--configuration 'processor.CTP_PROJECT_KEY=<value>' \
--configuration 'processor.CTP_CLIENT_ID=<value>' \
--configuration 'processor.CTP_AUTH_URL=<value>' \
--configuration 'processor.CTP_API_URL=<value>' \
--configuration 'processor.CTP_SESSION_URL=<value>' \
--configuration 'processor.CTP_CHECKOUT_URL=<value>' \
--configuration 'processor.CTP_JWKS_URL=<value>' \
--configuration 'processor.CTP_JWT_ISSUER=<value>' \
--configuration 'processor.STRIPE_PUBLISHABLE_KEY=<value>' \
--configuration 'processor.MERCHANT_RETURN_URL=<value>' \
--configuration 'processor.ALLOWED_ORIGINS=<value>'
--configuration flags too — the platform stores them encrypted: --configuration 'processor.CTP_CLIENT_SECRET=<value>' \
--configuration 'processor.STRIPE_SECRET_KEY=<value>' \
--configuration 'processor.STRIPE_WEBHOOK_SIGNING_SECRET=<value>'
connect.yaml (processor.KEY or enabler.KEY). Global (shared) config uses bare KEY=value.The deployment must include every application declared inconnect.yaml— including theassetsenabler, even though it takes no config. If you build the deployment draft by hand (e.g. via the REST API) and list onlyprocessor, the deploy may appear to succeed but is malformed: the enabler never deploys (no enabler URL is produced), and a laterredeployfails with the confusingDeploymentApplicationDoNotBelong— "deployment does not include application: 'enabler'". Include the enabler with empty config arrays:{ "applicationName": "enabler", "standardConfiguration": [], "securedConfiguration": [] }. The CLI'sdeployment createhandles this for you; raw API/scripted drafts are where this bites.
Step 4 — Get the URLs
commercetools connect deployment describe --key <your-deployment-key>
redeploy — the URLs do not change when you redeploy the same deployment. But a delete + recreate gives new URLs (the host id is per-deployment). If you ever recreate a deployment — e.g. to fix a malformed one that omitted an app — you must update everything that hardcoded the old URL: the BFF/storefront env (PROCESSOR_URL/ENABLER_URL) and the Stripe webhook endpoint (the old URL is now dead, so events silently stop arriving and transactions hang in Pending). Prefer redeploy over recreate whenever possible precisely to keep the URLs stable.Step 5 — Register the Stripe webhook
After you have the processor URL, go to the Stripe dashboard and register the webhook:
-
Stripe Dashboard → Developers → Webhooks → Add endpoint
-
Endpoint URL:
{processorUrl}/stripe/webhooks -
Subscribe to the events this user's flow needs — don't copy a fixed list. Look up Stripe's webhook-event catalog and select per the use case, as described in connector-contract.md → Webhook events. (For a typical Stripe flow that often means events like
payment_intent.succeeded,payment_intent.amount_capturable_updatedfor manual capture,payment_intent.payment_failed, andcharge.refunded— but confirm against Stripe's current docs and the user's capture/refund/dispute requirements.) -
Copy the signing secret (
whsec_…) -
Update the deployment's secured config via
redeploy— there is nodeployment updateCLI command, and the Connect REST API does not accept asetApplicationConfigurationaction (onlyredeployis a valid discriminator):commercetools connect deployment redeploy \ --key <your-deployment-key> \ --configuration 'processor.STRIPE_WEBHOOK_SIGNING_SECRET=<whsec_…>' \ --configuration 'processor.CTP_CLIENT_SECRET=<value>' \ --configuration 'processor.STRIPE_SECRET_KEY=<value>'After a redeploy the deployment goes back throughDeploying— same wait as the initial deploy. URLs remain stable.To pick up a newly published connector version, add--updateConnector:commercetools connect deployment redeploy \ --key <your-deployment-key> \ --updateConnector \ --configuration 'processor.KEY=value'Without--updateConnector,redeploykeeps the current connector version and silently does not update the deployed code — it only refreshes config and restarts. -
Optionally set
processor.STRIPE_WEBHOOK_ID=<we_…>for the post-undeploy cleanup script
Checklist
- CLI authenticated with
manage_connectors+manage_connectors_deployments - Repo is public (or private via SSH URL with
connect-mugranted read access) -
connectorstaged createused--repository-urlending in.git, included--creator-email - Production-ready before publish (applies to private too):
commercetools connect validatepasses before staging; no debug/console.loglogging, dev mocks, test scaffolding, commented-out code, or local-only config left in the repo; no hardcoded secrets/URLs; deps current; apps stateless -
engines.nodepinned (e.g.20.x) in every app'spackage.json(image-scan base image); dependency CVEs resolved by upgrading, not downgrading - Every app has a passing
testscript; Vitest apps route through a wrapper that ignores the buildpack's injected Jest flags - Repo contains only connector apps — no storefront/BFF/harness folder whose lockfile would be SCA-scanned
- Deployment draft lists all apps from
connect.yaml, including theassetsenabler (empty config) — else redeploy fails and no enabler URL is produced -
connectorstaged publishcompleted (status =published) -
deployment createpassed all required config, secrets in secured config - processor URL + enabler URL captured from
deployment describe - Stripe webhook registered at
{processorUrl}/stripe/webhooks; signing secret stored in secured config
Deploy a public payment connector
Two clients — don't conflate them
This trips people up, and conflating them is the usual cause of auth/scope failures:
| Client | Used for | Scopes |
|---|---|---|
| CLI / deploy client | authenticating the CLI to create the deployment | manage_connectors_deployments:{projectKey} + view_connectors:{projectKey} (to resolve the connector) — or manage_project:{projectKey}, which covers both. Plus manage_api_clients:{projectKey} if the connector auto-generates its runtime API client credentials; manage_project does not cover that one, and without it the deploy fails with 403 access denied. manage_connectors is the creator scope for staging/publishing your own connector — not needed to install someone else's — see Connect authorization and modify a connector |
| Connector runtime client | the credentials the deployed connector uses to call commercetools at runtime (create Payments, read sessions) | auto-generated at deploy time — the Connect platform shows the scopes it needs (e.g. manage_payments, view_sessions) during the deploy step and provisions them; you usually don't hand-create this client |
manage_deployments (not a real scope — it's manage_connectors_deployments:{projectKey}), or pre-creating a runtime client with payment scopes and trying to authenticate the CLI with it. The CLI client needs the connector/deployment scopes above; the payment/session scopes belong to the auto-provisioned runtime client.manage_project does not cover manage_api_clients — and no token-time trick changes that. The API Clients API states: "Due to the sensitive nature of this API, it can not be used with the manage_project:{projectKey} scope, but only with manage_api_clients:{projectKey}." So whenever the connector auto-generates its runtime credentials, manage_api_clients:{projectKey} must be granted on the CLI/deploy client itself (Connect — modify a connector documents the requirement and the 403 access denied symptom). Because scopes are immutable after an API Client is created, adding it means provisioning a new client — so decide before you create one.Step 1 — Authenticate the CLI
--region must match the project's region:commercetools auth login --client-credentials \
--client-id <CLI_CLIENT_ID> \
--client-secret <CLI_CLIENT_SECRET> \
--region <region e.g. europe-west1.gcp> \
--project-key <projectKey>
manage_connectors_deployments:{projectKey} + view_connectors:{projectKey} (or manage_project:{projectKey}, which covers both), plus manage_api_clients:{projectKey} on that same client if the connector auto-generates its runtime credentials — see the caveat above. Docs are explicit that the scope goes on the API Client running the deployment, not into the connector's connect.yaml.Step 2 — Deploy the public connector directly
connectorstaged step (that command stages your own connector for certification, which is a different, build-side flow). Deploy it:commercetools connect deployment create \
--region <region> \
--connector-key <public-connector-key> # or --connector-id <id> \
--type sandbox # preview | sandbox | production \
--key <your-deployment-key> \
--configuration '<applicationName>.<KEY>=<value>' \
--configuration '<KEY>=<value>'
- Pass the config you derived in Step 2 of the skill via repeated
--configurationflags ({applicationName}.{key}=valuefor app-specific,{key}=valuefor global). Secrets go here too — they land in the connector's secured config, not in the browser. - During this step the platform surfaces the runtime scopes the connector will be granted (the auto-generated client) — review them; that's expected, not an error.
- Find the connector's key/id in the Connect marketplace (Merchant Center → Connect) or via the Connect API.
Step 3 — Get the URLs back
commercetools connect deployment describe --key <your-deployment-key>). Bring those back to the skill's Step 2 (config) / Step 4 (backend) — they're what the BFF and enabler point at. A URL is stable across redeploys of the same deployment but a fresh deployment create gets a new one, so read them from config rather than hardcoding.A required config value that only exists after the first deploy
required: true, but the webhook endpoint can only be registered at the provider once the processor URL exists — which only exists after deploying. (The same shape appears on the custom-connector path, see deploy-custom-connector.md.) Break the cycle:deployment createwith a recognizable placeholder (e.g.placeholder_pending) for those keys, so the deploy doesn't fail on the required check.deployment describe --key <key>→ read the real processor URL.- Register the webhook endpoint at the provider against that URL.
deployment redeploy --key <key> --configuration '<app>.<WEBHOOK_ID_KEY>=<value>' --configuration '<app>.<WEBHOOK_SECRET_KEY>=<value>'to replace the placeholders.
redeploy merges rather than replaces config (deployment-installation.md, Pattern 4), step 4 only needs the keys you're actually changing — but a placeholder left un-replaced also stays put silently, so verify the final values rather than assuming the redeploy fixed everything.If something fails
403 access deniedon deploy → first checkmanage_api_clients:{projectKey}if the connector auto-generates credentials; that one is easy to miss becausemanage_projectdoesn't imply it. Otherwise the client is missingmanage_connectors_deployments:{projectKey}/view_connectors:{projectKey}(or you used a non-existent scope likemanage_deployments). Scopes can't be edited after client creation — provision a new client and re-login.- "connector not found" → wrong
--connector-key/--connector-id, or the connector isn't available to your organization yet (install it from the marketplace first). - Region mismatch →
--regionon bothauth loginanddeployment createmust equal the project's region. - Anything about building, bundling, staging, or certifying a connector → that's commercetools-connect, not this path.
Checklist
- CLI authenticated with a client that has
manage_connectors_deployments:{projectKey}+view_connectors:{projectKey}— ormanage_project:{projectKey}— andmanage_api_clients:{projectKey}on top if credentials are auto-generated (manage_projectdoes not cover it) - Deployed via
deployment create --connector-key …(noconnectorstagedfor a public connector) - Config passed via
--configuration; secrets in secured config, never the browser - Runtime scopes reviewed at deploy time (auto-generated client — expected)
- processor URL + enabler URL captured for the integration steps
The full-flow integration test
Charge, that the PSP webhook actually reaches the processor and finalizes the transaction. That's the gap this test closes: one automated test that drives the real, deployed pieces end to end and asserts the trace each step leaves in commercetools.Prerequisites
What it is (and isn't)
- It runs against a real deployed connector (processor + enabler URLs from config) and a real commercetools project, using the PSP's test cards — never live cards, never production keys.
- It reuses the test-harness.md flow as its driver for the browser half (session → enabler → submit), and verification.md as its oracle for the backend half (find the Payment, read its transactions).
- It is not a unit test and should not run on every commit. It needs secrets and a live deployment, it's slower, and a PSP sandbox hiccup can make it flake. Run it in a dedicated job (nightly, pre-release, or post-deploy smoke), gated on the connector config being present — skip with a loud, explicit message when config is absent so a missing secret reads as "not configured here," never as a silent pass. A silent pass on a missing secret is the same as having no test.
- Keep it to one or a few scenarios. Its value is breadth (it touches everything), not depth (the unit tests own the edge cases).
The flow it asserts
1. Mint session server-side (BFF) -> assert: sessionId returned; response carries no secrets
2. Drive enabler + submit a test card -> assert: onComplete fired / no enabler error
3. Find the Payment (verification.md) -> assert: cart.paymentInfo has a Payment; transaction is Success;
paymentInterface matches the connector; exactly one Payment
4. Place the Order -> assert: Order created; cartState -> Ordered; idempotent on orderNumber
5. Capture (if manual) via processor -> assert: a Charge/Success transaction appears on the Payment
6. Refund via the processor route -> assert: a Refund transaction appears; Payment Intents API never called
7. Webhook reconciliation (async PSP) -> assert: transaction reaches Success after the webhook (poll, don't sleep)
Steps 5–7 are conditional on the requirements from Step 1: skip capture if the flow is immediate-charge, skip the webhook wait for a fully-synchronous method. Assert only what the configured flow actually does — a test that asserts a manual capture against an automatic-capture deployment is testing the wrong contract.
Shape
submit() returns, so the Payment isn't Success the instant the browser says "done." Poll with a timeout; never a fixed sleep. A fixed sleep is either too short (flaky) or too long (slow) — polling is both faster and more reliable.import { describe, it, expect, beforeAll } from 'vitest';
import { loadConnectorConfig } from './support/config';
import { runHarnessFlow } from './support/harness'; // the test-harness.md flow, scripted
import { findPaymentForCart, findPaymentsForCart, getPayment, getCart } from './support/ct';
import { placeOrder, capture, refund } from '../src/backend';
const cfg = loadConnectorConfig(); // PROCESSOR_URL, ENABLER_URL, CT creds, project, region
const itLive = cfg ? it : it.skip; // skip (loudly) when no deployment is configured
// poll until a predicate holds, so async webhook settlement doesn't force a brittle sleep
async function until<T>(fn: () => Promise<T>, ok: (v: T) => boolean, { tries = 20, gapMs = 1500 } = {}) {
for (let i = 0; i < tries; i++) {
const v = await fn();
if (ok(v)) return v;
await new Promise(r => setTimeout(r, gapMs));
}
throw new Error('condition not met within timeout — suspect the webhook (see backend-integration.md)');
}
describe('connector full flow (live deployment, test card)', () => {
itLive('session -> pay -> Order -> capture -> refund leaves the right CT trace', async () => {
// 1-2. server-side session + drive the enabler to submit a test card
const { sessionId, cartId, result } = await runHarnessFlow(cfg, { testCard: '4242424242424242' });
expect(result.error).toBeUndefined();
// 3. the processor wrote the Payment — find it via the cart (verification.md)
const payment = await until(
() => findPaymentForCart(cfg, cartId),
p => !!p && p.transactions.some(t => t.state === 'Success'),
);
expect(payment.paymentMethodInfo.paymentInterface).toBe(cfg.paymentInterface); // e.g. 'checkout-stripe'
const payments = await findPaymentsForCart(cfg, cartId);
expect(payments).toHaveLength(1); // no duplicate Payment (frontend didn't create one)
// 4. place the Order — and prove idempotency by doing it twice with the same orderNumber
const orderNumber = `it-${sessionId}`; // deterministic per run; reused on retry
const order = await placeOrder({ cartId, orderNumber });
const again = await placeOrder({ cartId, orderNumber });
expect(again.id).toBe(order.id); // converges on one Order
expect(order.orderState).toBeDefined();
const cart = await getCart(cfg, cartId);
expect(cart.cartState).toBe('Ordered');
// 5-6. capture then refund via the processor's operation routes
if (cfg.captureMode === 'manual') {
await capture({ paymentId: payment.id });
const captured = await until(() => getPayment(cfg, payment.id),
p => p.transactions.some(t => t.type === 'Charge' && t.state === 'Success'));
expect(captured).toBeTruthy();
}
await refund({ paymentId: payment.id, amount: { centAmount: 100 } });
const refunded = await until(() => getPayment(cfg, payment.id),
p => p.transactions.some(t => t.type === 'Refund'));
expect(refunded).toBeTruthy();
}, 90_000); // generous timeout: cold starts + webhook settlement
});
runHarnessFlow is the test-harness.md 8-step flow scripted instead of clicked — driven headlessly (e.g. Playwright loading the enabler UMD, or, if the connector supports it, replaying the processor calls the enabler would make). Stay close to the harness you already proved by hand; the integration test is that harness with assertions and an Order/capture/refund tail bolted on.Reading a failure
| First failing step | Most likely cause | Where |
|---|---|---|
| 1 — no sessionId / secret leaked | BFF wiring, session metadata mismatch | connector-contract.md pitfalls 1–2 |
| 2 — enabler error / no onComplete | enabler load, cold start, ready timing | connector-contract.md pitfalls 5, 7, 10 |
3 — no Payment, or stuck Pending | submit never reached processor, or async webhook | verification.md, backend-integration.md |
| 3 — duplicate Payment | frontend wrongly created a Payment | connector-contract.md |
| 4 — Order not created / not idempotent | gate or orderNumber reuse wrong | backend-integration.md |
| 6 — refund 404/wrong call | reached for the Payment Intents API | backend-integration.md |
7 — never reaches Success | webhook not delivered/verified | provider reference → webhook setup |
Checklist
Gate: only write this test after the unit suite from backend-tdd.md exits 0.
- Unit suite green before this test was written — not after
- One end-to-end test drives a real deployed connector with a PSP test card (never live keys)
- It asserts the CT trace at each commit point (session, Payment+Success, Order, capture, refund), so a failure localizes the broken seam
- Reuses the test-harness.md flow as the driver and verification.md as the oracle
- Async settlement handled by polling with a timeout (
until()helper), not a fixed sleep - Asserts no duplicate Payment and that capture/refund went through the processor routes (not the Payment Intents API)
- Skips loudly (explicit
it.skiporconsole.warnwith a clear message) when deployment/secrets are absent — a silent pass on a missing secret is a broken test - Runs in a dedicated job (nightly/pre-release/post-deploy), not on every commit
Payment connector — direct integration (backend-focused)
- processor (a
service) — talks to the PSP, orchestrates payment operations, and owns the commercetools Payment object (creates it, adds transactions). Its behavior is driven by itsconnect.yamlconfig. You authenticate to it with a Checkout Session. - enabler (an
assetsbundle) — a browser JS library on top of the PSP's UI components. It renders the payment UI and calls the processor. This is the frontend touchpoint — necessary, but a thin slice of the work.
@commercetools/checkout-browser-sdk (that's the hosted Checkout product → commercetools-checkout); you do not create Payment objects yourself (the processor does); and capture/refund go through the processor, not the Checkout Payment Intents API. If you're building the connector itself, that's commercetools-connect."Checkout" is overloaded. Lowercase = the buying journey (always present). Uppercase Checkout = the commercetools product that runs that journey for you. On this path there is a checkout, but no Checkout product — which is exactly why the Payment is owned by the processor and refunds use the processor's routes, not the Payment Intents API. See backend-integration.md.
Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<payment terms from the user's request, e.g. 'payment connector processor session capture refund webhook'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root, where scripts/docs-search.mjs lives.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or https://docs.commercetools.com for deeper follow-up.Step 1 — Extract requirements (do this before any config or code)
connect.yaml values, and the wrong default silently bakes in the wrong behavior. So extract the requirements first; each answer maps to a concrete config key in Step 2. Ask the user (don't assume):- Which PSP / connector, and is it deployed? Get the connector and version and, if deployed, its processor URL and enabler URL (Merchant Center deployment view, or the Connect deployments API).
- Region and project? e.g.
europe-west1.gcp, projectmy-project— the Sessions API host and theCTP_*_URLconfig are region-specific. - Capture mode? Charge immediately, or authorize now and capture later (on fulfillment)? → drives the capture-method config and when you create the Order.
- Saved payment methods / returning customers? Should cards be saved for reuse? → drives the save-cards config and requires a
customerIdon the cart. - Refunds / partial captures? Will the business do partial refunds or split captures? → drives the multi-operations config.
- Which payment methods, and drop-in vs. web components? Drop-in (one element) is the default; web components give per-method layout control.
- Storefront origin(s) and post-payment return URL? → drives CORS and the return-URL config (a frequent silent breaker).
- Sync or async settlement? Some methods/PSPs finalize via webhook → drives whether Order creation waits on the webhook.
- Anything special or non-standard? (always ask — open-ended) The eight questions above cover the common shape, but they don't cover everything, and the requirements that decide config-vs-fork-vs-custom are often the ones a fixed list never asks. So explicitly ask the user: "Beyond the above, are there any specific constraints or behaviors you need?" Prompt with examples to jog memory — compliance/regulatory (PCI scope, SCA/3DS exemptions, local mandates), B2B (purchase-order numbers, invoices, multi-buyer approval), subscriptions/recurring or installments, multi-currency or per-market pricing, marketplace split payments/payouts, existing PSP contract terms or a specific PSP account/merchant id, custom fraud or risk-scoring, surcharging, stored-credential mandates, or anything that must appear on the PSP side (metadata, descriptors). Capture each as its own requirement line; don't force it into one of the eight slots.
Step 1.5 — Is a certified connector enough? (decide before wiring or building)
docs-search script/the Knowledge MCP), compare the requirements PSP-by-method-by-capability, and name the connector version you checked.- Public connector covers everything → install + configure (Step 2). Don't build. Installing it (CLI auth, scopes,
deployment create) is covered in deploy-public-connector.md — note it is not theconnectorstagedflow. - Supported PSP, gap looks like a capability → prove it isn't config first. Most "missing" behaviors (partial refunds, manual capture, saved cards) are
connect.yamltoggles → back to rung 1. See config-from-requirements.md. - Supported PSP, genuine gap config can't close → fork/extend the public connector (its repo is open source); add only the delta and deploy as an Organization connector. Don't rebuild — you'd throw away a working, maintained connector. Hand off to commercetools-connect. For monitoring the connector you build: deployment logs, structured logging, and the poison-message runbook are in
observability-operations.md. - No public connector for the PSP at all → build from the payment integration template. This can be done inline (within this skill session) when the user explicitly asks to build custom — see the stripe.md "Building a custom Stripe connector" section for the key gotchas (raw body, API version, POST vs GET route). For staging, publishing, and deploying the built connector see deploy-custom-connector.md. Hand off to commercetools-connect when the full Connect publish/certification lifecycle is the goal. For monitoring:
observability-operations.md.
Step 2 — Derive the provider config from the requirements
connect.yaml values for the chosen connector, and give a one-line why for each so the user can sanity-check it. The mapping (which requirement → which key) and the provider-specific key names/defaults live in the provider reference — read config-from-requirements.md for the provider-agnostic mapping table and the worked example, plus the matching provider reference (stripe.md) for exact key names, defaults, and secured-vs-standard split.connect.yaml has no published JSON Schema — its structure is defined only by the docs (Configure connect.yaml), so use only the documented envelope keys (deployAs/applicationType/configuration/inheritAs, each config item being {key, description, required, default?}) and don't invent fields. And it must live at the repository root, never in a nested folder (processor/, src/) — a misplaced file silently fails to deploy. Both are covered in config-from-requirements.md → The connect.yaml envelope.Produce, for the user:
- a filled standardConfiguration block (region URLs, capture method, saved-cards, multi-ops, billing collection, return URL, allowed origins, payment-interface name, …),
- the securedConfiguration keys they must supply (PSP secret key, webhook signing secret, CT client id/secret) — names only, never invent values,
- the API-client scopes the connector needs,
- a short rationale per non-obvious key tied back to their requirement.
MERCHANT_RETURN_URL must be an absolute URL with a scheme; ALLOWED_ORIGINS must include the storefront origin; scopes must cover managing payments + reading sessions. These appear again as runtime pitfalls in connector-contract.md.deployment create --connector-key, passing this config) — see deploy-public-connector.md, which also lists the correct Connect scopes and warns against the wrong-scope / connectorstaged pitfalls. Building or staging your own connector is the broader Connect flow → commercetools-connect. Either way, hand over the config block you derived here.Step 3 — Frontend touchpoint (reference)
submit(). This contract is the same across PSPs and is fully covered — including the load/timing pitfalls (UMD vs ES, the ready event) — in connector-contract.md. For a quick proof-of-life before wiring the real storefront, scaffold the throwaway harness in test-harness.md. Treat this as a supporting step: the substance of this skill is the config (Step 2) and the backend (Step 4).Step 4 — Build the backend (the main body of work), test-first
- Write a failing test that names the behavior and asserts the outcome.
- Run it. Confirm it fails for the right reason — not a missing import, not a wrong mock, but because the behavior is absent. A test that passes before you've written the code is testing nothing and must be fixed before proceeding.
- Write the least code that makes it pass. No extra logic, no generalizing ahead of the next test.
- Refactor with the test as a safety net. Then repeat for the next behavior.
Success, processor-owns-the-Payment, the IDOR guard) are invisible at the call site and only surface under conditions that are tedious to reproduce by hand — a retried webhook, a stale cart version, an unsettled async PSP. Each is one cheap assertion. Writing the test first pins the behavior and leaves a tripwire so the next change can't quietly undo it.- Every behavior listed in the backend-tdd.md checklist has a passing test.
- The test suite runs clean with
npm testand no secrets in the environment.
- Server-side session creation (BFF) — mint token/cart/session on the server so secrets and
manage_sessionsnever reach the browser; verify cart ownership (IDOR) and create the session as late as possible. The browser gets onlysessionId+ processor/enabler URLs. - Order creation — convert the cart to an Order after authorization completes (and, for async settlement, after the webhook confirms
Success), with a unique pre-generatedorderNumberfor idempotency. Timing follows the capture mode chosen in Step 1. - Post-purchase operations — capture / refund / cancel on the authorized Payment via the processor's own operation routes, not the Checkout Payment Intents API (which only works for payments the Checkout product created). Whether partial captures/refunds are even available depends on the multi-ops config from Step 2.
- Webhook reconciliation — treat the commercetools Payment (driven by the PSP webhook the processor verifies) as the authoritative state, not the browser's
onComplete. A transaction stuckPendingalmost always means the webhook.
Step 5 — Verify the round trip, then lock it in with a full-flow integration test
onComplete/return URL fired, and a commercetools Payment exists for the cart with a transaction (Authorization or Charge) in state Success, and — for production — the Order was created and a refund path works. See verification.md for the manual round-trip check.References
| Need | Reference |
|---|---|
| Is a certified connector enough?: fit-check a use case against public connectors vs. building custom, using live marketplace/docs data | connector-selection.md |
Deploy a public connector: CLI auth, the correct Connect scopes, and deployment create --connector-key (not connectorstaged) | deploy-public-connector.md |
Deploy a custom connector: connectorstaged create → publish → deployment create for Organization connectors (rung 3/4), with CLI pitfalls (URL format, private repo, required flags) and the production-readiness scan that runs at publish (SAST/SCA, no dev logs/code) — for private connectors too | deploy-custom-connector.md |
Requirements → config mapping: which requirement drives which connect.yaml key, with a worked example producing a filled config + rationale | config-from-requirements.md |
| The backend: server-side session/BFF, Order creation after payment, capture/refund/cancel via the processor, webhook reconciliation, who owns the Payment | backend-integration.md |
| Test-drive the backend: the red-green loop, what to assert vs. mock per piece (BFF/Order/capture-refund/webhook), turning the skill's invariants into Vitest regression tests | backend-tdd.md |
| Full-flow integration test: one end-to-end test against a real deployed connector + test card, asserting the CT trace at each commit point (the capstone of Step 5) | integration-test.md |
Stripe specifics: connector repo/version, exact connect.yaml keys (standard/secured) + defaults, enabler bundle name/global, test cards, webhook setup | stripe.md |
The provider-agnostic frontend contract: 8-step flow, Sessions API body, enabler load (UMD vs ES), processor routes + X-Session-Id auth, full pitfall catalog | connector-contract.md |
| Verifying the round trip: querying the Payment, reading transactions, confirming state | verification.md |
| A standalone throwaway harness to prove a deployed connector before building the real storefront | test-harness.md |
| Monitoring a forked/custom connector: deployment logs (CLI + Merchant Center), structured logging, poison-message / dead-letter runbook | commercetools-connect → observability-operations.md |
stripe.md and extending the mapping table — the requirements, the backend, and the flow do not change.Checklist
Requirements
- PSP/connector + version; processor URL and enabler URL (or routed to deploy)
- Region + project; capture mode; saved-cards? partial refunds/captures? methods; origins + return URL; sync/async settlement
- Asked the open-ended "anything special/non-standard?" question; captured each special requirement as its own line
- Requirements block written and confirmed with the user; special requirements flagged into the Step 1.5 fit-check
Connector fit (decide before wiring/building)
- Checked live marketplace + supported-PSPs docs (not memory); named the connector + version
- PSP, methods, integration type, capabilities, region compared to the requirements; apparent gaps re-checked as config
- Ladder rung presented to the user and chosen by them: configure (1) · config-closes-gap (2) · fork/extend public connector (3) · build from template (4)
- For a real gap on a PSP that has a public connector, chose fork/extend over rebuild
Config (the deliverable)
- Only documented
connect.yamlenvelope fields used (no invented keys); file placed at the repository root, not a nested folder - standardConfiguration filled from the requirements, with a rationale per non-obvious key
- securedConfiguration keys listed (values supplied by the user, never invented)
- API-client scopes cover managing payments + reading sessions
-
MERCHANT_RETURN_URLabsolute w/ scheme;ALLOWED_ORIGINSincludes the storefront origin - Capture-method / saved-cards / multi-ops config match the chosen flow
Backend
- Token/cart/session creation server-side; browser gets only
sessionId+ processor/enabler URLs - Order created from cart after authorization (and webhook
Successfor async), idempotent viaorderNumber - Capture/refund/cancel routed through the processor's operation routes (not the Payment Intents API)
- Webhook reconciliation in place;
Pendingtransactions traced to webhook delivery
- Vitest (or equivalent) installed and
npm testruns before any implementation code is written - Each backend behavior written test-first: failing test confirmed red for the right reason → least code to pass → refactor
- No implementation function was written before its test — if you find yourself writing code without a red test, stop and write the test first
- Boundary mocked (PSP/processor/Sessions API behind a port); orchestration logic not mocked; unit suite runs with no deployment/secrets
- Happy path pinned per piece (session minted, Order created once marked
Ordered, capture/refund recorded) — the one a broad refactor silently breaks - Invariants pinned as tests: IDOR rejection, no-secret-leak, Order idempotent on
orderNumber, gate-on-Success(bothPendingandFailureblocked), capture/refund via processor (Payment Intents API untouched), webhook idempotent on redelivery -
npm testruns clean with zero secrets in the environment
Verification
- Test-card payment completed; commercetools Payment found with a
Successtransaction - (Production) Order created; a refund through the processor succeeds
- Full-flow integration test drives the real deployed connector with a test card, asserts the CT trace at each commit point, polls (not sleeps) for async settlement, and skips loudly when unconfigured
Stripe payment connector
The connector
- Connector: Stripe Payment for Checkout (
stripe-payment-connector). - Source:
stripe/stripe-commercetools-checkout-app— a monorepo withprocessor/(service) andenabler/(assets). - Verify the version you're integrating before trusting any specific key — Stripe iterates the connector. Config keys are read from a recent release; re-check the deployment's
connect.yamlif behavior differs.
Enabler bundle (browser)
- File:
connector-enabler.umd.js(andconnector-enabler.es.js). Load the UMD one via<script>— see contract pitfall 5. - UMD global:
window.Connector→window.Connector.Enabler. - Internally imports
@stripe/stripe-js, which is exactly why dynamic ESimport()is fragile here.
<script src="https://assets-….{region}.commercetools.app/connector-enabler.umd.js"></script>
<script>
const { Enabler } = window.Connector;
const enabler = new Enabler({ processorUrl, sessionId, locale: 'en-US', onComplete, onError });
const dropin = await (await enabler.createDropinBuilder('embedded')).build({ showPayButton: false });
dropin.mount('#dropin-container'); // then wait for `ready` before enabling Pay (pitfall 7)
</script>
Configuration keys (connect.yaml)
securedConfiguration and are never logged or returned. Don't hardcode any of them in the frontend — the publishable key and appearance reach the browser via the processor's GET /operations/config.Secured (secrets):
| Key | Purpose |
|---|---|
CTP_CLIENT_ID | commercetools API client id |
CTP_CLIENT_SECRET | commercetools API client secret |
STRIPE_SECRET_KEY | Stripe secret API key |
STRIPE_WEBHOOK_SIGNING_SECRET | verifies inbound Stripe webhooks |
connect.yaml for the complete list and current defaults):| Key | Notes |
|---|---|
CTP_PROJECT_KEY | project key |
CTP_AUTH_URL / CTP_API_URL / CTP_SESSION_URL | region hosts; defaults point at europe-west1.gcp — set to your region |
CTP_CHECKOUT_URL | required |
CTP_JWKS_URL / CTP_JWT_ISSUER | Merchant Center JWKS + issuer for session JWT validation |
STRIPE_PUBLISHABLE_KEY | Stripe publishable key (reaches the browser via the processor) |
STRIPE_WEBHOOK_ID | the Stripe webhook endpoint id the connector manages |
STRIPE_CAPTURE_METHOD | automatic (immediate capture) or manual (authorize, capture later). Default automatic. Drives the capture-mode requirement and when you create the Order. |
STRIPE_SAVED_PAYMENT_METHODS_CONFIG | JSON, e.g. {"payment_method_save":"enabled"}. Default {"payment_method_save":"disabled"}. Enable for saved-cards requirement — needs a customerId on the cart. |
STRIPE_PAYMENT_INTENT_SETUP_FUTURE_USAGE | "Setup future usage" for the PaymentIntent — pairs with saved payment methods. |
STRIPE_ENABLE_MULTI_OPERATIONS | true/false (default false). Enables multicapture + multirefund; also requires multicapture enabled in the Stripe account. Set for partial-refund/split-capture requirement. Don't enable speculatively — it changes transaction handling. |
STRIPE_COLLECT_BILLING_ADDRESS | auto | never | if_required (required; default auto). Whether the Payment Element collects billing address. |
STRIPE_API_VERSION | pinned Stripe API version. Do not hardcode this in documentation or generated code. Derive it from the installed stripe npm package rather than pinning a literal — the value changes with each major SDK release and a stale value causes a TypeScript type error. Note that stripe/esm/apiVersion.js is not in the package's exports map, so it can't be imported directly; see the Stripe API version section below for the supported ways to read it. |
STRIPE_LAYOUT / STRIPE_APPEARANCE_PAYMENT_ELEMENT / STRIPE_EXPRESS_ELEMENT_OPTIONS | Payment Element layout/appearance + express button options (JSON; cosmetic, safe to leave default) |
MERCHANT_RETURN_URL | required; must be an absolute URL with a scheme (contract pitfall 6) |
ALLOWED_ORIGINS | required; comma-separated list; must include every frontend origin that calls the processor (CORS) |
PAYMENT_INTERFACE | the paymentMethodInfo.paymentInterface written on the Payment; default checkout-stripe |
Session metadata for Stripe
metadata carries what this deployment expects — either the Checkout Application applicationKey, or processorUrl set to the connector's processor URL — and that the cart total is non-zero. See contract pitfalls 2 and 3.metadata.processorUrl, not applicationKey. The template's session-auth hook validates that the session's metadata.processorUrl matches the processor's own deployed URL. There is no Checkout Application involved. Use:{ "metadata": { "processorUrl": "https://service-….europe-west1.gcp.3.sandbox.commercetools.app" } }
applicationKey here gets you a 401 that looks like a session problem but is actually a metadata mismatch. The split is not certified-vs-custom — processorUrl works against any deployed connector; applicationKey just has preconditions a custom connector doesn't meet:- A Checkout Application must exist for the Project — created in the Merchant Center or via the Checkout Applications API (
POST /{projectKey}/applications), with the Payment Integrations API configuring its payment methods. - Creating one requires a Connector installed from the Connect marketplace — that is what the Application's Payment Integrations bind to (Connectors and Applications). This is why a purely custom connector can't back the
applicationKeypath.
processorUrl; reach for applicationKey only once a Checkout Application is confirmed to exist for the Project.API client scopes
400 invalid_scope (not a 403), which surfaces as a generic "Permissions exceeded" error at runtime.| Actor | Minimum required scopes |
|---|---|
| Storefront BFF (session creation, order creation) | manage_sessions:{projectKey}, manage_orders:{projectKey} |
| Processor (CT API client used for payments, cart reads, session validation) | manage_payments:{projectKey}, view_sessions:{projectKey}, manage_orders:{projectKey} |
Notes:
- Checkout splits session scopes:
manage_sessions:{projectKey}grants creating a session (the Storefront BFF needs this), whileview_sessions:{projectKey}grants reading one — the latter is the scope required for connectors to interact with Checkout and validate sessions, so the Processor needsview_sessions. See Checkout Scopes. manage_orderscovers reading carts (needed for cart version lookups andaddPayment) — do not requestview_ordersormanage_my_ordersunless the client was explicitly granted them.- The processor and storefront BFF can share a single API client in development, but should use separate clients in production to enforce least-privilege.
Webhook setup
STRIPE_WEBHOOK_ID) and verifies it with STRIPE_WEBHOOK_SIGNING_SECRET. If a payment authorizes in the UI but the commercetools Payment transaction never moves to Success, suspect the webhook: confirm the endpoint exists in the Stripe dashboard, points at the processor, and the signing secret matches.Test cards
| Card | Outcome |
|---|---|
4242 4242 4242 4242 | succeeds, no authentication |
4000 0025 0000 3155 | requires 3D Secure authentication |
4000 0000 0000 9995 | declined (insufficient funds) |
Any future expiry, any CVC, any postal code.
Building a custom Stripe connector (from the payment-integration template)
When building your own connector (ladder rung 4 — no public connector, or forking), two things differ from the public connector experience:
GET /payments (the enabler calls it for you — see contract pitfall 8). When you build from the template you own that route and should implement it as POST /payments. Don't let the "GET" note in connector-contract.md confuse you — it applies to the public connector; in your own processor you write the HTTP method.stripe.webhooks.constructEvent() requires the raw unparsed request body (a Buffer), not the JSON-parsed body. Fastify parses bodies by default. Use the fastify-raw-body npm package (note: the scoped @fastify/raw-body does not exist — it will 404 on install):npm install fastify-raw-body
import rawBody from 'fastify-raw-body';
await server.register(rawBody, {
field: 'rawBody',
global: false, // opt-in per route, not global
encoding: false, // keep as Buffer, not string
runFirst: true,
routes: ['/stripe/webhooks'],
});
(request as any).rawBody as the Buffer to pass to constructEvent.fastify-raw-bodyv5 replaces the JSON content-type parser globally. Despiteglobal: false, v5 replaces Fastify's default JSON content-type parser for ALL routes (not just webhook routes). Theglobalflag only controls thepreParsinghook, not the parser replacement. This means anyPOSTroute that receivesContent-Type: application/jsonwith an empty body ("") will be rejected by the patchedalmostDefaultJsonParser— evenPOST /payments. The fix: always sendbody: "{}"(a valid empty JSON object) from the enabler'sfetchcall toPOST /payments, never an empty string or no body at all.
stripe.elements() requires mode + real cart amount (deferred-intent pattern). Without mode, amount, and currency, the Stripe Payment Element mounts as a blank box with no error — a silent failure. The certified connector solves this with a GET /config-element/:paymentComponent endpoint (session-authenticated) that returns the real cart amount, currency, capture method, and layout. This endpoint is not in the payment-integration template by default — you must add it. The enabler fetches /operations/config and /config-element/payment in parallel, then calls:stripe.elements({
mode: 'payment',
amount: cartElement.cartInfo.amount, // centAmount from the CT cart
currency: cartElement.cartInfo.currency.toLowerCase(),
capture_method: cartElement.captureMethod, // 'automatic' | 'manual'
});
getCartIdFromContext()) and call ctCartService.getPaymentAmount({ cart }):// GET /config-element/:paymentComponent — session-authenticated
async initializeCartPayment() {
const ctCart = await this.ctCartService.getCart({ id: getCartIdFromContext() });
const amount = await this.ctCartService.getPaymentAmount({ cart: ctCart });
return {
cartInfo: { amount: amount.centAmount, currency: amount.currencyCode },
captureMethod: getConfig().stripeCaptureMethod,
collectBillingAddress: getConfig().stripeCollectBillingAddress,
layout: JSON.stringify({ type: 'tabs', defaultCollapsed: false }),
};
}
automatic_payment_methods, not payment_method_types. When Elements is initialized in deferred-intent / automatic mode (no explicit payment_method_types list — which is the correct pattern when using GET /config-element/payment), the PaymentIntent created in POST /payments must also use automatic_payment_methods: { enabled: true }. Using payment_method_types: ['card'] causes a Stripe 400: "Payment details were collected through Stripe Elements using automatic payment methods and cannot be confirmed through the API configured with payment_method_types." The payment-integration template scaffolds payment_method_types: ['card'] by default — remove it and replace:await stripe.paymentIntents.create({
amount: amountPlanned.centAmount,
currency: amountPlanned.currencyCode.toLowerCase(),
capture_method: cfg.stripeCaptureMethod,
automatic_payment_methods: { enabled: true }, // not payment_method_types: ['card']
metadata: { ... },
});
clientSecret inside submit(), not at mount time. With stripe.elements({ mode: 'payment', ... }), stripe.confirmPayment() needs a clientSecret — but the PaymentIntent doesn't exist yet when the element mounts. Create it server-side inside submit(), then confirm. (This is the Stripe instance of connector-contract.md pitfall 12.)// 1. Validate the form
const { error: submitError } = await elements.submit();
if (submitError) { /* handle */ return; }
// 2. Create the PaymentIntent server-side NOW (not at mount time)
const res = await fetch(`${processorUrl}/payments`, {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'X-Session-Id': sessionId },
body: '{}',
});
const { clientSecret } = await res.json();
// 3. Confirm with the clientSecret
const { error, paymentIntent } = await stripe.confirmPayment({
clientSecret, // ← required for deferred-intent
elements,
confirmParams: { return_url: merchantReturnUrl },
redirect: 'if_required',
});
stripe.confirmPayment({ elements, confirmParams }) without clientSecret throws IntegrationError: You must pass in a clientSecret. Calling POST /payments at mount time instead of submit time creates a PaymentIntent before the user has confirmed — abandoned intents accumulate in Stripe.ch_xxx), not a PaymentIntent id. For automatic capture flows the webhook writes interactionId: paymentIntent.id (pi_xxx) on the Success transaction — that's what POST /payments/:id/refund receives as stripeChargeId. But stripe.refunds.create operates on charges, not PaymentIntents. Passing a pi_xxx id returns 404 No such charge. Fix: retrieve the charge id from Stripe before refunding — the PaymentIntent's latest_charge field carries it:const pi = await stripe.paymentIntents.retrieve(paymentIntentId);
const stripeChargeId = pi.latest_charge as string; // ch_xxx
paymentIntent.latest_charge) as the interactionId for Charge-type transactions, so the CT Payment itself carries the refundable id. (This is the Stripe instance of connector-contract.md pitfall 16.)card_error and validation_error (from both elements.submit() and stripe.confirmPayment()) are user-recoverable — show them inline near the payment form and clear on the next change event so the user can correct and retry without the storefront intervening. Non-recoverable errors (invalid_request_error, api_error) bubble to onError. Pattern:// In mount():
this.errorEl = document.createElement('div');
this.errorEl.setAttribute('role', 'alert');
container.appendChild(this.errorEl);
paymentElement.on('change', () => { this.errorEl.textContent = ''; });
// In submit(), after elements.submit():
if (submitError?.type === 'validation_error') {
this.errorEl.textContent = submitError.message ?? 'Please complete your payment details.';
return;
}
// After stripe.confirmPayment():
if (confirmError?.type === 'card_error' || confirmError?.type === 'validation_error') {
this.errorEl.textContent = confirmError.message ?? 'Payment failed. Please check your card details.';
return;
}
// non-recoverable → onError(confirmError, { paymentReference })
stripe/esm/apiVersion is not exposed via the package's exports map — importing it directly fails at runtime (ERR_PACKAGE_PATH_NOT_EXPORTED) and TypeScript can't find it (no .d.ts in esm/). Use whichever approach fits your module system and build setup — any of these are fine:- Read from disk at startup (CJS processors):
fs.readFileSync('node_modules/stripe/esm/apiVersion.js')and regex-extract the value. Works without any build step. - Build-time codegen: a
prebuildscript that runsnode -e "..."and writes the version to a generatedsrc/generated/stripeApiVersion.tsfile that TypeScript can import normally. - Pin it explicitly and own the update: hardcode the string (e.g.
'2024-06-20'), add a comment like// update when upgrading stripe SDK, and enforce it in CI with a check that compares against the installed version. Honest and often the most pragmatic choice.
'' or omitted) — Stripe will use its own latest version server-side, which may differ from what the SDK expects and cause subtle type mismatches.npm test at publish and its examples use Jest, which is what the connector templates assume. Vitest can work, but its CLI aborts on any unknown option it's passed — so make each app's test script call Vitest with a fixed, explicit arg list rather than letting extra arguments reach it. The simplest way is a small wrapper script:// scripts/run-tests.mjs → "test": "node scripts/run-tests.mjs"
import { spawnSync } from 'node:child_process';
const r = spawnSync(process.execPath, ['node_modules/vitest/vitest.mjs', 'run', '--coverage'], { stdio: 'inherit' });
process.exit(r.status ?? 1); // forward exit code so real failures still fail the build
test script — since tests are mandatory and reviewed at publish, an assets/enabler app with no test script won't pass validation. Give it the same wrapper plus at least one real test. And put coverage config in vitest.config.ts, not CLI flags, so the wrapper stays the single source of truth.Quick reference
- Bundle:
connector-enabler.umd.js, globalwindow.Connector. - Auth to processor:
X-Session-Id(contract pitfall 9). - Capture mode:
STRIPE_CAPTURE_METHOD(automatic|manual). - Secrets:
STRIPE_SECRET_KEY,STRIPE_WEBHOOK_SIGNING_SECRET,CTP_CLIENT_SECRET,CTP_CLIENT_ID. - 401 from processor → session
metadata/ cart-total check. - Raw body for webhooks:
fastify-raw-body(not@fastify/raw-body— that package doesn't exist). - API version: do not hardcode and do not import
stripe/esm/apiVersiondirectly (not in exports map). Options:fs.readFileSyncat startup, build-time codegen, or an explicit pinned string with a CI check. See the "Stripe API version" section above. - Payment Element blank box (no error) →
stripe.elements()missingmode/amount/currency— addGET /config-element/paymentto the processor; fetch it in parallel with/operations/configbefore initializing Elements. POST /payments500 with empty body →fastify-raw-bodyv5 global JSON parser replacement. Sendbody: "{}"(not""or no body) from the enabler's fetch.- PaymentIntent 400 "cannot be confirmed … configured with payment_method_types" → template default
payment_method_types: ['card']conflicts with automatic Elements mode. Replace withautomatic_payment_methods: { enabled: true }. - Enabler error handling:
card_error/validation_error→ inline message + clear onchange;invalid_request_error/api_error→onError. - Vitest test script failing at publish though it passes locally → Vitest aborts on unknown CLI options; route
testthrough a wrapper that calls Vitest with a fixed arg list. Prefer Jest (templates assume it). Every app (incl. enabler) needs atestscript. See the "Prefer Jest for connector apps" note above. - Image security analysis fails but SAST/SCA pass → base-image OS CVE, pin
engines.node(e.g.20.x) in every app'spackage.json. Dependency-CVE fixes are upgrades, never downgrades. See deploy-custom-connector.md.
Test harness
Shape
ready, submit.Security note: a real app does steps 1–3 (token, cart, session) server-side so client credentials andmanage_sessionsnever reach the browser. A local harness may do them client-side for speed, but say so and never deploy it.
Config the harness needs
CT_AUTH_URL, CT_API_URL # region hosts
CT_SESSION_HOST # https://session.{region}.commercetools.com
CT_PROJECT_KEY
CT_CLIENT_ID, CT_CLIENT_SECRET # client with manage_sessions (+ cart/payment read for verify)
PROCESSOR_URL # deployed connector processor URL
ENABLER_URL # deployed connector enabler URL (serves connector-enabler.umd.js)
CHECKOUT_APPLICATION_KEY # or PROCESSOR_URL again, per what the connector's session metadata expects
connector-env) without a .env extension, Vite's loadEnv() won't pick it up — it only reads files whose names start with .env. Use fs.readFileSync + a custom parser in vite.config.js instead:import fs from 'fs';
function parseEnvFile(filePath) {
return Object.fromEntries(
fs.readFileSync(filePath, 'utf8')
.split('\n')
.filter(l => l && !l.startsWith('#') && l.includes('='))
.map(l => { const i = l.indexOf('='); return [l.slice(0,i).trim(), l.slice(i+1).trim()]; })
);
}
const env = parseEnvFile('../connector-env');
export default { define: { __PROCESSOR_URL__: JSON.stringify(env.PROCESSOR_URL), /* … */ } };
The flow (pseudocode)
// 1) token (client_credentials)
const token = await oauth(CT_AUTH_URL, CT_CLIENT_ID, CT_CLIENT_SECRET, 'manage_sessions:'+CT_PROJECT_KEY);
// 2) non-zero cart (ExternalAmount avoids needing a tax category — see contract pitfall 3)
const cart = await post(`${CT_API_URL}/${CT_PROJECT_KEY}/carts`, token, {
currency: 'EUR', taxMode: 'ExternalAmount',
customLineItems: [{ name:{en:'Test item'}, slug:'test-item', quantity:1,
money:{currencyCode:'EUR',centAmount:1999},
externalTaxRate:{name:'test',amount:0,country:'DE'} }],
});
// 3) session — cartRef + processor-matching metadata (contract pitfalls 1, 2)
const session = await post(`${CT_SESSION_HOST}/${CT_PROJECT_KEY}/sessions`, token, {
cart: { cartRef: { id: cart.id } },
metadata: { applicationKey: CHECKOUT_APPLICATION_KEY }, // or { processorUrl: PROCESSOR_URL }
});
// 4) warm the processor (contract pitfall 10)
await fetch(`${PROCESSOR_URL}/operations/status`).catch(()=>{});
// 5) load enabler UMD (contract pitfall 5) — inject a <script> and await its load
await loadScript(`${ENABLER_URL}/connector-enabler.umd.js`);
const { Enabler } = window.Connector; // global is provider-specific
// 6) construct + build
const enabler = new Enabler({
processorUrl: PROCESSOR_URL, sessionId: session.id, locale: 'en-US',
onComplete: (r) => setStatus('paid: ' + JSON.stringify(r)),
onError: (e) => setStatus('error: ' + (e?.message ?? e?.code)),
});
const dropin = await (await enabler.createDropinBuilder('embedded')).build({ showPayButton: false });
// 7) mount + wait for ready (contract pitfall 7)
dropin.mount('#dropin-container');
container.addEventListener('ready', () => enablePayButton(), { once: true });
setTimeout(enablePayButton, 5000); // fallback if no ready event
// 8) on Pay click
payButton.onclick = () => dropin.submit();
Automating the card-entry step
ready) automate cleanly and prove the integration risk that actually matters. Step 8 is different: card fields live inside a cross-origin iframe owned by the payment provider, and the automation approach decides whether this works at all.| Approach | Against a cross-origin card iframe |
|---|---|
| Page-context script injection (JS evaluated inside the page) | Cannot work — same-origin policy blocks reaching into the iframe's DOM, regardless of tooling |
| Synthetic keyboard events from the parent frame (Tab-to-next-field) | Unreliable — key events don't reliably cross the frame boundary, and some widgets grow mid-fill so a repeated click lands somewhere else |
| A browser driver controlling the browser externally (Playwright, Puppeteer) | Works — frame-scoped locators address the iframe's contents directly |
With a driver, scope to the frame and select by visible label rather than placeholder text:
const frame = page.frameLocator('iframe[title="Secure payment input frame"]'); // title is provider-specific
await frame.getByLabel('Card number').fill('4242 4242 4242 4242');
await frame.getByLabel('Expiration date').fill('12/34');
await frame.getByLabel('CVC').fill('123');
await page.getByRole('button', { name: /pay/i }).click(); // the Pay button is in the parent page
If no iframe-aware driver is available, don't burn time on synthetic events — pick one:
- Human completes the submit. Automate through step 7, then have a person enter the test card. Legitimate; this harness is disposable by design.
- Bypass the widget entirely. Create and confirm a payment directly against the provider's API, create the matching commercetools Payment (
interfaceId= the provider's transaction id, plus aCharge/Successtransaction for the amount),addPaymentit to the cart, then call your place-order endpoint. This skips the widget's own payment creation but does exercise the real order-confirmation path — including BFF scope gaps, which is exactly where a live charge otherwise strands a customer (backend-integration.md).
After it works
- Verify the Payment (→ verification.md).
- Move steps 1–3 server-side for the real integration, and add Order creation + post-purchase operations (→ backend-integration.md); the browser only ever gets the
sessionId, processor URL, and enabler URL. - Delete the harness or scrub its secrets.
Checklist
- harness reads processor/enabler URLs and CT creds from config, not hardcoded
- non-zero cart; session with
cartRef+ correctmetadata - enabler loaded from UMD bundle;
ready-gated Pay button - one test-card payment completed and verified as a CT Payment
- card entry either driven by an iframe-aware browser driver, or deliberately left to a human / an API-level payment — not attempted via synthetic key events
- harness not deployed; secrets removed afterward
Verifying the round trip
paymentMethodInfo.paymentInterface is whatever the connector's PAYMENT_INTERFACE is set to (Stripe default checkout-stripe).What success looks like
dropin.submit():-
The enabler's
onCompletefires (or the browser is sent toMERCHANT_RETURN_URL). -
The processor has created a Payment whose
paymentMethodInfo.paymentInterfacematches the connector (e.g.stripe) and added a transaction:Charge/ stateSuccessfor immediate capture (STRIPE_CAPTURE_METHOD=automatic), orAuthorization/ stateSuccessfor authorize-now/capture-later (manual).
The interface value comes fromPAYMENT_INTERFACE(Stripe defaultcheckout-stripe). -
The Payment is linked to the cart (
cart.paymentInfo.payments).
Finding the Payment
paymentInfo:# Get the cart; paymentInfo.payments holds the Payment references the processor linked
curl -s "{api}/{projectKey}/carts/{cartId}" -H "Authorization: Bearer {token}"
Then fetch each referenced Payment and inspect its transactions:
curl -s "{api}/{projectKey}/payments/{paymentId}" -H "Authorization: Bearer {token}"
Look for, in the Payment:
paymentMethodInfo.paymentInterface= the connector's interfacetransactions[]containing aChargeorAuthorizationwithstate: "Success"interfaceIdset to the PSP's payment/intent reference- optionally
interfaceInteractions[]holding the raw PSP payload (audit trail)
interfaceId if you captured the PSP reference. Reading scopes needed: view_payments (and view_orders to read the cart).If the Payment is missing or stuck
| Symptom | Likely cause | Where |
|---|---|---|
| No Payment at all | submit() never reached the processor; or processor 401/502 | contract pitfalls 2, 4, 7 |
Payment exists, transaction stuck Pending | async PSP webhook not delivered/verified | provider reference → webhook setup; backend-integration.md → webhook reconciliation |
Payment with Failure transaction | declined card / PSP rejection | check PSP dashboard + the test card used |
| Duplicate Payments | frontend also creating Payments (wrong path) | the processor owns the Payment — don't create it yourself |
Checklist
-
onCompletefired or return URL was reached - Cart
paymentInfo.paymentsreferences at least one Payment - That Payment has a
SuccessCharge/Authorizationtransaction -
paymentInterfacematches the connector;interfaceIdis set - No duplicate Payments (a sign the frontend wrongly created one)
Build or fork a PIM connector
If forking, change only the delta (a new resource type, a transform, a direction) and keep the working sync engine, keying, and dependency handling — don't rebuild.
Decision 1 — Which ingestion API?
| Import API | HTTP (Products) API | |
|---|---|---|
| Shape | Asynchronous, bulk; submit then poll | Synchronous, transactional; immediate result |
| Best for | Initial catalog load, periodic full refresh, large scheduled batches | Real-time incremental updates, single-product fixes |
| Superpower | Automatic reference resolution — submit products/categories/types in any order within 48 h; up to 20 resources/request | Instant validation and errors; full update-action control |
| Watch out | Reference resolution ≠ data validity (SKU uniqueness etc. still checked by the commerce API); poll operations to a terminal state | You resolve references and ordering yourself; rate limits under high volume |
| Reference | Import API overview, best practices | Products API, product drafts / import endpoints |
Decision 2 — Which Connect application shape?
- Event-driven / near-real-time → a
serviceas inbound webhook: the PIM (or its middleware) calls your endpoint when a product changes; you transform and upsert. This is the commercetools-connect skill's inbound webhook mode — not an API Extension. Contract: 5-min service timeout (not the 2 s extension limit); you authenticate the caller and make the write idempotent. → service-applications.md, security.md. - Scheduled / bulk → a
job: poll the PIM for deltas (or do a full pull), transform, and submit via the Import API. Contract: 30-min request timeout, no built-in concurrency guard (you own overlap locking), restart-safe checkpointing so a re-run resumes cleanly. → job-applications.md. - Both is common and recommended: a
servicewebhook for live changes plus ajobfor nightly full reconciliation that heals anything the webhook missed. A single connector declares both applications inconnect.yaml. - Bi-directional only: if some commercetools attributes must flow back to the PIM, add an
eventapp subscribing to product Messages (at-least-once, no ordering, idempotent). Keep it scoped to CT-owned attributes only, and add self-change filtering so your own inbound writes don't loop back out. → event-applications.md.
Webhooks: which way they point
- Inbound (the one that matters): PIM → connector. The event-driven path is the PIM calling your
serviceendpoint when a product changes. Two setup obligations come with it: (a) register your endpoint with the PIM's event system (e.g. Akeneo's event subscriptions) so it will call you — do this in an idempotentpostDeploylifecycle script or via the PIM's own config; (b) verify the PIM's signature on every inbound event. Note the scheme is often HMAC (a shared secret insecuredConfiguration), not the JWT the security.md describes for commercetools-issued destinations — check the PIM's current docs for the exact header and algorithm, don't assume JWT. - Outbound is the commercetools API, not a webhook. The connector's outbound traffic is HTTP/Import API calls to commercetools — normal authenticated REST, not webhooks.
- commercetools Subscriptions are queue deliveries, not webhooks you call — relevant only on the bi-directional
eventpath above.
Full vs incremental
- Full sends the whole catalog — right for the initial load and periodic complete refreshes; slower and leaves the catalog partially updated mid-run. Best on the Import API.
- Incremental sends only what changed since the last run — faster and time-critical-friendly, but the PIM must support change tracking; if it doesn't, the connector needs its own change-detection (e.g. a stored hash/updatedAt per product) rather than reprocessing everything. (docs)
Idempotency (non-negotiable)
key from data mapping Principle 7 — create if absent, update if present. This is what makes the connector safe under the realities of both shapes: a webhook redelivered at-least-once, two job runs overlapping, a full re-import over existing data. Never blind-create (duplicates) and never full-overwrite another source's attributes (clobbering). On the HTTP API, upsert = get-by-key-then-update-actions or create; on the Import API, submitting the same key updates in place. State the idempotency strategy in one sentence before writing the handler — if you can't, you're not ready.Delete / unpublish handling
Then follow the build-side contracts
- Inbound webhook authentication + least-privilege scopes (
manage_products,manage_categories,manage_product_types, and Import API scopes as needed — notmanage_project) → security.md - Idempotent
postDeploy/preUndeploylifecycle scripts (register the webhook/subscription, create Product Types & AttributeGroups as-code) → lifecycle-scripts.md - Test-first: pure mapping unit tests, then a bounded sandbox-only live sync run with a pre-flight item count and catalog-size gate → testing.md; router-level auth-rejection matrix, duplicate-delivery/idempotency, malformed-payload handling →
commercetools-connect→ testing.md - Structured logs with a correlation key (the PIM product id), health endpoint, poison-message/replay runbook → observability-operations.md
connect.yamlat the repo root, documented envelope keys only; scaffold/validate with the Connect CLI → connect-cli.md; deploy/stage/publish → deployment-installation.md
Checklist
- Ingestion API chosen per volume/cadence (Import API bulk, HTTP API real-time; often both)
- Application shape matches the cadence (
servicewebhook,job, or both;eventonly for bi-directional write-back) - Full and incremental strategy decided; incremental has real change detection (or a documented fallback)
- Every write is an upsert by stable key; idempotency strategy stated in one sentence
- Delete/unpublish semantics defined (not implicit); reconciliation detects disappearances
- Handed the type-agnostic contracts (auth, scopes, lifecycle, tests, observability, deploy) back to the commercetools-connect skill and met its production-readiness gate
Is a public PIM connector enough?
There are two kinds of connector:
- Public connectors — listed in the Connect marketplace and actually deployable as a commercetools Connect application (they have a connector repo /
connect.yamland install via the Connect CLI). Some are built by commercetools, most PIM connectors by partners. If one covers the use case, this is almost always the right choice: install + configure, don't build. - Organization (custom/private) connectors — deployed for your organization only, either a fork of an open-source connector you extend, or one built from scratch using the connect skill's
service/jobpatterns. Both are commercetools-connect tasks → build-connector.md.
Don't hardcode "what's supported" — check it live
- Run the skill's
docs-searchstep and/or query the commercetools Knowledge MCP for "PIM connector product data integration". - Browse the live Connect marketplace — Product Information Management category for listings and versions: marketplace PIM integrations.
- For a partner listing, its own marketplace page / repo / docs is the source of truth for what it maps and how it's configured (e.g. the Akeneo listing).
- Confirm it's a Connect connector, not just an integration — the category mixes both. See the verification step below before counting a listing as a rung-1/2 option.
Beware the direction trap. The same PIM category also lists adjacent entries that are not PIM ingestion — search/recommendations (e.g. GroupBy), 3D/AR (e.g. Threekit), generic iPaaS middleware — and some push commercetools → external (syndication, the product-export template). A PIM ingestion connector must sync PIM → commercetools. Confirm the direction before counting a listing as a fit.
Not every marketplace listing is a Connect connector — verify it
connect.yaml; the Connect CLI registry is authoritative over the listing — see deployment-installation.md), then ask the user what to do.If it's a good match but is not a Connect connector
connect.yaml / CLI / lifecycle / deploy patterns don't apply). Point to the vendor's own onboarding, and offer the in-skill alternative: build a Connect connector for this PIM, or fork an open-source one if it exists (rungs 3–4 → build-connector.md).Only a listing that passes this check counts as a rung-1/2 (configure) option below.
Present the options first: install or modify (before considering build)
- Install it as-is → deploy the public connector and close any gaps with configuration / attribute mapping (ladder rungs 1–2). This is the default recommendation whenever a listed connector matches the PIM. →
deployment-installation.md. - Modify it (fork) → the connector's a fit but has a genuine gap config can't close (e.g. bi-directional write-back, a missing resource type). Fork the open-source connector, add only the delta, deploy as an Organization connector (rung 3). → build-connector.md.
The fit check
Compare the requirements gathered in Step 1 against what a candidate public connector actually does. Check each dimension:
| Dimension | Question | If not covered → which rung |
|---|---|---|
| PIM system | Is the user's PIM available as a public connector? | No connector for this PIM → rung 4 (build). |
| Direction | One-way (PIM → CT) vs bi-directional — does the connector match? | Most public PIM connectors are one-way; bi-directional need → fork (rung 3) or build (rung 4). |
| Sync scope | Does it sync what's needed — Product Types, products/variants, categories, attributes, media, prices? | Missing resource type → re-check as config; if genuinely absent → fork (rung 3). |
| Attribute mapping | Can it map the user's PIM attributes onto their Product Types, including localization and channels? | Almost always config/attribute-mapping, not code → rung 2. See data-mapping.md. |
| Cadence | Event-driven, scheduled delta, full re-sync — does it offer what's needed? | Missing cadence → fork (rung 3). |
| Special requirements | Reference entities, measurement conversion, Product Selections/Tailoring per store, variant/family quirks, approval workflow | Judge each: config (rung 2), small fork (rung 3), or build (rung 4). |
The decision ladder
- Is the listing a deployable Connect connector at all? → if not (a partner/SaaS integration), it's outside this skill: surface it with the not-a-Connect-solution warning above and offer the build/fork rungs instead. Only Connect-deployable listings reach rung 1. → Not every marketplace listing is a Connect connector.
- Public connector covers everything → install + configure. Don't build. The common, recommended case. Deploying it:
deployment-installation.md. - Right PIM, gap looks like a capability → first prove it's not config / attribute mapping. Field mapping, locale/channel selection, category mapping, and which attributes sync are configuration on most PIM connectors → back to rung 1. See data-mapping.md and the connector's own config docs (looked up live).
- Right PIM, genuine gap config can't close → fork/extend the public connector. Add only the delta (a new resource type, a transform, bi-directional write-back) and deploy as an Organization connector — you keep the working sync engine, keying, and dependency handling. → build-connector.md, a commercetools-connect task.
- No public connector for the PIM at all → build using the connect skill's
service(inbound webhook) and/orjobpatterns, ingesting via the Import API or HTTP API. The from-scratch path, justified only when there's nothing to fork. → build-connector.md.
Only rungs 3–4 leave this sub-area (hand off to build/fork); the flow resumes once the connector is deployed. Record the decision, the rung, and the connector version checked in the requirements block — so the rest of the work is grounded in a real, confirmed connector, not an assumed one.
Checklist
- Checked the live marketplace PIM category (not memory); cited the connector + version
- Verified each candidate is a deployable Connect connector, not a partner/SaaS integration listing (Connect affordance / repo / CLI registry — not the marketing page)
- A good-match listing that is not a Connect connector was surfaced with the not-a-Connect-solution warning and the build/fork alternative, not treated as rung 1
- Listed the available public PIM connectors to the user (name, vendor, direction, what it syncs) before any build discussion
- For a matching connector, offered install-as-is vs modify/fork explicitly; build proposed only when no listed connector matches the PIM
- Confirmed the candidate syncs PIM → commercetools (direction trap avoided)
- PIM, direction, sync scope, cadence, and special requirements each compared to the requirements
- Apparent capability gaps re-checked as config / attribute mapping (rung 2) before considering any build
- When a public connector exists but has a real gap, chose fork/extend (rung 3) over build-from-scratch
- Decision + rung + connector version recorded: configure (1), config/mapping (2), fork (3 → build-connector.md), or build (4 → build-connector.md)
From PIM model to commercetools product model
Principle 1 — Map only commerce-relevant data
Not every PIM attribute belongs in commercetools. Transfer only what search, display, pricing, or fulfillment needs; leave the rest in the PIM (it stays the source of truth and can be fetched on demand if ever needed). Every attribute you sync is one more thing to keep consistent — a smaller, sharper catalog is cheaper to run and faster to query.
Principle 2 — Do NOT map Product Types 1:1 with PIM families
- Design a small set of flexible Product Types driven by how products are sold and searched, not by the PIM's taxonomy.
- Give each Product Type a stable set of attributes; absorb PIM structural variety through attribute values, not new Product Types.
- A PIM with hundreds of families usually maps to a handful of commercetools Product Types. If you find yourself minting a Product Type per family, stop — that coupling is the anti-pattern.
Principle 3 — Prioritize search-critical attributes; consolidate the rest
- Search/filter/display-critical (brand, color, size, material, key specs) → map each to its own typed Product Type attribute. Type it precisely —
enum/lenumfor controlled vocabularies (so faceting works),number+ a unit for measures,booleanfor flags,ltext/textfor copy. Precise types are what make query predicates and search facets work. - Supplementary (long-tail specs shown but never filtered) → consolidate into a single JSON/
textattribute rather than exploding into dozens of rarely-used fields. This keeps the Product Type lean and the catalog queryable.
enum/set of enum (lenum if the labels are localized); a PIM metric/measurement attribute becomes a number plus a unit (convert to one target unit at map time — don't ship mixed units). Key each enum option on the PIM's stable option code, not its localized label — labels change per translation and would silently break faceting.Principle 4 — Localization
{ "en-US": "...", "de-DE": "..." }). Map each PIM locale to a commercetools locale explicitly — PIM locale codes don't always match (en_US vs en-US), and a mismatch silently drops translations. Decide which locales are in scope (Step 1) and only sync those. For attributes that vary by locale in the PIM but shouldn't in commercetools (e.g. a unit system), resolve to one value at map time.Principle 5 — Categories are a keyed tree, resolved by reference
key; a Product references its categories by key, and a child Category references its parent by key. When importing, you don't need categories to exist first — the Import API resolves references asynchronously (it holds an operation up to 48 h waiting for the referenced Category/Product Type to arrive, then retries). So you can submit products and categories in any order within that window — but the keys must match exactly. Derive category keys deterministically from a stable PIM identifier, never from a localized name (names change and aren't unique).Principle 6 — Media, price, and inventory each have their own path
- Media / assets. Map PIM image/asset URLs onto Product Variant images (or Assets for richer metadata). If the PIM only holds asset references into a DAM, sync the resolved public URLs. Large binary sets are better handled in the bulk/job path than on the hot webhook path.
- Price and inventory — keep them SEPARATE from content (docs). They change far more often and are more time-critical than descriptions/images. Implement them as their own event-based integrations even when they originate in the same system, so a slow nightly catalog sync never blocks a price or stock update. Prices map to embedded Prices or Standalone Prices; inventory to InventoryEntry by SKU (variant
availabilityupdates asynchronously after the InventoryEntry lands).
Principle 7 — Every resource gets a stable key (idempotency backbone)
sku), Price, Category, Product Type — a unique key derived from a stable PIM identifier (the PIM's product id / variant id, not a name or a position). Then every write is an upsert by key: create if absent, update if present. This makes re-delivery of a webhook, an overlapping job, and a full re-import all no-ops rather than duplicate-creators. A resource without a stable key cannot be safely re-synced — fix the key before writing any sync code.Principle 8 — Source of truth and read-only enforcement
Principle 9 — Scopes/channels and reference/related data (the concepts that catch people)
Two recurring PIM concepts don't have a 1:1 commercetools counterpart and need an explicit decision — whichever PIM you're on (the names differ; the shape doesn't):
- Scope / channel / context. Many PIMs scope attribute values by a channel or context (e.g. Akeneo channels, inriver segments/channels), so one attribute holds different values per scope. Choose which scope's values feed commercetools — syncing the wrong one produces correct-looking but wrong storefront data, and ignoring scopes entirely mixes contexts. If different scopes must feed different storefronts, that's a Product Selection / Product Tailoring decision (which Products/values each Store sees), not just attribute mapping.
- Reference / related entities. PIMs model related objects as first-class links (e.g. Akeneo reference entities, related products, cross-sells). Map these to commercetools attributes or product references — but this is often the gap a public connector doesn't cover, so confirm the chosen connector maps them; if not, it's a common fork trigger (→ build-connector.md).
Worked example (sketch)
tshirt, hoodie, jeans, each with dozens of family-specific attributes, 3 locales (en-US, de-DE, fr-FR), category tree by department.- Product Types: one
apparelProduct Type (not three) with attributesbrand(enum),color(lenum, localized labels),size(enum),material(set of enum),care-instructions(ltext), andspec-sheet(text holding consolidated JSON for the long-tail). Family differences live in attribute values, not new types. - Variants: one Product per style, one Variant per color/size combination;
key=pim-<productId>, variantkey/sku=pim-<variantId>/ the real SKU. - Categories:
key=dept-<pimCategoryId>, parent by key; products reference categories by key and let the Import API resolve. - Localization: PIM
en_US/de_DE/fr_FR→ LocalizedStringen-US/de-DE/fr-FR; other locales dropped per scope. - Price/inventory: separate event integrations keyed by SKU; not part of the content sync.
Checklist
- Only commerce-relevant attributes mapped; the rest left in the PIM
- A small set of flexible Product Types (not 1:1 with PIM families); structural variety absorbed as attribute values
- Search-critical attributes typed precisely (enum/lenum/number+unit/boolean); supplementary consolidated into one JSON/text attribute
- PIM locales explicitly mapped to commercetools locales; out-of-scope locales dropped
- Categories keyed from stable PIM ids (not names); products & parents reference by key; Import API resolves references
- Media mapped; price and inventory kept as separate integrations, keyed by SKU
- Every resource has a stable
keyfrom a PIM identifier → every write is an upsert (safe to re-run) - Source of truth decided per attribute; externally-owned attributes made read-only via an AttributeGroup; multi-source writes scoped to owned attributes only
- PIM scope/channel chosen (multi-scope → multi-Store routed through Product Selections/Tailoring); reference/related-entity mapping confirmed against the connector or flagged as a fork trigger
PIM connector — product data sync (build or integrate)
connect.yaml, lifecycle scripts, testing, deploy) are the commercetools-connect skill; this sub-area owns the PIM-specific job end to end — from "is there a connector already?" through configuring one, forking it, or building one, to the data model mapping and sync architecture that decide whether the catalog stays correct.service inbound webhook and/or a job), so this whole sub-area is server-side.Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<PIM terms from the request, e.g. 'product data integration import API product type attribute mapping categories'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root, where scripts/docs-search.mjs lives.) The two most load-bearing docs for this sub-area are the Integrate product data tutorial and the Import API overview — read them. You may additionally use the commercetools Knowledge MCP for deeper follow-up.Step 1 — Extract requirements (before any config or code)
- Which PIM system, and is a connector deployed? Name and version. If a public connector is in play, get its marketplace listing and version.
- Source of truth per attribute. Which system owns which field? A PIM typically owns enriched content (names, descriptions, images, specs); an ERP may own SKU/price/inventory. Multiple sources add sequencing and conflict rules.
- Direction — one-way or bi-directional? One-way (PIM → commercetools) is the simple, recommended default. Bi-directional means defining which attributes sync back and how conflicts resolve — flag it as expensive.
- Is product data editable in the Merchant Center? If product managers edit in commercetools, decide which attributes are read-only from the PIM (enforce with an AttributeGroup so externally-owned fields can't be hand-edited).
- Cadence — full vs incremental, event-driven vs bulk? Initial load and periodic refresh are full/bulk; time-critical fixes are incremental. Real-time correctness → event-driven; large nightly volumes → bulk/scheduled. → drives
jobvsservice-webhook (Step 4). - Volume and locales. Catalog size (drives Import API vs HTTP API) and which locales/currencies/channels are in scope (drives localization mapping).
- What to sync, and what NOT to. Product Types, products/variants, categories, media, prices, inventory — and explicitly what to leave behind. Not every PIM attribute belongs in commercetools; map only commerce-relevant data.
- Price and inventory ownership. Even if they come from the same system, treat them as separate integrations (they update far more often than content) — confirm where they originate.
- Anything special? (always ask — open-ended) Reference entities / related products, measurement-unit conversion, variant/family modeling quirks, publish/staging rules, channel- or store-specific catalogs (Product Selections / Product Tailoring), approval workflows, GDPR/PII in product data. Capture each as its own requirement line; don't force it into a slot above.
Step 1.5 — List the available connectors, then offer install or modify (before building)
- Public connector covers it (and is Connect-deployable) → install + configure. Don't build. (Deploying a public connector:
deployment-installation.mdin the commercetools-connect skill.) - Right PIM, gap looks like a capability → prove it isn't config / attribute mapping first. Most "missing" behavior on a supported PIM (which attributes map where, locale/channel selection, category mapping) is configuration, not missing code → back to rung 1.
- Right PIM, genuine gap config can't close → fork/extend the public connector and deploy it as an Organization connector — you keep the working sync engine and change only the delta. → build-connector.md, hand off to commercetools-connect.
- No public connector for the PIM at all → build one using the connect skill's
service(inbound webhook) and/orjobpatterns, ingesting via the Import API or HTTP API. → build-connector.md.
Step 2 — If configuring a public connector: derive its config
connect.yaml configuration and its attribute-mapping setup (most PIM connectors externalize the field mapping as config, not code). The connect.yaml envelope rules — documented keys only (no invented fields), file at the repo root — are the same as any connector; see the payment sub-area's config-from-requirements pattern for the envelope and deployment-installation.md. For the provider-specific config keys and concept names, read the chosen connector's own current docs/repo (looked up live) — don't rely on a hardcoded per-vendor table, which goes stale; the vendor-neutral mapping method is data-mapping.md.Step 3 — Data mapping (the heart — applies to configure and build)
Step 4 — If building/forking: sync architecture
service, scheduled job, or both) and the ingestion API from volume (Import API for bulk/async, HTTP API for real-time), then make every write idempotent. This is build-connector.md; it hands the type-agnostic build contracts (service/job semantics, security, testing, deploy) back to the commercetools-connect skill.Step 5 — Verify the sync
rejected/validationFailed operations — reference resolution succeeding is not the same as the data being valid.References
| Need | Reference |
|---|---|
| Is a public connector enough?: live marketplace check, named PIM connectors, fit dimensions, the configure/fork/build ladder | connector-selection.md |
| Data mapping (the substance): Product Type strategy, attribute mapping, localization, categories, media, price/inventory separation, keys & idempotency, source-of-truth | data-mapping.md |
Build or fork a connector: Import API vs HTTP API, service webhook vs job (vs both), full vs incremental, idempotent upsert, dependency resolution, delete handling | build-connector.md |
| Testing & safely running a sync: pure mapping unit tests, then a bounded sandbox-only live run with pre-flight item count, catalog-size gate, and idempotency re-run | testing.md |
| Deploy/install a public or custom connector; regions; certification | commercetools-connect → deployment-installation.md |
| Inbound webhook auth, least-privilege scopes, secured config | commercetools-connect → security.md |
| Scheduled/on-demand job: schedule, 30-min timeout, overlap locking, checkpointing | commercetools-connect → job-applications.md |
| Structured logs, health, poison-message/replay runbook | commercetools-connect → observability-operations.md |
Checklist
Requirements
- PIM system + version; whether a public connector is deployed (and its version)
- Source of truth per attribute; direction (one-way default, bi-directional flagged as expensive)
- Editable-in-MC decision; externally-owned attributes marked read-only (AttributeGroup)
- Cadence (full/incremental, event/bulk); volume; locales/currencies/channels in scope
- What to sync and what to leave behind; price & inventory treated as separate integrations
- Asked the open-ended "anything special?" question; each special requirement captured as its own line
- Requirements block written and confirmed with the user
Connector fit (decide before wiring/building)
- Checked the live marketplace (not memory); named the connector + version
- Verified the candidate is a deployable Connect connector, not a partner/SaaS integration listing; a good-match non-connector surfaced with the not-a-Connect-solution warning
- PIM, direction, and sync scope compared to the requirements; apparent gaps re-checked as config/attribute-mapping
- Ladder rung presented to the user and chosen by them: configure (1) · config/mapping-closes-gap (2) · fork/extend (3) · build (4)
Data mapping (the deliverable that decides correctness)
- Product Type strategy chosen (flexible attributes, not 1:1 with PIM families); search-critical attributes mapped, supplementary consolidated
- Localization, categories, and media mapped; price/inventory kept separate
- Every resource keyed for idempotent upsert
Sync / verify
- Application shape matches the cadence; ingestion API matches the volume; writes idempotent
- Mapping unit-tested; live sync run against a sandbox only (never production), pre-flight item count run and large catalogs gated → testing.md
- A real change flowed end to end; a re-run left the catalog unchanged; bulk imports polled to terminal state with rejects inspected
Testing a PIM sync
Layer 1 — Mapping unit tests (no credentials, every commit)
- Locale mapping (
en_US→en-US); an out-of-scope locale is dropped, not passed through. - Enum keyed on the option code, not a localized label.
- Measurement units normalized to one unit.
- Keys derived from stable PIM ids (Principle 7) — the same input yields the same key every time (this is what makes the sync idempotent).
- Category/product references emitted by key.
- Attributes not in scope are omitted (Principle 1), not sent as
null.
service app) is tested at the router level — auth-rejection matrix, malformed-payload handling, duplicate-delivery idempotency — using the commercetools-connect skill's testing.md. This file adds only the PIM-specific live-run layer.Testing the inbound webhook locally (no live PIM needed)
service endpoint (see build-connector.md); you can exercise that whole path locally without the PIM reaching you:- Replay a captured event. Save a real PIM event payload as a fixture and POST it at the router with
supertest(unit) or at a locally-running connector (commercetools connect application dev/ the generated local server) withcurl. This covers signature verification, mapping, and idempotency — mock the commercetools side withmsw, or point at a sandbox for a real write. - Signature check with the sample secret. Compute the PIM's HMAC over the fixture body using a test secret and send it in the expected header — assert a valid signature is accepted and a tampered/missing one is rejected (401/403). No PIM involved.
- Duplicate delivery. POST the same event twice and assert the second is a no-op (upsert by key).
- Only for true end-to-end — having a hosted SaaS PIM actually deliver to your machine — expose the local endpoint through a public tunnel so the PIM can reach a public URL, and register that URL as the PIM's webhook target. For everyday testing, replay is faster and needs no tunnel or PIM account.
The outbound side is not a webhook — it's commercetools API calls, covered by the guarded sync run below (sandbox only).
Layer 2 — A guarded live sync run (SANDBOX ONLY)
Guard 1 — Sandbox credentials only, from .env, never production
- Load credentials from a
.envthat is gitignored and holds sandbox values only. Never put production credentials in it, never commit it, never echo it to logs. - Require an explicit opt-in marker (e.g.
CT_ENV=sandbox) and refuse to run without it — an accidental run should fail closed, not touch a project. Optionally pin an allowlist of permitted sandbox project keys and refuse any key not on it. - Use a least-privilege API client for the sandbox (
manage_products,manage_categories,manage_product_types— notmanage_project), so even a misfire is bounded → security.md.
// support/sandbox.ts — fail closed if this doesn't look like an explicit sandbox
import 'dotenv/config';
export function loadSandboxConfig() {
const { CT_ENV, CTP_PROJECT_KEY, CTP_CLIENT_ID, CTP_CLIENT_SECRET, PIM_BASE_URL } = process.env;
if (!CTP_PROJECT_KEY || !CTP_CLIENT_ID || !CTP_CLIENT_SECRET) return null; // → skip loudly (unconfigured)
if (CT_ENV !== 'sandbox') {
throw new Error('Refusing to run a live sync: set CT_ENV=sandbox to confirm a throwaway project. Never use production credentials.');
}
const allow = (process.env.SANDBOX_PROJECT_ALLOWLIST ?? '').split(',').filter(Boolean);
if (allow.length && !allow.includes(CTP_PROJECT_KEY)) {
throw new Error(`Project '${CTP_PROJECT_KEY}' is not in SANDBOX_PROJECT_ALLOWLIST — aborting.`);
}
return { projectKey: CTP_PROJECT_KEY, clientId: CTP_CLIENT_ID, clientSecret: CTP_CLIENT_SECRET, pimBaseUrl: PIM_BASE_URL };
}
Guard 2 — Pre-flight count, and gate on large catalogs
const SYNC_WARN_AT = 500; // warn + require confirmation above this
const SAMPLE_SIZE = 25; // default bounded first run
export async function preflight(cfg, { confirmedLarge = false, limit = SAMPLE_SIZE } = {}) {
const total = await countSourceItems(cfg); // PIM total, or the incremental delta count
console.warn(`[pim-sync] pre-flight: ${total} source items would be in scope.`);
if (total > SYNC_WARN_AT && !confirmedLarge) {
throw new Error(
`Large catalog: ${total} items exceeds the ${SYNC_WARN_AT} warn threshold. ` +
`Re-run with an explicit limit (e.g. --limit ${SAMPLE_SIZE}) for a sample, ` +
`or pass confirmedLarge to sync the full set deliberately.`);
}
return Math.min(total, limit ?? total); // the count this run will actually process
}
Guidance to give the user with the count:
- Default to a bounded sample (a handful to a few dozen products) for the first run — enough to prove the mapping and wiring, cheap to inspect and clean up.
- Only sync the full catalog deliberately, and prefer the Import API for it (async, bulk, up to 20 resources/request, reference resolution) over per-item HTTP calls — see build-connector.md. Mind Import API best practices for batching and rate limits.
- If the source can't give an exact total cheaply, at least bound the run with a hard
limit— never let a test run unbounded.
Guard 3 — Assert the trace, then re-run to prove idempotency
import { describe, it, expect } from 'vitest';
const cfg = loadSandboxConfig();
const itLive = cfg ? it : it.skip; // skip LOUDLY when unconfigured — never a silent pass
async function until(fn, ok, { tries = 30, gapMs = 2000 } = {}) {
for (let i = 0; i < tries; i++) { const v = await fn(); if (ok(v)) return v; await new Promise(r => setTimeout(r, gapMs)); }
throw new Error('import did not reach a terminal state in time');
}
describe('PIM sync (sandbox, bounded)', () => {
itLive('syncs a sample and is idempotent on re-run', async () => {
const count = await preflight(cfg, { limit: 25 }); // Guard 2 gates large catalogs here
const sample = await fetchSourceSample(cfg, count);
const first = await runSync(cfg, sample);
// bulk path: wait for a real terminal state — no operations still in flight.
// include waitForMasterVariant, or a product awaiting its master variant lets the poll return early.
const summary = await until(() => getImportSummary(cfg, first.containerKey),
s => s.unresolved === 0 && s.processing === 0 && s.waitForMasterVariant === 0);
expect(summary.rejected, JSON.stringify(summary.errors)).toBe(0);
expect(summary.validationFailed).toBe(0);
// assert the CT trace for a known sample product
const p = await getProductByKey(cfg, sample[0].expectedKey);
expect(p).toBeTruthy();
expect(p.masterData.staged.name['en-US']).toBe(sample[0].expectedName);
expect(p.masterData.staged.categories.length).toBeGreaterThan(0);
// re-run the same sample → no new products, versions unchanged where content is unchanged
const beforeCount = await countProducts(cfg);
await runSync(cfg, sample);
expect(await countProducts(cfg)).toBe(beforeCount); // upsert, not duplicate-create
}, 180_000); // generous: bulk import + polling
});
Clean up
afterAll. Never leave a shared sandbox full of half-synced fixtures.Checklist
Gate: Layer 1 (mapping unit tests) green before any live run.
- Mapping unit tests cover locale mapping, enum-by-code, unit normalization, stable keys, by-key references, out-of-scope omission — no credentials needed
- Webhook
servicetested at the router level (auth matrix, malformed payload, duplicate delivery) → testing.md - Sandbox only: credentials loaded from a gitignored
.env; run fails closed without an explicitCT_ENV=sandboxmarker; no production credentials ever used - Least-privilege sandbox API client (not
manage_project) - Pre-flight count runs first; warns above a threshold and refuses a large full sync without explicit confirmation; every run is bounded by a
limit - First run is a small bounded sample; full-catalog runs are deliberate and use the Import API
- Live run asserts the CT trace (product by key, mapped localized name, category assignment); bulk imports polled to terminal state with
rejected/validationFailedinspected - Idempotency re-run proves a repeat sync is a no-op (no duplicate products)
- Skips loudly when unconfigured; runs in a dedicated job, not every commit
- Test fixtures cleaned up (disposable sandbox or delete-by-key in teardown)
Requirements → promotion connector config
connect.yaml values. For a from-scratch build these are the keys you define. For an existing public connector, read its own connect.yaml for the authoritative key list — don't work from a copy, here or anywhere else (public-connectors.md); use the mapping below to decide what each of its keys should be set to.The requirement → config map
| Requirement (Step 1) | Config / decision | Why |
|---|---|---|
| Which engine + credentials | securedConfiguration: engine API key / application key | Secrets never in standardConfiguration, never hardcoded |
| Region + project | standardConfiguration: CTP_REGION; scopes via inheritAs | Host + client provisioning are region/project specific |
| Engine owns promotions | Discount mechanism = setDirectDiscounts; native Discount Codes become inert | Direct Discounts and Discount Codes are mutually exclusive (below) |
| Coupon/voucher codes | Cart custom type + field for the code, plus a field for the validation result | Native Discount Codes are unavailable once Direct Discounts are in play |
| Evaluation + redemption | Deploy both apps (evaluator + syncer); evaluation-only = just the evaluator | Redemption is a separate engine endpoint and a separate Connect app |
| Loyalty points / balances | Mirror-target setting (Customer Custom Field) or "engine is sole source of record" | Points must not silently diverge between systems |
| Rollback on cancel/return | Syncer subscribes to OrderStateChanged / return messages + order-state → action mapping | A cancelled order must not consume a coupon or keep points |
| Fail-open vs fail-closed | Outbound timeout + error behavior in the evaluator; documented in the README | Decides whether a down engine breaks carts or just drops discounts |
| Cart/customer attributes the engine needs | Attribute-mapping keys in standardConfiguration | The engine's rules can only match on what you forward |
| Discount line items need a tax category | standardConfiguration: CTP_TAX_CATEGORY_ID (only if using custom line items) | A custom line item requires a tax category; not needed for Direct Discounts |
How discounts land on the cart
The single most consequential choice, and the one that leaks into the storefront. Three mechanisms:
setDirectDiscounts — recommended
setDirectDiscounts action carrying the engine's computed discounts. Each entry is a DirectDiscountDraft with a required value (relative, absolute, fixed, or giftLineItem) and an optional target (line items, custom line items, shipping cost, total price, multi-buy, or pattern) — the same value/target vocabulary as Cart Discounts, so an engine's percentage, fixed-amount, free-shipping, and free-gift effects all have a native landing spot. Fetch the current shape with this skill's openApi-schemata.mjs --resource-name api-Cart-write rather than trusting a copied field list.- Always active and valid — no validity window, no
isActiveto manage. - Default
StackingModeStacking, and nosortOrder— they apply in array order, so you control precedence by ordering the array. An engine that returns effects in priority order maps directly; one that doesn't means you sort before writing. - The action replaces the cart's
directDiscountsarray — the evaluator always writes the complete current set, never a delta. - They transfer to the Order automatically when the Order is created from the Cart. Changing them afterwards is not a plain cart update — it needs the Order Edits
setDirectDiscountsaction. - They also work on Quotes (valid for that quote only) — relevant for B2B negotiated pricing.
The exclusivity rule. Direct Discounts and Discount Codes are mutually exclusive: if a Direct Discount is applied to a Cart or Order, any matching Cart Discounts in the Project are ignored. Practical consequence to state to the user before writing a line of code: once the engine owns the cart, your native Discount Codes and Cart Discounts stop affecting it. "Engine promotions plus our existing native promo codes on the same cart" is not a supported design — pick one owner (see Step 1 question 3). The docs state the ignoring behavior; they do not define an API-level rejection for mixing, so do not rely on the platform to error out and warn you — enforce the ownership decision in your own code and configuration.
Negative custom line items — legacy/compat
Engine-managed native Discount Codes — narrow
job) creates/mirrors Cart Discounts and Discount Codes in commercetools, and the platform evaluates them natively. Keeps native semantics, no extension on the cart hot path, and no exclusivity problem — but adds sync lag, and Cart Discount and Discount Code limits apply, which is exactly what "unique codes at scale" requirements break against. Viable for a modest, slow-changing campaign set; not for per-customer unique codes.Record the choice and why.
The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / configuration; inheritAs), and place the file at the repository root — a nested connect.yaml silently fails to deploy.Native client provisioning (prefer this)
CTP_CLIENT_ID/SECRET:inheritAs:
apiClient:
scopes:
- manage_extensions # evaluator postDeploy registers the Cart API Extension
- manage_subscriptions # syncer postDeploy registers the OrderCreated Subscription
- view_orders # syncer re-fetches the Order to build the redemption
- manage_types # only if postDeploy creates the coupon-code custom type
configuration:
standardConfiguration:
- key: PROMOTION_ENGINE_BASE_URL
description: Engine API base URL (sandbox or live)
- key: COUPON_CODE_FIELD
description: Cart custom field holding the shopper-entered coupon code
securedConfiguration:
- key: PROMOTION_ENGINE_API_KEY
description: Engine API key, used by both apps
Note:view_extensions/view_subscriptionsare not valid standalone scopes —manage_extensions/manage_subscriptionscover read + write. Declaring the non-existent view scopes fails client creation.
view_customers only if the evaluator forwards customer attributes, and manage_customers only if you mirror points back onto the Customer. Both are easy to over-grant — justify each.CTP_CLIENT_ID/SECRET/SCOPE as secured config; migrating to inheritAs.apiClient.scopes is the more native, lower-maintenance form and is worth doing on a fork or a fresh build (public-connectors.md).Per-app config
deployAs:
- name: promotion-evaluator
applicationType: service
endpoint: /promotionEvaluator
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the API Extension + custom type
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: FAIL_MODE
description: "'open' (drop discounts on engine error) or 'closed' (fail the cart update)"
- key: ENGINE_TIMEOUT_MS
description: Outbound engine timeout, must stay under the extension budget
- name: redemption-syncer
applicationType: event
endpoint: /redemptionSyncer
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the Subscription
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub or SNS"
- key: ROLLBACK_ORDER_STATES
description: Order states that trigger a redemption rollback (e.g. Cancelled)
<url>/promotionEvaluator), and the Express router must be mounted at the same base path — a mismatch 404s all platform traffic (project-structure.md).Worked example (in-house engine, from-scratch build)
europe-west1.gcp.Derived config:
inheritAs:
apiClient:
scopes: [manage_extensions, manage_subscriptions, view_orders, manage_types]
configuration:
standardConfiguration:
- key: PROMOTION_ENGINE_BASE_URL
description: PromoSvc base URL
- key: COUPON_CODE_FIELD
description: "Cart custom field with the entered code (promoSvcCouponCode)"
- key: CART_HASH_FIELD
description: "Cart custom field holding the promo-relevant cart hash (loop guard + call reduction)"
securedConfiguration:
- key: PROMOTION_ENGINE_API_KEY
description: PromoSvc API key (same key both apps)
deployAs:
- name: promotion-evaluator
applicationType: service
endpoint: /promotionEvaluator
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
configuration:
standardConfiguration:
- key: FAIL_MODE
description: "'open' — a PromoSvc outage drops discounts, never blocks the cart"
- key: ENGINE_TIMEOUT_MS
description: "800 — well under the extension budget"
- name: redemption-syncer
applicationType: event
endpoint: /redemptionSyncer
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub"
- key: ROLLBACK_ORDER_STATES
description: "Cancelled"
setDirectDiscounts so PromoSvc's amounts are authoritative and no custom line items pollute the cart; native Discount Codes will no longer affect these carts — coupon codes go through promoSvcCouponCode instead; both apps because redemption and point-awarding are real requirements, not just checkout display; manage_types only because postDeploy creates the two cart custom fields; fail-open with an 800 ms timeout so a PromoSvc incident degrades to "no promotions" rather than "no checkout".postDeploy, get-then-update so a redeploy doesn't blow away existing fields (lifecycle-scripts.md).connect.yaml for the real key names and apply this mapping to their values; the fixes worth making while you're forking are in public-connectors.md.Native, use, customise, or build?
Rung 0 first — is this native?
Before any marketplace lookup, test the requirement against the native surface:
| Native primitive | Covers |
|---|---|
| Product Discounts | Percentage/absolute off a price before the cart, predicate-scoped |
| Cart Discounts | Spend thresholds, tiered discounts, item/shipping/total targets, buy-X-get-Y (multiBuy*), free gifts (giftLineItem), pattern targets, per-Store scoping |
| Discount Codes | Promo/coupon codes with max-applications and per-customer limits |
| Discount Groups | "Only the best of these N discounts applies", plus deactivating a whole campaign in one request |
Project discountCombinationMode | Stacking vs BestDeal across Product and Cart Discounts |
| Direct Discounts | A discount computed elsewhere and applied to one cart/order/quote |
Check live data — don't answer from memory
Listings and their capabilities change. Before deciding among rungs 1/3/4:
- Search the Connect marketplace (
marketplace.commercetools.com/connectors) and the Promotions & Loyalty listings, plus the docs via thedocs-searchscript or the Knowledge MCP. - Distinguish an installable Connect connector from a partner integration you self-host — apply the commercetools-connect skill's Marketplace listings are not all Connect connectors rule; don't re-derive it here. It bites especially hard in this category: promotions & loyalty is crowded with partner-operated SaaS, so a listing is weak evidence that anything is deployable via Connect. Treating one as installable is a planning error, not a detail.
- Compare the requirements engine-by-capability (evaluation, coupon codes, loyalty, rollback on cancel, POS, regions).
- Name the connector and version you checked, and record it in the requirements block.
The promotion landscape (verify, but this is the shape)
| Engine | Marketplace presence | Source available? | Default rung |
|---|---|---|---|
| Talon.One | ✅ Listed, with a Connect connector | ✅ MIT (composable-com/ct-connect-talonone) — maintained by Orium, not Talon.One | 1 (use) — or 3 (customise), since the source is public. Check you have the right repo: Talon.One's own commercetools repos are a separate, PoC-grade accelerator (public-connectors.md) |
| Voucherify | ✅ Listed (plus a separate Gift Card listing) | ✅ MIT (voucherifyio/commerce-tools-integration) — but a standalone Node service, not a Connect app | 3 (customise/port) — see the caveat below |
| Dovetech Campaigns, Eagle Eye, NULogic, Annex Cloud, Currency Alliance, SheerID | ✅ Listed | Vendor-private — check the listing | 1 (use) if the listing covers it; otherwise partner conversation |
| In-house / unsupported engine | ❌ Nothing to install | — | 4 (build) |
Two consequences worth stating to the user early, because they change the effort estimate:
- "Just use the Talon.One connector" is a real answer — but name the repo, because there are three. The MIT-licensed Connect connector is Orium's; Talon.One's own commercetools repos are an accelerator their docs label proof-of-concept, not production. If the requirements fit the Connect connector, this is rung 1 and cheap.
- Voucherify's public integration is not a Connect connector. It is an MIT-licensed standalone Node service that registers its own API Extensions and is documented for Heroku/self-hosted deployment. Using it as Connect means porting it into a Connect app (
connect.yaml, lifecycle scripts, endpoint↔route wiring) — that is rung 3 work, not a marketplace install. Its logic (coupon validation, cart-custom-field code storage, redemption on payment) is excellent reference material either way.
The ladder (stop at the first rung that fits)
Rung 1 — Use a public connector as-is
deployment create --connector-key, or Merchant Center install) is the commercetools-connect skill's deployment-installation.md. Hand it the config you derive in config-from-requirements.md.Rung 2 — A gap that config can close
Rung 3 — Customise/fork a public connector
CTP_CLIENT_ID/SECRET/SCOPE instead of inheritAs.apiClient.scopes, non-secrets sitting in securedConfiguration, missing loop guards. The concrete list is in public-connectors.md. Hand off to commercetools-connect for the fork's build/stage/publish lifecycle.Rung 4 — Build for your own promotion service
No connector for the engine — an in-house promotion service, or a vendor with no listing → build it.
What you actually write on rung 4:
- The evaluator: cart → your service's evaluate request; response effects →
setDirectDiscounts(+ custom fields for coupon validity/messaging); the loop guard; fail-open. → promotion-contract.md - The redemption-syncer: order → redeem/commit call, idempotent on order id; optional rollback on cancel/return.
- Lifecycle scripts that idempotently register the Extension, the Subscription, and the custom type holding the coupon code. → lifecycle-scripts.md
- Config + scopes (config-from-requirements.md).
Recording the decision
Promotions: none · rung 0 · checked requirements 2026-07 — "20% off orders over €100 plus a SUMMER promo code" is Cart Discount + Discount Code; no engine, no connector.
Promotions: Talon.One · rung 3 (customise) · checked marketplace 2026-07, public MIT connectorcomposable-com/ct-connect-talonone· fits except loyalty-point rollback on returns, which config can't express → fork and add the return handler; also migrating its hand-supplied CTP credentials toinheritAs.apiClient.scopes.
Promotions: in-house "PromoSvc" · rung 4 (build) · checked marketplace 2026-07 — no listing, and no promotion template exists → scaffoldingservice+eventwith the Connect CLI.
Promotion connector — integrate an external promotion engine
- promotion-evaluator (a
serviceregistered as a cart API Extension) — the evaluate half. On cart changes, commercetools calls it synchronously; it sends the cart to the engine, gets back the discount effects, and writes them onto the cart (normally viasetDirectDiscounts). Nothing is consumed — this is a quote. - redemption-syncer (an
eventdriven by an OrderCreated Subscription) — the redeem half. After the order is placed, it asynchronously tells the engine the promotion was actually used: redeem the coupon, close the session, award loyalty points — and, for a full integration, roll that back when the order is cancelled or returned.
Evaluate vs. redeem is the mistake to internalize first. "Why is the coupon still showing as unused / why are no loyalty points awarded / why is there nothing in the engine's dashboard?" is almost always because only the evaluator is wired. Evaluating a cart consumes nothing; only the redeem call does. They are different engine endpoints and different Connect apps here.
- commercetools already has a promotion engine. Product Discounts, Cart Discounts, Discount Codes, Discount Groups, multi-buy/pattern targets, and gift line items cover a large share of real requirements natively — with no connector to run, secure, and pay per call. A connector is the right answer when the requirement genuinely exceeds that surface; it is the wrong answer when it merely restates it. This is rung 0 of the ladder below and you must rule it out explicitly, not silently.
- An external engine and native Discount Codes cannot both own a cart. Direct Discounts and Discount Codes are mutually exclusive: once a Direct Discount is on a Cart or Order, matching project Cart Discounts are ignored. So "the engine does promotions and we keep our native discount codes" is not a coherent design on the same cart — see promotion-contract.md.
Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<promotion terms from the user's request, e.g. 'cart discount direct discounts discount codes external promotion engine API extension'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or pricing-and-discounts-overview for deeper follow-up.Step 1 — Extract requirements (before any config or code)
Promotion behavior is downstream of marketing intent, and the wrong default silently gives money away or blocks checkout. Extract these first; each maps to a config key in Step 2 or a rung in Step 1.5. Ask the user (don't assume):
- Which engine, and why? Talon.One, Voucherify, Dovetech, Eagle Eye, NULogic, an in-house service, or undecided. Do they already have an account + API credentials? If undecided, Step 1.5 may end at rung 0 (native).
- What can't commercetools do natively? Name the specific requirement — bulk/unique code generation at scale, referral or loyalty programs, geofencing, per-customer targeting from a CDP, cross-channel (POS + web) budgets, real-time campaign experimentation. If the answer is "percentage off, spend thresholds, buy-X-get-Y, a promo code" — that is native (Cart Discounts + Discount Codes) and you should say so.
- Who owns promotions after this — the engine or commercetools? Because Direct Discounts and Discount Codes are mutually exclusive, a split ownership model on the same cart doesn't work. Get an explicit answer: all engine, or native with the engine only for a carved-out case.
- Coupon/voucher codes? Does the shopper type a code? Then decide where the code lives (a cart custom field, since native Discount Codes are off the table) and how an invalid code is reported back to the storefront.
- Loyalty points, wallets, or gift cards? Points/balances are the engine's system of record — decide what (if anything) is mirrored into commercetools. Gift cards are a payment method, not a discount → that's the gift card sub-area, not this one. This split matters in practice because the same vendor often does both (Voucherify has a separate Gift Card listing): a voucher that reduces the cart total is a promotion, while stored value that pays for the order is a Payment.
- Order lifecycle beyond creation? Should a cancellation or return roll back the redemption and claw back points? → drives whether the syncer subscribes to
OrderStateChanged/ return messages, not justOrderCreated. - Region and project? e.g.
europe-west1.gcp, projectmy-project— host and config are region-specific. - Fail-open or fail-closed? If the engine is slow or down, does the cart proceed without promotions (fail-open, the usual answer for promotions) or does the cart update fail (fail-closed)? See Step 3.
- Anything special or non-standard? (always ask — open-ended) B2B/quotes, multi-store or multi-currency budgets, marketplace/multi-seller, POS + web shared budgets, subscription/recurring orders, existing native discounts to migrate. Capture each as its own requirement line; don't force it into a slot above.
Step 1.5 — Native, use, customise, or build? (decide before wiring or building)
docs-search script / Knowledge MCP), and name the connector + version you checked. Details, the live-check procedure, and the per-engine landscape are in connector-selection.md.- Native commercetools discounts are enough → build no connector. Model it with Cart Discounts, Discount Codes, and Discount Groups. The docs' own common discount use cases table maps most standard promotions to native primitives. Say this plainly and stop.
- A public connector for the engine covers everything → install + configure it (Step 2). Don't build. Installation (CLI auth, scopes,
deployment create) is the commercetools-connect skill's deployment-installation.md; it is not theconnectorstagedflow. - Public connector, gap looks like a capability → prove it isn't config first. Most "missing" behaviors (which effects map to which action, attribute/custom-field mapping, which order states redeem vs roll back) are
connect.yamlvalues or Merchant Center settings → back to rung 1. See config-from-requirements.md. - Public connector, genuine gap config can't close → fork/customise it. The Talon.One Connect connector and the Voucherify integration are both MIT-licensed and public, so this is a real option — add only the delta and deploy as an Organization connector. Don't rebuild a working codebase. Provider specifics, and the known issues worth fixing while you're in there, are in public-connectors.md.
- No connector for the engine at all (an in-house or unsupported promotion service) → build it. Note the difference from payment and tax: there is no promotion-integration template. You scaffold a plain
service+eventconnector with the Connect CLI and implement the contract yourself — connect-cli.md for the scaffold, promotion-contract.md for what to build.
Step 2 — Derive the config from the requirements
connect.yaml values, with a one-line why for each. The mapping, the connect.yaml envelope, and a worked example are in config-from-requirements.md. The decisions that live here:- How discounts land on the cart:
setDirectDiscounts(recommended) vs. negative custom line items vs. engine-managed native codes. This is the promotion equivalent of choosing a tax mode, and it is the one choice that leaks into the storefront. → config-from-requirements.md. - Where the coupon code lives — a cart custom field plus the custom type that
postDeploycreates idempotently. - API-client scopes — declare them in
inheritAs.apiClient.scopesso Connect provisions a least-privilege client (manage_extensions,manage_subscriptions,view_orders, plusmanage_typesifpostDeploycreates the custom type), rather than hand-supplyingCTP_CLIENT_ID/SECRET. - Secured vs standard config — the engine API key is
securedConfiguration; region, behavioral toggles, and attribute mappings arestandardConfiguration.
Step 3 — The extension trigger, call reduction, and the loop guard (reference)
setDirectDiscounts write is itself a cart update, so a naive evaluator re-triggers itself. Three things to get right — full detail in promotion-contract.md:- Condition the trigger so it only fires on carts worth evaluating (
Activecart state, non-empty). - Short-circuit on an unchanged promo-relevant cart hash stored in a custom field — this is both the cost control and the loop guard.
- Decide fail-open vs fail-closed and mean it. For promotions the usual answer is fail-open: a down promo engine should return no discounts, not break every cart update. That is the opposite of a compliance-driven tax integration — state the choice in the connector README.
Step 4 — Build/verify the two apps (the main body of work), test-first
200/201 (never 202), not looping on its own writes, redeeming exactly once under redelivery, rolling back on cancel — are invisible at the call site and miserable to reproduce by hand. Each is one cheap assertion. Write the test first.- Evaluator (API Extension) — map cart → engine session/evaluate request; call the engine; map effects →
setDirectDiscounts(+ custom fields for coupon validity and campaign messaging); respond200fast; fail-open on engine error. - Redemption-syncer (Subscription) — on
OrderCreated, re-fetch the Order by id, redeem/close/award in the engine, idempotently on a stable key (the order id). For a full integration, also handle cancel/return → rollback.
Step 5 — Verify the round trip
References
| Need | Reference |
|---|---|
| Native, use, customise, or build?: the rung-0 native check, the live-marketplace procedure, the per-engine landscape (Talon.One, Voucherify, Dovetech, Eagle Eye, NULogic, in-house) | connector-selection.md |
Requirements → config mapping: how discounts land on the cart, coupon-code custom field, scopes, the connect.yaml envelope; worked example | config-from-requirements.md |
The two-app contract: the evaluator (effect→action mapping, setDirectDiscounts, the self-trigger loop guard, 200-not-202, fail-open) and the redemption-syncer (redeem/rollback lifecycle, idempotency); full pitfall catalog | promotion-contract.md |
| Which public integration to use, and what to fix when forking: Talon.One's Connect connector is a third party's while the vendor's own repo is a PoC accelerator; Voucherify's is a port, not an install. Points at each repo and the vendor docs for config/API facts instead of copying them | public-connectors.md |
| Verify the round trip: discount on the cart, redemption in the engine; the double-redemption, abandoned-cart, and cart-merge traps | verification.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
sortOrder semantics, Discount Groups, and Direct-Discounts-block-Discount-Codes as domain concepts live in commercetools-commerce-patterns; this sub-area covers the connector that drives them. Gift cards and stored value as a payment method are the gift card sub-area.Checklist
Requirements
- Engine chosen (or deliberately deferred) + account/credentials; region + project
- The specific requirement native discounts cannot meet is named — not just restated as "promotions"
- Promotion ownership decided: all engine or native + carved-out case (never split on one cart)
- Coupon-code entry path decided (custom field + invalid-code feedback), or explicitly out of scope
- Loyalty/points mirroring decided; gift cards routed to the gift card sub-area if applicable
- Rollback-on-cancel/return decided; fail-open vs fail-closed decided
- Asked the open-ended "anything special?" question; each special requirement its own line
- Requirements block written and confirmed; specials fed into the Step 1.5 fit-check
Path (decide before wiring/building)
- Rung 0 ruled out explicitly — native Cart Discounts/Discount Codes/Discount Groups can't do it, and you said why
- Checked live marketplace + docs (not memory); named the connector + version
- User asked to choose between use-as-is (1), customise/fork (3), and build-new (4)
- Rung recorded with rationale; for a real gap on a supported engine, chose fork over rebuild
- If rung 4: understood there is no promotion template — plain
service+eventscaffold
Config (the deliverable)
- Discount application mechanism chosen (
setDirectDiscountsunless a reason not to) with rationale - Discount-Codes-are-now-inert consequence stated to the user
- Only documented
connect.yamlenvelope fields; file at the repo root -
inheritAs.apiClient.scopesleast-privilege (+manage_typesonly ifpostDeploycreates types) - Engine credentials in
securedConfiguration; region/toggles/mappings instandardConfiguration
The two apps (build test-first — do not write a function body before its red test)
- Evaluator returns
200/201(never202); fail-open on engine error/timeout - Loop guard: promo-relevant cart hash in a custom field; own writes don't re-trigger evaluation
- Extension trigger conditioned to reduce engine calls (cart
Active, non-empty) - Syncer re-fetches the Order by id; redeems idempotently on a stable key; rolls back on cancel/return (if in scope)
- Boundary mocked; suite runs with no deployment/secrets
Verification
- Discount visible on the cart (
directDiscounts+discountOnTotalPrice/discountedPricePerQuantity) after a cart update - Order placed → syncer acks → redemption/points confirmed via the engine API
- Redelivering the same message does not redeem twice
- Cart-merge-on-login and anonymous→known session identity verified
The two-app promotion contract
App 1 — the evaluator (cart API Extension)
What triggers it
cart resource, actions: [Create, Update], registered by the app's postDeploy. Promotion engines bill and rate-limit per call and sit on the cart hot path, so condition the trigger to fire only on carts worth evaluating:{
"resourceTypeId": "cart",
"actions": ["Create", "Update"],
"condition": "cartState = \"Active\" and lineItems is not empty"
}
400 ExtensionPredicateEvaluationFailed and breaks the cart operation, so a wrong condition is worse than none.What it must return
setDirectDiscounts— the engine's discounts. The action replaces the whole array, so always emit the complete current set, never a delta. Order the array deliberately: Direct Discounts have nosortOrderand apply in array order.setCustomField(coupon result) — whether the entered code was accepted, and why not if rejected. This is how the storefront shows "code invalid" (see below).setCustomField(cart hash) — the promo-relevant cart fingerprint, for call reduction.setCustomField(campaign messaging) — optional: "spend €10 more for free shipping" style engine copy the storefront renders.
{ value, target }, the same vocabulary as Cart Discounts:| Engine effect | value | target |
|---|---|---|
| % off eligible items | relative (permyriad) | lineItems (+ predicate) |
| Fixed amount off items | absolute | lineItems |
| Fixed price for items ("3 for €5") | fixed | lineItems / pattern |
| % or amount off the cart total | relative / absolute | totalPrice |
| Free / discounted shipping | relative (10000 permyriad) / absolute | shipping |
| Free gift item | giftLineItem | (none — the draft carries the product/variant) |
| Buy X get Y at a discount | relative only | multiBuyLineItems / multiBuyCustomLineItems |
openApi-schemata.mjs --resource-name api-Cart-write (CartSetDirectDiscountsAction, DirectDiscountDraft, CartDiscountValueDraft, CartDiscountTarget) rather than trusting a copied list. Two mapping details that bite:relativevalues are permyriad (1/10000), not percent — 10% is1000. An engine returning10becomes a 0.1% discount if you forward it raw.- The target discriminator is
shipping, notshippingCost— the type name (CartDiscountShippingCostTarget) and the discriminator value differ. Getting this wrong is a rejected action, not a silent miscalculation. - Multi-buy targets take a percentage only.
multiBuyLineItems/multiBuyCustomLineItemsaccept arelativevalue; an engine effect expressing "buy 3, pay €5" as a fixed amount must map topattern(which accepts an amount, a fixed price, or a percentage) or tolineItems, not to a multi-buy target. - A
giftLineItemdiscount needs a product that exists in commercetools. An engine effect granting a free item the catalog doesn't have cannot be expressed; decide up front whether unmatched gift effects are dropped (with a log) or fail.
Rejecting a coupon code without breaking the cart
400 with errors — don't: that fails the entire cart update, so the shopper's real change (adding an item, setting an address) is lost too, and the storefront gets a generic platform error.200, write the validation outcome to a custom field, and let the storefront read it. A valid code produces discounts and a success flag; an invalid one produces no discounts and a rejection reason. Reserve 400 { errors: [...] } for genuinely invalid requests, not for business-rule outcomes. The public Voucherify integration stores codes and their status in cart custom fields for exactly this reason.The response-status trap
200 or 201. Any other status — including 202 — is treated as a failure to respond properly and fails the triggering cart operation (docs). A successful no-op is 200 with {} or { actions: [] }. Same trap as the tax sub-area; pin it with a test.Latency and fail mode
- Keep the outbound engine call on a tight timeout under the extension budget, aborting yourself rather than letting the platform time out.
- Default to fail-open for promotions. Return
200with no discount actions when the engine errors or times out: a promotion outage then degrades to "no promotions today" instead of "nobody can add to cart". This is the opposite default from a compliance-driven tax integration, where an untaxed cart may be unacceptable. If the business genuinely requires fail-closed (e.g. engine-managed contract pricing that must never be missing), say so explicitly in the README. - Fail-open has a consequence to state: a cart can persist without discounts the customer expected. Make the next successful evaluation self-healing — because the evaluator always writes the complete
directDiscountsarray, the following cart update repairs it automatically.
Call reduction (the biggest cost lever)
{ actions: [] } immediately. The certified tax connectors use the same hashCart pattern (tax-contract.md).Re-trigger and chaining
Two things people conflate:
- Your response does not re-invoke you. The extension is called before the result is persisted and its returned actions are applied within that same operation — writing
setDirectDiscountsin the response is not a fresh cart update and does not recurse. - Out-of-band cart writes by your own connector do. If another app (an
eventhandler, ajob) updates the cart via the API, that is a cart update and will trigger the evaluator. Filter your own changes (event-applications.md) or you get a call loop and duplicate engine charges.
cart, and a project allows at most 25 extensions. When both promotions and tax extend the cart, discounts must be applied before tax is computed, since tax is calculated on discounted amounts. commercetools supports extension chaining with declared dependencies (bounded: max 5 direct dependencies, max 3 layers deep, no cycles — violations surface as ExtensionChainTooWide / ExtensionChainTooDeep / CircularDependency). If a promotion connector lands in a project that already has a tax connector, work the ordering out deliberately rather than hoping.Keep the mapping pure and testable
[], invalid code returns 200 + rejection field (not 400).App 2 — the redemption-syncer (OrderCreated Subscription)
What triggers it
event application: you register a Subscription on the order resource for OrderCreated (plus OrderStateChanged / return messages if rollback is in scope) in the app's postDeploy; Connect provisions the queue and delivers each message as an HTTP POST to the app's endpoint. Envelope decoding (base64 message.data on GCP), PlatformFormat vs CloudEventsFormat, message-type filtering, and ack semantics are all the commercetools-connect skill's event-applications.md — don't re-derive them here.What it must do
- Re-fetch the Order by id from
resource.id— don't trust the possibly stale or omitted payload. - Redeem in the engine: consume the coupon, close the customer session, award loyalty points. This is the call that makes the promotion real and the only one that shows up in the engine's reporting.
- Be idempotent on a stable key — the order id. Redelivery is guaranteed, not hypothetical, and this is the one place where a bug costs money: a double redemption double-awards points and can consume a single-use coupon twice. Prefer the engine's own idempotency key / duplicate guard, and treat "already redeemed" as success, not an error to retry.
- Ack correctly. Reply
200for handled and irrelevant-but-acked messages; return non-2xx only for transient failures you want redelivered.
What it must not do
- Don't redeem from the cart. Only an order is a purchase. Redeeming at cart-evaluation time consumes coupons and awards points for carts that are abandoned — the single most damaging design error in this sub-area.
- Don't treat commercetools as the ledger for points/balances. The engine is the system of record. Mirroring a balance onto a Customer custom field is fine for display; reading it back as authoritative is not.
Full lifecycle (if in scope)
- Cancel → roll back the redemption and claw back points, on the order states the merchant designates as cancellations.
- Return → partial rollback, on return-shipment state changes.
- Order edit → re-evaluate: note that changing discounts on an existing Order is not a plain cart update — it needs the Order Edits
setDirectDiscountsaction.
Identity: the session key
- Use the cart id as the engine session key, and the customer id (or a stable anonymous id) as the profile key.
- Cart merge on login is the trap. When an anonymous cart merges into a customer's cart, the cart identity the engine has been evaluating can disappear, and per-customer usage limits may be attributed to the wrong profile. Decide explicitly what happens: re-evaluate under the surviving cart id, and re-key or close the abandoned session.
- Carry the same key into the redemption call, so the engine can tie the redeem back to the session it evaluated.
Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
Extension returns 202 | Every cart update fails | Return 200/201 only |
Invalid coupon returned as 400 | Shopper's whole cart update fails; generic error in the UI | Return 200 + rejection reason in a custom field |
| Redeeming at cart time | Coupons consumed and points awarded for abandoned carts | Redeem only in the OrderCreated syncer |
| Non-idempotent redemption | Redelivery double-redeems / double-awards points | Stable key = order id; "already redeemed" = success |
setDirectDiscounts emitted as a delta | Old discounts linger or vanish unpredictably | Always write the complete array |
relative value forwarded as percent | 10% becomes 0.1% | Convert to permyriad (10% = 1000) |
| Native Discount Codes still expected to work | Codes silently have no effect once Direct Discounts are set | Exclusivity is by design; pick one owner (config-from-requirements.md) |
| No hash / no trigger condition | Engine called on every cart keystroke; bill and rate limits blow up | Condition the trigger; hash promo-relevant fields |
| Hash omits a field the engine matches on | Wrong segment's discount served from a stale evaluation | Hash everything the rules can read |
| Own connector writes the cart out-of-band | Evaluator re-triggers in a loop; duplicate engine charges | Self-change filtering |
| Promotion + tax extension ordering unmanaged | Tax computed on undiscounted amounts | Order via extension chaining/dependencies; discounts before tax |
| >100 actions in one response | Cart operation fails | Fewer, broader-targeted Direct Discounts |
| Extension destination = base URL | Platform's calls 404 the app | Register destination as <CONNECT_SERVICE_URL>/promotionEvaluator |
postDeploy doesn't register the extension / custom type | Evaluator never fires, or setCustomField fails on a missing type | Wire connector:post-deploy idempotently for both |
| Cart merge on login ignored | Usage limits attributed to the wrong profile; session orphaned | Re-key/close the session on merge |
| Gift effect for a product not in the catalog | Mapping throws or silently drops the reward | Decide drop-with-log vs fail; assert it |
| Legacy SDK | Fails the commercetools-connect skill's pinned-version gate | @commercetools/platform-sdk@^8 + @commercetools/ts-client@^4 |
Test-first checklist (mirror in the suite)
Evaluator
- Each engine effect type maps to the right
value/target; permyriad conversion asserted - Complete-array replacement asserted (previous discounts don't leak)
- Returns
200(asserted — the202regression is the one to pin) - Invalid coupon →
200+ rejection custom field, not400 - Hash short-circuit returns
{ actions: [] }; hash covers every engine-visible field - Fail-open asserted for engine error and engine timeout
- Action count stays within the 100-action cap for a large cart
Syncer
- Decodes the envelope; acks irrelevant/test messages
- Re-fetches the Order by id
- Redeems idempotently on the order id; "already redeemed" treated as success
- Nothing is redeemed for a cart that never became an order
- (If in scope) rollback on configurable cancel/return states
Public promotion integrations — which one, and what to fix
connect.yaml and README and in the vendor's docs, they change, and whoever maintains them does it better than a copy here would. Read them at the source — links below.- Which artifact is the production one — for the engines here, that is not obvious, and the vendor's own documentation points elsewhere.
- What to change when you fork one — commercetools production-readiness judgment applied to someone else's connector.
Get the current facts from the source
For any public connector, in this order:
- The repo's
connect.yaml— applications, types, endpoints, scripts, and the fullstandardConfiguration/securedConfigurationsurface. This is the authoritative config contract; nothing else is. - The repo's README — install, credentials, and setup.
- The repo source —
postDeploy(which resources the extensions/subscriptions are registered on) and the effect-mapping module.connect.yamltells you the deployment shape; only the source tells you the behavior. - The vendor's API docs — the engine-side endpoints, session model, and effect vocabulary.
- The marketplace listing — for certified vs registered status, which changes. Record what you saw; don't assert it from memory.
Talon.One — mind which repository
Three different artifacts exist, and picking the wrong one is the most likely early mistake:
| Artifact | Maintained by | Use it? |
|---|---|---|
composable-com/ct-connect-talonone — a Connect connector, MIT | Orium (a systems integrator), not Talon.One | Yes — the production path, and the basis for a rung-1 install or a rung-3 fork |
talon-one/commercetools-talonone-accelerator — AWS/GCP microservice | Talon.One | No. Talon.One's own documentation describes it as an experimental method suited to proof-of-concept or simulation projects, not production |
talon-one/commercetools-talonone-connector — AWS connector | Talon.One | Separate, AWS-specific; not the Connect path |
The model (concepts only — the API is the vendor's to document)
Voucherify — a port, not an install
voucherifyio/commerce-tools-integration is MIT and instructive, but not a Connect application: it is a standalone Node service that registers its own API Extensions and is documented for self-hosted / Heroku deployment. Running it under Connect means porting it (a connect.yaml, lifecycle scripts, endpoint↔route wiring, env vars moved into standard/secured configuration). That is rung-3 work, not a marketplace install — plan it as such. Nothing in the vendor's docs frames it this way, because from their side it isn't a Connect product.Three of its design decisions are worth copying — or consciously rejecting:
- Discounts as negative custom line items by default, Direct Discounts behind a flag. Its docs are explicit that the custom-line-item path requires storefront changes and that the integration bypasses native Discount Codes, storing codes in cart custom fields instead — the exclusivity rule from config-from-requirements.md showing up in a shipped product. For a new build, invert the default: Direct Discounts first.
- Codes and their validation status in cart custom fields — the same pattern promotion-contract.md prescribes for rejecting a coupon without failing the cart update.
- Redemption on payment state →
Paid, not onOrderCreated. A defensible variation: it avoids consuming a coupon for an order that is never paid. Put this to the user as a real decision — redeem at order creation (simpler, matches "the promotion was used", needs rollback on cancellation) or at payment confirmation (nothing consumed for unpaid orders, but the discount is shown before it is consumed, and the syncer subscribes to payment/order-state messages instead). Either is fine; drifting between them by accident is not.
What to fix when you fork (rung 3)
connect.yaml and source rather than assuming it still applies:- Hand-supplied commercetools credentials →
inheritAs.apiClient.scopes.CTP_CLIENT_ID/CTP_CLIENT_SECRET/CTP_SCOPEas secured config means a human provisions and rotates an API client with whatever scopes they happened to grant. Declaring scopes lets Connect mint a least-privilege client instead. Highest-value single change, and both public promotion integrations need it. - Non-secrets in
securedConfiguration. A tax-category id, a locale, a region are configuration, not credentials. Their presence is also a signal: a tax-category id means the connector can represent discounts as custom line items (which require one) — check which mechanism you're inheriting and whether you want it (config-from-requirements.md). - A root
endpoint: /. Works, but makes the route↔endpoint contract easy to break and gives you nothing to distinguish apps by. Prefer a named endpoint with the router mounted to match (project-structure.md). npm installin lifecycle scripts →npm ci --omit=dev. Reproducible, and no dev dependencies in the deployed image.- One
servicedoing both halves. If the connector performs redemption synchronously inside anorderextension rather than anOrderCreatedSubscription, understand the trade you are inheriting: it puts the engine on the critical path of order creation, so an engine outage can block orders — fail-closed on the money path. For new work, split evaluate (service) from redeem (event) as overview.md describes. If you keep synchronous redemption, document that stance in the README. - Pinned SDK versions — check against the commercetools-connect skill's gate (
@commercetools/platform-sdk@^8+@commercetools/ts-client@^4).
Other engines
Verify the promotion round trip
Check 1 — the cart carries engine-computed discounts
Drive a cart update (add a line item, enter a coupon code) and inspect the cart:
directDiscountsis populated. This is the direct tell that the evaluator fired and mapped effects. Before the API Extension is registered and firing, it is simply empty — that means "not wired", not "no promotions apply".- The totals moved.
totalPricereflects the discount, and the per-item breakdown (discountedPricePerQuantity, anddiscountOnTotalPricefor a total-price target) shows where it landed. Confirm the exact reference/field shapes against the current schema with this skill'sopenApi-schemata.mjs --resource-name api-Cart-readrather than a remembered field list. - The version jumped more than your update alone would explain. The evaluator's
setDirectDiscountsandsetCustomFieldactions are extra writes folded into the same operation. - The coupon-result custom field is set — accepted, or rejected with a reason. An entered-but-invalid code should leave the cart update successful with a rejection reason, never a failed request.
directDiscounts and the totals. (Same flow a storefront BFF would run.)Check 2 — the order is redeemed in the engine
OrderCreated subscription deliver (or, locally without Pub/Sub, POST the base64 envelope to the syncer directly), then:- The syncer returns a positive ack (
200/204). - The engine's API confirms the redemption — coupon marked used, session closed, points awarded — keyed on the order id.
- It appears in the engine's reporting/dashboard. Cart evaluation never does; only redemption surfaces there. If the dashboard is empty, suspect "only the evaluator is wired" before suspecting the engine.
Check 3 — redelivery does not double-redeem
OrderCreated envelope twice and assert the engine shows one redemption and one point award. At-least-once delivery makes this a certainty in production, not an edge case.The traps (correct behavior that looks like a bug)
Trap 1 — native Discount Codes stopped working
Trap 2 — zero discount is usually a correct answer
Related: check the engine environment the connector points at. Evaluating against sandbox while inspecting the production dashboard produces exactly the "nothing is happening" symptom.
Trap 3 — discounts disappeared after an unrelated cart update
directDiscounts array, the next successful evaluation self-heals it. Verify both halves deliberately: force an engine failure and confirm the cart update still succeeds, then let the next update repair the discounts. If you see a stuck empty state instead, the evaluator is emitting deltas rather than the full array.Trap 4 — usage limits attributed to the wrong shopper
Log in with an anonymous cart that already carries an evaluated session. If the anonymous cart merges into the customer's cart, the cart identity the engine has been tracking can change, so per-customer usage limits and loyalty attribution can land on the wrong profile — or a single-use coupon can be spent twice across the two identities. Verify the anonymous → known transition explicitly; it is not covered by any happy-path test.
Trap 5 — points awarded for a cart that never became an order
Verification checklist
-
directDiscountspopulated and totals reduced after a cart update (extension registered + firing) - Coupon accepted → discount applied; coupon invalid → cart update succeeds with a rejection reason
- Order placed → syncer acks → redemption/points confirmed via the engine API and visible in its reporting
- Same envelope delivered twice → exactly one redemption and one point award
- Abandoned cart → no redemption, no points
- Forced engine failure → cart update still succeeds (fail-open), and the next update self-heals the discounts
- Anonymous → logged-in cart merge verified; usage limits and attribution land on the right profile
- Cancellation/return → rollback observed in the engine (if in scope)
- Understood: inert native Discount Codes and zero discount from a non-matching campaign are correct, not bugs
Requirements → search document + connector config
connect.yaml. For a public connector these are its documented keys; for a fork or a build these are the keys and apps you define. The official scaffold is the Product export template.The search document, in one line
objectID, carrying only what the storefront queries/filters/sorts/displays — resolved to one price context and one locale strategy, with categories denormalized and availability handled deliberately. Derive it from a Product Projection (staged=false), never the raw Product. The full decision method — granularity, price-context explosion, localization, category denormalization, Store assortment — is data-mapping.md; this file only maps the config those decisions imply.Requirements → app composition
| Job | Default app | Alternative |
|---|---|---|
| Full ingestion — (re)build the whole index from the catalog | service with an on-demand REST trigger (e.g. /fullSync), matching the Product export template | job on a schedule (properties.schedule) when nightly rebuilds are enough and no on-demand trigger is needed |
| Incremental updater — keep the index fresh on catalog changes | event on product/store/selection Subscriptions (e.g. /deltaSync) | job polling Product Projections on lastModifiedAt when the engine or ops model can't take a push |
/product-projections?staged=false with cursor pagination; Store-specific reads /in-store/key={storeKey}/product-projections and is driven by Product Selection messages (the Product export template is Store-specific — one Deployment per Store; don't model one Deployment per Store at scale — see data-mapping.md).The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the Connect docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / properties / configuration; inheritAs), and keep the file at the repository root — a nested connect.yaml silently fails to deploy.Native client provisioning (prefer this)
CTP_CLIENT_ID/CTP_CLIENT_SECRET (a pattern you may see in existing search connectors and should not copy — check which form a fork candidate uses, per connector-selection.md):inheritAs:
apiClient:
scopes:
- view_products # read Products / Product Projections (the catalog to index)
# add only as the requirements need:
# - view_product_selections # Store-specific: read a Store's Product Selection assignments
# - view_stores # Store-specific: resolve the Store / in-store projections
# - manage_subscriptions # the incremental-updater app, whose postDeploy registers the Subscription
Scope notes. The engine side needs no commercetools scope — it is reached with the engine's own API key. Grantmanage_subscriptionsonly to theeventapp (it registers the Subscription inpostDeploy); theservice/jobfull-export app needs onlyview_products(+ the Store read scopes for the Store-specific pattern). There is no write scope here — if you findmanage_productson a search connector, it is over-privileged. Check the current list in API scopes rather than guessing, and grant per app, not per connector.
Per-app config
securedConfiguration; index name, region, locale, price context, and behavioral toggles are standardConfiguration.deployAs:
- name: full-export # (re)build the whole index on demand
applicationType: service
endpoint: /fullSync # the Express router must mount at this same base path
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: SEARCH_INDEX_NAME
description: Target index / collection name
- key: PRICE_CONTEXT
description: "currency[,country[,customerGroup,channel]] used to select the indexed price"
- key: LOCALES
description: "Comma-separated locales to index, e.g. en-US,de-DE"
- key: STORE_KEY
description: "Store key for the Store-specific pattern; omit for whole-catalog"
required: false
securedConfiguration:
- key: SEARCH_ENGINE_API_KEY
description: Engine admin/write API key (index management + record writes)
- name: incremental-updater # keep the index fresh on catalog changes
applicationType: event
endpoint: /deltaSync
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the product/store/selection Subscription
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub or SNS"
Worked example (whole-catalog, Algolia-style, build/fork)
en-US, de-DE, fr-FR); one price context (EUR/DE); availability as a coarse inStock boolean only; near-real-time on publish/unpublish plus a nightly safety rebuild; europe-west1.gcp.SEARCH_INDEX_NAME=products), per-locale fields rather than an index per locale (three locales, one price context — a single index stays simple); objectID = the Product id; price selected with PRICE_CONTEXT=EUR,DE; inStock derived from variant availability but never treated as live stock; categories denormalized to name + breadcrumb path per locale (data-mapping.md).inheritAs:
apiClient:
scopes:
[
view_products, # read the catalog to index
manage_subscriptions, # incremental-updater postDeploy registers the Subscription
]
deployAs:
- name: full-export
applicationType: service
endpoint: /fullSync
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
configuration:
standardConfiguration:
- { key: SEARCH_INDEX_NAME, description: "Algolia index name" }
- { key: PRICE_CONTEXT, description: "EUR,DE" }
- { key: LOCALES, description: "en-US,de-DE,fr-FR" }
securedConfiguration:
- { key: SEARCH_ENGINE_API_KEY, description: "Algolia Admin API key" }
- name: incremental-updater
applicationType: event
endpoint: /deltaSync
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
configuration:
standardConfiguration:
- { key: CONNECT_SUBSCRIPTION_DESTINATION, description: "GoogleCloudPubSub" }
service app that pages /product-projections?staged=false (cursor on id), builds records, and does an atomic replace-all into products — triggered on demand and by the nightly schedule (a job variant, or a scheduler hitting /fullSync); one event app whose postDeploy registers a single Subscription on ProductPublished/ProductUnpublished (+ the Store/Product-Selection messages if Store-specific) and upserts/removes one record per message, idempotently on objectID. Scopes are read-only + manage_subscriptions; the Algolia Admin key is securedConfiguration. Correctness rules per app: search-contract.md.Native, use, fork, or build?
Rung 0 first — is this native?
| Native capability | Covers |
|---|---|
| Product Search | Full-text, fuzzy/prefix/wildcard matching, faceting (distinct/range/count/stats/filtered), sorting, price/Store/Product-Selection scoping. GA June 2024; facets GA October 2025 (Use Product Search) |
| Product Projection Search | The older search endpoint — full-text, filters, facets, localeProjection/storeProjection; returns full projections rather than ids |
| Scoping | Price selection (currency/country/Customer-Group/Channel), storeProjection, and B2B assortment scope resolve a buyer's prices and catalog in the same query (Product Search for B2B) |
Check live data — don't answer from memory
Listings and engine capabilities change. Before deciding among rungs 1/3/4:
- Search the Connect marketplace (
marketplace.commercetools.com/connectors) and the search/discovery listings, plus the docs via thedocs-searchscript or the Knowledge MCP. - Distinguish an installable Connect connector from a vendor-hosted integration — apply the commercetools-connect skill's Marketplace listings are not all Connect connectors rule; don't re-derive it here. It bites hard in search specifically (below).
- Compare the requirement engine-by-capability (indexing, merchandising, synonyms, recommendations, analytics, per-Store scope, locales).
- Name the connector/engine and version you checked, and record it in the requirements block.
The hosted-integration trap (search's sharpest case)
connect.yaml, nothing for Connect to deploy, and this skill cannot build or operate them. If one is a good functional match, surface it with a warning that it is not a Connect solution — the customer configures it in the engine, and there is no connector to build — then, if they need a Connect-deployed pipeline (own the mapping, run it in Connect's infrastructure, no dashboard dependency), offer the fork/build path below.The search landscape (verify live — this is only the shape)
As checked 2026-08:
| Artifact | What it is | Default rung |
|---|---|---|
Product export template (commercetools/connect-product-export-template) | The official Connect scaffold for outbound catalog export, explicitly positioned "for external services such as search". Store-specific: full-export (service, /fullSync) + incremental-updater (event, /deltaSync) | 4 (build) — start here for any engine |
commercetools/launchpad-algolia-sync | An open-source worked Algolia example built on that same two-app shape (full-ingestion + incremental-updater) | 3 (fork) for Algolia — read the repo live before forking |
| Engine dashboard integrations (e.g. "Algolia for commercetools") | Vendor-hosted, dashboard-configured — not a Connect connector | outside this skill — surface with the not-a-Connect-solution warning |
| Bespoke / unsupported engine | Nothing to install | 4 (build) — scaffold from the Product export template |
payment-integration, product-export, tax-integration, transactional-emails). So even a from-scratch engine is rung 4 from a scaffold, not from nothing. Budget accordingly — the plumbing exists; what you write is the engine's SDK calls and the mapping.The ladder (stop at the first rung that fits)
- Native Product Search is enough → build no connector (above). Say why and stop.
- A public connector for the engine covers everything, and it's a real Connect connector → install + configure it. Installation (CLI auth, scopes,
deployment create) is the commercetools-connect skill's deployment-installation.md. Hand it the config from config-from-requirements.md. - Right engine, gap looks like a capability → prove it isn't config first (index name, which fields are indexed, locale/price context, which Store). Most "missing" behavior is a
connect.yamlvalue or engine-side setting → back to rung 1. - Right engine, genuine gap config can't close, and source exists → fork it (for Algolia,
launchpad-algolia-sync), add only the delta, deploy as an Organization connector. Assess the candidate from its current repo (rootconnect.yaml, thefull/incrementalhandlers, the mapping,inheritAs.apiClient.scopesvs hand-supplied credentials) — not from memory. - No usable connector for the engine (a bespoke or unsupported engine) → build by scaffolding from the Product export template and implementing the engine's client + the mapping. What you build is search-contract.md; config is config-from-requirements.md.
Recording the decision
Search: none · rung 0 · checked requirements 2026-08 — "typo-tolerant search with brand/size facets and price sort" is native Product Search; no engine, no connector.
Search: Algolia · rung 3 (fork) · checked marketplace 2026-08 — the dashboard "Algolia for commercetools" is vendor-hosted (not Connect); forking the open-sourcelaunchpad-algolia-syncto add per-locale indices and migrate its hand-supplied CTP credentials toinheritAs.apiClient.scopes.
Search: in-house "DiscoverSvc" · rung 4 (build) · checked marketplace 2026-08 — no listing → scaffolding from the Product export template (full-exportservice +incremental-updaterevent) and writing the engine client + mapping.
From commercetools projection to search document
Principle 1 — Project the current, published data — never the raw Product
staged=false (the current projection), not the Product resource. Only published Products have a current projection; a storefront index must never contain staged edits or unpublished products (current/staged). This one choice prevents the most common data leak — draft content showing up in search. On the incremental path, ProductPublished carries the productProjection in its payload, so you can index it without a re-fetch; for other triggers, re-fetch the projection by id (Principle 9).Principle 2 — Every record gets a stable objectID (the idempotency backbone)
id for product-level records, or <productId>-<variantId> (or the SKU) for variant-level. This is what makes every write an upsert and every delete targetable: re-indexing the same product, redelivering a message, and re-running a full load must all converge, not duplicate. A record whose id you can't reconstruct from a later message is a record you can't update or delete — fix the key before writing any sync code. (Algolia calls this objectID; other engines call it the primary key — same role.)Principle 3 — Record granularity: product-level vs variant-level (a UX decision)
- Product-level (one record per Product): variant-specific facets (size, color) become sets aggregated across variants; a hit links to the PDP. Fewer records, simpler; the default for most catalogs.
- Variant-level (one record per Variant): each color/size is its own hit with its own image and price; needed when the grid shows "red shirt" and "blue shirt" separately. More records; watch the engine's record-count/price tiers.
Match it to how the storefront wants to display results, and keep it consistent — don't mix granularities in one index.
Principle 4 — The price-context explosion (the decision that bites hardest)
- Index one context (e.g.
EUR/DE) — simplest; correct only for a single-market storefront. Select it with the projection's price-selection parameters at map time. - Index facetable price fields per context (
price_EUR_DE,price_USD_US) — one record, several price fields; the storefront picks the field for the shopper's context. Scales to a handful of contexts. - Emit one record per context (a
contextattribute + a filter) — when contexts are many or B2B Customer-Group pricing must be searchable; multiplies record count.
Principle 5 — Localization: index-per-locale vs per-locale fields
{ "en-US": "…", "de-DE": "…" }). A search engine wants one language per searchable field (so stemming/synonyms are per-language). Two shapes:- One index per locale (
products_en,products_de) — the cleanest for language-specific relevance config; the Store-specific and multi-market default. - Per-locale fields in one index (
name_en,name_de) — fewer indices; the storefront queries the shopper's language fields. Fine for a few locales.
localeProjection so you only carry in-scope locales, and map locale codes explicitly (commercetools uses en-US, not en_US).Principle 6 — Denormalize categories, and know the fan-out cost
id/key, and carry the localized name as a display field.Principle 7 — Store assortment: whole-catalog vs Store-specific
- A
stores/productSelectionsfilter field on each record — one index, the storefront filters by the current Store. Simple; fine when tailored content per Store is minimal. - One index per Store — driven by the Store's Product Selection; use the Populate a Store-specific external search tutorial and read
/in-store/key={storeKey}/product-projections(withstoreProjection, which also resolves Store locales and prices). This is what the Product export template implements — one Deployment per Store.
StoreProductSelectionsChanged, ProductSelectionProductAdded/Removed, ProductSelectionVariantSelectionChanged) drive add/remove on the incremental path.Principle 8 — Availability is high-churn and eventually consistent — decide deliberately
ProductVariant.availability is eventually consistent and never authoritative. Decide explicitly:- Usual answer: index a coarse
inStockboolean (or a bucketed level) for filtering "in stock only", refreshed on a cadence — and let the storefront read live quantity from the Inventory API / native search at render time. - Never make the search index the source of truth for live stock, and never wire per-unit inventory events into the index — the write volume will overwhelm it for no UX gain.
Principle 9 — Trust the id, not the payload; and keep the mapping pure
ProductPublished (whose productProjection payload is the just-published state), re-fetch the projection by resource.id so the index converges on current state instead of replaying old deltas. Keep the projection→document transform a pure function — no network calls — so it is unit-testable without a deployment or engine key. Everything the engine needs to rank and display should be in the record; the connector's only job is to keep that record equal to the current projection.The relevance-config boundary (what does NOT live here)
Worked example (sketch)
en-US, de-DE), one price context (EUR/DE), one global index.objectID= Productid. Source =/product-projections?staged=false(full load) andProductPublished.productProjection(delta).- Fields:
name_en/name_de,description_en/description_de(per-locale,localeProjectionlimited to the two);brand,color(set across variants),sizes(set),categories(denormalized breadcrumb names per locale) +categoryIds(facet on stable id);price(selectedEUR/DE) +priceas a numeric sort/facet field;inStockboolean (coarse, refreshed nightly);imageUrl,slug_en/slug_de. - Left out: staged data, out-of-scope locales, per-unit inventory, internal-only attributes, every non-
EURprice. - Delta triggers:
ProductPublished→ upsert;ProductUnpublished/ProductDeleted→ removeobjectID; category rename → reindex affected products (or wait for the nightly rebuild).
objectID rule — that mapping is the deliverable, and it's identical whether a public connector consumes it as config or a custom connector implements it.Checklist
- Records built from the
currentprojection (staged=false) — never staged or unpublished data - Every record keyed on a stable
objectID(product id, or product-id+variant) → every write is an upsert, every delete targetable - Record granularity (product vs variant) chosen for the result UX and kept consistent
- Price context resolved to one strategy (single context · per-context fields · per-context records); no "wrong price in search"
- Locales handled (index-per-locale or per-locale fields);
localeProjectionlimits to in-scope locales; codes mapped (en-US) - Categories denormalized (names/breadcrumb) with the rename→reindex fan-out understood; facets keyed on stable id
- Store assortment reflected (filter field or index-per-store); Store-specific reads in-store projections; per-Store-Deployment scale limit noted
- Availability handled deliberately (coarse flag at most); index is not the live-stock source of truth
- Delta path re-fetches by id (except
ProductPublished); the mapping is a pure, unit-testable function - Relevance config (ranking/synonyms/merchandising) left to the engine, not encoded in the mapping
Search connector — outbound catalog indexing (build or integrate)
connect.yaml, lifecycle scripts, testing, deploy) are the commercetools-connect skill; this sub-area owns the search-specific job end to end — from "do you even need an external engine?" through configuring, forking, or building the connector, to the data mapping and sync architecture that keep the index correct.service/job full load plus an event/job that keeps it fresh), so this whole sub-area is server-side. The engine's relevance configuration (searchable fields, ranking, synonyms, merchandising rules, A/B tests) is owned in the engine, not here — the connector only feeds it correct, current data.Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<search terms from the request, e.g. 'integrate external search product export product projections staged product search subscriptions'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root, where scripts/docs-search.mjs lives.) The load-bearing docs for this sub-area are the two tutorials — Integrate external search (whole-catalog) and Populate a Store-specific external search (Store/Product-Selection-scoped) — the Product export template (the official scaffold), and, for the native gate, the Storefront search overview. Read them. You may additionally use the commercetools Knowledge MCP for deeper follow-up.Step 1 — Extract requirements (before any config or code)
The architecture is downstream of a handful of answers. Ask the user — don't assume:
- Which engine, and why this one over native? Algolia, Constructor, Bloomreach, Coveo, Elasticsearch/OpenSearch, Typesense, Meilisearch, or undecided. Do they already have an account + API keys? If undecided, Step 1.4 may end at rung 0 (native).
- What can native Product Search not do? Name the specific need — merchandising/curation, synonyms and query rules, recommendations, search analytics, A/B testing, learned ranking, or a headless engine the front end already talks to. "Typo-tolerant full-text with facets" is native (storefront-search-overview) — say so (Step 1.4).
- Whole-catalog or Store-specific? One global index, or a Store-scoped index driven by Product Selections / Product Tailoring? This picks the tutorial and the projection endpoint (Step 3).
- Which locales? Drives index-per-locale vs per-locale fields (
localeProjection). - Which price context(s)? Currency, country, Customer Group, Channel — a record can't hold every combination; you must pick (Step 2).
- Does availability/inventory belong in the index? High-churn and eventually consistent — usually a deliberate no or a coarse flag, never the live stock system of record.
- Record granularity — product-level or variant-level? A UX decision (one hit per product vs one per variant) that shapes the whole document.
- Catalog volume and cadence. Size drives batch/pagination; real-time correctness → event-driven, large periodic rebuilds → scheduled
job. - Anything special? (always ask — open-ended) B2B/scoped assortments, multi-currency budgets, category-tree depth, staged-vs-published rules, GDPR in product data. Capture each as its own requirement line; don't force it into a slot above.
Step 1.4 — Rung 0: is native search enough? (STRONG — rule it out first)
Step 1.5 — Native, use, fork, or build?
Step 2 — Data mapping (the heart)
objectID keying, record granularity, the price-context explosion, localization, category denormalization, Store assortment, and where (if anywhere) availability belongs. This is data-mapping.md. Get it wrong and the index drifts no matter how good the plumbing is.Step 3 — Sync architecture (the two apps)
service on-demand trigger or scheduled job) that reindexes the whole catalog atomically, and an incremental updater (event on product/store/selection Subscriptions, or a polling job) that keeps it fresh. The contract for each, and the pitfall catalog, is search-contract.md; the connect.yaml config derived from Step 1 is config-from-requirements.md. The official scaffold is the Product export template.Step 4 — Deploy
deployment create, regions, certification) is the commercetools-connect skill's deployment-installation.md.Step 5 — Verify the sync
References
| Need | Reference |
|---|---|
| Native, use, fork, or build?: the rung-0 native-search gate, the live-marketplace check, the hosted-integration trap, scaffolding from the Product export template | connector-selection.md |
Requirements → config: the search document shape, index/engine keys, connect.yaml envelope, scopes, secured config; worked example | config-from-requirements.md |
Data mapping (the substance): projection → flat document, objectID keying, record granularity, price-context explosion, localization, category denormalization, Store assortment, availability boundary | data-mapping.md |
| The two-app contract: full ingestion (cursor pagination, atomic/blue-green reindex, count check) + incremental updater (idempotent upsert, deletion propagation, staleness guard); full pitfall catalog | search-contract.md |
| Verify the sync: publish/unpublish/delete/full-load/idempotency/per-store checks; the eventual-consistency, availability-drift, and non-atomic-rebuild traps | verification.md |
| Deploy/install a public or custom connector; regions; certification | commercetools-connect → deployment-installation.md |
| Least-privilege scopes, secured config, engine-key handling | commercetools-connect → security.md |
| Scheduled/on-demand job: schedule, 30-min timeout, overlap locking, checkpointing | commercetools-connect → job-applications.md |
Checklist
Requirements
- Engine chosen (or deliberately deferred) + account/API keys; whole-catalog vs Store-specific decided
- The specific need native Product Search cannot meet is named — not just restated as "search"
- Locales, price context(s), and record granularity (product vs variant) decided
- Availability-in-index decision made deliberately (usually no / coarse flag)
- Volume + cadence captured; asked the open-ended "anything special?" question, each special its own line
- Requirements block written and confirmed with the user
Path (decide before wiring/building)
- Rung 0 ruled out explicitly — native Product Search / Product Projection Search can't do it, and you said why
- Checked live marketplace + docs (not memory); a hosted engine integration surfaced with the not-a-Connect-connector warning
- Path presented to the user and chosen by them: use (1) · config-closes-gap (2) · fork (3) · build-from-template (4)
Mapping + sync (the deliverables)
- Projection → flat document mapped;
objectIDkeyed for idempotent upsert; price context and localization resolved → data-mapping.md - Full ingestion reindexes atomically and verifies counts; incremental updater is idempotent and propagates deletions → search-contract.md
- A real change flowed end to end; a re-run left the index unchanged → verification.md
The two-app search-sync contract
full-export + incremental-updater).The rule that spans both apps: the index is a projection, keyed and idempotent
objectID (a stable commercetools id — data-mapping.md) and every removal targets that same id. The index is a derived copy of the published catalog: any record must be reproducible from the current Product Projection, and running either app twice must converge, not duplicate or double-delete. Both full loads and subscription messages are at-least-once, so idempotency is not optional.App 1 — full ingestion (service on-demand trigger, or job on a schedule)
Rebuilds the entire index from commercetools. It is the initial load, the disaster-recovery path, and the nightly backstop that repairs whatever the incremental path missed.
- Authenticate the trigger. A
servicewith a public/fullSyncendpoint must validate the caller (shared secret / signature) before kicking off a rebuild — an open reindex endpoint is a denial-of-wallet and data-exposure risk (security.md). (AuthorizationHeaderauthentication on an Extension'sHTTPdestination is the reverse mechanism, for commercetools calling an Extension — irrelevant here; there is no Extension in this sub-area.) - Page with a cursor, not offset. Read
/product-projections?staged=false&withTotal=false,sort=id asc,limit=100(or up to 500), and page withwhere=id > "<lastId>"(Integrate external search). Offset pagination breaks past a few thousand products; theid-cursor is stable and resumable. For the Store-specific pattern, iterate the Store's Product Selection assignments and read/in-store/key={storeKey}/product-projectionsinstead (Populate a Store-specific external search). - Map with the pure function (data-mapping.md) and bulk/batch writes to the engine — never one HTTP call per record.
- Reindex atomically — build-and-swap, never wipe-then-fill a live index. Build the new index into a temporary/secondary index (or tag every record with a build/generation id), then atomically swap it in and drop the stale set (Algolia's "replace all objects" / a blue-green index alias). Clearing the live index and refilling it leaves a half-empty index serving zero results for the length of the rebuild — the most visible search outage there is.
- Verify counts. After the swap, confirm the engine's record count matches the number of published products (± your granularity multiplier); a large mismatch means the map dropped or duplicated records — fail loudly, don't leave a bad index live.
- Respect the runtime. A
jobhas a 30-min timeout and needs overlap locking and checkpointing (job-applications.md); a very large catalog may need the full load chunked or moved to thejobshape. Keep the initial migration and the ongoing rebuild the same code path.
App 2 — incremental updater (event on Subscriptions, or a polling job)
Keeps the index in step with catalog changes between full loads.
- Subscribe once per message type and fan out in the handler — never one Subscription per index or per Store (the Project allows 50 Subscriptions). Register them idempotently in
postDeploy(get-then-create, never delete-then-recreate). The relevant Product Catalog Messages:ProductPublished→ upsert the record. Its payload carries theproductProjection(the just-publishedcurrentdata), so you can map it directly without a re-fetch.ProductUnpublished→ remove the record byobjectID. An unpublished product must leave the index or it becomes a ghost result linking to a dead PDP.ProductDeleted→ remove the record. (Design augmentation — not in the tutorial's set. Its payload field iscurrentProjection, notproductProjection; in practice a delete is usually preceded by an unpublish that already removed the record, so treat this as a belt-and-braces removal.)- Store / Product Selection messages (Store-specific):
StoreProductSelectionsChanged,ProductSelectionProductAdded→ add to that Store's index;ProductSelectionProductRemoved→ remove from it;ProductSelectionVariantSelectionChanged→ re-index;StoreCreated/StoreDeleted→ provision/tear down the Store's index.
- Decode the envelope, then ack correctly. The GCP transport wrapper is
{ "message": { "data": "<base64>" } }; decodemessage.data(base64 → JSON), validate the messagetype, and ack-and-ignore anything you don't handle (including the platform's test message). Return2xxfor handled and deliberately-ignored messages; non-2xxonly for transient failures you want redelivered. - Upsert is idempotent under redelivery — writing the same record twice is a no-op by construction (keyed on
objectID). - Deletion propagation is a first-class path, not an afterthought. Every removal trigger (unpublish, delete, removed-from-selection, and — if
inStockfiltering matters — dropping to zero stock) needs a defined action; a missing one leaves ghost records. - Guard against stale writes. With no ordering guarantee, an older message can arrive after a newer one. Except for
ProductPublished(whose payload is current), re-fetch the projection byresource.idso the index converges on current state; where the engine supports it, additionally guard on a version /lastModifiedAtso an out-of-order write can't overwrite newer data. - The polling
jobalternative: query/product-projections?where=lastModifiedAt > "<checkpoint>"on a schedule, upsert the page, advance the checkpoint. Simpler ops, but it cannot see deletions (a deleted product no longer appears in the query) — pair it with the nightly full rebuild to purge ghosts, or subscribe toProductUnpublished/ProductDeletedfor removals.
Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
| No deletion propagation on unpublish/delete | Unpublished products still appear in search; hits link to dead PDPs (ghost records) | Handle ProductUnpublished/ProductDeleted → remove by objectID |
| Wipe-then-fill a live index | Search returns zero/partial results for the whole rebuild window | Build-and-swap / replace-all-objects (atomic); drop the old set after the swap |
| Indexing staged / unpublished data | Draft content and unpublished products surface in search | Read the current projection (staged=false) only |
| Trusting a stale message payload | Older delta overwrites newer state; out-of-order writes | Re-fetch by resource.id (except ProductPublished); guard on version/lastModifiedAt |
| Price context mismatch | Wrong price in search results / facets | Select one context at map time; index per-context fields/records (data-mapping.md) |
| Category rename not fanned out | Stale category names/breadcrumbs on products | Reindex affected products on category messages; nightly rebuild as backstop |
| Per-unit inventory wired into the index | Write volume overwhelms the engine; cost spikes | Coarse inStock flag refreshed on cadence; live stock from the Inventory API |
| Offset pagination on the full load | Full load misses/duplicates products past a few thousand | Cursor on sort=id asc + where=id > "<lastId>" |
| One call per record | Full load times out / hits engine rate limits | Batch/bulk writes |
| No count check after reindex | A silently truncated index goes live | Assert engine count ≈ published-product count; fail loudly on mismatch |
| Envelope not decoded | Handler sees base64 garbage / crashes | Decode message.data (base64 → JSON), then validate type |
| Wrong ack | Handled message redelivered forever, or failures silently dropped | 2xx for handled/ignored; non-2xx only for retryable |
| One Subscription per index/Store | Hits the 50-Subscription Project limit | One Subscription per message type; fan out in the handler |
Unauthenticated /fullSync trigger | Anyone can trigger a full reindex (denial-of-wallet) | Validate a shared secret/signature before starting |
| Engine key over-scoped or in logs | Admin key leaked; compliance incident | Key in securedConfiguration; generic error responses; no payload dumps |
Route ≠ connect.yaml endpoint | Platform traffic / trigger 404s | Mount the router at the app's endpoint base path |
| Legacy SDK | Fails the commercetools-connect skill's pinned-version gate | @commercetools/platform-sdk@^8 + @commercetools/ts-client@^4 |
Test-first checklist (mirror in the suite)
Full ingestion
- Rejects unauthenticated / bad-signature trigger calls
- Pages with the
idcursor (assertswhere=id > "<lastId>"), not offset; maps via the pure function - Reindex is atomic (build-and-swap) — a rebuild never leaves the live index empty/partial
- Count check asserts engine count ≈ published-product count; mismatch fails the run
- Store-specific: iterates the Store's Product Selection and reads in-store projections
Incremental updater
-
ProductPublishedupserts from the payload projection; second delivery is a no-op -
ProductUnpublished/ProductDeletedremove the record byobjectID - Store/Selection add/remove messages add/remove from the right Store index
- Stale/out-of-order message doesn't overwrite newer state (re-fetch by id / version guard asserted)
- Envelope decode + ack matrix covered (handled, ignored, retryable)
- Polling-
jobvariant (if used): advances checkpoint; deletions covered by rebuild/removal messages - Boundary mocked (engine + commercetools APIs); suite runs with no deployment and no secrets
Verify the search sync
/fullSync trigger directly. Two of the checks below regularly look broken when they're actually correct — read the traps.Check 1 — a publish appears in the index (delta path)
Publish a Product (or change and re-publish one), then confirm:
- A record with the expected
objectIDexists in the engine, with the mapped fields — name/description in the in-scope locales, the selected-context price, denormalized categories, and image. - Searching for a term in the product's name returns it, and its facets (brand/color/size) are populated.
- Only the
current, published data is present — no staged edits, no unpublished siblings. If staged content shows up, the connector is reading the wrong projection (data-mapping.md). - Re-deliver the same
ProductPublishedmessage: nothing duplicates (idempotent upsert onobjectID).
localeProjection mapping is wrong — not that indexing failed.Check 2 — an unpublish/delete disappears (deletion propagation)
job variant is in use, confirm removals are covered by the removal messages or the nightly rebuild — a lastModifiedAt poll alone can't see deletions.Check 3 — a full ingestion matches the published catalog, atomically
/fullSync (or run the job) against a known catalog, then confirm:- The engine's record count equals the published-product count (× your granularity multiplier for variant-level). A mismatch means the map dropped or duplicated records.
- The index never went empty or partial during the rebuild. Query it while a rebuild runs (or inspect that the connector built into a temporary index and swapped) — a live index that returns zero/partial results mid-rebuild is the non-atomic-reindex bug (Trap 3), not a timing quirk.
- Re-run the full ingestion: the resulting index is identical (same count, same records) — the load is idempotent.
Check 4 — per-Store scope holds (Store-specific pattern)
The traps (behavior that looks like a bug — or hides one)
Trap 1 — the lag is eventual consistency, not a dropped update
Trap 2 — availability in the index drifts, and that's by design
inStock flag, it is a cadence-refreshed snapshot, not live stock — ProductVariant.availability itself lags and is eventually consistent (up to ~10 s). A search result showing "in stock" for something that just sold out is expected; the storefront must confirm live quantity from the Inventory API / native search at render or add-to-cart. Verify the flag refreshes on its cadence — don't expect it to track real-time stock.Trap 3 — a "flaky, half-empty" index during rebuilds is a non-atomic reindex
Trap 4 — sandbox catalog vs production
Verification checklist
- Publish → record present with mapped fields (locales, selected-context price, categories, image);
currentdata only; redelivery doesn't duplicate - Unpublish and delete → record gone from the index (no ghost results)
- Full ingestion → engine count ≈ published-product count; index never empty/partial mid-rebuild; re-run identical (idempotent)
- Store-specific: add/remove from a Product Selection scopes to the right Store index; in-store projection resolved Store locales/prices
- Confirmed eventual-consistency lag converges (not a dropped update)
-
inStock/availability treated as a cadence snapshot, not live stock; live quantity read from Inventory/native search - Contract verified on sandbox; volume + rate-limit behavior verified on a production-sized catalog; test records cleaned up
- No engine key or payload dumps in logs; the
/fullSynctrigger rejects unauthenticated calls
Requirements → shipping connector config
connect.yaml values. Give each a one-line why when you present it.connect.yaml and README — read those and use their names; the keys below are illustrative and will not match. For a build, these are keys you define, so the names are yours; what each carrier needs configured (service codes, package/unit conventions, account identifiers) comes from the carrier's own API docs.The requirement → config map
| Requirement (Step 1) | Config / decision | Why |
|---|---|---|
| Which shipping service + credentials | securedConfiguration: API key/secret plus whatever account identifiers that carrier's API requires (take the list from their docs) | Secrets never in standardConfiguration, never hardcoded |
| Quoting, execution, or both | Which applications you declare in deployAs | One service for rating, one event for labels/tracking; deploy only what's in scope |
| Landing mechanism (A/B/C) | standardConfiguration: e.g. RATE_STRATEGY: score | customShippingMethod | The hard-to-reverse decision from shipping-contract.md — make it explicit, not implicit in code |
| Enabled carriers / service levels | standardConfiguration: comma-separated carrier + service codes | The commonest post-launch change; must not need a code change |
| Origin address / ship-from | standardConfiguration (or per-Channel lookup if multi-warehouse) | Rates are origin-dependent; a wrong origin quietly misprices everything |
| Package defaults & dimensional weight | standardConfiguration: default package dims, weight unit, DIM divisor | Carriers price on dimensional weight; defaults belong in config, not constants |
| Markup / handling fee | standardConfiguration: percentage or flat amount | Merchants change this often and it is not a carrier setting |
| Latency + fallback | standardConfiguration: CARRIER_TIMEOUT_MS, QUOTE_CACHE_TTL_S, FALLBACK_SHIPPING_METHOD_KEY | The fail-open contract has to be operable without a redeploy |
| Which Order Messages trigger a label | standardConfiguration: message types / order-state gate; Subscription registered in postDeploy | Booking on the wrong trigger buys labels for unpaid orders |
Single vs Multiple shipping mode | Not config — it changes which update actions the code emits | Decide in requirements; it is fixed per Cart |
| Region + project | standardConfiguration: CTP_REGION; scopes via inheritAs | Hosts and client provisioning are region/project specific |
| Sandbox vs production carrier account | standardConfiguration: CARRIER_SANDBOX | Sandbox rates are usually list rates, not negotiated — see verification.md |
The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / configuration; inheritAs) and put the file at the repository root — a nested connect.yaml silently fails to deploy.Scopes (least privilege)
CTP_CLIENT_ID/SECRET. What a shipping connector actually needs:| Scope | Needed by | For |
|---|---|---|
manage_extensions | rate app postDeploy/preUndeploy | register/remove the Cart API Extension |
manage_shipping_methods | rate app postDeploy | provision the fallback Shipping Method (and the $0 carrier-quoted method for path C) |
view_tax_categories | rate app | resolve the taxCategory reference a custom shipping method must carry |
manage_subscriptions | label app postDeploy/preUndeploy | register/remove the Order Subscription |
manage_orders | label app | re-fetch the Order and write addDelivery / addParcelToDelivery / setParcelTrackingData |
manage_types | postDeploy | create the Custom Types for the quote-hash Cart field and the carrier shipment id on Delivery |
manage_key_value_documents | rate app (optional) | quote cache in CustomObjects, if not held in a Cart custom field |
view_products | rate app (optional) | read weight/dimension attributes when they live on the Product, not the Line Item |
view_extensions / view_subscriptions are not valid standalone scopes (manage_extensions / manage_subscriptions cover read + write; declaring the view variants fails client creation).Worked example — both applications
deployAs:
- name: shipping-rates
applicationType: service
endpoint: /shippingRates
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the Cart API Extension + fallback Shipping Method
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
- name: shipping-labels
applicationType: event
endpoint: /shippingLabels
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the Order Subscription
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
inheritAs:
apiClient:
scopes:
- manage_extensions
- manage_shipping_methods
- manage_subscriptions
- manage_orders
- manage_types
- view_tax_categories
configuration:
standardConfiguration:
- key: CTP_REGION
description: commercetools region, for example europe-west1.gcp
- key: RATE_STRATEGY
description: "How quoted rates land on the Cart: score or customShippingMethod"
default: customShippingMethod
- key: ENABLED_CARRIER_SERVICES
description: Comma-separated carrier/service codes to quote, in display order
- key: SHIP_FROM_POSTAL_CODE
description: Origin postal code used for rating
- key: SHIP_FROM_COUNTRY
description: Origin country code (ISO 3166-1 alpha-2)
- key: HANDLING_MARKUP_PERCENT
description: Percentage added to every quoted rate
default: "0"
- key: CARRIER_TIMEOUT_MS
description: Outbound carrier timeout; must stay well under the API Extension budget
default: "1200"
- key: QUOTE_CACHE_TTL_S
description: How long a quote may be reused for an unchanged cart
default: "300"
- key: FALLBACK_SHIPPING_METHOD_KEY
description: Shipping Method applied when the carrier is unavailable (fail-open)
- key: LABEL_TRIGGER_ORDER_STATE
description: Order state that gates label creation, for example Confirmed
- key: CARRIER_SANDBOX
description: "'true' to call the carrier sandbox instead of the live account"
default: "true"
securedConfiguration:
- key: CARRIER_API_KEY
description: Carrier or rate-service API key
- key: CARRIER_ACCOUNT_NUMBER
description: Carrier account number that negotiated rates are tied to
postDeploy has real work here
More than in most sub-areas, because the connector depends on commercetools resources existing:
- Fallback Shipping Method — create it get-then-update (only if absent), with a predicate that always matches and a rate in every currency in scope. Without it, fail-open has nothing to fall back to.
- Custom Types — the Cart field holding the quote hash/cached quote, and the Delivery field holding the carrier shipment id.
- The Cart API Extension — registered with an authenticated destination; use the discriminator value
AuthorizationHeader(not the schema type nameAuthorizationHeaderAuthentication, which fails withInvalidJsonInput), and confirm registration withGET /{projectKey}/extensionsafterwards (security.md). - Deploy-time credential validation — make one cheap carrier call (an account or rate ping) so bad credentials fail the deployment instead of the first shopper's cart.
Checklist
- Only documented
connect.yamlenvelope keys; file at the repository root - Applications declared match the scope actually agreed (rating, execution, or both)
- Landing strategy is an explicit config key, not implicit in code
- Carrier credentials + account number in
securedConfiguration; everything operational instandardConfiguration -
CARRIER_TIMEOUT_MSset below the API Extension budget; cache TTL set -
FALLBACK_SHIPPING_METHOD_KEYset and the method provisioned bypostDeploy - Scopes least-privilege and valid (no
view_extensions/view_subscriptions; no cart write scope for the extension) -
postDeployidempotent for Extension, Subscription, Types, and fallback Shipping Method;preUndeployremoves what it created -
postDeployvalidates carrier credentials at deploy time - Sandbox-vs-production carrier account switchable by config
Native, use, customise, or build?
Rung 0 — native Shipping Methods (the gate)
Most "we need a shipping integration" requests are a Shipping Method modeling problem. Answer this before anything else, and state the answer explicitly.
| Requirement as stated | Native mechanism | Connector needed? |
|---|---|---|
| Flat rate per country/region; free over a threshold | Zones + zone rates + freeAbove | No |
| Price by weight / volume / item count / distance band | Tiered rates over Cart Score (or a priceFunction) — the score is set with setShippingRateInput by whoever computes it | No |
| Price by an abstract bucket ("Light"/"Bulky") | Tiered rates over Cart Classification | No |
| Option only available for certain stores, addresses, warehouses, or cart contents | Shipping Method predicate | No |
| Same-day / click-and-collect as a distinct option | A Shipping Method (plus predicate); BOPIS modeling | No |
| Exact price known only from a third party, once, late in checkout | Cart freeze (SoftFreeze) + setCustomShippingMethod — or Order Edits after the fact | Not necessarily |
| Live multi-carrier rate shopping per cart, negotiated account rates, live service levels and delivery estimates | — | Yes |
| Labels, pickup-point selection, tracking numbers, returns labels from a carrier API | — | Yes |
Two constraints that decide borderline cases:
- The Project's
shippingRateInputTypeis a single choice. Cart Value, Cart Classification, or Cart Score — not a mix. If the score is already committed to another purpose, the tiered-rate route is closed and the case moves toward a connector. - 100 Shipping Methods per Project (soft limit, limits). A carrier × service level × zone matrix blows through this. If your native design needs dozens of near-identical methods, that is a signal the rates want to be quoted, not enumerated.
Rung 1 — is there a public connector? (live check, never memory)
-
Check live, programmatically.
shippingis a valid ConnectIntegrationType, so query the registry rather than browsing:GET {connect-host}/connectors/search?integrationTypes=shipping # add &text=<carrier or platform name> to narrow; &integrationTypes=oms if fulfilment is also in scopeEach result carrieskey,integrationTypes,creator,repository,configurations,supportedRegions,certified, andprivate— usecertified: true/private: falsefor public connectors, andrepositoryto judge whether rung 3 (fork) is even possible (deployment-installation.md). Also search the Connect marketplace and the integration docs (viadocs-search/ the Knowledge MCP) by the service name the user gave you. Name the connector key + version you found, or record "none exists". Availability changes; a connector you remember may not exist, and one you don't may. -
Apply the listings rule — it bites hard here. Shipping is one of the categories where a vendor's "commercetools integration" is most often their hosted service plus glue you write, not a deployable Connect application: a dashboard where you paste commercetools API credentials, a rate endpoint you are expected to call yourself, or an app in the vendor's own marketplace. None of that is Connect-deployable. Confirm a Connect affordance — a public connector repo, a root
connect.yaml, a Connect deploy action — before you call anything "install". Full rule: Marketplace listings are not all Connect connectors. -
If the fit is a vendor-hosted integration, say so plainly: the Connect build patterns (
connect.yaml, the Connect CLI, lifecycle scripts, the Connect deployment model) do not apply to it. Point at the vendor's onboarding, and offer the in-skill alternative — build a Connect connector that calls their API (rung 4), which is usually a thin client over the same endpoints.
The landscape (verify live, but this is the shape)
| What you may find for a shipping service | What it usually is | Default rung |
|---|---|---|
| A rate/checkout-rules engine advertising commercetools support | Their hosted rating service plus glue you write; frequently documented as requiring custom development | Verify Connect-deployability; usually 4 (thin connector over their rate API) |
| A label/shipping-execution platform ("connect your store, print labels") | Their SaaS pulling orders via API credentials you paste into their dashboard | Often out of Connect scope — say so; else 4 |
| A single carrier's own API | An API, not an integration | 4 |
| An OMS/WMS that already books carriers | Not a shipping connector at all | Hand to order-management |
| Nothing for the service | Common | 4 |
Rung 2 — prove the gap isn't configuration
Before forking anything, check whether the "missing" behavior is a config value on the existing connector or on the commercetools side:
- Enabled carriers and service levels, markup/discount, package/dimension defaults, origin address, currency and unit system → almost always
standardConfiguration. - Which options appear where, and their ordering/default → often Shipping Method
predicateandisDefaulton the commercetools side, not connector code. - Cut-off times, insurance, signature-on-delivery → frequently carrier-account settings, not connector code.
If config closes it, go back to rung 1.
Rung 3 — fork and customise
- Root
connect.yaml: which applications, whichinheritAs.apiClient.scopes, which config keys already exist. - The rate path: does it use
setShippingRateInputor a custom shipping method (this is the hard-to-change decision — shipping-contract.md). - Timeout and fallback handling on the extension path; whether it caches quotes.
- Idempotency on any delivery/parcel write-back.
- Test suite and last release; whether it targets
@commercetools/platform-sdk@^8+@commercetools/ts-client@^4.
Rung 4 — build a new one (the common case)
commercetools connect init (connect-cli.md) and replace the domain logic:| What you're building | Scaffold from | Why it maps |
|---|---|---|
| Rate quoting on the Cart | tax-integration | Structurally identical: a Cart API Extension that calls an external service and returns update actions, plus an OrderCreated Subscription. Swap tax calls for rate calls; the extension registration, postDeploy/preUndeploy, and envelope plumbing carry over unchanged. |
| Labels, shipments, tracking write-back | fulfilment-integration | Its order-export and order-updates applications already model "Order out, fulfilment data back", which is exactly the label/tracking loop. |
| Both | tax-integration for the extension app, fulfilment-integration for the event app | One connector, two applications in one connect.yaml. |
Present it, don't pick it
Once you have the live landscape, put the rungs to the user with a recommendation and the reasoning: native (0), install (1), config (2), fork (3), build (4). Record the chosen rung, the connectors checked, and their versions in the requirements block. If you recommend build, say which template you'll scaffold from and why.
Checklist
- Rung-0 native gate answered explicitly, with the mechanism named or ruled out
-
shippingRateInputTypeavailability and the 100-Shipping-Method limit checked against the proposed design - Ran
GET /connectors/search?integrationTypes=shipping(not memory) plus a marketplace check; cited connector key + version and itscertified/privateflags - Every candidate confirmed Connect-deployable (repo /
connect.yaml/ deploy action), not vendor-hosted - If vendor-hosted: said plainly that this skill doesn't cover it, and offered the build alternative
- Config-vs-code tested before recommending a fork
- Fork candidates assessed from the current repo against the production-readiness gate
- For a build: template chosen and justified (
tax-integrationfor rating,fulfilment-integrationfor execution) - Ladder presented to the user; they chose the rung; decision recorded
Shipping connector — integrate a carrier or rate service
Three facts to state before anything else
Say these out loud to the user early. Each one changes the plan, and each is the opposite of what the payment/gift-card experience suggests.
- There is no shipping connector contract. Checkout defines connector contracts for payment and gift cards (Connectors and applications); shipping is not one of them. There is no enabler/processor pair and no session-authenticated contract to conform to — a shipping connector is a plain Connect connector built from
service/event/jobapplications. Note the distinction:shippingis a valid ConnectIntegrationType, so it classifies a connector in the registry (and is how you search for one — see connector-selection.md); it does not prescribe the connector's shape. - There is no shipping application template. The documented templates are payment, product-export, tax, and transactional email (Application templates); the CLI additionally exposes
fulfilment-integration. You scaffold from the closest architectural twin — see connector-selection.md. - Most shipping vendors ship a vendor-hosted integration, not a Connect connector. Rate engines and label platforms commonly document "connect your commercetools store" flows that run on their infrastructure and require custom glue on yours. That is not something Connect deploys. Apply the commercetools-connect rule — Marketplace listings are not all Connect connectors — before calling anything installable.
Scope boundary — three neighbours
Shipping work lands in one of four places. Route it, don't absorb it:
| The user needs | Owner |
|---|---|
| Zones, Shipping Method modeling, tiered rates, cart score, shipping predicates, BOPIS — no external service in the loop | commercetools-commerce-patterns (rung 0 below) |
| Live rates, labels, or tracking from a carrier or rate service | this sub-area |
| Order orchestration, allocation, fulfilment status and inventory owned by an OMS/WMS (which usually also owns shipment + tracking) | order-management |
| Tax on the shipping line | tax |
Delivery/Parcel is a data-integrity bug, not redundancy.Rung 0 — can native Shipping Methods do this? (gate before designing a connector)
freeAbove, predicates, and tiered rates driven by the Project's shippingRateInputType setting (Cart Value, Cart Classification, or Cart Score — with priceFunction available inside Cart Score tiers) — see Shipping and Delivery Overview and the Shipping Methods reference. The docs' own guidance is that external calculation is for when tiered rates are not enough (Shipping Methods & Rates).- Rates are a table (per zone, per weight band, per cart value, free over a threshold) → native. Zones + tiers +
freeAbove. No connector. - Rates are a function of one computable number (weight, volume, distance, item count) → native tiers over Cart Score, optionally expressed as a
priceFunction, with the score set by whoever computes it. → tiered-rates-cart-score.md - Availability varies by store, warehouse, address, or cart contents → native shipping predicates. → shipping-predicates.md
- Only the price is unknown until an external system says so, and the flow can quote once late in checkout → still not necessarily a connector: the documented cart-freeze +
setCustomShippingMethodpattern covers it. → dynamic-shipping-costs.md - You need a connector when the carrier itself must be called per cart (multi-carrier rate shopping, negotiated/account-specific rates, live service levels and delivery estimates), or when labels/tracking/pickup points must be created against a carrier API.
shippingRateInputType is a single choice (Cart Value or Cart Classification or Cart Score). If cart score is already used for something else, that option is spent. Also note the soft limit of 100 Shipping Methods per Project (limits) — a design that mints a Shipping Method per carrier × service level × zone will hit it.Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<shipping terms from the user's request, e.g. 'shipping methods tiered rates cart score custom shipping method delivery parcel tracking'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root.) Use its output as primary grounding; the Knowledge MCP and Shipping and Delivery Overview are for deeper follow-up.connect.yaml and README) rather than recalled. Fetch that at the moment you need it; don't write carrier field names from memory.Step 1 — Extract requirements (before any config or code)
Shipping design is downstream of business facts, and the wrong default produces either a blocked checkout or a wrong price on a placed Order. Ask (don't assume):
- Which shipping service, and what does it actually do? A carrier API (one carrier), a multi-carrier aggregator, a rate/checkout-rules engine, or a shipping-execution/label platform. These need different applications. Do they have an account and API credentials already?
- Which side do you need — quoting, execution, or both? Rates at checkout, labels + tracking after the Order, or both. This decides the number of applications and is the single biggest scope lever.
- Is an OMS/WMS in the picture? If yes, who books the carrier? → boundary above.
- Shipping mode:
SingleorMultiple? Multiple (split shipments, per-line-item methods and addresses) changes every update action and is not reversible once set on a Cart. - Must quoted options appear in
GET /shipping-methods/matching-cart? The storefront's normal shipping-options list only shows native Shipping Methods. If the storefront reads that endpoint and you land carrier prices as custom shipping methods, the options never show up. → this is the Step 3 landing decision; get the answer here. - Rate granularity: per cart, per shipment/delivery group, or per line item? Are pickup points / parcel lockers in scope (address selection, not just price)?
- Latency and fallback: what should happen when the carrier is slow or down — block the cart or fall back to a native rate? Shipping is a fail-open candidate; get the business decision, don't pick it yourself.
- Region and project (e.g.
europe-west1.gcp, project key) — the CT API/Auth hosts are region-specific. - Anything special or non-standard? (always ask — open-ended) Dimensional weight, hazmat/dangerous goods, oversize surcharges, duties/DDP for cross-border, insurance, negotiated account rates, delivery-date promises, returns labels, multi-origin/warehouse routing, carrier cut-off times. Capture each as its own line; don't force it into a slot above.
Step 1.5 — Native, use a public connector, customise one, or build? (decide before building)
- Native Shipping Methods — the gate above. Stop here if it fits.
- Use a public connector directly — a Connect-deployable shipping connector for this service exists and covers the requirements → install and configure.
- The gap is config, not code — prove it before forking. Enabled carriers, service levels, markup, package defaults and fallback behavior are typically
connect.yamlvalues → back to rung 1. - Customise/fork — a real gap config can't close, and an open-source connector for the service exists → fork, add only the delta, deploy as an Organization connector.
- Build a new one — no connector for the service (the common case here) → build from the type-agnostic
service/event/jobpatterns, scaffolding from the closest template. Which template and why: connector-selection.md.
Step 2 — Derive the config from the requirements
connect.yaml values with a one-line why each: which applications, carrier credentials in securedConfiguration, enabled carriers/service levels/package defaults/markup/timeout/fallback in standardConfiguration, and least-privilege scopes. Mapping table, scopes, and a worked example: config-from-requirements.md.Step 3 — Build the applications, test-first
setShippingRateInput over native tiers vs. setCustomShippingMethod/addCustomShippingMethod), and what each choice costs you at matching-cart. Then build, red test first for each:- Rate application (
service, usually a Cart API Extension) — quote the carrier and land the result. This is where the extension response budget (2 s by default, configurable per Extension viasetTimeoutInMsup to 10 s), the fail-open decision, and quote caching live — set the timeout deliberately for the carrier's latency instead of inheriting the default. - Label/tracking application (
eventon Order Messages) — book the shipment, then writeaddDelivery→addParcelToDelivery→setParcelTrackingDataback onto the Order, idempotently. Skip this app entirely if an OMS owns it. - Optional
job— tracking-status polling or reconciliation where the carrier has no outbound webhook.
Step 4 — Verify the round trip
matching-cart, and sandbox rates that aren't the negotiated ones.References
| Need | Reference |
|---|---|
| Native, use, customise, or build? the rung-0 native gate, the live registry check, the vendor-hosted-integration trap, which template to scaffold from, how to assess a fork candidate | connector-selection.md |
The contract (read before coding): how quoted rates land on the Cart and the matching-cart consequence; the rate extension's latency/fail-open budget; the label + tracking write-back; Single vs Multiple mode; full pitfall catalog | shipping-contract.md |
Requirements → connect.yaml: applications, carrier credentials, carrier/service-level/package/markup/timeout config, least-privilege scopes; worked example | config-from-requirements.md |
| Verify the round trip: quote → option shown → Order priced → label → tracking; the blocked-cart, invisible-option, and wrong-rate traps | verification.md |
Native shipping modeling (rung 0): zones, tiers, cart score, predicates, BOPIS, the freeze + setCustomShippingMethod pattern | commercetools-commerce-patterns |
| OMS/WMS owns fulfilment, shipment status, and tracking write-back | order-management/overview.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
Adding another carrier or rate service later means a new mapping and new credentials — the landing decision, the application shapes, and the contract do not change.
Checklist
Scope and rung 0
- Quoting vs execution (labels/tracking) scoped explicitly; OMS/WMS boundary settled and only one writer of
Delivery/Parcel - Native Shipping Methods ruled out explicitly (zones/tiers/predicates/
freeAbove/freeze +setCustomShippingMethod), with the reason stated -
shippingRateInputTypenot already spent on another use case; 100-Shipping-Method soft limit not designed into
Requirements
- Shipping service identified and classified (carrier / aggregator / rules engine / label platform); credentials available
-
SinglevsMultipleshipping mode decided; rate granularity decided - Answered whether options must appear in
GET /shipping-methods/matching-cart - Fail-open vs fail-closed decided by the user, with the fallback rate named
- Asked the open-ended "anything special?" question; each special requirement its own line
- Requirements block written and confirmed
Path
- Checked the registry/marketplace live (not memory); named connector + version
- Verified any candidate is actually Connect-deployable, not a vendor-hosted integration
- Ladder rung presented to the user and chosen by them: native (0) · use public (1) · config-closes-gap (2) · fork (3) · build (4)
Build
- Landing mechanism chosen deliberately, with the
matching-cartconsequence understood - Extension timeout set deliberately (
setTimeoutInMs, 2 s default / 10 s max), outbound carrier timeout under it, plus a cache/short-circuit and the agreed fallback - Label/tracking app idempotent on a stable delivery key; re-fetches the Order before writing
- Carrier boundary mocked; suite runs with no deployment/secrets
- commercetools-connect production-readiness gate satisfied (commercetools-connect)
Verification
- A real cart shows a real carrier price; the placed Order carries that exact amount
- Carrier outage exercised — cart still completes per the agreed fallback
- Tracking number visible on the Order's Parcel (if in scope)
The shipping connector contract
The landing decision (read first)
| A — score/classification over native tiers | B — custom shipping method | C — quote once, late | |
|---|---|---|---|
| Update action | setShippingRateInput (Score or Classification) | setCustomShippingMethod (Single) / addCustomShippingMethod (Multiple) | setCustomShippingMethod after freezeCart |
| Who computes the price | commercetools, from the tier table | your connector, verbatim from the carrier | your connector, verbatim |
Appears in GET /shipping-methods/matching-cart | Yes — the matching tier is resolved and flagged isMatching | No — it is not a Shipping Method | No |
| Multiple carrier options side by side | Yes, one Shipping Method per option | Only via your own endpoint | No — one price |
| Arbitrary carrier amounts | No — must fit tiers or a priceFunction | Yes | Yes |
| Cost | the Project's single shippingRateInputType; tier tables to maintain | storefront must source the option list from you | needs a freeze step in the checkout flow |
matching-cart unchanged; your extension only supplies the score. Tier types and the price function are in Shipping and Delivery Overview → tiered shipping rates and Shipping Methods.matching-cart will never list your quoted options. The storefront must fetch the option list from a connector endpoint and then apply the chosen one. Plan that endpoint deliberately (see below); discovering it late means reworking the checkout UI.setCustomShippingMethod, fully worked in dynamic-shipping-costs.md. Note that reference's own warning: the freeze must use SoftFreeze, because HardFreeze blocks shipping-method updates and this pattern silently breaks under it.Mixing A and B in one connector is legitimate (A for the standard ladder, B for a quoted express option), but say so explicitly — the storefront then has two sources of options and must merge them.
Field shapes you'll actually emit
setCustomShippingMethod requires shippingMethodName and shippingRate, and optionally takes taxCategory, externalTaxRate, custom, and estimatedDelivery (from/until) — the natural home for a carrier's delivery-window promise. addCustomShippingMethod (Multiple mode) additionally requires shippingKey and shippingAddress, and accepts shippingRateInput and deliveries. ShippingRateDraft is price (required) plus optional freeAbove and tiers. Confirm the exact current shape from the Cart OAS rather than from memory:node scripts/openApi-schemata.mjs \
--resource-name "api-Cart-write" \
--app-name "<current-app>" --model "<current-model>" --skill-name "commercetools-integrations"
taxCategory, or externalTaxRate when the Project is in External tax mode. If an external tax connector also extends the Cart, the shipping rate must land before tax is computed — order the two deliberately with extension chaining rather than hoping, and note the project cap of 25 extensions. Coordinate with tax.Application 1 — the rate application (service)
The budget is the design
ExtensionBadResponse, ExtensionNoResponse). Carrier rate APIs, and multi-carrier rate shopping in particular, are the classic way to blow that budget. So:- Set an outbound timeout strictly under the extension budget, with headroom for your own work. Make it configuration, not a constant.
- Rate-shop in parallel, and degrade rather than wait. Return the carriers that answered; don't let the slowest define the response.
- Short-circuit when nothing rate-relevant changed. Hash the inputs that actually move the price — shipping address, line-item quantities/SKUs, weight, chosen service level — store the hash and the quote on the Cart (custom field or
CustomObject), and return no update actions when the hash matches. Most Cart updates in a checkout flow don't change any of them, and each avoided carrier call is a round trip you keep inside the budget — measure the carrier's actual latency and size the timeout from that, not from a number quoted here. Your own response does not re-invoke you — the returned actions are applied inside the same operation — but the storefront's next call, applying the shopper's chosen option, is a fresh Cart update that does re-trigger the extension. The short-circuit is what stops that from being a second carrier call. - Cache per quote-relevant hash, not per cart, with a TTL short enough that a stale rate can't reach an Order.
Fail-open is a business decision, and shipping usually wants it
postDeploy (a "Standard" rate that always matches), log the degradation with the correlation ID, and let checkout continue. But fail-open means a cart can be priced below cost, so get the decision from the user and record it, along with what the fallback rate is. Whatever you pick, state it in the connector README (deployment-installation.md).200 with your update actions. Do not use the extension to signal "no shipping available" by erroring — model unavailability as no matching option, and let the shipping predicate or an empty option list express it.The option-list endpoint (path B only)
matching-cart, the connector needs a second, plain HTTP route on the same service app — an inbound endpoint the storefront BFF calls to get [{ carrier, serviceLevel, price, estimatedDelivery }]. This is not an API Extension: the 5-minute service timeout applies, and you authenticate the caller yourself (security.md). Keep it and the extension reading the same cached quote so the list and the applied price cannot disagree.Application 2 — the label & tracking application (event)
Delivery/Parcel data is a defect.OrderCreated plus an OrderStateChanged/OrderShipmentStateChanged gate; don't book a label the instant an Order exists unless payment and fulfilment really are settled. Envelope decoding, ack semantics, and re-fetch-by-ID are the commercetools-connect contract (event-applications.md).The write-back is three Order update actions, in order:
addDelivery— the shipment. Always setdeliveryKeyto a value you can recompute (e.g.order.id+shipment index): it is your idempotency handle. InMultipleshipping mode also setshippingKeyto bind the Delivery to the right shipping entry.addParcelToDelivery— the physical parcel(s), with measurements and items.setParcelTrackingData—TrackingDatacarriestrackingId,carrier,provider,providerTransaction, andisReturn(use it for return labels so they don't read as outbound shipments).
--resource-name "api-Order-write") rather than from memory.deliveryKey already exists; if it does, ack and stop. Where the carrier API supports an idempotency key, send one derived from the same value — that closes the window between "label bought" and "Delivery written". Never keep a local dedup store.postDeploy, see lifecycle-scripts.md) so a later status update or void can find it.Optional job
Multiple shipping mode
Multiple mode (split shipments, per-line-item methods and addresses) changes the whole surface:- Rates are per shipping entry, keyed by
shippingKey; useaddShippingMethod/addCustomShippingMethod, notsetShippingMethod/setCustomShippingMethod. - Line items are bound to addresses through
itemShippingAddresses+shippingDetails. - Quoting is per group, so one Cart update can mean several carrier calls — the latency budget gets tighter, not looser. This is often the case that forces path A or an asynchronous quote.
- Deliveries must carry
shippingKey— optional in theaddDeliveryschema, so the platform won't reject you for omitting it; without it the Delivery isn't bound to a shipping entry, which is a silent data bug rather than an error.
shippingMode is fixed once the Cart is created. Decide it in requirements (Step 1), not while coding. Concepts: Shipping and Delivery Overview; a worked multi-address example: Multiple Shipping Addresses and Methods.Pitfall catalog
| Symptom | Cause | Fix |
|---|---|---|
| Quoted options never appear at checkout | Landed as a custom shipping method; the storefront reads matching-cart, which only returns Shipping Methods | Path A, or add the connector option-list endpoint and change the storefront to use it |
| Every cart update is slow, then carts start failing | Carrier call on the hot path with no short-circuit; extension exceeds its budget | Input hash + cached quote; outbound timeout under the budget; parallel rate shopping |
| Carts can't be updated at all during a carrier outage | Fail-closed extension | Fail-open to the postDeploy-provisioned fallback Shipping Method (business decision, documented) |
| Tier prices never change | Project shippingRateInputType not set, or not the type your tiers use | Set the Project's shippingRateInputType; tiers only apply when it is configured |
| Cart Score can't be used in a shipping predicate | Score isn't addressable in predicates | Mirror it to a Cart custom field — shipping-predicates.md |
| Score rejected or price wrong for fractional values | Cart Score must be a non-negative integer | Scale (×10/×100) and scale the tiers to match |
| Shipping line has no tax / wrong tax | Custom shipping method supplied without taxCategory, or External tax mode without externalTaxRate | Supply the tax category or external rate; coordinate with the tax extension |
| Freeze-and-quote pattern silently stops updating the rate | HardFreeze blocks shipping-method updates | Use SoftFreeze — dynamic-shipping-costs.md |
| Duplicate labels / duplicate Deliveries | Message redelivery with no idempotency handle | Recomputable deliveryKey, re-fetch-and-check before booking, carrier idempotency key |
| Tracking number never reaches the Order | Subscription not registered (postDeploy failed quietly), or the Message type isn't subscribed | Verify the Subscription exists; check postDeploy actually ran — lifecycle-scripts.md |
| Rates differ between test and production | The carrier's sandbox may not return your negotiated account rates (check what its docs say it returns) | Verify against a production carrier account before go-live — verification.md |
| Design needs dozens of near-identical Shipping Methods | Carrier × service level × zone enumerated natively | Quote instead of enumerate; the 100-method soft limit is real |
| Can't register the extension at all | Project already at the 25-API-Extension maximum | Consolidate extensions, or reconsider path A (no per-cart carrier call) |
Checklist
- Landing mechanism (A / B / C) chosen deliberately; the
matching-cartconsequence stated to the user - If path B: option-list endpoint designed, authenticated, and sharing the extension's cached quote
- Outbound carrier timeout configured under the extension budget; rate shopping parallelized
- Input-hash short-circuit implemented, so a Cart update that can't change the price makes no carrier call
- Fail-open/fail-closed decided by the user, fallback Shipping Method provisioned in
postDeploy, documented in the README - Tax on the shipping line handled (
taxCategoryorexternalTaxRate); interaction with a tax extension considered -
estimatedDeliverypopulated when the carrier returns a delivery window - Label/tracking app built only if no OMS owns shipment booking
-
deliveryKeyrecomputable; re-fetch-and-check before booking; carrier idempotency key where available -
shippingKeyset on rates and Deliveries inMultiplemode - Over-shipment guarded in connector code (the platform does not validate it)
- Carrier boundary mocked in tests; timeout, outage, and duplicate-delivery paths each have a test
Verify the shipping round trip
The round trip
Run these in order; each one fails differently.
- The extension is registered.
GET /{projectKey}/extensionsreturns your Cart extension with the destination and trigger you expect. ApostDeploythat silently didn't run is the single most common cause of "nothing happens" — check this first, not last (lifecycle-scripts.md). - A quote happens. Create a Cart, set a shipping address, and confirm from the connector logs (correlated by
X-Correlation-ID) that exactly one carrier call was made, with the origin, destination, and weight you expect. - The option is visible to the shopper. Path A:
GET /shipping-methods/matching-cart?cartId=…returns the methods with the correct tier resolved (isMatchingon the rate/tier). Path B: your connector's option-list endpoint returns the quoted options, and the storefront reads that, notmatching-cart. Verify whichever one the storefront actually calls — this is the failure that unit tests never catch. - The price lands on the Cart. Apply the chosen option and read the Cart's
shippingInfo: the amount matches the carrier quote exactly,shippingMethodNameis what you expect, and the tax on the shipping line is present and correct. - The rate survives to the Order. Place the Order and confirm
shippingInfo.priceon the Order equals the quoted amount. A rate that changes between quote and order is a cache-TTL or re-quote bug. - Idempotence of the hot path. Make an unrelated Cart update (change a custom field, set an email). Confirm no carrier call was made — the short-circuit works. This is what keeps the extension inside its budget in production.
- Label booked once. For an in-scope execution app: trigger the Order Message, confirm a shipment exists at the carrier, and confirm the Order has one
Deliverywith your computeddeliveryKey. - Tracking is on the Parcel. The Order's Parcel carries
trackingDatawith the carrier'strackingIdandcarrier, and the number resolves on the carrier's tracking page. - Redelivery is a no-op. Re-deliver the same Subscription message. No second label, no second Delivery, and the handler acks (
102/200/201/202/204).
Failure-path checks (do not skip)
These are the ones that hurt in production, and they can only be proven by breaking things on purpose.
- Carrier down. Point the connector at an unreachable carrier host (or force the timeout). The Cart update must still succeed and the agreed fallback must apply. If carts fail, the connector is fail-closed — go back and confirm that is what the user actually chose.
- Carrier slow. Inject latency above
CARRIER_TIMEOUT_MS. The extension must return within its budget on the fallback path, not ride the carrier's timeout into anExtensionNoResponse. - Bad credentials. Deploy with a wrong API key.
postDeploymust fail the deployment, not defer the failure to the first shopper. - Address the carrier rejects (undeliverable postcode, PO box for an express service). The shopper must see "no option", not a 500 that blocks the cart.
Traps that look like bugs
| What you see | What it actually is |
|---|---|
| The storefront shows no shipping options, but the connector logs a successful quote | Path B: quoted rates landed as a custom shipping method, which is not a Shipping Method and never appears in matching-cart. The storefront must call the connector's option-list endpoint. → shipping-contract.md |
| Every cart update fails with an extension error | Carrier call on the hot path exceeding the extension budget, or a fail-closed error path. The platform does not retry within the API call — the whole update fails |
| Rates are plausible but consistently higher than expected | The carrier sandbox may not price against your negotiated account rates — confirm what it returns in the carrier's docs, and re-verify against a production carrier account before go-live |
| Prices are right in one country and wrong in another | Origin (SHIP_FROM_*) or unit system misconfigured; or dimensional-weight divisor differs per carrier region |
| Tier prices never change with weight | The Project's shippingRateInputType isn't set, or isn't the type your tiers use — tiers only apply when it is configured |
| The score is rejected or rounds oddly | Cart Score must be a non-negative integer — scale fractional values and scale the tiers with them |
| A shipping predicate that should match doesn't | Cart Score isn't addressable in predicates; mirror it to a Cart custom field → shipping-predicates.md |
| The rate stops updating after the cart is frozen | HardFreeze blocks shipping-method updates; the quote-late pattern needs SoftFreeze → dynamic-shipping-costs.md |
| Two labels for one order | Redelivery with no idempotency handle — deliveryKey not recomputed, or not re-checked before booking |
| Tracking never appears | Subscription not registered, the wrong Message type subscribed, or the OMS (not this connector) actually owns the write-back |
| Shipping is untaxed on the Order | Custom shipping method emitted without taxCategory, or External tax mode without externalTaxRate |
| Deliveries exceed what was ordered | commercetools does not validate delivered quantities against ordered ones — the guard is yours |
Checklist
- Extension registered and confirmed via
GET /{projectKey}/extensions - Exactly one carrier call per rate-relevant change; zero on unrelated Cart updates
- Options visible through the endpoint the storefront actually calls
- Cart
shippingInfoand the placed Order carry the quoted amount, with correct tax - Carrier-down, carrier-slow, bad-credentials, and rejected-address paths each exercised deliberately
- Label booked once;
deliveryKeypresent; redelivery is a no-op - Tracking number on the Parcel and resolvable at the carrier
- Rates re-verified against a production carrier account before go-live
- Fail-open/fail-closed behavior matches what the user chose and what the README says
Avalara (and TaxJar) specifics
Avalara — the certified connector (ground truth)
mediaopt/avalara-commercetools-connector (open source, certified). It is the reference implementation of the two-app pattern, plus a Merchant Center config app.Three applications
| App | type | endpoint | Role |
|---|---|---|---|
service | service | /service | Calculator (cart API Extension) |
event | event | /event | Recorder (order Subscription): commit / void / refund / recalculate |
mc-app | merchant-center-custom-application | (MC) | Config/admin UI — credential test, address-origin validation, settings |
avatax npm SDK; Express; Jest. Both service+event run postDeploy: npm install && npm run build && npm run connector:post-deploy.Calculation (the service app)
- API Extension on
cart,[Create, Update], conditionshippingAddress is defined and shippingInfo is defined and lineItems is not empty— the strong call-reduction gate. - AvaTax call:
AvaTaxClient.createTransaction()(Avalara/api/v2/transactions/create). For the quote phase it setstype = SalesOrder (0)andcommit: false— a tax estimate that files nothing. - Tax mode
ExternalAmount, returned viachangeTaxModeplus the full set of tax actions:setLineItemTaxAmount,setCustomLineItemTaxAmount,setShippingMethodTaxAmount,setCartTotalTax.taxRatename isavaTaxRate,amountderived from the AvaTax response detail. - Idempotency / call reduction:
hashCart(cart)compared to a storedavalaraHashcustom field; recalculates only when the hash changed ortaxedPriceis absent, then persists the new hash. - Fail-closed: returns
400("No Avalara merchant configuration found.") on error/misconfig — blocks the cart rather than persisting untaxed. - Ship-from / ship-to:
shipFrom= configured origin address;shipTo=cart.shippingAddress.
Recording (the event app)
-
Subscription on
order, destination built from the injected broker (GoogleCloudPubSuborSNS), message typesOrderCreated,OrderStateChanged,OrderStateTransition,OrderReturnShipmentStateChanged. -
A transaction manager drives the lifecycle, mostly keyed on merchant-configured order-state ID lists (settings in a custom object), not hardcoded names:
commitTransaction— files the sale (onOrderCreatedif the booleancommitOnOrderCreation, or when state ∈commitOrderStates).voidOrRefundTransaction— on state ∈cancelOrderStates(plus a residual hardcodedorderState === 'Cancelled'check in theOrderStateChangedpath).partiallyRefundTransaction— on return-shipment state change, gated by the booleanactivateReturns(a flag, not a state-ID list).recalculateTransaction— for order edits.
The lesson to carry over on a fork/build: model the commit/cancel states as configurable lists (state keys differ per project); returns can be a simpler on/off flag.
Config keys (exact)
- service
standardConfiguration:CTP_REGION; custom-type keys/names for shipping, line item, category, shipping-method, customer, order (e.g.avalara-connector-custom-shipping,avalara-connector-order);AVATAX_PRODUCT_ATTRIBUTE_NAME(optional, defaultavatax-code). - service
securedConfiguration:CTP_PROJECT_KEY/CTP_CLIENT_ID/CTP_CLIENT_SECRET/CTP_SCOPE,AVALARA_USERNAME,AVALARA_PASSWORD,AVALARA_COMPANY_CODE,AVALARA_ENV,FRONTEND_API_KEY(optional). - event: standard
CTP_REGION+AVATAX_PRODUCT_ATTRIBUTE_NAME; secured = same CTP + Avalara keys. - mc-app: standard
CUSTOM_APPLICATION_ID,CLOUD_IDENTIFIER(defaultgcp-eu),ENTRY_POINT_URI_PATH.
Note: it suppliesCTP_CLIENT_ID/SECRETmanually (secured config), not viainheritAs.apiClient.scopes. On a fork or new build, prefer native client provisioning (config-from-requirements.md) — it's the more modern, lower-maintenance form.
Enterprise features worth knowing (they're config, not forks)
- Tax-code mapping, multi-level: product attribute (
AVATAX_PRODUCT_ATTRIBUTE_NAME, defaultavatax-code) → category custom fields (getCategoryTaxCodes) → shipping/custom-line-item types. - Exemptions:
getCustomerEntityUseCode(cart.customerId)reads the customer's Avalara entity-use code from a Customer custom field (avalaraEntityUseCode). - Address validation:
/avalara/check-address→client.resolveAddress()(/addresses/resolve), toggleable. - Settings in custom objects, managed by the MC app: address-validation toggle,
commitOnOrderCreation,commitOrderStates/cancelOrderStates,activateReturns, logging, tax-code map, entity-use codes.
TaxJar — the build-from-template contrast (rung 4)
The engine calls (the two halves)
- Calculate:
POST /v2/taxes(liveapi.taxjar.com, sandboxapi.sandbox.taxjar.com). Send destination address + line items (major-unitunit_price) + shipping; get backtax.amount_to_collect,tax.rate, andtax.breakdown.line_items[]/tax.breakdown.shipping. Stateless — stores nothing. - Record:
POST /v2/transactions/orders. Sendtransaction_id(= order id, for idempotency),transaction_date, destination,amount(net),shipping,sales_tax,line_items[]. This is what appears in the dashboard.
Mapping notes
- Convert commercetools minor units (
centAmount/fractionDigits) to TaxJar major-unit decimals once, in the mapper. - Emit all four
ExternalAmountactions; take per-line tax fromtax.breakdown.line_items[]keyed by the line id you sent, shipping tax fromtax.breakdown.shipping, and fall back to the effectivetax.ratewhen a breakdown entry is absent. - Product tax code: read a Custom Field (e.g.
taxjar-tax-code) and pass asproduct_tax_code; omit when absent (TaxJar treats it as fully taxable).
TaxJar-specific gotchas (learned from a real build)
to_stateis required on transactions — a destination without a state yields406 to_state can't be blank. Ensure the address carriesstate, and omit blank optional fields rather than sending empty strings.- Sandbox does not persist transactions.
POST /v2/transactions/ordersreturns201, but GET returns canned demo data and nothing appears in the dashboard. Transactions only show up on a live account. Prove recording against live (with cleanup), not sandbox — verification.md. - Zero tax without nexus. TaxJar only collects where the account has nexus; a destination outside your nexus correctly returns
amount_to_collect: 0. Test against a nexus region. - Duplicate = success. A redelivered order hits TaxJar's duplicate-
transaction_idguard (422); treat it as already-recorded.
Cross-engine summary
| Dimension | Avalara (certified) | TaxJar (from template) |
|---|---|---|
| Rung | 1 configure / 3 fork | 4 build |
| Calculate API | createTransaction (commit:false) | POST /v2/taxes |
| Record API | createTransaction (commit:true) | POST /v2/transactions/orders |
| Tax mode | ExternalAmount | ExternalAmount |
| Lifecycle | commit/void/refund/recalc on configured states | OrderCreated (add void/refund yourself) |
| Tax codes | product attr → category → type (multi-level) | single custom field passthrough |
| Exemptions | entity-use code from Customer field | add yourself |
| Address validation | yes (resolveAddress) | no |
| Config UI | MC app + custom objects | env/config only |
| Extra apps | + merchant-center-custom-application | none |
Requirements → tax connector config
connect.yaml values. For a certified connector these are its documented keys; for a from-template build these are the keys you define. Provider-exact key names/defaults are in avalara.md.The requirement → config map
| Requirement (Step 1) | Config / decision | Why |
|---|---|---|
| Which engine + credentials | securedConfiguration: engine API token or username/password/company-code | Secrets never in standardConfiguration, never hardcoded |
| Nexus regions | (engine-side account setting, not connect.yaml) | The engine only returns tax where you have nexus; a missing nexus is the usual "tax is zero" cause |
| Region + project | standardConfiguration: CTP_REGION; scopes via inheritAs | Host + client provisioning are region/project specific |
| Calculation + recording | Deploy both apps (calculator + syncer); calculation-only = just the calculator | Recording is a separate engine API and a separate Connect app |
| Void on cancel / refund on return | Syncer subscribes to OrderStateChanged / return messages + the order-state → action mapping | Filing must follow the order's real lifecycle, not just creation |
| Product tax categories/codes | Tax-code source setting (Product attribute name / Tax Category / Custom Field) | The calculator must know where to read each item's tax code |
| Tax-exempt buyers | Exemption/entity-use-code source (Customer Custom Field) | Passed to the engine so exempt buyers are taxed correctly |
| VAT-inclusive / rounding | Cart taxMode, taxCalculationMode, taxRoundingMode (and includedInPrice on the external rate) | Controls how the platform combines the external amounts |
Tax mode
changeTaxMode), it decides who owns the arithmetic:ExternalAmount(recommended). You supply the exact tax amounts; commercetools stores them as-is. No re-derivation, so no rounding drift between what the engine files and what the cart shows. This is what the tax docs recommend and what the certified Avalara connector uses. Requires taxing every priced element — line items, custom line items, and shipping — plus a cart total (see tax-contract.md).External. You supply tax rates; commercetools computes amounts. Simpler payloads, but the platform's rounding can differ from the engine's by cents — a reconciliation headache when the engine is the system of record for filing.Platform/Disabledare not external-engine modes.
ExternalAmount unless the user has a specific reason (e.g. they only have rates, not amounts). Record the choice and why.The connect.yaml envelope
connect.yaml has no published JSON Schema — its shape is defined only by the docs. Use only documented envelope keys (deployAs / applicationType / endpoint / scripts / configuration; inheritAs), and place the file at the repository root — a nested connect.yaml silently fails to deploy.Native client provisioning (prefer this)
CTP_CLIENT_ID/SECRET:inheritAs:
apiClient:
scopes:
- manage_extensions # calculator postDeploy registers the Cart API Extension
- manage_subscriptions # syncer postDeploy registers the OrderCreated Subscription
- view_orders # syncer re-fetches the Order to build the transaction
configuration:
standardConfiguration:
- key: TAX_SANDBOX
description: "'true' to call the engine sandbox instead of live"
securedConfiguration:
- key: TAX_PROVIDER_API_TOKEN
description: Tax engine API token, used by both apps
Note:view_extensions/view_subscriptionsare not valid standalone scopes —manage_extensions/manage_subscriptionscover read + write. Declaring the non-existent view scopes fails client creation.
CTP_CLIENT_ID/SECRET/SCOPE as secured config; migrating to inheritAs.apiClient.scopes is the more native, lower-maintenance form and is worth doing on a from-template build.Per-app config
deployAs:
- name: tax-calculator
applicationType: service
endpoint: /taxCalculator
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the API Extension
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
- name: order-syncer
applicationType: event
endpoint: /orderSyncer
scripts:
postDeploy: npm ci --omit=dev && npm run connector:post-deploy # registers the Subscription
preUndeploy: npm ci --omit=dev && npm run connector:pre-undeploy
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub or SNS"
Template gotcha: the official template'stax-calculatorpostDeploywas justnpm install— it never registered the extension, and its post-deploy pointed the destination at the app's base URL instead of<url>/taxCalculator. Wireconnector:post-deployfor both apps, and make the extension destination include the endpoint path.
Worked example (TaxJar, from-template build)
tax-code; no exemptions yet; ExternalAmount; europe-west1.gcp.Derived config:
inheritAs:
apiClient:
scopes: [manage_extensions, manage_subscriptions, view_orders]
configuration:
standardConfiguration:
- key: TAXJAR_SANDBOX
description: "'true' for sandbox; note the sandbox does not persist transactions"
securedConfiguration:
- key: TAX_PROVIDER_API_TOKEN
description: TaxJar API token (same token both apps)
deployAs:
- name: tax-calculator
applicationType: service
endpoint: /taxCalculator
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
- name: order-syncer
applicationType: event
endpoint: /orderSyncer
scripts: { postDeploy: "npm ci --omit=dev && npm run connector:post-deploy", preUndeploy: "npm ci --omit=dev && npm run connector:pre-undeploy" }
configuration:
standardConfiguration:
- key: CONNECT_SUBSCRIPTION_DESTINATION
description: "GoogleCloudPubSub or SNS"
ExternalAmount so TaxJar's amounts are authoritative; both apps because they want filing, not just checkout tax; manage_extensions+manage_subscriptions+view_orders are exactly what the two postDeploy scripts and the syncer's order re-fetch need — nothing more. Nexus is DE-only on the account, so US destinations will (correctly) show zero tax — flag this so it isn't mistaken for a bug (verification.md).AVATAX_PRODUCT_ATTRIBUTE_NAME, AVALARA_USERNAME/PASSWORD/COMPANY_CODE/ENV, commit/void order-state settings), see avalara.md.Is a certified tax connector enough?
Check live data first — don't answer from memory
Supported engines and their capabilities change. Before deciding:
- Search the Connect marketplace (
marketplace.commercetools.com/connectors) and the tax docs via thedocs-searchscript or the Knowledge MCP. - Compare the requirements engine-by-capability (calculation, recording/filing, void/refund, exemptions, address validation, regions).
- Name the connector and version you checked, and record it in the requirements block.
The tax landscape (verify, but this is the shape)
| Engine | Public certified connector? | Source available? | Default rung |
|---|---|---|---|
| Avalara (AvaTax) | ✅ Yes (marketplace) | ✅ Open source (mediaopt/avalara-commercetools-connector) | 1 (configure) — or 3 (fork) since the source is open |
| Vertex (O Series) | ✅ Yes (marketplace) | ❌ Partner-private | 1 (configure) — fork not possible without source |
| TaxJar | ❌ No public connector | — (only the generic template) | 4 (build from template) |
| Other (Sovos, ONESOURCE, …) | Check the marketplace | Varies | Likely 4 unless a listing exists |
The ladder (stop at the first rung that fits)
Rung 1 — Configure a certified connector (Avalara, Vertex)
deployment create --connector-key, or Merchant Center install) is the commercetools-connect skill's deployment-installation.md. Hand the connector the config you derive in config-from-requirements.md.Rung 2 — A gap that config can close
Rung 3 — Fork/extend the public connector (Avalara)
Rung 4 — Build from the tax template (TaxJar, or any engine with no connector)
What you actually write on rung 4:
- The calculator: cart → engine calculate-request, response → the four
ExternalAmountupdate actions (see tax-contract.md). - The syncer: order → engine record-transaction call, idempotent on order id; optionally void/refund on lifecycle.
- Config + scopes (config-from-requirements.md).
200 not 202; shipping and custom line items must be taxed or the Order won't create in ExternalAmount mode; legacy SDK versions). Those are catalogued in tax-contract.md and grounded, with a real engine, in avalara.md (which contrasts the certified Avalara approach with a from-template TaxJar build).Recording the decision
Tax: TaxJar · rung 4 (build) · checked marketplace 2026-07 — no public TaxJar connector, Avalara/Vertex exist but engine is fixed to TaxJar by an existing account · building both apps from the tax template.
Tax connector — integrate an external tax service (backend-focused)
- tax-calculator (a
serviceregistered as a cart API Extension) — the calculate half. On cart changes, commercetools calls it synchronously; it asks the tax engine for the tax on the current cart and returns cart update actions that put the tax onto the cart. This is a quote — nothing is filed. - order-syncer (an
eventdriven by an OrderCreated Subscription) — the record half. After an order is placed, it asynchronously records the finalized order as a transaction in the tax engine (for reporting/filing), and — for a full integration — commits/voids/refunds as the order's lifecycle changes.
Calculation vs. recording is the mistake to internalize first. "Why don't my transactions show up in the tax provider's dashboard?" is almost always because only the calculator is wired: the calculate API stores nothing; only the record API (the order-syncer) persists a transaction. They are different endpoints on the provider and different Connect apps here.
Workflow
Step 0 — Gather context (required, run first)
node scripts/docs-search.mjs \
--query "<tax terms from the user's request, e.g. 'tax connector external tax API extension cart tax order sync'>" \
--app-name "<current-app ex: claude, copilot, codex>" \
--model "<current-model>" \
--skill-name "commercetools-integrations" \
--limit 10
commercetools-integrations skill root.) Use its output as primary grounding. You may additionally use the commercetools Knowledge MCP or https://docs.commercetools.com/tutorials/tax-integration for deeper follow-up.Step 1 — Extract requirements (before any config or code)
Tax behavior is downstream of business facts, and the wrong default silently produces wrong or missing tax. Extract these first; each maps to a config key in Step 2 or a rung in Step 1.5. Ask the user (don't assume):
- Which tax engine, and why? Avalara/Vertex (enterprise US sales tax + global, compliance/filing), TaxJar (simpler US sales tax), or another. Do they already have an account + credentials?
- Where do they have nexus / an obligation to collect? Which countries/states. This decides which destinations produce non-zero tax and is a frequent "why is tax zero?" cause (see verification.md).
- Region and project? e.g.
europe-west1.gcp, projectmy-project— the API host and config are region-specific. - Do they need transaction recording / filing, or just calculation at checkout? Calculation-only (rare) is one app; recording (the norm for compliance) needs the order-syncer too. → decides whether you build one app or two.
- Order lifecycle beyond creation? Should cancellations void the filed transaction and returns refund it? → drives whether the syncer subscribes to
OrderStateChanged/return messages, not justOrderCreated. - Product tax categories / codes? Are products taxed differently (clothing, food, digital, luxury)? Where is the tax code stored — a Product attribute, a Tax Category, or a Custom Field? → drives the tax-code mapping in the calculator.
- Tax-exempt buyers? B2B/non-profit/government exemptions, exemption certificates or entity-use codes → stored on the Customer (Custom Field) and passed through.
- B2B / included-in-price / rounding needs? VAT-inclusive pricing (
includedInPrice),taxCalculationMode(LineItem vs UnitPrice),taxRoundingMode. - Anything special or non-standard? (always ask — open-ended) Marketplace/multi-seller, cross-border/customs, multi-currency, address validation, invoicing, or a specific engine account/company code. Capture each as its own requirement line; don't force it into a slot above.
ExternalAmount tax mode → tax code from a Product attribute → and say so explicitly.Step 1.5 — Is a certified connector enough? (decide before wiring or building)
docs-search script / Knowledge MCP), and name the connector + version you checked. The tax landscape as of writing: Avalara and Vertex have certified public connectors; TaxJar does not (build from template). See connector-selection.md.- Public connector covers everything → install + configure (Step 2). Don't build. Installing it (CLI auth, scopes,
deployment create) is the commercetools-connect skill's deployment-installation.md; it is not theconnectorstagedflow. - Supported engine, gap looks like a capability → prove it isn't config first. Most "missing" behaviors (which order states commit/void, tax-code source, exemptions) are
connect.yamlvalues or Merchant Center settings → back to rung 1. See config-from-requirements.md. - Supported engine, genuine gap config can't close → fork/extend the public connector (Avalara's is open source; see avalara.md); add only the delta and deploy as an Organization connector. Don't rebuild a working, maintained connector. Hand off to commercetools-connect for the build/publish lifecycle.
- No public connector for the engine at all (e.g. TaxJar) → build from the tax integration template. The template ships both apps as stubs; you implement the engine calls and the mapping. The exact contract, gotchas, and a worked engine are in tax-contract.md and avalara.md (with TaxJar as the from-scratch example). The template docs state it is for development purposes and needs further customization before production use, so read and test-drive the generated lifecycle scripts and validators rather than treating them as production-ready (connect-cli.md, Step 2).
Step 2 — Derive the config from the requirements
connect.yaml values for the chosen connector (or your own), with a one-line why for each. The mapping and the provider-specific key names/defaults are in config-from-requirements.md and avalara.md. Key decisions that live here:- Tax mode:
ExternalAmount(recommended) vsExternal.ExternalAmountmeans the engine's exact amounts are authoritative — no re-derivation, no rounding drift between what's filed and what's shown.Externalhas commercetools compute from a rate you supply. The docs and the certified connector both preferExternalAmount. → config-from-requirements.md. - API-client scopes the connector needs — declare them in
inheritAs.apiClient.scopesso Connect provisions a least-privilege client (manage_extensions,manage_subscriptions,view_orders), rather than hand-supplyingCTP_CLIENT_ID/SECRET. - Secured vs standard config — the engine API token/credentials are
securedConfiguration; region and behavioral toggles arestandardConfiguration.
Step 3 — The extension trigger & call-reduction (reference)
taxMode="ExternalAmount", a shipping address is set, line items exist), and consider hashing the cart to skip redundant calls. Full contract and the call-reduction pattern: tax-contract.md.Step 4 — Build/verify the two apps (the main body of work), test-first
200/201 (not 202), taxing shipping and custom line items too (or the Order can't be created in ExternalAmount mode), idempotent recording, committing only on the right order states — are invisible at the call site and tedious to reproduce by hand. Each is one cheap assertion. Write the test first.- Calculator (API Extension) — map cart → engine request; call the engine's calculate API; map the response to
setLineItemTaxAmount+setCustomLineItemTaxAmount+setShippingMethodTaxAmount+setCartTotalTax(andchangeTaxModeif you own that); respond200fast; decide fail-open vs fail-closed. - Order-syncer (Subscription) — on
OrderCreated, re-fetch the Order by id, map it to the engine's record/commit transaction API, POST idempotently (stabletransaction_id= order id). For a full integration, also handle cancel→void and return→refund.
Step 5 — Verify the round trip
taxedPrice appears on the cart (it's absent until the extension is registered and firing), and — with a live engine account whose nexus covers the destination — a transaction is recorded. See verification.md, which also covers the two traps that make people think it's broken when it isn't: sandbox accounts often don't persist transactions, and an engine returns zero tax where you have no nexus.References
| Need | Reference |
|---|---|
| Is a certified connector enough?: certified (Avalara/Vertex) vs fork vs build-from-template (TaxJar); live-marketplace check; per-engine dimension table | connector-selection.md |
Requirements → config mapping: tax mode, nexus, tax-code source, exemptions, scopes; the connect.yaml envelope; worked example | config-from-requirements.md |
| The two-app contract: the calculator (ExternalAmount, all four tax actions, 200-not-202, fail modes, call reduction) and the syncer (commit/void/refund lifecycle, idempotency); full pitfall catalog | tax-contract.md |
Avalara specifics (ground truth from the certified connector): exact connect.yaml keys, AvaTax createTransaction (quote vs commit), tax-code/entity-use mapping, MC config app, address validation — plus TaxJar as the build-from-template contrast | avalara.md |
| Verify the round trip: taxedPrice on the cart, transaction recorded; the sandbox-doesn't-persist and no-nexus-means-zero traps | verification.md |
| Build/publish/certify lifecycle, deploy, scopes, production-readiness gate (type-agnostic) | commercetools-connect |
avalara.md and extending the selection table — the two-app architecture, the contract, and the flow do not change.Checklist
Requirements
- Engine chosen + account/credentials; nexus regions known; region + project
- Calculation-only vs calculation+recording decided; order lifecycle (void/refund) decided
- Tax-code source (Product attribute / Tax Category / Custom Field) and exemption model identified
- Asked the open-ended "anything special?" question; each special requirement its own line
- Requirements block written and confirmed; specials fed into the Step 1.5 fit-check
Connector fit (decide before wiring/building)
- Checked live marketplace + tax docs (not memory); named the connector + version
- Ladder rung presented to the user and chosen by them: configure (1) · config-closes-gap (2) · fork/extend (3) · build from template (4)
- For a real gap on an engine with a public connector, chose fork over rebuild
Config (the deliverable)
- Tax mode chosen (
ExternalAmountunless a reason not to) with rationale - Only documented
connect.yamlenvelope fields; file at the repo root -
inheritAs.apiClient.scopes=manage_extensions,manage_subscriptions,view_orders(least-privilege) - Engine credentials in
securedConfiguration; region/toggles instandardConfiguration
The two apps (build test-first — do not write a function body before its red test)
- Calculator returns
200/201(never202); taxes line items and custom line items and shipping;changeTaxModeif it owns the mode - Extension trigger conditioned to reduce engine calls (mode set, address present, non-empty)
- Syncer re-fetches the Order by id; records idempotently on stable
transaction_id; commits/voids/refunds on the right states (if in scope) - Boundary mocked; suite runs with no deployment/secrets
Verification
-
taxedPricepresent on the cart after a cart update (extension registered + firing) - With a live account whose nexus covers the destination, a transaction is recorded
- Understood: sandbox may not persist transactions; zero tax at a no-nexus destination is correct, not a bug
The two-app tax contract
App 1 — the calculator (cart API Extension)
What triggers it
cart resource, actions: [Create, Update], registered by the app's postDeploy. External engines bill per call and rate-limit, so condition the trigger to fire only when the cart is worth taxing:{
"resourceTypeId": "cart",
"actions": ["Create", "Update"],
"condition": "taxMode = \"ExternalAmount\""
}
shippingAddress is defined and shippingInfo is defined and lineItems is not empty — a stronger gate that avoids calling the engine until the cart can actually be taxed. Match the gate to when a correct quote is even possible: tax can't be selected without a destination address.What it must return
ExternalAmount mode you must tax every priced element or the cart is inconsistent and — critically — the Order cannot be created:setLineItemTaxAmount— per line itemsetCustomLineItemTaxAmount— per custom line item (easy to forget; a cart with a custom line item fails without it)setShippingMethodTaxAmount— the shipping method (a cart with shipping fails Order creation without it — see the pitfall below)setCartTotalTax— the cart-level total grosschangeTaxMode→ExternalAmount— only if the connector owns the mode. The certified Avalara connector sets it itself; a from-template build often assumes the storefront already set it (and the trigger condition enforces it). Decide which, and be consistent.
externalTaxAmount carries totalGross (net + tax, in minor units) and a taxRate (name, amount as a 0–1 decimal, country, optional state, includedInPrice). Exact shapes: avalara.md.The response-status trap (this one silently breaks every cart)
200 or 201. Any other status — including 202 — is treated by commercetools as "failed to respond properly" and fails the triggering cart operation. The official template shipped a HTTP_STATUS_SUCCESS_ACCEPTED = 202 constant and returned it; on a from-template build, fix this first. A validation rejection uses 400 with { errors: [...] }; a successful no-op is 200 with {} or { actions: [] }.Latency and fail mode
- Keep the outbound engine call on a tight timeout under the extension budget (e.g. ~1.2 s), aborting rather than letting the platform time out.
- Decide fail-open vs fail-closed deliberately. Fail-closed (return
500/400, blocking the cart) guarantees no untaxed cart persists — the certified Avalara connector does this (400when misconfigured). Fail-open (return200with no actions) keeps checkout alive at the risk of a temporarily untaxed cart. State the choice in the README. - Reduce calls. Skip the engine when nothing tax-relevant changed — hash the tax-relevant cart fields (line items, quantities, address, shipping) and store the hash in a cart Custom Field; on the next call, if the hash matches and
taxedPriceis already set, return no actions. The certified Avalara connector does exactly this (hashCart→avalaraHashcustom field). It's the biggest single cost lever.
Keep the mapping pure and testable
[].App 2 — the order-syncer (OrderCreated Subscription)
What triggers it
event application, not a hand-wired Pub/Sub consumer: Connect provisions the queue/destination and delivers each message as an HTTP POST to the app's endpoint (port 8080). You register a Subscription on the order resource for OrderCreated (and, for a full integration, OrderStateChanged / OrderStateTransition / return-shipment messages) in the app's postDeploy — you don't manage the transport.- Transport wrapper (GCP): on a Google Cloud destination the payload arrives wrapped as
{ "message": { "data": "<base64>", ... } }—message.datais base64-encoded JSON, so decode it first. (Other destinations wrap differently; Connect abstracts which one.) - Message format: the decoded message is either PlatformFormat (
{ notificationType: "Message", type: "OrderCreated", resource: { typeId, id }, ... }) or CloudEventsFormat ({ specversion, type: "com.commercetools.order.message.OrderCreated", data: { ...same fields... } }), set when the Subscription is created. Readtypeandresource.idfrom whichever you get, and validate the message type before acting (ack-and-ignore the platform's test/probe messages).
OrderCreated.What it must do
- Re-fetch the Order by id from the message's
resource.id— don't trust the (possibly stale/omitted) payload. This is the required pattern for at-least-once delivery. - Record the transaction via the engine's record/commit API (Avalara
createTransactionwithcommit: true; TaxJarPOST /v2/transactions/orders). This is what appears in the engine's dashboard — the calculator's quote never does. - Be idempotent. Use a stable
transaction_id= the order id, so a redelivered message hits the engine's duplicate guard (TaxJar returns422; treat as already-recorded). Redelivery is guaranteed, not hypothetical. - Ack correctly. Reply
200for handled and irrelevant-but-acked messages — the Connect event contract expects a200(docs); a positive ack tells the platform "don't redeliver." Return non-2xx only for transient failures you want redelivered (Subscriptions retry unacked messages, at-least-once). Ack the platform's test/probe messages too.
Full lifecycle (if in scope)
OrderCreated:- Cancel → void the filed transaction (on the order states the merchant designates as cancellations).
- Return → refund (partial), on return-shipment state changes.
- Order edit → recalculate.
commitOrderStates, cancelOrderStates, activateReturns) stored in custom objects — not hardcoded state names. If the requirements include void/refund, model it the same way (configurable states), because state keys differ per project.Pitfall catalog
| Pitfall | Symptom | Fix |
|---|---|---|
Extension returns 202 | Every cart update fails | Return 200/201 only |
Shipping not taxed in ExternalAmount | Order creation fails: "shipping method is missing an external tax amount and rate" | Emit setShippingMethodTaxAmount |
| Custom line items not taxed | Order creation fails on carts with custom line items | Emit setCustomLineItemTaxAmount |
| Extension destination = base URL | Platform's calls 404 the app | Register destination as <CONNECT_SERVICE_URL>/taxCalculator |
postDeploy doesn't register the extension | Extension never fires; taxedPrice never appears | Wire connector:post-deploy, not just npm install |
| No trigger condition | Engine called on every cart keystroke; bill/limits blow up | Condition on mode + address + non-empty; hash to dedup |
| Syncer trusts the payload | Missing/stale order data → wrong or failed transaction | Re-fetch the Order by resource.id |
| Non-idempotent recording | Redelivery double-files a transaction | Stable transaction_id = order id; treat duplicate (422) as success |
| Config-validation throws a string status | Process crashes (ERR_HTTP_INVALID_STATUS_CODE) | Guard: only res.status() on integer codes |
| Engine requires a state (e.g. TaxJar transactions) | 406 to_state can't be blank | Ensure the destination address carries state; omit blank optional fields |
| Legacy SDK | Fails the commercetools-connect skill's pinned-version gate | @commercetools/platform-sdk@^8 + @commercetools/ts-client@^4 (both the template and the certified Avalara connector still ship the legacy sdk-client-v2 — upgrade anyway) |
Test-first checklist (mirror in the suite)
Calculator
- Emits all four action types; money minor↔major conversion correct
- Shipping and custom line items taxed
- Returns
200(asserted — the202regression is the one to pin) - No-op/short-circuit paths return
{ actions: [] } - Fail mode (open vs closed) asserted for an engine error
Syncer
- Decodes the base64 envelope; acks irrelevant messages
- Re-fetches the Order by id
- Records idempotently on a stable
transaction_id; duplicate treated as success - (If in scope) commit/void/refund keyed on configurable order states
Verify the tax round trip
Check 1 — the cart carries engine-computed tax
Drive a cart update (add a line item, set the shipping address) and inspect the cart:
taxedPriceis present. Before the API Extension is registered and firing,taxedPriceis simply absent — that's the tell that the extension isn't wired, not that tax is zero. After it fires,taxedPrice.totalNet/totalGross/totalTaxare populated.- The version jumped more than your update alone would explain. The extension's
setLineItemTaxAmount/setShippingMethodTaxAmount/setCartTotalTaxactions are extra writes — an add-line-item that lands the cart several versions higher is the extension firing. - Shipping and custom line items are taxed, not just line items — otherwise Order creation will later fail in
ExternalAmountmode.
ExternalAmount mode, set a destination address in a nexus region, add a priced line item, and read back taxedPrice. (Same flow a storefront BFF would run.)Check 2 — the order is recorded as a transaction
OrderCreated subscription deliver (or, locally without Pub/Sub, POST the base64 OrderCreated envelope to the syncer directly), then:- The syncer returns a positive ack (
204/200). - The engine's API confirms the transaction (by
transaction_id= order id). - It appears in the engine's dashboard — the calculator's quote never does; only this recording step surfaces there.
The two traps (correct behavior that looks like a bug)
Trap 1 — the sandbox doesn't persist transactions
POST .../transactions (returning 201) but don't store the record: a subsequent GET returns canned demo data, and nothing shows in the sandbox dashboard. TaxJar's sandbox behaves exactly this way. So an empty Transactions tab after a successful sync is expected on sandbox, not a failure.- Switch the engine base URL / sandbox flag to live and supply the live token.
- Use a destination in a region the live account has nexus in.
- Treat these as real records — delete the test transactions afterward (engines expose a delete-transaction API) so they don't pollute filing/reporting.
Trap 2 — no nexus means zero tax (correctly)
amount_to_collect: 0, taxedPrice.totalTax: 0. This is not a wiring bug; it's the engine doing its job. Before concluding "tax isn't calculating," confirm the destination is a nexus region (check the engine account's nexus settings — e.g. TaxJar GET /v2/nexus/regions). A cart shipping to a nexus region should return non-zero tax; one shipping elsewhere should return zero.Verification checklist
-
taxedPricepresent on the cart after a cart update (extension registered + firing) - Line items and custom line items and shipping all carry tax (Order creation succeeds)
- Order placed → syncer acks → transaction confirmed via the engine API
- Transaction visible in the dashboard on a live account (sandbox may not persist)
- Non-zero tax at a nexus destination; zero at a non-nexus destination (both correct)
- Live test transactions cleaned up afterward