Observability

What is Application Performance Monitoring?

Using an Application Performance Monitoring (APM) service provides you with insights into how well your web application performs. An APM can show which parts of the application are running within their bounds and expectations, as well as which parts may need to be optimized. This allows you to proactively make changes to your web application if performance degrades.

How can SDKs be monitored?

commercetools SDKs use a middleware approach for creating an API client. You can stack middleware together to modify the request or response based on your requirements.

This approach makes it possible to collect metrics or details about each request as the middleware gives a entry point before the request is sent and right after the response is returned.

Monitor using New Relic

New Relic is a APM which supports different languages, frameworks, and platforms to retrieve metrics and traces.

The following guide will instruct you how to integrate New Relic with commercetools SDKs.

Java

Prerequisites

• Version 13.1 (or later) of the Java SDK.
• New Relic Java agent configured

Include the New Relic middleware in the SDK Client

The Java SDK supports New Relic with the commercetools-monitoring-newrelic module.

Once commercetools-monitoring-newrelic is added as a dependency to your application, you must add the New Relic middleware to the SDK client using the .withTelemetryMiddleware() method.

ApiHttpClient apiHttpClient = ApiRootBuilder
  .of()
  .defaultClient(ServiceRegion.GCP_EUROPE_WEST1.getApiUrl())
  .withTelemetryMiddleware(new NewRelicTelemetryMiddleware())
  .buildClient();

The NewRelicTelemetryMiddleware middleware will read a NewRelicContext from the request, restore the transaction of the context, and report the details of the request and response to the NewRelic agent as an external call.

For being able to create a trace per web request for example from a Spring boot application, you must create a client with the NewRelic transaction. This can be done using the ContextClient of the SDK.

ContextApiHttpClient contextClient = ContextApiHttpClient.of(
  apiHttpClient,
  NewRelicContext.of(NewRelic.getAgent().getTransaction()),
  false // don't close the ApiHttpClient
);
ProjectApiRoot apiRoot = ProjectApiRoot.fromClient(projectKey, contextClient);

The ContextClient adds the NewRelicContext object to every request made to the API to ensure that even asynchronous calls in different threads are instrumented.

As it is a best practice that the ApiClient is reused across the application lifetime, you should configure the ContextClient to not close the inner client when itself gets closed.

PHP

Prerequisites

• PHP 8.1 (or later)
• NewRelic Agent & Daemon configured as per New Relic documentation

Include the New Relic monitoring in the SDK

The New Relic PHP Agent supports the Guzzle HTTP client, so each API call can be monitored by New Relic without extra configuration in the SDK Client.

To see how to include New Relic in the PHP SDK, you can view an example application that uses a Docker environment.

TypeScript

Prerequisites

• Version 2.3.0 (or later) of the TypeScript SDK
• Version 1.0.0 (or later) of the @commercetools/ts-sdk-apm
• New Relic Node.js agent configured

The TypeScript SDK uses @commercetools/ts-sdk-apm and the middleware builder method withTelemetryMiddleware() to add the New Relic APM and openTelemetry agent to collect and upload metric and trace data to New Relic.

Include the monitoring package in your SDK client

The following code example demonstrates how set up your SDK client for using New Relic monitoring.

// Import the @commercetools/ts-sdk-apm package
import { createTelemetryMiddleware } from '@commercetools/ts-sdk-apm'
import { ClientBuilder } from '@commercetools/sdk-client-v2'

// Configure the telemetry options
const telemetryOptions = {
  createTelemetryMiddleware
}

// Create the client with the withTelemetryMiddleware() middleware
const client = new ClientBuilder()
  .withClientCredentialsFlow(...)
  .withHttpMiddleware(...)
  .withTelemetryMiddleware(telemetryOptions) // telemetry middleware
  ...
  .build()

Add the New Relic agent configuration

Once the New Relic agent is installed, create a file named newrelic.js in your project root and copy the following code into it. You should modify app_name and license_key to match your New Relic profile.

('use strict');
/**
 * New Relic agent configuration.
 *
 * See lib/config/default.js in the agent distribution for a more complete
 * description of configuration variables and their potential values.
 */
exports.config = {
  /**
   * Array of application names.
   */
  app_name: [process.env.NEW_RELIC_APP_NAME],
  /**
   * Your New Relic license key.
   */
  license_key: process.env.NEW_RELIC_LICENSE_KEY,
  logging: {
    /**
     * Level at which to log. 'trace' is most useful to New Relic when diagnosing
     * issues with the agent, 'info' and higher will impose the least overhead on
     * production applications.
     */
    level: 'info',
  },
  /**
   * When true, all request headers except for those listed in attributes.exclude
   * will be captured for all traces, unless otherwise specified in a destination's
   * attributes include/exclude lists.
   */
  allow_all_headers: true,
  attributes: {
    /**
     * Prefix of attributes to exclude from all destinations. Allows * as wildcard
     * at end.
     *
     * NOTE: If excluding headers, they must be in camelCase form to be filtered.
     *
     * @env NEW_RELIC_ATTRIBUTES_EXCLUDE
     */
    exclude: [
      'request.headers.cookie',
      'request.headers.authorization',
      'request.headers.proxyAuthorization',
      'request.headers.setCookie*',
      'request.headers.x*',
      'response.headers.cookie',
      'response.headers.authorization',
      'response.headers.proxyAuthorization',
      'response.headers.setCookie*',
      'response.headers.x*',
    ],
  },
};

You should require the configuration file during application launch either by adding it at the first line of the application entry module/file or as a command-line argument. Both approaches produce the same result.

// app.js
require('./path/to/newrelic.js') // require the nodejs New relic agent configuration here
import express from 'express'
...
const app = express()

app.listen('8000', function() {
  console.log(`server listening on port ${8000}`)
})

{
  "script": {
    "start": "node -r ./path/to/newrelic.js app.js"
  }
}

An example of the command-line configuration can be found in the newrelic-express-apm example.

Monitoring your SDK

You can now send requests in your application to commercetools Composable Commerce through your SDK and monitor the reported data dashboard, telemetry, and performance statistics in New Relic.

The Service map shows all the service(s) (both internal and external) the request calls throughout its lifecycle.

Every request to the API will now be shown as a call to an external service or as a segment in the distributed traces view of your application.

The Distributed tracing tab shows the statistics of the requests traced:

Opening a traced request displays more details of the request itself and the called external services (like commercetools):

In the commercetools segment the attributes display the request details, including the called URI and the response status code and timings.