A complete guide to instrumenting Next.JS with OpenTelemetry and Pino Logger

Have you ever wondered if you could use the same structured log or tracing formats when switching between frameworks or different languages?

OpenTelemetry(OTel) achieves this by standardizing the logs, traces, and metrics signals. Collecting logs, traces, and metrics is called instrumentation of applications, and it improves their observability. OTel provides its own SDK to implement traces and metrics for most languages and instrumentation libraries to integrate into third-party logging libraries.

Without further ado, let’s get into how you can incorporate this amazing technology into your web application today.

Web application

As the title shows, this tutorial is focused on integrating traces and logs of OpenTelemetry with a NextJS web application. The NextJS application is set up with the app router, and it uses fetch API to make network requests to a Fastify API. The Fastify API fetches data from a database and returns it to the front end. I use Typescript in this document, but nothing changes for JavaScript besides file extensions.

Three-tier web application architecture

We use the Pino logger in both of these applications. It is lightweight, efficient, and integrated into many libraries and frameworks, including Fastify.

NextJS official documentation

Instrumentation of a NextJS application is done by enabling the experimental instrumentation hook in the NextJS config file and adding a new file called instrumentaion.ts to the root of your project.

NextJS documentation for setting up OTel can be found in the following links.

Overview of Instrumentation: https://nextjs.org/docs/app/building-your-application/optimizing/instrumentation

More info on `instrumentaion.ts`: https://nextjs.org/docs/app/api-reference/file-conventions/instrumentation

OTel integration: https://nextjs.org/docs/app/building-your-application/optimizing/instrumentation

Then, why am I writing this guide?

While the above documents provide a starting point for trace instrumentation with OTel, they need to provide in-depth insight into the configuration or any information on log instrumentation or context propagation.

I ran into multiple problems while instrumenting the NextJS application due to weird issues like import order, and I spent hours researching solutions. Also, before the official OTel Node SDK was released, there were many third-party instrumentation libraries, and I was confused by the sheer number of options available.

I’m writing this guide so that you do not have to waste hours researching solutions to the same issues I did.

So, let’s start from the top!

1. Enable the instrumentation hook in the NextJS config file.

First, you must enable instrumentation in the framework by adding the following configuration to your next.config.ts file.

module.exports = {
  experimental: {
    instrumentationHook: true,
  },
}

This tells NextJS to look for the instrumentaion file in the root directory of your project when starting the server.

The next step is to set up the instrumentaion.ts file in the root or src directory, depending on your project initialization options. This file must be in the same directory level as the app or the pages directory.

This file must contain an export name register that the NextJS framework will call. You can implement theregister function using @vercel/otel or manually.

2a. Set up instrumentaion with @vercel/otel — Option 1

This approach uses the @vercel/otel package that includes a few basic OTel configurations. It will do the instrumentation with little effort.

First, install the required packages.

npm install --save @vercel/otel @opentelemetry/sdk-logs @opentelemetry/api-logs @opentelemetry/instrumentation

Then, create the instrumenaion.ts file in the root directory and add the following. You can add a custom trace exporter to support your service provider, but it’s optional.

import { registerOTel } from '@vercel/otel';
 
import { registerOTel } from '@vercel/otel';
import { MyCustomExporter } from './my-custom-exporter';
 
export function register() {
  registerOTel({
    serviceName: 'your-project-name',
    traceExporter: new MyCustomExporter(),
  });
};

@vercel/otel is a good option for adding only trace signals to your NextJS instrumentation. It supports context propagation, custom samplers, and trace exporters, among other things. You can file all customization options here. @vercel/otel library works in both edge and node NextJSserver environments.

The issue with the above library is that it does not allow the addition of more instrumentation at this point. For example, if you need to instrument your existing third-party logger and add traceIdand spanId you must do it with a different approach.

The second approach is more of a manual configuration

2b. Set up instrumentaion with @opentelemetry/sdk-node — Option 2

If you want complete control over OTel configuration, you can use the node SDK to set up instrumentation.

Important: However, at this point, node SDK is not supported by the NextJS edge server runtime environment, meaning if you want traces in middleware, you have to go with the first option of using @vercel/otel. You can read more about these NextJS server runtimes here.

NextJS will call the register function inside the instrumentation file in both node and edge server runtimes. As mentioned before, you must import it conditionally because the edge runtime does not support node SDK.

First, install all dependencies.

npm install @opentelemetry/sdk-node @opentelemetry/resources @opentelemetry/semantic-conventions @opentelemetry/sdk-trace-node @opentelemetry/exporter-trace-otlp-http

Then create the instrumentaion.ts file.

// instrumentation.ts
export async function register() {
  if (process.env.NEXT_RUNTIME === 'nodejs') {
    await import('./instrumentation.node.ts')
  }
}

Initialize the OTel node SDK in the instrumentaion.node.tsfile as follows.

// instrumentaion.node.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { SEMRESATTRS_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-node';
 
const sdk = new NodeSDK({
  resource: new Resource({
    [SEMRESATTRS_SERVICE_NAME]: 'your-project-name',
  }),
  spanProcessor: new SimpleSpanProcessor(new OTLPTraceExporter({
      url: '<trace collector url>'
  })),
});

process.on('SIGTERM', () =>
    sdk
        .shutdown()
        .then(
            () => console.log('OTEL SDK shut down successfully'),
            (err) => console.log('Error shutting down OTEL SDK', err)
        )
        .finally(() => process.exit(0))
);

sdk.start();

How do we instrument the logger and the fetch API?

Now, we come to the good part. You can instrument your libraries and inject trace information into them in two ways.

The first option is to use the node auto instrumentation library, which will instrument all available libraries, including the logger and the fetch API. However, this comes at the price of a larger build size. The other way is to selectively add them to the instrumentation array in the Node SDK initializer in the instrumentaion.node.ts file.

First, install the required instrumentation libraries. NextJS uses a modified version of the Undici fetch API to make requests to backend services.

npm install --save @opentelemetry/instrumentation-undici @opentelemetry/instrumentation-pino

Now, add these instrumentations to the instrumentation.node.ts file.

// instrumentaion.node.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { SEMRESATTRS_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-node';
 
const sdk = new NodeSDK({
  resource: new Resource({
    [SEMRESATTRS_SERVICE_NAME]: 'your-project-name',
  }),
  spanProcessor: new SimpleSpanProcessor(new OTLPTraceExporter({
      url: '<trace collector url>'
  })),
  instrumentations: [
       new PinoInstrumentation({
          disableLogSending: true
          // Pino instrumentation options.
      }),
      new UndiciInstrumentation({
      // Undici fetch instrumentation options.
      })
  ]
});

process.on('SIGTERM', () =>
    sdk
        .shutdown()
        .then(
            () => console.log('OTEL SDK shut down successfully'),
            (err) => console.log('Error shutting down OTEL SDK', err)
        )
        .finally(() => process.exit(0))
);

sdk.start();

Adding the above instrumentation libraries will enable context propagation via the fetch API and inject trace information into the Pino logs. You can inject additional trace information into the request in the Fetch API or customize trace information log keys in Pino in the options of each instrumentation constructor.

How do we remove health checks from trace information?

Suppose you are deploying the NextJS application as a Docker container. In that case, you will probably have a health check route set up, which the Docker orchestration service of your choice will use to determine whether the application is healthy.

You should probably remove this from traces as this will create a large volume of traces depending on the frequency of health checks. Otherwise, they will clutter your traces and increase the instrumentation cost.

You can do this by implementing a custom sampler. Samplers are usually used to record a portion of your traffic in traces, as recording all of it is not required to make informed decisions or debugging in most cases.

You can add it to the node SDK as follows.

// instrumentaion.node.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { SEMRESATTRS_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-node';

class NextJSSampler implements Sampler {
    shouldSample(
        _context: Context,
        _traceId: string,
        _spanName: string,
        _spanKind: SpanKind,
        attributes: Attributes
    ): SamplingResult {
        return {
            decision:
                // change the health check route accordingly
                attributes['next.route'] === '/health'
                    ? SamplingDecision.NOT_RECORD
                    : SamplingDecision.RECORD_AND_SAMPLED,
        };
    }

    toString(): string {
        return 'Next JS Sampler - Exclude Health Route';
    }
}
 
const sdk = new NodeSDK({
  resource: new Resource({
    [SEMRESATTRS_SERVICE_NAME]: 'your-project-name',
  }),
  spanProcessor: new SimpleSpanProcessor(new OTLPTraceExporter({
      url: '<trace collector url>'
  })),
  instrumentations: [
       new PinoInstrumentation({
          disableLogSending: true
      // Pino instrumentation options.
      }),
      new UndiciInstrumentation({
      // Undici fetch instrumentation options.
      })
  ],
  sampler: new NextJSSampler()
});

sdk.start();

Debugging

Trace information is not present in Pino logs.

This issue can occur because of two main reasons.

1. You have a Pino import before the OTel Node SDK is initialized.

For library instrumentations to work, those libraries must not be imported into any module before the node SDK is initialized.

For example, if you use the Pino logger in the instrumentaion.ts or instrumentaion.node.ts file, the Pino instrumentation will not work, and you will not see the trace information in the logs anywhere, as the Pino module will be loaded into the Node.js module cache before the OTel SDK initialization.

2. You have used an incorrect export from the Pino library.

The Pino class has multiple exports in the Pino library. The Pino instrumentation only instruments the entire module export.

// intstrumented
import pino = require('pino');
const logger = (pino as any)();

// not instrumented
//import { pino } from 'pino';
//const logger = pino();

// instrumented only if "esModuleInterop": true is in tsconfig.js
import pino from 'pino';
const logger = pino();

No traces are exported at all.

If you do not see any traces in your logs service provider, try exporting traces to the console to see whether they are recorded. You can replace OTLPTraceExporter with inbuilt ConsoleSpanExporter to achieve it.

const sdk = new NodeSDK({
  resource: new Resource({
    [SEMRESATTRS_SERVICE_NAME]: 'your-project-name',
  }),
  spanProcessor: new SimpleSpanProcessor(new ConsoleSpanExporter()),
  instrumentations: [
       new PinoInstrumentation({
          disableLogSending: true
      // Pino instrumentation options.
      }),
      new UndiciInstrumentation({
      // Undici fetch instrumentation options.
      })
  ],
  sampler: new NextJSSampler()
});

If all traces are recorded as expected, ensure you have provided the required credentials for the OTLPTraceExporter configuration. For brevity, I did not include them in the above code segments.

const sdk = new NodeSDK({
  resource: new Resource({
    [SEMRESATTRS_SERVICE_NAME]: 'your-project-name',
  }),
  spanProcessor: new SimpleSpanProcessor(new OTLPTraceExporter({
      url: '<trace collector url>',
      headers: {
          Authorization: 'YOUR_API_KEY_HERE'
      }
  })),
  instrumentations: [
       new PinoInstrumentation({
          disableLogSending: true
      // Pino instrumentation options.
      }),
      new UndiciInstrumentation({
      // Undici fetch instrumentation options.
      })
  ],
  sampler: new NextJSSampler()
});

Enable OTel diagnostic logs.

OTel Node SDK diagnostic logs can help you identify issues if the instrumentation fails.

You can enable diagnostic logs without any code changes by setting the OTEL_LOG_LEVEL=debug environment variable or adding the logger to the instrumentation file.

// instrumentaion.node.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { SEMRESATTRS_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-node';

// tracing.ts or main index.ts
import { diag, DiagConsoleLogger, DiagLogLevel } from '@opentelemetry/api';

if(process.env.ENABLE_OTEL_LOGS === "true") diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

class NextJSSampler implements Sampler {
    shouldSample(
        _context: Context,
        _traceId: string,
        _spanName: string,
        _spanKind: SpanKind,
        attributes: Attributes
    ): SamplingResult {
        return {
            decision:
                // change the health check route accordingly
                attributes['next.route'] === '/health'
                    ? SamplingDecision.NOT_RECORD
                    : SamplingDecision.RECORD_AND_SAMPLED,
        };
    }

    toString(): string {
        return 'Next JS Sampler - Exclude Health Route';
    }
}
 
const sdk = new NodeSDK({
  resource: new Resource({
    [SEMRESATTRS_SERVICE_NAME]: 'your-project-name',
  }),
  spanProcessor: new SimpleSpanProcessor(new OTLPTraceExporter({
      url: '<trace collector url>'
  })),
  instrumentations: [
       new PinoInstrumentation({
          disableLogSending: true
      // Pino instrumentation options.
      }),
      new UndiciInstrumentation({
      // Undici fetch instrumentation options.
      })
  ],
  sampler: new NextJSSampler()
});

sdk.start();

Summary

In this article, we discussed how you can instrument your NextJS application with OpenTelemetry and Pino logger. We enabled the instrumentation hook in the Next config file and proceeded to set up instrumentaion and instrumentaion.node . Then, we debugged the setup and explored the most common issue. I hope this guide was helpful to you!