Charles Humble for New Relic

While New Relic has been involved in various open-source projects for some time, these efforts have received increased focus in recent months, as the company has made a concerted push into open source and made the New Relic One platform programmable. The recently launched is intended to act as a hub for developers to discover, use, and contribute to the open-source projects maintained and supported by the vendor. Areas the projects cover include instrumentation, third-party integrations, custom apps, tooling templates and example code.

Among the approximately 180 projects, you’ll find the source code for a large number of the Nerdpacks available through the application catalogue on New Relic One. These provide working, usable examples of the programmatic capabilities of the platform. Nerdpacks are Webpack applications that use GraphQL to interact with the data available in New Relic, and React components for visualization and other UI. These are the same tools that New Relic’s developers use to build the platform itself.

Examples of Nerdpacks include Cloud Optimize, which evaluates the size and utilization of your AWS, GCP and Azure cloud instances and shows the amount of money you could save by right-sizing; Container Explorer, which gives you a global view of all containers associated with an account; and (Top)[https://opensource.newrelic.com/projects/newrelic/nr1-top], a simple process monitor inspired by the Unix top command:

The source for the New Relic open-source website itself, built using Gatsby, is also available.

In addition, you can find a number of libraries, including GraphQL Java Core, which provides helpers for integrating (graphql-java)[https://github.com/graphql-java/graphql-java] with a JVM application, simplifying input mapping and schema configuration. Ben Evans, Principal Engineer & JVM Technologies Architect at New Relic, explained that

We use GraphQL as a key "glue" technology - where the schema acts as an interface definition and a contract between the back end and front end (and because we have programmability capability customers can use the same contract to build their own rich front ends if they want).

This means we quite naturally fall into the "schema first" view of the world - where we want to enable teams to write the schema and then auto-generate based upon that schema, rather than adding Java annotations to pre-existing services... Of course, the schema-first approach may not work for everyone - but it works for us so we thought we'd share those pieces that enable it.

There are three major community-led projects that New Relic doesn’t host, but to which it contributes. The first is the OpenTelemetry standard, currently incubating with the CNCF. This project, formed through a merger of OpenTracing and OpenCensus, is a multi-vendor effort that provides a single set of APIs, libraries, agents, and collector services to capture distributed traces and metrics from your application. New Relic contributes by defining specifications, implementing parts of the specification in language SDKs, reviewing pull requests from other contributors, and serving as a maintainer in the various OpenTelemetry sub projects. The company has two full-time engineers working on the project - John Watson is a Java maintainer, and Tyler Yahn is a Go maintainer. John is also highly active on the project’s Gitter.

Secondly, New Relic is a participant in the W3C Distributed Tracing Working Group. The W3C project is closely linked to OpenTelemetry, which supports the emerging W3C standard as one of the available options for context propagation. As Alois Reitbauer has written in an explainer about the W3C Trace Context sub-project

The W3C Trace Context specification is a set of new standards being developed by open source and commercial tool providers that defines a unified approach to context and event correlation within distributed systems, such as microservices environments. Having such a standard will enable end-to-end transaction tracing within distributed applications across a range of monitoring tools.

The third community effort is AdoptOpenJDK (soon to be moving to the Eclipse Foundation and renamed Eclipse Adoptium). AdoptOpenJDK started in 2017 as an advocacy and on-boarding program for OpenJDK (the OSS project behind Java itself), but has since shifted its efforts to providing a long-term, stable build platform for OpenJDK. New Relic is a Silver Sponsor, with a pool of around five engineers involved in the project. New Relic's involvement came about, Evans explained, after Oracle changed the terms under which they distribute their Java binaries.

We decided that we would adopt the same upgrade cadence as we see our customers using and that meant sticking with LTS Java releases. In turn, that meant choosing a vendor to be our supplier for open-source Java binaries.

AdoptOpenJDK offers free, high quality binaries - and they are focused on the binaries, not on making source code contributions. After Oracle, AdoptOpenJDK is the stand-out distribution that we see our customers using, so it was an easy choice to make them our supplier as well.

The decision to get involved on a technical level came about due to a confluence of: a) New Relic engineers who wanted to spend more time working on open-source software and b) Technical efforts within AdoptOpenJDK that are relevant to New Relic. Our services and backend is pretty much all Java and Kotlin so we have a key strategic interest in the success of the project. So at one level it's no more complicated than the familiar: "Contribute to what matters to you" mantra of open source.

In terms of what New Relic contributes to the project Evans told me that

As New Relic is an Observability company, it's probably no surprise that we're interested in anything that intersects our business. So, we're interested in things like Java Flight Recorder (JFR) and innovations in serviceability. We are also very interested in performance improvements and avoiding regressions - both for customers and for our own high-performance services.

For example, when we transitioned to using AdoptOpenJDK last year, we had a programme of chasing down and fully investigating every performance regression that was reported when moving from Oracle to OpenJDK. From our knowledge of how the OpenJDK project works we were basically sure that any regressions should be false positives, but our approach was very much "Trust But Verify". As you might hope, we were able to prove that all the reports were false positives and there were no real regressions caused by the upgrade to OpenJDK.

It's not a very glamorous example, but sometimes making more perfect software requires detailed, meticulous work. Developers and teams need the confidence when upgrading that their systems won't break or regress in subtle ways. We're happy to do our bit to help provide that confidence for our customers, ourselves, and the whole community.