This is Part 6 of our Open Source Creator Series on “How to Measure Community Building" to help you – the open source technology creators – understand and bootstrap some of the essential non-technical elements of building a successful project.

(See Part 1 – Licensing Fundamentals; Part 2 – Product Marketing; Part 3 - How to Choose an Open Source License; Part 4 -- Documentation and Technical Writing 101; Part 5 -- Understanding 'Distribution' in Open Source Licensing)

Community building is table stakes in the success of any open source project.

Even outside of the open source context, community is considered a competitive advantage and moat in many industries, from retail to gaming to fitness. (For a deeper dive, see “When Community Becomes Your Competitive Advantage” in the Harvard Business Review.)

However, open source community building, especially offline activities, is notoriously hard to measure, track, and analyze. While we’ve all been to our fair share of meetups, conferences, and “summits” (and probably hosted a few of them ourselves), were they worth it? Did the community meaningfully grow? Was printing all those stickers and swags worth the money? Did we collect and track the right numbers to measure progress?
Conference stickers

To develop a better framework for measuring community, we can look to a different industry for guidance and fresh ideas: political campaigns.

Some Personal Context

I started my career in political campaigns in the U.S., as a field organizer for then candidate Senator Obama in 2008. Thinking back, a field organizer’s job was basically community building. My day consisted of calling supporters to do volunteer activities, hosting events to drum up more support, bringing in guest speakers (called “surrogates” in politics) to these events, and selling the vision and plan of our candidate (essentially our “product”).

Another big chunk of my day was data entry. We logged everything: interactions on my phone conversations, contact rate, event attendance, volunteer recruitment rate, volunteer show up rate, and a myriad of other numbers to constantly measure the effectiveness of what we do everyday.

Regardless of your misgivings about politics in general or specific politicians, the winning campaigns that produce political victories are all giant community building exercises that are data driven, meticulously measured, and constantly optimized. They are well-oiled community building machines.

When I entered the world of open source a few years ago, the community building part felt familiar and natural. What surprised me was how little community building as an operation, especially the offline activities, is quantified and measured.

Three Metrics to Track

Taking a page from the best-run political campaigns I’ve seen, here are the three most important metrics to track and optimize for an open source community:

• # of community ambassadors
• # of return attendees (defined as: people who attended your activities two or more times)
• rate of churned attendees (defined as: percentage of people who attended your activities only once, or said they would come but didn’t show up)

Community Ambassadors

The function of a “community ambassador” is a user or enthusiast of your project, who is willing to consistently host local meetups or activities where she or he lives. Growing the number of community ambassadors and supporting them with resources and guidance is core to your community’s strength and scale. You can probably hire for these if you have a lot of funding, but pure volunteers speak more to the allure of your project.

These ambassadors should be your best friends, where you understand inside and out why they are motivated to evangelize your project in front of both peers and strangers. Their feedback on your project is also valuable and should be a critical part of your roadmap development process. You can strategically cultivate ambassadors in different tech hubs geographically around the world, so your project can count on someone with local knowledge to reach and serve users of different business cultures with different needs. The beauty of open source is that it’s global by default; take advantage of it!

Some developer hubs to consider: SF/Bay Area, New York City, Seattle, Vancouver, Toronto, Berlin, Beijing, Hangzhou, Shenzhen (especially hardware), Bangalore, Seoul, Singapore, Amsterdam. (Please ping me if I missed any!)

There’s an adage in political campaigns: “meet voters where they are.” The same concept can be re-appropriated to open source communities: “meet developers where they are.”

An example of this is the Cloud Native Ambassadors program of the Cloud Native Computing Foundation. (Note: not every ambassador on this page is consistently active; your community can do better!)

Return Attendees

The number of return attendees is crucial to measuring the usefulness or stickiness of your community activities. Tracking return attendees is how you can draw a meaningful line between “the curious” and “the serious.”

Trying to grow this number should be an obvious goal. However, that’s not the only goal. This is the group whose motivation you want to understand the clearest. This is the group who reflects your project’s user persona. This is the group that can probably give you the most valuable feedback. This is the group who will become your future community ambassadors.

Putting it differently: this is your 1,000 true fans (if you can keep them).

Having hosted and attended my fair share of community meetups, my observation is that most people participate to learn about a technical topic, search for tools to solve problems at work, or network for their next job opportunity. What they are not looking for is “being marketed to”.

There is a growing trend of developer community events becoming marketing events, especially when participants are from companies flushed with funding or have a strong marketing department who wants to “control messaging”. I personally find this trend troubling, because it undermines the core purpose of any open source community: learning.

Thus, be laser-focused on “technical education”. If a developer community gets taken over by marketing campaigns, your “return attendees” metric won’t be pretty.

Churned Attendees Rate

Tracking “churned attendees” is the flip side of the same coin as “returned attendees”, so I won’t belabor the point.

One note of caution: be brutally honest when measuring this number and don’t fool yourself (or others). If someone signed up but didn’t show up, it doesn’t mean much. If someone showed up once and never came back, it doesn’t mean much. Routinely sit down and assess why someone isn’t showing up to re-evaluate and refine your community program and activities.

Don’t build the wrong incentives into your community building operation to reward the wrong metric.

Of course, there are other things you can track, but more data does not necessarily mean clearer insight. Focusing your energy on these three metrics will make the most impact on your community building operation. An open source community, where the number of ambassadors and return attendees are trending up, and churned attendees rate is trending down, is one that’s healthy and growing in the right way.

(For the politically curious, the corresponding terms on a political campaign for these three metrics are typically: neighborhood captains, super volunteers, and flake rate.)

Value of In-Person Connection

I purposely focused this post on measuring offline community activities, not online, because online activities are inherently more trackable and intuitive to digital-native open source creators.

Offline community activities are essential to any project’s journey to reaching traction and prominence. I have yet to see a successful project that does not have a sizable offline presence, regardless of its online popularity.

Why is this the case? Why can’t an open source community, usually born online, just stay and grow online?

Because technology choice is ultimately a human decision, thus face-to-face interaction is an irreplaceable element of new technology adoption. No one wants to be the guinea pig. No one wants to be the “first”. The most effective way to not feel like the “first” is to literally see other human beings trying out or learning about the same thing.

Being in the same room as other developers, doing exercises on the same technology, and doing that regularly is the most effective way to build trust towards that project.

And with trust comes traction.