Measuring Community

Community is a buzzword. While there is a ton of interest by companies to build communities around their products and services, the reality is that these efforts usually amount to nothing more than pet projects. Once cash gets tight and budgets get slashed, community is often first on the chopping block.

I have been thinking about this issue deeply for the past four years. This is why I have been heads down working on the second edition of my book Community-in-a-Box. Having dedicated a significant portion of my career on building communities, I feel many companies have missed out on the opportunity to properly tap into the power of community.

The hardest part for companies though is the question of how to connect community work to business results. It has been the missing link to take community from a mere toy in the corporate toolbox to a key pillar for corporate innovation and growth.  This is where most of my time has been spent in putting together the second edition; creating the bridge between community activity and business goals that can lead to better, more data-driven community management.

So, enjoy this excerpt from the updated book that covers what exactly is entailed in creating that data-driven bridge. And if you have thoughts on this topic, please do share!

Many organizations view community with both interest and skepticism. They understand the benefits of communities and appreciate the goodwill a strong community can provide. They recognize that community can be a significant contributor to the business. At the same time, they question whether community truly has measurable business impact.

Is it so important to measure business impact? As Peter Drucker, the famous management guru, once said, “You can’t manage what you don’t measure.” Regardless of the type of community, your goal as a leader should be to continuously improve the quality and impact of the community for its members, something that I discussed in the Measuring Impact chapter.

Demonstrating impact is even more important for organization-aligned communities. The teams supporting communities are fighting for company resources with every other corporate department. There are competing business interests, corporate politics, and annual budget wrangling. Deciding what to prioritize and fund often comes down to which departments and teams can prove their impact. Teams that can present their case in a clear, data-supported way that aligns with quantifiable business goals and KPIs often get the most funding and support.

Whenever the economy hits a downturn, the first thing companies do is cut costs. In determining which teams and projects get cut, those who have not done the work to prove their contribution to business impact are the first to get the axe. This has been a recurring theme for community teams that grow in size and scope during the good years and get decimated in the lean years. The five years from 2020 to 2025 were a particularly vicious boom-bust cycle where community was one of the hottest business trends, only to see entire teams gutted when the economy softened. If they had shown quantifiable business impact, they might have been able to justify their existence and avoid getting downsized. The problem they face is trying to determine how to collect the data to show business impact.

The question of how to measure community comes up regularly. It is one of the most discussed and debated topics in community management forums. Even organizations like Amazon, which is very data-driven, struggle to make sense of what and how to measure. Taking the next step in figuring out how community activities create business value with specific data points is even more daunting. When CFOs and corporate executives are looking at annual budget requests and determining what to keep and cut, this is the information they are seeking to justify their decisions.

Speaking of Amazon, one of the most fundamental lessons I learned while there was the concept of “Working Backwards.” It is a simple framework for serving customer needs and developing products. Instead of building a product and finding customers who need the product, working backward encourages entrepreneurs to start by understanding the customer's problem first and then creating a solution based on that knowledge.

This concept can also be useful for mapping the business value of community to community-based initiatives. For example, an organization may want to know the ROI of a community program. The mistake most community teams make is that they start with basic community-specific metrics like the number of members or attendees at events and then make this magical leap to ROI. There is no direct correlation between the two, so the argument supporting community is not convincing.

Having worked for several enterprise technology providers, I can confidently state that pinning down ROI is rarely a straightforward process. The way we would do this is to “back into the number.” That means we would start with some key business metrics that a company needs to improve, then tie that back to benefits and features in our product that could deliver a measurable improvement.

Knowing the end goal makes it easier to tie the community to business objectives. It is usually not a straight path, though. We often need “bridge metrics” to form connections that logically tie the upstream effects of community activity to goals. The framework used to build this bridge and establish correlation is the Three Levels of Community Metrics:

• Level 1 (”C1”) – Metrics tying community output to relevant business goals, typically at an OKR (objectives & key results) level,

• Level 2 (”C2”) – Metrics derived from community activity that establish the “bridge” between measurable actions and goals,

• Level 3 (”C3”) – Metrics directly related to the health of the community, in other words, the Community Pulse Metrics.

To understand how to use this framework, let’s review a few examples of how to understand how capturing C1, C2 & C3 metrics can help a technology company define the value it receives from its developer community.

• Revenue Example – The company’s developer community hosts a lot of user group meetups around the globe. They see the user groups as a source of high-quality leads. Still, they want to know specifically the potential of the user groups to generate additional revenue beyond pure marketing and sales. They have been tracking the number of attendees and noticed when examining the level of engagement that there was an uptick in repeat attendees (C3). A certain percentage of these repeat attendees have scanned a QR code to sign up for their developer newsletter (C2). Over time, some of those QR code scans, tied to a member of the community, convert into paying users (C1).

• Cost Example – The company wanted to ease the burden placed on its support team to answer the high volume of customer issues. They set up an online community forum to enable users to post questions and other users to provide answers. They track the number and percentage of questions answered by forum members (C3). Through web tracking, they can see users finding answers to questions they search for and upvoting those answers (C2). When they examine the number of support tickets per topic area, they see a drop in tickets being opened for topics that have answered questions discovered through search (C1).

• Recruitment Example – The company needs to increase hiring for open solution architect roles. The pipeline of qualified candidates, however, could not meet the recruiting quota. The recruiting team pulls data on the developer community on who has been the most active in sharing the company’s content (C3). When they review the shared content, a small percentage are creating original content like blogs and videos, thus being tagged as high-value contributors (C2). Through various outreach efforts, they have been able to initiate hiring cycles for some of these high-value contributors (C3).

Why only define three levels? Because there is a bridge that is often required to connect community activity or engagement with business value. However, adding more levels makes the connection between business and community value more tenuous. The result is a more convoluted story, and it dilutes the impact you want to convey why the investment in community is valuable from a business perspective.

The value of this approach is that it shows a logical correlation between what happens in the community and how that results in business impact. It provides community teams tangible levers to experiment and adjust certain behaviors and actions in the community to drive business goals and KPIs. You begin to “connect the dots” for stakeholders in a way that is much clearer and more direct.

Using the Three Levels of Community Metrics avoids two common errors when determining business impact. First, it does not start at some high-level and ambiguous business metric like revenue, cost, or productivity. It begins with a more attainable and practical business objective that a stakeholder would be directly responsible for as part of their objectives. Any C1 metric must have an internal owner and be a priority in order to be useful for this exercise. Second, it does not include vanity metrics that have no value for the business and do nothing to help understand business impact. We can spend our energy on actionable metrics.

There is a valid argument that some of the metrics in the above example are not strictly correlated. Many factors could influence the C1 metrics beyond the examples outside of the metrics cited. Sometimes, correlations are assumed rather than direct. Assumptions are okay as long as you can back up your assumptions with logic and show a range of effects. Just like any experiment, you need to create baselines and control groups when implementing the framework in order to figure out the correlative effects and whether your assumptions are valid.

Another common objection is that C1 and C2 could be the same depending on the type of community initiative. This is especially true if the impact made by the community is more immediate and direct. Here, it may be better not to include bridge metrics at all in order to show a correlation, as it is fairly straightforward and obvious.

The framework presented here will go a long way in supporting the case for continued investment in community efforts. Moving from vanity metrics, sporadic anecdotes, and cheerleading towards a more quantitative and data-driven approach gives community teams a more powerful tool to build the support they need to show business impact. Perhaps the most exciting benefit of the Three Levels of Community Metrics approach is that it can give community teams a seat at the table with an organization’s leadership as a domain that is as important as sales, product, or support!

Mark Birch

It is the final stretch on finalizing the book and it has been a slog to wrap up! This is why I skipped sending a newsletter last week as I was drowning in edits to the book. The good thing is that the book is much stronger this time around!

Though I have been super busy on the book, I did have some time to get around town and do some content related activities. I had a chance to attend a few events, including on for the Taiwan Gold Card holders, as well as a pitch event for wellness startups.

I also was a guest on not just one, but two, podcasts! I started the week off as a guest on the Eddy LIVE 艾迪直播. It is hosted out of a Mexican Restaurant in Taipei and is the longest running English podcast in Taiwan. It is I had a blast talking about what brought me to Taiwan and my observations on the startup culture over tacos and margaritas!

Then I wrapped up the week as a guest on the TechSoda podcast by seasoned journalist Judy Lin covering the latest technology news from Taiwan. It was an awesome, hour-long conversation about building communities and busting up misconceptions such as why community is not just marketing. The episode should be out next week and I will share it in the next newsletter. Cheers!