As NGOs turn to technology to reach and serve a greater number of clients, the question arises of how to accurately measure the impact of digital programs, which require a different approach to in-person initiatives. Given Signpost's reliance on digital channels, Signpost measures users in a distinct way from most other IRC programs. One advantage of digital programs is their strong potential for broad global reach, which in-person programs cannot usually achieve. While the depth of engagement may be less than some in-person programming, digital programs offer a way to provide people with critical information who may not otherwise be reached.
Digitally tracking unique users presents specific challenges, including data privacy concerns, anonymous users and duplicate accounts. These challenges limit the ability to collect user demographic information like age, gender and location. Due to privacy concerns, demographic information is not usually available at an individual level. For example, Facebook pre-aggregates demographic data for privacy and security reasons, as well as it being easier to process at a larger scale.
For Signpost, measuring users in an ever-changing technological landscape requires taking into account the data policies and governance of each digital tool used. For example, companies like Meta and Google determine the data made available to third-party teams and how it is reported, calculated and available via their APIs, which can change frequently. Therefore, Signpost's measurement strategy makes the best use of available data while adapting to organizational changes.
As Signpost scales to 30+ programs, manual data collection becomes time-consuming, inefficient, and costly. Therefore, a reporting solution that allows for efficient, automated, and consistent global understanding of users is necessary. To measure users, any data point used must meet three criteria: 1) queryable via the Application Programming Interface (API) of the company whose tool we are using (ie. Meta, Google), 2) reported in a daily format and 3) measures unique individuals.
When measuring users, the depth of user engagement becomes a key question. Who can be counted as a user of Signpost platforms? How do we differentiate between someone who scrolled past a Facebook post on their feed versus someone who read an article on a Signpost website, reached out to ask a specific question and received a response from a moderator?
To measure users, Signpost has developed three composite metrics that allow understanding unique users at three different scales outlined below, depending on the level of engagement sought. By using such measures, Signpost can obtain a more comprehensive view of user engagement across all programs and channels. While each metric provides a different level of insight, they are all critical for understanding the impact of Signpost programs within the digital environment in which we operate. It is important to note that Signpost's measurement strategy will continue to evolve and adapt as we expand our toolkit to include other digital platforms beyond Google, Facebook and Instagram, to TikTok, YouTube, and beyond.