And for that matter, to where exactly we’re trying to get?
One use of numerical measurement: to describe direction
Continuing to reflect on some of the topics that I’ve covered in the last couple of posts, I want to spend more time today on numerical measurement and its alternatives.
While the answer to the question posed in the title may seem like a resounding “yes”, I’d like to suggest that in some circumstances, it may actually be “no”.
To be clear, I’m not a measurement or metrics hater. And this is not a rant about metrics. But given that I’ve spent quite a bit of time calling out their deficiencies, I owe it to myself and others to offer alternatives that are not meant to completely replace them, but to expand the toolbox so we can use the more effective tool to the problem at hand.
I won’t repeat my whole case on the challenges with numerical measurement but will just briefly mention that they typically elicit some non-trivial “operational” challenges in both the collection and interpretation of the data. And perhaps more importantly, they also pose some more “strategic” challenges — they reduce a highly complex reality into a very simplified representation. Sometimes that’s incredibly helpful — separating signal from noise and creating clarity on what’s truly important. But often times over-simplifaction leads to solutions with both cognitive and behavioral flaws when applied back in the complex reality.
The alternatives to this approach depend on the purpose we were trying to accomplish with numerical measurement begin with, something that I’ve noticed I haven’t given enough attention to in the past. Introducing some distinctions there helps to identify viable alternatives, at least in the two cases outlined below.
Measurement to articulate direction
One common use case of numerical measurement is to articulate direction. By describing where we are right now and where we want to get to, we implicitly define the direction in which we want to go:
We want to get from point A to point B = we need to drive in direction C
We want to improve margins from 13% to 15% = we need to improve margins/become more efficient
*nerd alert* describing a vector using the coordinates of its start and end points
Often times the start and end points are rather meaningless in and of themselves. It’s the direction or delta between them that matters.
Yet describing the start and end point are not required to describe the direction. I can still “drive south” without saying “get from SF to LA”. Not the perfect example, I know, but hopefully still gets the point across.
So alternatively, we can use either “even over” statements or slightly more detailed polarity maps to describe the direction we want to go.
Measurement to choose between options
Another common use case of numerical measurement is to choose between options.
For example, choosing what driver to focus on next in order to improve employee engagement. This is often done by having employees rate each one of the drivers using a Likert Scale, translating each rung in the scale to a numerical score and sorting the drivers from the lowest rated to the highest rated or from the one that worsened the most to the one that improved the most.
The distinction between cardinal and ordinal utility can help us find an alternative. We don’t even have to go into the debate about the feasibility of truly measuring the cardinal utility of each one of the drivers and simply say that since we’re only using the measurement to choose between the options, the ordinal utility is sufficient. And in that case, there’s a simpler alternative to numerically rating each one of the drivers in isolation: asking employees to stack-rank the drivers from the one we should focus on the most, to the one we should focus on the least.
It’s been interesting to notice the difference between concepts that I thought will be useful to keep in mind, immediately after reading a certain article, and the ones that actually proved out to be useful several months later when I find myself referencing them over and over again.
One kind of figurative language that is especially important is metaphor. It is a way for individuals grounded in different contexts and with different experiences to understand something intuitively through the use of imagination and symbols without the need for analysis or generalization.Through metaphors, people put together what they know in new ways and begin to express what they know but cannot yet say. As such, metaphor is highly effective in fostering direct commitment to the creative process in the early stages of knowledge creation…
But while metaphor triggers the knowledge-creation process, it alone is not enough to complete it. The next step is analogy. Whereas metaphor is mostly driven by intuition and links images that at first glance seem remote from each other, analogy is a more structured process of reconciling contradictions and making distinctions. Put another way, by clarifying how
the two ideas in one phrase actually are alike and not alike, the contradictions incorporated into metaphors are harmonized by analogy. In this respect, analogy is an intermediate step between pure imagination and logical
thinking.
The context in which it is most present for me right now is thinking about employees as customers, which I’d argue for many organizations is still “stuck” in the metaphor stage of knowledge creation. But before I jump to the opportunity that lies ahead of us, I want to acknowledge the celebration-worthy progress that the current stage represents.
Thinking about employees as customers is a massive step forward compared to the previous organizing metaphor: employee as resources/machines. First and foremost it acknowledges that employees are human beings and need to be treated as such. It reminded us that employees are in choice about their actions: they choose to join, they choose to stay, and they can choose to leave. It also served as a directional inspiration for how to address many employee challenges by borrowing concepts and ideas from the customer domain:
Lead generation/business development → Sourcing
Sales → Recruiting
Sales Marketing → Recruiting Marketing
Product brand → Employer brand
Product value proposition → Employee value proposition
Net Promoter Score (NPS) → employee Net Promoter Score (eNPS)
The metaphor continues to provide inspiration to this day, with more customer concepts making their way to the employee domain. A more recent example is recognizing specific “moments that matter” in the customer’s lifecycle, which require special design and attention, as also relevant for employees.
However, while the metaphor continues to move us forward in some ways, its drag, or downside, if you will, is also starting to become more apparent in cultural challenges such as unjustified entitlement or learned helplessness among employees which in turn make efforts to improve the shared working experience somewhere between extremely hard to impossible to execute on.
A more concrete example is the heavy reliance on surveys as the primary means of engaging with employees, a tool that was borrowed directly from the customer domain to the employee domain. Employee interaction needs to be bi-directional and iterative, and it needs to revolve not just around the present state but also around creative problem solving: what each of us can do about it. Yet surveys tend to move the conversation exactly in the opposite direction.
Nonaka’s work paints a clear path forward: moving away from metaphor and towards analogy. While the key focus in the former is around looking for similarities as sources for inspiration, the key focus in the latter is around looking for differences (distinctions) and addressing them, creating a more refined representation of reality.
At the root of most of the customer concepts that get pulled into the employee domain and end up backfiring seem to be a handful of distinctions, ways in which customers and employees are NOT alike. I suspect I’ll continue to refine these over time but here’s what I have so far:
The core interaction between customer and service provider has a clear division of roles: I, the customer, have a problem that I’m trying to solve, and you, the service provider, are supposed to provide me with a solution to it. Inside the organization, it’s not that clear cut: we are working to accomplish a shared mission together, and division or roles and authority is more dynamic and less absolute. We are all part of the problem and we are all part of the solution.
Cross-customer interaction, as it pertains to the organization, is relatively weak (mostly word-of-mouth reputation) so thinking about the way the organization interacts with each customer in isolation is a pretty accurate description of reality. Cross-employee interaction, as it pertains to the organization is very strong — tight collaboration to accomplish shared goals. So the way the organization interacts with each employee cannot be thought of in isolation.
Acknowledging these differences and designing ways of working together with them in mind is an important frontier in the future of work.
Author’s note: it’s been a while since I had a chance to write a post completely “from scratch”, not having it based on a particular article or book. This one has been brewing in my head for some time now and I’m excited to share it with you all!
“You can’t manage what you can’t measure.” — Peter Drucker
“If you give a manager a numerical target, he’ll make it even if he has to destroy the company in the process.” –W. Edwards Deming
“Not everything that counts can be counted, and not everything that can be counted counts.” — Bruce Cameron
This trio of quotes captures beautifully the fundamental tension that we’re all trying to navigate when we work hard to make our organizations better. Without a clear measure of progress, it’s hard to know whether we’re making progress and whether our current efforts help advance us towards our goal or move us away from it (Drucker). However, not everything that we care about can be measured (Cameron), and sometimes trying to force the issue and measure something can lead to pretty painful, unintended consequences (Deming).
The current state of DIB efforts
Nowhere is this tension felt more today than in our collective efforts to make our organizations more diverse and our behaviors more inclusive, fostering a deep sense of belonging among our teammates. Figuring out what to measure and what progress looks like remains a heavily debated topic.
Measuring diversity is becoming a more popular practice because it seems easy at first. But when we dig a little deeper and grapple with less easy to measure aspects, such as socioeconomic status (see Aline Lerner’s response here), not to mention intersectionality, Deming’s observation seems closer to the truth.
Measuring inclusion is perhaps more critical since it seems to have a more profound business impact. Not to mention that improving diversity without any follow up deliberate action will most likely decrease inclusion. However, inclusion turns out to be more difficult to measure and improve.
Often stumped by this challenge, many HR organizations turn to their “silver bullet” measurement tool and attempt to use our all-purpose-hammer: the survey. Yet, as the folks at Cultivate so eloquently point out, survey data suffers from a myriad of human biases: from recency bias, through acquiescence bias, to self-reporting and social desirability bias. And I will further add some more “mechanical” challenges such as selection bias (partial participation) and proper statistical analysis of the results.
Supporting inclusion also requires a different “type” of measurement. Since improving inclusion requires human behavior change, feedback (measurement) needs to be a lot more frequent and timely in order to make a difference. Learning today that there was something that I could have done differently two months ago is not so useful. Learning about it immediately, or even an hour later can be transformational, since the window for corrective action is still open.
To find a solution to this conundrum, we need to take a slight detour and familiarize ourselves with a much lesser known tool in our toolbox, that’s currently undergoing a profound revolution.
Organizational Network Analysis
Organizational Network Analysis (ONA for short) is the process of studying the relational and communication patterns within an organization through the use of models (graphs) of said relationships/interactions and conducting analysis, often statistical in nature, do derive various insights at both the group and individual levels. For example, these models/graphs, often referred to as “sociograms”, can be used to evaluate the overall level of “closeness”/”density” of relationships inside the organization by measuring the average “distance” (number of connections) that it takes to get from any one place in the network to any other place in the network. At the individual level, it is fairly easy to identify “outliers” — the people that are least connected to everyone else in the organization. A slightly more comprehensive overview of ONA can be found here.
While the roots of ONA can be traced all the way back to the work of Emile Durkheim in the early 1890s, real research began in earnest in the 1930s and made significant leaps forward in the 1970s and 1990s as more sophisticated technology unlocked more complex analysis of the data. Today, ONA is offered as a standard service by both top-4 consulting shops like Deloitte and boutique consulting firms specializing purely in ONA like Culture Optix and Tree Intelligence.
But ONA never achieved wide, mainstream adoption. Most HR organizations today don’t even know that the tool exists, let alone use it in their day-to-day practice. I believe this is due to two main reasons:
The cost of ONA in terms of time, energy and effort remained high. Even though technology helped in the analysis portion, the data collection process required for the construction and update of the sociograms remained mostly analog, relying heavily on survey data with all their drawbacks covered above, significantly constraining both the type of data that can be collected and the frequency by which it can be collected.
The benefits of ONA remained fuzzy. Partly due to the data collection constraints, partly due to the relevant research still being in its adolescence stage, and partly due to not-so-great product management, the value proposition of using ONA and the types of organizational challenges that it can help address remained too broad and too shallow, never scratching a big enough itch to justify the complex execution and analysis.
But all of this is now changing.
The digital revolution
In the last two decades organizations have been undergoing a digital revolution in the way they collaborate and work together: from the pervasive use of email, through video conferencing, to instant messaging. Furthermore, many analog activities still generate some digital “footprint” — from calendar invites to digital work artifacts like documents, spreadsheets, and code.
This revolution opened the floodgates of data towards a new era of ONA in which not only can sociograms be constructed and maintained almost effortlessly, in real-time and with no human bias, but also the richness and granularity of the data that can be analyzed exceed by orders of magnitude what was possible a decade ago.
Companies like CrossLead, Kumu/Compass, and Cultivate are the early pioneers that have started exploring this rich sea of opportunities.
And companies like Humanyze continue to push the envelope even further by creating solutions that deliberately generate digital footprints to the analog interactions that currently don’t organic ones.
DIB + ONA = ❤
By now you should probably be able to tell where I’m going with this:
I believe improving inclusion is the “killer app”, the “thin edge of the wedge” if you will, for the broad adoption of next-generation ONA.
ONA, with its analytical orientation towards identifying individual and group relational patterns, and the ability to perform it seamlessly on an on-going basis is perfectly positioned to close the currently-broken feedback loop and provide us with the close-to-real-time feedback needed to drive real behavior change.
ONA can help us identify the overall state and trend, as well as both the “bright spots” (to learn from) and “hot zones” (to help) across many inclusive dimensions including but not limited to:
Use of gendered language
Communication silos and the people who connect them
Outsiders and bridges
Balance of communication frequency across teammates
Balance of communication time/reciprocity
Communication inside/outside working hours
Communication sentiment (positive, negative, etc.)
Marrying DIB and ONA presents an opportunity to leverage the heightened awareness around this hot-button, critical topic and gain an edge in a red hot super-competitive market for both HR leaders and software vendors alike.
In this piece, Hagel makes an observation that deeply resonates with me on the current focus of the conversation around personal and organizational development:
The key assumption in the room was that it was all about the mind. They assumed that our assumptions and beliefs shape what we feel and what we do. In this view of the world, emotions are a distraction, or at best a second order effect, and it’s ultimately all about our mind.
He then offers an alternative viewpoint:
Our emotions aren’t just derivative of our assumptions and beliefs. Emotions shape our perceptions, assumptions, thoughts and beliefs as well. If you try to shape assumptions and beliefs without paying attention to the emotions that already exist, good luck.
We need to move beyond mindset and expand our horizons to address our heartset: what are the emotions that filter how we perceive the world, shape what we believe and influence how we act?
The rest of the piece explores Hagel’s thesis around the origin of the mindset-focused viewpoint, and an attempt at sketching out a path forward that better integrates “heartset”, primarily making the case for the power of narratives in shaping emotions. The latter ties in well with the piece from two weeks ago about wise interventions.
While I read Hagel’s description of the current mindset-focused conversation as somewhat critical, I view it through a more appreciative lens: as the first step in breaking into the behaviorist view of human beings as a black box and a bold attempt to develop a more holistic view of humans that takes “what’s inside” into account.
Having said that, I agree that the mindset-focused narrative is incomplete, and the relationship between thoughts and emotions is bi-directional: thoughts shape emotions and emotions shape thoughts. If we think of emotions as labels that we’ve assigned to a subset of sensory, felt experiences perhaps the broader aspiration should be to integrate the cognitive perception (mindset) with a more holistic somatic perception into a unified view of human experience.
Org Hacking 