This post consists of three sections: the first is a synthesis of other people’s great ideas about what makes core principles particularly strong and effective. The second are lessons learned from looking at the core principles of other companies, looking for common patterns. The third is a disciplined top-down and bottoms-up method for creating/updating core principles. Primarily suitable for organizations where a more emergent or fully top-down exercise will not work given the organization’s size.

1. Theory: A Framework for Strong Organizational DNA

prin·ci·ple – ˈprinsəpəl/ – noun

A fundamental truth or proposition that serves as the foundation for a system of belief or behavior or for a chain of reasoning.

A vision-based, mission-driven organizational framework (adopted from Moz) 

Consists of the following:

• Core Principles: the architecture behind our decisions, infusing all our efforts – the intrinsic principles we believe in ahead of financial or strategic success. They are the framework by which we make decisions, hire/evaluate/reward people, and create culture.
• Core Purpose: The reason we exist – (aka mission) This is how we want to contribute to the world professionally, and how we want to make our living. We will forever pursue this purpose, though it can never be fully achieved.
• Strategic Vision: How we accomplish our purpose – How best to execute our core purpose (based on our abilities and market conditions) over the next 10 years 
• BHAG (big, hairy audacious goal): Our accountability to results – an easily measurable representation of whether we’re achieving our strategic vision 
• Strategic Initiatives: near-term methodology that we pursue – Key items we want to accomplish in order to see our strategic vision realized, and give us the best chance to hit our BHAG
• Tactics – everything anyone in the company works on should be mapped to 1+ strategic initiative 

Strong Principles

There are no good or bad principles, only weak or strong principles.

A principle is strong to the degree that it’s honest, clear, and reinforced:

• Honest principles are born of self-assessment and embrace the virtues and faults of leaders and their organizations. This honesty is the prerequisite of clarity.
• Clear principles are defined. They make hard decisions to precisely describe who/what is and isn’t a culture fit in a way that deliberately excludes huge chunks of the population and their beliefs
• Reinforced principles are practiced. They have well-worn pathways, and decisions can be made reflexively because of the clarity of the culture and obvious prioritization of principles.

Strong principles come at a cost

Strong principles require tough decisions to be made in order for the principles to be upheld.  If you establish principles that are never challenged, these principles aren’t serving any real purpose.

Strong principles may empower an employee to say NO to a decision from a superior on the grounds that it violates a core principle.

An organization considering defining its principles must first come to terms with the fact that, when properly practiced, principles inflict pain. They make some employees feel like outcasts. They limit an organization’s strategic and operational freedom and constrain the behavior of its people. They leave executives open to heavy criticism for even minor violations. And they demand constant vigilance. If you’re not willing to accept the pain real principles incur, don’t bother going to the trouble of formulating them.

There’s a fine line between strong principles and dogma. Principles should be thought of as “bounded ideals”, constrained by the messiness and complexity of reality. We should try as hard as we can to live up to them, and everyone should feel empowered to speak up when we violate them.

REMEMBER: Your core principles are not what you think they are, they’re what the team thinks they are.

Types of Principles

A comprehensive principle system incorporates people, product and process:

• People Principles. This is the category that startups tend to focus on when they consider principles. In this category, your principles should help you decide who you should hire, who you should fire and how you treat each other while you work together.
• Process Principles. principles associated with process link to how decisions are made in a company. This includes how you report progress, hold people accountable and trust each other. For example, most startups will say that trust is vital, but won’t construct a process that reflects and protects that priority.
• Product Principles. In most cases, it’s less instinctual to associate your product with your principles. Your type of product, its features, how you treat your customers and how you respond to threats and opportunities around your product are built into your principle system.

A more dynamic view of principles categorizes them to the following categories:

• Core principles are the deeply ingrained principles that guide all of a company’s actions; they serve as its cultural cornerstones. Collins and Porras succinctly define core principles as being inherent and sacrosanct; they can never be compromised, either for convenience or short-term economic gain.
• Aspirational principles are those that a company needs to succeed in the future but currently lacks. A company may need to develop a new principle to support a new strategy, for example, or to meet the requirements of a changing market or industry.
• Permission-to-play principles simply reflect the minimum behavioral and social standards required of any employee. They tend not to vary much across companies, particularly those working in the same region or industry, which means that, by definition, they never really help distinguish a company from its competitors. (integrity, ethics, quality)
• Accidental principles arise spontaneously without being cultivated by leadership and take hold over time. They usually reflect the common interests or personalities of the organization’s employees. Accidental principles can be good for a company, such as when they create an atmosphere of inclusivity. But they can also be negative forces, foreclosing new opportunities.

Ongoing work is needed to transition aspirational principles into core, and eliminating accidental ones.

Cultural Fit

Fit should be defined by:

1. Shared beliefs – the things that you collectively hold to be true about the world (e.g. good people tend to have traits like X, the right way to treat others is Y, what’s appropriate/inappropriate at work is Z).
2. Shared priorities – what matters in terms of big, overarching things like work/life balance, short vs. long term commitment, how decisions are made, etc.
3. Stylistic cohesion – some people don’t work well together, others find themselves able and inspired to do more when surrounded by a certain type. Cohesion isn’t about finding lots of people who are the same, but about making sure there’s no one on the team that detracts from others and that many get more enjoyment and progress from the diverse perspectives their co-workers bring.

Certain kinds of diversity are excellent in creating a better culture – gender, background, experience, education, etc, when diverse, tend to produce interesting, productive, and fun work environments. But, diversity on beliefs (around morality or on what work should be like) or priorities (minimum viable effort vs. “there’s little else I’d rather be doing,” or “we should sell this business as soon as there’s an opportunity” vs. “let’s build for the long term”) tends to produce a lot of ugly strife.

Vibe, Amenities and Events

Culture (and specifically, principles) is not vibe. Vibe represents the emotional side of the company. Like all emotions, vibe can be fairly volatile and is highly influenced by outside factors. The biggest influence on vibe is typically success.  Most companies that are doing well tend to have an overall positive vibe.

Amenities and events (Secret Santa gift exchanges, Karaoke nights, Bean bag chairs, Nerf gun fights, Catered lunches, Team events) tend to have a positive impact on vibe, but require thoughtful design in order to proactively advance your core principles.

 

2. Practice: Observations from looking at other companies’s Core Principles  

I’ve looked at the Core Principles of 10 leading companies, searching for common traits and patterns. Here are our key lessons learned:

• Core principles should be a set of principles, which if hired against and acted against, creates a true north which defines good day to day work habits and decisions — and reduces the need for cumbersome micromanagement and processes
• The fewer the better. Especially in light of the the following point below, having more than 5 core principles become meaningless, because that means that “everything” is important
• When we strip away the creative wordsmithing, core principles become a lot less company-specific. They tend to fall into 10 broad categories. Each company essentially picks the ones that matter the most to it:   

1. Mission/value orientation – “focus on impact”, “champion the mission”, “we commit maniacally to both the mission and our metrics”
2. Customer focus – “focus on the customer and all else will follow”, “our members come first”, ‘we look at the long term and solve for the customer” , “we put our patients first”
3. Risk taking/creativity – “be bold”, “take intelligent risks”, “we dare to be different and challenge the status quo”, “we challenge conventional thinking”
4. Ownership  – “act like and owner”,  “we favor autonomy and take ownership”, “take ownership” , “be a cereal entrepreneur”
5. Transparency – “be open” x2, “we share openly and are remarkably transparent”,”we are transparent until it hurts”
6. Excellence – “it’s best to do one thing really really well”, “great just isn’t good enough”, “demand excellence”, “every frame matters”
7. Humility/Learning – “learn from failure”, “turn friction into learning”, “be humble”, “make others successful”, “
8. Simplicity – “simplify”, “we build simple”, “refactor”  
9. Tenacity, grit – “live the spirit of challenge”,  “embrace ambiguity”, “we embrace challenge with grit”, “embrace the adventure”
10. Speed

3. Method: Developing Your Core Principles

Part I-A: Founder self-reflection:

• Answer the following questions (first things that come to mind, don’t over-think it):
  • What are my strengths?
  • What am I outstanding at?
  • What sets me apart from the people around me?
  • What do I value about the people around me?
  • When I look at my friends, what are the characteristics they have in common?
  • What qualities drive me crazy about people?
  • How do I make my best decisions? (Think of a recent decision you made that had a good outcome. What process led to that?)
  • What am I bad at?
• Answer:
  • Using the Reinventing Orgs framework – what kind of organization is your company right now? (Impulsive, conformist, achiever, pluralistic, evolutionary)
  • What kind of organization would you want your company to be? (Impulsive, conformist, achiever, pluralistic, evolutionary)
  • Given the kind of organization you want your company to be, how would you define your role as CEO?

Part I-B:

• Allow all employees to contribute their input to Session I (of Part II) by filling in an online “self-reflection” form with similar questions

Part II: All-day group offsite

• Share this doc in advance
• Share Reinventing Org deck in advance

 

• Kick-off

 

• Process and workshop overview
• Q&A/ short discussion on materials (clarification, not a debate)

 

• Session I – divergence/exploration

 

• Goal is to avoid bias from CEO’s self-reflection at this stage
• Company Self-reflection (brainstorm-style, each person puts their answer on a sticky note on a flip chart with the relevant question):
  • What are our strengths?
  • What are we outstanding at?
  • What sets us apart from the companies around us?
  • How do we make our best decisions? (Think of a recent decision we made that had a good outcome. What process led to that?)
  • What are we bad at?
  • What do you value about the people around you?
  • When you look at your friends, what are the characteristics they have in common?
  • What qualities drive you crazy about people?
• Add CEO’s answers (different color)

 

• Break (+silent/individual review)

• Session II – convergence/consolidation

 

• Plannary: 3-4 people share observations from/reactions to first session
• In groups, each group tackles 1-2 of the company self-reflection questions:
  • Find meaningful cluster/themes for the responses
  • Formulate into suggested core principles. Formulate as commandments (start the sentence with a verb: “think at scale”, “draw the owl”). Bonus points: add an “even over” section articulating the main trade-off (“build working software even over creating comprehensive documentation”). Don’t fret over word-smithing too much.
  • Test for applicability – is the core principle you identified can help inform:
    • The way we should treat each other
    • The way we should build our products/deliver our services
    • The way we should run our processes

Not every principle should inform all three, but if a principle doesn’t inform any – it’s not a useful principle

• Plannary: each group presents their principles for 3-5 mins followed by 3-5 mins of questions
• Lunch break  

 

• Session III – pressure-testing and selection  

 

• Dot-vote on following attributes:
  • The people who are successful at our companyembody this principle
  • The long term success of our company will be harder without this principle
  • This principle sets us apart from other companies
  • This principle helps informing the way we should treat each other
  • This principle helps informing the way we should build our products/deliver our services
  • This principle helps informing the way we should run our processes

• Select winners (do another round of dot-voting if there are no clear winners)

 

• Group Session IV – action planning

 

• Each group is assigned a small set of processes and articulates gaps/ divergence from core principles + plan to close them.
• Present back to plannary

 

• Group Session V – next steps

 

• Polish session – identify 2-3 additional participants
• Present to rest of company – 3-5 people + wrangler.
• Work on closing the gaps – assign people to individual process action plans. Opt-in, not mandatory. This team did all the heavy lifting so they get first dibs.

Part III: Polish session

• CEO as Editor in Chief – final refinements. Compare against self-reflection.
• Marketing support / someone with good copy-writing skills?
• 2-3 other people

Part IV: Roll-out

• Presented at a company all-hands meeting by non-execs
  • Process we followed
  • Core Principles we came up with
  • Action Plan – current participants + calls for additional people. No more than 5 per process.

Appendix: Bibliography

 

1. https://moz.com/rand/diving-deep-on-tagfee/
2. http://scottberkun.com/2014/critique-dont-fuck-up-culture/
3. https://medium.com/evergreen-business-weekly/most-company-culture-posts-are-fluffy-bullshit-here-is-what-you-actually-need-to-know-1cf8597a5c2c#.gbuctkn7n
4. https://medium.com/@dunn/creating-culture-21a117803f80#.clhlubw1e
5. https://moz.com/rand/vision-based-framework/
6. https://moz.com/rand/what-company-culture-is-and-is-not/
7. http://www.feld.com/archives/2012/06/startup-culture-principles-vs-vibe.html
8. https://hbr.org/2002/07/make-your-principles-mean-something
9. http://firstround.com/review/This-Culture-Deck-Powers-the-Worlds-Toughest-Work/
10. http://firstround.com/review/Culture-isnt-Kumbaya-stuff/
11. http://firstround.com/review/80-of-Your-Culture-is-Your-Founder/
12. http://firstround.com/review/draw-the-owl-and-other-company-principles-you-didnt-know-you-should-have/
13. http://ssir.org/articles/entry/how_to_create_a_culture_manifesto_for_your_organization_and_why_its_a_good
14. http://www.tablegroup.com/imo/media/doc/D2%20Create%20Clarity.pdf
15. http://www.tablegroup.com/imo/media/doc/Develop_a_Playbook.pdf

Buster Benson‘s

Codex Vitae

Is a fantastic idea and as a bonus, a great rabbit-hole-entrance to his writing in general.

Inspired by Robin Sloan’s Mr Penumbra’s 24-Hour Bookstore, he decided to attempt to write his own Codex Vitae or Book of Life, that represents everything he has learned in his life.

The Codex Vita starts by articulating several classes of beliefs: metabeliefs, perceptions, observations and predictions.

It then covers a set of topic and ideas that drives Buster’s worldview. And then his “personal canon”: articles and books that he was most inspired by, and his key writings.

The idea of creating such artifact as a tool for self exploration, reflection and personal growth strongly resonates with me. So much so that I have “forked” (sort of) Buster’s repo on GitHub and created my own branch here.

At the time of writing this post, it only contains the first pass at the (mathematical) intersection of his beliefs and mine. I hope to add to it in the coming weeks.

 

Tiago Forte of Forte Labs wrote a wonderful 4-part series about the “Future of Productivity”.

Its title doesn’t do it justice, in my opinion at least, as it’s a much deeper treatise on the essence of human behavior, which has applicability well beyond the limited scope of productivity. Though much of this significance body of work  (~4,000 words each) is oriented around the overall goal of improving productivity, I’d argue that much of can either be used as-is or slightly generalized to be applicable for an even-bigger goal of living up to one’s full potential.

Part 1:  Productivity for Precious Snowflakes – discusses a mindset-driven approach for productive work and mastering it using four core skills: alignment, specialization, workflow and aggregation.

Part 2: Emergent Productivity: A People-Centered Equation for Modern Work – discusses the tensions between an employee and the business under the existing transactional paradigm, and the way they can be resolved under a new paradigm in which rather than viewing work as a common ground between the company and employees, employees are viewed as the common ground between the company and work. This simple switch leads to rather surprising outcomes.

Part 3: The Holy Grail of Self-Improvement – discusses Tiago’s take on habit formation and behavior change. It starts off with a few prominent behavior change framework like “identity-based habits“, but the more interesting part explores the notion of “emergent habits” under the assumption that human behavior is a complex (as opposed to a complicated) system. This leads to some thoughtful insights on the way behavior is shaped and habits can be formed.

Part 4: Meta-Skills, Marco-Laws and the Power of Constraints – discusses the core components of a self-knowledge based productivity improvement framework: meta-skills – skills that allow you to leverage other skills, macro-laws – fundamental axioms about yourself designed to guide the next phase of exploration and learning.

Take a breath, this is going to be a long-ish post in the standards of this blog.

In this post, I’d like to explain why I think “employee engagement” and/or “employee satisfaction” are vanity metrics, that are likely to become a distraction from true improvements to organizational health. Don’t worry. It’s not all bad. I also like to propose an alternative path that’s more likely to lead to the end state we’re really optimizing for.

This is a 3-part piece: in the first part, I’ll present the thinking, mostly not my own, that led me to see this disjoint. In the second part, I’ll anchor it in some real world examples. Finally, I’ll propose an approach that I believe can take us closer to the right path.

Part I: What’s wrong with “employee engagement”?

I can’t remember what led me to discover this “ancient” (+20 years old) piece by Chris Argyris,  one of the fathers of the field of “organizational development”:

Good Communications that Blocks Learning

But ever since I did, I come back and re-read it quite regularly. Which is rather rare in my case, and a good indication that it contains some fundamental truth, that unfortunately is still far from being considered self-evident in the People Ops space. It’s a +6,000 words piece, that is well worth a thorough read, but I’ve quoted some of the most relevant parts to the discussion at hand below (emphasis is mine).

Chris makes his case known from the get-go:

Years ago, when corporations still wanted employees who did only what they were told, employee surveys and walk-around management were appropriate and effective tools. They can still produce useful information about routine issues like cafeteria service and parking privileges, and they can still generate valuable quantitative data in support of programs like total quality management. What they do not do is get people to reflect on their work and behavior. They do not encourage individual accountability. And they do not surface the kinds of deep and potentially threatening or embarrassing information that can motivate learning and produce real change.

And explains the false rationale more deeply:

In the name of positive thinking, in other words, managers often censor what everyone needs to say and hear. For the sake of “morale” and “considerateness,” they deprive employees and themselves of the opportunity to take responsibility for their own behavior by learning to understand it. Because double-loop learning depends on questioning one’s own assumptions and behavior, this apparently benevolent strategy is actually antilearning. Admittedly, being considerate and positive can contribute to the solution of single-loop problems like cutting costs. But it will never help people figure out why they lived with problems for years on end, why they covered up those problems, why they covered up the cover-up, why they were so good at pointing to the responsibility of others and so slow to focus on their own.

At the heart of the problem, is a false role design principle:

First, consider the way roles and responsibilities are assigned in manager-employee (or leader-subordinate) conversations, interviews, and surveys. There seem to be two rules. Rule number one is that employees are to be truthful and forthcoming about the world they work in, about norms, procedures, and the strengths and weaknesses of their superiors. All other aspects of their role in the life of the organization—their goals, feelings, failings, and conflicted motives—are taken for granted and remain unexamined. Rule number two is that top-level managers, who play an intensely scrutinized role in the life of the company, are to assume virtually all responsibility for employee well-being and organizational success. Employees must tell the truth as they see it; leaders must modify their own and the company’s behavior. In other words, employees educate, and managers act.

Which manifests itself in tools like employee surveys:

Employee surveys like the one Acme conducted—and like most other forms of leader-subordinate communication—have a fundamentally antimanagement bias whenever they deal with double-loop issues. They encourage employees not to reflect on their own behavior and attitudes. By assigning all the responsibility for fixing problems to management, they encourage managers not to relinquish the top-down, command-and-control mind-set that prevents empowerment.

This “positivity” also highlights more fundamental issues:

But this emphasis on being positive is plainly counterproductive. First, it overlooks the critical role that dissatisfaction, low morale, and negative attitudes can play—often should play—in giving an accurate picture of organizational reality, especially with regard to threatening or sensitive issues. (For example, if employees are helping to eliminate their own jobs, why should we expect or encourage them to display high morale or disguise their mixed feelings?) Second, it condescendingly assumes that employees can only function in a cheerful world, even if the cheer is false. We make no such assumption about senior executives. We expect leaders to stand up and take their punches like adults, and we recognize that their best performance is often linked to shaky morale, job insecurity, high levels of frustration, and a vigilant focus on negatives. But leaders have a tendency to treat everyone below the top, including many of their managers, like members of a more fragile race, who can be productive only if they are contented.

Chugging along with these attitudes in place, is simply not sustainable in today’s competitive reality:

Today, facing competitive pressures an earlier generation could hardly have imagined, managers need employees who think constantly and creatively about the needs of the organization. They need employees with as much intrinsic motivation and as deep a sense of organizational stewardship as any company executive. To bring this about, corporate communications must demand more of everyone involved. Leaders and subordinates alike—those who ask and those who answer—must all begin struggling with a new level of self-awareness, candor, and responsibility.

Bottom line: the focus on positive “employee engagement” / “employee satisfaction” further exacerbates the “employees educate, managers act” dynamics and the sense of learned helplessness of employees in dealing with organizational health issues.

Part II: Real world examples

Let’s explore two prominent trends in the PeopleOps space. Both are getting some serious attention in recent years and experience some notion of forward progress. But evaluated through Argyris’ double-loop learning lens, are we really making progress or still standing in place?

Employee engagement/satisfaction surveys:

Whether you use a tried-and-true tool like the Gallup Q12, decided to give an innovative startup like CultureAmp a try,  or opted to build your own home-brewed, most-likely-Google-Forms-based tool, the fundamental interaction design remains the same: you, the employee, should truthfully tell us what’s going on, and we, management, will solve these issues for you. Allow me to demonstrate with a few Q12 questions (my additions in italics, intentionally extreme/exaggerated to drive the point home):

• You DO NOT know what is expected of you at work, but never asked your manger to clarify her expectations
• You DO NOT have the materials and equipment you need to do your work right, but never asked for them
• At work, your opinions DO NOT seem to count, but you never made an effort to articulate them clearly
• Your associates or fellow employees ARE NOT committed to doing quality work, but you never referred a candidate to the company, or gave one of your peers honest feedback
• You DO NOT have a best friend at work, but never made the effort to make one

To be clear, I’m not trying to make the case that these should all be problems for the employee to solve on her own – that would be just as senseless as making them purely the problem of management. I’m trying to show that the employee’s own behavior plays a critical role in both the problem and the solution.

360 Feedback

Again, the tool, be it Workday, Reflektive or your own home-brewed solution, doesn’t change the fundamental interaction design: you, the reviewer, should truthfully tell us/me what’s going on, and we (management)/I (the employee) are/is responsible for addressing it. Even in the more progressive setup, where feedback is shared/solicited directly to/by the employee, the dynamic stays the same: one side calls out the problem, and the other side is responsible for solving it.

Part III: Having “skin in the game”

The good news is that starting to move in the right direction, doesn’t require some dramatic change management effort, or a massive paradigm shift. The glass half-full of “a tool is only as good as the process it enables” is that it’s unlikely that you’ll have to throw your existing tool out the window, and have all those hours upon hours of implementation and adoption go to waste.

A small tweak to the interaction design can be a great first step. I’ve been contemplating how to label this tweak: “dyadic” sounds way too academic, “reciprocal” is not quite there. “Skin in the game” seems to be the closest, but I’m open to better label ideas. The idea is to model the interaction based on the fact that both sides are accountable for solving the problem, and they both have roles to play in the solution, albeit different ones (in most cases).

The pattern looks something like this:

• The problem, why it’s a problem, and how much of a problem it is.
• What I will do to make it better
• What you can do to make it better

Consider the 360 feedback example I covered earlier:

• “You could really use some work on your communication skills”

Vs.

• “You could really use some work on your communication skills. Here’s what I will do to help you work on them: <list of things that I will do> and here’s what you can do to work on them: <list of things that you can do>”

Which feedback is more likely to result in you improving your communication skills, and which one is more likely to end in inaction?

Would this require more from me as someone providing feedback? Absolutely! But to repeat the final words form the Argyris piece:

Leaders and subordinates alike—those who ask and those who answer—must all begin struggling with a new level of self-awareness, candor, and responsibility.

Data analysis is core to almost any role in the knowledge economy. We either do the analysis ourselves, on data that we collected, or try to learn from analysis done by others.

In both cases we are looking for insights – trying to find meaningful patterns in the data that have some functional use – for example: supporting a business decisions.

Many of us are able to tell apart an average from a mean, and have the similar image in our heads of a semi-shaded bell curve, when someone talks about a standard deviation.

But at least in my own case, it seems that I have left behind any functional understanding of deeper statistical concepts somewhere in my second year of undergrad. That is, until some well-intentioned past colleagues have taught me to know better. I’ve been grateful for them ever since.

Of a slightly more extensive list of tools and concepts, I’ve picked two that I view as absolutely critical to master, if we want to have any real chance of success in our quest to discern signal from noise:

Confidence Interval – measurement is an imperfect process. Focusing solely on point estimates coming out of a measurement exercise (average, mean, etc.) typically creates a false sense of certainty in the accuracy of those estimates. Confidence intervals show the reliability of point estimates, by specifying a range within which the parameter is estimated to lie,  given a required confidence level (typically 90%, 95%, 99%).

Two-sample t-test – behind the scary name lies a critical ideal. Comparative analysis is an essential part of almost every form of data analysis. Often times we care less about an absolute computed value in isolation, and more about understanding its significance in the context of other values. Be it measurement of the same parameter in a different time, in a different population or a different parameter altogether. The two-sample t-test enables us to determine, under a given required confidence level, whether two parameters are indeed different from one another.

Let’s connect these tools with more real world examples:

Suppose, that for whatever reason we’ve decided to run an “employee engagement survey” to regularly measure our employee engagement scores (why that’s a bad idea is a topic for a different time). It’s an overly-simplified survey where we simply ask employees to rate their engagement on a scale between 1 and 7.

We’ve decided that if, on average, we score less than 5 – it means we have an engagement problem and further action must be taken. The survey results came back and the average was 4.8. Should we drop whatever we’re doing right now and start figuring out ways to solve our engagement problem? Well, that depends. Depends on what? Depends on the reliability of that point estimate. How might we go about assessing that? Confidence Interval!

We also decided to slice the data by department. While the average score for the marketing department was 5.8, the average score for the engineering department was 5.3. Wow! This seems like a serious problem. Should we drop whatever we’re doing right now and start figuring out what’s going on in engineering that their engagement score is so much lower than marketing’s? Well, that depends. Depends on what? Depends on whether those two averages are indeed different. How might we go about assessing that? A two-sample t-test!