Data for Decisions

Data Matters

In one of my first full-time software engineering roles I found myself doing what I would later come to know as data engineering. Our simplified workflow was to ingest data from the product database and a few vendors into our data warehouse.¹ From there we used business intelligence tooling to create reports at various cadences. This reporting would later prove to be an essential part of the business, helping to establish product-market fit and accelerate revenue growth.²

Over the years, data has become increasingly important as we build businesses. Many folks have come to realize the value of leveraging data to build products and deliver value. From startups to large corporations, virtually every tech company now recognizes the value of data-driven decision making. Moving from workflows where we operate largely in the dark, making decisions based on intuition, to decisioning based on data, is transformative.

However, it's important to point out that the value of data is not the data itself so much as the insights we draw from it.³ As an example, we might have web analytics which indicate various things about our web traffic: number of unique visitors, page views per session, the geo region a user is from. While this can tell us something about our site, for instance how popular it is on a given day, it isn't necessarily driving the business forward.

From Data to Insights

Indeed, to make data useful we need to make it relevant to our work. If say our business leverages web traffic to sell products, then our web analytics data becomes more interesting as we understand the movement of that traffic through the larger map of our site. In this example, we might build insights from web analytical data by describing an expected funnel, from landing page to purchase, to understand how well it performs at various stages.

Insights will vary from company to company and product to product. Measuring a web purchasing funnel is useful if a product fits a certain model of e-commerce, but less so if your sales pipeline is mostly offline or handled through a process of high-touch lead generation. So what's important here is that the data be adapted to the right insights that fit the needs of the business.

In fact the importance of fitting insights to the needs of the business shouldn't be understated and a common pitfall of data work is to overlook this.⁴

Asking the Right Questions

In order to avoid working on the wrong kinds of insights, we need to ensure we're framing the work appropriately. Like other product work, it's often helpful to think about what it is we're trying to achieve. With data more specifically, we can start with a question:

"What decisions will we make with this data?"

We should be able to readily answer this question. If we believe it isn't related to any decision or part of a decision making process, then we should take a step back and think more deeply about the goals and how they relate to the value we're driving. Ideally we'll invest in data work that's tied directly to business decisions because without this the value of the data itself is unclear.⁵

Then we need to reflect on how we answered this question. By doing so, we can begin to see a clearer picture of how this work is situated within the larger landscape of the business. For instance, if we'll use this data to make a decision about adjusting our marketing spend, then we might continue by querying for the frequency of this decision. From this, we begin to understand essential requirements, like data freshness.⁶

Furthermore this gives us a tool for quantifying the relative impact and need. If the data will be used for casual exploration or occasional validation of secondary or tertiary business metrics then the priority of that work is meaningfully different as compared to data that will be used to drive day-to-day business decisioning, such as marketing spend.

By orienting around decisions, we have a basic framework for steering data insights to their strongest potential, maximizing impact and broader alignment within the business.

Decisions Over Data

Data is a powerful tool that's inarguably a key piece of how technology companies succeed today. Whether the business context is big or small, data-driven approaches are powerful.

That said, data is only useful insomuch as we can align it with relevant insights. It's easy to lose sight of this and forget that data needs to be shaped into relevant insights for our specific business context.

To avoid investing in the wrong data work, we should start by identifying what decisions we'll make with it. Not only does this help filter asks it also lays the foundation for requirements and prioritization.

Like software more broadly, data is a means to an end and by providing lightweight structure to orient ourselves around specific goals we can maximize the value it provides.

Footnotes

1. At the time, Airflow wasn't a thing, and so we used a tool called Luigi (you know, because data pipelines involve plumbing). We had a number of workflows that would take data from one location and enrich it in various ways, ultimately storing it in our data warehouse, Redshift. ↩

2. It's hard to imagine how the business would have succeeded without this data-driven approach. If it had, it would have been on our lucky guesses as opposed to the empirical approach we took. ↩

3. There are exceptions to this, but even then the value of the data is still ultimately what we do with it. Until we begin using it in this way, its value is entirely theoretical. ↩

4. More specifically there's a temptation to grab as much data as we possibly can before we know what to do with it. This is counterproductive for a multitude of reasons, including the fact that well-formed data is a necessary input to worthwhile insights. ↩

5. Data for the sake of data isn't valuable. It's a signal for caution if we don't understand what the data is for and how it'll be used. ↩

6. The specifics here matter and ensuring there's a dialogue from this initial question is important. ↩