Creating a data stack for Hacker News was fun and revealing at the same time
“If the statistics are boring, you’ve got the wrong numbers.” So said statistician and Yale professor Edward Tufte. And we couldn’t agree more. Crunching the numbers is inherently fun. That’s because analyzing data is like hunting for buried treasure. Sometimes you get lucky and find a gem of wisdom with minimal effort. Oftentimes, you will need to dig deeper beyond the surface-level information to discover the valuable riches of insight hidden within the data. To purloin another analogy, creating a data stack is much like cooking (which can also be fun!). Before you end up with a delicious dish of data, you must first source and store your ingredients (trackable items) in your kitchen (data warehouse), prepare your ingredients (collate and clean the data), put everything in the oven (data analysis), until, finally, you have a ready-to-eat-meal of data insights.
It was this thinking about preparing a fresh (and tasty) data stack just for fun that brought the website Hacker News to our attention. Hacker News (often shortened to simply HN) prides itself on providing intellectual discourse revolving around the general topics of computer science and entrepreneurship. Run by venture investor group Y Combinator, the site acts as a social news website where users can post thought-provoking content. Think of the glory days of Reddit.
Table of Content
Why create a tracking plan and data stack for hacker news?
Data stack creation principles
1. Have a plan
2. Ensure consistency
3. Communicate actionable data
Final thoughts
Why create a tracking plan and data stack for hacker news?
A tracking plan is a singular and centralized source of truth in a company. A well-designed tracking plan makes it easy to achieve 2-3 times more insights from your analytics data. A tracking plan aligned to your business model, goals, and user flow makes sure that your business is on the right path to being data-driven. Let’s be honest: some websites don’t require a tracking plan. Hacker News is definitely in this category. It’s essentially a discussion board for developers to share their insights as well as the great things they are building. Creating a data stack for HN was essentially a labor of love. Normally, you only track and gather data if it helps you make business decisions.
However, this didn’t stop us from creating a data stack for them.
It’s what we do after all.
Our passion is unlocking the power of actionable data to help your business grow.
It ends up being the case that, in its simplicity of design, HN is a perfect example to use to help explain the principles we use when we create a data stack.
Data stack creation principles
1. Have a plan
Yes, we’re talking about tracking plans again (see, we told you they were important!). The wonderful thing about using tracking plans is that they are tied to your business goals. The data you collect will let you know straight away whether you’re on track. That said, the data must be standardized across your products. The best way to standardize the data you collect is with a tracking plan.
What we did to begin creating a data stack for HN was define the different kinds of business objects that hacker news contains as well as the actions that can happen with these objects. This gave us our trackable events and put us on the right path to tracking the different kinds of things that can happen to a post made to HN (i.e. upvoted, downvoted, flagged, deleted, dead, etc.). Why is this important? Applied to a business website or software product, this principle will reveal how users are engaging with different experiences and products. This actionable data then informs business decision-makers what’s working well and identifies opportunities for growth.
2. Ensure consistency
Using a great tracking plan will ensure data quality and consistency. Your tracking plan should specify three critical factors:
- The events you will track
- Where they’ll be tracked
- The reason for tracking them
This will help enormously when it comes time to clean and analyze your data. And a tracking plan also serves as a roadmap, a project management tool, and a single source of truth in your business.
To achieve consistency with our HN data stack, we focused on tracking engaging posts. We really wanted to see and know what an engaging post on Hacker News looked like. So, we defined it. 10 upvotes and three comments got the tick of approval as an engaging post. And then we could begin to look at total amounts of upvotes as a means of consistent comparison. We can then see which categories of posts (i.e. products in the business world) were performing much better than others based on their totals. Now, this is much like creating a fully event-driven system where we must have consistency to end up with totals. Which is important for the next principle.
3. Communicate actionable data
Communicating crystal clear actionable data insights empowers businesses to make strategic, data-driven decisions. The goal of data analysis should be to find the compelling story, the gem of wisdom, the insight hidden in the data. And to communicate it compellingly. As data analysts, our job is to make the data impactful for the uninitiated. We need to give the teams in different departments of an organization actionable data they can use to make their jobs easier, whether that’s in sales, marketing, logistics, or customer support.
What we found out about user interaction with Hacker News was quite revelatory. If you would like to know what it was, check out this video. Suffice to say, after creating a data stack for HN, we could now easily create a better pipeline of users who are on hacker news and get them consuming more content. And in the business world, this means converting them into customers. Which is the plan, after all.
Final thoughts
If you would like to put these principles into practice in harnessing the power of data to help your business grow, we’d love to hear from you. If you are just starting out on your journey towards becoming a data-driven business, we can show you how our expertise can help you fast-track you to your destination. And if you’re not sure what you need, why not take action and schedule a free 30 minute call? It’s only when you put data into action that your business can grow.