Prototyping for Data Teams
Hello again!
Hello, welcome back, and thanks for reading! If you enjoy this issue please consider sharing or subscribing.
We’re Back!
I took a break from this newsletter to work on some of the projects I wrote about in this issue. I’m trying to make time to write again and part of that process is trying to nudge this newsletter towards a shorter and more sustainable format.
I think the tone I’m aiming for is less like the comprehensive Medium-style guides of my first issues and instead more like informal sketches of ideas I’m kicking around. Something more like an email you would send a friend when you come across an interesting article or have an idea you want to share.
You’re that friend. We’re data friends now.
Prototyping for Data Teams
The main project I’ve been working on for the last few months is organizing some thoughts around the role of prototyping for the modern data practitioner.
My thesis goes something like this. The expanding role of modern data teams, powered by executive-level buy-in and self-service SaaS tools, had allowed data teams to take on projects that are higher impact and more complex then their previous work. This is interesting and exciting but because the stakes are now higher the costs of building the wrong things are also higher.
One of the ways to reduce project risk is to create a prototype. But to prototype a data product (even something as basic as a dashboard) you often have to use data that is somehow approximate, simulated, or outright faked. I think this flies in the face of the natural tendency of data teams to obsesses over data accuracy and as a result we tend to skip this step. The result is that we don't really start validating if we've built the right thing until after we've built it, at which point we’re already dealing with a sunk cost.
My first public iteration of this idea came last month when I spoke at the Data Quality Meetup hosted by DataFold. It's a great event I encourage you to check out. You can read a write-up of the entire event and see my section in the video below.
Hitting the Conference Circuit
I read a blog post years ago by someone who was a frequent speaker at tech conferences about how they approached the conference submission process (unfortunately, I can't find the link anymore). The author's method was to pick one topic at the beginning of the year, make the best presentation they could around that topic, and then submit iterations of that talk to every conference for the rest of the year. It’s a simple concept but it was mind-blowing to me at the time.
This newsletter has already gotten me the habit of publicly articulating my ideas and then revisiting and rehashing them. From there, it was a natural progression to bring the same approach to my conference presentations. Case in point, I submitted an updated version of my Data Quality Meetup talk to the dbt Coalesce Conference in December. And then again to the Fivetran Modern Data Stack Conference in September.
You can read my Coalesce Conference submission here and watch my pitch video here.
Submitting to conferences is scary. You could get rejected, you could get accepted and not be happy with your talk. And there’s nothing like watching a video of yourself speaking to make you realize what a weirdly twitchy and awkward human you are. 🤪
Submitting to conferences means opening yourself up to failure. So let’s talk about failure.
Failure as a Goal and Failure as a Process
2 years ago when I was last on the job market, I read a blog post about someone trying to get to 100 rejections in their job hunt. I can’t find the original post, but it was certainly inspired by this TED Talk. I know TED talks are something of a meme these days but this was a useful nudge for me.
I realized that if I wasn't getting rejections then I wasn't being ambitious enough in my job search. So even thought it left me anxious to the point of insomnia, I applied to everything I was even remotely interested in: from seed stage startups to billion-dollar companies, and from VC firms to boutique analytics consultancies.1 It made my job hunt much scarier but also more interesting and fulfilling. I think I ended up feeling much more settled in my final decision as well.
It’s in this same spirit I’m intentionally telling you about the conferences I’ve applied to without knowing if my talks were accepted.
I’m doing this because I think it’s healthy to make sure we’re “failing” just a little bit, and even to loosen up our concept of what counts as success. I’m doing it because I want to choose to celebrate the hard work I put into this process and what I was able to produce.
Let me be clear, I hate failure and rejection. I’m naturally anxious and risk-adverse. I mean, I leave the house with two raincoats. But I’ve been learning to let some of that go when it doesn’t serve me.
Lastly, I hope it’s not presumptuous of me to assume that you might need the same reminder to put yourself out there a little more.
Seeing as we’re data friends now.
Odds & Ends
Around the Internet:
3 Things: This newsletter is increasingly influenced by the format of my friend Elaine's "3 Things" newsletter. Concise, original, and consistent, Elaine is a VC who digs into 3 business rabbit holes each week she finds interesting. The breadth of her ideas are astounding to me and I admire her delivery.
Listening:
Flow State: If I have your phone number and you’re into music I’ve probably already texted you about this one. “Flow State” is a substack that comes out every work day with “2 hours of music perfect for working”. They lean heavily into ambient, experimental, electronic, and jazz artists, which minimal lyrics and more upbeat selections on Friday. My only complaint is that its too much content!
Reading:
Irrational Exuberance: Robert J. Schiller is a noble laureate in economics and Irrational Exuberance is probably his most famous book about behavioral economics and specifically the wisdom/folly of crowds in pricing assets such as stocks and real estate. The topic is fascinating but this book is a textbook in disguise. Incredibly exhaustive and discursive I was barely able to get though it even though I’m glad I did.
The Box: I enthusiastically read a whole book about shipping containers in case you wanted to know exactly how much of a nerd I can be.
90% of Everything: Another book about logistics, this time about the shipping industry. I thought this was a nearly perfect non-fiction book. Informative but not overly serious and with a narrative arc that doesn’t overshadow the content.
I should acknowledge there’s an abundance of research that shows that as a straight white male with a nondescript name that obscures my immigrant identity and a traditional STEM background, I’m generally taken much more seriously as a candidate for a broad range of positions than folks from other backgrounds.