Castles in the Sky
What I’ve learned in my startup journey with HashiCorp
“Oh tell me why
Do we build castles in the sky?
Oh tell me why
Are the castles way up high?”
- Ian Van Dahl, Castles in the Sky
Silicon Valley is a place built on dreams. And there’s no bigger dream powering the Valley than being part of a successful startup and building technology that changes the world.
In 2016 I joined a small startup called HashiCorp. The company was founded by two University of Washington CS majors I deeply respected, and as their first dedicated product manager it was my job to help the team build a product and company that balanced supporting the open source community while competing against some of the biggest names in enterprise security.
Five years and a lot of hard work later I had the extreme privilege of joining them and some of my colleagues in ringing the opening bell at NASDAQ during our company’s IPO.
I write for two reasons: to tell stories I think matter and should be told, and to reflect on my own experiences to try and learn something lasting from what I’ve seen and heard. As I’ve spent the last month processing this last half decade of my life, I’m surprised how the common narrative of the Silicon Valley startup dream is at times very different from what I experienced.
Being part of building something like HashiCorp is an extremely rare privilege. I’m so grateful to have been blessed with this journey, and I want to share my story: what it felt like, what I learned along the way, and how I feel moving forward.
On My Way
I joined HashiCorp as the first product manager for Vault, HashiCorp’s flagship security product. But why I was recruited to serve as Vault’s PM is due to an education and series of experiences that began almost a decade prior and an hour south on US 101.
Like most computer science students at San Jose State University in the late 00's, I originally studied CS because I erroneously thought it had to do with computers.
What I found instead was that it was actually a field of applied math. Unfortunately I sucked at math. I failed Calc 2 the first time I took it, barely scraped by in courses like discrete math, and was so disillusioned by the time I was into the meat of my upper division CS courses that I was more excited about economics than I was computer science.
CS 166: Computer Security changed that. A popular elective taught by a hilarious former NSA cryptologist, CS 166 introduced concepts like information warfare and cryptography via coding and hacking practicum.
Cryptography changed how I looked at math. Crypto wasn’t some austere or esoteric “math for math’s sake” exercise. It was an art: a creative exercise where you used whatever you could from math to defend secrets or steal them. Cryptography made math real for me, and it drove me on a wild journey through the rest of my college and into my career in tech.
Back then, the primary place you applied crypto was in defense. I explored the military via USAF ROTC, but codebreakers were far less important than linguists for the military at the time. Rather than continuing a commission that likely would have had me working in HUMINT instead of SIGINT, I took a different path.
At 22 I became a product manager at NetApp. NetApp had an amazing internship program, and thanks to NetApp’s CTO I was allowed to interview for a role as NetApp’s first PM for Product Security and Encryption.
My exposure to defense, education, and likely my low comp asks as a new college grad helped me get the job. And given security was largely a cost center for most organizations at the time, having me work on off broadway areas like filesystem encryption and federal compliance regulations would seemingly keep me in “safe” parts of the company while I learned our business and product management as a whole.
At least, that was the plan. A week and a half after I graduated, Chelsea Manning’s contributions to Wikileaks highlighted a glaring lack of multimedia encryption within the DoD. Then someone hacked an Iranian nuclear facility to cause a nuclear meltdown. Not long after, a state intelligence organization hacked RSA (the leader of enterprise cryptography software in the world at the time) and used data they stole to steal secrets from American aerospace defense contractors. And so on.
Security and cryptography became front and center to NetApp’s business. As a major federal contractor for the United States and other NATO governments, building encryption technology to respond to the threat of state sponsored hacking became elemental to our success.
I was blessed with the experience of a lifetime. I was an early-twenties PM who went from working in the back room on FIPS 140 compliance to attending classified meetings and working alongside the co-founder of a Fortune 500 company to build technology to stop spies and professional hackers from stealing the most sensitive secrets in the world.
My experiences at NetApp later drove me into an adventure across venture capital and my first foray at a security startup. While I bounced around Silicon Valley for most of my twenties, I spent that time focused on building/investing in infrastructure technology for security and cryptography.
Six years after I graduated college and joined NetApp, I found myself coming back to a role frighteningly similar to the one that launched me into my career.
All thanks to a life-changingly terrible breakup.
Lost
In 2016 my life changed. My girlfriend and I broke up, and with our relationship went my passions for my career.
I wrote about what I felt personally. But there were significant professional implications to our breakup. My ex was/is an amazing fund manager. This made her one of the largest pursekeepers in the United States to private equity and alternative asset classes: including and especially venture capital.
Back then I was a VC principal — a type of intermediate rank between the junior position of an associate and the vaunted role of partner. This deserves a blog post for itself, but the role of principal or its equivalent is a difficult one. Your job is a decade-long paper chase: you hunt to acquire enough big “wins” (i.e.: funded startups that exit in a way where you can be personally attributed as a first investor) to earn a partnership spot.
This means that for 8+ years you remain on point: fighting for investments without the ability to sign your own checks, balancing inter-firm politics, and trying to find ways to not go insane while waiting your turn for the fund to get large enough to support another partner or someone to leave.
The latter is extremely difficult. Seemingly everyone recruited into Silicon Valley venture capital shares a type of high-functioning neurosis where we constantly seek an adrenaline cocktail of personally attributed success mixed with the knowledge we’ve made an impact/done a “good job.” Having this be actualized in your job requirements and having to wait for half a decade or more for proof we’ve “done well” can be maddening. In fact, I’d argue some of the famous VC outbursts over the last 10 years are great examples of this.
When my ex and I broke up I lost my passion for that paper chase. Being in venture meant she and her colleagues were going to be a constantly visible part of our small corner of private equity. And struggling with depression, it became harder and harder to muster the strength to steel myself for a decade+ long fight to prove the world wrong with my investment thesis.
It became clear I needed to do something else.
I tried to find a time in my life where I felt personally and professionally fulfilled by my career. I found that time in the work I was doing at NetApp. I missed building technology that protected the world’s secrets. I missed the constantly changing landscape of security and the artistry of cryptography. And most of all, I missed working with a team of people who were passionate about doing all of that too.
My VC firm at the time, Amplify Partners, was wonderful enough to give me the space and funding to find something that satisfied the above. I explored everything: going back to NetApp or similar companies like Cisco to pick up where I left off, doing crypto/security product management at the FANGs, and even explored going back into the government proper.
Candidly I was reluctant to do the latter given some of the implications it’d have on my life. But I needed to rediscover my passions that got me into tech in the first place. All options were on the table.
As I measured these different paths, I sought out guidance from someone I’d been quietly talking to for the last few years. Armon was the co-founder of HashiCorp, a company I’d sought out 3 years earlier as a potential investment for the VC firm I was an associate at. He and his co-founder Mitchell were building some of the most progressive open source technologies in the world and at the bleeding edge of building infrastructure technology.
Armon and I met during a happy hour at one of HashiCorp’s early investors, True Ventures. We got along well as some of the only people there who wrote code there, and over the years I sought his advice on investments I pursued or technology I was working on as an operator.
I met up with Armon at a coffee shop in the Mission to chop it up and ask for his advice. But this time I wasn’t dilligencing a company. I was dilligencing my life. I asked him about what he thought about the opportunities I was looking at, and told him what I wanted: to go build secure systems to stop hackers alongside people who were as nerdy and passionate about that pursuit as I was.
I remember Armon calmly and quietly listening to my rambling, drinking his coffee, then leaning in across the table to ask with a visible smirk:
“What do you know about Vault?”
HashiCorp Vault was one of several mid-’10’s projects to bring a new type of security software, secrets managers, to the then-growing world of developer and cloud infrastructure. Unlike other projects it was both aggressively open source and abstracted the cryptography necessary to protect those secrets.
Secrets managers were, and remain, disruptive. Prior to them, you handled secrets like credit card numbers or passwords differently depending on that data’s intended purpose. Accessing one secret or another on the same software or hardware might entail using a different API or different identity infrastructure, ensuring there was often a lot of complexity in protecting sensitive data.
For example: to write an application to pull a password encrypted within/by a Hardware Security Module (or HSM, basically a big secure computer for protecting very important data with strong cryptography) you would typically need to do the following:
- Deploy a LDAP or Active Directory server to manage identities and the rights that each identity would have to access certain secrets. If you already had identity, federate that identity to your new system or that HSM. This is something you’d deploy with a lot of Powershell, a .NET language of your choice, or in the UI.
- Setup all of those identities and access controls to access those secrets. Again, lots of Powershell or UI work but now potentially including C/C++ depending on your HSM manufacturer.
- Actually write the application that, given a verification of identity, requested the secret. For most HSMs this was something written in Java or C++often using libraries rarely updated as they got higher on the stack. While there were/are protocols like PKCS#11 and KMIP that made this less vendor-specific, the quality of vendor-supported libraries for these open protocols often meant it was safer to write code in the native libraries for each HSM.
Basically if you wanted to write a simple application to get a secret, you needed to be fluent in 2–4 programming languages and spend a whole mess of time and money to do it. Knowing this, it makes sense why so many teams and organizations didn’t encrypt their passwords over the last 15 years: it was a huge pain in the ass.
Vault grew out of a recognition that for a world where we were using clouds to host and develop software this experience wouldn’t work. Uniquely, Vault also didn’t care what clouds (or really what systems in general) it ran on. Mitchell and Armon were absolutely brilliant in that they completely abstracted Vault’s underlying architecture: how it stored data, identified a user and their rights/privileges, and protected secrets with cryptography.
Basically, Vault could/can run anywhere: across on-prem, clouds, even Raspberry Pis. And you didn’t need to know a ton of programming languages. You only needed to pass REST API calls, which ensured with Vault’s popularity a growing number of popular fameworks/libraries had native support.
Vault essentially spoke whatever language you wanted and ran on wherever software or hardware you had available.
As someone who grew up using HSMs and writing code there in the grim darkness of 2010, Vault was godsend. Armon explained that HashiCorp was looking to explore some of the same kind of enterprise infrastructure problems I dealt with at NetApp, but using Vault as the vanguard for them. And that he was looking for a product manager who was as excited at working both in the world of the Global 2000 as well as responding to tickets on GitHub and vibing with the open source community.
Three days later, I started my interview cycle at HashiCorp.
Feels Like We’re Falling for the First Time
Right off the bat, it was clear that I was a little different than most of the current employees at HashiCorp and in the Vault community proper.
Mitchell and Armon rightfully recruited from their friends, colleagues at Kiip, and the open source community for most of the first 50 employees at HashiCorp. This ensured that the HashiCorp I was interviewing for was a lean, engineering-focused place where everyone already sort of knew each other and shared a similar developer-focused perspective on the world.
In comparison, I was the epitome of an evil suit. I was technically competant enough to keep up. But I’d spend most of my career building enterprise technology and often for groups like big banks and the military. My limited professional experience with the open source community was focused almost wholely on threat intelligence — essentially writing code to share the raw data identifying hackers among a few dozen other security researchers.
This is very different than collaborating with the community on managing a project like Vault, which today has a thousand contributors and a myriad of other open source components managed by teams numbering in the thousands to tens of thousands.
HashiCorp’s collaboration in the open source community meant that I’d have a lot to learn as a PM. The community was/will never just be a place where we draw code we want to make Vault better: it’s a living, breathing stakeholder that all of HashiCorp works with to solve problems. That’s a complex challenge that I never faced in my experience as an enterprise software product manager.
There was also a major cultural difference between me and a lot of the early HashiCorp team. A lot of my colleagues were the cultural personifications of the San Francisco startup hacker. In comparison, I was an ex-enterprise IT product manager who was coming back into software from venture capital. My technical experience was in using software and hardware considered legacy or “non-target” by some of interviewers. And even though I was able to get through the rigorous process, it wasn’t unbloodied.
At times it felt like I was being hired as an IBM employee into early Apple. It was like I was entering the room thirty seconds after Steve Jobs extolled the Macintosh team as pirates raiding IBM and Microsoft’s fortresses while wearing a Royal Navy outfit with a DOS hat on.
No greater/more hilarious was this than when I did my first tech talk three days after I joined.
While giving a presentation on state sponsored espionage and cryptography, someone in the back of the room shouted a question about Vault’s cryptography and accused it of being weakened to help the NSA better hack into it. Given my experience as someone who worked with the government, I was almost certainly there to help ensure Vault was “backdoored” for the US to spy on everyone.
Nobody really knew how to handle that one. We all kind of silently watched as the guy kept on with his conspiracy theories as he went for some of our free pizza, and kept facing us as he walked out the door. In one of the more awkward professional experiences of my life, I had to reassure the remaining group that I wasn’t a government plant and that Vault’s crypto was all open source and auditable.
Armon later laughed after hearing about this. “Congratulations on successfully infiltrating HashiCorp,” he joked.
Run those Red Lights
I had a lot to learn as Vault’s PM. And I had to learn it fast.
Luckily I wasn’t alone. I was joined on the Vault team by three amazing engineers: Jeff Mitchell, Vishal Nayak, and Brian Kassouf. All three had extensive experience working on Vault prior to me joining, with Jeff recruited previously as Vault’s first engineer-cum-product lead and having authored significant portions of Vault himself.
One thing I appreciated a lot about HashiCorp — and still very much appreciate — is the insane quality of our engineering team. All three of Vault’s early team members were more than comfortable doing traditionally “un-engineer” things like working with customers on support cases or even attending pre-sales meetings with me as we sold what would become Vault Enterprise to the Global 2000. They are passionate, empathetic, and amazing colleagues.
These qualities were essential, because I frankly wasn’t prepared for the pace of HashiCorp circa 2016/2017. Vault exploded in popularity at this time, and it became clear that it was the opportunity we needed to build the enterprise business we wanted to support the company. To do that meant a lot of work, and as Vault’s PM I initially balanced product marketing, traditional product management (which also was complicated by learning how to work with the OSS community), directly supporting partnerships and sales, and even legal compliance.
To be blunt, I dropped a lot of things. I’d forget to send emails. I didn’t attend community meetings. I struggled to keep up with the demands of my role, and at times even Armon admitted he was disappointed with some aspects of my performance.
The undersung heroes of this time was my team. Jeff, Brian, and Vishal all stepped in to help me balance the swaying house of cards. At times they all played Vault’s “Open Source” product manager, working with the community to elegantly deliver new features and improvements in ways I learned from/emulated.
I grew a lot personally and professionally during all of this. To keep pace with the job I became much more organized, and learned time management out of a necessity to squeeze every minute I could out of the day. To best learn what I could I submlimated my ego — learning to ask questions, ask for help, and be open to learning from anyone regardless of job title or role. As a former VC, this was incredibly difficult and involved a lot of deprogramming.
By the end of 2017, the Vault team was somewhere around ten people. We had gone from essentially pre-revenue to well north of $10mm in ARR and were starting to operate as a functional and well-oiled machine. All of that was due to folks like Jeff, Brian, and Vishal — as well as “flex” members of our team like Armon, Mitchell, Seth Vargo, and Julia Friedman who would show up to do “hero coding” sessions to help deliver major features like Autounseal and Replication or deploy infrastructure to keep everything running.
We didn’t feel like a traditional “top down” product team. It was like the Fellowship of the Rings: everyone did everything, some people did things better and rightfully focused on those tasks that involved them, and overall we were all just fighting as hard as we could to throw the ring into Mount Doom.
The Vault team, and the company as a whole, would grow extremely quickly over the next few years. Still this culture of “everyone works hard, everyone matters” remained and continues to be a big part of HashiCorp’s ethos.
A great example of this was in the launch of Vault 1.0 in December 2018. The 1.0 release was a major cornerstone in Vault’s evolution, and we built a new website to christen it. Unfortunately the site broke the night before the launch at HashiConf (our company’s public convention for our products) and we didn’t have enough engineers to help fix the site.
By this time I’d found my stride as Vault’s PM and had no problems with rolling up my sleeves. But as some of the team members and I jumped in to help do our own hero coding at 1AM in the morning, we saw someone else contributing to the private repo: Mitchell. Mitchell had seen the drama unfolding over Slack and without being prompted threw himself in to help.
I remember stopping mid-commit and feeling astonished at it all. There was no ego. Everyone just wanted to see Vault succeed, and we all did what we could to make it happen.
That was nothing short of magical.
Maybe We’ll Try in Another Life
One fundamental part of the startup narrative is the idea that you need to work a lot. No Silicon Valley startup story is complete without long hours and deep personal sacrifices. There certainly were both during the evolution of Vault. But they were never desired, and as soon as humanly possible the team worked to expand to reduce burn out among the ranks.
To do the latter, there was and still is a strong culture of servant leadership at the company. Whether it’s Mitchell working to help build the Vault website or Armon writing a significant portion of Vault Enterprise’s replication code, leaders at HashiCorp were/are supposed to see their position and privilege as a responsibility to support the greater team and make things easier for everyone.
Another example of this could be seen in mastering builds for Vault. Early on in Vault’s history, there was a manual aspect to mastering a release. This often meant someone on the team would have to stay up late (usually midnight to 1:00am) to cut the release and redirect all of our licensing and documentation to the new version.
Jeff Mitchell (then Vault’s engineering manager) and I volunteered to be those people. I’d go into the office at 9pm and, while loudly blasting trance music into the empty building, work with Jeff and Vault’s then-product marketing manager Chris Kent to cut the release.
I’d spend the next day woefully low on sleep and clearly messed up. But I’d be vindicated by the rush of working on a small team to build something big, and as a product manager there is no greater high than shipping.
But this took its toll. It all took its toll. I didn’t know it at the time, but in many ways I was “hiding” in my work at HashiCorp to avoid dealing with the painful realities of what brought me to the company. I quietly struggled with mental health throughout Vault’s meteoric rise, eschewing therapy for the adrenaline cocktail of working 60+ hour weeks by day and raging my face off with my friends to “burn” away the stress by night.
It would take five years, another breakup, and lost friendships to finally force me to go back to therapy. And when I did I realized that I had major issues to work through if I wanted to have healthy relationships in my life.
Yes, working a lot and getting the insane gratification of having a visible and personal role in building one of the fastest growing security startups in history was great. But the way I was doing it was unhealthy. I was self-medicating on that feeling in dangerous ways that made it difficult to be my partner or my close friend.
There are a lot of lessons I’ve learned about building startups from my experience at HashiCorp. But this one is key: we need to stop glorifying overwork. “Hustle porn” is so toxic because it erroneously glorifies overwork rather than calling it what it is: a sometimes unfortunately necessary evil whose personal sacrifices often disenfranchise employees and hurt the people involved.
I’m not sure if I’d do another early stage startup after HashiCorp. But if I did, I’d work as hard as I could to minimize and de-glorify overwork not just for my team — but for me as well.
And I’d definitely do it with a therapist or counselor in tow.
Blue Sky Action
Vault, and HashiCorp, grew faster than I thought possible.
Five years after I joined, Vault had gone from one of a handful of popular open source secrets management projects to the de facto standard for secrets management in the world. Major open source infrastructure like Kubernetes included Vault as part of its reference architectures for its use, and Vault became the most popular secrets manager for open source — as well as one of the most popular open source projects in history.
Our IPO was a special moment in that history.
HashiCorp has grown over 40x since I joined. Vault today is a team of over a hundred engineers, designers, product managers, product marketing managers, and many more working with the community to build amazing things. Boundary, another product I had the pleasure of first prototyping and building alongside Jeff and members of the Vault team, has also joined Vault as another security product at HashiCorp with a stellar team in the dozens.
One challenge with this growth is that I don’t know everyone in the company, and sometimes even my team, anymore. I also don’t get to see all of the early Vault team members as much as I’d like. For example, Jeff (who I worked side by side with throughout Vault’s ascendance over the years) has mostly been working on Boundary while I’ve been focused on Vault and building HashiCorp’s federal division.
Going public was great. Being able to see Times Square emblazoned with all of our products was the experience of a lifetime, and I joined a lot of my early colleagues in getting emotional in the 20-degree weather as we realized everything we worked for, well, matters.
But the best part of the IPO for me wasn’t being able to sell shares or even that experience of watching Times Square and Wall Street light up about our company. It was just getting a chance to spend time with my early colleagues I went on this journey with. Life and work have all taken us in different directions, and being able to just get together to reminisce and vibe was the best part of that entire week.
When I look back on the IPO, it’s not the listing ceremony or even the bell ring I’m going to most remember. It’s Jeff and I spending the better part of an evening at Le Bain talking about TPM chips and remote attestation. It’s Mitchell loudly realizing that I’m probably the reason everyone on Sand Hill Road originally called HashiCorp “VagrantCorp” (yeah, my bad). It’s crying when Armon thanked me for working hard to make people understand why Vault mattered.
At the end of the day, I learned that the best part of building a startup isn’t building technology that changes the world. It’s the people you get to build it with.
You’ll Be OK
I don’t plan on leaving HashiCorp right now.
The IPO, while a lifechanging event for me, is just another mile-marker in our company’s story. We have so many amazing things we’re working on, and I’m excited to join a (thankfully much larger) team to build those things and more over the next few years.
But there are some changes I want to make as 2022 begins. I haven’t invested as much in myself personally as I would have wanted. This means making sure I maintain a healthy work life balance, and pursue supporting myself and my friends with the same kind of energy I pursued in building Vault.
This last part is key. I simply could have not have survived the last five years without my friends. Just as I’ve learned professionally how to be a successful product manager working on an amazing team of engineers I look up to, I’ve learned how to be a good person by exploring this life alongside friends I similarly admire.
One thing I learned as a VC was that founders and other early employees frequently have an identity crisis after an exit. HashiCorp isn’t a startup anymore and I’m not some late-twenties kid trying to tear down the establishment in an economic guerilla war.
In many cases, we are the establishment.
I’ve been working through this preemptively in therapy. I definitely feel the need to redefine my identity given its foundation has shifted.
What I’d like to do is explore building an identity that isn’t tied strictly to my job at HashiCorp. I want to just be a good person, who focuses on learning how I can use my privilege to support making things better for other people who also have to overcome significant challenges to find their place in the world. Whether that’s at HashiCorp as a new product manager, or as a POC/LGBTQ+/URM in tech, I want to explore how I can make the climb more accessible and hopefully a little less painful for others.
And most of all: I want to just be a good friend. I want to spend time reinvesting in my personal relationships, and go on some of those adventures with my favorite people we always dreamed of doing.
TL;DR
I have been blessed with the journey of a lifetime. I got to join a small group of people I admire in building technology that’s changed the world.
That journey at times has been difficult, or at times even painful. It’s taught me that some of the truisms of the Silicon Valley startup narrative were bullshit. It’s made me come to terms with the importance of taking care of myself and others, and taught me the responsibility of using my privilege to make things better for the people around me.
It’s been a long road. But I’m so grateful I’ve had a chance to walk it, and I’m excited to see where it goes.
