Forward, the Light Brigade
Why I joined CoreWeave
Cannon to right of them,
Cannon to left of them,
Cannon in front of them,
Volleyed and thundered;
Stormed at with shot and shell,
Boldly they rode and well,
Into the jaws of Death,
Into the mouth of hell
Rode the six hundred.
- Tennyson, Charge of the Light Brigade
I’ve spent my career in information security focused on the adversary and their relationship with an ever-changing world.
The adversary will always exist. Sensitive data and infrastructure will always need to be protected against them.
But as the world changes technology, and the adversary, will change. In that change there will always be something new to learn: new societal and technological shifts to appreciate, support, and protect against a constantly evolving enemy that needs to be respected but resisted.
The modern world has become inseparably linked with its online shadow. Since I graduated in 2010, computing has become a native part of our daily lives. There have been multiple shifts within that time frame that have further tied modern society to critical online infrastructure: web 2.0 consumer internet, mobile, etc.
So too have the mechanisms to build and run software powering that critical infrastructure. When I graduated, infrastructure (and especially security) was an arcane topic that was divorced from how software engineers built and ran their code. Today there are disciplines dedicated to uniting how developers write and run code: DevOps, DevSecOps, etc.
The adversary has also changed. Most modern adversaries are no longer individuals or small groups. They are professionals directly managed or contracted by governments or multi-national organized crime. The trends of the ’10s have only accelerated, and as seen in the Iran-Israel War the terminal result of many modern Advanced Persistent Threats (APTs) is preparation for/ultimate execution of cyberwarfare against critical infrastructure.
As we enter the middle of the 2020’s I believe there is another tidal wave of change coming: to society, the critical software infrastructure supporting society, and to the adversaries attempting to compromise that infrastructure. That tidal wave is Generative AI.
How I Feel About GenAI
I’m definitely an AI bull. But I’m also a healthy skeptic. I believe in (and have seen firsthand) AI’s transformative impact on fields like software engineering and defense. I have watched how GenAI copilots and agents have enabled skilled human professionals in those fields to become massively more effective in solving problems.
I believe that many technical fields are going to transform into ones where skilled humans are commanding fleets of agents that perform analysis and engage in autonomous operations on their behalf. And I believe/hope that the net result of this generation of tech is us interacting with computers more like how characters in Star Trek do: talking, reasoning, and partnering together to solve massively hard problems for the betterment of humanity and society.
But I’m also a skeptic on the field. I don’t necessarily believe in AGI, nor do I believe that GenAI is or will become a conscious entity anymore than my iPhone’s autocorrect that erroneously believes I want to want to use the word “duck.”
Like other transformative and buzzy technologies, Generative AI is far from without its problems. There are significant legal and ethical issues to solve in how we train and operate Generative AI systems. The industry has also attracted more than its fair share of con men — people who have jumped from other formerly trendy areas of tech to make a quick buck or clout rather than to contribute to the field or solve major problems with AI.
Security is one of those major problems. The systems for training and managing inference for GenAI often utilize DevSecOps infrastructure that has been battle tested (sometimes quite literally) in the last generation of tech: Kubernetes, Terraform, Tensorflow, etc.
But they are being deployed and run in highly novel ways for GenAI. New infrastructure technologies necessary for running training and inference at scale (e.g.: Retrieval Augmented Generation or RAG, protocols like MCP or A2A, etc.) are also just starting to get proven against large, enterprise-grade security requirements and use cases.
The net result of this is that if we’re going to deliver that Star Trek-style world of human computer interaction, and thereby realize the potential societal gains from GenAI at enterprise scale, we now need to rapidly secure a massive new surface area: one where most of the data mankind hosts online is now being stored, used, and interacted with in a novel fashion with increasingly novel technology.
These challenges, when combined with the highly resourced and often state-sponsored modern adversary, provoke a potential existential risk to the success of GenAI. But they also provide an opportunity: one where teams have the ability to create new technology, best practices, and frameworks to secure how humanity is going to operate with computers for the next generation.
That dichotomy reminds me of what I saw firsthand building what would become Secrets Management and Infrastructure as Code.
But the scale is much, much larger this time. And rather than just build software to protect how enterprises hosting most of humanity’s data securely move it onto their hyperscaler’s infrastructures, I’m excited to build a new hyperscaler that securely enables how the world’s enterprises host critical training and inference infrastructure at scale.
Why CoreWeave?
Five months ago I joined CoreWeave because I believed it was the best opportunity to build that new hyperscaler. CoreWeave stood out to me for a few reasons:
The Team
The team at CoreWeave was shockingly technical and driven. During my interview process I met with a broad swath of the company’s security, engineering, and product management leadership.
All of them had previous experience building and running major enterprise infrastructure in marquee, successful use cases. Many of them designed and ran previous infrastructure that CoreWeave’s customers had run on. Some even wrote books that I read at HashiCorp and recommended to my team.
There were many times where I felt like the dumbest person in the room throughout my interviews and first few months at CoreWeave. That was scary, but it also felt great: it meant that I was learning.
More important than all of this, everyone was nice. There is a genuine sense of unity and passion uniting the company to enable the world’s largest enterprises to embrace GenAI and support this next phase of humanity’s interaction with computing. And while things have sometimes been a little exciting, CoreWeave’s “Let’s Fucking Go” culture feels far more like a high-growth startup than the several thousand person organization it has become.
I liked the idea of being in the trenches with the people I met in my interview and thought I could learn a lot from them. In the months since I’ve joined, those feelings have only been reinforced.
The Technology
Most of the other hyperscalers have built inference and training services on their existing general purpose compute infrastructure. This makes a lot of sense: they have invested tens to hundreds of billions of dollars in building that infrastructure, and it was readily available when they needed to build something like a Sagemaker or a GCP Slurm.
CoreWeave took a very different approach owing to the fact that it was created nearly a decade later. From its hardware foundations (i.e.: how literally GPU servers are racked and networked) up to the final presentation layer to the DevOps professional and AI developer interacting with the CoreWeave Cloud Platform, CoreWeave built custom infrastructure tailor-made for genAI and high performance computing.
The result of this is that CoreWeave’s infrastructure has been ranked regularly as the top platform for GenAI in the world for the enterprise. Their SemiAnalysis results and the experiences of developers I found online seemed to prove that CoreWeave’s customized, bare metal approach to building infrastructure specifically designed for AI was novel.
I particularly was impressed by what they had done with CKS, CoreWeave Kuberentes Service. CKS is a customized, API-compliant version of Kubernetes that CoreWeave develops and maintains. It’s unique in that it’s a form of “bare metal” Kubernetes: it does not require a hypervisor to sit between it and the hardware it utilizes to run.
The benefit of bare metal is that it is often significantly more performant than its other hypervisor-hosted counterparts. The drawback is that it’s — to be blunt — kind of a fucking pain in the ass to run at scale: particularly for enterprise grade, ephemeral infrastructure.
If you’re running K8S on proxmox, ESXi, or Hyper-V the problems of securely/performantly allocating and freeing networking and compute resources on a per-job, per process basis can be pushed to those hypervisors. If you’re running bare metal, it is your problem. You must figure out how you securely and efficiently allocate resources to Kubernetes so it can use them for jobs that will appear at seemingly random across a potentially global surface area.
Until I interviewed at CoreWeave, I had only seen a large bare metal Kubernetes infrastructure like CKS successfully operate in two places: within Google and within the US DoD. That CoreWeave had been able to successfully build high performance, scalable, and secure infrastructure run by CKS was a testament to its engineering and product team. And it was incredibly novel technology: both for hyperscalers and within tech as a whole.
So What’s Next?
I’ve joined CoreWeave to lead the creation of their Security Products organization.
This organization builds products and features that enable customers to deploy enterprise AI workloads and sensitive data for those workloads onto CoreWeave. A good example of one of these products is CoreWeave IAM, the company’s multi-cloud native Identity and Access Management infrastructure designed to securely manage authorization to AI workloads across any kind of heterogenous identity infrastructure.
We recently released a major suite of features with CoreWeave IAM highlighting our organization’s unique perspective on multi-cloud security: allowing for example you to automate the provisioning and management of SLURM users/credentials for model training from any modern IdP through CoreWeave IAM’s Automated User Provisioning (AUP) feature.
I’m excited to join some of the smartest people I’ve ever met in leveraging our experiences building some of the largest, most secure enterprise infrastructures in history to power and protect the rise of genAI.
And I’m even more excited to share stories from this journey, and what I’ve learned from defending this new generation of tech against a new generation of adversaries.
Once more, unto the breach dear friends.
