I hope everyone enjoyed their Fourth of July! Due to the holiday week, I decided to push this out a week later than normal but it’s a good one!
Serverless is growing. In fact, according to Omnia, serverless computing is the fastest growing cloud computing service. The same report anticipates growth to $41bn by 2028 (currently at $19bn)!
But that’s just compute. I would be very interested in learning more about these numbers if we were to include serverless data and serverless AI. One thing I loved about this was how the report included serverless containers and DevOps tools, not just Functions-as-a-Service (FaaS).
This report comes out as InfoWorld writes and article about serverless fading away. Unfortunately, their information is very inaccurate. They define serverless as FaaS. Now in fairness, FaaS was the original serverless offering. I discuss this in my history of serverless blog post. However, we are way past FaaS being the only definition for serverless.
I would invite the author to review more of my blog to learn some of the awesome things happening in serverless. In fact, let’s cover the last three weeks.
Meta Loves Serverless for AI
Recently Meta Engineering released a blog post detailing their usage of serverless Jupyter Notebooks. Meta uses a Jupyter Notebook platform known as “Bento”. It leverages WASM which you may remember from a previous blog post.
You may not be familiar with Jupyter Notebooks so let’s level set. In simplest terms, it is a web-based interactive computational environment. It allows users to write Python programs in their browser and execute them using a specific Python kernel. You can then export these applications as “notebooks” with the “.ipynb” extension.
It was still called IPython when I was first learning Python in the 2000s but it has since evolved to include Julia and R along with Python (hence the name Jupyter). Now it has taken on it’s own life and is incredibly popular with data scientists. One of the artifacts that data scientists store are ipynb files.
Meta wanted to simplify this process for data scientists. Naturally, serverless was the perfect fit. As mentioned, they went straight past containers and are building using WASM architecture. This may be the first real example I have seen using WASM for AI/ML type workloads. This is a next level serverless in my opinion!
Even OpenAI loves serverless RAG
OpenAI recently announced that they acquired Rockset. I actually just learned about Rockset but apparently they are a leading data analytics platform. Back in April of 2023, Rockset introduced a serverless vector search service. This platform can now serve as a Serverless RAG offering for OpenAI.
Earlier this year, Pinecone (arguably the most popular vector database) created a serverless vector database for similar reasons. This provides the basis of a serverless RAG offering. It only charges for computing resources when they are utilized, true serverless. This made waves in the GenAI world when announced and we have seen many competitors come up as a result.
OpenAI’s primary business is ChatGPT. As with any foundational LLM, end users want to customize their experience. RAG is arguably the best method for doing this. It only makes sense that OpenAI would want to offer a serverless RAG to their users. It will keep them in the OpenAI ecosystem.
Databricks Embraces Serverless
Databricks is going serverless! The whole platform. At least that’s what Ali Ghodsi said at their Data + AI Summit.
Databricks is a major data, analytics, and AI company that was founded by the original creators of Apache Spark. They pioneered the concept of a data lakehouse, a platform that combines data warehousing with AI. While not a public company, they are a MAJOR player in data warehousing and analytics and have been a big player in AI/ML as of late. They even released an open LLM called DBRX (which may also be their stock symbol when they do IPO).
Databricks currently operates on all three major public clouds. In order to run Databricks, historically you did have to setup the compute layer on your own. There were a lot of tools to help automate it, but it still required you to “turn knobs”.
Databricks has designed their own Serverless API abstraction that handles the compute for their end user, regardless of what cloud you use. This is a game changer in my opinion. One of my biggest complaints about serverless (in particular PaaS and FaaS) is that you almost always experienced vendor lock-in as the platforms tend to be highly opinionated and proprietary.
Serverless containers is a way to break this system since containers bundle a runtime with application code. Databricks has found a way to make it work for their users. It’s still a proprietary system but it sounds like it can be deployed anywhere.
Databricks has recognized that their customers just want to use their product, they don’t want to have to provision or tune anything. This is why Databricks loves serverless.
Capital One and Serverless Transformation
Digital Journal recently did a piece on Sateesh Kumar Undrajavarapu, a senior manager of software engineering at Capital One. He is actively trying to advance the banking industry by embracing serverless technology. This is an interesting story because Capital One is not a tech vendor or tech company. It’s a financial institution and an end-user.
The finance industry is not exactly known for its advancements in technology. In fact, it is estimated that 43% of all international banking systems rely on COBOL, a programming language that’s largely considered “dead” as very few people are actively learning it.
However, serverless has the potential to totally reshape banking operations. By decoupling the application development from infrastructure management, they can scale better to fit their customer’s needs. It can also improve developer agility by removing distractions. Undrajavarapu successfully migrated Capital One’s critical applications to AWS serverless architecture which helped them achieve 99.99% service availability and reduced production incidents by 80% as stated in the article.
In addition to this, they have found tremendous cost savings. As Undrajavarpu states “Serverless technologies have allowed Capital One to innovate and scale operations while achieving substantial cost savings”.
But of course there are challenges. I am not going to sit here and tell you that the move to serverless is all rainbows and butterflies. Capital One had a legacy architecture that was presumably largely monolithic. Refactoring that takes times and can cause friction and, quite frankly, headaches. But with great leadership, companies can follow Capital One’s lead. I mean, if a bank can figure it out, anyone can. Not to throw shade at banks, but again, they tend to be slow to adopt new technological concepts. It’s not their fault, regulations contribute to heavily to this.
But again, if you take one thing away from that story, it is possible to modernize legacy applications to a modern, cloud native, infrastructure.
Another Serverless SQL Service? Duck Yeah!
Let’s close today’s newsletter talking about a startup. Admittedly, I had never heard of DuckDB prior to reading this article. But I fell down the rabbit hole and wanted to share this story. DuckDB is an open source SQL database that is great for analytics.
MotherDuck is a startup that is responsible for the creation of DuckDB and they are commercializing it. It’s similar to how MongoDB is both a company and an open source (sorta) database. Their serverless database is now GA. It removes the overhead of running a database. Focus on usage, not on infra.
While it’s still too early to see what this adoption will look like in the long run, it is yet another example of a serverless database company entering the marketplace. Why are there so many serverless data startups lately? Why are they getting funding? There is clearly a need and a desire from enterprises to simplify their database usage while controlling costs and these companies are filling the gaps.
One thing I have seen a lot of these past three years or so is customers trying to reasonably reduce their cloud bill. They have no problem paying for what they are using but they want to see if there is any ways that they can optimize their usage to help drive down costs.
My hypothesis is that this is what’s driving the popularity of serverless lately and why we are seeing these statups.
Final Thoughts
Some people are saying that serverless is dead and to them I say that they just aren’t paying attention. I shared some news stories that include startups, pre-IPO, public companies, and even a legacy bank, all of which are adopting serverless architecture in one way or another.
Again, I think some people’s mindset is stuck with the idea that serverless = FaaS and that’s probably what’s driving these misconceptions around the industry.
We need to decouple serverless from FaaS and get back to the original meaning of serverless. Serverless is not about FaaS, it’s about obfuscating infrastructure from the developer so that they can focus on code, not infrastructure. If we can shift the paradigm, we will be amazed at what serverless can do!
—Photo courtesy Emiliano Arano on Pexels—