First Look at Kubecost

Getting detailed costs for Kubernetes is not a trivial task. Most solutions don't understand Kubernetes and so give you one lump cost for the entire cluster. As Kubernetes continues to be a platform for building other platforms and must support multi-tenancy let alone multi-team, a lump-sum type of approach just isn't good enough any more. What we need is a tool that natively, deeply understands Kubernetes and also understands usage in order to assign granular cost. In this article, I'm taking my first look at Kubecost, an application designed to deliver just this.

I joined the company behind Kubecost, Stackwatch, around the middle of August 2023 and so am about a month in. Prior to that, I spent some time with Kubecost to understand how it works and even began to dig into some of the details. Costing is near and dear to my heart having spent many years as a consultant to enterprise companies employing VMware's cloud technologies. One of the key technologies in use in those environments was some cost control application. For many, this wasn't a nice-to-have capability but a need-to-have. In fact, some of the companies for whom I built cloud management platforms lived and died by that cost information. And often times this devout need wasn't even driven by chargeback but sometimes just showback (or what we called "shameback"). That all worked well for things like virtual machines and cloud resources like Elastic Load Balancers and other PaaS offerings, but Kubernetes has historically been a black box. With the rise of more advanced cluster provisioning tools and workflows, practically gone are the days when companies converged everything into a mono-cluster architecture. Multi-tenancy, whatever the definition, involves disparate entities or organizational concepts sharing the same space and so costing has to take that into account.

This anti-black-box-ery is where Kubecost can be extremely useful. Kubecost is able to get granular usage information on the entire cluster--down to the container level--and pair that with either your own custom pricing, standard pricing, or--get this--your cloud provider specific bill and show you exactly what is costing you how much. With support for all the major cloud providers and, obviously, on-premises clusters too, Kubecost can slice and dice your costs in a load of different and highly useful ways as you'll see. If I had to provide a concise, non-nonsense list of the major features, it would be the following:

• Detailed cost breakdown of Kubernetes: Get as detailed as per-label or per-container understanding of costs in a Kubernetes cluster.
• Standalone and federated topologies: Get costs from the cluster on which Kubecost is installed or all of your clusters funneled down to one installation, it's your choice.
• Cloud services costs out-of-cluster: Connect Kubecost to your cloud provider and let it figure out what things other than Kubernetes are costing you money.
• Optimization and right-sizing recommendations: Let Kubecost tell you where you can save money, how, and even take steps to get you there automatically.
• Spending alerts and governance: Set budgets and send alerts when spend is going too high, budgets are overrun, and others.
• View and send reports: No monitoring tool would be complete without them. Get, view, and send reports of all kinds on Kubecost's findings.
• Fully private data (no sharing your costs with someone else): Your data stays with you. Nothing gets sent to Stackwatch, your cloud provider, or anywhere else.
• Deployable on-premises and as SaaS: Install Kubecost in an air-gapped environment, self-hosted on your cloud, or just throw the agent on all your clusters and point to Kubecost's SaaS instance and call it a day.

Another awesome thing about Kubecost which I wish more vendors did was provide you a fully-functional sandbox without you having to lift a finger or provide so much as an email to use. Kubecost has a public demo instance which I'll be using for the remainder of this article and so if you wanted to kick the tires yourself with no investment in deploying anything, this is the way to go. This is a "first look" article so I'm not diving deep into the weeds here.

When you first login, you'll be presented with the Kubecost dashboard showing you the overview of your spending at a glance.

What's nice about this is it gives you all the pertinent, high-level information immediately and everything focused on the last week. You've got Kubernetes costs (for all connected clusters if you have the enterprise version), total costs including non-Kubernetes, possible monthly savings, and your projected efficiency. You can scroll down to see more breakdowns by things like Namespaces, cloud costs, and even network costs.

On the Monitor page under Allocations, you've got a breakdown of your Kubernetes spend by default shown per-Namespace for the last week. At first glance, this might look fairly straightfoward, and it is, but one super valuable thing to call out that you're seeing these costs across all your clusters simultaneously (if you're on enterprise). So, for example, this default Namespace view shows you what that Namespace is costing for all clusters being monitored with the same name. This means your dev, staging, and prod clusters which share the same Namespace for designations like apps or teams get aggregated into a single view. That's huge if you've ever had to manage multiple clusters like I have had to in the past.

You can also change the view to aggregate multi-dimensionally. For example, if you wanted to look at per-Namespace costs also per-cluster, you can do that with the selection filter.

Moving on, the Assets page under Monitor shows you all of the infrastructure-level components (your assets) which are the cost drivers. This is where you can see things like what all your Nodes are costing you, what your individual cloud providers are charging per-account, etc.

The Clusters subpage shows you, not surprisingly, what your individual Kubernetes clusters are costing and if they're still reporting data.

And, finally, Cloud Costs show you a breakdown of your cloud-provider's non-Kubernetes spending which allows viewing on a per-service and other basis. It also maps how much of those resources are being consumed by Kubernetes under management which is very useful.

The Reports page has your quintessential reports which basically allow any of the prior views you saw to be captured and viewed or sent as a report.

You can configure these reports as part of the installation process (via Helm) which is a nice touch especially if you're looking to automate the deployment of Kubecost across individual clusters. Kubecost also gives you the ability to create custom advanced reports by diving directly into the metrics if you need even more power.

Over on the Savings page, you can see that based on the observations Kubecost has made of usage and deducing costs from that usage all the ways it can save you money. Many tools have something similar, but this is the first one that's both Kubernetes focused as well as cloud focused. Here, you can see that it's offering many different options which all result in some savings whether that be rightsizing the resource requests figures in your Pods, removing derelict workloads, and making changes in your cloud account such as using reserved or spot instances.

When you go to the Alerts page, this is where Kubecost allows you to configure alerts based on a variety of conditions including budget/spending conditions, status changes to Kubecost itself, and even changes to the health of your cluster which Kubecost is also periodically checking. And in addition to the standard email recipients, you can configure Slack and Microsoft Teams webhooks which is a nice touch.

The Health page shows a health score of common Kubernetes misconfigurations and runtime events such as resource pressure, OOMKilled Pods, non-highly-available control plane, and also a couple cost-related ones. Although you might have other monitoring applications which handle most of these things, it is handy to be able to get another tool's viewpoint on many of the common problems which tend to arise when running Kubernetes in production.

Everyone likes Budgets and so Kubecost gives you the ability to define a budget based on a Namespace or cluster over on the Govern page. This is a fairly new feature so I would expect so maturity around this ability, but it does satisfy at least what I've seen as the majority of the budgeting requests.

That's it for the first look at Kubecost but hopefully you can see that it can be quite a useful tool. In later articles I'll dig into each of these sections in more depth so stay on the lookout.