Introduction
There is no one sizes fits when it comes to Azure and cost optimization, but the focus of this session is to explain some tips & tricks during my daily life as a Cloud Solutions Architect
Some general tasks can be done monthly/quarterly to be sure that you Azure environment is up to date, taking into consideration that the optimization and your business run are the most important things here
Be advised that not all the things that can be done in Azure are being covered in this post, probably because at the time of the writing I didn’t have to
Why this post?
Every design in Azure has cost implications, before architecting something, we must consider the budget that we will need for the Project itself, taking into consideration thinks like:
- Identity different boundaries for scale up
- Redundancy
- BCP taking into consideration the cost of the solution
- Design and set up scalable architectures, focusing on metrics & performance
- Start small and scale out as soon as the required performance needs it (I really love that one)
- Choose PaaS and SaaS over IaaS, pay only for what you use as a consumer
- Always, monitor, Audit & optimize the cost related
Ok I get it, but what are we going to cover?
For the next minutes I will explain some guidelines about cost optimization, in particular for the following topics:
- Use of ARI
- Use of Dev subscriptions
- Optimal use of Azure App Services
- Optimal use of Auto-Scale in App Services
- Azure Data Factory Failed Pipelines
- PaaS SQL Optimization
- Cosmos DB
- VM Right Sizing
- Azure Hybrid Benefit
- Blob Storage Lifecycle
- Networking
- Clean Orphan Resources
- RIs
- Use of Log Analytics
- Use of Azure Advisor
- Cost Management Preview (ACO Insights)
- Azure Governance Dashboard
- Closing
Before starting…
Before starting this post, I would recommend to create an Azure Inventory from your environment, with this tool, it is pretty simple: https://github.com/microsoft/ARI
And as you can observe, it will give you a great overview of what type of resources are you having, which use, locations, etc…Also, some of the sheets can be used to optimize your Azure Cost environment
Also, another tip that I want to give, is you can start your journey to Cost Optimization with a self-done assessment, but it can give you some guidelines about where are you: https://docs.microsoft.com/en-us/assessments/
Use of Dev Subscriptions
Using the top-to-bottom approach, the first thing to pay attention to is Azure Dev/Test subscriptions, which are applicable for both enterprise and pay-as-you-gooffers. By placing your dev resources in those subscriptions, you will get lower prices for most common Azure services for the cost of excluding them from the regular vendor SLA commitments.
Optimal use of Azure App Services
First, check that standard Plans and Premium plans has an associated application
I have seen a lot of empty App Services Plan, which leads to unnecessary cost to the customer, remember that having a right governance in your subscriptions it is algo a cost measure.
Another thing that I tend to do is to check the metrics for the plan, and check if are being used properly (scale down in case is needed, but remember the features needed in each case)
Optimal use of Auto-Scale in App Services
In my case, I can be able to scale down my resources, but first check features between standard and premium plans (or even between std o premium!). Also what it can be done is to scale up/down based on a schedule https://techcommunity.microsoft.com/t5/apps-on-azure-blog/azure-app-service-automatic-scaling/ba-p/2983300
Very useful for those workloads where we know that only are needed in certain periods of time
Azure Data Factory
Review the failing pipelines, if a pipeline is constantly failing to run, probably it will impact into the cost of your resource, so take action
Again, I have reviewed a lot of Pipelines in DF which are continuosly failing… take care of that as well
Paas SQL Optimization
With monitor, check if the database needs all the DTUs provisioned, one thing I love to do, is to play with the different available plans for the SQL, if you’re running a version with a lot of DTU’s, implement a runbook in order to reduce the plan when you don’t need it, for example, you can use: GitHub – francesco-sodano/azure-sql-db-autoscaling: This ARM Template deploys an Azure SQL Database with DTU Consumption plan (with a new Azure SQL Server) including all the resources required to perform Auto Scaling (scale up and scale down) based on Metric Alerts using a function app. Again, very useful for those workloads where we know that only are needed in certain periods of time
Focus on those DBs with 40%-80% of the DTU capacity, those are the most imporant to be scale up
Check if you really need the Georeplication, probably you don’t need to replicate your DB across regions (important point!!!), remember the first bullets of this post, we need to start small and then plan big, if we start to put all the georeplication modes to DBs that are not being use or for those in test, you’re wasting your money
Cosmos DB
With the help of metrics, review the use for the correct size & throughput
Consider autoscaling for those type of resources (it avoids consuming unnecessary resources)
Consider serverless options for Dev & Test environments or those environments where intermittent traffic is it used: Consumption-based serverless offer in Azure Cosmos DB | Microsoft Learn
VM Right Sizing
One thing I love to do, is to shutdown VMs based in a Schedule
The Schedule is set up with a tag in the resource, and the operation is done by an Automation Account (it could be a Logic App as well). For example, I love to use the following script:
Scheduled Virtual Machine Shutdown/Startup – Microsoft Azure | Automys
You can setup the following tag in the VMs
And your VMs will automatically shutdown and start in the configured schedule, which for those test and PRE environments where Azure Reservations does not fit, are simply great, you will save a bunch of computation hours with this simply script
In order to cut costs, we can use spot VMs for non priority tasks (it helps to save some money vs other azure VM sizes), you can get more info in: Use Azure Spot Virtual Machines – Azure Virtual Machines | Microsoft Learn
Get rid of those old VM sizes
One thing that I do for all of my clients in order to optimize cost, is to check which version size are running for the VMs, this can be extracted from the ARI (remember the first tool):
Why? Because as you probably know, Microsoft is always optimising hardware in the Datacenter, so they are pulling new version of the VM size, so what’s the point? the older is the VM size, higher VM cost, so check out if there is any new VM size, and you will be able to save some money from each VM size.
Imagine that you have 100 VMs running in a v2 series, and changing from v2 to v5, represents a change in cost of 20€/VM/month, so in total the save is 2000€/month with only changing the VM to a newer version, not bad uh?
Azure Hybrid benefit
First question is: Do you have a software assurance with Microsoft? If the answer is yes, don’t waste more time and money, and apply it to your Azure Resources, it Will help to sabe up to 40% in cost (for VMs and SQL)
If you want to know how much you can save with this, you can use the Azure Calculator for this purpose: https://azure.microsoft.com/en-us/pricing/hybrid-benefit/#calculator
Storage Lifecycle
- Everyone loves to sabe data, but what can be done to save some money? Use lifecycle J Optimize costs by automatically managing the data lifecycle – Azure Storage | Microsoft Learn
- Transition information to a cooler Storage tier, to optimize performance and cost
- Delete information at the end of the lifecycle
With this procedure I was able to save a lot of money in a recent IoT Project, all the information was stored in blobs, but once a certain period of time passed, we moved the information from one tier to another in order to cut storage costs
Networking
Check out costs related with networking, it may scary you
You Will need to identify which applications are using most of the egress bandwidth and review & redesign your infrastructure accordingly
Check which gateways are not being used, probably those which have a throughput lower tan 900MB/day
Check you Azure Express Route Circuits, probably the first provision of the circuit was greater than needed
So, check Azure Monitor: Monitor – Microsoft Azure
Are you sure that everything that you have in your subscription are being used? Use this workbook and take action in your subscription: Azure Orphan Resources (microsoft.com)
Save Azure costs deleting those unused disks, Public IP’s which are consuming Storage and account cost (remember that in Azure Advisor we have these recommendations as well):
I’m sure that you will save a bunch of €€€ with this procedure
Use of Log Analytics
If you’re using Log Analytics to monitor your Azure Resoruces, you should add a Daily Cap into your Log Analytics Workspace: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/daily-cap#view-the-effect-of-the-daily-cap
Also a few tips:
- Use Azure Monitor Agent and Data Collection rules over Log Analytics agent
- Set retention per table and leave the workspace retention to its default
- Set archival tier per table – To meet certain compliance rules, you may need some of the data available for a longer period of time
- Configure diagnostic settings with only the logs that are needed and used
Use of Azure Advisor
I must admit that I’m a fan of Azure Advisor, for any Project that i have, always i tend to revise Advisor in order to cut Azure costs
It helps to detect if a Virtual Machine runs on a VM size GREATER than what it needs (based on CPU utilization under 5% in the last 14 days). If the Azure Advisor reports an overprovisioned machine, you need to investigate its use and resize it to a more suitable size.
For this VM rightsizing purpose, I also use a script from Jos Lieben, which helps to put your underused VM in the right size in terms of load: Automatic modular rightsizing of Azure VM’s with special focus on Azure Virtual Desktop | Liebensraum
Reserved Instances
Reserved instances allow us to reduce cost, there are a lot of resources that can be reserved, take them into account when you’re designing your infrastructure
As you can see, there are a lot of Azure resources available to be reserved, make use of them 🙂
In Azure Advisor always recommend to reserve instances of our resources, don’t forget it
Azure Budgets
Send notification when a certain amount of money is spent, this can be set at resource group or subscription level, and for example email to the application/subscription owner
Azure Cost Management
Remember to keep a closer look to the latest updates from Cost Management: https://azure.microsoft.com/en-us/blog/microsoft-cost-management-updates-november-2022/ I’m sure that you’ll take profit of those new features 😉
Insights, is the new feature of ACM allows us to have some insights about our daily spending in Azure resources, we can detect what is a tendency, and what is a cost anomaly in our subscriptions
Azure Governance Dashboard
If you want to deploy a High-Level Visualization in PowerBI of your azure resources, you can implement the CCO Dashboard from GitHub: https://github.com/Azure/ccodashboard
I know that this is more related with governance, but it helps to have a bird’s eye into the different resources and Azure subscriptions.
PRO TIP
If you really like those cost recommendations, there is a toll in github: https://github.com/helderpinto/AzureOptimizationEngine which can enchange the Azure Advisor recommendations and help you to optimize your environment
Closing
That’s all, probably some of the recommendations are already being followed by you, but I hope this post was interesting to you 😊
Till next time, merry Christmas and happy holidays!
Albandrod's Memory 