Best Practices about how to cut costs in Azure

by albandrod, posted in Azure, Best Practices, cost

Introduction

There is no one sizes fits when it comes to Azure and cost optimization, but the focus of this session is to explain some tips & tricks during my daily life as a Cloud Solutions Architect​

Some general tasks can be done monthly/quarterly to be sure that you Azure environment is up to date, taking into consideration that the optimization and your business run are the most important things here​

Be advised that not all the things that can be done in Azure are being covered in this post, probably because at the time of the writing I didn’t have to

Why this post?

Every design in Azure has cost implications, before architecting something, we must consider the budget that we will need for the Project itself, taking into consideration thinks like:

  • Identity different boundaries for scale up
  • Redundancy
  • BCP taking into consideration the cost of the solution
  • Design and set up scalable architectures, focusing on metrics & performance
  • Start small and scale out as soon as the required performance needs it​ (I really love that one)
  • Choose PaaS and SaaS over IaaS, pay only for what you use as a consumer​
  • Always, monitor, Audit & optimize the cost related

Ok I get it, but what are we going to cover?

For the next minutes I will explain some guidelines about cost optimization, in particular for the following topics:

  • Use of ARI
  • Use of Dev subscriptions
  • Optimal use of Azure App Services​
  • Optimal use of Auto-Scale in App Services​
  • Azure Data Factory Failed Pipelines
  • PaaS SQL Optimization
  • Cosmos DB
  • VM Right Sizing
  • Azure Hybrid Benefit​
  • Blob Storage Lifecycle​
  • Networking
  • Clean Orphan Resources​
  • RIs​
  • Use of Log Analytics
  • Use of Azure Advisor
  • Cost Management Preview (ACO Insights)
  • Azure Governance Dashboard
  • Closing

Before starting…

Before starting this post, I would recommend to create an Azure Inventory from your environment, with this tool, it is pretty simple: https://github.com/microsoft/ARI

And as you can observe, it will give you a great overview of what type of resources are you having, which use, locations, etc…Also, some of the sheets can be used to optimize your Azure Cost environment

Also, another tip that I want to give, is you can start your journey to Cost Optimization with a self-done assessment, but it can give you some guidelines about where are you: https://docs.microsoft.com/en-us/assessments/

Use of Dev Subscriptions

Using the top-to-bottom approach, the first thing to pay attention to is Azure Dev/Test subscriptions, which are applicable for both enterprise and pay-as-you-gooffers. By placing your dev resources in those subscriptions, you will get lower prices for most common Azure services for the cost of excluding them from the regular vendor SLA commitments. 

Optimal use of Azure App Services
First, check that standard Plans and Premium plans has an associated application​

I have seen a lot of empty App Services Plan, which leads to unnecessary cost to the customer, remember that having a right governance in your subscriptions it is algo a cost measure.

Another thing that I tend to do is to check the metrics for the plan, and check if are being used properly (scale down in case is needed, but remember the features needed in each case)

Optimal use of Auto-Scale in App Services​

In my case, I can be able to scale down my resources, but first check features between standard and premium plans (or even between std o premium!). Also what it can be done is to scale up/down based on a schedule https://techcommunity.microsoft.com/t5/apps-on-azure-blog/azure-app-service-automatic-scaling/ba-p/2983300

Very useful for those workloads where we know that only are needed in certain periods of time

Azure Data Factory​

Review the failing pipelines​, if a pipeline is constantly failing to run, probably it will impact into the cost of your resource, so take action

Again, I have reviewed a lot of Pipelines in DF which are continuosly failing… take care of that as well

Paas SQL Optimization​

With monitor, check if the database needs all the DTUs provisioned, one thing I love to do, is to play with the different available plans for the SQL, if you’re running a version with a lot of DTU’s, implement a runbook in order to reduce the plan when you don’t need it​, for example, you can use: GitHub – francesco-sodano/azure-sql-db-autoscaling: This ARM Template deploys an Azure SQL Database with DTU Consumption plan (with a new Azure SQL Server) including all the resources required to perform Auto Scaling (scale up and scale down) based on Metric Alerts using a function app. Again, very useful for those workloads where we know that only are needed in certain periods of time

Focus on those DBs with 40%-80% of the DTU capacity​, those are the most imporant to be scale up

Check if you really need the Georeplication, probably you don’t need to replicate your DB across regions (important point!!!), remember the first bullets of this post, we need to start small and then plan big, if we start to put all the georeplication modes to DBs that are not being use or for those in test, you’re wasting your money

Cosmos DB​

With the help of metrics, review the use for the correct size & throughput ​

Consider autoscaling for those type of resources (it avoids consuming unnecessary resources)​

Consider serverless options for Dev & Test environments or those environments where intermittent traffic is it used: Consumption-based serverless offer in Azure Cosmos DB | Microsoft Learn

VM Right Sizing​
One thing I love to do, is to shutdown VMs based in a Schedule​

The Schedule is set up with a tag in the resource, and the operation is done by an Automation Account (it could be a Logic App as well)​. For example, I love to use the following script:

Scheduled Virtual Machine Shutdown/Startup – Microsoft Azure | Automys

You can setup the following tag in the VMs

And your VMs will automatically shutdown and start in the configured schedule, which for those test and PRE environments where Azure Reservations does not fit, are simply great, you will save a bunch of computation hours with this simply script

In order to cut costs, we can use spot VMs for non priority tasks (it helps to save some money vs other azure VM sizes), you can get more info in: Use Azure Spot Virtual Machines – Azure Virtual Machines | Microsoft Learn

Get rid of those old VM sizes

One thing that I do for all of my clients in order to optimize cost, is to check which version size are running for the VMs, this can be extracted from the ARI (remember the first tool):

Why? Because as you probably know, Microsoft is always optimising hardware in the Datacenter, so they are pulling new version of the VM size, so what’s the point? the older is the VM size, higher VM cost, so check out if there is any new VM size, and you will be able to save some money from each VM size.

Imagine that you have 100 VMs running in a v2 series, and changing from v2 to v5, represents a change in cost of 20€/VM/month, so in total the save is 2000€/month with only changing the VM to a newer version, not bad uh?

Azure Hybrid benefit

First question is: Do you have a software assurance with Microsoft? If the answer is yes, don’t waste more time and money, and apply it to your Azure Resources, it Will help to sabe up to 40% in cost (for VMs and SQL)

If you want to know how much you can save with this, you can use the Azure Calculator for this purpose: https://azure.microsoft.com/en-us/pricing/hybrid-benefit/#calculator

Storage Lifecycle

With this procedure I was able to save a lot of money in a recent IoT Project, all the information was stored in blobs, but once a certain period of time passed, we moved the information from one tier to another in order to cut storage costs

Networking

Check out costs related with networking, it may scary you​

You Will need to identify which applications are using most of the egress bandwidth and review & redesign your infrastructure accordingly​

Check which gateways are not being used, probably those which have a throughput lower tan 900MB/day​

Check you Azure Express Route Circuits, probably the first provision of the circuit was greater than needed

So, check Azure Monitor: Monitor – Microsoft Azure

Clean Orphan Resources

Are you sure that everything that you have in your subscription are being used? Use this workbook and take action in your subscription: Azure Orphan Resources (microsoft.com)

Save Azure costs deleting those unused disks, Public IP’s which are consuming Storage and account cost (remember that in Azure Advisor we have these recommendations as well):

I’m sure that you will save a bunch of €€€ with this procedure

Use of Log Analytics

If you’re using Log Analytics to monitor your Azure Resoruces, you should add a Daily Cap into your Log Analytics Workspace: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/daily-cap#view-the-effect-of-the-daily-cap

Also a few tips:

  • Use Azure Monitor Agent and Data Collection rules over Log Analytics agent
  • Set retention per table and leave the workspace retention to its default
  • Set archival tier per table – To meet certain compliance rules, you may need some of the data available for a longer period of time
  • Configure diagnostic settings with only the logs that are needed and used

Use of Azure Advisor
I must admit that I’m a fan of Azure Advisor, for any Project that i have, always i tend to revise Advisor​ in order to cut Azure costs

It helps to detect if a Virtual Machine runs on a VM size GREATER than what it needs (based on CPU utilization under 5% in the last 14 days). If the Azure Advisor reports an overprovisioned machine, you need to investigate its use and resize it to a more suitable size.​

For this VM rightsizing purpose, I also use a script from Jos Lieben, which helps to put your underused VM in the right size in terms of load: Automatic modular rightsizing of Azure VM’s with special focus on Azure Virtual Desktop | Liebensraum

Reserved Instances

Reserved instances allow us to reduce cost, there are a lot of resources that can be reserved, take them into account when you’re designing your infrastructure

As you can see, there are a lot of Azure resources available to be reserved, make use of them 🙂

In Azure Advisor always recommend to reserve instances of our resources, don’t forget it​

Azure Budgets

Send notification when a certain amount of money is spent, this can be set at resource group or subscription level, and for example email to the application/subscription owner

Azure Cost Management

Remember to keep a closer look to the latest updates from Cost Management: https://azure.microsoft.com/en-us/blog/microsoft-cost-management-updates-november-2022/ I’m sure that you’ll take profit of those new features 😉

Insights, is the new feature of ACM allows us to have some insights about our daily spending in Azure resources, we can detect what is a tendency, and what is a cost anomaly in our subscriptions

Azure Governance Dashboard

If you want to deploy a High-Level Visualization in PowerBI of your azure resources, you can implement the CCO Dashboard from GitHub: https://github.com/Azure/ccodashboard

I know that this is more related with governance, but it helps to have a bird’s eye into the different resources and Azure subscriptions.

PRO TIP

If you really like those cost recommendations, there is a toll in github: https://github.com/helderpinto/AzureOptimizationEngine which can enchange the Azure Advisor recommendations and help you to optimize your environment

Closing

That’s all, probably some of the recommendations are already being followed by you, but I hope this post was interesting to you 😊

Till next time, merry Christmas and happy holidays!

Seven Essential Security Configurations for Microsoft 365

by albandrod, posted in AAD, O365, security

Security Essentials in Microsoft 365 are a must, probably, most of this recommendations are being followed by most of us, but just in case and as a reminder:

 

Are users configured with multi-factor authentication? Multi-factor authentication is necessary control for users that protects them from password attacks such as password guessing and credential theft. If a Microsoft 365 user account is compromised, an attacker may gain access to the user’s emails, files, chat history, and other sensitive data. So imagine if this happens to an admin… probably: GAME OVER

If the organization’s on-prem Active Directory is synchronized with Azure Active Directory, are only necessary objects synchronized? Organizations will commonly synchronize their on-prem AD with Azure AD. However, it is a best security practice to only sync those AD objects that require use within Azure AD

 

Is the number of users configured as administrators in Microsoft 365 appropriate for the size of the organization? Having more than one administrator in Microsoft 365 ensures that if one administrator is unavailable, another user can make changes to the tenant. as always my recommendation and Microsoft is that should be no more than five Global Admins (remember to have emergency access account as well)

 

Are dedicated administrative accounts used? Separate administrative accounts from personal accounts, and something important administrative personnel should use their privileged accounts only when it is required.

 

Are tenant Global administrators configured with working email addresses? Microsoft 365 Global Admins receive a variety of important email notifications that include service status, security events, and other information. So, it is important that organizations ensure that global admins use an email address that is configured to a working address.

 

Are Azure AD User Settings configured from non-default settings? By default, non-administrative users may access the Azure AD administrative portal and perform several different actions including:

• Register custom-developed applications for use within Azure AD

• Access the Azure AD administrative portal

• Allow user to connect their Azure AD accounts with their
LinkedIn account

• Invite external guest users

• Invited guest users can invite additional guest users

 

Each of these settings may have a security impact, depending on how the organization. If the your organization has not tackled any of these default settings to be more restrictive, you’ll need to do it, there are a lot of configurations to be done

 

Are users restricted from creating auto-forwarding rules within Outlook? When a user creates an auto-forwarding rule, emails sent to the account are automatically forwarded without user notification to an email box that the organization does not control. This may expose the organization to risk of loss of sensitive data.

 

As always, there a lot of best practices to follow, the previous recommendations are only a few of them, but it’s up to you to apply them or be in the risky way, stay safe!

 

Conditional Access Tips From the Trenches

by albandrod, posted in AAD, conditional Access, security

I want to drop some lines about my experience deploying several projects of Azure Conditional Access

  • Always exclude your emergency accounts from the conditional access policies (remember if you don’t have an Emergency Account, you’re late), this is something that I always tell to my customers and I will never give up
  • Don’t enable new policies without communicate properly to the organization, and also to foresee the impact in the users (you will save a lot of tickets from the customer service)
  • Don’t enable policies that requires compliant or hybridAzureAdJoin devices without verifying the state of the devices in the Azure Portal (same as before, you will save tickets and system interruption from the end users)
  • Careful with including in the policies the application “all apps”, is it possible to have a disgusting surprise (in my case, it happened to me in Azure, I put some exclusions, but it seems that sometimes the portal calls randomly to other APIs that cannot be controlled (and do not exist in AAD), so the user received a block in the portal even if the policy makes sense for you)
  •  If you go ahead with the policy with “All Cloud Apps” for the policies bases on devices, be sure to exclude in the policy the app “Intune Enrollment” or you won’t be able to enrol new devices in the portal
  • It is very easy to include multiple cases in one policy, but if you want to troubleshoot of what is happening, is it easier to segment the policy in multiple policies. Eat the elephant bite by bite, we have to put in the balance having and managing several policies or be able to troubleshoot correctly.
  • Is it recommended to include a naming convention in your policies, in a bird’s eye, you will be able to know what is the use for each policy (user, device, administrator, guest)

So, this is all, probably you’re following most of these recommendations, but if not, don be a fool 😉

Till next time!

OATH Hardware Tokens for AzureAD

by albandrod, posted in AAD, Best Practices, security

As you probably have been reading in my previous posts, I’ve been talking about FIDO2 keys, and how it can be used as a secondary authentication when signing in AzureAD.

PassWordless Authentication with Fido 2 Keys – Part 2
PassWordless Authentication with Fido 2 Keys

Today, I want to talk about OATH hardware Tokens, known as Time-based One Time Password Tokens as well.

As you are aware, some authentication methods can be used as the primary factor when you sign in to an application or device, such as using a FIDO2 security key or a password. Other authentication methods are only available as a secondary factor when you use Azure AD Multi-Factor Authentication or SSPR.

The following table outlines when an authentication method can be used during a sign-in event:

But, OATH TOTP is an open standard that specifies how one-time password (OTP) codes are generated. OATH TOTP can be implemented using either software or hardware to generate the codes. OATH TOTP hardware tokens typically come with a secret key, or seed, pre-programmed in the token.

In this post, I will show you how the OTP C200 token from Feitian can be configured in Azure AD and how it works.

First of all, what you have to do is to register the key in Azure AD, in order to do this, you will need the Serial Number from the Key, and the secret key provided by the manufacturer, and then you need to create a CSV file with all the information:

Once you have done this, these keys must be input into Azure AD: Multifactor authentication – Microsoft Azure

Upload the file, and activate the key in the portal, once it is have been done, it will show you a screen like the following:

If you have any error during the upload, it will be shown in the portal itself:

You must consider that you can activate a maximum of 200 OATH tokens every 5 minutes.

Also, as you probably figure out, users may have a combination or OATH Hardware tokens, Authenticator App, FiDO Keys, etc…

Be aware that users con configure their default sign in method in the security info web: My Sign-Ins | Security Info | Microsoft.com

So, once the key has been configured for the user, which is the flow to access to the account?

I have compared the Authentication flow with the Fido2 Key Flow, the difference that you can appreciate is with FiDO2 Keys is not necessary to include my password

Finally, check out the following table from Microsoft, where you can see different persona cases and which passwordless technology can be used for each one of them

IMHO, FiDO Keys are great, but thinking as an end user they have problem: the first setup: We must rely on end user about how they configure the key and associate it with azure AD (remember the previous table). FiDO keys has the advantage to be able to be used to sign in instead of using a password in the computer.

In the other hand, OAUTH keys are great, because you as an administrator, can configure the keys in the AAD Portal, and once have been activated provide them to end users, without necessity to do any other action from the end user perspective, and the most important part, are very easy to use

Thanks to Feitian for providing such amazing tokens

How to stop Azure Application Gateway and Azure Firewall

by albandrod, posted in Azure

Hi folks, summer is here and my holidays are very near, so I’m wrapping everything up to close my laptop and relax for a few weeks.

But before my deserved rest, I need to give you an FinOps advice:

If you’re like me and often makes demo setups in your Azure subscription that involve resources like Azure Firewall and Application Gateways, you probably have realize that there is no easy way to gracefully shutdown all those “hungry” resources to save some money.

To stop VMs, we can simply use the Azure Portal start/stop buttons, or use automation accounts or whatever, but Azure Portal doesn’t allow you to stop application gateway or Az Firewall. In such cases, Azure PowerShell helps:

# Get Azure Application Gateway
$appgw = Get-AzApplicationGateway -Name "appgw_name" -ResourceGroupName "rg_name"
 
# Stop the Azure Application Gateway
Stop-AzApplicationGateway -ApplicationGateway $appgw
 
# Start the Azure Application Gateway
Start-AzApplicationGateway -ApplicationGateway $appgw

After executing the stop, we will be able to see that the Operational State change after 1 minute or so:

and for the AzFirewall we can use the following:

$firewall=Get-AzFirewall -ResourceGroupName rgName -Name azFw
$firewall.Deallocate()
$firewall | Set-AzFirewall

$vnet = Get-AzVirtualNetwork -ResourceGroupName rgName -Name anotherVNetName
$pip = Get-AzPublicIpAddress -ResourceGroupName rgName -Name publicIpName
$firewall.Allocate($vnet, $pip)
$firewall | Set-AzFirewall

Now, you know how to save some money using those resources and I’m able to go to holidays to rest a while

Happy holidays!

Admin App for Microsoft Teams

by albandrod, posted in Teams

Microsoft has announced the release of a new Admin App for Microsoft Teams. This app simplifies top management options and showcases them in one place.

  • Optimize ‎Microsoft Teams‎ meetings, messages, webinars, and more for everyone in your organization
  • Add and remove users and reset their passwords
  • Add and remove teams and manage team members
  • Assign and remove licenses for users

Discover advanced settings, training, and support resources

To get started with the new Admin app experience, you will first need to head to the Microsoft Teams app store. Then, search for “Admin” and download the app in Microsoft Teams.

Once you have installed the app, you can manage some tenant settings and Teams configurations:

Admin app It is a great step to put teams as a central hub for everything, making easier for IT Pros to access settings and perform key management tasks across Microsoft 365 and Teams, but keep in mind that the Admin app is available for all users in the App Store, but you must have administrative rights to use it.

Multilingual invitations for Teams Meetings

by albandrod, posted in O365

This new feature for Teams, allows administrator to customise meeting invitations, to display the information of meeting in up to two languages ton all email platforms

In order to enable this, we can apply a new policy in their admin portal by enabling the MeetingInviteLanguages parameter in the CsTeamsMeetingPolicy at the user or group level, or for the entire organization

To enable this, we must use PowerShell, so let’s go:

#connect to Teams PowerShell

Import-Module MicrosoftTeams

Set-ExecutionPolicy -ExecutionPolicy Unrestricted

Connect-MicrosoftTeams

#check the current configuration

Get-CsTeamsMeetingPolicy -identity global | fl *lang*

#apply the new config

Set-CsTeamsMeetingPolicy -Identity Global -MeetingInviteLanguages "en-US,es-ES"

It may take several hours before the policy becomes active, but in the end, we will be able to find this in a new teams meeting:

Hope that helps, till nex time!

You should remove that basic authentication from Exchange…

by albandrod, posted in AAD, security

Now more than ever, you should disable your legacy authentication in Exchange Online, last year Microsoft announced that they will remove that basic Authentication next October (https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-may-2022/ba-p/3301866)

Did you know that:

  • More than 99 percent of password spray attacks use legacy authentication protocols
  • More than 97 percent of credential stuffing attacks use legacy authentication
  • Azure AD accounts in organizations that have disabled legacy authentication experience 67 percent fewer compromises than those where legacy authentication is enabled
thumbnail image 1 of blog post titled New tools to block legacy authentication in your organization

Disabling legacy authentication for users is a must-do on your identity security checklist

Why? becasue is a security gap and Microsoft took a lot of effor to promote Modern Authentication. So, you can disable now this basic Authentication in very simple steps:

If you don’t want to use this option, you can block the access to those protocols by using Conditional Access, understanding in a very simple way the impact of the policy.

What are you waiting for?