Implement FinOps guide build-out #777
Description
📝 Scenario
As a FinOps practitioner, I need to learn, document, measure, adopt, and automate FinOps capabilities in order to maximize cloud ROI for my organization
🚀 Goal
Flesh out the Implementing FinOps guide to cover all aspects of maturity based on the FinOps assessment guide. We want to make sure our Implementing FinOps guide includes everything practitioners need to know in order to gain knowledge, document their processes, measure KPIs, drive organizational adoption, and automate each of these for each FinOps capability.
To accomplish this, we will need to review each capability and identify:
- Knowledge: Is there anything missing or outdated in the capability guides that should be added, updated, or removed?
- Metrics: How can we measure success of the capability and the nested elements and action items within the capability?
- Leverage the KPI library and use case library
- Process: What process would we recommend for customers to leverage in order to implement this capability?
- Automation: How can we automate knowledge sharing, metrics (e.g., collection, reporting, alerting), and related processes (e.g., approvals, next actions)?
- Adoption: Do we have any recommendations (or ways to automate) broad adoption for the capability across the organization?
As each capability is reviewed with these 5 lenses in mind, add any ideas on changes we could make to fill gaps and address opportunities in the task lists below. This may be documentation updates, features in existing tools, or potentially new tools that could be added to the toolkit. This issue is for documenting all the ideas and we will review them and prioritize them later, starting with documentation and then moving into new and updated tools.
📋 Understand cloud cost and usage
### Data ingestion
- [ ] Review the Data ingestion capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] State best practices for organization to begin discussing and develop further knowledge on Data Ingestion
- [ ] Highlight FOCUS Validator
- [ ] Promote community engagement at the tech community site FinOps - Microsoft Community Hub
- [ ] Highlight enabling or disabling overwriting to track data ingestion
- [ ] Discuss approaches to take to implement Policies & Governance related to data ingestion frequency, redundancy, auditing, and data protection
- [ ] Inform on maintaining data ingestion, selection of repository size, performance, resiliency, and availability throughout the lifecycle
- [ ] Discuss cost associated with data ingestion
- [ ] Highlight ingestion consideration of sustainability data (carbon emissions), resource health, and other data to provide wholistic insight to cloud usage and cost
- [ ] State process for exporting data through Cost Management (lookback period and daily export)
- [ ] Highlight tools to increase efficiency in exporting data from cost management compared to manually exporting
- [ ] State how to test processes for Efficacy, Validity, and Prevalence
- [ ] Discuss preparing Landing Zone for Data Ingestion
- [ ] Provide process to obtain feedback/ insights from Personas on what is needed within the reports
- [ ] Discuss how access is enabled to the data repository for personas
- [ ] Include how to perform quality checks
- [ ] Include how to check for data completeness
- [ ] Showcase automation of data ingestion on a daily basis
- [ ] Highlight FinOps Hubs toolkit ability to manage exports
- [ ] Highlight how to implement poll to obtain insights/requests from Personas to determine what data is needed to ingest
### Allocation
- [ ] Review the Allocation capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Include importance of maintaining tagging and hierarchy strategy on a regular cadence
- [ ] Highlight strategy for identifying Resource Groups/Resources that are missing cost related tags
### Reporting + analytics
- [ ] Review the Reporting + analytics capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Include recommendation to understand billing constructs such as Reservations, Savings Plans, Azure Hybrid Benefit
- [ ] Role requirements needed to assign to different personas to leverage native tools to read cost details
- [ ] Highlight Azure Copliot for Cost Management
- [ ] Highlight Azure Advisor (Score/Recommendations) / Advisor Cost Optimization Workbook to identify opportunities of improvement & report on Sustainability
- [ ] Highlight Advisor Alerts/ Recommendation Digests
- [ ] Highlight Pricing Model filter to compare on-demand vs reservation/ saving plan/ hybrid benefit cost distribution
- [ ] Implement budgets to analyze within Cost Analysis
- [ ] Highlight leveraging tagging & Cost Analysis
- [ ] Highlight using Cost Analysis to evaluate commitment coverage vs on-demand (Filter By Pricing Model)
- [ ] Mention Microsoft Fabric Capacity Metrics App
- [ ] Highlight Emissions Impact Dashboard
- [ ] Highlight Application Insights
- [ ] Highlight Log Analytics for monitoring
- [ ] Highlight Resource health
- [ ] Highlight Sentinel, Security Center, Defender for supporting investigative reporting (Include how to share/ distribute reports for all tooling)
- [ ] Mention importance of maintaining guidelines for reporting including data sensitivity
- [ ] Highlight policy for tracking tagging compliance
- [ ] Highlight Use case / Develop cost reporting templates for each persona
- [ ] Mention how personas can provide guidance on reporting and data analytic use cases to FinOps team
- [ ] Showcase process for tracking anomalies, utilization, cost outliers, budgets and forecast variances
- [ ] Guidance on how to define a “reporting & analytics” scope
- [ ] Process to enable access control, guidance on identifying data sensitivity, and inclusions for each persona
- [ ] How to translate analytic results into work queues used by engineering (Data in the path)
- [ ] Highlight key metrics that can be used to help organizations track adoption of reporting & analytics and how these insights are informing decisions (Ex. How many teams have reports, % of self-service enabled, how many subscribe emails, % teams with feedback pipelines)
- [ ] How to set up environment for each persona to leverage data analytics and reporting provided by FinOps teams (showcase for each tool/report)
- [ ] How to track which teams have implemented self-service reporting and analytics
- [ ] How to automate communication of analytic results from each tool/report to personas
- [ ] Automate poll to request feedback from personas to further develop reports
- [ ] Inform on Tag Reconcile
### Anomaly management
- [ ] Review the Anomaly management capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
📋 Optimize cloud cost and usage
### Architecting for cloud
- [ ] Review the Architecting for cloud capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Embed Resource links to languages (Bicep / Terraform) & tools https://learn.microsoft.com/en-us/azure/?product=popular#languages-and-tools
- [ ] Highlight Assessments available for reviewing Architecture ie Cloud Adoption Strategy Evaluator https://learn.microsoft.com/en-us/assessments/8fefc6d5-97ac-42b3-8e97-d82701e55bab/
- [ ] Include Resource link for Azure Resource Manager https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/overview
- [ ] Include links to Microsoft Learn training: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/#build-your-skills-with-microsoft-learn-training
- [ ] Inform on Architecting for Sustainability
- [ ] Include resource for Azure services: https://azure.microsoft.com/en-us/products/
- [ ] Highlight link to the Onboarding capability as a Related FinOps Capability
### Workload optimization
- [ ] Review the Workload optimization capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Introduce Azure Monitor for monitoring metrics/demand and to evaluate against Advisor recommendations such as rightsizing
- [ ] Highlight Azure Monitor Alerts
- [ ] Update links for Azure Automation and Azure Functions as these direct to the same site
- [ ] Verify Azure Optimization Engine link located in Building on the basics section
- [ ] Inform of the Orphan Resources workbook to identify idle/orphan resources https://techcommunity.microsoft.com/t5/fasttrack-for-azure/azure-orphan-resources/ba-p/3492198
- [ ] Highlight the Well-Architected Framework Service Guides https://learn.microsoft.com/en-us/azure/well-architected/service-guides/?product=popular
- [ ] Highlight recommendations for selecting the right services https://learn.microsoft.com/en-us/azure/well-architected/performance-efficiency/select-services
- [ ] Introduce Azure Architecture Center for optimizing your workload https://learn.microsoft.com/en-us/azure/architecture/
- [ ] Inform on Alerting Strategy https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/manage/monitor/response#successful-alerting-strategy
- [ ] Highlight Bult-In Policy to Govern Compute SKUS https://learn.microsoft.com/en-us/azure/governance/policy/samples/built-in-policies#compute
- [ ] Highlight Built-In Policy for Region Governance https://learn.microsoft.com/en-us/azure/governance/policy/samples/built-in-policies#general
- [ ] Highlight Automation based on Budget Alerts https://learn.microsoft.com/en-us/azure/cost-management-billing/manage/cost-management-budget-scenario
- [ ] Add the process to conduct a Well-Architected Assessment Review https://learn.microsoft.com/en-us/assessments/azure-architecture-review/
- [ ] Highlight the use of Azure Virtual Machine Selector tool https://azure.microsoft.com/en-us/pricing/vm-selector/
- [ ] Mention Azure activity logs to track optimization actions taken
- [ ] Inform on metric for Power schedule adherence percentage per month = (Total number of hours the system is required to run/Total number of hours the system is running), may use Azure monitor, log analytics, workbooks for visualization, and alerting systems to raise awareness. Reference: https://www.finops.org/wg/how-to-measure-power-schedule-adherence-rate-percentage/
- [ ] Inform on tracking Percentage Resource Utilization (CPU/ Memory/ Throughput/ IOPS/ Storage Utilization Rate)
- [ ] Inform on tracking Percentage of Legacy Resource (ex. V2 storage that provides lower rates and additional features)
- [ ] Inform on tracking Percentage of Storage on a Frequent Access Tier
- [ ] Storage accounts on premium tier without lifecycle management enabled
- [ ] Tracking percentage of Idle/ Orphan resources
- [ ] Tracking Auto-scaling efficiency rate (Auto-scaling efficiency rate = Maximum capacity cost of running workload to meet workload demand / Cost of running workload with auto-scaling to meet same workload demand. The higher the efficiency rate the more effective the auto-scaling is.)
### Rate optimization
- [ ] Review the Rate optimization capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Highlight link to learn what reservations are https://learn.microsoft.com/en-us/azure/cost-management-billing/reservations/save-compute-costs-reservations
- [ ] Highlight link to lean what savings plan are https://learn.microsoft.com/en-us/azure/cost-management-billing/savings-plan/savings-plan-compute-overview
- [ ] Inform on Spot Virtual machines
- [ ] Inform on tiering to save on storage resource types or highlight the WAF Performance pillar to support the development of the Rate Optimization strategy, for ex determine if standard storage vs premium storage for storing snapshots
- [ ] Inform on understanding agreements and how they impact rates such as EA, MCA, MACC https://learn.microsoft.com/en-us/azure/cost-management-billing/manage/ea-pricing-overview
- [ ] Inform on Dev/Test offer
- [ ] Highlight the approach to Configure Advisor alerts by recommendation type for reserved instances and savings plan
- [ ] Highlight the use of the Advisor Cost Optimization workbook Rate Optimization section
- [ ] Highlight AHUB and the use of Azure Hybrid Benefit Calculator https://azure.microsoft.com/en-us/pricing/hybrid-benefit/#calculator
### Licensing + SaaS
- [ ] Review the Licensing + SaaS capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### Cloud sustainability
- [ ] Review the Cloud sustainability capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Highlight differnet personas that may require specific metrics for Cloud Sustainability
- [ ] Provide information on Microsoft Sustainability Manager https://learn.microsoft.com/en-us/industry/well-architected/sustainability/sustainability-manager-centralized-monitoring
- [ ] Highlight Microsoft’s Sustainability Learning Center https://www.microsoft.com/en-us/sustainability/learning-center
- [ ] Highlight collaboration with partners https://appsource.microsoft.com/en-us/marketplace/cloudsIndustry?page=1&industry=sustainability
- [ ] Update Related FinOps Capabilities (update title for Workload Optimization, Remove or change Managing Commitment Discounts, update Measuring Unit Costs to Unit Economics)
- [ ] Highlight Microsoft Sustainability Manager Assessment https://learn.microsoft.com/en-us/assessments/333d9326-e34c-40e1-88d9-8342afceec36/
📋 Quantify business value
### Planning + estimating
- [ ] Review the Planning + estimating capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Highlight the use of the Azure VM Selector tool https://azure.microsoft.com/en-us/pricing/vm-selector/
- [ ] Highlight the use of the Azure Hybrid Benefit Calculator https://azure.microsoft.com/en-us/pricing/hybrid-benefit/#calculator
- [ ] Highlight CAF/ Azure Landing Zone and WAF to identify infrastructure to include in Estimate and to leverage for guidance in migrating, modernizing, innovating, and relocating
- [ ] Highlight exporting from Azure Pricing Calculator
- [ ] Highlight Emissions Savings Estimator https://gw.us-il301.gateway.prod.island.powerapps.com/customerenrollmentservice/estimator/index.html to estimate (MTCO2e) Azure vs On-premise emissions
- [ ] Highlight the use of Microsoft Delivery Plans extension in Azure DevOps https://marketplace.visualstudio.com/items?itemname=ms.vss-plans to help with estimating/ tracking timelines
- [ ] Can highlight the Strategic Migration Assessment and Readiness Tool https://learn.microsoft.com/en-us/assessments/Strategic-Migration-Assessment/ to support with Migration planning
- [ ] Inform on leveraging Cost Analysis to evaluate cost of similar workloads, or to use to evaluate cost of new workloads over a short period of time to obtain actual cost of trial-run
- [ ] Inform on Dev/Test offer as initial environment for planning workloads or incorporating as a stage in the trial-run
- [ ] Metric: Percentage of workload planned and estimated
- [ ] Metric: Percentage variance of estimated cost vs actual costs
### Forecasting
- [ ] Review the Forecasting capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### Budgeting
- [ ] Review the Budgeting capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Update title of Related Capability from “Chargeback and Finance Integration” to “Chargeback & Invoicing”
- [ ] Highlight Copilot for Azure, Cost Analysis, and Monitor as tools for identifying and investigating reasons for budget overages
- [ ] Inform on daily caps for resources to avoid/ limit unnecessary charges due to unexpected spikes in usage
- [ ] Inform on Planning & Estimating Capability + Pricing Calculator to help develop budget estimates for new workloads
- [ ] Highlight Workload Optimization/ Rate Optimization/ (Advisor or Cost Optimization Workbook) as capabilities and tools to use to adjust workloads if cost are forecasting past budgets
- [ ] Highlight creating a budget using ARM https://learn.microsoft.com/en-us/azure/cost-management-billing/costs/quick-create-budget-template?tabs=no-filter%2Cportal
- [ ] KPI/Metric: Percentage to Budget (Favorable to Budget vs Unfavorable to Budget amounts)
- [ ] KPI/Metric: Percentage of Workloads at Budget vs Actual Variance at/ below 20%, 15%, 12% variance
- [ ] KPI/Metric: Percentage Variance of Budget vs Forecasted Cost
- [ ] KPI/Metric: Allocated Budget Usage or burn rate (Actual Spend for Time Period/ Budget for Time Period)
- [ ] Highlight leveraging custom policy definition to govern budgets being assigned to a desired scope (reference as example https://www.azadvertizer.net/azpolicyadvertizer/Deploy-Budget.html)
### Benchmarking
- [ ] Review the Benchmarking capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### Unit economics
- [ ] Review the Unit economics capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
📋 Manage the FinOps practice
### FinOps practice operations
- [ ] Review the FinOps practice operations capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### FinOps education + enablement
- [ ] Review the FinOps education + enablement capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### Invoicing + chargeback
- [ ] Review the Invoicing + chargeback capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### FinOps assessment
- [ ] Review the FinOps assessment capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### Onboarding workloads
- [ ] Review the Onboarding workloads capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
- [ ] Adjust reference of the Forecasting capability to the Planning & Estimating capability
### Cloud policy + governance
- [ ] Review the Cloud policy + governance capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### FinOps tools + services
- [ ] Review the FinOps tools + services capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
### Intersecting disciplines
- [ ] Review the Intersecting disciplines capability to identify knowledge gaps, metrics, processes, automation opportunities, and adoption guidance
🙋♀️ Ask for the community
We'd love to get your feedback! If you see something missing, please suggest it in a comment and we'll add it to the list. If you'd like to pick up one of the suggestions, either start a discussion or create an issue for it and let us know!