Automation Matrics and KPIs

 Hey Folks,

Below is useful information on automation matrics and KPIs

summary of the main differences between metrics and KPIs: 

• KPIs measure performance based on key business goals while metrics measure performance or progress for specific business activities. 
• KPIs are strategic while metrics are often operational or tactical.
• Metrics are lower-level indicators specific to a department while KPIs can be tracked by various departments working towards the same goal. 
• Metrics provide context to your business activities, KPIs allow for strategic decision-making.  

·       Every KPI is a metric, but not every metric is a KPI

Why Measure

• Automation Effectiveness
• Speed to Market:
• You want to create a culture of accountability
• Continuous Improvement:

 

Automation Matrix

•            Automatable Test Cases ----% Automatable = (# of automatable tests / # of total tests) * 100

•            Automation Script Effectiveness---Automation Script Effectiveness = (# of defects found by automation / # of defects opened) * 100

•            Automation Pass Rate---Pass Rate % = (# of cases that passed / # of test cases executed) * 100

•            Automation Execution Time----Execution Time = End Time – Start Time

•            Automation Test Coverage---Automation Test Coverage % = (# of automation tests / # of total tests) * 100

•            Automation Stability ---Automation Stability % = per test (# of failures / # of executions) * 100

•            Build Stability---Build Stability % = (# of build failures / # of builds) * 100

•            In-Sprint Automation---In-Sprint Automation % = (# of scripts created in-sprint / # of scripts created post-sprint) * 100

•            Automation Progress---Automation Progress % = (# of automated tests / # of tests that are automatable) * 100

•            Automation Pyramid---Automation Pyramid = (# of tests at each level / # of total tests) * 100

 

DevOps KPIs.

•            Meantime to failure recovery - This is the average time taken to recover from a failure.

•            Deployment frequency - The frequency in which the deployment occurs.

•            Percentage of failed deployments - The number of times the deployment fails.

 

Cheers,

TJ