Cloud Testing

Hi folks,

Few things on cloud testing

 Cloud testing type

• System testing
• Interoperability testing – infrastructure
• Performance
• Availability
• Multi tenancy- security
• Disaster recover

 

Cloud testing key objectives

• Ensure excellence of cloud based applications
• Authenticate software as service in cloud environments
• Verify automatic cloud based functional service
• Analyze inter operation capabilities and compitability

 

Consideration for cloud based testing APIs

• Data security and privacy
• Data migration
• Interface compitability
• Enterprise application integration

 

Cloud Test environment types

• Cloud based enterprise test environment
• Private/public test environment
• Hybrid cloud test environment

 

Challenges of cloud testing

• Lack of environment control
• Lack of transparency
• Query limitation
• Self modification and autonomic behavior
• Asynchronous interaction
• Time and load dependent behavior
• Huge configuration space
• Ultra large scale
• Low observity

 

Challenge SAAS with dummy data

Place holder of simultate the real data with no impact

• Simulate behavior of real data
• Can be quickly created to test API
• Allow developer to focus on test

Role of dummy data in test API

• Nominally occupy space of real data
• Act as stub to avoid api testing issue
• Data for security testing

 

Cloud testing benefits

• Flexible, comprehensive and indicative testing
• Reduce cost of ownership
• Promote green IT
• Geographic transparency and traceability

 

Cloud testing strategy

• Stress- testing till breaking point
• Performance- measure response time
• Functional
• Latency
• Security 
• Capacity testing= max capacity for current future hardware
• Governance risk compliance test
• Load- system stability

 

Service model as Test as a servie (TAAS)

• Taas process management
• Access TAAS  service
• On demand test environment
• Service option
• Service provisioning
• Service execution

 

TAAS benefit

• Reduce cycle time
• Reduce business risk
• Improve agility with highly automated solutions
• Helps leverage transactional pricing model for predictable and affordable solutions

 

TAAS provider- IBM, SOASTA, sauselab,skytap,vmlogics,

Automation Matrics and KPIs

 Hey Folks,

Below is useful information on automation matrics and KPIs

summary of the main differences between metrics and KPIs: 

• KPIs measure performance based on key business goals while metrics measure performance or progress for specific business activities. 
• KPIs are strategic while metrics are often operational or tactical.
• Metrics are lower-level indicators specific to a department while KPIs can be tracked by various departments working towards the same goal. 
• Metrics provide context to your business activities, KPIs allow for strategic decision-making.  

·       Every KPI is a metric, but not every metric is a KPI

Why Measure

• Automation Effectiveness
• Speed to Market:
• You want to create a culture of accountability
• Continuous Improvement:

 

Automation Matrix

•            Automatable Test Cases ----% Automatable = (# of automatable tests / # of total tests) * 100

•            Automation Script Effectiveness---Automation Script Effectiveness = (# of defects found by automation / # of defects opened) * 100

•            Automation Pass Rate---Pass Rate % = (# of cases that passed / # of test cases executed) * 100

•            Automation Execution Time----Execution Time = End Time – Start Time

•            Automation Test Coverage---Automation Test Coverage % = (# of automation tests / # of total tests) * 100

•            Automation Stability ---Automation Stability % = per test (# of failures / # of executions) * 100

•            Build Stability---Build Stability % = (# of build failures / # of builds) * 100

•            In-Sprint Automation---In-Sprint Automation % = (# of scripts created in-sprint / # of scripts created post-sprint) * 100

•            Automation Progress---Automation Progress % = (# of automated tests / # of tests that are automatable) * 100

•            Automation Pyramid---Automation Pyramid = (# of tests at each level / # of total tests) * 100

 

DevOps KPIs.

•            Meantime to failure recovery - This is the average time taken to recover from a failure.

•            Deployment frequency - The frequency in which the deployment occurs.

•            Percentage of failed deployments - The number of times the deployment fails.

 

Cheers,

TJ

Smart way of Testing

Hey Folks,

Time to come up with some smart tools, which make your life easier.

1. To find the xpath from xml file : 

xpather.com 

2. To find the WSDL APIs: 

Install Google Chrome extension - wsdler

3. To test mobile app on emulator:

Install Google Chrome extension - Android online emulator

4. To do browser and excel automation without investing much time on commercial or freeware tool

Install Google Chrome extension - iMacros

Cheers,

TJ

Test Metrics

Hi,

Couple of Test Matrics formulas.

Formulae :

Test Coverage (%) = (No. of Test Cases Executed / No. of Test Cases Planned) * 100

Test Case Effectiveness (%) = (1 - (No. of Inadeqaute Test Cases Detected / Actual No. of Test Cases)) * 100

Quality of Testing (%) = (No. of Defects Accepted / No. of Defects Reported) * 100

Automation Suite Test Coverage (%) = (No. of Test Cases Automated / Total No. of Test Cases) * 100

Environmental Availability (%) = (1 - (Total Down Time / Total Cycle Time)) * 100 Regards, Trupti

Defect Tracking or Defect/Bug Life Cycle

Hey folks,

Defect Tracking or bug life cycle is vary from company to company and tools to tools.

Defect Tracking

To track defects, a defect workflow process has been implemented. Defect workflow training will be conducted for all test engineers. The steps in the defect workflow process are as follows:

a) When a defect is generated initially, the status is set to "New". (Note: How to document the defect, what fields need to be filled in and so on, also need to be specified.)

b) The Tester selects the type of defects:

Bug

Cosmetic

Enhancement

Omission

c) The tester then selects the priority of the defect:

Critical - fatal error

High - require immediate attention

Medium - needs to be resolved as soon as possible but not a showstopper

Low - cosmetic error

d) A designated person (in some companies, the software manager; in other companies, a special board) evaluates the defect and assigns a status and makes modifications of type of defect and/or priority if applicable).

The status "Open" is assigned if it is a valid defect.

The status "Close" is assigned if it is a duplicate defect or user error. The reason for "closing" the defect needs to be documented.

The status "Deferred" is assigned if the defect will be addressed in a later release.

The status "Enhancement" is assigned if the defect is an enhancement requirement.

e) If the status is determined to be "Open", the software manager (or other designated person) assigns the defect to the responsible person (developer) and sets the status to "Assigned".

f) Once the developer is working on the defect, the status can be set to "Work in Progress".

g) After the defect has been fixed, the developer documents the fix in the defect tracking tool and sets the status to .fixed,. if it was fixed, or "Duplicate", if the defect is a duplication (specifying the duplicated defect). The status can also be set to "As Designed", if the function executes correctly. At the same time, the developer reassigns the defect to the originator.

h) Once a new build is received with the implemented fix, the test engineer retests the fix and other possible affected code. If the defect has been corrected with the fix, the test engineer sets the status to "Close". If the defect has not been corrected with the fix, the test engineer sets the status to .Reopen.. Defect correction is the responsibility of system developers; defect detection is the responsibility of the AMSI test team. The test leads will manage the testing process, but the defects will fall under the purview of the configuration management group. When a software defect is identified during testing of the application, the tester will notify system developers by entering the defect into the PVCS Tracker tool and filling out the applicable information.

20 Essential Addons for Fire fox

Hey guys,

If you are using fire fox as your testing browser, then you must have these 20 essential firefox addons.I found this article, once I wondering for some information. Hope you will also like it.

http://www.testinggeek.com/twenty-essential-firefox-addons-for-testing

Thanks,
Trupti

Testing Cheat sheet & Tips

Hi Friends,

Please find below some tips on Testing with cheat sheet.

Test Heuristics Cheat Sheet by Elisabeth Hendrickson - This is one of the best cheat sheat that you could find for testing. 

Heuristic Test Strategy Model by James Bach - Again, a must read.

Heuristic Risk-Based Testing by James Bach - A must read for all testers.



Thanks,

Prioritization of tests

Prioritization of tests

Hey folks,

Always Testers get less time to test the application, in this type of situation, one must take a decision to prioritization of his test. Here are some points we can take in consideration for prioritize it. If you found some more please add in comments, so all can get good information. I try my best to give you good information, hope you guys like it.

Principle

Prioritize tests So that, whenever you stop testing, you have done the best testing In the time available.

• How to Prioritize
• Possible ranking criteria
• Test where a failure would be:
• most severe
• more visible
• More likely
• Ask the customer to prioritize the requirements
• Critical to customer’s business
• Areas changed most often
• Areas with most problems in the past
• Most complex areas, or technically critical 

Statement, Branch/Decision and Path Coverage testing technique with example

How to calculate Statement, Branch/Decision and Path Coverage

Statement Coverage:

In this the test case is executed in such a way that every statement of the code is

executed at least once.

Branch/Decision Coverage:

Test coverage criteria requires enough test cases such that each condition in a decision

takes on all possible outcomes at least once, and each point of entry to a program or

subroutine is invoked at least once. That is, every branch (decision) taken each way,

true and false. It helps in validating all the branches in the code making sure that no

branch leads to abnormal behavior of the application.

Path Coverage:

In this the test case is executed in such a way that every path is executed at least once.

All possible control paths taken, including all loop paths taken zero, once, and multiple

(ideally, maximum) items in path coverage technique, the test cases are prepared based

on the logical complexity measure of a procedural design. In this type of testing every

statement in the program is guaranteed to be executed at least one time. Flow Graph,

Cyclomatic Complexity and Graph Metrics are used to arrive at basis path

How to calculate Statement Coverage, Branch Coverage and Path Coverage?

Draw the flow in the following way-

· Nodes represent entries, exits, decisions and each statement of code.

· Edges represent non-branching and branching links between nodes.

Example:

Read P

Read Q

IF P+Q > 100 THEN

Print “Large”

ENDIF

If P > 50 THEN

Print “P Large”

ENDIF

Calculate statement coverage, branch coverage and path coverage.

Solution:

The flow chart is-

Statement Coverage (SC):

To calculate Statement Coverage, find out the shortest number of paths following

which all the nodes will be covered. Here by traversing through path 1A-2C-3D-E-4G-5H all

the nodes are covered. So by traveling through only one path all the nodes 12345 are covered,

so the Statement coverage in this case is 1.

Branch Coverage (BC):

To calculate Branch Coverage, find out the minimum number of paths which will

ensure covering of all the edges. In this case there is no single path which will ensure coverage

of all the edges at one go. By following paths 1A-2C-3D-E-4G-5H, maximum numbers of

edges (A, C, D, E, G and H) are covered but edges B and F are left. To covers these edges we can follow 1A-2B-E-4F. By the combining the above two paths we can ensure of traveling

through all the paths. Hence Branch Coverage is 2. The aim is to cover all possible true/false

decisions.

Path Coverage (PC):

Path Coverage ensures covering of all the paths from start to end.

All possible paths are-

1A-2B-E-4F

1A-2B-E-4G-5H

1A-2C-3D-E-4G-5H

1A-2C-3D-E-4F

So path coverage is 4.

Thus for the above example SC=1, BC=2 and PC=4.

Memorize these….

100% LCSAJ coverage will imply 100% Branch/Decision coverage

100% Path coverage will imply 100% Statement coverage

100% Branch/Decision coverage will imply 100% Statement coverage

100% Path coverage will imply 100% Branch/Decision coverage

Branch coverage and Decision coverage are same.

*LCSAJ = Linear Code Sequence and Jump.