What are the common problems with software automation

What are the common problems with software automation?

Read more »

Automation testing Tools

 Test automation tools

Tool name	Produced by	Latest version
QTP	HP	11.0
IBM	IBM Rational	8.1.0.3
Parasoft SOAtest	Parasoft	9.0
Ranorex	Ranorex GmbH	3.0
Rational robot	IBM Rational	2003
Selenium	Open source	1.0.10
SilkTest	Micro Focus	2010 R2
TestComplete	SmartBear Software	8.2
TestPartner	Micro Focus	6.3
TOSCA Testsuite	TRICENTIS Technology & Consulting	7.2.1
VS	Microsoft	2010
WATIR	Open source	1.6.5
WebUI Test Studio	Telerik, Inc.	2010.3

Read more »

General Automation Process In Real Time Industries

                 

1. Requirement Gathering.
In this stage Test Engineers will collect the requirement documents and review the documents and they analyse the documents from where to automate it.
2.Identify the Best tool to fit.
In this stage Test engineers will analyse which tool is the best to automate the application.
3.Proof of Concept
Here Test engineers will give a proof how to automate the software to the client.
Some times client may give end to end scenarios or we may add end-to-end scenarios.
4.Feasibility Analysis
It is detailed study of the requirements in order to check whether all the requirements are possible are not.
Test Engineers will check where to automate the application and for example cmd prompt test cases will not be automated.
5.Estimation
*Based on Man hours we calculate Man days
6.Frame works
There are different types of frame works and test lead will decide the best frame work for the software.
Frame works are Keyword Frame work, Linear Frame work, Modular Frame work, Data driven Frame work, Hybrid Frame Work.
7.Develop Automation Scripts
Here the Test Engineers will develop the scripts by Knowing the requirement document, when the build is released then they debug the scripts and execute it
8.Failure Analysis
Here we should plot the data how it is failed
Why the script is failing?
Reasons for failures
Data issues
Application may synchronise
Object identification Issues
Run time errors,Defects
9.Defect Tracking
Test Engineers will track the defect where it is failed.
10.Publish Results
We should give proper Reason where it is failed and in which part it is effecting.

Read more »

Fifth Generation Scriptless and Advanced Test Automation Technologies

By Jeff Hinz, CTO, TESTars Test Competence Centre.with Martin Gijsen, Principal Test Automation Architect, DeAnalist.nlReviewed for accuracy by Christopher J. Scharer, Director VIVIT Worldwide and Maurice Siteur, Managing Consultant, in Testing, Capgemin
Explaining fourth generation test automation Jeff wrote a whitepaper to get clients knowledgeable in basic test automation principles. For those needingan introduction to the basics of test automation please refer to the white paper Jeff wrote addressing the subject, Test Automation Awarenessi. To understand advanced test automation, it is important to understand the progression/evolution of test automation approaches

1st Generation – Record and Playback

                                      

2nd Generation – Use/reuse of functions in test scripts

3rd Generation – Data Driven scripts/functions

4th Generation – Action word (keyword) scripts/func.

5th Generation – Scriptless Automation

Read more »

Top 10 automation Testing Tools

Quick Test Professional(QTP)

HP QuickTest Professional software provides functional and regression test automation for software applications and environments.Part of the HP Quality Center tool suite, HP QuickTest Professional can be used for enterprise quality assurance
HP QuickTest Professional supports keyword and scripting interfaces and features a graphical user interface. It uses the Visual Basic Scripting Edition (VBScript) scripting language to specify a test procedure, and to manipulate the objects and controls of the application under test.
HP QuickTest Professional was originally written by Mercury Interactive. Mercury Interactive was subsequently acquired by Hewlett Packard (HP) in 2006.HP QuickTest Professional is currently available from HP Software Division.

IBM Rational Functional Tester 

IBM Rational Functional Tester is a tool for automated testing of software applications from the Rational Software division of IBM. It allows users to create tests that mimic the actions and assessments of a human tester.[1] It is primarily used by Software Quality Assurance teams to perform automated regression testing.

Selenium

Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) [1] to write tests in a number of popular programming languages, including C#, Java, Groovy, Perl, PHP, Python and Ruby. The tests can then be run against most modern web browsers. Selenium deploys on Windows, Linux, and Macintosh platforms.

TestComplete 

TestComplete is an automated testing tool, developed by SmartBear Software which aims to allow testers to create software quality tests. Tests can be recorded, manually scripted or created manually with keyword operations and used for automated playback and error logging.
TestComplete is used for testing many different application types including Web, Windows, WPF, Flash, Flex, Silverlight, .NET and Java .It automates front end UI/functional testing and back-end testing like database, and HTTP load testing

Telerik-Test Studio 

Test Studio [2] is used for testing all type of web applications without the need of writing code. The tool comes in two editions – a Visual Studio plug-in for software developers, and a standalone edition for QA professionals. Test Studio supports various web technologies such as HTML, AJAX, Silverlight, ASP.NET MVC, JavaScript

Visual Studio Test Professional 

Visual Studio Test Professional is an integrated testing toolset developed by Microsoft that delivers a complete plan-test-track workflow for in-context collaboration between testers and developers, in order to increase testers’ visibility to the overall project.

SilkTest

SilkTest is an automation tool for testing the functionality of enterprise applications in most versions of Windows, Solaris 9 and 10, and Red Hat Enterprise Linux WS 2.1 and 3.0.[1] It was produced by Segue Software which was acquired by Borland in 2006. Borland was acquired by Micro Focus International in 2009. SilkTest became the strategic product to eventually replace TestPartner, which was acquired by Micro Focus International from Compuware.
SilkTest offers various clients: - SilkTest Classic uses the domain specific 4Test language for automation scripting. It is an object oriented language similar to C++. It uses the concepts of classes, objects, and inheritance. - Silk4J allows to do automation in Eclipse using Java as scripting language - Silk4Net allows the same in Visual Studio using VB or C# - SilkTest Workbench allows automation testing on a visual level (similar to former TestPartner)

SOAtest 

Parasoft SOAtest is "a comprehensive testing and analysis tool suite for tailored to the unique testing and validation needs of Service Oriented Architectures." It helps testers test the message layer functionality of their services by automating their testing and supports numerous transport protocols including: HTTP 1.0, HTTP/1.1, JMS, MQ, RMI, SMTP, TIBCO, .NET WCF HTTP, .NET WCF TCP. Basic testing functionality include static analysis through WSDL testing, functional unit testing, regression testing, security testing, and load testing. Parasoft SOAtest introduced application behavior virtualization, which creates copies of applications or back-end systems that are otherwise difficult to access or exercise during development and testing. Application behavior virtualization was extended and is now available in Parasoft Virtualize. SOAtest is used by companies such as Sabre, Lufthansa Cargo AG, Siemens AG, MedicAlert, and AOL

TOSCA Test suite 

TOSCA Testsuite is a software tool for the automated execution of functional and regression software testing. In addition to test automation functions, TOSCA includes integrated test management, a graphical user interface (GUI), a command line interface (CLI) and an application programming interface (API). TOSCA Testsuite is developed by the Austrian software company TRICENTIS Technology & Consulting GmbH based in Vienna. In 2011, TOSCA was included in Gartner Inc.’s "Magic Quadrant for Integrated Software Quality Suites" report as a “visionary”.

Watir 

Web Application Testing in Ruby (or Watir, pronounced "water") is a toolkit used to automate browser-based tests during web application development. This automated test tool uses the Ruby (programming language) scripting language to drive Internet Explorer, Mozilla Firefox, Google Chrome, Opera and Safari, and is available as a RubyGems gem. Watir was primarily developed by Bret Pettichord and Paul Rogers.

Testing Anywhere 

Testing Anywhere is software produced by San Jose-based Automation Anywhere, Inc. (est. 2003). The software allows testers and developers to test applications, Web sites, objects, controls and GUI front-ends.The company has been featured in SD Times for its creation of "Object Avatars", and has over 25,000 customers.

                                             

Ranorex 

Ranorex is a Windows GUI test automation framework for testing many different application types including Web 2.0 applications, Win32, MFC, WPF, Flash/Flex, .NET and Java (SWT). Ranorex doesn't have a scripting language of its own. The user (e.g. the software tester) can use the functionalities of the programming languages like C# and VB.NET as a base, and enlarge it with the GUI automation functionality of Ranorex.

HTTP Test Tool (htt)

HTTP Test Tool (htt) is an open source scriptable protocol test tool for HTTP protocol based products like web servers, web browsers, web applications or ICAP.
Software products are per se not error free. Adding new features, fixing errors and upgrading library versions can all introduce new errors and security holes. A protocol test tool helps to detect errors within HTTP protocol based Software. It is very simple and can therefore do bad requests/response to emulate bad or malicious server/client software. There are many more protocol test tools for HTTP like OpenSTA, ApacheBench, Selenium... which also can do stress testing.

QF-Test 

QF-Test (the successor of qftestJUI, available since 2001) from Quality First Software is a cross-platform software tool for the GUI test automation limited to Java/Swing, SWT, Eclipse plug-ins and RCP applications, ULC, Java applets, Java Web Start and cross-browser test automation of web-based applications (HTML, AJAX, ExtJS, GWT, RAP, Qooxdoo, RichFaces).

Read more »

Benefit analysis - Manual versus Automated Testing

Problems with Manual Testing: Some of the problems with manual testing are:

1. Less Reliable: Manual testing is not reliable, as there is no yardstick available to find out whether the actual and expected results have been compared. We just rely on the tester's words.
2. 2) High Risk: A manual testing process is subject to high risks of oversights and mistakes. People get tired, they may be temporarily inattentive, they may have too many tasks on hand, they may be insufficiently trained and so on. Hence, unintentionally mistakes happen in entering data, in setting parameters, in execution and in comparisons.
3. Incomplete Coverage: Testing is quite complex when we have mix of multiple platforms, O.S. Servers, clients, channels, business processes etc. Testing is non-exhaustive. Full manual regression testing is impractical.
4. Time Consuming: Limited test resources makes manual testing simply too time consuming. As per a study done, 90% of all IT projects are delivered late due to manual testing.
5. Facts and Fiction: The fiction is that manual testing is done while the fact is only some manual testing is done depending upon the feasibility.
6. It is worth noting that the manual testing is used to do the documentation of tests, creating testing related guides according to data queries, providing structures for helping run the tests on temporary basis and measuring the test results. 
7. Manual testing is considered to be costly and time-consuming; hence we use automated testing to cut down the time and cost.

Benefits of Automated Testing: On the contrary, Automated testing is having many benefits.
Automated testing is the process of automating the manual testing process. We use automated testing to substitute or provide a supplement to manual testing with the use of a comprehensive suite of testing tools. Automated testing tools assist software testers to evaluate the quality of the software by automating the mechanical aspects of the software testing task. The benefits of automation are better software quality, lesser time for marketing, repeatability of testing procedures & reduced cost of testing. We shall now list some more benefits of test automation. They are given below

1. Automated execution of test cases is faster than manual execution. This saves time. This time can also be utilized to develop additional test cases, thereby improving the coverage of testing.
2. Test automation can free test engineers from mundane tasks and make them focus on more creative tasks.
3. Automated tests can be more reliable. This is because manually running the tests may result in boredom and fatigue, more chances of human error. While automated testing overcomes all these shortcomings.
4. Automation helps in immediate testing, as it need not wait for the availability of test engineers. In fact,
5. Automation = Lesser Person Dependence
6. Test cases for certain types of testing such as reliability testing, stress testing, load and performance testing cannot be executed without automation. For example, if we want to study the behavior of a system with millions of users logged in, there is no way one can perform these tests without using automated tools.
7. Manual testing requires the presence of test engineers but automated tests can be made to run round the clock, (24 x 7) environment. So, automated testing provides round the clock coverage.
8. Tests, once automated, take comparatively far less resources to execute. A manual test suite requiring 10 persons to execute it over 31 days i.e., 31 x 10 = 310 man days, may take just 10 man-days for execution, if automated. Thus, a ratio of 1 : 31 is achieved.
9. Automation produces a repository of different tests, which helps us to train test engineers to increase their knowledge.
10. Automation does not end with developing programs for the test cases. Automation includes many other activities like selecting the right product build, generating the right test data, and analyzing results and so on.Automation should have scripts that produce test data to maximize coverage of permutations and combinations of input and expected output for result comparison. They are called as test data generators.

It is important for automation to relinquish the control back to test engineers in situations where further sets of actions to be taken are not known.
As the objective of testing is to catch defects early, the automated tests can be given to developers so that they can execute them as part of unit testing.
Drawbacks of Automated Testing: Despite of many benefits, pace of test-automation is slow.
Some of its disadvantages are as under:

1. An average automated test suite development is normally 3-5 times the cost of a complete manual test cycle.
2. Automation is too cumbersome. Who would automate? Who would train? Who would maintain? These issues complicates the matter.
3. In many organizations, test automation is not even a discussion issue.
4. There are some organizations where there is practically no awareness or only some awareness on test automation.
5. Automation is not an item of higher priority for management. It does not make much difference to many organizations.
6. Automation would require additional trained staff. There is no staff for the purpose. Automation actually allows testing professionals to concentrate on their real profession of creating tests and test cases, rather than doing the mechanical job of test execution.

Conclusion:
Test automation is a partial solution and not a complete solution. One does not go in for automation because it is easy. It is painful and resource-consuming exercise but once it is done, it has numerous benefits.

Read more »