We are a small company specializing in Test Automation Software. We believe that Quality Assurance is not a subset of Development – it is a separate field. We believe that QA Engineers should focus on Quality Assurance ideals, automation and results rather than Development. And there are a few things we can do to help you focus
· We can automate the entire Test Suite run
· We can remove any machine and Suite configuration steps
· We can reduce the number of steps and tools you have to use to complete the Test Suite Life Cycle (Defining the Suite through viewing the Suite Results)
These 3 things become important when managing large-scale Test Suites
We’ve written a Test Automation Product (currently for JUnit – but easily extended to other platforms) that tries to accommodate people that have the same beliefs. We provide a Test Environment that lets your QA Engineers (easily) extend their JUnit Test Suite to run on multiple machines in our Pool of Test Machines. While some understanding of the environment is required, they do not need to develop a multi-threaded Test Suite nor do they have to pre-configure the Test Environment. Similarly, once the Test Suite is registered with us other people associated with the Test Suite can run it against any number of Test Clients – without having to understand the technical details of the Test Suite
· You do not need to own or maintain the Test machines
· You can schedule your Test to run at a particular time
· You can easily communicate between Test machines during your Test run – regardless of the number of Test runs
· You can specify what type of machines to run your Test against. For instance, you can choose to run against a machine that has a range of
o Hard drive space
o Available memory
o O/S
· You do not need to hard-code the number of Test machines that will run your Test Suite
· Once the Test Suite is registered a non-QA person can easily rerun it without knowing the intricate details of the Suite, against any type of computer they choose in our Pool of Test Machines and against any number of Machines in our Pool
· You do not need to install or maintain 3rd-party Testing/Scripting Applications (like Ant, Selenium, Perl, SQL). All you need to do is select which Applications you want to include in running your Test Suite – and they will be installed automatically as a function of the Test Suite
· Results are stored and viewable from our central Test Server
· Results and Test Suites are retrieved through the same Interface
· Your QA engineers do not need to learn a new scripting language just to configure the Test environment
· JDK1.6 or higher
· Win32 (WinXP or higher)
· Test Client Install Directory must contain
o Apache Commons Codec library
o InstallMinimalTestClient.class
These are the basic steps:
· Install our Development Test Client. The Client is used to register new Test Suites, view Results, administer your Test Suite and give you a Development Environment that simulates a Test Machine. Detailed steps are available here
· Create the Install Script for your Test Suite. This includes a list of the Test Components you need, as well as a list of triggers that define how to install your Test Suite, how to run your Test Suite and how to upload the Results to our Test Server (a sample is provided that can be used as a Template for defining your own Install Script).
· Register your Test Suite with our Test Server. This includes uploading the Install Script for your Test Suite as well as all pertinent files
· Define how many times you want to run your Test Suite, the schedule you want to run your Test Suite on and what types of machines you want to run it against
· Register your Work Database
· Run your Test Suite locally to verify it is configured properly, and to give you some sense of performance
· Finalize your Test Suite. Finalizing your Test Suite will freeze the state on the Test Server, and mark your Suite as ready to run on the Test Server
· Your Test Suite will be run within our Pool of private Test Machines, and results will be uploaded to the Test Server
· View your Test Results on the Test Server
· Always assume that the Test machine is clean prior to running your Install Script
· This version of the product has been customized around JUnit. Other Test Frameworks will be supported in the future based on interest
· All 3rd party Test components should be installed by us (by referencing them in your Install Script) – not as a component of your Test Suite. If some Test component is missing contact us and we’ll work on adding it to our Test Server. This reduces the number of duplicate file references and incorrect directory references
· Decide at the beginning whether you need to maintain information between Test runs. For example: Client names, Test state, Random Strings. If you need to share this type of information, then you should define a Work Database within your Install Script – and you will have to include code to connect to the Work DB within your Test Suite (Straight-forward examples are provided). The Work DB will be created automatically and the connection information will be distributed to all Test Machines that run your Test Suite
· Think about whether it’s worth it to break up your Test Suite into distinct pieces – and whether it’s possible in our Test Environment to run each piece on a separate Test Machine by splitting up your Test Suite and fully utilizing the Work DB
· If you require an installer to setup your Application – you will probably need a Silent Installer to use it on our Test Client (along with a Silent Uninstaller)
As mentioned in Best Practices, the Work DB is used to maintain information between Test runs. A separate Work DB may be created for every Test Suite – and connection details are provided automatically to Test Clients that run the Test Suite. You can also define a setup SQL script within your Install Script to be run when the Work DB is created. Once the Test run is complete the Work DB will be removed
· We are currently in Alpha. If you would like to participate or have questions please contact us
· To view the Developer’s Readme file, click here
· To view the Architecture’s Readme file, click here
· Download a presentation describing Qovalam here
Install and Sample Walkthrough (file download is 300mb. Extracted file is 700mb)
· Testing an Instant Messenger Application
· Testing an Email System
· Testing a Client/Server Application
· Testing an Application that takes too long to complete against 1 machine – with a Test Script that can be split up into smaller scripts that run simultaneously on multiple machines
· Testing a specific Bug against a specific type of machine
· Testing a distributed Database
· We have a long list of Enhancements to implement. While we will try to implement them as soon as possible – we could really use some help
· If a company wants to have a completely secure instance of our Test Server we can license it (and set it up internally). We can setup each Desktop within your organization as a Test Client with a sleep schedule – which will give you the ability to use all of your Desktops during off-hours. Think of it as your own private (virtual) super computer
· We have over 40 years of experience in Test Automation. We love this stuff, but we think many companies make incorrect assumptions about our field. And the best way to demonstrate this is by offering a better way to do things
· Our Test Server is setup to support RollOver Test runs (wherein Test runs will be rerun on a Test Machine if some have not started yet.) While this gives us the ability to support an unlimited number of Test Runs – it also introduces some limitations to the Work Database. For now it’s been enabled – but this may change if/when we configure more Test Machines (based on interest in our product).
· Similarly, you can use a Work DB with our Server in RollOver mode – but maintaining Test State between Test Clients may not be feasible (since you may not be able to guarantee that all Test Clients run simultaneously)
· Our eventual goal is to charge for usage of the Test Machines. For now it is free
· If you want to test a Bug against a specific type of machine the basic steps have to change slightly. We can accommodate you as needed – just contact us
· We’ve started extending to other platforms besides JUnit (PHP test frameworks, Ruby, etc.) While these have not been fully integrated yet, they can already be used with a little extra effort. We’ll help you setup your platform within our framework – just ask
· We can isolate Test Data from a Test Suite with a little effort. You would register your Test Suite with us, and your customer would upload their Test Data to our Server. Your Test Suite can then run against real data that we would be responsible for
· We can act as a Firewall between Test Development and Data
1. We can secure access to customer data within our Test Framework – the data will only be accessible during the Test Run and only through our fully-automated Test Clients
2. We can introduce 3rd-party validation of the Test Suite. The Test Suite will be run in our secure environment with no User manipulation – so if it runs in our Framework you will know that it is fully automated with no requisite custom configuration