"Testware or Shelfware" - The reality

From a Software vendor's perspective it is easy to be involved in the planning of a project, work with a team in justifying the acquisition of a tool, go in and train a targeted set of users, fulfil the acquisition objectives, see the return on investment achieved and leave behind a happy customer.

Several months down the line, that project team has dispersed and the knowledge of the tool has dissipated. Even if the project leader is aware that a tool is available, he could be forgiven for thinking, "Perhaps it would be easier to carry on the way we've always done it?" i.e. manually.

What companies need is someone to take responsibility for making potential users aware of the capabilities of their tool set and ensuring that the potential benefits and productivity are achieved across the entire organisation.

We appreciate that in these increasing competitive times a role like this does not always exist within an organisation and our company are trying to assist in filling that gap in as many practical ways as we can.

We have introduced a Technical Account Management (TAM) facility to major customers, to assist in this process. The TAM can be utilised in a variety of ways depending on the particular needs of the customer, but at one site; for example, the TAM attends the support teams monthly meetings, with a regular agenda item to update one product group.

There is, also, the option to invite the TAM to specific project planning meetings to advise on making best use of their tool set for that particular project, again ensuring that a wider audience is aware of the functionality that they have. In addition, we are inviting customers to technical briefing sessions across the country.

This enables us to inform our customers of new features that have been incorporated within the products, whilst allowing us to keep them informed of our future plans and in the process provide a platform for customers to keep us up-to-date on their strategic direction.

In addition our Professional Services division can provide assistance at the required level from resourcing the project, to on-site training or "hand-holding", ensuring that our customers don’t have "Shelfware".

A synopsis of a couple of different customer projects may give some food for thought on different business areas where a variety of the tools could be used:

Financial package upgrade:

A large financial institution needed to upgrade their financial package that processed their core business. This is an on-line green screen application running on OS/390.

As is typical in these situations, there was a business critical deadline to implement this new release, 'Detailed User Acceptance' testing was required and IT were told that there was an infinite number of rating permutations. However there was limited availability of testers, plus a lack of the skill set required for test automation.

After a planning meeting with Professional Services a project plan was agreed, utilising the business knowledge of the users and filling the technical gap with two technicians. The planned number of test conditions was agreed at 100,000.

The users used their existing facility to create batches of spreadsheets containing their test conditions and expected resultant quote. The technicians transferred this data to the OS/390 platform and drove automated test scripts from these data files using the scripting language available with the tool.

The quotes, generated by the new release of the application, were scraped from the resulting screens into a file. This file was subsequently transferred back to the PC and automatically compared with the expected values.

Using a mainframe tool meant that some of this activity could be run as batch jobs i.e. background tasks, avoiding putting any excess activity onto the network.

The project was successfully completed a week ahead of schedule, automating a task effectively and accurately that would have taken many months to complete manually.

Due to the success of this project this company is considering other uses of their automation tool e.g. the transfer of a recently acquired company’s 40,000 policies into their system. The estimate for this to be done manually is 12 months.

Successful deployment of the company's website

With recent, high profile incidents where problems with other companies' web sites have had direct impact on share value, the management team saw the successful deployment of their web-site as a business critical project. They now see these IT systems as an integral part of the company's underlying business processes.

As a company's web site develops, the application is becoming further and further integrated into the diversity of their legacy systems and as it continues to grow the number of components, the complexity and the potential number of bottlenecks increases.

Historically, when load/stress testing had been carried out the testers viewed their role as – 'creating the stress to the point at which the system broke'. They didn't have the correct skill set to find out the "where", "why" or "what" of the problem.

Also, as load testing was traditionally carried out at the end of the project and the implementation date loomed, the best that could be hoped for was a patch for the problem. i.e. as the application was performing badly the company may have invested in extra bandwidth or faster PCs, not necessarily solving the problem.

The operators team had been responsible for monitoring production performance, but there was little communication between them and the test team. Their expertise in system management tools was largely untapped in the pre-deployment environment.

The company has implemented a load/manage process throughout the build cycle, right from the database design and network planning. A benchmark environment has been devised:

• database of a specified size created by their automated test tool
• specified number of user transactions simulated and submitted at a specified speed, using their load testing tool
• the response times measured at various PC’s and reported on using the automated testing tool
• bottlenecks are highlighted, solutions trialled, corrective action initiated (manually or automatically) by network management tools

This enables the customer to make fact-based "production readiness" decisions and hence has increased the rate of successful implementation and ensuring availability and reliability.

Using an Automated Test Tool as "middleware" between web and mainframe

As is often the case for companies these days, this High street bank's web applications were growing faster than they could be integrated with their back end, OS/390 applications. Hence the initiation of a project to utilise their Windows-based automated test tool to integrate these 2 environments.

When a new application request is generated via the web site, the application is amended to insert a new record into a new transaction table, at the same time as a master data record is inserted in the master table. The automated test tool is constantly polling for new records in this new transaction table.

The test tool then schedules the appropriate slave machine to process that particular application. This slave machine accesses the corresponding master record and uses this data to drive a request to the credit authorisation application.

Once a decision is generated by the authorisation application, this decision is captured along with other relevant information and a record is inserted into the results table, etc, etc. There is more processing subsequently, again, involving polling of files, creation and deletion of records. This whole process has already been automated.

This has allowed the bank to move ahead more quickly with their web activity than would otherwise have been the case - very critical for this company in this very competitive area.

In conclusion, when we have customers using what may have purely been viewed as "test" automation tools for the projects discussed above as well as:

• Audit
• Training / Demonstrations
• Automatic database population
• Data transfer
• Analysis of production problems
• Automatic screen population from files
• Automation of repetitive tasks

We are continually trying, and encouraging our customers, to "Think outside the box".

In the current economic climate of acquisitions, mergers, Euro possibilities and changes in Government regulations there is much IT work to be done where these tools will be coming into their own.

It is worth noting as well, that test tools are maturing along with the market place. Additional facilities have been provided to make them more flexible, easier to use and to work with emerging technologies, so what does your company need to do?

• Ensure that all the appropriate personnel are aware of and up-to-date with the functionality provided by their tool set, via Technical Account Management, Product Update Briefings, Technical Helpdesk, On-line Technical Support, etc.
• Acquire the knowledge, time required to successfully complete a project by utilising resources available from your vendor, whether that be on-site training, services staff on-site for all or part of the project.

Pauline Havelock, Compuware Limited