by Barry Burke

In today’s fast paced IT world, it is important to be able to deal with change quickly and efficiently. In our private lives and as consumers, we expect rapid responses when interacting with the internet or mobile apps. The same is true when we use enterprise software in our working lives. Slow response times can be directly monetised in hard currency and can also have a damaging impact in terms of the relationship a business has with its customers and suppliers.

What is benchmark testing?

So, what has benchmark testing got to do with this? Benchmark testing is the process of load testing a single component, or an entire end to end IT system, to determine the performance characteristics of the underlying application.

Benchmark tests should be easily repeatable so that any performance measurements captured can be compared against subsequent benchmark tests. This enables documented changes to be made to the application or infrastructure to determine if there is a performance improvement or degradation. This is essential information to have to ensure the ongoing optimization of your JD Edwards system.

What changes can trigger the need to benchmark test?

You may have JD Edwards installed on premise or have (some or all) component workloads running in the cloud. Regardless of where JD Edwards is running, there are a myriad of change events that may necessitate benchmark testing, where you need to compare performance before and after the change event.

Here are just a few examples:

• Moving JDE workloads to the cloud
• Tools release installs
• Database upgrades/major patching and/or tuning
• Platform changes
• Hardware changes
• New company/subsidiary on-boarding
• Disaster Recovery/Continuity tests
• Significant application/configuration changes
• The performance of a system audit
• Significant security model changes

Benchmark testing objectives

Benchmarking enables you to ensure the application meets the minimum requirements stated in your SLA. Guaranteed response times against number of concurrent users should be an established metric, agreed by key stakeholders within your organization and your cloud service provider. This means repeating benchmark tests, tuning and improving and retesting until required performance levels are met.

Another key objective is to determine the breaking point of the application; increasing demand and load until it fails. This allows support to establish the point of failure and understand what specifically in the change event caused the application to fail.

Benchmark testing should be performed using the same software and hardware components as in production. This will give the best possible chance of accurately identifying failures. Each test should be identical to ensure a like for like comparison across the test runs.

It is essential that any failure is documented and that you know how far into the test the fail occurred. How many users were simulated at that time? What applications were running? Where exactly did it fail? It is also essential to be able to distinguish between an application failure and an infrastructure failure. It may be that more memory, kernels, storage changes are required to meet the demands of the change event.

Benchmark testing challenges

Why don’t businesses do a lot more benchmark testing of their business-critical applications? Could it be because it is considered too time consuming, – expensive and complex? Perhaps it’s because it requires a perceived level of collaboration across the business. By failing to perform regular benchmark testing, organizations are failing to mitigate a very significant risk whose impact can be felt right up to boardroom level.

There are many load testing products in the market place that either come with a hefty price tag or are too difficult to make work with JD Edwards applications. This puts customers off.

At DWS, we set out to make benchmark testing accessible to all JD Edwards EnterpriseOne customers. We have done this with Dimension LoadTest™. LoadTest is a solution that is native to JD Edwards applications, is easy to set up, and has a very flexible and affordable SaaS pricing model. It relies on scripts that can be added to LoadTest definitions, which can in turn be assigned to multiple agents to simulate large numbers of virtual users on the JD Edwards system. These load tests can then be run again in a repeatable process to enable accurate benchmark testing on a regular basis.

As the JD Edwards community marches on and adopts the 9.2 release and the Continuous Delivery model, we anticipate load testing as a service will become an essential part of the JDE product lifecycle.

If you would like to learn more load testing, please contact us today to discuss your requirements.