The Apdex: what is it, and what can you do with it?

We all know that applications need to be fast and available. That’s why many organisations actively monitor the performance of their most important applications and can take prompt action if performance deteriorates. However, the question we are frequently asked is “Are my response times good enough?” In the end, response time is an end user satisfied with? And with what response times does IT make an optimal contribution to business goals? With the Apdex, response times are indexed to make them more useful and more widely applicable.

Context-dependent

Is a 2-second response time fast? Or would 5 do just as well? At what point do end users start to complain, or do you start to lose money? Unfortunately, there is no simple answer to that. For some applications, the fastest technically achievable time is around 8 seconds, and that is fine in the perception of the end users.

Take, for example, logging in once per day to a VDI environment. But when it comes to a web store? In that case, research shows that 40% of visitors give up if the loading time lasts more than 3 seconds. In short: good response times depend on their context. The Apdex allows you to quantify and qualify this context.

What is the Apdex?

The Apdex (Application Performance Index is an open industry standard for indexing performance. In the ‘Apdex Alliance’, several organisations have jointly developed a formula, based on which the indexation takes place. The idea behind the Apdex is to enable one to measure whether response times meet the expectation of the end user, the business or the IT manager. These expectations have to be defined and converted into parameters in the formula.

The management of a company always runs the risk of underestimating what happens when employees are challenged on their routines. It is a challenge to have to think in new ways, and therefore habitual thinking is a substantial obstacle among employees when it comes to migration to the cloud.

It is important to turn this into something positive. The internal resistance also stems from the fact that the employees want to use their professionalism in the best possible way and make a difference for the company. With new work routines in the cloud, you are challenged on your professionalism and may become a little less efficient for a while. So how do you make the process a good experience?

How does it work?

In the Apdex formula, response times are categorised into three bandwidths: ‘satisfied’, ‘tolerated’ and ‘frustrated’. For example:

• does logging into application X take a maximum of four seconds? If so, that measurement is marked as satisfied and is assigned a value of 1 point.
• Does it logging in take from 4.1 to 7 seconds? If so, that measurement is marked as tolerated and is assigned a value of half a point.
• Measurements of longer than 7.1 seconds are marked as frustrated. This measurement is assigned 0 points.

The number of points is divided by the number of measurements and plotted on a scale of 0 to 1. For example, 100 measurements have been made:

• 60 measurements were ‘satisfied’ and are assigned 1 point (60*1= 60)
• 30 measurements were ‘tolerated’ and are assigned 0.5 point (30*0.5= 15)
• 10 measurements were ‘frustrated’ and are assigned no points (10*0 = 0)

Now a weighted performance index is calculated in the form of a single figure for the entire application for a specific moment. Now new possibilities arise. For example, the standardisation of satisfaction: with an Apdex value of 0.85, users can be regarded as ‘satisfied’. The aim, therefore, is to attain at least 0.85. In addition, it allows the satisfaction of users of different applications to be compared to each other. Moreover, the Apdex can provide an important guidance resource for investment in specific, poorly performing applications.

Adding context with parameters

To determine what the parameters of the various bandwidths of response times should be, it is important to involve the end user and/or business. For each transaction, there is discussion about which values are necessary, achievable and desirable. Then it is a simple matter to convert the measured performance figures into an Apdex figure per application or even for an entire chain.

For example, a doctor who makes use of an Electronic Patients Dossier. Not all of the transactions in this application have the same parameters:

• The log-in transaction takes place once per day. Both the doctor and IT have no problem with it taking between 6 and 10 seconds. Considering the dossiers that have to be loaded, there are also technical reasons for this. The parameters are set as follows:
  • Satisfied: 1-7 seconds
  • Tolerated: 7-10 seconds
  • Frustrated: more than 11 seconds
• The transaction find patient is executed dozens of times per day. That means it is important for this to happen quickly, otherwise the doctor loses too much time from his already short consultation session.
  • Satisfied: 1-3 seconds
  • Tolerated: 4-5 seconds
  • Frustrated: more than 6 seconds

The parameters are not obvious for all transactions or chains. Specialised parties can help to determine logical values based on experience, the type of application, research, and discussions with keys users and functional management.

Which means?

When the parameters have been set, these can be calculated automatically and are displayed on a dashboard or included in reports. As with many implementations, using Apdex is not just about the technical aspect. You also have to involve the people and processes from the start. People have to be aware of the impact of a low Apdex figure, and processes have to be configured in such a way that improvement programmes can be quickly deployed. The Apdex functions here as ‘glueware’ between people, processes and technology; the business, IT and the management all look to a single point of truth which is understandable and relevant for all parties.