Correlations in vessel performance indicators

For this purpose, you set up a system to collect data from the ship and a system for analysing the data in order to study the effect of the improved setup over a long time both before and after the improvements were made.

In your data analysis you calculate a performance indicator which was selected to describe the effect of the improvements. Your boss is very sceptical about the effect of the improvements and he is literally looking over your shoulder every day when you get an update on the performance indicator. Quickly you observe, that the performance indicator jumps up and down every day more or less randomly. Your boss is getting nervous.

You look more carefully into the data and you realise that the jumps are not entirely random: There is a weak tendency that the performance indicator is higher when the ship sails faster. In statistical terms this tendency is a correlation. Now, you explain to your boss, that she should not rely too much on the performance indicator of interest, because the changes can simply be due to the ship sailing at different speeds day by day. If your boss is a sensible person – after all most bosses turn out to be sensible – then she will accept your disclaimer and trust your weekly reporting without looking nervously over your shoulder.

However, it turns out that in order to achieve company funding for the improvement project, the data collection and your data analysis, your boss has promised to the CEO that your performance indicator will prove beyond any reasonable doubt whether the improvement project had a significant positive effect or not. 

Unfortunately, one day the CEO  decides that all ships must reduce general speeds by 1 knot in order to save fuel due to rising fuel prices. 

With your knowledge of the correlation between speed and performance indicator you are now able to predict that not only will the performance indicator not show any improved performance. It will actually show a worse performance due to the reduced speed! Maybe you will be able to explain to your boss and the CEO, that this is actually due to the reduced speed, but you will have a very hard time explaining that you are unable to prove beyond reasonable doubt whether the improvements had any effect or not. After all it is difficult to explain to a CEO, that he ruined your experiment by making the data inconclusive and rendered the investment useless. 

Although this scenario may seem farfetched, it is not entirely unrealistic. In a recent study (“Quantitative Comparison of Different Key Performance Indicators” downloadable from link at this page) we observed that three commonly used performance indicators are to different extend correlated to speed or weather conditions – or both. Hence, if you rely too much on these performance indicators as in the example above, then you may end up in a bad corner. 

One of the performance indicators we studied is specified by the ISO 19030 standard. According to the standard, the performance indicator is very precise provided you average over long times. This is a common statistical argument. However, the argument is not valid if the performance indicator is prone to correlations with properties that change systematically. The standard does not account for correlations and does not make any disclaimers about general changes of operations. If an external parameter as speed is changing systematically then presence of correlations will affect the performance indicator systematically and this cannot be accounted for by averaging. In effect the performance indicator will be unreliable and possibly misleading.

Our conclusion is clear: You should definitely care about correlations if your work or job depends on performance indicators. None of the performance indicators we studied were immune to correlations. Hence, you should carefully make disclaimers about the validity of the performance indicators if external parameters show systematic changes or tendencies.