Google ends support for popular browser benchmark

Google decided to discontinue its widely known browser benchmark Octane, after determining that it no longer helped the web as it was originally intended. Ever since its launch the benchmarks were used by developers worldwide in order to asses how well the browser runs the included JavaScript.

As years have passed, Octane failed to keep up the pace with ongoing developments. This meant that many of the JavaScript optimizations it suggested were no longer optimal in a real world scenario and in some cases the suggestions actually had a detrimental effect on performance. Not only that, but according to Google, it was found out that some developers even went so far as to cheat into getting higher scores on the test. This was possible due to a bug in Octane which allowed websites to get a 15% performance boost score even though the changes done to trigger this had no effect on the actual performance.

Octane, Google writes, "The engine didn't capture important use cases for the modern web... and often doesn't work the way real-world code does... Unfortunately, similar issues exist in other static or synthetic benchmarks". The general message here should not be that one system was considered outdated and as a result it was discontinued but that the benchmarks we all rely on can easily outlive their usefulness.