Sun suspected of test rigging

Pendragon and Sun battle it out over alleged falsification of test results.

Pendragon Software, developer of the CaffeineMark Java performance results. benchmark, has hit out at Sun Microsystems, after allegedly accusing the workstation giant of cheating.

In a statement, Pendragon claimed that Sun's Just In Time (JIT) Java compiler produces a 'misleading score' by including part of the benchmark code.

Sun had previously claimed that the JIT compiler allows Java applications to run 50 per cent faster on Sun's Solaris operating system than on Windows NT. However, the benchmark creators disputed this fact.

In one of the nine modules that constitute the CaffeineMark test, Sun alleged the Solaris JIT compiler was 50 times faster than any system to date.

Pendragon admitted it was this claim that first aroused its suspicions.

After a slight modification in the test, the company discovered that all other Java environments produced similar results as previously, but Sun Solaris was suddenly 300 times slower.

Ivan Philips, president of Pendragon Software, said the company requested that Sun retract its statement. However, he added: 'Though we made a request, there has been no retraction of the original press release issued by Sun Microsystems. So we felt it would be fair to all Java technology vendors to set the record straight.'

He claimed that the fastest overall CaffeineMark 3 scores are from Windows NT systems running on Intel Pentium II processors at 300MHz.

But Sun responded to the allegations by claiming they had been stirred up by Microsoft. 'Sun blew the doors off the CaffeineMark 3 benchmark, beating NT by 50 per cent. Now Microsoft has gone whining to Pendragon about being beaten. Too bad,' said the statement from Sun.

A Sun representative denied the company had cheated on the benchmark.

'We optimised the product to deliver better performance than the competition,' he claimed.