FutureMark Choosing Drivers!

This is the place to discuss the latest computer hardware issues and technology. Please keep the discussion ON TOPIC!
Post Reply

Your opinion

FutureMark is still respectable
0
No votes
FutureMark needs to explain
2
33%
FutureMark means nothing to me anymore
4
67%
 
Total votes: 6

User avatar
Illuminati
Site Admin
Site Admin
Posts: 2378
Joined: Mon Oct 06, 2003 8:48 am
Location: Wright City, Missouri, USA
Contact:

FutureMark Choosing Drivers!

Post by Illuminati »

I pulled this text from the news post I wrote:
It seems as if we were not the only site to have FutureMark email us about our 3DMARK2003 testing. The email from FutureMark asked us to remove our results from our HIS 9600XT Turbo review because the 53.03 driver we used on the NVidia card we compared it to was not on FutureMark's "Approved Driver List". Further research of this issue revealed that FutureMark actually put this clause in their End-User License Agreement for 3DMARK2003:

"PLEASE NOTE that Section 11 of this Agreement contains terms, which restrict Your right to publish any Results obtained by You using drivers that do not fulfil Futuremark Optimization Guidelines requirements. Updated Optimization Guidelines and list of drivers verified by Futuremark can be found on Futuremark’s website at http://www.futuremark.com."

It seems this afternoon, Overclocker Cafe posted a blurb explaining their thoughts on this issue.

"It almost comes across as if Futuremark is happy only as long as ATi isn’t besting NVidia on its benchmark. That is of course only my opinion. You draw your own conclusions. I want to say I do not know nor have heard of any questionable ties between Futuremark and NVidia nor do I wish to suggest collusion between the two. As for us here, we are now one of the numerous other sites who will not be using 3D Mark 2003 again."

We also disagree with FutureMark's "Approved Driver List" dictating which drivers can be used for 3DMARK2003 testing, thus our future reviews will not feature a comparison using the 3DMARK2003 benchmark until further notice. We feel our readers should see results based on the most recently released drivers for all our products in all tests. And just so everyone understands, this also means that today's released ATI Cat 4.1 drivers are not legal to use with published 3DMARK2003 scores.
Please voice your opinion about this issue!
Justin West
Server Admin & Forum Moderator
Follow me on Twitter | Find us on Facebook
User avatar
Apoptosis
Site Admin
Site Admin
Posts: 33941
Joined: Sun Oct 05, 2003 8:45 pm
Location: St. Louis, Missouri
Contact:

Post by Apoptosis »

Still gathering my thoughts on this one
User avatar
Illuminati
Site Admin
Site Admin
Posts: 2378
Joined: Mon Oct 06, 2003 8:48 am
Location: Wright City, Missouri, USA
Contact:

Post by Illuminati »

I just checked the "Approved Drivers List" today, and the CAT 4.1 drivers that were released yesterday are now on the list. So everyone can publish results using the CAT 4.1's now.
Justin West
Server Admin & Forum Moderator
Follow me on Twitter | Find us on Facebook
User avatar
Illuminati
Site Admin
Site Admin
Posts: 2378
Joined: Mon Oct 06, 2003 8:48 am
Location: Wright City, Missouri, USA
Contact:

Post by Illuminati »

Another update after some more research:
Adrian over at the Rojak Pot has posted an editorial response to the same situation that happened to us regarding use of un-"Approved" 53.03 drivers on nVidia cards. I found this article to be the most conclusive so far in my research/quest for the truth. Adrian points out some very interesting points indeed, most of which I agree with. They even go into a quick driver comparison test using 3dMark2003.
"Granted, they tested NVIDIA's optimization and found that it only affected the Pixel Shader 2.0 test and had no effect on the overall 3DMark score. Still, that should not have been the ONLY approved driver on the list."
In all this research, I also came across a slightly older article posted at Elite Bastards which shows a driver comparison using a different card. The one thing I noticed right away, the 53.03 driver they used was released by eVGA and not WHQL certified.. whereas all the recent 53.03 drivers are released by nVidia themselves and are indeed WHQL certified. But these results are still interesting and could show differences in a card affected by the 53.03 driver optimizations.

I hope to soon have driver comparison results in my hand using our GFFX 5900 Ultra. I already have results in my hand from our GFFX 5600 Ultra and see no difference between the drivers... much like Adrian's article noted.
Justin West
Server Admin & Forum Moderator
Follow me on Twitter | Find us on Facebook
crim
Legit Little One
Legit Little One
Posts: 1
Joined: Fri Jan 23, 2004 11:32 am

Post by crim »

Futuremark has always meaned nothing to me. What are you doing, playing games or playing a benchmark?
User avatar
Apoptosis
Site Admin
Site Admin
Posts: 33941
Joined: Sun Oct 05, 2003 8:45 pm
Location: St. Louis, Missouri
Contact:

Post by Apoptosis »

haha... I like Crim's point of view ;)
User avatar
Illuminati
Site Admin
Site Admin
Posts: 2378
Joined: Mon Oct 06, 2003 8:48 am
Location: Wright City, Missouri, USA
Contact:

Post by Illuminati »

crim wrote:Futuremark has always meaned nothing to me. What are you doing, playing games or playing a benchmark?
The point that a lot of other readers have pointed out is that Synthetic Benchmarks were a to predict how well the card could perform in future games. However, it seems that 3DMark2003 does not have that ability anymore due to nVidia's driver optimizations specifically for 3dMark2003.

I will be posting an editorial on this topic in the near future that will reveal the research I have done the past few days.
Justin West
Server Admin & Forum Moderator
Follow me on Twitter | Find us on Facebook
Post Reply