Today we put a new spin on our benchmarking and for the first time run multiple applications for true multi-tasking benchmarking. We run the dual core Intel 840 versus the single cored Intel 640 processor.
"The Intel 640 took a 24% hit in performance when run with the virus scanner enabled, which is a significant difference in performance. The Intel 840 with Hyper Threading enabled came in only 4% slower, and that isn't a significant performance difference. Dual core easily proves to be the victor here."
Read the article HERE
Let us know what you think of our testing style as it's the first time it has been used and also any feedback you have on the article in general!
Intel Dual Core: Multi-Tasking Benchmarking
-
gvblake22
- Legit Extremist
- Posts: 1111
- Joined: Thu Feb 17, 2005 9:39 am
- Location: Northern Michigan
- Contact:
That is a VERY interesting article with a lot of suprises! Good work!
Much more Quantitative than Anandtech's 840 mutitasking article
I think it is especially interesting to see how HT doesn't seem to help performance in any of your benchmarks, in some cases it even hinders performance! It is very clear cut though that although the dual cores do not offer any performance increase in single threaded apps, when multiple tasks and/or multi-threaded apps are introduced into the mix the advantage of having that "extra CPU" on board seems to help quite a bit.
From what I can gather, Dual Cores will be more beneficial to the "power user" who is constantly multitasking as much as possible (similar to the benchmark situations you have run) while those users who will only be running mildly CPU intenisve programs simultaneously or game without much happening in the background, a good, powerful single core CPU will do just fine.
Good read, thanks!
Much more Quantitative than Anandtech's 840 mutitasking article
I think it is especially interesting to see how HT doesn't seem to help performance in any of your benchmarks, in some cases it even hinders performance! It is very clear cut though that although the dual cores do not offer any performance increase in single threaded apps, when multiple tasks and/or multi-threaded apps are introduced into the mix the advantage of having that "extra CPU" on board seems to help quite a bit.
From what I can gather, Dual Cores will be more beneficial to the "power user" who is constantly multitasking as much as possible (similar to the benchmark situations you have run) while those users who will only be running mildly CPU intenisve programs simultaneously or game without much happening in the background, a good, powerful single core CPU will do just fine.
Good read, thanks!
What I find the common trend to be when it comes to benchmarking both the dual core and Hyperthreading enabled processors is a slight misunderstanding of how the technologies work.
Neither Windows(XP, XPx64, or 2003) nor the processor itself has thread allotment, so unless the software itself utilizes SMP or load balancing, the benefit from multitasking in a dual processor, dual core, or Hyperthreading enabled enviornment will not be fully realized, let alone be benchmarked, until an operating system and/or processor with the ability to assign threads is available.
However, Tom's Hardware has a tool that allows you to manually assign which logical CPU the thread is assigned to. This allows threads to be properly appropriated to individual tasks.
It might be worth a shot to throw a few benchmarks at it:
http://www6.tomshardware.com/cpu/20040528/index.html
You could set Doom III to CPU0, while the other tasks could be assigned to either CPU1, or individually divided among the remaning 3 logical processors.
Neither Windows(XP, XPx64, or 2003) nor the processor itself has thread allotment, so unless the software itself utilizes SMP or load balancing, the benefit from multitasking in a dual processor, dual core, or Hyperthreading enabled enviornment will not be fully realized, let alone be benchmarked, until an operating system and/or processor with the ability to assign threads is available.
However, Tom's Hardware has a tool that allows you to manually assign which logical CPU the thread is assigned to. This allows threads to be properly appropriated to individual tasks.
It might be worth a shot to throw a few benchmarks at it:
http://www6.tomshardware.com/cpu/20040528/index.html
You could set Doom III to CPU0, while the other tasks could be assigned to either CPU1, or individually divided among the remaning 3 logical processors.
-
Illuminati
- Site Admin
- Posts: 2378
- Joined: Mon Oct 06, 2003 8:48 am
- Location: Wright City, Missouri, USA
- Contact:
teqguy,
I don't think we are misunderstanding how the technologies work. Hyperthreading gives the illusion to the Operating system and overlaying software that there is more than one cpu, but then still serializing the execution tasks as they go through the execution cycle of the processor.
(for everyone else's benefit, Dual core offers two physical cores, which means that instead of serializing the various threads, two threads can actually be executed in parallel... meaning at the exact same nano-second).
To my understanding, Windows XP fully supports threads... if it didn't then none of the overlaying SMP software would know how to assign itself to a different processor. It is the operating system's responsibility to organize and synchronize the various threads to the hardware they may be requesting.
Using Tom's tool to benchmark with is not necessarily a good idea. A tool like that forces a process to be assigned to a certain CPU. This overrides the operating system's default load-balancing algorithms. In doing so, you risk an unequal balance in cpu utilization. if doom3 is assigned to cpu0, and all threads assigned cpu1, then when some of the other processes you know about are finished, cpu1 is left empty and cpu0 very busy because it is handling doom3 + any other processes that must use cpu0, so now the benefit of dual core is lost.
It is fairly well known that the operating systems built in load-balancing is best at default... especially with some Linux flavors... but Windows has this too.
I don't think we are misunderstanding how the technologies work. Hyperthreading gives the illusion to the Operating system and overlaying software that there is more than one cpu, but then still serializing the execution tasks as they go through the execution cycle of the processor.
(for everyone else's benefit, Dual core offers two physical cores, which means that instead of serializing the various threads, two threads can actually be executed in parallel... meaning at the exact same nano-second).
To my understanding, Windows XP fully supports threads... if it didn't then none of the overlaying SMP software would know how to assign itself to a different processor. It is the operating system's responsibility to organize and synchronize the various threads to the hardware they may be requesting.
Using Tom's tool to benchmark with is not necessarily a good idea. A tool like that forces a process to be assigned to a certain CPU. This overrides the operating system's default load-balancing algorithms. In doing so, you risk an unequal balance in cpu utilization. if doom3 is assigned to cpu0, and all threads assigned cpu1, then when some of the other processes you know about are finished, cpu1 is left empty and cpu0 very busy because it is handling doom3 + any other processes that must use cpu0, so now the benefit of dual core is lost.
It is fairly well known that the operating systems built in load-balancing is best at default... especially with some Linux flavors... but Windows has this too.
My concern is, as the article on Tom's Hardware clearly illustrates, Windows(XP in particular) has little to no thread appropriation or even a concept of true load balancing. While it does acknowledge Hyperthreading and additional logical processors, it does not manage them whatsoever. That's entirely dependent on the software.
The software can't allocate threads that have not been properly appropriated by the operating system, regardless of whether or not the operating system makes an acknowledgement that they are actual entities.
Additionally, without software to specifically utilize the secondary, tertiary, or quarternary logical processors, those threads remain virtually idle unless otherwise put to use by the operating system and/or other software.
The software can't allocate threads that have not been properly appropriated by the operating system, regardless of whether or not the operating system makes an acknowledgement that they are actual entities.
Additionally, without software to specifically utilize the secondary, tertiary, or quarternary logical processors, those threads remain virtually idle unless otherwise put to use by the operating system and/or other software.
another way to look at it....from a casual users perspective....
Why not just use the CPU as it would be used by a typical user....how many people, other than obviously teqguy, are going to go to Tom's hardware, dowwnload this utility, and implement it?
People are going to buy a CPU, put it in their system, and run it letting the system manage the resources. Good in depth articles are awesome, but people want to see how this technology is going to impact them.
Why not just use the CPU as it would be used by a typical user....how many people, other than obviously teqguy, are going to go to Tom's hardware, dowwnload this utility, and implement it?
People are going to buy a CPU, put it in their system, and run it letting the system manage the resources. Good in depth articles are awesome, but people want to see how this technology is going to impact them.
In order for parallelism or load balancing to be of any current benefit, the concept has to extend beyond the software developer. The "average joe" consumer will eventually be able to disregard parallelism and load balancing, as it will be a standard in the industry. However, in the present time, the concept is theoretical for some and applicable for others.
If the only way to get an actual indication of the multi-tasking potential of a dual core processor is by using a third party tool, then so be it.
At least those with speculation will be given something that's somewhat coherent as to how a dual core processor actually performs.
If the only way to get an actual indication of the multi-tasking potential of a dual core processor is by using a third party tool, then so be it.
At least those with speculation will be given something that's somewhat coherent as to how a dual core processor actually performs.
-
Apoptosis
- Site Admin
- Posts: 33941
- Joined: Sun Oct 05, 2003 8:45 pm
- Location: St. Louis, Missouri
- Contact:
yeah, I actually read that story several months ago when it first came out over at Toms and while it does better address thread usage it's not realistic to the average reader. LR can never suit 100% of our readers, but the majority care about the present and what works with little or no extra effort. So, the point of this article was to address the many questions i received on software applications that are being used today on Windows XP Pro. Maybe in the future we can look at the individual threads closer.
Thanks for understanding.Apoptosis wrote:yeah, I actually read that story several months ago when it first came out over at Toms and while it does better address thread usage it's not realistic to the average reader. LR can never suit 100% of our readers, but the majority care about the present and what works with little or no extra effort. So, the point of this article was to address the many questions i received on software applications that are being used today on Windows XP Pro. Maybe in the future we can look at the individual threads closer.
A person wants to buy something and see the benefit it provides without having to download additional programs and tweaks to get it to do what it should.
there is always tweaking and overclocking etc to fine tune performance, but 99% of people are not going to get so involved as to use thread management, most probably don't even understand it.
As this technology develops, as all do, then software will be written to maximize the performance of it...until then, our job is to show people what they can expect simply by purchasing these products and using them as they come.
there is always tweaking and overclocking etc to fine tune performance, but 99% of people are not going to get so involved as to use thread management, most probably don't even understand it.
As this technology develops, as all do, then software will be written to maximize the performance of it...until then, our job is to show people what they can expect simply by purchasing these products and using them as they come.
Last edited by LVCapo on Mon Apr 11, 2005 6:15 pm, edited 1 time in total.
True, but that doesn't help the "average joe" in understanding why they should fork out for a dual core processor.capper5016 wrote:until then, our job is to show people what they can expect simply by purchasing these products and using them as they come.
I'd hate to see dual core processors fall victim to the same fate as SLi.
-
Apoptosis
- Site Admin
- Posts: 33941
- Joined: Sun Oct 05, 2003 8:45 pm
- Location: St. Louis, Missouri
- Contact:
One thing not to forget about is the pricing on these new processors. Pricing alone is why dual core will not have the same fate as SLi.
A closer look at pricing:
Intel Pentium D 2.8GHz - $241
Intel Pentium 4 630 (3.0GHz) - $226
Intel Pentium 4 520 (2.8GHz) - $167
The Intel 630 is the cheapest priced and lowest frequency 6XX series processor on the market. It is also only priced $15 under then upcoming Pentium D 2.8GHz. Intel is bringing out dual core for just dollars more than the previous generation processors. As you can tell from todays article dual core has advantages even without setting up the threads and for cheap. I say cheap because you are basically getting another prescott cpu for an extra $15 or so.
Hopefully now that you look at the low end pricing and even high end pricing you will note that consumers will not be "forking out" like you stated. Intel said on the record that the Intel 840 will be roughly $1000 which is $200 below the price of the 3.73 EE when it came out at $1200 for trays of one thousand.
I'm actually impressed and shocked that Intel has been able to keeping pricing at the points they are today. Pretty nice eh?
A closer look at pricing:
Intel Pentium D 2.8GHz - $241
Intel Pentium 4 630 (3.0GHz) - $226
Intel Pentium 4 520 (2.8GHz) - $167
The Intel 630 is the cheapest priced and lowest frequency 6XX series processor on the market. It is also only priced $15 under then upcoming Pentium D 2.8GHz. Intel is bringing out dual core for just dollars more than the previous generation processors. As you can tell from todays article dual core has advantages even without setting up the threads and for cheap. I say cheap because you are basically getting another prescott cpu for an extra $15 or so.
Hopefully now that you look at the low end pricing and even high end pricing you will note that consumers will not be "forking out" like you stated. Intel said on the record that the Intel 840 will be roughly $1000 which is $200 below the price of the 3.73 EE when it came out at $1200 for trays of one thousand.
I'm actually impressed and shocked that Intel has been able to keeping pricing at the points they are today. Pretty nice eh?
Well, it balances out when you factor in the cost of the new i955 based boards required to use a Pentium D.
I think Intel is starting to realize that the "if you can't beat them, join them" saying isn't the worst marketing strategy they could have.
If they manage to uproot AMD's standing in the budget market, they'll have a two pronged approach at driving them out of the market.
The lower Intel goes, the lower AMD must go... and when you consider which company has fewer resources, dependencies, and a smaller hold of the market, it's not hard to see AMD gone within 4-7 years.
I think Intel is starting to realize that the "if you can't beat them, join them" saying isn't the worst marketing strategy they could have.
If they manage to uproot AMD's standing in the budget market, they'll have a two pronged approach at driving them out of the market.
The lower Intel goes, the lower AMD must go... and when you consider which company has fewer resources, dependencies, and a smaller hold of the market, it's not hard to see AMD gone within 4-7 years.
-
Apoptosis
- Site Admin
- Posts: 33941
- Joined: Sun Oct 05, 2003 8:45 pm
- Location: St. Louis, Missouri
- Contact:
well keep in mind that the Intel 955X chipset is the same as the 925XE chipset in terms of pricing. So once again new technology for roughly the same MSRP.
I'm not going to thread crap on here, but check this thread about chipset pricing: http://forums.legitreviews.com/viewtopic.php?t=1649
So again no balancing!
I'm not going to thread crap on here, but check this thread about chipset pricing: http://forums.legitreviews.com/viewtopic.php?t=1649
So again no balancing!
I've been saying that I am neither a fan of nor a believer in SLI...seems the market disagrees... I just can't see buying a $240-270 motherboard, slapping in a pair of $200 video cards, and bragging that you are getting the same performance as a $140 motherboard and $420 video card......doesn't make much sense to me....but it is geared more to the enthusiast than the casual user.teqguy wrote:Well, basically they took the Double Mint gum theme song and made it an applicable concept...
-
Illuminati
- Site Admin
- Posts: 2378
- Joined: Mon Oct 06, 2003 8:48 am
- Location: Wright City, Missouri, USA
- Contact:
ok, I read the article from Toms that is linked above... here are my comments after reading that article...
First off, I have to say that I'm disappointed that the authors did not outline how they ran those tests. as far as any readers of that article can tell, that was not a multitasking article at all. It appears that only the application shown in the graph was being run during the tests, so really this article and the multitasking article LR released this morning are not even comparable because Toms article covers NO multitasking.
Secondly, Using a utility like Toms (which appears to just be a different UI to the Task Manager ability) is still not a viable tool for the average user because the average user is not going to have the required knowledge to efficiently manually assign tasks to cpus to achieve optimal system performance.
Experienced users who are knowledgable in single and multi threaded apps, and multitasking concepts will be able to achieve better performance of A FEW apps, but not without the expense of performance from other tasks. So lets say doom3 and a virus scanner are running... if doom3 is set to always use the physical CPU and the virus scanner kicks on, doom3 should have a slightly higher framerate than if doom3 sometimes used the logical cpu.... however, this will cause the virus scanner to take longer to complete, thus extending the total time doom3 is running with its crippled framerate... to most avid gamers, it will be a toss-up to decide which method is better.
All Toms article explains is which of the apps they tested are threaded and which are not... it does not show that their tool is better or more efficient when used in a heavy multitasking scenario... in fact, I can assure everyone that only the single threaded app that is assigned to the physical cpu will benefit, while the rest of the thousands of processes on your system will slightly suffer.
First off, I have to say that I'm disappointed that the authors did not outline how they ran those tests. as far as any readers of that article can tell, that was not a multitasking article at all. It appears that only the application shown in the graph was being run during the tests, so really this article and the multitasking article LR released this morning are not even comparable because Toms article covers NO multitasking.
Secondly, Using a utility like Toms (which appears to just be a different UI to the Task Manager ability) is still not a viable tool for the average user because the average user is not going to have the required knowledge to efficiently manually assign tasks to cpus to achieve optimal system performance.
Experienced users who are knowledgable in single and multi threaded apps, and multitasking concepts will be able to achieve better performance of A FEW apps, but not without the expense of performance from other tasks. So lets say doom3 and a virus scanner are running... if doom3 is set to always use the physical CPU and the virus scanner kicks on, doom3 should have a slightly higher framerate than if doom3 sometimes used the logical cpu.... however, this will cause the virus scanner to take longer to complete, thus extending the total time doom3 is running with its crippled framerate... to most avid gamers, it will be a toss-up to decide which method is better.
All Toms article explains is which of the apps they tested are threaded and which are not... it does not show that their tool is better or more efficient when used in a heavy multitasking scenario... in fact, I can assure everyone that only the single threaded app that is assigned to the physical cpu will benefit, while the rest of the thousands of processes on your system will slightly suffer.
I wasn't trying to shed any light on the article, but rather the tool that's included with it.
What I gathered from the LR review is that all of the tests were conducted without manually specifying which application was assigned to which logical processor.
This would lead me to believe that all of the processes were automatically executed on the primary logical processor and not making use of the second logical processor at all.
However, if you ran the tests conducted in this review with that tool, you'd probably see two applications entirely uninhibted by each other, which is ideally how load balanced multitasking should be. Correct?
I could be wrong, but it wouldn't hurt to give it a shot, would it?
What I gathered from the LR review is that all of the tests were conducted without manually specifying which application was assigned to which logical processor.
This would lead me to believe that all of the processes were automatically executed on the primary logical processor and not making use of the second logical processor at all.
However, if you ran the tests conducted in this review with that tool, you'd probably see two applications entirely uninhibted by each other, which is ideally how load balanced multitasking should be. Correct?
I could be wrong, but it wouldn't hurt to give it a shot, would it?
-
Apoptosis
- Site Admin
- Posts: 33941
- Joined: Sun Oct 05, 2003 8:45 pm
- Location: St. Louis, Missouri
- Contact:
I 100% see your point as I did when I looked at your post the first time. I just don't think you see where we are coming from. Our first article on dual core showed single application testing. Most using synthetic benchmarks like Sandra, Everest, Futuremark, and so on. After the article was posted i received over a 100 e-mails and read dozens of posts on other forums where people were unhappy with not seeing real multi-tasking benchmarks. When I did my phone interviews with Reuters news, CNN, and the Wall Street Journal all three wanted to see more real world numbers for investors and their readers. Real world meaning applications they actually use. Downloading files with IE/FireFox, Virus Scans, watching DVD's/MP3's Encoding, Decoding, Gaming, Folding are all part of what consumers are doing today on Wix XP Pro/Home.
So as a reviewer I have to come up with benchmarks to be used that are acurate and realistic. I also have to figure in time and how many times they should be run and then take the average of the scores. (None of my benchmarks are a single test run) The keyword thus far is realistic. So i can test two ways.
Real versus thread management
This was an easy one for me because no one even mentioned managing the threads after my Dual Core article with 4 threads. Does Intel expect the public to manage their threads? Heck no! Do you expect more than 1% of the consumers with dual core to manage their threads? nope Does anyone with two thread on their Intel PC's manage their threads now, meaning today? nope So like I said a couple days ago I can't make everyone happy, but the goal of Legit Reviews is to hit the mainstream enthusiast market. This is the market we have aimed the site at and is also who is in mind when the articles are thought up.
So as a reviewer I have to come up with benchmarks to be used that are acurate and realistic. I also have to figure in time and how many times they should be run and then take the average of the scores. (None of my benchmarks are a single test run) The keyword thus far is realistic. So i can test two ways.
Real versus thread management
This was an easy one for me because no one even mentioned managing the threads after my Dual Core article with 4 threads. Does Intel expect the public to manage their threads? Heck no! Do you expect more than 1% of the consumers with dual core to manage their threads? nope Does anyone with two thread on their Intel PC's manage their threads now, meaning today? nope So like I said a couple days ago I can't make everyone happy, but the goal of Legit Reviews is to hit the mainstream enthusiast market. This is the market we have aimed the site at and is also who is in mind when the articles are thought up.