Real World DirectX 10 Performance: It Ain't Pretty
by Derek Wilson on July 5, 2007 9:00 AM EST- Posted in
- GPUs
Call of Juarez
There has been quite a bit of controversy and drama surrounding the journey of Call of Juarez from DirectX 9 to DirectX 10. As many may remember, AMD handed out demos of the DirectX 10 version of Call of Juarez prior to the launch of R600. This build didn't fully support NVIDIA hardware, so many review sites opted not to test it. On its own, this is certainly fine and no cause for worry. It's only normal to expect a company to want to show off something cool running on their hardware even if it isn't as fully functional as the final product will be.
But, after NVIDIA found out about this, they set out to help Techland bring their demo up to par and get it to run properly on G80 based systems. Some publications were able to get an updated build of the game from Techland which included NVIDIA's fixes. When we requested the same from them, they declined to provide us with this updated code. They cited the fact that they would be releasing a finalized benchmark in the near future. Again, this was fine with us and nothing out of the ordinary. We would have liked to get our hands on the NVIDIA update, but it's Techland's code and they can do what they want with it.
Move forward to the release of the of the Call of Juarez benchmark we currently have for testing, and now we have a more interesting situation on our hands. Techland decided to implement something they call "HDR Correct" antialiasing. This feature is designed to properly blend polygon edges in cases with very high contrast due to HDR lighting. Using a straight average or even a "gamma corrected" blend of MSAA samples can result in artifacts in extreme cases when paired with HDR.
The real caveat here is that doing HDR correct AA requires custom MSAA resolve. AMD hardware must always necessarily perform AA resolves in the shader hardware (as the R/RV6xx line lack dedicated MSAA resolve hardware in their render backends), so this isn't a big deal for them. NVIDIA's MSAA hardware, on the other hand, is bypassed. The ability of DX10 to allow individual MSAA samples to be read back is used to perform the custom AA resolve work. This incurs quite a large performance hit for what NVIDIA says is little to no image quality gain. Unfortunately, we are unable to compare the two methods ourselves, as we don't have the version of the benchmark that actually ran using NVIDIA's MSAA hardware.
NVIDIA also tells us that some code was altered in Call of Juarez's parallax occlusion mapping shader that does nothing but degrade the performance of this shader on NVIDIA hardware. Again, we are unable to verify this claim ourselves. There are also other minor changes that NVIDIA feels unnecessarily paint AMD hardware in a better light than the previous version of the benchmark.
But Techland's response to all of this is that game developers are the one's who have the final say in what happens with their code. This is definitely a good thing, and we generally expect developers to want to deliver the best experience possible to their users. We certainly can't argue with this sentiment. But whether or not anything is going on under the surface, it's very clear that Techland and NVIDIA are having some relationship issues.
No matter what's really going on, it's better for the gamer if hardware designers and software developers are all able to work closely together to design high quality games that deliver a consistent experience to the end user. We want to see all of this as just an unfortunate series of miscommunications. And no matter what the reason, we are here today with what Techland has given us. The performance of their code as it is written is the only thing that really matters, as that is what gamers will experience. We will leave all other speculation in the hands of the reader.
So, what are the important DirectX 10 features that this benchmark uses? We see geometry shaders to simulate water particle effects, alpha-to-coverage for smooth leaf and grass edges, and custom MSAA resolve for HDR correct AA.
Call of Juarez Performance
The AMD Radeon HD 2900 XT clearly outperforms the GeForce 8800 GTS here. At the low end, none of our cards are playable under any option the Call of Juarez benchmark presents. While all the numbers shown here are with large shadow maps and high quality shadows, even without these features, the 2400 XT only posted about 10 fps at 1024x768. We didn't bother to test it against the rest of our cards because it just couldn't stack up.
Call of Juarez 4xAA Performance
With 4xAA enabled, our low-end NVIDIA hardware really tanks. Remember that even these cards must resolve all MSAA samples in their shader hardware. AMD's parts are designed to always handle AA in this manner, but NVIDIA's parts only support the feature inasmuch as DX10 requires it.
We do see some strange numbers from the low-end NVIDIA cards at 1600x1200, but its likely that they performed so poorly here that rendering certain aspects of the scene failed to the point of improving performance (in other words, it's likely not everything was rendered properly even though we didn't notice anything).
There has been quite a bit of controversy and drama surrounding the journey of Call of Juarez from DirectX 9 to DirectX 10. As many may remember, AMD handed out demos of the DirectX 10 version of Call of Juarez prior to the launch of R600. This build didn't fully support NVIDIA hardware, so many review sites opted not to test it. On its own, this is certainly fine and no cause for worry. It's only normal to expect a company to want to show off something cool running on their hardware even if it isn't as fully functional as the final product will be.
But, after NVIDIA found out about this, they set out to help Techland bring their demo up to par and get it to run properly on G80 based systems. Some publications were able to get an updated build of the game from Techland which included NVIDIA's fixes. When we requested the same from them, they declined to provide us with this updated code. They cited the fact that they would be releasing a finalized benchmark in the near future. Again, this was fine with us and nothing out of the ordinary. We would have liked to get our hands on the NVIDIA update, but it's Techland's code and they can do what they want with it.
Move forward to the release of the of the Call of Juarez benchmark we currently have for testing, and now we have a more interesting situation on our hands. Techland decided to implement something they call "HDR Correct" antialiasing. This feature is designed to properly blend polygon edges in cases with very high contrast due to HDR lighting. Using a straight average or even a "gamma corrected" blend of MSAA samples can result in artifacts in extreme cases when paired with HDR.
The real caveat here is that doing HDR correct AA requires custom MSAA resolve. AMD hardware must always necessarily perform AA resolves in the shader hardware (as the R/RV6xx line lack dedicated MSAA resolve hardware in their render backends), so this isn't a big deal for them. NVIDIA's MSAA hardware, on the other hand, is bypassed. The ability of DX10 to allow individual MSAA samples to be read back is used to perform the custom AA resolve work. This incurs quite a large performance hit for what NVIDIA says is little to no image quality gain. Unfortunately, we are unable to compare the two methods ourselves, as we don't have the version of the benchmark that actually ran using NVIDIA's MSAA hardware.
NVIDIA also tells us that some code was altered in Call of Juarez's parallax occlusion mapping shader that does nothing but degrade the performance of this shader on NVIDIA hardware. Again, we are unable to verify this claim ourselves. There are also other minor changes that NVIDIA feels unnecessarily paint AMD hardware in a better light than the previous version of the benchmark.
But Techland's response to all of this is that game developers are the one's who have the final say in what happens with their code. This is definitely a good thing, and we generally expect developers to want to deliver the best experience possible to their users. We certainly can't argue with this sentiment. But whether or not anything is going on under the surface, it's very clear that Techland and NVIDIA are having some relationship issues.
No matter what's really going on, it's better for the gamer if hardware designers and software developers are all able to work closely together to design high quality games that deliver a consistent experience to the end user. We want to see all of this as just an unfortunate series of miscommunications. And no matter what the reason, we are here today with what Techland has given us. The performance of their code as it is written is the only thing that really matters, as that is what gamers will experience. We will leave all other speculation in the hands of the reader.
So, what are the important DirectX 10 features that this benchmark uses? We see geometry shaders to simulate water particle effects, alpha-to-coverage for smooth leaf and grass edges, and custom MSAA resolve for HDR correct AA.
The AMD Radeon HD 2900 XT clearly outperforms the GeForce 8800 GTS here. At the low end, none of our cards are playable under any option the Call of Juarez benchmark presents. While all the numbers shown here are with large shadow maps and high quality shadows, even without these features, the 2400 XT only posted about 10 fps at 1024x768. We didn't bother to test it against the rest of our cards because it just couldn't stack up.
With 4xAA enabled, our low-end NVIDIA hardware really tanks. Remember that even these cards must resolve all MSAA samples in their shader hardware. AMD's parts are designed to always handle AA in this manner, but NVIDIA's parts only support the feature inasmuch as DX10 requires it.
We do see some strange numbers from the low-end NVIDIA cards at 1600x1200, but its likely that they performed so poorly here that rendering certain aspects of the scene failed to the point of improving performance (in other words, it's likely not everything was rendered properly even though we didn't notice anything).
Facebook
Twitter
RSS
59 Comments
View All Comments
misaki - Thursday, July 5, 2007 - link
So Nvidia and AMD actually complained about their "mainstream" parts being below par?It sounds to me like they are seriously out of touch. Media center PCs will get their 8400 and 2400 cards for h264 acceleration. Gamers with lots of money will buy those $400+ cards as usual. But your average gamer in the $200 market is stuck with junk that is unplayable for dx10 games and performs like previous gen hardware that just barely makes the grade for current games already on the market.
What is so hard to understand?
MadBoris - Thursday, July 5, 2007 - link
I'm glad that some of the things that were said in the article, are starting to be said. DX10 reminds me of Physx, great concept but can't possibly succeed due to certain hurdles in the technology being able to actually take off. Unfortunately DX10 also isn't going to be what we all hoped it would be in real performance, video drivers are not the only reason.There is something real counterintuitive with quality native DX10 rendering support.
There is very little incentive for a developer to produce a good DX10 renderer when developers have DX9 support on Vista along with many of their real current goals of console support for more customer base. With only so many hours a day, console support is much more lucrative with new people having access to buying a title, that will also actually run just fine with only DX9 on a Vista platform, never needing or really benefitting from DX10.
The costs of making DX10 games outweigh the benefits, and the benefits aren't currently that palpable in the first place. Furthermore, actual DX10 performance will rarely ever be all that positive, something that is too early to prove, but rather negative even though apples to apples DX9 to DX10 comparisons cannot be made.
As to the low end parts, it's just marketing, theirs no "real" value DX10 for $100 -$150 for gaming. If people want a good video value purchase for games, $200 is where the good quality/price starts and it's mostly in a previous generation, not some value card for $150. Whether we agree with that price point being 'worth it' is another matter altogether and is purely personal preference. I don't know why Nvidia or ATI even introduced low end DX10 compatible hardware when customers will only get angry at developers or video mfr's for the blunder of underpowered hardware for high end game titles. This low end DX10 hardware mystified me. it was either going to slow down DX10 progress or have to be ignored. It seems obvious that all high end games will have to pretty much ignore the low end parts for achieving acceptable framerates with DX10 for new eye candy titles. They should have left DX10 out entirely in low end, but they had to include it because of competition between AMD/Nvidia, neither wanted to leave the other with a marketing advantage of saying, look we have DX10. DX10 with it's lofty goals of being able to render more in a scene and produce even greater quality eyecandy is at odds with low price. Higher quality rendering will always be at odds with low price, they are mutually exclusive. Low price never is going to give you good performance and quality, people should really start being realistic as to what they are paying for at a $100 - $150 price point, it's a consumer expectations problem. Low end hardware will work fine for games like Sims and those games will target low end hardware, but not high end games for higher resolutions and decent frame rates.
In the end, with future goggles on I think the picture is becoming quite clear that DX10 will become one of the DX API's that becomes mainly skipped (if a decent DX successor becomes available in next few years). The only time it will really make sense to go above DX9 native support is when Vista saturates >%85 gaming market share. In the several years that that will take, DX11 or higher should be available and will be superior to DX10, so DX10 in hindsight will really end up being just a marketing ploy to upgrade to Vista, little more.
Glad the mask is starting to come off and more people are being able to see the picture clearer that making any purchases around DX10 with a GPU or OS is silly and bound to cause frustration.
strafejumper - Thursday, July 5, 2007 - link
i upgraded recently - but ended up only spending under $300 for a core2duo systemthis is why - people were saying get a DX10 card - future proof
i decided to keep my old agp card because i felt real dx10 games and real dx10 hardware were not here yet.
i'm happy i made i feel the right choice and didn't spend money on new psu, sata drives etc. so i could have a $$$ dx10 card only to play call of juarez at 14 fps.
stromgald - Thursday, July 5, 2007 - link
I have to agree. I was considering trying to get an 8600 for my SFF PC, but after looking at this, I'm probably going to hold off until the next generation. HTPC and SFF PCs just can't handle the heat an 8800 series generates, and I want at least playable DX10 performance.jay401 - Thursday, July 5, 2007 - link
Seriously, F them. It's pathetic they're trying to pawn off half-assed hardware as "mid-range enthusiast" parts when they can't even perform as good as the mid-range from the previous generation. Jerks.
jay401 - Thursday, July 5, 2007 - link
Another new article showing how DX10 Vista performance propaganda is garbage.Gotta love people who try to act superior b/c they bought Vista for gaming when all it does is suck up more system resources and uses DX10, the combination of which will easily inhibit performance on equivalent hardware.
BigLan - Thursday, July 5, 2007 - link
"They both argued that graphics cards are no longer just about 3D, and additional video decode hardware and DX10 support add a lot of value above the previous generation."Yeah, I don't buy into this either. I've pretty much given up on 'video decode,' be it avivo or purevideo. You end up stuck with using a specific product, rather than ati or nvidia opening the features for any developer to access. Right now, it's only useful with the latest windvd, powerdvd or nero but you have to hope your driver version is the right one, and doesn't (and probably never will) work for x264 or xvid content.
Purevideo is horribly branded by nvidia - is it card features that everyone has access to, or do you have to buy it from them? And has ati actually released their avivo video converter to the public? Could I use it to compress some of my recorded tv shows from mpeg to xvid?
Maybe this is like the mpeg2 decoder situation was in 98/99, in which case we should just wait for cpu speeds to increase and mean that we don't need video decode acceleration.
titan7 - Thursday, July 12, 2007 - link
I agree. How much quicker would things be if all the video transistors were spent on more shader processors? I want my video card for video games. I want my dvd player for movies.vailr - Thursday, July 5, 2007 - link
Comparing mid-range DX10 cards:(lowest prices found via froogle.com; shipping not included)
Radeon 2600XT 256Mb ~$145
http://www.ewiz.com/detail.php?p=AT-2600XT&c=f...">http://www.ewiz.com/detail.php?p=AT-2600XT&c=f...
nVidia 8600GT 256Mb ~$100 (after $15 MIR)
http://www.newegg.com/Product/Product.asp?Item=N82...">http://www.newegg.com/Product/Product.asp?Item=N82...
How is the 2600XT worth the added $45 v. the 8600GT?
Comdrpopnfresh - Thursday, July 5, 2007 - link
I have a 7600gt oc'd to 635/800. I get 30+fps with better than default settings, with AAx2 set @ 1024x768. Why do these cards not seem to do much better (given they are @ 1280x1024)?