Real World DirectX 10 Performance: It Ain't Pretty
by Derek Wilson on July 5, 2007 9:00 AM EST- Posted in
- GPUs
Company of Heroes
While Company of Heroes was first out of the gate with a DirectX 10 version, Relic didn't simply recompile their DX9 code for DX10; Company of Heroes was planned for DX10 from the start before there was any hardware available to test with. We are told that it's quite difficult to develop a game when going only by the specifications of the API. Apparently Relic was very aggressive in their use of DX10 specific features and had to scale back their effort to better fit the actual hardware that ended up hitting the street.
In spite of the fact that Microsoft requires support for specific features in order to be certified as a DX10 part, requiring a minimum level of performance for features is not part of the deal. This certainly made it hard for early adopters to produce workable code before the arrival of hardware, as developers had no idea which features would run fastest and most efficiently.
In the end, a lot of the DX10 specific features included in CoH had to be rewritten in a way that could have been implemented on DX9 as well. That's not to say that DX10 exclusive features aren't there (they do make use of geometry shaders in new effects); it's just that doing things in a way similar to how they are currently done offers better performance and consistency between hardware platforms. Let's take a look at some of what has been added in with the DX10 version.
The lighting model has been upgraded to be completely per pixel with softer and more shadows. All lights can cast shadows, making night scenes more detailed than on the DX9 version. These shadows are created by generating cube maps on the fly from each light source and using a combination of instancing and geometry shading to create the effect.
Company of Heroes DirectX 9
Company of Heroes DirectX 10
There is more debris and grass around levels to add detail to terrain. Rather than textures, actual geometry is used (through instancing and geometry shaders) to create procedurally generated "litter" like rocks and short grass.
Triple buffering is enabled by default, but has been disabled (along with vsync) for our tests.
We discovered that our cards with 256MB of RAM or less had trouble running with 4xAA and DirectX 10. Apparently this is a known issue with CoH on 32-bit Vista running out of addressable memory. Relic says the solution is to switch to the 64-bit version of the OS, which we haven't had time to test out quite yet.
DirectX 9 Tests
Company of Heroes DX9 Performance
Company of Heroes 4xAA DX9 Performance
Under DX9, the Radeon HD 2900 XT performs quite well when running Company of Heroes. The card is able to keep up with the 8800 GTX here. In spite of a little heavier hit from enabling 4xAA, the 2900 XT still manages to best it's 8800 GTS competition. But the story changes when we move to DX10.
DirectX 10 Tests
Company of Heroes DX10 Performance
Company of Heroes 4xAA DX10 Performance
When running with all the DX10 features enabled, the HD 2900 XT falls to just below the performance of the GeForce 8800 GTS. Once again, the low-end NVIDIA and AMD cards are unable to run at playable framerates under DX10, though the NVIDIA cards do lead AMD.
Enabling 4xAA further hurts the 2900 XT relative to the rest of the pack. We will try to stick with Windows Vista x64 in the future in order to run numbers with this game on hardware with less RAM.
While Company of Heroes was first out of the gate with a DirectX 10 version, Relic didn't simply recompile their DX9 code for DX10; Company of Heroes was planned for DX10 from the start before there was any hardware available to test with. We are told that it's quite difficult to develop a game when going only by the specifications of the API. Apparently Relic was very aggressive in their use of DX10 specific features and had to scale back their effort to better fit the actual hardware that ended up hitting the street.
In spite of the fact that Microsoft requires support for specific features in order to be certified as a DX10 part, requiring a minimum level of performance for features is not part of the deal. This certainly made it hard for early adopters to produce workable code before the arrival of hardware, as developers had no idea which features would run fastest and most efficiently.
In the end, a lot of the DX10 specific features included in CoH had to be rewritten in a way that could have been implemented on DX9 as well. That's not to say that DX10 exclusive features aren't there (they do make use of geometry shaders in new effects); it's just that doing things in a way similar to how they are currently done offers better performance and consistency between hardware platforms. Let's take a look at some of what has been added in with the DX10 version.
The lighting model has been upgraded to be completely per pixel with softer and more shadows. All lights can cast shadows, making night scenes more detailed than on the DX9 version. These shadows are created by generating cube maps on the fly from each light source and using a combination of instancing and geometry shading to create the effect.
Company of Heroes DirectX 9
Company of Heroes DirectX 10
There is more debris and grass around levels to add detail to terrain. Rather than textures, actual geometry is used (through instancing and geometry shaders) to create procedurally generated "litter" like rocks and short grass.
Triple buffering is enabled by default, but has been disabled (along with vsync) for our tests.
We discovered that our cards with 256MB of RAM or less had trouble running with 4xAA and DirectX 10. Apparently this is a known issue with CoH on 32-bit Vista running out of addressable memory. Relic says the solution is to switch to the 64-bit version of the OS, which we haven't had time to test out quite yet.
DirectX 9 Tests
Under DX9, the Radeon HD 2900 XT performs quite well when running Company of Heroes. The card is able to keep up with the 8800 GTX here. In spite of a little heavier hit from enabling 4xAA, the 2900 XT still manages to best it's 8800 GTS competition. But the story changes when we move to DX10.
DirectX 10 Tests
When running with all the DX10 features enabled, the HD 2900 XT falls to just below the performance of the GeForce 8800 GTS. Once again, the low-end NVIDIA and AMD cards are unable to run at playable framerates under DX10, though the NVIDIA cards do lead AMD.
Enabling 4xAA further hurts the 2900 XT relative to the rest of the pack. We will try to stick with Windows Vista x64 in the future in order to run numbers with this game on hardware with less RAM.
Facebook
Twitter
RSS
59 Comments
View All Comments
jay401 - Friday, July 6, 2007 - link
They're right though, the charts need work. They are not intuitive and there are multiple better ways to present 'percent change' data that would make sense on first glance without the reader having to decipher an unintuitive method that is contrary to the readability of the article.DerekWilson - Friday, July 6, 2007 - link
The dx9 vs dx10 scaling graphs have been altered to present the data in a different way.Please let me know if this is still not adequate.
Andyvan - Thursday, July 5, 2007 - link
I had the exact same reaction to the charts. For the Lost Planet chart with the two colors, either pick better (more standard) colors, or make the performance drop bars grow to the left (or down), and the performance increase bars grow to the right (or up).-- Andyvan
PrinceGaz - Thursday, July 5, 2007 - link
The best thing to do with those charts is change them to show relative performance in DX10 compared to DX9, with 100% meaning no change (same performance in DX10 as DX9). Improvements with DX10 give scores above 100%, reduced performance gives a result below 100%.Doing that would make the graphs much easier to understand than the current mess.
sterlinglittle - Thursday, July 5, 2007 - link
This might be a silly question as I can't recall the current status of MultiGPU performance with Vista drivers. Will it be possible to test these games with SLI/CrossFire configurations soon?gigahertz20 - Thursday, July 5, 2007 - link
The results show exactly why I am waiting to buy a DX10 video card, all these people who rushed out to buy a Geforce 8800GTX or AMD 2900XT..hah..especially all those 2900XT fanboys who said the R600 would destroy the 8800GTX in DX10 benchmarks because it has 320 stream processors and a 512-bit memory interface....well guess what, the benchmarks are in and they show the R600 is still the power hunry POS video card it is.KeithTalent - Thursday, July 5, 2007 - link
I'm not sure how it is a 'hah' to the people that purchased these cards as they still blow everything else out of the water in DX9, I mean it is not even close.So for those of us running at higher resolutions (1920x1200 or higher), an 8800/2900 or two made perfect sense (and still does). I doubt very many people were expecting great DX10 performance right away anyway, particularly as the games available barely make use of it.
KT
Sceptor - Thursday, July 5, 2007 - link
I agree with your idea, I've always skipped over one generation of hardware to another.Especially when users are still "testing" Vista gaming for Microsoft, Nvidia and AMD I see no need to part with my money until performance is at least on par with DX9.
Good article...Nice to see some real numbers on DX10 vs DX9
DerekWilson - Thursday, July 5, 2007 - link
there are applications where the 2900 xt does outperform its competition, as is shown by call of juarez.it really depends on how developers go forward. we'll have to wait and see what happens. judging the future of AMD/NVIDIA competition under dx10 isn't really feasible with only 3 apps to go by.
one thing's for sure though, we'd love to see better performance out of the mainstream parts from both camps. And having some parts to fill in the gap between the lower and higher end hardware would be nice too.
defter - Thursday, July 5, 2007 - link
I think the point here is that many claimed that "R6xx is designed for DX10, don't judge it based on DX9 performance blah blah blah". Those claims gave the impression, that relative DX10 performance of R6xx series will be much better than their DX9 performance.Your tests show that on average, R6xx takes a HIGHER performance hit from moving to DX10. Thus, under DX10 R6xx is even SLOWER than it was under DX9.