Last month, we reviewed the HP Z620 workstation and came away impressed with the system’s performance and flexibility. Today, we’re headed back to answer two additional questions. First, should Hyper-Threading be enabled when we’ve got so many cores to work with already? Second, how much performance can software upgrades deliver? Is it better to stick with older packages and put money towards new hardware, or eschew brand-new equipment in favor of the latest copy of 3ds Max, Lightwave, or Maya?
Let’s start with Hyper-Threading. We began by testing performance in the SPECapc benchmarks for Lightwave 9.6, Maya 2009, and 3ds Max 2011. We ran the Maya test in the 2012 edition of the program.
In Lightwave 9.6, the rendering benchmark is actually moderately faster with HT disabled than with it enabled. This is likely explained by the program’s multi-threading support — LW9.6 tops out at 16 threads, which means core resources go unused in a 32-thread system. It also increases the chance that two threads may be allocated to the same core (physical + virtual) rather than using one physical core for each thread.
In Maya, leaving HT enabled or disabled has very little effect on the program’s performance. HT-disabled may have a very slight edge of ~3%, but that’s well within any margin of error.
Next up, 3ds Max 2011 with SP3 and hotfixes installed. The official SPEC test shows the HT-disabled Z620 as slightly faster in the GPU Composite test, but the two configurations are otherwise neck-and-neck.
Here’s our custom Star Destroyers render, designed by Ansel Hsiao of Fractalsponge.net. Here, Hyper-Threading makes a positive difference, knocking just over a minute off our render time.
In Maxwell Render, turning HT off lowers performance in both 1.6.0 and 2.5.1. We’ve also tested the newest version of the program, 2.7.0, but readers should be aware that this release incorporates a built-in benchmark scene. The good news is that there’s no longer any reason to hunt down and use the old one — the bad news is that it’s still called “Benchwell,” despite looking nothing like the previous demo.
In the future, we will use the newer “Benchwell” scene. For now, we’ve broken out the graph data separately.
Hyper-Threading is even more effective in the new scene (and with the new software). Activating HT knocked 20% off our render time in MR 2.5.1, and improves performance in 2.7.0 by 25%.
Don’t jump to conclusions — we aren’t finished yet
These results, taken in aggregate, appear to demonstrate that Hyper-Threading should nearly always be enabled. It’s not true.
Next, let’s look at software updates. Performance enhancements aren’t always delivered with new product versions, but over several years we expect to see performance trend upwards for a given workload. Because most of the SPEC benchmarks are version-specific (Maya is the one exception to this), we compared render times for specific scenes rather than running a wide-spectrum scripted test.
The benefit of this approach is that it gave us the opportunity to compare scaling with modern scenes as well as using scenes created for older versions of the program. As before, we’ll start with Lightwave.
The first Lightwave scene we modeled is an old lighting demo from the early 2000s named Agypten.lws. The second is a scene from the SPEC LW9.6 benchmark named ELS. Hyper-Threading is enabled in both cases.
LW11 delivered a noticeable speed improvement even when working with the ancient Agypten.lws; render-time here improved by 14%. The ELS speedup, however, was downright breathtaking — LW11 knocked a whopping 80% off the scene’s render time compared to Lightwave 9.6. We ran this test multiple times in both applications to confirm the difference, and tested with HT on as well as off to make certain that LW9.6 wasn’t mishandling the file due to a greater-than-expected number of cores.
Maya’s overall score only goes up by six percent, but the program’s CPU score jumps by nearly 41%. I/O performance also increases modestly. If you’re doing CPU-centric work, the upgrade could easily justify itself as far as time savings.
So far, our results indicate that while Hyper-Threading matters, software updates can also have a major impact on overall performance. With 3ds Max, all our previous trends go out the window — so much so, that we had to return to the Hyper-Threading question in order to understand why results looked the way they did. Up until now, we’ve shown you two different 3ds Max tests — the SPECapc 2011 test and a custom Star Destroyer render that uses 3ds Max 2012.
Look at what happens when we take two of the individual scenes from the SPEC test and render them across 3ds Max 2011, 2012, and 2013. All service packs and product updates have been installed for each program. The scenes we’re testing include our Star Destroyer render, the Space Flyby render from SPEC, and the Train-passenger20 render from the same suite. Here’s multi-year performance with Hyper-Threading enabled.
Performance in 3ds Max 2012 when using Mental Ray is absolutely terrible. It takes 2.66x longer to render our Space Flyby and 3.51x as long to render the Train-Passenger20 test scene. Even our Star Destroyer scene, which was created using 3ds Max 2012, is 24% slower when rendered in that program as compared to 3ds Max 2011 . 3ds Max 2013′s performance is much closer to 2011′s, but remains significantly slower overall.
Why is 3ds Max 2012 so much slower than 2011? It’s not because the program is doing more work behind the scenes. We used Windows 7′s built-in Perfmon tool to measure CPU usage across all cores while rendering the same scene in 3ds Max 2011 as well as in 2012.
The Train scene – 3ds Max 2011
The same scene, rendered in 3ds Max 2012
That’s the same file, at the same point in the workload. So what happens if we turn HT off?
Here’s how the two graphs compare against each other.
- 3ds Max 2011: Disabling Hyper-Threading in 3ds Max 2011 makes Space Flyby and our Star Destroyers render 3% and 8.7% more quickly. Train-Passenger, however, renders ~15% more slowly.
- 3ds Max 2012: Turning HT off in 3ds Max 2012 cuts the Space Flyby render time by 40%. Star Destroyer render time goes up 16%, but render time on the Train-Passenger scene falls by 27 minutes. Even with Hyper-Threading off, 3ds Max 2012 is consistently and obviously the slowest of the three programs.
- 3ds Max 2013: Overall performance is significantly better than 3ds Max 2012, but not as fast as 2011. Of the three versions of the program we tested, 3ds Max 2012 was the least affected by Hyper-Threading one way or the other.
The last question we wanted to answer was whether there was something particular to these newer scenes or Mental Ray itself that was responsible for the wide performance variation. To that end, we adopted three additional tests — a simple benchmark of the default scanline render, the Mental Ray 2009 benchmark from this website, and SPEC’s older “Architecture” Mental Ray benchmark, as included in the 3ds Max 2009 benchmark.
These results imply that Hyper-Threading’s impact on performance is difficult to predict and depends on both the production renderer and the type of scene. 3ds Max 2012 is particularly sensitive to HT — or, at least, the version of Mental Ray that shipped with 3ds Max 2012 contains routines which are improperly scheduled when HT is enabled. This performance degradation is something a number of 3ds Max 2012 users noticed, but our research failed to identify a specific culprit. Render settings, RAM usage, and a number of other parameters remain steady across the three program versions, even as performance in those scenes swings wildly.
If you aren’t using 3ds Max, we’d recommend checking out new versions of the major software packages, even if you’ve got a copy that’s only 2-3 years old. HT is an asset in virtually every case, and the performance benefits of a software upgrade might push out the need for new hardware in certain cases. This is most likely to hold true for dual-socket users or six-core Westmere-class hardware. If you’re using Core 2 Quad-era equipment from 2006-2007, you’re more likely to benefit from faster CPUs that aren’t hogtied by Intel’s ancient GTL+-based FSB.
If you’re using 3ds Max 2012 and Mental Ray, you’re likely to benefit from turning HT off, especially if you’re rendering scenes you initially built using 3ds Max 2011. We don’t want to oversell the benefit — not without having more data on the cause — but there’s good reason to try it if you’re looking to improve production render times.
If you’ve got 3ds Max 2011 and you’re happy with its features, we’d be hard-pressed to recommend an upgrade at this point. The 2011 version of the program nearly always leads the pack (our secondary Mental Ray benchmark from 2009 is the sole exception to that).