Stuttering in D3D application

magwo001

Hello,

we're experiencing stuttering in our D3D game. With stuttering, I mean that the game loop will generally take 20-30 milliseconds to complete, running smoothly at ~30 fps. But then, like once per second or so, the game loop will take like 150 milliseonds or more to complete.

The stuttering happens on many different computers so I don't think it's a driver/hardware/setup problem, but rather something that we're doing in our rendering that's causing the stuttering. Many other games run fine on these computers.

I've profiled parts of the code.. and usually the huge delays come from D3D Present() method or the actual rendering. Does this indicate that there's a problem with the rendering command queue being filled up and stalled or something


Are there any known common mistakes one can make in D3D code that will cause the application to stutter like this, once every 20-30 frames or so It seems like there should be a simple remedy for this, since the framerate is generally quite decent - it's just that the application seems to halt occasionally.

I know that a few other games have had problems like these, namely U1, UT2003, DDO.. and probably some others.

Ideas

Alexey Barkovoy

Most often stuttering is caused by dynamic creation of resources during gameplay. Other reason can be that you allocated more resources than total size of video memory and occasionally Direct3D runtime will swap textures in/out of video memory.

magwo001

We are not loading or unloading any resources during the time that the hickups occur.

Your second suggestion seems somewhat more likely, but still unlikely, because I'm running this on a 128 MB video card, and NvPerfHud4 reports a usage of about 25 MB video memory.


I just get the feeling that this is some setting that has gone wrong, or that we're overlooking a simple detail.

JeremyAtGosub

Full screen or windowed mode I've seen what you described in Windowed mode on many different computers. I've only seen it happen on one computer in full screen mode. My guess for Windowed mode is that it has something to do with GDI (because, the more stuff GDI is doing, the worse it gets and the delay seems to always be in the present function). My guess for full screen mode is that another process is blocking the application about once a second (because the delay seems to happen at random times during execution). I know it's not GC related in either case.

-Jeremy

Alexey Barkovoy

Other reason could be what at these periods of time you are starting to draw opbjects using new shaders. And these shaders are being compiled by internal driver compiler, causing stuttering.

ThomasAG

I have experienced the exact same problem.

The problem turned out to be due to greedy threads and windows insisting on doing some background work. I had a render thread and a thread optimizing a large mesh. The optimizer was trying to use 100% cpu time (on a dual core system) and the renderer was just rendering along.

When some background work had to be done, my optimizer was not about to be stopped, so the renderer was blocked for a short while, while something went on in the background, and then ran again.

The solution was to down prioritize my mesh optimization, so when windows insisted on getting cpu time, the optimizer was paused, and not the renderer. Problem went away.

My experience relates heavyly to dual core systems with two threads, but perhaps it can shed some light on your problem.

magwo001

The issue seems to appear in both window and fullscreen mode. At the moment we're using only fixed-function due to the need to support very old hardware, so it shouldn't be shader recompilations.

I noticed something strange in fullscreen mode also. The framerate seems to be fluctuating in a semi-sinusoidal fashion.. like a cyclic pattern as observed in two different frequencies that are added.

In fullscreen the framerate cycles between about 10 FPS and 40-50 FPS.. the time for a complete cycle is about 10-15 seconds.. quite weird.

Regarding threads, we are not using more than one thread for the game loop, but we ARE developing on dual-core machines.


I'm sure I seem somewhat confused, which is because I didn't actually write the 3D engine in the first place. The guy who wrote it in the first place is busy with other tasks.

Jack Hoxley

Even if you're not using multiple threads yourself you could be influenced by other threads in the system. I figure they're better at it these days, but AV software used to be guilty of grabbing lots of CPU time every now and then and leading multimedia apps to randomly pause...

Another issue I've had in the past was with log-files - on high logging levels I was streaming lots of text to a file and whenever the buffer was committed to disk it'd pause slightly (I/O bound) and then carry on.

One other thing that hasn't been suggested so far is your CPU/GPU synchronization. Your CPU-based application can keep on piling work on the GPU and get several frames ahead; once the command buffer is full D3D will force your application to stall until it can accept more commands - this stall often happens on Present() calls.

IIRC the "Accurately profiling Direct3D API Calls" paper in the SDK has code to force a pipeline stall using queries. You normally wouldn't want to do this, but if you place one of these in EVERY frame then you'll stop the CPU getting too far ahead and you effectively replace one big occasional stall for lots of small stalls thus smoothing out the overall frame-rate. Trying this could be a useful way of determining if Direct3D or the GPU is at all responsible...

hth
Jack

magwo001

Interesting!

I added the buffer flushing code code just before the Present() call:
// 1. Create an event query from the current device
IDirect3DQuery9* pEvent;
m_pD3DDevice->CreateQuery(D3DQUERYTYPE_EVENT, &pEvent);

// 2. Add an end marker to the command buffer queue.
pEvent->Issue(D3DISSUE_END);

// 3. Empty the command buffer and wait until the GPU is idle.
while(S_FALSE == pEvent->GetData( NULL, 0, D3DGETDATA_FLUSH ))
;


It seems like the stuttering is completely removed.

We tried two different scene complexities that resulted in following behaviours:
Heavy scene without flushing: 24 FPS - stuttering
Heavy scene with flushing: 15 FPS - no stuttering

Light scene without flushing: 38 FPS - stuttering
Light scene with flushing: 20 FPS - no stuttering


So, I hope I'm drawing the correct conclusion when I say that the stuttering is induced by command buffer-related problems.

HOWEVER, this does not fix our problem since the framerate is lowered to (expected) an unacceptably low value.

What are the suggested fixes to this problem
And, why are we running into this problem, while most people don't seem to

Jack Hoxley

You're mostly correct on your conclusion, but it is more general in that the CPU is getting ahead of the GPU. The command buffer filling up is more your indicator than your cause.

I would recommend using "PIX for Windows" along with that article on profiling API calls to try and build up some information about where your time is being spent. Which bits of work require little CPU time but a lot of GPU time Profiling is absolutely key here - you NEED results to both determine where your problem is as well as to measure any improvements/fixes.

Could also be worth considering the hardware used - it is always possible that your GPU is too slow (or relatively, your CPU is too fast).

hth
Jack

magwo001

Yes, I've tried using PIX, and gathered some information with it. However, it's very complex and I'm not actually a pure 3D coder - I just fill in for the person who would normally handle this issue.

I was not able to draw any concrete conclusion from the PIX data. I just saw that the dt was very high every now and then, and that the PIX-calculated framerate was dropping firmly to 4-5 FPS at those points.

But I'm not really getting your suggestion about the CPU being too fast. Any other game is running fine on this hardware configuration, so why would it be in our case that the "CPU is getting ahead of the GPU" It seems strange and illogical.


Surely this must be a fairly common problem with common solutions

Jack Hoxley

It can take a bit of time/effort to learn how to get the most from PIX, so may well not be the best tool for you.

In order for the CPU and GPU to be running in parallel at maximum efficiency they must be producing (CPU) and consuming (GPU) at a roughly equal rate. If the GPU consumes too quickly it'll be idle whilst waiting for the CPU to catch up, alternatively if the CPU produces too quickly then the GPU will have to block it until its cleared some of the backlog.

You wouldn't be going too wrong to consider the CPU/GPU dynamic in the standard producer-consumer multiprogramming concept.

My comment about the CPU being too fast isn't usually the case - it was more a throw-away comment to illustrate the possibility. It is more likely that the high-end GPU's (X1950's and 8800's) will be able to outpace many CPU's and thus the mismatch results in idle time rather than blocked time which rarely causes a noticeable problem.

Comparing your situation against other applications isn't really a good idea - they'll interact with the API/GPU in different ways that may well mean that the CPU/GPU utilization is better balanced. Running PIX against these apps might well reveal some interesting comparisons.

Looking at how many Draw calls you make per frame, how many times you read/write resources (you did comment that you don't create/destroy resources, but modifying them can be equally bad), state changes and so on...

Given that you mention that the frame-rate with forced synchronization is 15-25fps then have you considered that you're just pushing the hardware too hard I can quite easily write graphics demos that exhibit similar characteristics where I make only a few draw calls with 100's of 1000's of polygons (or even millions) each. Throw in a complex shader and you can quite easily choke a good GPU without bothering the CPU too much...

hth
Jack

magwo001

Ok I just discovered something that changes a lot of things.

You can have yor network (LAN) adapter show an icon in your systray to show if you're connected or not. When I uncheck the option to display it, and it goes away, ALL the stuttering problems vanish and framerate is smooth.

So apparently it seems that the stuttering is caused by windows 2D rendering. Correct


Are there known fixes to this problem Is it caused by starving the windows scheduler or something

magwo001

Also I just discovered that lowering the amount of 2D hardware acceleration in Windows also removes the stuttering completely. What does this tell us

JeremyAtGosub

> So apparently it seems that the stuttering is caused by windows 2D rendering. Correct \

That's been my experience when working in windowed mode. Like I said above, the more stuff GDI is drawing, the more stuttering there is. And that's for all the computers I've tested on. It looks like Present() sometimes blocks for too long (like 100 milliseconds or so).

Is there a way to know if Present() (or BeginScene/EndScene) is going to block on the next call Maybe waiting for them to un-block before calling would help.

-Jeremy