R4d1um

I use a open source codec/video library (libavcodec). This lib exposes the video frames

as 24_bit RGB bitmaps in memory.

All I want to do now, is displaying those frames in a window as fast as possible.

I tried a very naive approach:

for ( unsigned int i = 0 ; i < Framenumber; i++ )

{
Load->get_nth_frame( ID, i , RGBFrame );
DXDev->BeginScene();
Backbuffer->LockRect( &LR, NULL, 0);

// [ memblit Copies RGBFrame pixelwise to Backbuffer , same function as

//D3DXLoadSurfaceFromMemory ( Backbuffer, NULL, &SrcRect, RGBFrame,

//D3DFMT_R8G8B8, Pitch, NULL, &SrcRect, D3DX_FILTER_NONE, 0);]

memblit ( RGBFrame, (char*)LR.pBits, w, h, LR.Pitch );
Backbuffer->UnlockRect( );
DXDev->EndScene();
DXDev->Present( NULL, NULL, NULL, NULL );
}

It turns out, that this is far to slow.

(I am not sure, if this is due to the used technique, or some other directx

misconfiguration.)

Anyway, which is the best approach you would recommend for displaying video

through DirectX

Thx



A.Gallus



Re: Game Technologies: Graphics Displaying Videoframes with DirectX

BLANC Guillaume

As I've answered to someone else in that post, I would recommend to don't lock the back bugffer but rather to fill a texture (using the some lock/unlock technic) and then render that texture to the back buffer.

This can seems quite similar to what you've done, but modifying the back buffer introduce a synchronization stage between the cpu and the gpu (during the lock). During that time only one of the component will be able to work : the cpu. So avoiding that has the advantage to keep cpu and gpu tasks parallel and thus get more performances. It can also allow to do some of the video format conversion in a pixel shader, wich can be a huge optimization in some cases.

In the other post I gave you the link, I've explained a bit how to do and also speak about non square and/or non power of two texture sizes issues..





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

R4d1um

Yes, i have read you post, and gave it a try.

Unfortunately my video device is not capable to handle dynamic textures:

Driver does not support dynamic textures.

So, is there another prossibility I have a very old Savage as Graphic Adapter. But all videoplayers work well on my pc.

So there must be a way to do this in directx or at least with another library.

Thx

A. Gallus





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

Bad Habit

One of the samples in DirectShow called Texture3D9 is similar to what you want. This is what readme says:

//-----------------------------------------------------------------------------
// Name: DirectShow Sample -- Texture3D9
//
// Copyright (c) Microsoft Corporation. All rights reserved.
//-----------------------------------------------------------------------------

Draws video on a Microsoft DirectX 9.0 texture surface. This sample builds
on the Direct3D Tutorial05 sample, adding support for rendering video
onto a Direct3D9 surface.


Note: This sample does not support changing the display properties
of the monitor while the sample is running.





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

R4d1um

Thx, i manged it in the meantime by optimizing my naive approach.

I will use your suggestions in a future version of my video player, and when i have a new GFX card.

Thx a lot!

A.Gallus





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

lib_team

One can see D3DUSAGE_DYNAMIC as just a hint for DirectX for memory management and uploads. The algorithm will also work well without this flag so you can still have a try with your savage. You'll keep advantage of cpu/gpu de-synchronization..




Re: Game Technologies: Graphics Displaying Videoframes with DirectX

R4d1um

OK, i will try it then.

If i understand it right i should do the following:

1.) Load my frame as an RGB bitmap into system memory.

2.) Subdivide the frame in smaller parts, which their extends beeing a power of two, e.g. let's say my frame has a height of 732 pixels, and the width would be 512 (power of 2) then i would require 6 Textures to subdivide the height: 512 + 128 + 64 +16 + 8 + 4 = 732!

3.) Lock each texture's surface, copy individual parts of the RGB bitmap adeaquatly to that surface, unlock surface.

4.) Get the handle to the window in which i would like the frame(s) to appear, render the textures to the backbuffer(s) of that window.

I have several questions here:

Q1.) Why does this approach work faster than one in which i simply copy the RGB bitmap directly to the window backbuffer via "D3DXLoadSurfaceFromMemory()" or an own copy mechanism Maybe you could explain me in more detail the synchronization problem between CPU/GPU.

Q2.) Is DirectX the best choice for this task, what's with SDL for example

Q3.) Is it better to use two textures (512 + 256) instead of 6 textures in the example above

Thx in advance

A.Gallus

Greetings

A.Gallus





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

R4d1um

I tried it without the dynamic flag for my textures. Of course the textures are not lockable then with 3DPOOL_DEFAULT.

So how should i bring in the bitmap data in the videoram I would have to setup a bunch of surfaces, copy the rgb data into that surfaces

and then copy from that surfaces to the textures surface by UpdateSurface() ! That can't be the solution.

Help appreciated!

By the way using other Flags than 3DPOOL_DEFAULT did't have performance advantages.

Greetings

A.Gallus





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

JeremyAtGosub

I think you should skip step number two above, and use the next larger size texture while changing the UV coordinates.  So copy your 732 pixel picture to a 1024 pixel texture and then use a U coordinate of 0.7148. 

There are examples of how to write to a texture in the SDK.

-Jeremy





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

BLANC Guillaume

I'll try to be clearer:

>> Q1.) Why does this approach work faster than one in which i simply copy the RGB bitmap directly to the window backbuffer via "D3DXLoadSurfaceFromMemory()" or an own copy mechanism Maybe you could explain me in more detail the synchronization problem between CPU/GPU.

The problem is not the copy mechanism : all copy functions are good (as soon as they do not do format conversion) because they're all copying from and to the main memory. Indeed, DirectX (especialy the driver I think) is internaly doing the main to video memory copy.

The problem when locking the back buffer, is that while that time the gpu cannot do anything. It must wait for the cpu to finish. Indeed it may not be a huge waste for video rendering compared with 3d rendering, unless you can use the gpu for something important like strectching (zoom, full screen, ...), converting video pixel format from yuv to rgb, doing some special effects,...

Remember you're using two processors : a CPU and a GPU. Imagine a case where both processors should do something to render a frame, the best performance is reached when both processors are not waiting for each other and are working in parallel. This is an important thing to do when working with 3d rendering as well as when working with multi-processors. To acheive this, you can use a buffer technic which could be for the video example like that:
- on the cpu : do IOs and video decompression
- on the gpu : do format conversion, stretching and effects
This is a very common thing that 3d API automaticaly implies if you do not stall the pipeline with a back buffer lock (either read or write). The two cpu and gpu tasks are running in parallel : while the cpu is working to decompress the next frame to display, the gpu is rendering the current one. On next tick, cpu sends new images to the gpu and immediatly starts processing a new frame. Processor load is that way balanced.

>> Q2.) Is DirectX the best choice for this task, what's with SDL for example

If you use DirectX, this is because you need some acceleration for things like strecthing, zoming, effects and so on, and because you want to do it by yourself with full control. So indeed if that's not the point for you, then use SDL or even Windows DC and you'll be fine. You won't need any directx back buffer and will directly blit your video on screen (different from a back buffer which is a dedicated word for buffered rendering).

>> Q3.) Is it better to use two textures (512 + 256) instead of 6 textures in the example above

Jeremy's idea is a very good one. I'm used to highly constraint systems so I didn't think about it by myself.


>> I tried it without the dynamic flag for my textures. Of course the textures are not lockable then with 3DPOOL_DEFAULT. So how should i bring in the bitmap data in the videoram

You should then use D3DPOOL_MANAGED which will work fine, even if it won't be the best. Also, you'll never be responsible for copying to video ram. You're only copying from main to main memory, Directx will itself copy surfaces to video memory when the surface will be needed. By the way, you could avoid a memory copy if the codec was able to decompress video pixels directly in the locked surface. It would be a quite big perfomance improvement I think.

>> and then copy from that surfaces to the textures surface by UpdateSurface()

There's a misunderstanding : surfaces ARE texture parts! Textures are made of surfaces. UpdateSurface is not needed at all. Again see the other post I've sent you for a little more explanation about surfaces vs textures.

-----------

This post is a bit long but I hope it could helps. Again, if you don't need to do anything accelerated by the gpu, then I think you could reconsider not using Directx, but rather Windows API DC or SDL equivalents..





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

R4d1um

First of all thx Guillaume and Jeremy, a lot is clearer now to me.

I set up a simple renderloop, drawing just a triangle which changes color over time.

No Textures, no video.

Even then, the rendering is very very slow, and a 700 * 400 window featuring that triangle flickers like hell.

Furthermore my card only is only able to do "D3DCREATE_SOFTWARE_VERTEXPROCESSING".

From all that, i deduce, that the way we discussed is only viable with a much more performant GFX card.

So, i am looking for another approach , maybe without directx.

All i want to do, is displaying a XRGB32 bitmap in memory in a window, it's screen also having this pixelformat.

Furthermore, let's assume, that the frame has the same extends as the window. So really no processing is needed

anymore. I just have to copy that bitmap as fast as possible 1:1 to the Frontbuffer(preferably)/Backbuffer.

I had a look at various open source videoplayers.....

Normally they don't use Vertexprocessing/Texturing for display, only if a very capable gfx card is in the system

or if windows vista is detected (which implies a capable gfx card) they use the approach suggested here.

Normally, they surpisingly use SDL or DirectDraw. So the discussed approach has many advantages

in the future, when the mass of gfx cards are more capable.

Up to now, it seems that most videoplayers stick with a simple 2D API.

I thought, that Directx9 (not using DirectDraw) would have the same capabilities as DirectX versions below 9.

But because Directx9 is not able to render to the frontbuffer directly, DirectX9 actually means a severe disadvantage for older cards.

Under this circumstances, maybe some of you is able to give me a few hints, how i could do it in DirectDraw/SDL, and if

i am right with my thoughts.

Greetings

A.Gallus





Re: Game Technologies: Graphics Displaying Videoframes with DirectX

BLANC Guillaume

Indeed directx might not be the right choice for simply rendering video without any effect, zoom, incrustation an so on. Windows API is probably enought for that.

As we're far from the initial post subject now (Displaying Videoframes with DirectX), maybe you could flag this one as closed/answered and open new oneson other forums that best suit your needs.