Alex-MyRpg

Hi,

I'm currently writing a library (for XNA/DirectX) for animation/soundless video playback, and I have a few design questions for the pros out there if you don't mind answering them :

  1. I'm having my animation class inherit from game component, and update the frame depending on game time in the overriden Update method, and render it in the overidden Draw method, but even when I preload my textures to render, playback is choppy. Not sure exactly why, but I notice this only when I have significant number of frames (I noticed it with 100 frames, but *not* with 10/15). This problem may be linked and solved by the 2nd point, so please read on.
  2. When I preload all the textures for the frames in the animation (and the no. of frames is small, as explained in point 1), the animation plays back perfectly. Now I've also added the capability for future frame textures to be loaded on-the-fly asynchronously (using delegates) whilst the animation is playing, in order to a) I can't preload all the textures for a big video since they can have 1000s of frames... ba) To minimize loading time. From this I've discovered that the problem is caused by the slow loading time of frame textures in comparison to the playback rate (typically about 300ms compared to 30ms). Clearly the playback will overtake the on-the-fly preloading rather quickly, causing playback to wait up, making it appear choppy. So all I want to do it somehow quicken the preloading time for textures. This is my current code:

' Create new frame texture and store it in the array.
Dim creationParams As New TextureCreationParameters( _
0, 0, 0, 0, SurfaceFormat.Unknown, ResourceUsage.None, ResourcePool.Managed, _
Nothing, FilterOptions.None, FilterOptions.None)

_frameTextures(frame) = Texture2D.FromFile(graphicsDevice, _dataStream, byteLength, creationParams)

As you can see I'm loading from a stream (FileStream in my tests). I've fiddled around with the parameters quite a bit but this code makes it the quickest - still not quick enough .

I hope that some of you may have ideas on how video rendering is done by other programs, or more likely just some new ideas. Thanks in advance for any help!

Alex

P.S. If there are any premade animation playback libraries for XNA already please do let me know, but I've searched and I doubt it anyway since XNA is still very young.




Re: XNA Framework Video/Animation Playback

David Weller - MSFT

Moved to XNA Framework forum




Re: XNA Framework Video/Animation Playback

Shawn Hargreaves

Creating textures for each frame isn't probably the most efficient way, for two reasons:
  • Creating and destroying GPU resources is pretty slow. This is the kind of thing you really want to be doing just once in an initialize function, rather than inside your rendering loop.
  • Loading textures using FromFile means you aren't going to be getting much in the way of data compression. This may be fine if your animation is very low resolution, but for decent quality video, uncompressed format is going to take so much space, the hard drive may well not be able to keep up with the transfer rate this is requiring!
To fix the first problem, a typical approach would be to create probably three textures using the Dynamic usage flag in your initialize function. Then you cycle between them. Each time you want to render a frame, pick the next texture from your list of three, call SetData on it to fill it with a new frame of image data, then use it to render, and rotate the list so you get a different texture on the next frame.

The reason to have three textures rather than just one is that the GPU renders asynchronously with the CPU, so if you called SetData immediately the next frame on the same texture you just used, the CPU would have to stall until the GPU was finished rendering out of that texture. Using three in rotation gives the GPU time to have finished rendering the previous frames before the CPU tries to stuff new data into each texture.

To reduce the amount of bandwidth you are using (assuming this turns out to be a problem), you will need to do some kind of video compression. That's not an area I know much about though!





Re: XNA Framework Video/Animation Playback

Alex-MyRpg

Thanks very much for your reply. I think you've pretty much covered exactly how to do it. I'll have to check out exactly how to use SetData (do I use it on the Texture directly or on it's surface ) An example would be great, but you've been very helpful so far. At the moment I'm loading from JPEG images in a screen, which is good for compression (same size as an MPEG), but not good enough for loading speed.
I'll update here on my progress - and oh, would anyone (including you) perhaps be interested in this Video (i.e. Animation) component for XNA when it's complete Who knows, I may even add sound later
if people are interested.




Re: XNA Framework Video/Animation Playback

Alex-MyRpg

Is anyone interested then I've very nearly completed the class thanks to Shawn's help, so I may post it up if there is any interest...



Re: XNA Framework Video/Animation Playback

thedo

I'd certainly be interested.

What format do you use for loading the animations Is it proprietary or a regular format like mpeg, or wmv

Cheers


Neil






Re: XNA Framework Video/Animation Playback

bill spinhoven

I am interested to sounds real nice !

Bill





Re: XNA Framework Video/Animation Playback

Alex-MyRpg

Great, nice to hear people interested. Currently I use my own custom format and I have a seperate converter from AVI > my format. In the future I may adapt it to handle AVIs directly, or possibly even MPEGs, but the next step is to get sound and that's going to be tricky, since there is no equivalent of DirectSound (or none that has the same level of control) in XNA :S
Still, my converter handles compressed AVIs, which are really the same size as MPEGs.
Anyway I've tidied up the source a bit and I just have a few changes to make to it, and the converter before I release both. I shall be posting a download link here any time in the next day or so hopefully. :)

Alex

P.S. Not sure how much medium-term interested this will generate but I'm happy for people to suggest or make changes to the component so I can incorperate them into subsequent releases.




Re: XNA Framework Video/Animation Playback

CaptainTenneal

I too would be EXTREEEEEEEEEEMELY interested in this functionality as i'm currently trying to get this working myself.



Re: XNA Framework Video/Animation Playback

a.d.m

I'm definitely interested in video playback as well. I'm especially interested in accessing video from a webcam (on PC... not worried about Xbox 360 development right now).




Re: XNA Framework Video/Animation Playback

Alex-MyRpg

Good to hear some interest. Now it's not ready *quite* yet, but would you maybe like me to tidy up it all up so you have a drag & drop AVI > my format converter Then I can just post the code for the class to use in your game. :)




Re: XNA Framework Video/Animation Playback

errolian

Alex-MyRpg wrote:
Good to hear some interest. Now it's not ready *quite* yet, but would you maybe like me to tidy up it all up so you have a drag & drop AVI > my format converter Then I can just post the code for the class to use in your game. :)


Yes please!






Re: XNA Framework Video/Animation Playback

Remmie

Looks very interesting.Why dont you make an online page for this project so others can help in working on this codec. Anyways keep us informed!
Keep up the good work!




Re: XNA Framework Video/Animation Playback

Alex-MyRpg

Sure, perhaps I'll start up a SourceForge.net project for helper XNA classes, mainly this one at first. :)-




Re: XNA Framework Video/Animation Playback

NoNameHere

I was just thinking of using some video in a game and would greatly appreciate using your class.