Transitioning to WPF

Little Miss Sunshine

Hi all,

We have a legacy application written using MFC/Win32 GDI whose primary function is to
display JPEG images from various sources (e.g. cameras) at a constant frame rate (which can
be up to 30 frames per second).

On a given target hardware our legacy app can display 20 "160x120" jpegs at 5 fps consuming
about 30-40% CPU. For JPEG decoding we use Intel's JPEG decoder which is based on
Intel's performance primitives.

We are planning to use WPF for our next version of the application. To compare the performance,
I wrote a simple app that displays 20 "160x120" jpegs at 5 fps. Now the CPU usage has doubled to about
70-80% on the same taraget hardware.

The legacy app used to first write to an in-memory bitmap and then would render the bitmap to
display - the classic double-bufferring solution. My test WPF app does not do any of that, it
just creates 20 "Image" objects and updates the "Src" attribute to the decoded JPEG image.
I use the JpegBitmapDecoder which is part of the WPF infrastructure libraries.

I would like to know if my rather naive solution in WPF is correct for the problem at hand If not,
I would appreciate any other alternative ways of doing this to get comparable or even better
performance. On thing to mention is, in the legacy app, the JPEG image is decoded on
the thread it is received in and rendered on a different thread. In my WPF app, I am not
sure where it gets decoded, but I believe it is in the UI thread (I use Dispatcher.BeginInvoke()).
To BeginInvoke, I pass the byte[] array containing the encoded JPEG image.

Below I have some snippets of the code to illustrate what I am doing:

Callback method called to handle an incoming JPEG:

public void onJPEGData(CMR_CLI.JPEG jpeg, Byte[] imgBuf, DateTime cap_time)
{
MemoryStream strm = new MemoryStream((Byte[])imgBuf);
img.Dispatcher.BeginInvoke
(
DispatcherPriority.Normal,
new UpdateImage(updateImage),
strm
);

Set JPEG image:

delegate void UpdateImage(Stream arg);
void updateImage(Stream strm)
{
BitmapDecoder dec = JpegBitmapDecoder.Create
(
strm,
BitmapCreateOptions.None,
BitmapCacheOption.None
);

foreach (BitmapFrame f in dec.Frames)
{
// img - System.Windows.Controls.Image
img.Source = f;
break;
}
}

Thanks & Kind regards

LMS

Jeremiah Morrill

I do not know what hardware you are running on, or how your legacy application works, but a 160x120 image at 30 FPS should not take up 30 - 40% CPU. On my 3.4ghz P4, I am able to do 30 FPS M-JPEG 704x480 @ 30 FPS at around 15 - 20% CPU. This was using the Intel JPEG library and Microsoft's M-JPEG DirectShow filter used just slightly higher CPU. Are you resusing the same buffers for decompression and rendering Reallocating a bitmap for each frame can be very heavy. In the task manager, do you see the page faults going nuts

In WPF, you are seeing a major peformance loss because of this:

1.) (Speculating)Allocate new memory to hold the JPEG, you should see some (not alot) performance increase by reusing the same byte array for each JPEG.

2.) Allocate new memory for a new bitmap and decompress the jpeg

3.) Set the new bitmap source in the image

4.) Let the garbage collector clean up the old image

5.) Repeat

#2, #3, #4 are your problem areas. The bad news is WPF is a totally retained UI framework, meaning you don't really have access to much low level stuff to do any custom hi-performance stuff. A lot of us are crossing our fingers that this will be addressed in the future.

In the mean time, there a couple work-arounds I threw together. If you have a DirectShow solution (some sort of MJPEG source filter that maybe reads from an Axis webcam), there is this: http://jmorrill.hjtcentral.com/Home/tabid/428/EntryID/15/Default.aspx

Or, what might be a more simple solution for you, is this hack that lets you get access to the pointer of a BitmapSource's pixels. With this, I suggest getting the pointer to the BitmapSource's pixels buffer. Have your JPEG decoder library decompress to the BitmapSource's pixel buffer. I have been able to get 7% CPU usage on a 30 FPS MPEG4 video using this technique: http://jmorrill.hjtcentral.com/Home/tabid/428/EntryID/16/Default.aspx

I'm interested in what you are trying to do, so hit me up if you have any questions: jeremiah.morrill [at] gmail.com

-Jer

Little Miss Sunshine

Hi Jer,

Thanks for your reply.

I think the actual hardware that I am running on is not so important. The point I was trying to make was that on the same machine the WPF app consumes twice as much CPU as legacy windows App.

I know that my approach is rather simplistic and I was hoping that there are better ways of doing it in WPF instead of the approach I am taking. I will take a closer look at your second solution which is to update the Bitmap source's pixels directly. Now, I don't want to use Intel's JPEG decoder. If possible, I want to use the one that comes with WPF. Is it possible to use your second solution with WPF's JPEG decoder

I can't be very specific about what I am trying to achieve, but in a nutshell, the application I am trying to develop in WPF will display video (mpeg-4 and motion JPEG) from multiple sources (e.g. cameras) up to 20-30 sources needs to be
supported. Even more if possible,

Regards.

LMS

Jeremiah Morrill

I think I know what you are trying to make. We might be in the same industry and making competing applications . No reason to not give a hand though.

Unfortunatly, I don't think you'd be able to use WPFs JPEG decoder with this technique because it will just reallocate a new BitmapSource each time.

Your best bet for performance is a DirectShow based solution (first hack), this way the video will be rendered via the MediaElement. I made a DirectShow source filter for fun that took a custom URL, like "mjpeg://addressort/path_to_mjpeg". I was able to pass that URL to a MediaElement and it loaded up perfectly and only had a minor performance hit when compared to a raw directshow implementation. Unfortunatly, this is not an easy answer. Learning DirectShow can be a pain.

I wish you luck. Hopefully you don't have to drop down to using Win32 Interop (Host your legacy controls in WPF). Nobody likes doing that

-Jer

Transitioning to WPF

Re: Windows Presentation Foundation (WPF) Transitioning to WPF

Re: Windows Presentation Foundation (WPF) Transitioning to WPF

Re: Windows Presentation Foundation (WPF) Transitioning to WPF