paulergiker82

Hello everybody,


I was just wondering whether there is a possibility to 'transform' mouse data into Ink data if entered. The trigger of my question is the rather low sampling rate for a mouse input device.
I would like to have more 'smooth' lines (just like the smoothness of Ink data).
I thought about calculating additional points between those captured in a Stroke, but I don't know how to update/display my new calculated Stroke (display the new one instead the old one).
Furthermore, is it maybe possible to do this manipulation when a Stroke is entered (and not after) I'm just concerned about the performance.
Or is there maybe totally different approach to this problem

Thanks in advance!

paul


Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

Stefan Wick - MSFT

Hi Paul,

which flavor of our Ink APIs are you using (WPF, Winforms or COM) Which component are you using to collect the ink (InkCanvas, InkOverlay, InkCollector, InkPicture, RealTimeStylus)

There are several ways to do this, but the specifics depend on which ink components you are using in your app.

The simplest approach is to set 'DrawingAttributes.FitToCurve = true' on your stroke. Have tou tried that

http://msdn2.microsoft.com/en-us/library/microsoft.ink.drawingattributes.fittocurve.aspx

Thanks,

Stefan Wick

Microsoft - Windows Experience






Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

paulergiker82

Thanks Stefan,

I'm using WPF and to collect the Ink, InkOverlay as well as InkCollector are used.

I already tried 'DrawingAttributes.FitToCurve' but I'm not satisfied.

That is why I try to find a 'better' solution.

I'm also working with a SmartBoard Overlay; big TV with an 'Overlay' which is connected via USB to the PC.

Do I have the same problem of a low sampling rate because it's connected via USB

About which ways are you actually talking. I'm very keen to hear them.

Thanks in advance

paul





Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

Stefan Wick - MSFT

Hi Paul,

I am still a little unclear on which of our APIs you are using for what.

>>I'm using WPF and to collect the Ink, InkOverlay as well as InkCollector are used

InkOverlay/Collector are not designed for usage in WPF so I am a bit surprised about this statement. WPF has its own, complete new ink object model and an InkCanvas element that is designed for collecting ink on WPF surfaces.

Can you explain how/why you are using InkOverlay/Collector in WPF - and why you are not using an InkCanvas

Are you storing the ink in System.Windows.Ink.Stroke objects (WPF strokes) or in Microsoft.Ink.Stroke objects ('Winforms' Strokes, as provided by InkOverlay/Collector)

Thanks,

Stefan Wick

Microsoft - Windows Experience






Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

paulergiker82

Hi,

I'm only using Microsoft.Ink.Stroke objects and never System.Windows.Ink.Stroke so I guess I use Winforms. I'm sorry for the wrong info abut the used API. Actually, what I'm using (WPF or Winforms) does not depend on me, I have to work with an already existing application.

Thanks

paul





Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

Stefan Wick - MSFT

Hi Paul,

Got it. Before looking at more advanced solutions, I want to confirm that setting FitToCurve=true is indeed not sufficient for your scenario.

I just wrote a little test program and used with my mouse and with the touchpad of my Laptop. The resulting strokes look pretty smooth to me. One key thing to note here is that setting DefaultDrawingAttributes.FitToCurve=true is not enough. You actually have to force a repaint for the stroke after it has been collected to renderer the smoothed stroke.

I have included my test program below. Please try it out and let me know whether or not this works for you.

Thanks,

Stefan Wick

Microsoft - Windows Experience

Code Snippet

using System;
using System.Drawing;
using System.Windows.Forms;
using Microsoft.Ink;

class SmootherInk : Form
{
public SmootherInk()
{
_inkOverlay = new InkOverlay(this);
_inkOverlay.DefaultDrawingAttributes.FitToCurve = true;
_inkOverlay.Stroke += new InkCollectorStrokeEventHandler(_inkOverlay_Stroke);
_inkOverlay.Enabled = true;
}

private void _inkOverlay_Stroke(object sender, InkCollectorStrokeEventArgs e)
{
Rectangle rect = e.Stroke.GetBoundingBox();
using (Graphics g = this.CreateGraphics())
{
InkSpaceToPixel(g, ref rect);
}
this.Invalidate(rect);
}

private void InkSpaceToPixel(Graphics g, ref Rectangle r)
{
Renderer renderer = new Renderer();
Point leftTop = new Point(r.X, r.Y);
Point widthHeight = new Point(r.Width, r.Height);
_inkOverlay.Renderer.InkSpaceToPixel(g, ref leftTop);
_inkOverlay.Renderer.InkSpaceToPixel(g, ref widthHeight);
r.Location = leftTop;
r.Size = new Size(widthHeight);
}

static void Main(string[] args)
{
Application.Run(new SmootherInk());
}

private InkOverlay _inkOverlay;
}






Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

paulergiker82

Hello Stefan,

thanks for little prog. I've already written a prog to see how successful FitToCurve actually works. I must admit it is quit good.

But I'm still keen to know some other approaches to solve this problem. I have to present my boss the results and I don't want to say that 'activate this attribute' is the only thing we could do. As far as this is the easiest solution with really good results it might be the one which is going to be used but I want to try some other solutions as well (even if they are not as good as this one). That is why I'm keen to hear about some other possible solutions/approaches.

Thanks

paul





Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

Stefan Wick - MSFT

Hi Paul,

when using InkOverlay (or InkCollector or InkPicture), you can swap out each newly collected stroke with your own, new and smoother version of the stroke in the 'Stroke' event handler.

Here is an example that shows how to do this. For simplicity sake, my algorithm actually un-smoothes the ink, but I am sure you get the concept. Basically you take control over the points on the stroke, and you can add, change or remove points according to the smoothing algorithm of your choice.

FYI: not sure if it is a requirement or useful in any way for you, but if you used a RealTimeStylus [1] object (instead of an InkOverlay) for collecting the strokes, you could actually manipulate the points on the stroke while the user collects them(as opposed to after the stroke has been collected).

Thanks,

Stefan Wick

Microsoft - Windows Experience

[1] http://msdn2.microsoft.com/en-us/library/ms824830.aspx

Code Snippet

using System;
using System.Drawing;
using System.Windows.Forms;
using Microsoft.Ink;

class SmootherInk : Form
{
public SmootherInk()
{
_inkOverlay = new InkOverlay(this);
_inkOverlay.Stroke += new InkCollectorStrokeEventHandler(_inkOverlay_Stroke);
_inkOverlay.Enabled = true;
}

private void _inkOverlay_Stroke(object sender, InkCollectorStrokeEventArgs e)
{
e.Cancel = true;
Point[] oldPoints = e.Stroke.GetPoints();

// algorithm to smooth ink (for simplicity sake, this example unsmoothes the ink)
Point[] newPoints = new Point[2];
newPoints[0] = oldPoints[0];
newPoints[1] = oldPoints[oldPoints.Length - 1];

_inkOverlay.Ink.CreateStroke(newPoints);
}

static void Main(string[] args)
{
Application.Run(new SmootherInk());
}

private InkOverlay _inkOverlay;
}






Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

paulergiker82

Thanks a lot for your help Stefan.

I'm implementing right now some interpolation algorithms (spline functions).

Just one last question. Is there a way to figure out whether 'mouse data' or 'pen data' was entered

I've already written a prog which gives me some information about the stroke (length of point array ...) but even with this infos I can only guess in which way/with which utility the data was entered. Is there maybe a more precise way

If so, I could check the way the data was entered (So no pen data interpolation would be necessary).

Thanks a lot

paul





Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

Stefan Wick - MSFT

Hi Paul,

one way to do this is by checking the Cursor.Name property in the Stroke event. It equals "Mouse" if a mouse device has been used.

Thanks,

Stefan Wick

Microsoft - Windows Experience






Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

paulergiker82

Thanks again,

I used the Cursor.Name property in the Stroke event to figure out which device was used. Surprisingly, it showed mouse when I was using the mouse or the pen. I tried it on an overlay (smartboard is mounted on your screen and connected via USB). Maybe it is because the overlay is connected via USB When I tried it on a tablet PC it was ok.

Do you have an idea wy it says 'mouse' Furthermore, if mouse is indicated does it mean that the sampling rate of the overlay-pen combination is similar to the overhead-mouse combination (that both sampling rates are low)

Thanks

paul





Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

Stefan Wick - MSFT

Hi Paul,

The value returned by Cursor.Name depends on what the device driver reports to the system. Apparently, your USB connected pen is using a mouse driver and not a proper stylus HID driver. As a result, you will not receive the same high sampling rate that you would get from a proper stylus device.

You could get a measurement for the sampling rate by dividing the number of points you have received by the time span between CursorDown event and Stroke event.

Thanks,

Stefan Wick

Microsoft - Windows Experience






Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

paulergiker82

Hello Stefan,

the stroke points are not really captured in a linear manner. I mean that not the same amount of points are always send in one time unit. It depends on the speed of pen/mouse movement, or am I wrong

When I analyze the sampling rate(SR) it is always unstable. With the mouse the the SR is between 0.04 and 0.1 and with the tablet PC around 0.13. Do you have an idea why it's just a little bit higher with the tablet (0.03). Should it not be more

paul





Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

Stefan Wick - MSFT

Hi Paul,

the sampling rate is a constant for a stylus device regardless high fast you move it. The rate you are seeing (133 points per second) is correct for most current models.

For mouse devices, the rate depends on how fast you move the device.

Besides the (temporal) sampling rate of the received points you might also want to compare the (spatial) granularity between different devices for your scenario.

Thanks,

Stefan Wick

Microsoft - Windows Experience






Re: Notebook, Tablet PC, and UMPC Development Increasing smoothness of ink Stroke

paulergiker82

Hi Stefan,

is it possible to change the behavior of the mouse concerning the unstable sampling rate. Furthermore, is it possible to force the mouse (or any other input device like the overlay) to 'work' with the highest possible sampling rate or at least with a constant one

I'm just wondering which component is responsible for the unstable rate. Is it the operating system, the mouse itself, the driver ...

Thanks

paul