stuartaw

I'm evaluating the use of UIA for assistive technology. I've got most of what I need working. However the big problem at the moment is that various controls are not reported by UIA or UISpy whereas AccExplorer32 finds them OK.

For example - right-click in a Word document and open the 'Font' dialog. UISpy cannot see any of the check-boxes (and some other controls) whereas AccExplorer correctly reports the contents of the dialog. This applies to Word 2000 and 2003. In fact UIA can detect the check-boxes if you use AutomationElement.FromPoint, but none of the TreeWalker methods or Element.Find... methods return these.

The problem is not restricted to Office, this is merely a good example of it.

I know there are threading problems with UIA (eg with any browser software as provider), but the above is unaffected by which thread the UIA iteration is done in - I get the same results in my code as UISpy produces.

Can anyone help I was under the impression UIA would provide most of the information that MSAA could provide for pre-Vista apps. Am I doing something wrong

All this is running under XPSP2, using the 6.0.6000 build of .Net3 and Vista SDK.

Stuart


Re: Microsoft UI Automation Numerous controls missing from UIA which MSAA finds

Padmavar

This seems to be the same problem that I mentioned in an earlier thread regarding 'Broken links in Automation tree'.

The checkboxes themselves are identified as automation elements (Name: Emboss, ControlType: check box), and they have the Font window (Name: font, ClassName: bosa_sdm_Microsoft Office Word 12.0, ControlType: window) as parent. But the Font window does not seem to include the checkboxes as children in the automation tree.

I have seen these problems mainly in Office though - PowerPoint, Word, Office Project etc.





Re: Microsoft UI Automation Numerous controls missing from UIA which MSAA finds

Marco Achtziger

I have seen the same behaviour in Outlook. Some custom controls from Microsoft do not correctly implement the necessary interfaces i think. I think strip controls are also still a problem. For these controls i use fallbacks to MSAA whenever needed. But i think this is not the best solution to do it because of the sometimes strange behaviour of MSAA.





Re: Microsoft UI Automation Numerous controls missing from UIA which MSAA finds

stuartaw

Thanks for the answers.

Padmavar wrote:

This seems to be the same problem that I mentioned in an earlier thread regarding 'Broken links in Automation tree'.


Yes I think you're right. It is not just Office, there are some (different) controls missing from other applications also, but Office seems to be the worst.


I have seen the same behaviour in Outlook. Some custom controls from Microsoft do not correctly implement the necessary interfaces i think. I think strip controls are also still a problem. For these controls i use fallbacks to MSAA whenever needed. But i think this is not the best solution to do it because of the sometimes strange behaviour of MSAA


It rather weakens the case for using UIA in AT apps, if you also need to use MSAA. Disappointing, since I thought the MS legacy provider was reading from MSAA itself, and therefore would show all the controls MSAA showed.

One of the main purposes of the job I am currently doing is investigative - to determine how usable UIA is for AT, and it's advantages (or not) over MSAA or IA2. It seems a bit hard to recommend the current buggy version. Pity.

Stuart




Re: Microsoft UI Automation Numerous controls missing from UIA which MSAA finds

Karl Bridge - Microsoft

If a provider does not implement UI Automation, the "fallback" is to use IAccessible to retrieve as much information as possible. The issues you are encountering can probably be traced to a faulty or invalid IAccessible implementation.

Karl






Re: Microsoft UI Automation Numerous controls missing from UIA which MSAA finds

stuartaw

Karl Bridge - Microsoft wrote:

If a provider does not implement UI Automation, the "fallback" is to use IAccessible to retrieve as much information as possible. The issues you are encountering can probably be traced to a faulty or invalid IAccessible implementation.

Karl



Hi Karl,

Thanks for the response. The odd thing is that the information seems to be available thru IAccessible (eg looking in AccExplorer32), so the controls must be supporting IA to a large extent.

In fact the charity I've done this work for is considering funding work to write their own MSAA fallback to fill in the gaps in the information from UIA, as described by another poster above. This seems to be duplicating work that UIA is supposed to do; but I see no alternative if UIA is to be used for AT applications.

Stuart

PS - essentially all of the provider apps we've been testing with are Microsoft ones.




Re: Microsoft UI Automation Numerous controls missing from UIA which MSAA finds

Marco Achtziger

That the information is available in AccExplorer does not mean that the implementation of IAccessible is not faulty. I've seen that mostly the navigation of the elements is not correct, e.g. you cannot navigate to the parent or next sibbling of an element. The children of an IAccessible are mostly accessible. That's why one can navigate top-down in the tree. But i also think that normally UIA should do that work for the developer as the fallback to MSAA from managed code is relatively errorprone. I encountered many types of different exceptions when accessing an IAccessible e.g. which i do not think should be thrown (e.g. NotImplementedException) from controls.