Michael Hunter (MSFT)

Some time back Alan Page stopped by my office asking for suggestions for effectively automating UI that is undergoing lots of change. Learn my answer (which won't be any surprise to longtime readers of my blogs) at http://msdn.microsoft.com/testing/automatinguichanging.asx.


Re: Software Testing Discussion Automating UI that is changing

automated_testing

Michael,

All of the commercial GUI automation tools have an object repository, or dictionary that allows for logical naming and script reference.

Even http://www.openqa.org/wet/ that is free has a ¡°object depot¡± which serves the same purpose.

Couple this with a Keyword Framework or other functions that are basically action\object and you pretty much have what you describe existing several years ago.

I have to admit though you are the first person I have heard using ¡®abstraction¡¯ in this type of framework.

The ones I have built have separated the objects the extent to what is instantiated, i.e. I did not abstract ¡®Button¡¯ for Radio or Push.

Do you think there is mileage in this abstract layer, and could you give examples of how much time is saved versus having an Object Based framework without inheritance

If you have code examples of what you have done here that may help me.

Good job explaining the concept to Alan though. Also I am sending the guy a YoYo for Christmas so expect him in front of your "whiteboard" in the new year for instructions (just kidding Alan, but don¡¯t give up your day job!).





Re: Software Testing Discussion Automating UI that is changing

Michael Hunter (MSFT)

I believe, and have seen, that abstracting away the details of UI widgets does save time and effort and maintenance. My experience is that while UI details change frequently, UI semantics change much less frequently. Since my tests are written in terms of the semantics of my UI, rather than its details, my tests do not need to change when those details change as long as the semantics stay the same.

Thanks for letting me know about the object repositories the commercial automation tools have. I haven't used any of them, so what they do and don't support is an unknown to me. As it happens, for various reasons I was not able to use a commercial tool five years ago when I started building this framework, which (also as it happens) does a lot more than logical naming of UI widgets. See http://www.thebraidytester.com/stack.html for more details, including code samples. (I didn't talk about the rest of the stack because I wanted to ease Alan into it. <g/> If he stops by again I'll talk about the other parts.)






Re: Software Testing Discussion Automating UI that is changing

automated_testing

Thanks for the clarification.

Changes to the "UI detail level" that do not change the "UI semantic level" are accommodated in most (if not all) commercial and Open Source tools in that space (GUI automation) by changing the POM or vendor¡¯s equivalent.

That said there are distinct advantages to a ¡®home grown¡¯ solution but ¡®make or buy¡¯ decisions will vary from company to company.





Re: Software Testing Discussion Automating UI that is changing

Alan Page

A Yo-Yo Oh man - I think I'm getting a bad reputation.





Re: Software Testing Discussion Automating UI that is changing

Frederic Torres

Micahel,

We are working on a new feature for our web functional testing tool InCisif.net, where the record mode can generate part of the abstract class you talked about in your video.

The class is generated starting from the HTML <FORM> tag and take into account standard controls.

You can find a sample of the class at www.incisif.net/tutorial/CustomerDialog.cs.txt. I wrote myself the methods
OpenCustomerMainMenu(), EditNewCustomer(), OK(), but the rest of the code was generated.

This idea was to create a layer to abstract the Add/Edit customer test process.

You can view how we use the class at www.incisif.net/tutorial/TestCustomerDialog.cs.txt.

The abstract class inherits from our class InCisif.net.Library.ClassDialog, to get some methods for free,
like ClassDialog.SaveContext(), which save all the control's value of a dialog.

And the method ClassDialog.Assert(ContextSaved) that verify that the control's value from the saved context match this one from the dialog.

My idea is that is once we have a class that abstract a user interface, we can build some generic behaviors on top of it.

The goal being to reduce the test code size.

Feedback wanted.

Thanks.

Frederic Torres
www.InCisif.net
Web Testing with C# or VB.NET







Re: Software Testing Discussion Automating UI that is changing

halperinko

We have been using VBA - to create our own Testing/Application "language" to be used while planning our tests (similar to having unique commands for excel vs. word in their VB sripting language).

So the testers developing automation do not have to deal with common tasks, like opening a specific window, or setting-up a common service.

But they still gain from using a comfortable development sorrounding of VB, allowing quick writing and identification of basic mistakes.

At the same time, we also gain from being able to start automating when there is actually no application there yet - as the test script does not rely on the actual application, and we can start as soon as required abstract commands are there.

We can also set several layers of abstraction, so it becomes easier to investigate a suspected bug, by drilling in deeper & deeper into the logs.

In the background, our infrastructure team "connects" the abstract commands to the real world as we start to get it.

A key advantage of this method, is the we do not rely on specific GUI - we can run the test once from the Element Manager application, then from the Craft terminal which is a web based application with similar abilities, or even through Command Line Interface, or we can set a value from one & read the result using the other ---- The test case code remains the same !

We can easily switch between products, test equipment models etc.





Re: Software Testing Discussion Automating UI that is changing

automated_testing

Halperinko,

This sounds interesting but raises some questions regarding the level of detail in your UI design documents.

If you can ¡°start automating when there is actually no application there yet¡±, my guess is you have an abstract (semantic) view of the GUI (that may not change that much) before the GUI is written.

For my better understanding:-

Are you following a 'water fall' SDLC rather than prototyping for the GUI

Could you comment on the design documents that you are using to build the test commands, that is an example of what you call the ¡°required abstract commands¡±

I think this would help me better understand what you are doing.