Accessible and Reliable

Recently, Eric asked me to speak at one of our company meetings about the future of the OpenSpan platform. He didn't want me to speak about releases or features, but rather about the platform as a whole. What is our vision for the OpenSpan platform?

This is actually a topic I think a lot about. For every feature we design and every release we plan, I ask the question: "How will this push the platform forward?" One of the things I think we've done very well at OpenSpan is to stay true to the principles that led us to create OpenSpan in the first place. When Stephen and I first started planning the platform we each brought two sets of distinct experiences to the table.

Stephen had spent the past several years creating visual programming environments. He believed strongly that the barrier that prevented more business analysts from programming was not the concepts of programming but rather the syntax of programming. Ultimately, many of the concepts programmers use everyday, conditions and loops in particular, are familiar to every advanced Excel or Access user. Other concepts, such as types, casts, threads and objects are either unnecessary for many tasks or presented in such a way as to be impenetrable to the lay person. Stephen was committed to providing an environment where business users could develop or participate in the development of automations.

I had spent the past few years creating custom desktop automation solutions for enterprise customers with diverse application requirements. I had become painstakingly familiar with the available APIs: COM libraries for Lotus Notes, Internet Explorer, Office, Siebel and others; MSAA and windows functions for Win32 applications; HLLAPI, EHLLAPI, etc. for mainframe emulators. I had also become painfully familiar with the limitations of these APIs: narrow or incomplete APIs; little or no support for events; frustratingly inconsistent implementations of supposedly standard APIs. My experiences had convinced me that there was a better way to automate these applications.

In our very first face to face conversation, Stephen and I decided two things: OpenSpan would be accessible to both developers and non-developers and OpenSpan would be completely reliable. If you clicked a button with OpenSpan, it would click every time.

I would be lying if I claimed that we knew that these principles would lead us to our current automation engine and injection library. Some ideas like controls, targets and match rules evolved quickly. Other ideas like asynchronous links and keys evolved in response to customer requirements. Whatever happened, though, we stayed true to our principles. When we reached the limits of message hooks and accessibility interfaces, we scrapped them and wrote an injection and hooking library from the ground up. When we encountered a control we didn't support, we reverse engineered the platform to get at the real object. When a public interface didn't supply the events we needed, we made our own with some well placed hooks.

Over time, some new principles evolved:

1. A button is a button is a button. In other words, hide the platform implementation details. Every control in OpenSpan should act the same regardless of the actual platform.


2. Never hide functionality. If a platform provides functionality that another platform does not provide, expose it. A great example of this is HTML controls. They follow the same pattern as other OpenSpan controls, but add extra properties to manipulate the inner HTML, set attributes, etc.


3. Always talk to the object. Pretty quickly it became apparent that every time we tried to take a shortcut, it burned us. GDI hooking. Sucks. MSAA. Sucks. Message hooks. Suck. There's always something to tweak or modify or re-implement for each environment. The only API you can rely on is the one the original developer of the application used. So what if it's not exposed? Sure, it takes more time initially, but we only need to do it once and then it works every time. I'm very proud of the fact that OpenSpan implementations never need engineers on site to develop integrations. If there is a control we don't support, we write a translator and publish it for all of our customers. One and done.


4. Know everything about the object. It's not enough to talk to the object. We know when an object is created. We know when an object is destroyed. We intercept events before the object receives them. 'Nuff said.


5. There is no such thing as too advanced. Early on we had a debate about threading. Should we hide threading from our users? Should the platform make threading decisions for them? Ultimately, we decided not to hide threading because doing so would inevitably limit our users. We made threading accessible by making it easy to visualize. Do people screw themselves up with threading? Sure, but people burn down their house with turkey fryers every year too. That doesn't mean fried turkey doesn't taste good.


6. Developers do not always want to drag and drop. This one is pretty obvious. Even though we want the platform to be accessible to non-developers, we want it to be just as accessible to developers. For developers, sometimes it's just easier to write code. And when it is easier, there's nothing more frustrating than not being able to. From the very beginning, we made sure that developers could write C# and VB .NET scripts and use their own .NET components in automations.

So what is our vision for the OpenSpan platform?
OpenSpan was built from the ground up to be a accessible and reliable desktop automation platform. Those two simple words, accessible and reliable, have directly guided the evolution of our most valuable IP: automation and injection. Our vision is to extend those two technologies into new areas. We are actively working on new ways to expose them to developers including our upcoming Visual Studio plug-in, Lotus Expeditor container and translator SDK. We are also working to apply these technologies in new environments. In the future, we will move beyond the desktop to provide automation and injection on the server. When we do, you can be sure it will be just as accessible and reliable as our current platform.

Inside Out: Synchronous Click

In 4.1 we introduced a new set of methods: PerformSynchronousClick, PerformSynchronousDoubleClick and PerformSynchronousRightClick. These methods are only available in our windows adapter currently, but will be extended to other technology stacks in the future. These new methods complement our existing PerformClick, PerformDoubleClick and PerformRightClick. So why do we need synchronous versions of these methods? And why are the existing methods not synchronous? What does synchronous mean in this context anyway?

To understand these functions, let's take a step back and talk about the Windows message loop. All Windows programs (at least those with a user interface) feature some variation of the following:

MSG msg;
while( GetMessage( &msg, NULL, 0, 0 ) != 0)
{
   TranslateMessage(&msg); 
   DispatchMessage(&msg); 
}

This simple code snippet is the prototypical Windows message loop. Essentially, GetMessage asks Windows for a message and waits. When Windows has a message (such as a keyboard or mouse message), GetMessage returns. The caller then passes the message to TranslateMessage (which turns keyboard messages into char messages), and then DispatchMessage (which passes the message to the actual destination window).

However, not all messages are returned by GetMessage. Only posted messages (input messsages and messages posted using PostMessage) are returned by GetMessage. So what happens to sent messages (messages sent using SendMessage or one of its variants)? These messages are dispatched synchronously to the destination window by GetMessage before it returns. The sequence looks like this:

1. Enter GetMessage.


2. Dispatch all sent messages to the destination windows.


3. Get the next posted message and fill the msg structure.


4. Exit GetMessage.

In truth, it gets a little more complicated than this, particularly when you're dealing with another application's message loop. Applications can and do add all sorts of additional processing to the message loop to do special input filtering, show modal dialogs, etc. Moreover, simulating even the simplest click can involve over twenty messages, some sent, some posted, depending on the styles of the destination window.

Unfortunately, directly sending messages to the destination window will not yield the correct behavior because if the application does any special message processing in the message loop we will have bypassed it entirely. Moreover, we never know when one of the twenty messages we send will result in a long running application process that we don't want to wait for. For these reasons, prior to 4.1, we did not provide any synchronous click functions at all. We filled the message queue and let'er rip.

For most scenarios, this works fine. OpenSpan always waits for new controls to be created using our implicit WaitForCreate methods so there are very few instances where we need to wait for the actual result of a click. However, one of those situations involves our prototypical demo application: calculator. Using the standard click methods, the following automation will return 0 or 4 rather than the desired 8. With the information above can you understand why?



That's right, each step of the automation doesn't wait for the previous step to complete. Thus, the automation gets the text before calculator has updated the text field with the final result. The new synchronous click methods do wait for all of the click messages to be processed. Thus, the automation below will return 8 every time.



But what if clicking a button creates a modal dialog? Will OpenSpan wait for the modal dialog to be dismissed by the user before continuing? What if there is no user? What if I want to dismiss the dialog in my automation? Oh noesss!

Simmer down. The short answer is that we do not wait for the modal dialog to be dismissed. Modal dialogs are nothing more than a nested GetMessage loop inside the outer GetMessage loop. Usually the sequence looks something like this:

1. GetMessage returns a mouse up message.


2. DispatchMessage passes the mouse up message to a window.


3. The window procedure creates the modal dialog and shows it.


4. The window procedure calls GetMessage and enters a new message loop to process messages posted to the modal dialog.


5. The user dismisses the modal dialog.


6. The windows procedure exits the message loop and returns.


7. DispatchMessage returns and the outer message loop resumes.

OpenSpan simply waits until step 4, when GetMessage is called again after the mouse up message and returns. Perceptive readers with knowledge of hooking may have guessed how we do this, but for everyone else, it's sufficient to say that we ensure that each time an application calls GetMessage (or any of its variants such as PeekMessage) the next simulated message is always returned. Some other time, maybe I'll talk about how we ensure that OpenSpan messages receive priority over other messages and are always processed in order. Until then, I'll let everyone guess in the comments (everyone except OpenSpan employees, that would be cheating) and give the correct answer a free "Do It on the Desktop" t-shirt.

4.1 Feature Spotlight: Automation Stack

It's that time again! As our 4.1 release nears completion, it's time to start posting about our new features. One of the new features we're testing currently is a new automation stack implementation. As our platform has evolved, we have begun to use our automation engine in new and more complicated environments. Previously, most of our environments were single-threaded automations of applications on the user desktop. Now, our platform is being used on the server to automate back-office requests, process messages and expose applications as web services. In this new environment, it quickly became clearly that our automation engine needed a proper stack to ensure that our automations remained thread safe.

If you're not familiar with OpenSpan, one of your first questions might be how the heck don't you already have a stack? To explain that, I need to go under the hood for a little bit. Looking at the automation your first instinct would be to assume that OpenSpan is a visual programming language that generates source code. For example, the automation below could be expressed in a C# class as:

private string varString;
private void Setup()
{
   btnSubmit.Click += delegate
   {
      varString = txtPhone.Text;
      webBrowser.Navigate(txtUrl.Text);
   };
}

Pretty straightforward, right? How about this automation? In this case we've made one of the links asynchronous indicating that the next step should be performed on a new thread.

private string varString;
private void Setup()
{
   btnSubmit.Click += delegate
   {
      varString = txtPhone.Text;
      NavigateDelegate navDel = delegate(string url)
      {
         webBrowser.Navigate(url);
      };
      navDel.BeginInvoke(txtUrl.Text, null, null);
   };
}

Now it's starting to get more complicated. Every time we need to execute an asynchronous link we would need to generate a delegate type. I'm using anonymous methods which makes it easier, but it's still pretty difficult. How about this automation? We have two execution threads from different events converging on a single block. Easy to express in OpenSpan, but we would need to use another method or delegate in C#.

private string varString;
private void Setup()
{
   btnSubmit.Click += delegate { NextStep(); };
   webForm.Submitting += delegate(object sender, CancelEventArgs e)
   {
      NextStep();
   };
}

private void NextStep()
{
   varString = txtPhone.Text;
   NavigateDelegate navDel = delegate(string url)
   {
      webBrowser.Navigate(url);
   };
   navDel.BeginInvoke(txtUrl.Text, null, null);
}

Even more complicated, and I haven't even added entry points to the automation. So what can we conclude from this exercise? As you might have guessed, OpenSpan does not generate source code. Instead, we model execution flow using a serialized object graph. This gives us greater flexibility to express logical concepts such as branches, joins, etc. without the limitations of traditional syntax. However, because we do not generate code we are unable to rely on the existing thread stack to manage our local variables for us. Thus, prior to 4.1, we simply didn't have a stack at all.

So, how does the new stack work? For the most part, it works like the normal thread stack we all know and love. When an automation is entered, all of the local variables and components are allocated and pushed onto the automation stack. So far so good, right? But what about those pesky asynchronous links? In OpenSpan it's perfectly legal to have two links on different threads in the same automation updating variables.


So what should we do here? In a traditional thread stack model, each asynchronous link gets it's own stack. OpenSpan, however, makes creating new threads absurdly easy. Thus, it's easy to create a number of parallel threads to accomplish tasks such as updating three or four application simultaneously. Should we allocate new variables and components every time we execute a new asynchronous link? Should we initialize them to their original defaults or should we initialize them to their current values?

As you probably guessed, the correct answer is none of the above. The new automation stack is not a thread stack. Local components and variables are allocated on a per automation basis not on a per thread basis. Thus, each of those asynchronous links in the example above are actually updating the same variable. This approach preserves the ease of threading OpenSpan traditionally provides, while adding the additional safety of stack-based variables.

Now that I've explained how the new automation stack works, I'll cover some of the common questions I've been asked about this feature.

What if I want my variables to be available to all automations?
No problem. OpenSpan automations now have a global and a local tray. Components in the global tray behave just like they did in previous versions of OpenSpan. They can be updated from any automation at any time. Components in the local tray can only be used on that automation and are not exposed to other automations.

When are local and global components created?
Global components are created when the project is started and can be used in any automation. Local components are only created when an automation is run. Local components can only be used in the automation that contains them.

When are local and global components destroyed?
Global components are destroyed when the project is stopped. Local components are destroyed when all the threads in the automation have completed.

When should I use a local component?
Local components are appropriate when the component in question does not need to be accessed from other automations or when an automation can be run simultaneously on multiple threads. Local components are recreated every time an automation is run and thus will not maintain state across multiple runs. For instance, you could not create a local variable to keep track of the number of times an automation is run. Every time the automation runs the counter variable would be recreated and initialized to zero.

What happens when I upgrade my old automations?
Variables and components from old automations will automatically be placed in the global tray during the upgrade process. Thus, your old automations will continue to function as they did before. To take advantage of the automation stack, you can easily make any global components local by right-click and selected the "Make Local" menu item.