How OpenSpan Could Support the Google Chrome OS (and Linux in General)

Last week I mentioned that I would follow-up my original post on the Google Chrome OS with a post on how OpenSpan could support the Google Chrome OS (and Linux in general). I emphasize could, because we currently have no plans to support the Google Chrome OS or Linux. Thus, this post is really just informed speculation on how our technology could be applied to Linux.

Diagram 1
Diagram 2
Diagram 3Typically, when we talk about OpenSpan, we focus on our Windows translator, Scout, as the base layer which everything builds upon. In fact, we often say "OpenSpan sits in between the application and Windows". We then talk about how we use the Scout layer to detect when additional technologies are loaded into an application so we can load translators to expose those technologies. For example, to explain how we support Java, we might whiteboard something like Diagram 1.

However, Diagram 1 isn't really correct. There's actually another layer below Windows, the instrumentation layer, responsible for hooking x86-64 byte code. Of course, once you're dealing with byte code, you're dealing with the processor, not the OS. Thus, our hooking library can hook functions on both Windows and Linux systems. When we add the instrumentation layer, our Java stack looks like Diagram 2.

So what would it look like if we added Linux to the mix? You might think that adding Linux would mean two different stacks, one on top of Windows, another on top Linux. However, this isn't the case. For technologies that run on both Windows and Linux, the stacks would actually reconverge at that layer. Diagram 3 shows what our Java stack looks like with Linux.

Of course, there's only a few UI frameworks that work on both Windows and Linux (see list here), most importantly Java and HTML. Unfortunately, HTML is a markup language rather than a binary specification so we have to adapt to each browser rather than HTML itself. However, we could still create a common set of HTML element objects that interact with each browser implementation through an abstraction layer. It gets even more interesting if you start thinking about server technologies.

To summarize, above the OS layer, most multi-platform technologies will reconverge on a common set of classes. Thus, although we will need to write a Linux layer, we will not need to write a new Java layer or two separate FireFox layers. Don't get me wrong, supporting Linux would still require a significant degree of effort. However, it is a manageable amount of effort because we can build on the layers we already support.

Google Chrome OS

Today, Google announced on their blog that they are working on a OS based on their Chrome browser and Linux. The blogosphere has been expecting this announcement for a couple of years now. It's been pretty obvious that Google was positioning itself to compete against all of Microsoft's core products, adding first Google Apps to compete with Office and then Google Chrome to compete against Internet Explorer. A Google OS is the final piece, taking on Windows.

I find it amusing the excitement that a Google OS generates. It seems to speak more to a dislike of Microsoft than to an excitement about Google. After all, if Google is successful, it will be in a more dominant position than Microsoft ever was, controlling the operating system, productivity applications, email and search (which effectively controls the Internet). That doesn't sound like a particularly desirable outcome. We all know monopolies tend to stagnate. After all, it took FireFox to drive Microsoft to improve Internet Explorer, Java to drive Microsoft to scrap COM and now Google Apps to drive Microsoft to update Office. I would hate to see Google replace Microsoft as another monopoly.

On the other hand, operating systems are a bit of a special case. Having a single dominant operating system leads to innovation at the next layer. OpenSpan only has to develop for Windows currently, which lowers our costs and allows us to focus more effectively. Standards are supposed to alleviate this problem, but standards are equally problematic. I was a web developer when Netscape and Internet Explorer both rolled out their own implementations of HTML 4.0. Having to develop cross-browser scripting code was an absolute pain in the ass. I've had similar experiences dealing with different Java server environments, each with their own little quirks. "Write once, run anywhere," usually means "Write once, debug everywhere." Moreover, standards tend to be produced slowly and are very rarely perfect. Browser and JVM creators are notorious for jumping the gun and rolling out versions that are compliant with an interim standard, but not the final standard. They are also notorious for rolling out innovations that are not compatible with other browser or JVM implementations. Developers are constantly presented with the choice of targeting the lowest common denominator and sacrificing functionality, targeting a specific implementation and sacrificing market share, or targeting multiple implementations and sacrificing time and money.

Of course, Google will argue that they will not be dominant or monopolistic, because the Chrome OS is designed to bring web applications to the desktop. Here's Google's description of the Chrome OS architecture:

"The software architecture is simple — Google Chrome running within a new windowing system on top of a Linux kernel. For application developers, the web is the platform. All web-based applications will automatically work and new applications can be written using your favorite web technologies."

Google's statement reminds me of the Palm webOS (on the Pre), which allows you to build softphone apps that use web technologies. These local apps are built with HTML and javascript, but they still rely on webOS services to interact with the local system. Is this how the Google Chrome OS will work? Or will the Chrome OS only allow users to interact with applications in the cloud, ie. no local storage, etc.? Most likely, the Chrome OS will do both, and some applications will be compatible with Windows, Mac and Linux and some will not.

If Google wants to demonstrate a commitment to open standards, it could roll out a Chrome OS virtualization layer to run applications on Windows. This would actually be a pretty smart strategy, extending the reach of the OS while deflecting potential criticism. Likewise, to gain widespread adoption, providing a Windows virtualization layer to run Windows applications on the Chrome OS will be required. Ultimately, virtualization may make much of this mute, particularly if we move from machine virtualization to OS virtualization (thus eliminating the need for OS licenses). It will also be interesting to see if Google open sources the entire Chrome OS. The Chrome browser is mostly open source (as Chromium), although Google does add some small functionality (mainly branding) on top of the open source implementation.

The most fascinating aspect of this will be Microsoft's eventual response if Google is successful. Right now, Google's search revenue funds free applications like Google Apps, Chrome and the Chrome OS. Google is going to need to make money off of these eventually, as I'm guessing most people don't want advertising on their desktop. In the meantime, however, being free is a really big competitive advantage for Google. Will Microsoft eventually be forced to make Windows free as well? Will Microsoft have to move more aggressively to a cloud model to compete? Whatever happens, Google has succeeded in making the desktop market interesting for the first time in a long time, and it hasn't even released anything yet.

Today or tomorrow I'll follow up this post with some thoughts on how the changing OS market impacts OpenSpan and provide some insights on how our architecture is already positioned for the post-Windows world.

Accessible and Reliable

Recently, Eric asked me to speak at one of our company meetings about the future of the OpenSpan platform. He didn't want me to speak about releases or features, but rather about the platform as a whole. What is our vision for the OpenSpan platform?

This is actually a topic I think a lot about. For every feature we design and every release we plan, I ask the question: "How will this push the platform forward?" One of the things I think we've done very well at OpenSpan is to stay true to the principles that led us to create OpenSpan in the first place. When Stephen and I first started planning the platform we each brought two sets of distinct experiences to the table.

Stephen had spent the past several years creating visual programming environments. He believed strongly that the barrier that prevented more business analysts from programming was not the concepts of programming but rather the syntax of programming. Ultimately, many of the concepts programmers use everyday, conditions and loops in particular, are familiar to every advanced Excel or Access user. Other concepts, such as types, casts, threads and objects are either unnecessary for many tasks or presented in such a way as to be impenetrable to the lay person. Stephen was committed to providing an environment where business users could develop or participate in the development of automations.

I had spent the past few years creating custom desktop automation solutions for enterprise customers with diverse application requirements. I had become painstakingly familiar with the available APIs: COM libraries for Lotus Notes, Internet Explorer, Office, Siebel and others; MSAA and windows functions for Win32 applications; HLLAPI, EHLLAPI, etc. for mainframe emulators. I had also become painfully familiar with the limitations of these APIs: narrow or incomplete APIs; little or no support for events; frustratingly inconsistent implementations of supposedly standard APIs. My experiences had convinced me that there was a better way to automate these applications.

In our very first face to face conversation, Stephen and I decided two things: OpenSpan would be accessible to both developers and non-developers and OpenSpan would be completely reliable. If you clicked a button with OpenSpan, it would click every time.

I would be lying if I claimed that we knew that these principles would lead us to our current automation engine and injection library. Some ideas like controls, targets and match rules evolved quickly. Other ideas like asynchronous links and keys evolved in response to customer requirements. Whatever happened, though, we stayed true to our principles. When we reached the limits of message hooks and accessibility interfaces, we scrapped them and wrote an injection and hooking library from the ground up. When we encountered a control we didn't support, we reverse engineered the platform to get at the real object. When a public interface didn't supply the events we needed, we made our own with some well placed hooks.

Over time, some new principles evolved:

1. A button is a button is a button. In other words, hide the platform implementation details. Every control in OpenSpan should act the same regardless of the actual platform.


2. Never hide functionality. If a platform provides functionality that another platform does not provide, expose it. A great example of this is HTML controls. They follow the same pattern as other OpenSpan controls, but add extra properties to manipulate the inner HTML, set attributes, etc.


3. Always talk to the object. Pretty quickly it became apparent that every time we tried to take a shortcut, it burned us. GDI hooking. Sucks. MSAA. Sucks. Message hooks. Suck. There's always something to tweak or modify or re-implement for each environment. The only API you can rely on is the one the original developer of the application used. So what if it's not exposed? Sure, it takes more time initially, but we only need to do it once and then it works every time. I'm very proud of the fact that OpenSpan implementations never need engineers on site to develop integrations. If there is a control we don't support, we write a translator and publish it for all of our customers. One and done.


4. Know everything about the object. It's not enough to talk to the object. We know when an object is created. We know when an object is destroyed. We intercept events before the object receives them. 'Nuff said.


5. There is no such thing as too advanced. Early on we had a debate about threading. Should we hide threading from our users? Should the platform make threading decisions for them? Ultimately, we decided not to hide threading because doing so would inevitably limit our users. We made threading accessible by making it easy to visualize. Do people screw themselves up with threading? Sure, but people burn down their house with turkey fryers every year too. That doesn't mean fried turkey doesn't taste good.


6. Developers do not always want to drag and drop. This one is pretty obvious. Even though we want the platform to be accessible to non-developers, we want it to be just as accessible to developers. For developers, sometimes it's just easier to write code. And when it is easier, there's nothing more frustrating than not being able to. From the very beginning, we made sure that developers could write C# and VB .NET scripts and use their own .NET components in automations.

So what is our vision for the OpenSpan platform?
OpenSpan was built from the ground up to be a accessible and reliable desktop automation platform. Those two simple words, accessible and reliable, have directly guided the evolution of our most valuable IP: automation and injection. Our vision is to extend those two technologies into new areas. We are actively working on new ways to expose them to developers including our upcoming Visual Studio plug-in, Lotus Expeditor container and translator SDK. We are also working to apply these technologies in new environments. In the future, we will move beyond the desktop to provide automation and injection on the server. When we do, you can be sure it will be just as accessible and reliable as our current platform.

Inside Out: Synchronous Click

In 4.1 we introduced a new set of methods: PerformSynchronousClick, PerformSynchronousDoubleClick and PerformSynchronousRightClick. These methods are only available in our windows adapter currently, but will be extended to other technology stacks in the future. These new methods complement our existing PerformClick, PerformDoubleClick and PerformRightClick. So why do we need synchronous versions of these methods? And why are the existing methods not synchronous? What does synchronous mean in this context anyway?

To understand these functions, let's take a step back and talk about the Windows message loop. All Windows programs (at least those with a user interface) feature some variation of the following:

MSG msg;
while( GetMessage( &msg, NULL, 0, 0 ) != 0)
{
   TranslateMessage(&msg); 
   DispatchMessage(&msg); 
}

This simple code snippet is the prototypical Windows message loop. Essentially, GetMessage asks Windows for a message and waits. When Windows has a message (such as a keyboard or mouse message), GetMessage returns. The caller then passes the message to TranslateMessage (which turns keyboard messages into char messages), and then DispatchMessage (which passes the message to the actual destination window).

However, not all messages are returned by GetMessage. Only posted messages (input messsages and messages posted using PostMessage) are returned by GetMessage. So what happens to sent messages (messages sent using SendMessage or one of its variants)? These messages are dispatched synchronously to the destination window by GetMessage before it returns. The sequence looks like this:

1. Enter GetMessage.


2. Dispatch all sent messages to the destination windows.


3. Get the next posted message and fill the msg structure.


4. Exit GetMessage.

In truth, it gets a little more complicated than this, particularly when you're dealing with another application's message loop. Applications can and do add all sorts of additional processing to the message loop to do special input filtering, show modal dialogs, etc. Moreover, simulating even the simplest click can involve over twenty messages, some sent, some posted, depending on the styles of the destination window.

Unfortunately, directly sending messages to the destination window will not yield the correct behavior because if the application does any special message processing in the message loop we will have bypassed it entirely. Moreover, we never know when one of the twenty messages we send will result in a long running application process that we don't want to wait for. For these reasons, prior to 4.1, we did not provide any synchronous click functions at all. We filled the message queue and let'er rip.

For most scenarios, this works fine. OpenSpan always waits for new controls to be created using our implicit WaitForCreate methods so there are very few instances where we need to wait for the actual result of a click. However, one of those situations involves our prototypical demo application: calculator. Using the standard click methods, the following automation will return 0 or 4 rather than the desired 8. With the information above can you understand why?



That's right, each step of the automation doesn't wait for the previous step to complete. Thus, the automation gets the text before calculator has updated the text field with the final result. The new synchronous click methods do wait for all of the click messages to be processed. Thus, the automation below will return 8 every time.



But what if clicking a button creates a modal dialog? Will OpenSpan wait for the modal dialog to be dismissed by the user before continuing? What if there is no user? What if I want to dismiss the dialog in my automation? Oh noesss!

Simmer down. The short answer is that we do not wait for the modal dialog to be dismissed. Modal dialogs are nothing more than a nested GetMessage loop inside the outer GetMessage loop. Usually the sequence looks something like this:

1. GetMessage returns a mouse up message.


2. DispatchMessage passes the mouse up message to a window.


3. The window procedure creates the modal dialog and shows it.


4. The window procedure calls GetMessage and enters a new message loop to process messages posted to the modal dialog.


5. The user dismisses the modal dialog.


6. The windows procedure exits the message loop and returns.


7. DispatchMessage returns and the outer message loop resumes.

OpenSpan simply waits until step 4, when GetMessage is called again after the mouse up message and returns. Perceptive readers with knowledge of hooking may have guessed how we do this, but for everyone else, it's sufficient to say that we ensure that each time an application calls GetMessage (or any of its variants such as PeekMessage) the next simulated message is always returned. Some other time, maybe I'll talk about how we ensure that OpenSpan messages receive priority over other messages and are always processed in order. Until then, I'll let everyone guess in the comments (everyone except OpenSpan employees, that would be cheating) and give the correct answer a free "Do It on the Desktop" t-shirt.

4.1 Feature Spotlight: Automation Stack

It's that time again! As our 4.1 release nears completion, it's time to start posting about our new features. One of the new features we're testing currently is a new automation stack implementation. As our platform has evolved, we have begun to use our automation engine in new and more complicated environments. Previously, most of our environments were single-threaded automations of applications on the user desktop. Now, our platform is being used on the server to automate back-office requests, process messages and expose applications as web services. In this new environment, it quickly became clearly that our automation engine needed a proper stack to ensure that our automations remained thread safe.

If you're not familiar with OpenSpan, one of your first questions might be how the heck don't you already have a stack? To explain that, I need to go under the hood for a little bit. Looking at the automation your first instinct would be to assume that OpenSpan is a visual programming language that generates source code. For example, the automation below could be expressed in a C# class as:

private string varString;
private void Setup()
{
   btnSubmit.Click += delegate
   {
      varString = txtPhone.Text;
      webBrowser.Navigate(txtUrl.Text);
   };
}

Pretty straightforward, right? How about this automation? In this case we've made one of the links asynchronous indicating that the next step should be performed on a new thread.

private string varString;
private void Setup()
{
   btnSubmit.Click += delegate
   {
      varString = txtPhone.Text;
      NavigateDelegate navDel = delegate(string url)
      {
         webBrowser.Navigate(url);
      };
      navDel.BeginInvoke(txtUrl.Text, null, null);
   };
}

Now it's starting to get more complicated. Every time we need to execute an asynchronous link we would need to generate a delegate type. I'm using anonymous methods which makes it easier, but it's still pretty difficult. How about this automation? We have two execution threads from different events converging on a single block. Easy to express in OpenSpan, but we would need to use another method or delegate in C#.

private string varString;
private void Setup()
{
   btnSubmit.Click += delegate { NextStep(); };
   webForm.Submitting += delegate(object sender, CancelEventArgs e)
   {
      NextStep();
   };
}

private void NextStep()
{
   varString = txtPhone.Text;
   NavigateDelegate navDel = delegate(string url)
   {
      webBrowser.Navigate(url);
   };
   navDel.BeginInvoke(txtUrl.Text, null, null);
}

Even more complicated, and I haven't even added entry points to the automation. So what can we conclude from this exercise? As you might have guessed, OpenSpan does not generate source code. Instead, we model execution flow using a serialized object graph. This gives us greater flexibility to express logical concepts such as branches, joins, etc. without the limitations of traditional syntax. However, because we do not generate code we are unable to rely on the existing thread stack to manage our local variables for us. Thus, prior to 4.1, we simply didn't have a stack at all.

So, how does the new stack work? For the most part, it works like the normal thread stack we all know and love. When an automation is entered, all of the local variables and components are allocated and pushed onto the automation stack. So far so good, right? But what about those pesky asynchronous links? In OpenSpan it's perfectly legal to have two links on different threads in the same automation updating variables.


So what should we do here? In a traditional thread stack model, each asynchronous link gets it's own stack. OpenSpan, however, makes creating new threads absurdly easy. Thus, it's easy to create a number of parallel threads to accomplish tasks such as updating three or four application simultaneously. Should we allocate new variables and components every time we execute a new asynchronous link? Should we initialize them to their original defaults or should we initialize them to their current values?

As you probably guessed, the correct answer is none of the above. The new automation stack is not a thread stack. Local components and variables are allocated on a per automation basis not on a per thread basis. Thus, each of those asynchronous links in the example above are actually updating the same variable. This approach preserves the ease of threading OpenSpan traditionally provides, while adding the additional safety of stack-based variables.

Now that I've explained how the new automation stack works, I'll cover some of the common questions I've been asked about this feature.

What if I want my variables to be available to all automations?
No problem. OpenSpan automations now have a global and a local tray. Components in the global tray behave just like they did in previous versions of OpenSpan. They can be updated from any automation at any time. Components in the local tray can only be used on that automation and are not exposed to other automations.

When are local and global components created?
Global components are created when the project is started and can be used in any automation. Local components are only created when an automation is run. Local components can only be used in the automation that contains them.

When are local and global components destroyed?
Global components are destroyed when the project is stopped. Local components are destroyed when all the threads in the automation have completed.

When should I use a local component?
Local components are appropriate when the component in question does not need to be accessed from other automations or when an automation can be run simultaneously on multiple threads. Local components are recreated every time an automation is run and thus will not maintain state across multiple runs. For instance, you could not create a local variable to keep track of the number of times an automation is run. Every time the automation runs the counter variable would be recreated and initialized to zero.

What happens when I upgrade my old automations?
Variables and components from old automations will automatically be placed in the global tray during the upgrade process. Thus, your old automations will continue to function as they did before. To take advantage of the automation stack, you can easily make any global components local by right-click and selected the "Make Local" menu item.

Desktop Service Enablement: What's the Point?

As many of you know, OpenSpan began making a big push into SOA technologies last year. We even came up with a new marketing slogan: “OpenSpan the last mile of SOA.” While we’ve always had the capacity to consume web services, we wanted to add the ability to expose our automations as services. This culminated in the announcement last week of the OpenSpan Platform SOA Desktop Edition, which allows you to create an automation within or across applications and then expose that automation as a service deployed on the desktop (we call this desktop service enablement). While many of our customers are very excited about this new capability, I have been asked by others “that’s cool but what’s the point?"

I have to admit this reaction makes me want to jump up and down screaming: “What’s the point! What’s the point! Are you kidding! This isn’t just cool! This is totally freaking awesome!”

But… after taking a deep breath, I realize that this is really our fault for not explaining ourselves clearly. As anyone who’s read my blog knows, I think the software industry has treated the desktop environment like some provincial backwater for a quite a while now. Sure you need it to keep your empire running but you sure don’t want to go there if you can avoid it. Even though most of us spend most of our working lives in the Windows environment, the desktop is just the place everyone’s applications happen to end up. All of the cool stuff happens on the server.

Gradually, that opinion seems to be changing. ClickOnce, JNLP, WPF, Adobe Flex and other emerging desktop technologies are using the web as a delivery mechanism while using the power of the desktop to create rich interactive applications for users. Even Web 2.0 and AJAX are examples of this trend. The server delivers the HTML and JavaScript for the site once. After the initial delivery, all of the presentation and interaction logic runs on the desktop, only calling the server when new data is required. However, one thing hasn’t changed, once these applications get to the desktop they are still running in silos, usually isolated from the rest of the desktop for security purposes and unable to interact with the desktop environment.

OpenSpan desktop service enablement is the platform that allows these applications to integrate with the rest of the desktop. With OpenSpan services exposed at the desktop, these applications can now call services on the local machine just like they would their own backend services. What if your AJAX portal could now display not just data from your backend, but from your soft phone running on the desktop, the host system with your billing data, and your Java email application? What if your users could click a link in the portal and generate calendar invitation in Outlook? What if they could click another link and start an automated billing process?

OpenSpan desktop service enablement allows you to integrate any server delivered application with the rest of your desktop. Now, instead of using OpenSpan windows forms, you can create a user interface using the technology of your choice but still use OpenSpan to pull and push data, invoke automations and respond to events. Instead of being a “dumb” application that OpenSpan automates and monitors, your application can be a “smart” application that interacts and controls OpenSpan. That’s not just cool, that’s totally freaking awesome.

3.2 Feature Spotlight: Debugging

Over the next few weeks I’ll be writing some articles to spotlight new features in our 3.2 release. For many of our users, the most exciting new feature is debugging. For the first time, you now have the ability to debug OpenSpan automations in real time using paradigms familiar to users of Visual Studio, Eclipse and other IDEs such as breakpoints, step functions, and watch, call stack and thread windows.

Breakpoints

The essence of debugging is giving the developer the ability to stop execution and examine the state of an application. Stopping execution is referred to as breaking. By breaking execution a developer can find flaws in his logic, scenarios he did not anticipate, or invalid data. The most convenient way to break execution is a breakpoint. A breakpoint is simply a way of telling an application to “stop when you get here.” When the application is run, the debugger will break execution when the breakpoint is reached.

To set a breakpoint in an OpenSpan automation, simply highlight a link, right click and select “Toggle Breakpoint.” A red dot will now appear on the link indicating that a breakpoint has been set. If you right click on an existing breakpoint, you can select “Disable” to temporarily prevent the debugger from breaking when the breakpoint is reached, or “Delete” to remove the breakpoint entirely.

When a breakpoint is reached within an automation, the executing link will become a dashed, animated line. The blocks on either side of the link will also be outlined. The executed block will have a solid blue outline whereas the executing block will have a dotted blue outline.
You can view all of the breakpoints you have set within a project in the “Breakpoints” window. The Breakpoints window lists each breakpoint by automation. You can double-click on a breakpoint to go to that automation. You can also enable or disable a breakpoint using the checkbox next to it. If you right-click on an automation, you can also enable, disable or delete all breakpoints within that automation.

Step Functions

Once a breakpoint has been reached, you can begin “stepping” through an automation. By using the step functions, “Step Over” or “Step Into”, you can walk through your logic at your own pace. The “Step Over” function moves the automation to the next yellow execution link. The “Step Into” function moves the automation to the next blue data link, or if there are no blue data links, to the next yellow execution link. Once you have finished stepping through your automation, you can select “Continue” to resume execution until your next breakpoint is reached. The “Step Over”, “Step Into” and “Continue” options are available within the “Debug” menu and toolbar, but most developers simply use the F10, F11 and F5 keys respectively.

Watch Windows

When debugging, you can view the value of any property by hovering over its data port in the automation. If you want to “watch” how a property value changes while you step through your automation, you can right-click on the property and select “Add Watch.” The property will now be displayed in the “Watch Window.” Anytime the property changes, the watch window will update to reflect the new value. The watch window can display properties utilized in any automation, even ones that are not open or executing. OpenSpan studio also provides a special type of watch window called the “Locals” window. The “Locals” window will automatically display all of the properties in the currently executing automation.

Call Stack and Threads Windows

In addition to the “Watch” and “Locals” windows, OpenSpan Studio provides the “Call Stack” and “Threads” windows to help debug automations. The “Call Stack” window shows the previously executed blocks within an automation. This allows you to see what path was executed within your automation to get to your breakpoint. The “Threads” window shows all of the currently executing automation threads. This allows you to debug complicated multi-thread scenarios within your project.

Future Directions

With the 3.2 release, OpenSpan has for the first time provided automation developers with integrated debugging. We are already discussing additional debugging features for our future releases such as runtime match rule debugging. Let me know what other debugging features you would like to see.