Ouch, over a year?

Can't believe it's been over a year since I posted. Things were busy. So here's the latest:

I'm now CTO of OpenSpan. I've taken on a lot of additional responsibilities around product strategy and customers, which leads to...

Stephen is now Chief Scientist. He's officially the undisputed god of the deep, dark, awesome, unfathomable underpinnings or our technology and is devoting himself full-time to internals and RE, which leads to...

Danny is now VP of Development. He's taking over running our team operationally, delivering release and all that good stuff, which leads to...

We're growing and we're hiring. If you're a data guru (NoSQL, MySQL, Hadoop) or a cloud guru (Ruby, AWS) or a web guru (HTML, jQuery, CSS) or a reverse engineering guru (injection, hooking, disassembly) let me know!

Handling web pages that navigate back to themselves

I recently worked through a customer issue that revolved around a page that reloaded itself when a form was submitted. This is a pretty common scenario, but it can be a little tricky to handle in OpenSpan, especially if the reloaded page doesn't actually create any new elements to wait upon. The trick is to wait for the Destroyed event to fire on the web page after you call the method that triggers the reload. This is pretty easy to do in an automation by using the "Wait for this event" feature.

First, drag the Destroyed event from the page onto the automation surface. Right click and select for "Wait for this event". The event will now transform into an object you can wait for.

_{Selecting the "Wait for this event" command}

Next, connect the Setup port to the method that will trigger the reload, in this case the PerformClick method.

_{The right way to trigger the event.}

Note, you must use the Setup port to ensure that OpenSpan is ready to wait for the event when you trigger it. If you call PerformClick before you enter the event block there is always the chance that the Destroyed event will fire between the two blocks in which case the event will be missed.

_{The wrong way to trigger the event.}

Finally, connect the Fired port to the blocks that should execute after the page has reloaded.

_{The complete solution.}

As you can see, waiting for an event is very easy in an automation, but how would you accomplish the same thing in code? You could spin your own eventing mechanism, but this would be a fair amount of work and require a good knowledge of threading. Fortunately there is another way to access the wait for event functionality. Every OpenSpan control supports a GetEvent method which takes a special EventName object. This method actually pre-dates the automation "Wait for this event" functionality. GetEvent takes a parameter of type EventName which is simply a wrapper around the name of the event. GetEvent returns an Event object which exposes a Wait method. Events objects are managed internally by the control so you do not have to worry about disposing the object. The following code snippet illustrates how to use GetEvent.

public void DoSearch()
{
 // Get event
 Event waitForDestroy = mSample.EventSamplePage.GetEvent(new EventName("Destroyed"));

 // Perform click
 mSample.Next.PerformClick();

 // Wait for event
 if (waitForDestroy.WaitForEvent(10000))
 {
  // Next steps
 }
 else
 {
  // Handle error
 }
}

Hosting OpenSpan: A complete ISynchronizeInvoke wrapper for the Dispatcher

As I mentioned in my last post, I've been working on a sample where I host OpenSpan adapters inside of a WPF Prism application. Here's a screenshot that will help give you a sense of it.


When I started fleshing out the sample, I realized that WPF didn't provide an object that implemented ISynchronizeInvoke that I could pass to the adapter Start method. As a little background, we added an overload to the Start method that takes an instance of ISynchronizeInvoke as a convenience for developers using adapters within windows forms applications. Normally, within the OpenSpan runtime, adapters fire asynchronous events on thread pool threads. Our automation surface abstracts the invocations required to interact with windows forms from visual developers.

When hosting adapters directly in .NET, we quickly realized that it was painful to force developers to invoke onto the windows forms thread anytime they needed to update their UI within an adapter event. Thus, we added an overload to the Start method where developers could pass in an instance of ISynchronizeInvoke. In practice, since all windows forms controls implement ISynchronizeInvoke, developers simply pass in their form or user control. When an adapter is started with an instance of ISynchronizeInvoke, it will automatically invoke all events onto the right thread.

However, the WPF Dispatcher object, which provides equivalent functionality to ISynchronizeInvoke, does not actually implement ISynchronizeInvoke. A quick search revealed that other folks had run into the same issue with WPF and created wrappers. However, looking over the wrappers, they weren't entirely complete. It seems that nobody had implemented every method or property required. In particular I knew that our adapters depended on the EndInvoke method and the IAsyncResult.AsyncWaitHandle property.

To remedy this, I implemented my own wrapper. In addition to the public DispatchWrapper, I created a couple of nested private classes, DispatchOperationWrapper and DispatcherOperationWaitHandle, to hide the IAsyncResult and WaitHandle implementation details. Unlike some of the samples I found the implementation below doesn't use any looping or sleeps to wait. Also note that the implementation for IAsyncResult.CompletedSynchronously always return false. This conforms to the IAsyncResult guidance on MSDN.

using System;
using System.Threading;
using System.Windows.Threading;

namespace OpenSpan.Samples
{
 public class DispatcherWrapper : ISynchronizeInvoke
 {
  private Dispatcher _Dispatcher;

  public DispatcherWrapper(Dispatcher dispatcher)
  {
   _Dispatcher = dispatcher;
  }

  #region ISynchronizeInvoke Members
  public IAsyncResult BeginInvoke(Delegate method, object[] args)
  {
   DispatcherOperation op = _Dispatcher.BeginInvoke(method, args);
   return new DispatcherOperationWrapper(op);
  }

  public object EndInvoke(IAsyncResult result)
  {
   DispatcherOperationWrapper wrapper = result as DispatcherOperationWrapper;
   if (wrapper != null)
   {
    wrapper.Operation.Wait();
    return wrapper.Operation.Result;
   }
   throw new ArgumentException("Result does not wrap a DispatchOperation");
  }

  public object Invoke(Delegate method, object[] args)
  {
   return _Dispatcher.Invoke(method, args);
  }

  public bool InvokeRequired
  {
   get { return _Dispatcher.CheckAccess(); }
  }

  #endregion

  private class DispatcherOperationWrapper : IAsyncResult
  {
   private DispatcherOperationWaitHandle _WaitHandle;
   private DispatcherOperation _Operation;
   private object _State;

   public DispatcherOperationWrapper(DispatcherOperation operation)
   {
    _Operation = operation;
   }

   public DispatcherOperationWrapper(DispatcherOperation operation, object state)
    : this(operation)
   {
    _State = state;
   }

   public DispatcherOperation Operation
   {
    get
    {
     return _Operation;
    }
   }

   #region IAsyncResult Members
   public object AsyncState
   {
    get { return _State; }
   }

   public WaitHandle AsyncWaitHandle
   {
    get
    {
     if (_WaitHandle == null)
     {
      _WaitHandle = new DispatcherOperationWaitHandle(_Operation);
     }
     return _WaitHandle;
    }
   }

   public bool CompletedSynchronously
   {
    get { return false; }
   }

   public bool IsCompleted
   {
    get { return (_Operation.Status == DispatcherOperationStatus.Completed); }
   }
   #endregion

   private class DispatcherOperationWaitHandle : WaitHandle
   {
    private DispatcherOperation _Operation;

    public DispatcherOperationWaitHandle(DispatcherOperation operation)
    {
     _Operation = operation;
    }

    public override bool WaitOne()
    {
     DispatcherOperationStatus status = _Operation.Wait();
     return (status == DispatcherOperationStatus.Completed);
    }

    public override bool WaitOne(int milliseconds)
    {
     return this.WaitOne(new TimeSpan(0, 0, 0, 0, milliseconds));
    }

    public override bool WaitOne(int milliseconds, bool exitContext)
    {
     return WaitOne(milliseconds);
    }

    public override bool WaitOne(TimeSpan timeout)
    {
     DispatcherOperationStatus status = _Operation.Wait(timeout);
     return (status == DispatcherOperationStatus.Completed);
    }

    public override bool WaitOne(TimeSpan timeout, bool exitContext)
    {
     return this.WaitOne(timeout);
    }
   }
  }
 }
}

Hosting OpenSpan: You Can't Start Adapters Synchronously in a Load Event

This past week I've been working on a sample illustrating how to host OpenSpan within WPF. Specifically, I've been working on using our re-parenting control to host external applications within the Prism application block from Microsoft.

For my first attempt at using the re-parenting control, I created a WPF user control with this XAML:

<UserControl x:Class="StockTraderRI.Modules.Research.GoogleFinanceView"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:os="clr-namespace:OpenSpan.Controls.Reparenting.TabbedReparentContainer;assembly=OpenSpan.Controls"
    Height="Auto" Width="Auto">
    <DockPanel LastChildFill="True">
        <TextBlock Text="{Binding TickerSymbol}" Style="{StaticResource CurrentSymbolTitle}" DockPanel.Dock="Top" />
        <WindowsFormsHost Name="_WindowsFormsHost">
            <os:TabbedReparentContainer Load="OnReparentContainerLoad" Disposed="OnReparentContainerDisposed" />
        </WindowsFormsHost>
    </DockPanel>
</UserControl>

As you can see I declaratively added the TabbedReparentContainer within the WindowsFormHost. In the load event handler I created and started the adapter:

private void OnReparentContainerLoad(object sender, EventArgs e)
{
 _Google = new GoogleFinanceAdapter();
 _Google.HomePage.Created += OnGoogleHomePageCreated;
 _Google.Start();
}

The setup was very easy so I was feeling pretty confident until I ran the project and received this exception: "Unable to start message form."

I was puzzled since I knew that the exception occurs when we cannot start the windows form we use internally for inter-process communication. What could be stopping us from starting the windows form? To find out, I downloaded the latest version of Reflector and started debugging. If you haven't used it, the latest version of Reflector let's you debug decompiled .NET assemblies within Visual Studio. With Reflector it was pretty easy to see that we were stuck in System.Windows.Forms.NativeWindow:

public virtual void CreateHandle(CreateParams cp)
{
 System.Windows.Forms.IntSecurity.CreateAnyWindow.Demand();
 if (((cp.Style & 0x40000000) != 0x40000000) || (cp.Parent == IntPtr.Zero))
 {
  System.Windows.Forms.IntSecurity.TopLevelWindow.Demand();
 }
 lock (this)
 {
  this.CheckReleased();
  WindowClass class2 = WindowClass.Create(cp.ClassName, cp.ClassStyle);
  lock (createWindowSyncObject) // <== STUCK
  {
   // ...
  }
 }
}

So there you have it. When .NET is creating a window on one thread, you cannot start another thread and create a window on it at the same time. I'm not really sure why this restriction exists as there isn't anything inside of the lock that looks particularly unsafe. Nevertheless, because of this, you cannot start adapters synchronously within the Load event, at least in Beta 2.

I'm continuing to work on this sample and will keep posting other interesting things I find over the next few days.

When doing something the "right" way is wrong

Francis had an interesting post earlier this week: Not using OpenSpan? You could be losing millions. His basic premise is that if you're not using OpenSpan, you're losing millions of dollars. The funny thing is that he's right. This isn't just hyperbole. Our customers really do save millions of dollars. So why isn't the world beating a path to our door? Simply put, it's because the technology community thinks we are the "wrong" way of solving problems.

Are your users having to cut and paste and toggle between too many applications? An OpenSpan developer can develop a solution to that problem in minutes and deploy it in days. But the "right" way of solving that problem is to integrate the applications on the back-end and provide a new user interface. How long is that going to take? I bet it's longer than a few days.

Are your users having to navigate between too many screens to complete a task? An OpenSpan developer can develop a solution to that problem in minute and deploy it in days. But the "right" way of solving that problem is to rewrite the application and provide a better user interface. How long is that going to take? I bet it's longer than a few days.

I could keep going, but I think you get the picture. Of course, the real question is what is "right"? If "right" is defined as the most elegant solution that will be most extensible in the future, then OpenSpan is the wrong solution. If "right" is defined as the most cost-effective and rapid solution that will provide value to the business, then OpenSpan is the "right" solution.

Ultimately, the "right" answer it to do both. Solve the business problems in the short term with OpenSpan and in the long term with your strategic architecture. Most IT organizations have backlogs that contain thousands of feature requests. Usability issues, unless they are absolutely horrendous, almost always get pushed below strategic features. And with the resources available and the time alloted, that's the right decision. Many of those big features will save hundreds of millions of dollars. While those pesky little features are only worth tens of millions of dollars.

But why waste those tens of millions of dollars when you could implement them quickly and cheaply with OpenSpan? Just today, we had a deal killed by a CTO who said, "Don't worry about that, we're reimplementing that application anyway. It's a waste of money." Really? It's a waste of money to buy a product where the ROI is typically delivered in less than three months? Whose cost of ownership is a tenth of the savings it delivers? That enables a team of less than five developers to save a company millions?

I try not to criticize the organizations we sell to, but it's incredible the narrow attitudes that prevail in our industry. Real agile practitioners know that you get the best results by delivering value rapidly. Every day, I tell my team to ask themselves "What's the value of what I'm doing? Is there anyway I can deliver the same value better or faster?" Although agile is on the rise, it seems clear that the core principles behind agile have yet to filter up to the decision makers within organizations.

Here's the bottom line: If you're not using OpenSpan, you're not delivering the value you should deliver. You're wasting money and you're hurting your business.

OpenSpan 4.5 Public Beta is Here!

After over a year of hard work, the next OpenSpan release is upon us. This morning we released the public beta of OpenSpan 4.5. For the first time, OpenSpan Studio will be free for anyone to download and evaluate.

For this release we rewrote our design environment to utilize Visual Studio Extensibility (VSX). OpenSpan Studio will now be available as both a Visual Studio 2008 plug-in and a stand-alone IDE. All OpenSpan developers will benefit from new features enabled by VSX such as source control integration, undo-redo and project references. However, we're most excited by the new features we can offer .NET developers.

In 4.5, we have changed the output of OpenSpan projects to be .NET assemblies. At design-time, we still utilize XML documents to store the data, but we have added a build step that generates and compiles a class for the project and each project item (adapter, automation, etc.). The generated classes are identical to the classes utilized in OpenSpan automations. Anything you can do in OpenSpan automations you can now do with C#, VB or any .NET language.

So how do we anticipate people using this? Well, currently we have a number of customers who embed our functionality into new or existing windows forms or WPF applications they have created. Our new features will make this integration simple and seamless, enabling more complex integrations than ever before. In particular, with our re-parenting control, you can quickly and easily integrate web pages, terminal screens and Java applications into your composite application or mash-up not only visually, but functionally as well. Anytime you are creating a new .NET application, you should think about the benefits using OpenSpan can bring.

Needless to say this is a big step for us, one that has necessitated not just code changes but organizational changes as well. In order to support the needs of developers, we have introduced a new developer portal with forums and code samples and also a new incident-based support model. I hope you'll check out the beta and give us feedback on the forums. To get started, check out the OpenSpan Community.

How OpenSpan Could Support the Google Chrome OS (and Linux in General)

Last week I mentioned that I would follow-up my original post on the Google Chrome OS with a post on how OpenSpan could support the Google Chrome OS (and Linux in general). I emphasize could, because we currently have no plans to support the Google Chrome OS or Linux. Thus, this post is really just informed speculation on how our technology could be applied to Linux.

Diagram 1
Diagram 2
Diagram 3Typically, when we talk about OpenSpan, we focus on our Windows translator, Scout, as the base layer which everything builds upon. In fact, we often say "OpenSpan sits in between the application and Windows". We then talk about how we use the Scout layer to detect when additional technologies are loaded into an application so we can load translators to expose those technologies. For example, to explain how we support Java, we might whiteboard something like Diagram 1.

However, Diagram 1 isn't really correct. There's actually another layer below Windows, the instrumentation layer, responsible for hooking x86-64 byte code. Of course, once you're dealing with byte code, you're dealing with the processor, not the OS. Thus, our hooking library can hook functions on both Windows and Linux systems. When we add the instrumentation layer, our Java stack looks like Diagram 2.

So what would it look like if we added Linux to the mix? You might think that adding Linux would mean two different stacks, one on top of Windows, another on top Linux. However, this isn't the case. For technologies that run on both Windows and Linux, the stacks would actually reconverge at that layer. Diagram 3 shows what our Java stack looks like with Linux.

Of course, there's only a few UI frameworks that work on both Windows and Linux (see list here), most importantly Java and HTML. Unfortunately, HTML is a markup language rather than a binary specification so we have to adapt to each browser rather than HTML itself. However, we could still create a common set of HTML element objects that interact with each browser implementation through an abstraction layer. It gets even more interesting if you start thinking about server technologies.

To summarize, above the OS layer, most multi-platform technologies will reconverge on a common set of classes. Thus, although we will need to write a Linux layer, we will not need to write a new Java layer or two separate FireFox layers. Don't get me wrong, supporting Linux would still require a significant degree of effort. However, it is a manageable amount of effort because we can build on the layers we already support.