Compare Merge Viewer Example: Merging Word Documents

I just finished writing up a Wiki article describing how to implement a custom (i.e. non-text based) compare merge viewer. The example I used was a Word document comparison. Hopefully I'll be able to get the code into the Eclipse 3.5 stream once floodgates open for 3.5 development but, in the meantime, there's a link in the article from which you can download the bundle (it's a fragment of the org.eclipse.compare plug-in). The code is compatible with 3.3 and 3.4 so if you are interested in either seeing how to implement a custom compare merge viewer or you want to try out the Word document compare viewer, check out the article.

My first (and second) OSGi Service

It seems everywhere I look these days, I see OSGi mentioned. So in an attempt to be fashionable (and because I think it was the right thing to do) I have made a couple of new features that have been added the Eclipse Platform available as OSGI services (I should note that these features will appear in the next integration build). The new services are:

• IProxyService: a service that allows clients to access and modify the proxy settings for HTTP, HTTPS (or SSL) and SOCKS and ensures that the values specified are put into the corresponding Java system properties. This service is located in the org.eclipse.core.net plug-in and there is an associated preference page for setting the proxies.
• IJSchService: a service that complements the JSch SSH2 client by ensuring that JSch is properly configured (using settings from the SSH2 preference page) when clients attempt to make SSH2 connections. The service also uses the proxy service to configure JSch proxies. The JSch service is found in the org.eclipse.jsch.core plug-in while a generic prompter can be found in the org.eclipse.jsch.ui plug-in.

In order to use these services, you need to obtain them using OSGi. Niel provides some general articles on working with OSGi services. To illustrate, here's an example of how you can use the JSch service (and, indirectly, the proxy service) to make a connection using a proxy.

The first thing you need to do is add a service tracker in you bundle activator (or, in the old world lingo, add a tracker to your plug-in class). Here's the code I needed to add to my Activator class.

private ServiceTracker tracker;

public void start(BundleContext context) throws Exception {
   super.start(context);
   tracker = new ServiceTracker(
      getBundle().getBundleContext(),
      IJSchService.class.getName(),
       null);
   tracker.open();
}

public void stop(BundleContext context) throws Exception {
   tracker.close();
   super.stop(context);
}

public org.eclipse.jsch.core.IJSchService getJSchService() {
   return (IJSchService)tracker.getService();
}

Once you have that, you can make an SSH2 connection in the following way:

IJSchService service = CVSProviderPlugin.getPlugin().getJSchService();
if (service == null)
   // Service is not available
Session session = service.createSession(hostname, port, username);
int timeout = 60000;
session.setTimeout(timeout);
if (password != null)
   session.setPassword(password);
session.setUserInfo(new org.eclipse.jsch.ui.UserInfoPrompter(session));
service.connect(session, timeout, monitor);

The above connection methods will use any proxies that are specified in the proxy service. However, with the JSch service, you can also use proxies with other types of connections (e.g. CVS pserver). Here's how:

private Proxy getProxy() {
   IJSchService service = Activator.getInstance().getJSchService();
   if (service == null)
      return null;
   Proxy proxy = service.getProxyForHost(
      hostName,
      IProxyData.HTTPS_PROXY_TYPE);
   if (proxy == null)
      proxy = service.getProxyForHost(
         hostName,
         IProxyData.SOCKS_PROXY_TYPE);
   return proxy;
}

If the returned proxy is null, you can make a direct connection. Otherwise, you can use the Proxy#connect method to connect to a specific host. There's a helper method on the JSch service to connect to a proxy in a responsive fashion (i.e. respond to cancellation) that you can use as well.

service.connect(proxy, hostName, port, timeout, monitor)

So, there you have it. No more need to define static classes for accessing singleton services. Initially, I found accessing the OSGi services a bit more cumbersome than the static class approach but I think the added flexibility is worth it.

Responsive Remote Connections

We've had a couple of contributions for fetching contents directly from a URL:

• Add a URL to Apply Patch Wizard (bug 73683)
• Add a URL to Project Set Import: (bug 162608)

These are great ideas but the problem is that the contributions don't treat accessing the contents from the remote system any differently than accessing contents from the local file system. Unfortunately, there are several issues related to fetching content over remote connections that make this problematic. Some problems were less severe, such as manual cancellation not being instantaneous, while others were more severe, such as Eclipse hanging and requiring a restart.

We encountered these issues several years ago with the CVS client and came up with solutions that allowed the remote connections to remain responsive. I thought it would be worthwhile to summarize the issues and what we came up with to solve the problems. I would be curious to know if others have encountered similar problems and, if so, how they solved them. Also, I'd be curious to know if the situation has improved in later versions of Java (our coding was done against 1.4 or earlier).

Problem 1: Connecting to a Socket may hang indefinitely

When initially connecting to a socket in Java, there is no way to specify a timeout. In most cases, this is not a problem as the connection will either succeed in a fairly short time or fail fairly quickly. However, we found that, under some circumstances, the connection attempt would hang indefinitely.

To work around this, we wrote a createSocket method that performed the open in a separate thread while the caller thread blocked until the socket was opened. However, the caller thread would only sleep for a second at a time and would also only loop for a predefined number of times (default of 60). That way, the connection would only block for a maximum of 60 seconds (or whatever value the user specified as the CVS timeout preference) and would respond to cancellation with a second. For those that are curious, the implementation of the create socket is in the CVS/Core Util class.

Problem 2: Users expect cancellation to happen immediately

Once you have a socket open, you can specify the read and write timeouts for the socket streams. Thus, you can ensure that the connection will be dropped if it has been blocked for too long. However, after waiting for a while, the user may decide to explicitly cancel the operation and would expect this to occur immediately. To make this work, we created a wrapper stream that would only block on the read for a second. If no content arrived in that time, the stream would poll to see if the user canceled. If a cancel did occur, the read was terminated. Otherwise, the read was reattempted until either content became available or the number of attempts exceeded the timeout (default 60).

The other benefit of this stream wrapper is that appropriate progress messages could be shown to the user (e.g. 350 or 2000 bytes read). The stream implementation is internal to the Team/Core plug-in in the PollingInputStream class.

Problem 3: Some streams do not have a timeout

The final problem we found was that some streams do not have a timeout. Specifically, when reading the output from a system process (for the EXT connection method), under some circumstances, the read could block indefinitely. To solve this, we used a similar approach to problem 1. All the reading was performed in a separate thread and any read content was passed between the reading thread and the caller thread using a shared buffer. The implementation for this is found in the TimeoutInputStream class in the Team/Core plug-in.

Thoughts?

As I mentioned earlier, I would be curious to know how others have addressed these issues. Is there facilities available from the JDK that make this easier or are there better approaches that should be considered? Let me know what you think.

SSH2 Support in Eclipse

One of the things that has happened in Eclipse 3.2 is that the SSH2 client used by CVS has become a separate plug-in: com.jcraft.jsch. The Jsch library has been used since 3.0 for the SSH2 connection (extssh) support and is now also used to provide proxy support for pserver connections. In order to make the Jsch library accessible to the Core CVS plug-in, we needed to move it out of the extssh plug-in and into a separate plug-in. As a result of this, other plug-ins in Eclipse 3.2 can now make use of the Jsch library to make SSH2 connections.

It was a patch provided by Eugene Kuleshov to add the pserver proxy support that was the catalyst that resulted in the split. Atsuhiko Yamanaka from JCraft also played an active role in making this happen (Jcraft has also just posted an SFTP plugin on their website that makes use of the Jsch plugin to provide SFTP file transfers). I think this is a great example of how community members have worked together to make Eclipse better.

Common Navigator and the Synchronize view

Thanks to Chris, I saw Michael's Common Navigator post over the weekend. The framework is new in Eclipse 3.2 and there have been a number of questions about it on the eclipse-platform newsgroup already. Michael gives a pretty thourough description of the features of the framework. I would encourage anyone who provides a tree-based view of their model in Eclipse to check it out.

As it turns out, the new model-based synchronization support that was added in Eclipse 3.2 makes use of the Common Navigator framework to allow models like Java to appear in the Synchronize view. Here's a screenshot that shows our Team Example model, a Java model and the Resource model all in the same CVS synchronization.



For those that are curious, the Team Example model is available in the org.eclipse.team.examples.filesystem plugin which you can get from the :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse repository or in the RC5 or later Eclipse Platform Example plug-ins download. The purpose of the example is to illustrate how to integrate logical models like Java with Team providers like CVS and includes Common Navigator extensions for both the Project Explorer and the Synchronize view. The RC5 version of the example is in reasonable shape but the version in HEAD is more complete.

Don't speak until spoken too

As the 3.2 release date approaches, I thought that it would be a good time to install Callisto. We're performing an RC2 2-day test pass on the Platform so I didn't really plan on digging too deeply into the features of Callisto right away but I thought it may add a bit of variety to the test pass to have some new stuff to look at. It turns out that even just having Callisto installed while doing my Platform testing was a useful exercise and here is why.

I had several plugin projects in my workspace and needed to close a few. Upon doing so, I got an exception from a validator that was trying to write out some project preferences. There were two problems with this. Firstly, the validator should not have been writing the preferences out during the project close delta (hence the exception). However, the larger problem is that I was doing plugin developement and the validator that was writing out the preferences had nothing to do with developing plugins.

There's been a lot of talk about Eclipse and Callisto as a Platform. An important aspect of this is how invisible a component is if it is present in an install but is not being used. We've really felt this with the Eclipse CVS client since it is part of the SDK but is not of interest to all users of Eclipse. One can think of Callisto as a mega-SDK so the same rules apply. However, we developers become so focused on improving the feature set of our components, that it is hard to adequately test how well our component behaves when it is not being used.

So, if you would like to help out testing for Callisto but you only use a small subset of the features, it is still worthwhile to install the whole thing. Then you can help tests the parts you care about and also help to ensure that the parts you aren't using remain silent.

Image Management

Just to follow up on what Tod mentioned in his blog about image management, the ResourceManager and ImageRegistry classes are not just about maqnaging the lifecycle of an image, they're also about how images can be shared. I thought it would be worthwhile to give a brief description of each of these classes.

ResourceManager: A resource manager can be used to keep track of a set of created Images. The client of this class is still responsible for creating and disposing of images but creation and disposal is done by calling the createImage(ImageDescriptor) and destryImage(ImageDescriptor) methods on ResourceManager. The obvious question is why would you want to use a resource manager if you still need to dispose of the image at some point. The reason is that the resource manager will ensure that, for a particular descriptor, the image is only created once, thus saving precious OS resources. The manager uses reference counting to ensure that the image is only disposed when it has been destroyed the same number of times it has been created.

So the advantage of using a resource manager is that you can reuse images but the disadvantage is that you still need to dispose of the images when you are done with them. However, disposal can be simplified somewhat by using a LocalResourceManager. Let's say I have a set of images that need to be available while a particular viewer is opened but can be disposed when the viewer is closed. I can create a LocalResourceManager that wraps the JFace resource manager (JFaceResource.getResources()) and use it to create any images I need. I can add a dispose listener to the control of the viewer that will dispose of the manager when the viewer is closed. This way, I don't need to ensure that I match each create with a dispose. I can just happily call create when I need an image, knowing that the manager will never create the same image twice and will dispose of them all when I call dispose on the manager.

There is another advantage of managing images this way. The image created from a LocalResourceManager for a particular descriptor is also pushed up to the parent manager(e.g. the JFace resource manager). This means that the image is accessible through the parent manager or through other local managers that share the same parent. This means that at most one image is created for a particular descriptor but the image will be disposed when it is no longer used anywhere.

ImageRegistry: An ImageRegistry wraps a ResourceManager and allows image descriptors and images to be referenced by a key of type String. Also, when creating an image registry, you can populate it with image descriptors and the registry will only create the corresponding image the first time it is referenced. Because an ImageRegistry wraps a ResourceManager, you get all those benefits as well. As with a ResourceManager, an ImageRegistry needs to be disposed of when the images in the registry are no longer needed and disposing a registry will free all the images in its resource manager if they are not used by another manager.

However, as Tod pointed out, one must be careful when adding images to either the JFace image registry (JFaceResources.getImageRegistry()) or the image registry of a plugin (AbstractUIPlugin#getImageRegistry()) since both of these registries are only disposed when the workbench is closed. I think a better approach would be to create ImageRegistries that are local to views and editors and have these registries wrap LocalResourceManagers so that the created images are shared when possible but are disposed when no longer in use.

Disclaimer: I started out as a Core level guy who has been thrust into this UI level stuff. It turns out that most of the image mangement code I've written up until this point has been done using a HashMap. I just recently stumbled on this API when fixing an image disposal problem. Given that 3.2 development on the Platform is pretty much frozen at this point, I'll need to wait until 3.3 before I can put this into practice (i.e. treat the above as ideas and discussion points instead of the carved-in-stone way to do image management).

Yet another blog about Eclipse

Now that I'm a committer on Platform/UI as well as Team/CVS, I figured it would be worthwhile to start a blog to capture some of the things I come across while developing and maintaining these components.