CopyOnWrite Wrappers Part 2 (now CIS Wrappers)

There was some good feedback on my first pass. To summarize:

1. The implementation is not a "true" CopyOnWrite. It is more of a "concurrent iteration safe wrapper". Implying that it is CopyOnWrite will confuse users.

2. Does the code pass the JSR-166 Unit tests?

3. For speed: Why are the getters synchronized? Could you use atomics?

4. What are you trying to solve, why not use a (some other class)?

I have updated the code, at http://flyingspaniel.wikidot.com/cow to address those comments:

1. To better indicate that these are not true CopyOnWrite, they are now named CISListWrapper and CISMapWrapper, where CIS stands for "Concurrent iteration safe". There are improved comments. Users who are familiar with CopyOnWrite behavior should not be confused or disappointed.

2. In addition to my own unit test (CISWrapperTest.java) I modified some of the JSR 166 unit tests, renaming them, making them more generic, and taking out a few tests (serialization) that didn't really apply (or work for me). This required a few changes in my code - mainly adding equals(), hashCode() and toString() methods that I had neglected.

3. As implemented, the reads do need to be synchronized. (also here). I wasn't concerned about ultimate speed, and this has the tremendous advantage of being more "idiot-proof". The wrapped Collection need not be synchronized. Removing the synchronization might require a user to add a Collections.synchronizedXXX() wrapper around the underlying collection, adding one more layer and eliminating any speed benefit.

I did consider changing the synchronization from the wraper to the wrapee. That is, instead of the current

public synchronized boolean containsKey(Object key) {

return wrapee.containsKey(key);

}

use instead:

public boolean containsKey(Object key) {

synchronized(wrapee) {

return wrapee.containsKey(key);

}

}

This is "left as an exercise for the reader" and might offer modest speed increases if the underlying collection were itself synchronized. If you really want the utmost in speed on simple reads, extend or use something like ConcurrentHashMap, or use one of the "true" CopyOnWriteMaps:

org.apache.mina.util.CopyOnWriteMap

and the Atlassian Concurrency Utilities.

4. So why use the CIS code? The "true" CopyOnWrite wrappers make a copy of your Map, and, for the most part, they copy the data into a basic HashMap or TreeMap. If that behavior is suitable, you are better off using their wrappers. Of course, if that basic behavior is suitable and you are seeking ultimate speed, you could consider rewriting your code to use the "standard" java.util.concurrent.ConcurrentHashMap.

If you have an existing List or Map that is not thread-safe or iteration-safe, and it has special or complex behavior that is not a simple HashMap or TreeMap, then this pure wrapper is useful. For example, you have a class com.mycompany.FunkyMap and it:

1. validates inputs and throws Exceptions
2. has special behavior for null keys or values
3. has unusual rules for sorting
4. implements some security rules on puts and gets. (and throws Exceptions)
5. logs stuff
6. encrypts values and stores them on a database
7. is some facade or proxy (say, for an ORM)
8. is buggy and you have set some breakpoints in your IDE or added printlns.
9. is a singleton

In order to keep this behavior in a pure CopyOnWrite, the copy would have to itself be a FunkyMap, not a TreeMap. One could use reflection, but there goes all the speed bonus. To their credit, the Atlassian Utilities do provide for the possibility of subclasses. It's a small bit of work and may not be suitable for all cases.

CopyOnWrite Wrappers

Java has a CopyOnWriteArrayList is a very useful class in java.util.concurrent which, when iterating, takes "snapshots" of the underlying array, and therefore never throws a ConcurrentModificationException.  It is highly effective and efficient when you need to preclude interference among concurrent threads.  But it only implements two types of collection behavior: ArrayList and a CopyOnWriteArraySet.  Unlike many of the other java.util.Collections goodies, like Collections.unmodifiableList, it is not implemented as a wrapper class, but as an actual class.  So, if you want behavior other than an ArrayList or ArraySet, or you need to protect one of your own special List implementations, you are out of luck.  Nor is there any version for a Map, i.e. there is no CopyOnWriteMap.

So I implemented it.  The files are on my wiki page at http://flyingspaniel.wikidot.com/cow.

COWListWrapper wraps any List, providing "CopyOnWrite" behavior for iteration.
COWMapWrapper wraps a Map, providing "CopyOnWrite" behavior when you iterate using keySet(), entrySet(), or values().

There are some hopefully useful utility classes that they use:

UnmodifiableCollection is an actual class, not a wrapper utility, to wrap an underlying Collection with unmodifiable behavior.  It includes static inner classes, UnmodifiableCollection.List and UnmodifiableCollection.Set, that add List or Set behavior.

UnmodifiableIterator is a class implementing ListIterator that works efficiently with the above classes and prohibits modification during iteration.

Enjoy!

A couple of notes.  For any CopyOnWrite implementation, even the "official" Java ones, you only get the iteration safety if you actually use the iterator( ).  If you use the old fashioned "C-style" integer loop

for (int i=0; i < myList.size(); i++) {
  doSomethingWith( myList.get(i));
}

and a separate thread is modifying myList, all bets are off.  You should use the new style:

for (E element:myList) {
  doSomethingWith(element);
}

The standard Java CopyOnWrite classes know exactly what collection behavior (ArrayList or ArraySet)  to implement and, at construction, they make a safe copy any incoming data, and thereafter are completely divorced from the original collection.  My wrapper classes do not, and cannot, know all possible List or Map behaviors.  They use the passed in (wrapped) List or Map for storage and most of the implementation.  For example, a call to add(Object o) is passed to the underlying object, and it implements the behavior and storage.  In other words, my wrappers do not make a "safe" copy of the data at construction time, and remain linked to the underlying collection.  You should not use the wrapped collection directly - all calls and modifications must be done through the the wrapper.  The "safe" copy of data is done only for iteration.

Recently added:
  UnmodifiableCollection.toString() implemented (useful in the unit test)
  COWWrapperTest  unit tests

What's the Key for the next "big" parallel programming language?

This has been a popular subject on Artima, e.g.here &  here.  Most of the discussion has revolved around syntax, simplicity, speed, etc.  IMO, this is interesting, but, ultimately, not important.


In order for a parallel programming language to take off and grab mindshare, it must be well integrated with low cost GPUs from ATI and NVIDIA.  Because, ultimately, to take advantage of parallel programming, one either has to have a million dollar compute cluster, or a $100 graphics card.  My bet is on cheap.


A pure, elegant language that doesn't readily talk to CUDA will lose in the marketplace to some hack language that does.  It will be Beta vs. VHS, or 68000  vs. 8088 revisited.  So Clojure, Scala, Fantom, Groovy enthusiasts, write some GPU libraries!

Java RMI is a PITA

My background is mainly in Java desktop applications, not "EE server stuff".  But I had an idea for a simple server app, and, having recently attended an excellent pair of meetups sponsored by the SF Java User's Group, I was psyched to write a server app.  The app involves the instrument software sending status messages to a central server.  So first, how to do that?

In the past, I've often done this by opening a ServerSocket on the server, listening to connection requests from the client on a known port, creating a short lived Socket, and streaming over the data.  It works and I know how to code it.  But it's not particularly robust, nor scalable, and it's "so 1980s".

Since the client (at least initially) is written in Java, I next thought to use RMI.  From the little I knew it was "simple".  The drawback, that it only worked for Java programs, wasn't a deal breaker.  Sun/Oracle provide some good documentation.

As you can see from their docs, basic coding is simple.  Define an interface for the service, implement it on the server, and call it from the client.  Since the messages I was passing were basic Strings, not any fancy business classes, no extra classes needed to be declared.  In my case, the interface was one method

public interface MyInterface extends Remote {

   public void setStatus(String instrumentID, String status) throws RemoteException;

}

and the implementation was also quite simple, storing the message in a HashMap.  Well, simple once I realized that the call needed to declare that it throws a RemoteException.  Forget that and it won't work.  Also, both server and client need a main() to do basic registration.  But there's much more.  The Devil is in the details.

Setting up the classes and .jar files to compile is a minor PITA.  Even in my very trivial case.  Mainly getting everything onto the right classpath.  As you can imagine from the tutorial, this could be complex for a complex RMI call, or if the classes depend on numerous other classes or jar files.

On the server, somebody must manually start the registry, rmiregistry.  This seems a needless step, because in main the server class calls

MyImpl theServer = new MyImpl();

Naming.rebind(NAME, myImpl);

Why can't the java.rmi.Naming check to see that rmiregistry is running?  There's probably some complex cases where you don't want this, or a security issue, but why not handle the simple case?  BTW, just like a ServerSocket, the client needs to know the host and port for the rmiregistry.  No big advantage to RMI here. (yes, once you have multiple RMI calls, it's simpler cause you need only assign one port, not many)

There's a possible classpath issue on the server, where you may have to set java.rmi.server.codebase. Why?  In my case the one interface that my client uses, MyInterface, has already been included and resolved in the code - it gets compiled!  Isn't the idea to code to an interface and try not to think about the implementations?  Sure, there's probably a good reason, but another hassle.

Much more tedious is the SecurityManager stuff.  In your main, you have to provide a SecurityManager. 

System.setSecurityManager(new RMISecurityManager());

 Why?  Why have security and force people to put in their own, especially when the code examples all grant AllPermissions?  Seems like saying "we have a lock, but you must replace it with an open door".  And, frankly, I never even got this to work.  Probably got some of the -Ds wrong in the runtime arguments.  Luckily, I found this RMI tutorial  where they override key security checks to do nothing.

System.setSecurityManager (new RMISecurityManager() {

public void checkConnect (String host, int port) {}
  public void checkConnect (String host, int port, Object context) {}
});

By doing this I got RMI to work.  But the security hassles seem designed to lead the user to provide no security, and, IMO, the whole RMI setup issue is a PITA.  Besides, RMI is so "2000".  If you Google "Java RMI sucks", you'll get a lot of hits.  This one was useful.

I next looked into REST, much more trendy and cutting edge, for which JavaEE 6 provides support.  It worked for me.  More in a later blog.

A pat on my back

Last year, I wrote a couple of blog posts showing how simple it is use an Object Oriented Database with ZK. The original and one followup.

In thanks, db4o has named me a "Most Valued Professional" for 2010. Thanks db40 for the nice recognition.  And thanks for the nice insulated coffee-mug!

Auto-Alignment of JXTable TableHeaders

As a followup of the previous post, here's example code to automatically align the headers of a JXTable with the data underneath, using the new fangled SwingX renderers.

import org.jdesktop.swingx.renderer.DefaultTableRenderer;

public class AutoAlignedHeaderRenderer extends DefaultTableRenderer {
  
  // constructors etc. omitted  ...

  @Override
  public Component getTableCellRendererComponent(JTable table, Object value, boolean selected, boolean focused, int row, int column) {
   
    TableCellRenderer dataRenderer = table.getCellRenderer(row, column);

    int align = SwingConstants.LEFT;  // default to left

    if (dataRenderer instanceof JLabel) { // backwards compatibility
      align = ((JLabel)dataRenderer).getHorizontalAlignment();
    }
    else if (dataRenderer instanceof AbstractRenderer) {
      AbstractRenderer ar = (AbstractRenderer)dataRenderer;      
      align = ar.getComponentProvider().getHorizontalAlignment();
    }
    
    componentController.setHorizontalAlignment(align);

    return super.getTableCellRendererComponent(table, value, selected, focused, row, column);

  }

}

Auto-Alignment of JTable TableHeaders

Most Java programmers are familiar with setting up various TableCellRenderers for the contents of JTables.  Often the JTable.getDefaultRenderer() defaults are fine, if you set the proper class for each column.  Else you create your own class, typically subclassing JLabel or DefaultTableCellRenderer.  Numeric data is typically right aligned/justified, perhaps with a fixed number of decimal points, so that the digits line up nicely.  Text, boolean, and graphical data are more often left or center aligned.  In all, this entails some work, but the time spent seems to be justified

Ideally, the table headers, which are usually just the names of each column, should also be aligned to match their data.  You can painstakingly set up a custom renderer for each TableColumn, via something like myTable.getColumn().setHeaderRenderer().  For the common case where it is just a text label, this is tedium and overkill.  One could write some utility method to help out.

When recently confronted with this problem, I had a brainstorm that worked easily and simply for my client's code.  I had already painstaking setup the TableCellRenderers for each column.  Instead of duplicating that effort, why not just ask them?  The code looks like this:

public Component getTableCellRendererComponent(JTable table, Object value,
boolean selected, boolean focused, int row, int column)
{
// get the existing JLabel used for a renderer, for example
  JLabel jl = (JLabel) super.getTableCellRendererComponent(table, value, selected, focused, row, column);

// get the corresponding data renderer
  TableCellRenderer dr = table.getCellRenderer(row, column);
  if (dr instanceof JLabel)
    jl.setHorizontalAlignment(((JLabel)dr).getHorizontalAlignment());
  else
    jl.setHorizontalAlignment(SwingConstants.LEFT);  // or whatever works for a default for you

... // more code removed, 
  return jl;
}

}


You might worry that all the sets of HorizontalAlignment are slow and will fire a lot of property change events, but, assuming you are using a subclass of DefaultTableCellRenderer, they won't.  And ain't modern fast CPUs great?

Anyway, hoping this will help one of your projects.  I'm less familiar with the newfangled SwingX Highlighters, but seems like this would be a good place for them

Fixing the ClassCastException in LayoutComparator.compare()

My client's code was sporadically afflicted with a ClassCastException.  As you can see from the stack trace below, there is no client code involved, just Swing (and swingx) code

java.lang.ClassCastException
at javax.swing.LayoutComparator.compare(LayoutComparator.java:61)
at java.util.Arrays.mergeSort(Arrays.java:1293)
at java.util.Arrays.mergeSort(Arrays.java:1282)
at java.util.Arrays.sort(Arrays.java:1210)
at java.util.Collections.sort(Collections.java:159)
at javax.swing.SortingFocusTraversalPolicy.enumerateAndSortCycle(SortingFocusTraversalPolicy.java:119)
at javax.swing.SortingFocusTraversalPolicy.getFirstComponent(SortingFocusTraversalPolicy.java:434)
at javax.swing.LayoutFocusTraversalPolicy.getFirstComponent(LayoutFocusTraversalPolicy.java:148
at javax.swing.SortingFocusTraversalPolicy.getDefaultComponent(SortingFocusTraversalPolicy.java:511)
at java.awt.FocusTraversalPolicy.getInitialComponent(FocusTraversalPolicy.java:152)
at java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:340)
at java.awt.Component.dispatchEventImpl(Component.java:4502)
at java.awt.Container.dispatchEventImpl(Container.java:2099)
at java.awt.Window.dispatchEventImpl(Window.java:2475)
at java.awt.Component.dispatchEvent(Component.java:4460)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at java.awt.SequencedEvent.dispatch(SequencedEvent.java:101)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

Google searches revealed that others have seen this error, and supposedly it is caused by Swing components being added or removed while not on the Event Dispatch Thread.  O.K., makes some sense.  But how can you determine which rogue component is causing the problem?  Simple-minded attempts to "hook-into" this stack trace to catch and view the cause are rebuffed by the fact that Component.dispatchEvent()  is final, and all of the dispatchEventImpl()s are package access, not protected.

The place you can access a FocusTraversalPolicy() is in your Container.  However, you cannot access the Comparator it uses, because getComparator() is protected.  So, you have to write your own TraversalPolicy.  The first part is easy.

public class BetterFocusTraversalPolicy extends LayoutFocusTraversalPolicy {

   public BetterFocusTraversalPolicy() {
      setComparator(new BetterLayoutComparator());
   }
}

So, what to use for the "better" Comparator?  I copied and pasted from javax.swing.LayoutComparator, changing a couple of lines.  Here is the original:

public int compare(Object o1, Object o2) {
... some code deleted ...
if (a == null) {
   // 'a' is not part of a Window hierarchy. Can't cope.
   throw new ClassCastException();
}

My first reaction is "why throw a ClassCastException?  (instead of say, an IllegalArgumentException.  But the deeper issue is that the code provides no information about the rogue Component that is not part of the hierarchy.  The proper behavior would be to put o1.toString() into the exception:, e.g.

public int compare(Object o1, Object o2) {
... some code deleted ...
if (a == null) {
   // 'a' is not part of a Window hierarchy. Can't cope.
   throw new ClassCastException(o1.toString());
}

Making this change quickly revealed the problem. One of the axes for a graph was Time, and it was subject to programmatic change in the data collection thread. Correcting this fixed the bug.
But it would sure be simpler if LayoutComparator provided the information in the first place.

Yet more JXTreeTable Knowledge - autoselecting a row after construction

This JXTreeTable knowledge - I'm just full of it.  :-)

My current client wants to read persisted data into a JXTreeTable, nothing special there, but then wants to automatically select the root node, which, in their application, would usually save the user a click, plus it enables all the cool gizmos for beginners to see.  My simple brute force approach was to read the data, set that into the model, and call

setRowSelectionInterval(0,0);

It wasn't working,  I could occasionally see the first row briefly highlight, but then it would go away, with no selection on the table, no cool enabled gizmos.  Drat.  After wasting time on futile efforts to set the selection later or more often, I eventually set breakpoints in setRowSelectionInterval() and clearSelection() to see who else was calling them.


The JXTreeTable constructor indirectly calls clearSelection().  It actually happens twice, through setModel() and setRowSorter().  But these happen during construction, well before my call, and are completely expected.

Somewhere after I read in the data, model.setRoot() gets called.  This triggers TreeTableMode.setColumnIdentifiers, which calls
TreeModelSupport.fireNewRoot(), which calls
TreeModelSupport.fireTreeStructureChanged(), which broadcasts to all the TreeModelListeners.

Other than sounding like all the "begats" in Genesis, this still makes sense.  One of the listeners is JXTreeTable's internal TreeModelListener, with a method, treeStructureChanged().  This method calls delayedFireTableStructureChanged().   

Now we are getting somewhere, that "delayed" should be a big hint. The code does the fairly obvious:

   SwingUtilities.invokeLater(new Runnable() {
      public void run() {
         fireTableStructureChanged();
      }
   });

Eventually, fireTableStructureChanged() will call JTable.clearSelectionAndLeadAnchor(), which, duh, calls clearSelection().

But, since this is called via invokeLater, it ends up happening after my call to select row 0.  Now, sharp purists would, at this time, point out that it was a mistake all along for me to select a row on a UI from a non-AWT thread.  But, but...  Anyway, the simple fix was, in my code, to also delay the selection.  After reading in the data and setting it into the model, instead of directly calling setRowSelectionInterval(0,0), wrap that call inside an invokeLater().  i.e.

SwingUtilities.invokeLater(new Runnable() {
   public void run() {
      myTable.setRowSelectionInterval(0,0);
   } 

});

Now it works.  So, if you are struggling with selecting a row after setting up a JXTreeTable, now you know the trick too.

Stupid JXTreeTable Tricks

For one project for one of my clients, I've been using the new SwingX JXTreeTable. It works. Not sure if it is all that much better than some of the earlier code you'll find on the web, but it basically works. And I never used tree tables much before, so, being "new" is not an issue.  Now, it's a bit offputting to look at the source code and see two data model adapter classes and four TreeTableHacker classes, most of which have comments like:

     * Temporary class to have all the hacking at one place. Naturally, it will
     * change a lot. The base class has the "stable" behaviour as of around
     * jun2006 (before starting the fix for 332-swingx).

or

     * Note: currently this class looks a bit funny

or, my favorite

     * A more (or less, depending in pov :-) aggressiv hacker.

Anyway, I found a few weird bugs or features that cost me some hours and were easily fixed.

1)  JXTreeTable ignores your attempts to resize the columns.  You can resize them, but as soon as any of the table data changes, it takes over like a bad episode of Twilight Zone and ignores your settings.  After an hour of debugging to convince myself that my code to resize the columns was really working, and trying every possible combination of  setMinWidth(), setWidth, and/or setPreferredWidth(), a little Googleing found this thread.  When changing the content of the table, JXTreetable always recreates the columns, even in the most common case where the number of columns hasn't changed.  Silly.  Anyway, you can complain about their design. or you can simply add this line of code somewhere:

setAutoCreateColumnsFromModel(false);

2) JXTreeTable messes with the colors of your Renderers.  If you are used to "traditional" JTable Renderers, you are in for a surprise as JXTreetable will, by default, apply one of their newfangled Highlighters to your renderer.  (For the record, Highlighters look like a good idea, but forcing them on you isn't)  Here is part of a comment from JXTable source:

     * Client code which solves the problem at the core (that is in a
     * well-behaved DefaultTableCellRenderer) can disable the hack
     * by removing the client property or by subclassing and override this
     * to do nothing.

So, lets review.  Well-behaved code must go out of it's way to avoid getting broken, while poorly written code is automatically "fixed".  Seems bass-ackwards to me.  I guess I have to start writing lousy, ill-behaved renderers.  :-)

Anyway, you can get around this issue by adding this line to your code:

putClientProperty(USE_DTCR_COLORMEMORY_HACK, null);

(the "null" could also be Boolean.FALSE as well).

Hope this helps and save you some time.