Apache NetBeans Wiki Index

Note: These pages are being reviewed.

Folder ordering using numeric sort.


merged to trunk (Jun 16 2007) changeset Issue #103187 Current SFS with positions


Previous state

The Filesystems API specifies no order in folder children (FileObject.getChildren). But the Datasystems API does assign a meaning to the order of DataFolder.getChildren. Currently there are four ways a folder may get an order:

  1. If the folder attribute OpenIDE-Folder-Order is specified, it is used. (Any children not mentioned are put at the end.) The format is a/b/c where a etc. are file or subfolder names. DataFolder.setOrder sets this attribute.

    1. If the folder attribute OpenIDE-Folder-SortMode is specified, it is used. The format is a single character, e.g. M for sort by modification time. This method is hardly used any more, although the sort mode attribute is still shown in the property sheet for folder nodes.

    2. If there are some relative ordering attributes, the folder is topologically sorted. The attribute a/b, if set to the value Boolean.TRUE, means that a should come somewhere before b in the folder (not necessarily immediately before).

    3. Otherwise, the fallback order is alphabetical (same as OpenIDE-Folder-SortMode=F).

although these methods can apply to any folder, in practice only folders on the system filesystem, i.e. defined in XML layers, are explicitly ordered.

Problems with current state

The third way is the only one suitable for a modular system. It has been in use since NetBeans 3.1. Unfortunately it suffers from several drawbacks:

  1. New developers are confused by ordering attributes. For example, it is common to assume that a/b means a will immediate precede b, which is not the case. It is also common to assume that a/b=false is equivalent to b/a=true, which is not the case (it has no effect).

    1. Typing relative ordering attributes in an XML layer is slow and results in bloat since most filenames have to be written three times:

<file name="a"/>
<attr name="a/b" boolvalue="true"/>
<file name="b"/>
<attr name="b/c" boolvalue="true"/>
<file name="c"/>
<attr name="c/d" boolvalue="true"/>
<file name="d"/>
  1. When many unrelated modules contribute to a folder, it is often not clear which modules are more fundamental. (The convention is to have more optional modules declare order relative to more fundamental modules, so the fundamental modules need not know about the optional modules.)

To ensure that the folder order will hold up even when some modules are disabled, it is common to overspecify ordering, which can lead to a near-quadratic number of attrs; when this is not done, it is common for folders to fall into haphazard order as the set of modules changes in unexpected ways.

  1. Attributes often need to be edited to accommodate renames or other changes of unrelated files.

  2. Contradictory orderings lead to `TopologicalSortException`s, which are unfriendly and difficult to debug.

  3. The module development support has a difficult time writing out ordering attributes when the developer uses drag-and-drop to reorder files in an XML layer.


To address these problems, the solution is to add a fifth means of ordering a folder. Every file in the folder could have a Number-valued attribute position. Files would then be sorted (in increasing order) by position. This would also be consistent with Lookups.metaInfServices.

For example:

<file name="a"><attr name="position" intvalue="100"/></file>
<file name="b"><attr name="position" intvalue="200"/></file>
<file name="c"><attr name="position" intvalue="300"/></file>
<file name="d"><attr name="position" intvalue="400"/></file>


Ordering semantics

Normally positive integers would be used for positions, but floats or negative integers could be used for emergencies in case an item needed to be inserted between two adjacent integers.

Files with no marked position would be placed at the end (and a brief warning logged). Files with the same position would be ordered alphabetically (and a brief warning logged). As an exception, you may mark files with the position 0 to indicate that their position is irrelevant; no warning is logged if several such files exist in the same folder.

Numeric and relative ordering can coexist, for backwards compatibility.

Uses of relative ordering attributes should be logged as warnings to assist in migration.

Setting order

DataFolder.setOrder should remove any of the old ordering methods in effect and set positions on each file in the folder. The tricky part is to avoid changing the positions of files which already have positions unless necessary to accommodate the new order.

For example, given an initial folder content:

a (#100)
b (#200)
c (#300)

and asked to set the order to d a c b, it would be best to change only two attributes, e.g.:

d (#0)
a (#100)
c (#300)
b (#400)

or something similar.

I.e. first need to compute a minimal set of transpositions.

decompose permutation graph into disjoint cycles.

Then use some heuristics to decide which of a pair in a transposition to "move", and what its new position should be. Heuristics could include:

  1. Prefer to add a position to a file which lacks one than to change an existing position.

    1. Prefer round numbers like 100 to numbers like 123, and prefer integers to floats.

    2. Prefer to move a locally modified file to an untouched one. (Can be implemented by looking at position of a `MultiFileObject’s leader filesystem.)

Java APIs

Introduce methods to order a folder in FileUtil in the Filesystems API, and to set a new order. This avoids duplicating somewhat subtle code. Since callers might be ignoring some files in a folder (e.g. *.form) the list of files to consider needs to be passed as well.

class FileUtil {
  public static List<FileObject> getOrder(Collection<FileObject> children, boolean logWarnings);
  /** @postcondition children = getOrder(children, false) */
  public static void setOrder(List<FileObject> children) throws IOException;
  public static boolean affectsOrder(FileAttributeEvent event);
  // ...

(used in: core/startup, core/windows, editor/mimelookup/impl, openide/loaders)


For menu folders and other places where there is a distinguished null value or other separator, it is generally permitted to have extra separators. (Leading, trailing, or adjacent duplicate separators are ignored.) With numeric ordering, a simple convention could help group items into separated blocks.

For example:

<file name="sep500"><attr name="position" intvalue="500"/></file>
<file name="cut"><attr name="position" intvalue="600"/></file>
<file name="copy"><attr name="position" intvalue="700"/></file>
<file name="paste"><attr name="position" intvalue="800"/></file>
<file name="delete"><attr name="position" intvalue="900"/></file>
<file name="sep1000"><attr name="position" intvalue="1000"/></file>
<file name="undo"><attr name="position" intvalue="1200"/></file>
<file name="redo"><attr name="position" intvalue="1300"/></file>
<file name="sep1500"><attr name="position" intvalue="1500"/></file>

will display as:


but it is easy to add new items at the top, bottom, or middle of any block; add new blocks at any position; divide existing blocks; etc.

The editor folders under SideBar were using a position attribute for a different purpose. These have been converted (compatibly) to use location instead.

Changed code

Scope is "Big IDE" with all clusters (incl. CND and Profiler).

Readers of relative ordering attrs

  • FolderOrder (openide/loaders; also DataFolder and FolderList) This is the canonical reader of ordering attributes.

  • CompoundFolderChildren (editor/mimelookup/impl; also FolderChildren) Does its own reading to order the result of merging together several folders. Víťa agrees it could probably be changed to use MultiFileSystem instead, or could use any new sorting API (if it supported parallel folders), or could directly implement sorting by position.

  • OptionUtilities (editor) Víťa says it is semi-obsolete, but similar to CompoundFolderChildren.

  • ModeParser (core/windows) Prefers to operate at Filesystems API level, for efficiency and predictability. (Using DataFolder is slower and introduces asynchronous behavior.)

  • RecognizeInstanceFiles (core/startup) Cannot refer to Datasystems API.

Writers of relative ordering attrs

  • LanguagesManager (languages/engine) (needs tuning)

  • MenuFolderNode and ToolbarFolderNode (core/windows)

  • CreatedModifiedFiles (apisupport/project; also ui.wizard.action.DataModel)

  • WritableXMLFileSystem (apisupport/project)

  • LanguageRegistry (scripting/gsf) (needs tuning)

  • MidpPaletteProvider (mobility/designer2/midp) (done though untested)

Uses of relative ordering attrs

Fixed in bulk mode using apisupport/relative2position.

Commit validation

ValidateLayerConsistencyTest (in core) should verify that:

  1. No relative ordering attributes are in use on any folder.

    1. Neither OpenIDE-Folder-Order nor OpenIDE-Folder-SortMode are used.

    2. Any position attribute has a numeric value.

    3. If any file (or subfolder) in a folder has a position attribute, then they all do; and all the values are distinct.

Implemented. Run not only in the trunk Hudson project (i.e. full IDE), but also in nbms-and-javadoc (to check experimental modules).

API Docs Updated

Misc. impl still left to do

  • Fix up LanguagesManager and LanguageRegistry. (some fixes done already; remainder probably best left to domain developers)

  • Change FileUtil.setOrder to be more conservative: avoid changing existing position attributes if possible. ''(in progress; cf. issue #110981)''

  • Fix up various ordering attrs which are not quite right. Especially files which claim to be ordered in folders which do not care. Also Editors/text/+xml/* (e.g. Ant context menu) are generally not right. (generally will be left to whoever handles UI spec conformance bugs)

  • nbbuild/build.xml#index-layer-paths ought to order files. (done)

  • Need some way of marking a file as not intended to be ordered. E.g. position="0" or position="none". Useful for e.g. hidden subfolders. getOrder can put these wherever it likes but should never warn about them. Issue #107550 (done)

  • Clean up experimental modules. Current errors (done)