Visualisation clients

# Chapter 3 Visualisation clients

- Control
- Viewlets
- Viewers
- Scenarios

To visualise the viewables of an annotated program, the library lib(java_vc) provides a Java based graphical visualisation client.

A new Java visualisation client (Java VC) can be started from the tools menu of tkECLiPSe or using the predicate start_vc/1 in lib(java_vc). The single argument will return a unique name for the created client which can be used to close the client if required. While the Java VC is running, it will react automatically to the creation of viewables during ECLiPSe execution, but it cannot visualise viewables which were created before the Java VC was running.

> Figure 3.1: The initial Java VC screen before any viewables have been created.

## 3.1 Control

When running a visualisation-annotated ECLiPSe program with a Java VC attached, control of the ECLiPSe process may pass between ECLiPSe and the VC throughout the program run. That is to say at certain key events in the program, ECLiPSe will pause in its running of the program and wait for user interaction with the VC before continuing. In such circumstances, the VC is said to hold the control.

Table 3.1 details the default behaviour for each of the visualisation events which may occur, and indicates whether or not this default behaviour can be altered.

> Event | Triggered by | Default hold | Alterable
> viewable creation | viewable_create/2 viewable_create/3 viewable_create/4 | yes | no
> viewable expansion | viewable_expand/3 viewable_expand/4 | no | yes
> viewable contraction | Backtracked over a viewable expansion | no | yes
> viewable destruction | Backtracked over a viewable creation | yes | yes
> forward update | One or more elements in a viewable have been updated, ie. had their domain reduced or have been instantiated | no | yes
> backward update | A forward update has been backtracked over | no | yes

> Table 3.1: VC default behaviour for visualisation event.

Should the VC hold, control can be passed back to ECLiPSe by pressing the Resume button at the bottom of the VC window, or by setting the auto resume timer. The Resume button and the auto resume timer are disabled when ECLiPSe has control, see Figure 3.2.

> Figure 3.2: The VC showing the auto resume menu option and timer slider.

## 3.2 Viewlets

The Java VC provides many ways of visualising any single element of a viewable.

1. Textually, as though the element had been printed with write/1. This is suitable for all viewable types.
2. As a rectangular bar on a scale representing the current bounds of a numeric_bounds type viewable element. Bounds viewlets can be aligned either vertically or horizontally.
3. As a node in a graph, similar to the simple textual representation but enclosed in a geometric shaped node.
4. As an edge in a graph, with the textual representation attached as a label to the edge.
5. With a colour which varies in shade and hue in response to events occurring on the variable.

When rendered on the screen these representations are referred to as viewlets. Figure 3.3 shows the same variable rendered using a number of viewlet types.

> Figure 3.3: The FD variable with initial domain 0..10, reduced to 3..5 as rendered by text, bound, node and fade viewlets.

## 3.3 Viewers

The Java VC currently contains five different methods for rendering an entire viewable. Each of these methods can be thought of as a window looking onto the viewable and is referred to as a viewer.

Upon a viewable being created, the user is presented with a dialog box asking which of the available viewers they wish to view the viewable with.

The currently available viewers are

TextTable
  Renders any type of 1D and 2D viewables as a grid of textual descriptions of the elements.
BoundsTable
  Renders numeric_bounds 1D and 2D viewables as a grid of rectangles representing the size of the numeric domains.
FadeTable
  Renders 1D and 2D viewables as a grid of coloured rectangles whose colour changes represent domain changes in the viewable elements.
Desktop
  Allows the user to place all available representations of the viewable elements anywhere on a desktop window. Also enables the loading of an arbitrary background image from file, and for placing images alongside viewlets.
Network
  Renders graph(fixed) viewables graphically as connected nodes, where the textual representation of the viewable elements is displayed at nodes and along edges.
Network (0/1)
  Similar to the Network viewer except that if the edge annotation can be interpreted as the number 0, then the edge is not drawn. If it can be interpreted as the number 1, it is drawn in black. Any other value has the edge draw in gray.
Network (Capacity)
  Similar to the Network viewer except that the edge labels are interpreted as fractions indicating the capacity of a link in a flow network. 0.0 indicating unused (thin black line) up to 1.0 indicating full usage (thick black line) and any number greater than one indicating over utilisation (very think red line). If the edge data cannot be interpreted as a number (eg. it is a variable) it is assumed to be 0.
Gantt
  Interprets the first three rows of any 2D viewable with numeric_bounds elements (and at least 3 rows) as being the start times, durations and resource requirements of a scheduling problem. The resulting schedule/partial schedule is rendered as a gantt chart.
Bar chart
  Renders any n-dimensional numeric_bounds viewable as a bar chart. Extra dimensions will be separated by gaps in the chart.

> Figure 3.4: The VC showing some of the applicable viewers for the SEND+MORE=MONEY example.

> Figure 3.5: The VC showing the network viewer displaying the graph example.

> Figure 3.6: The VC showing various viewers for the changeable solver example.

Common to all viewers are the three menus Options, Select and View, the latter two also being accessible by pressing the right mouse button.

### 3.3.1 Options menu

The options menu contains controls for viewer-wide properties.

Hold at expansions
  Determines whether this viewer will hold control when the viewable is expanded.
Hold at contractions
  Determines whether this viewer will hold control when the viewable is contracted.
Hold at destruction
  Determines whether this viewer will hold control when the viewable is destroyed. This option is useful for examining the state of the viewable immediately before the creation is backtracked over.
View propagation steps
  Controls how frequently the visualisation client is informed of forward update events.
  fine
    Events are sent as soon as they occur.
  coarse
    Events are sent at priority 8 in the ECLiPSe program. Typically this means that all the propagation that occurs as a result of a single user level search step are sent together.
  timed
    Events are collected and sent at regular timed intervals.
Track updates
  When set, the viewer will attempt to ensure that all updates are visible within the window. This can be important when visualising large viewables which may not easily fit the window.

Figure 3.7 shows the default settings for the Options menu. Note that the View propagation steps options are disabled because ECLiPSe has control and the update granularity can only be changed when the Java VC is holding control.

> Figure 3.7: The options menu, common to all viewers.

### 3.3.2 Select menu

Contains convenience commands for dealing with the currently selected set of viewlets.

Selecting individual viewlets can be done clicking on them with the left mouse button, whilst selecting ranges can be done by dragging the mouse across a range of viewlets.

Select all viewlets
  Sets the selection to the entire viewable.
Select updating viewlets(s)
  Sets the selection to only those viewlets which have been marked as updating (either forward or backward). This option is only enabled when the Java VC has control, since it requires the state of the viewables to remain constant during the selection process.
Clear selection
  Clears the selection.

> Figure 3.8: The select menu, common to all viewers.

### 3.3.3 View menu

So as to facilitate visualisation of large viewables, all viewers have the ability to zoom in and out. All the options are self explanatory and will not be expanded further upon except to mention that the Zoom to fit width and Zoom to fit height options operate on the whole viewer and not just the selected viewlets.

> Figure 3.9: The view menu, common to all viewers.

Both the network and desktop viewers have an extra item on the view menu, Toggle high quality. This toggles between quick rendering and high quality views, and may help to make the VC more reactive under high load.

### 3.3.4 Viewlet actions

Within a viewer, as previously mentioned, any number of viewlets may be selected. These viewlets, once selected can have actions performed on them. The actions are selected by pressing the right mouse button in order to bring up the context sensitive actions menu. If the viewlets in the selection are of different types then all the available actions are displayed and once one has been selected, it will be applied to all applicable viewlets in the selection. This is a change from previous versions of the visualisation client, which would display only those actions common to all viewlets.

#### Hold on update

The most common action, which can be performed on any type of viewlet is the Hold on updates action, which, when set, indicates that the Java VC should hold control whenever any sort of update event is issued for the corresponding viewable element. The Hold on updates property of a viewlet is indicated by a slight “greying” out of the viewlet, or in the case of viewlets attached to edges in the network viewer, the edge is drawn “dotted” instead of solid.

Figure 3.10 shows the graphical effect of setting the Hold on update property of a text viewlet.

> Figure 3.10: The sequence of actions required to select Hold on update for a viewlet

Table 3.2 lists the available viewlet actions and indicates for which type the actions are valid.

> Name | Description | Applicable
> Hold on updates | Causes the VC to hold control on forward or backward update events for the selected viewlets. | all
> Fade update history | Toggles using the background color of the viewlet to indicate recent update history. This has the effect of fading from green to white in the event of a forward update and from red to white for backward updates. | text, node, fade, edge
> View bounds in detail | Pops up a window detailing the original bounds and the current bounds for the single selected viewlet. | bound
> Align bounds | Causes the selected viewlets to use the same underlying scale when displaying the bounds. This allows variables whose initial bounds were different to be visually compared. | bound
> Toggle horizontal/vertical range bar | Toggles the rotation of the bar for all bounds viewlets | bound

> Table 3.2: The available viewlet actions and associated types.

### 3.3.5 Desktop/Network viewers

All the table viewers have essentially the same functionality – they do not allow flexible placement of viewables and both deal only with 1 or 2 dimensional viewables. A more flexible viewer is provided in the Desktop viewer.

This viewer aims to implement the common desktop metaphor by providing the user with a rectangular region of the screen upon which viewlets can be dropped, stacked and moved around as though they were pieces of paper on a desk.

#### Adding viewlets

Typically, viewlets will be added to a desktop immediately after the viewer has been created. To minimise the overhead of having to layout the viewlets each time the user’s program is run (a potentially time consuming task), the Java VC provides an automatic recording and repeat mechanism which is triggered every time a viewer is created. Section 3.4 explains this feature in more detail.

Adding viewlets to a Desktop viewer is done by selecting the required viewlet type from the Insert menu. This menu will contain only those viewlet types which are appropriate for the type of the viewable.

Once an appropriate viewlet type has been selected, the range selection dialog will pop up, from which any combination of dimension ranges may be selected.

Figure 3.11 shows the range select dialog for the on going SEND+MORE=MONEY example.

> Figure 3.11: The range selection dialog for the SEND+MORE=MONEY example

At least one selection must be made from each of the dimensions, though it is possible to select multiple values in each dimension.

Figures 3.12 and 3.13 illustrate the default layout of viewlets when 1 and 2 dimensional ranges are selected. The desktop will automatically resize to ensure that all viewlets fit. Attempts to move a viewlet off the desktop will cause it to grow.

> Figure 3.12: The result of selecting a 1D range

> Figure 3.13: The result of selecting a 2D range

Higher dimension range selections result in a stacked 2D grid, with progressive dimensions appearing underneath the initially visible grid.

### 3.3.6 Adding images

As well as viewlets, the Desktop viewer can show icons loaded from disk by selecting the Image option from the Insert menu. This brings up a file selection dialog from which the user may select an image file to load. The loaded image will be added to the viewer as a small icon which is selectable and movable like other items on the desktop. Currently there is no way to increase the size of the loaded image.

#### Background images

In keeping with the computer GUI desktop metaphor, the user may set the background image for the desktop viewer. Aside from making the viewer look pretty this feature is intended to allow graphical context to be associated with the visualisation of a program. For example the background image could be a diagram representing the network topology and the values being visualised could be the flows through various parts of the network. By placing the viewlets near the appropriate nodes on the background image the user could more easily visualise the network flow problem.

Background images are loaded by selecting the Import background image option from the Background menu and are removed by selecting the Clear background option. Currently only GIF, PNG and JPEG format images can be loaded.

In keeping with our SEND+MORE=MONEY example, figure 3.14 shows the problem visualised on a desktop viewer, placed over a background image1.

> Figure 3.14: The SEND+MORE=MONEY example displayed on a Desktop viewer with a background image

### 3.3.7 Layout

Items on the desktop may be manually positioned by selecting (single click) and dragging (click-and-move) them. New items may be added to the current selection by holding down the Ctrl key whilst clicking with the left mouse button. Ranges of items are selected by clicking on the background of the desktop and dragging a selection rectangle around the desired items. When dragging a selection all items move, except lines on the Network viewer.

It is also possible to use one of the automatic layout options available from the Graph menu. These options make use of the external graph layout tools dot, neato and twopi from the AT&T Labs Research project Graphviz2. These tools should be automatically installed as part of the ECLiPSe installation procedure.

### 3.3.8 Gantt charts

The Gantt chart viewer has many of the same options as the Network viewer previously mentioned but in addition, the Gantt menu provides access to options that control how transparent the individual gantt task bars are drawn. By selecting the transparent option, regions where tasks overlap will be rendered in a darker colour. The darker the colour, the more tasks overlap there.

When either the start time or the duration of a task is a variable, then the task will be draw as two connected bars indicating the earliest & shortest possible occurrence of the task and the latest & longest possible occurrence.

Above the gantt chart is a numeric scale indicating time. By clicking and dragging this scale can be expanded or shrunk so as to fit the gantt chart into the window. This feature works independently of the zoom.

> Figure 3.15: The VC showing the Gantt viewer for a scheduling example. Note the highlighted task showing the earliest start/shortest and latest/longest times of the task.

### 3.3.9 Printing

To print the current state of almost all viewers, right-click on the background and select the Print option from the popup menu. This will bring up the print dialog as shown in figure 3.16.

> Figure 3.16: The print options dialog box.

## 3.4 Scenarios

To make the process of setting up the visualisation environment and the laying out of viewers and viewlets quicker, the Java VC provides a record and playback feature where all user visible changes and actions that are performed following the creation of a viewable are recorded in a visualisation scenario. This action sequence can then be optionally re-played the next time a viewable of the same name is created.

The common use case is as follows.

1. Start Java VC.
2. Run program which creates viewable “foo” for the first time.
3. Select viewers for “foo”.
4. Arrange viewer windows on screen, resize and scale to taste. Optionally insert and layout viewlets on a Desktop viewer.
5. Press Resume button to continue running program.
6. Watch visualisation of program run until viewable is destroyed (ie. is backtracked over).
7. Re-run program, after having made some changes.
8. Answer yes to the prompt to reinstate visualisation preferences for viewable “foo”.
9. Watch as things magically re-arrange themselves into the configuration you previously had.
10. (optional) Make some more layout changes.
11. Press Resume again.
12. Repeat.

To make long running visualisation projects easier and also to assist in running demonstrations, these visualisation preferences can be saved to disk and loaded back into memory at any time. The loading and saving of scenarios is achieved by using the Load and Save options of the File menu. The most common point at which a scenario is saved is just after laying out all the viewers and just before passing control back to ECLiPSe. It should be noted that the scenarios (settings) for many different viewables can be saved into/loaded from a single file, this is to aid visualisation of large programs.

1
  Background image ©1999-2003 www.barrysclipart.com
2
  http://www.research.att.com/sw/tools/graphviz/