moodss

(Modular Object Oriented Dynamic SpreadSheet)

Contents

About this document

This document contains general information, reference information and examples designed to help the user understand the moodss application and the programmer write modules for it.

Introduction

Quite often, one needs to monitor changing data, whether it might come from a system, such as the different processes running on a Unix server, or from a network, such as the kind and volume of traffic that runs through it.

Most often, such data can be organized in a table with rows of information, each column representing a different type of data. For example, in the case of processes running on a system, rows might be sorted according to their unique process identifier, and columns might represent such values as CPU usage, memory usage, owner's name, time of creation, ...

The software used to view this type of information comes in different forms and shapes. Unix users might be familiar with the "top" application which presents rows of process data as lines of text, whereas RMON (Remote MONitoring) SNMP software usually uses multiple windows with graphical displays, curves, pie charts, multiple configuration dialog boxes, even 3D visualization modules to present network traffic, connection matrices, ...

In most cases, data comes from one or several tables. A common interface, graphical with menus, drag'n'drop capability, a table widget and graphical data viewers such as multiple line graphs, bar and pie charts, could be used. The user could then sort the table rows, select one or more cells, rows, columns, to launch viewers such as other tables, charts, ... best suited to the way data should be presented. In effect, what is needed is a spreadsheet that is capable of dealing with dynamically changing data.

Moodss (Modular Object Oriented Dynamic SpreadSheet) is an attempt at answering these needs. It is composed of a main part (the core) and an unlimited number of modules, each module interfacing to a specific type of data. The core is written in the great Tcl language (http://sunscript.sun.com/ or http://www.scriptics.com/ ) using object oriented techniques thanks to the stooop and scwoop packages (http://www.mygale.org/~jfontain/). The module function is to describe the type of data that it is also in charge of retrieving and formatting when asked by the core. Modules can be written in plain Tcl and optionally use dynamically linked library written in the C language (modules are packages in the Tcl sense).

As features are added to moodss, different ways of viewing data will be made available while the modules will stay the same. The goal of moodss is to become a nice feature packed generic way of viewing data. Moodss can be used to monitor any type of data, since the simplest cases can fit in a table with a single row.

As moodss is written in Tcl and uses well supported extensions (Tktable and BLT), it will run on most Tcl/Tk supported platforms: UNIX and Windows (I do not know if Tktable and BLT are available for the MacIntosh). Some modules may be specific to a platform, but the core is guaranteed to run on them all.

After reading and understanding this document, you should be able to write your own modules in order to monitor the data that you are interested in.

Moodss is free software. You can redistribute it and/or modify it under the terms of a BSD type license: please see the COPYRIGHT file or use the main window About help menu for more information.

Required software

For the current version (2.1), the following packages must be installed before attempting to install moodss (make sure to check the INSTALL file for the latest information):
* many thanks to the authors for these great packages
 
The stooop library and required scwoop and pie widgets are included in the self-contained moodss file. Therefore, it is not required to install the stooop, scwoop and tkpiechart packages, unless you want to work on the moodss source code itself. However, should you want to get more information on those extensions, you could find the latest versions at:

Architecture

The moodss application is composed of the core software and one or several modules. Modules are implemented as Tcl packages and thus usually comprise a Tcl source file and a pkgIndex.tcl file required by the Tcl package implementation.

The core will load the module whose name was passed as a command line parameter and will start displaying the module data in a table. The table is then updated at the frequency defined by the poll time, which the user may change. For example, to launch moodss with the random module, just type:

$ wish moodss random
All the module code and data is kept in a separate namespace. The module data is stored is a single array including some configuration data used when the module is loaded by the core, and variable data (displayed in the application table and eventual graphical viewers) which the module must update when requested by the core.

The original data table represents the first data view, from which cells can be selected and when dropped through a drag'n'drop operation into a graph, bar chart or pie chart iconic site, result in the creation of graphical data viewers. In turn, these viewers can display more table cells, which when dropped into the graphical viewer, result in the creation of corresponding data graph lines, bars, or slices.

All data viewers can be moved and resized at will through a simple internal window manager.

Core

User interface

Soon after the application launch, tabular data is displayed in a tkTable widget with automatic scroll bars, between the menu bar, the drop sites with graphical viewers and trash icons and a message area, as one can see right below:
 
moodss initial main window view

The message area is used to display status information, such as when the data is being updated, and help information, as the user moves the mouse pointer over sensitive areas, such as table column headers. Further help is provided through widget tips (also known as balloons) when appropriate, and of course the Help menu.

The window title shows the name of the current module along with the poll time.

The File menu only contains the exit menu entry, used to gracefully quit the moodss application.

The Options menu contains the Poll time entry which when selected launches the corresponding dialog box, as shown below:
 

poll time dialog box

The user can select a new poll time among the module choices from a spin entry widget, or directly type in a new value, as long as it is not smaller than the module minimum poll time, in which case a message dialog box warns the user.

Table data can be sorted at any time by simply clicking on a column title. Clicking on the same column title again sorts the data in opposite order, thus toggling between increasing and decreasing orders.
When sorting, the selected column is used as a reference, meaning that all rows will be rearranged so that the selected column appears sorted, with values either increasing or decreasing.
The reference column for sorting always appear in a darker shade of gray.

Aside from the main table, graphical viewers can be created for viewing table cell data behavior over time. For example, to create a cell graphical plot, one must place the mouse pointer over a data cell in the main table, hold down the first mouse button (the left one for a right handed user) while dragging over to the left-most icon below the menu bar (when dragging an object, as the mouse pointer passes over possible drop sites, they are highlighted with a thin black surrounding border for user feedback). Releasing the mouse button at this time results in the creation of a BLT graph viewer (such as can be seen below). The create procedure is identical for other types of viewers: side-by-side bars charts, stacked bars charts, 2D pie charts and 3D pie charts*.

*note: if you know of any other nice viewers (like 3D graphs) that work with Tcl, please let me know so I can integrate them. Many thanks in advance...

Once a viewer exists, it also acts as a drop site for data cells. Dropping a cell directly in the viewer results in a corresponding line, bar or slice being created and automatically updated. Each new graphical element is assigned a new and different color.

All data viewers can be moved and resized thanks to handling areas in the data viewer borders. When moving the mouse pointer over these areas, the mouse cursor changes to indicate the available action. Corner handles allow resizing in both x and y axis. Handles in the middle of the sides allow resizing in either the x or y axis direction. All other areas can be used for moving the data viewer. Clicking on any part of the border changes the stacking order: the viewer being clicked on either goes below (eventually becomes hidden) the other viewers, or becomes fully visible (on top, eventualy hiding other viewers). Further description of this small window manager functionality is useless, as it behaves like a standard window manager (let me know if it does not).

You may delete one or more graph lines from a graph viewer by selecting them through their labels in the upper-right side of the graph viewer (several elements can be selected by depressing the control key as the first mouse button is pressed). Then dragging from the viewer to the trash drop site (the bullet hole) on the upper-right side of the main window and releasing the first mouse button result in the corresponding graph lines to be destroyed. If there are no remaining lines, the graph itself is destroyed.

The bar charts viewers behave identically.

The pie charts viewers can only be deleted as a whole at this time, since slice elements cannot be selectively selected.

 

moodss window with graph data viewers

The Help menu contains Module help, and About general information entries.

Command line

Launching moodss is very simple: just pass the data module name as parameter, as in:
$ wish moodss random
You may eventually specify a poll time in seconds using:
$ wish moodss -update 25 random

Modules

All examples are drawn from the random sample module.

source

Since a module is a package, it must have a name in the Tcl sense. For example, at the beginning of the random.tcl file, one can find the following statement:
package provide random 1.0
This line simply states that this is the 1.0 version of the random package. Please note that the package name must also be used as the module namespace name (see below).
namespace
All module procedures and data are kept in a specific namespace bearing the module name. For example (from the random.tcl source file):
namespace eval random {
    array set data {
        ...
    }
    proc update {} {
        ...
    }
}
configuration
The module configuration defines the data table column headers, help information, ... This data never changes during the lifetime of the application.

All the module configuration data is stored as array members of the array named data within the module namespace. For example:

namespace eval random {
    array set data {
        updates 0
        0,label name 0,type ascii 0,message {user name}
        1,label cpu 1,type real 1,message {cpu usage in percent}
        2,label disk 2,type integer 2,message {disk usage in megabytes}
        3,label command 3,type dictionary 3,message {most time consuming command}
        pollTimes {10 5 20 30 60 120 300}
        columns 4
        sort {1 decreasing}
        helpText {
This is a simple demonstration module ...
        }
    }
    ...
}
The updates member is a counter used to keep track of the number of times that the module data was updated, and is also used by the core to detect when module data display should be updated (see variable data for more information).

The columns member defines the number of columns in the table to be displayed by the core.

The n,label members define the text to be displayed as column titles. There must be as many n,label members as they are columns.
The n,type members define the type of the corresponding column data. Valid types are simply those that the Tcl lsort command can handle: ascii, dictionary, integer and real. There must be as many n,type members as they are columns.
The n,message members define the text of the help message to be displayed in the message area (see User Interface) as the user moves the mouse pointer over column titles. It should be composed of only a few words, just enough to actually help the user understand what the column data means. There must be as many n,message members as they are columns.
Note that column numbers start at 0.

The pollTimes member is a list of valid poll times (in seconds) for the module. The list need not be ordered, as its first element represents the default poll time value to be used when the moodss application starts. This value may be overridden by a command line argument. The smallest value in the list is used by the core as the lowest possible poll time and checked against when the user enters a new value through the poll time dialog box. The list must not be empty.
Note that the list is also used by moodss as a set of possible choices in the dialog box used to set the new poll time. The user may still directly input any value as long as it is greater than or equal to the minimum value.

The sort list defines the index of the column which should be initially used as a reference for sorting the data table rows, and in which order (increasing or decreasing) the rows should be sorted. The column index for sorting works like the -index Tcl lsort command option, that is rows are sorted so that that specific column appears sorted in the specified order.

The helpText member specifies a text of any length, to be displayed when the user requests help information on the current module from within the help menu.

variable data
The tabular data (variable data) that the module code must update is stored in the same data array as the module configuration data.

The core invokes the module update procedure (which obviously must exist) when it is time to refresh the data display (table and eventually graphical viewers). At this time, the update procedure may update the tabular data straight away (synchronous operation) or launch a request for later data update (asynchronous operation).

It actually does not matter when the data is updated. The core will know that fresh data is available when the updates array member is set (actually incremented as it also serves as a counter for the number of updates so far).
It is the module programmer's responsibility to increment this counter right after all tabular data has been updated.

For example, retrieving information for the processes running on a machine is a local operation that can be achieved in a reasonably small amount of time. In such a case, data would be updated immediately and the updates variable incremented at the same time.
But if the data has to be retrieved from across a network, waiting for it to come back would cause a delay that the user would certainly notice, as the application would not respond to mouse or keyboard input during the whole time that it would take to fetch the whole data. In such cases, it is easier to let the update procedure return immediately without setting the updates variable, which would be incremented at a later time, only when the data would become available. For example, when waiting for data to come across a network connection, the Tcl fileevent command could be used on a non blocking channel, where the script to be evaluated when the channel becomes readable would increment the updates array member.
Thus, data update can occur synchronously or asynchronously.

For example:

namespace eval random {
    ...
    proc update {} {
        variable data
        array set data "
            0,0 john    0,1 1234 0,2 4567 0,3 cc
            1,0 william 1,1 8901 1,2 2345 1,3 xedit
            2,0 anny    2,1 6789 2,2 0123 2,3 ps
            4,0 peter   4,1 4567 4,2 8901 4,3 ls
            6,0 laura   6,1 2345 6,2 6789 6,3 emacs
            3,0 robert  3,1 1234 3,2 5678 3,3 top
        "
        incr data(updates)
    }
}
The tabular data array index is the row number followed by the column number separated by a comma. The column number must start from 0 up to the total number of columns minus 1 (no holes are allowed in the column sequence).
The row number can take any integer value and be defined in any order, as long as it is unique during the lifetime of the module data. If a new row is created, it must take a value that was never used: the index of a row that has disappeared cannot be reused. Row numbers need not be consecutive.

When all rows (or only those table cells that have changed) have been updated, the updates member array must be incremented so that the core knows that it can update the table data display.

installation

A module is a package in the Tcl sense. When writing a module, you must then provide a pkgIndex.tcl file along with the module code file. The pkgIndex.tcl file is very simple, as the following example shows:
package ifneeded random 1.0 "source [file join $dir random.tcl]"
The line above says that if the random package is needed, the Tcl core should source the random.tcl module source code from the directory where it was installed. 1.0 is the version number for the package.

Modules can be installed at any valid place that the Tcl core allows (look at the pkg_mkIndex manual page for more information).

When you unpack moodss, you will find the sample modules in sub-directories. The current directory (.) is appended to the auto_load global list variable so that sample modules can be found when moodss is run from the unpacking directory.

For example, if you unpacked moodss in /home/joe/moodss-2.1/, you will find the random module package in /home/joe/moodss-2.1/random/ so that the following will work:

$ cd /home/joe/moodss-2.1/
$ wish moodss random
You can install your new modules in the default location: /usr/local/lib/ on Unix. For example, if you move the files in /home/joe/moodss-2.1/random/ to /usr/local/lib/random/, moodss (actually Tcl :) will still be able to find the random module (again, look at the pkg_mkIndex manual page for more information).

Please take a look at the INSTALL file for the latest information on how to install the moodss application itself.

Future developments

The following features will eventually be added to the core: Sub modules would be launched from the main module (probably through menus), in order to display specific areas of the data being observed, or even different but related information (this concept needs to be defined, input is welcomed).

I welcome any suggestion for new features that you may need in your specific use of moodss.

Miscellaneous information

For downloading Tcl software (such as stooop, scwoop, ...), visit my web page.

Send your comments, complaints, ... to Jean-Luc Fontaine.