Journal reference: Computer Networks and ISDN Systems, Volume 28, issues 7–11, p. 1355.

Active FORMs

Paul Thistlewaite and Steve Ball
Cooperative Research Centre for Advanced Computational Systems
Australian National University, Canberra.

1. Introduction

The addition of the FORM component to HTML has opened the way for much greater functionality in browsers that support this component. However, on-line forms have been used for some time, and the design and implementation requirements for the effective provision and use of on-line forms are now well-understood (see, for example, Yourdon in [1]). In assessing how HTML forms meet these design and implementation requirements, it is clear that they are, in many respects, deficient. These deficiencies invariably relate to the inability of HTML forms to change their contents in response to client-side activity, without intervention by the server that provided the form, and which can only happen at the specific request of the user on client-side (by submitting the form). Overcoming these deficiencies appears to require some client-side capability for modifying the form.

In Section 2 of this paper, we discuss the current capabilities and deficiencies of HTML forms. In Section 3, we describe a system we have built using Tcl/Tk [2] that provides active capabilities on the client-side of a HTML form, which overcomes such deficiencies. In Section 4 of the paper, we describe the work that remains to be done to make our system for active forms more functional.

2. Current HTML FORMs Capabilities and Requirements

2.1 Uses of Forms

Forms are used by WWW servers and clients as a way of handling input and interaction between client and server. The two principal uses of forms are:

to provide structured information to the server for later processing, and
to provide arguments to an application running on the server.

For example, a form may be used to provide structured information for a credit transaction, an order form, an application form for membership of a society or an application to participate in a conference, or more complex information, such as a tax return form.

Alternatively, inputs to a form may provide arguments to an application, such as a search engine, or parameters to a CGI script, thereby altering the behaviour of that application and the subsequent information provided by the server to the client.

2.2 Components of a Form

The elements of a form typically include:

one or more fields, requiring inputs of different types (date, dollar amount, zip code, mouse clicks(s)) or dimensionality (length of name, number of items in a list).
each field may have associated textual or graphical information, such as a name, an explanation of the intent of the author of the form in providing the field, examples of typical field values, instructions for completing the form, a topographical map, and so on.
each field may have a default value.
the individual fields are arranged in a particular sequence, and the form may be partitioned into zones or blocks of related fields.
and there is a mechanism for lodging or submitting the form to the server that provided it.

2.3 Requirements of Forms

Field Types and Constraints

The type of value acceptable as input to a particular field is, in general, constrained. These constraints are generally more particular than just typical "type" identification (eg., zip code rather than just integer, town-name rather than just string, ...) and often have other restrictions associated with them, such as an allowable range (eg., positive dollar amounts less than $10 million). As well as these internal constraints on allowable values for a field, a given field may have external constraints placed on it by the values given for other fields (for example, a user who entered "reseller" in the "customer-type" field may be allowed to place orders to a greater total value than one who entered "consumer"). Currently, HTML-forms provide only limited capability for typing fields, and no capability for providing relational or external constraints on a field.

Verification that the input to a particular field satisfies the constraints (internal or external) of the field should occur at the time that the user enters the value into the relevant field (see [1] or [5]), and if the supplied value fails to satisfy these constraints, then an appropriate pop-up message should occur at this time, detailing the nature of the error. Currently, HTML-forms provide no way of verifying such constraints until the form is submitted and examined by a server-side form processing program.

Field Labels and Entry Instructions

An advanced user, or one more familiar with a particular form (eg. an accountant with a tax return form), may prefer to see less explanatory material for the fields on the form, or to see such material only selectively, or to a selective level of detail. While HTML provides a mechanism for a user to choose some global property for the form (eg., form-wide extent of informational messages), there is currently no mechanism for selectively displaying information about particular fields (unless, of course, the client submits an appropriately annotated form to the server and receives a new form back).

Default or Computed Field Values

Currently, HTML-forms can contain a default value for any field. This default is, however, a constant value. In many circumstances, the appropriate default value for a field may be dependant on the value the user has entered for one or more other fields on the form. In some circumstances, the values entered for other fields might determine the correct value for a given field without the need for the user to input anything in the slot for this field. For example, having entered an address, the appropriate zip code may be derivable in most circumstances. Currently, HTML-forms provide no way for varying the default value for a field once the form has been sent by the server (without resending it), or for computing and entering a value for a field whose possible values are determined by the values entered elsewhere on the form.

Field Zones and Sequencing

Often, fields on a form can be naturally grouped into zones of related items: eg. on an order form, the zones might be:

those related to the customer's name and address,
those related to the items and quantities being ordered,
payment details, and
supplier-side credit authorisation.

Just as commonly, the requirement that the user provide a value for a given field, or even an entire zone of fields, may vary depending on what values the user has supplied for other fields. For example, on a medical survey form, if the user has previously stated that he is male, then the zone of fields or questions relating to female-specific medical conditions can be deleted from the form entirely, or ideally, only exhibited in the overall form if and when the user indicates that she is female. Currently, HTML-forms provide no mechanism for selectively enabling or disabling some segment of the form conditionally in response to user activity with the form, without intervention by the server at the specific request of the user.

Field Input Semantics

Fields have semantics with respect to user interaction, either explictly stated in the HTML standard, implied by the standard, or implemented by a browser. Sometimes these semantics conflict with a form author's desired user interface paradigm. For example, pressing the mouse button on an image field causes the form to be submitted, and the coordinates of the mouse click are included with the submission. These semantics imply that a form cannot support even simple user interface semantics, such as including the coordinates of two or more button presses, or the tracking of pointer motion over the image.

3. Active FORMs

A WWW browser, known as SurfIt! [3], has been developed which has the ability to download Tcl/Tk [2] scripts and execute them safely as applets (see Figure 1). A Tcl applet is a document whose MIME Content-Type is application/x-tcl. SurfIt! supports the use of Tcl applets for client-side scripting of fill-out forms. This section provides detail on the support provided by SurfIt! for client-side scripting of forms.

Figure 1: The SurfIt! WWW browser.

3.1 Use of HTML for Form Applets

HTML Version 3.0 [4] allows the SCRIPT attribute for the FORM tag to associate a client-side script with a fill-out form. When SurfIt! encounters such an element, the document referenced by the URI given as the value of the SCRIPT attribute is loaded and then the browser executes the document as an applet attached to the form. For example:

<FORM SCRIPT="/tcl/form.tcl" METHOD=GET ACTION="/cgi-bin/submission">

The applet is started as soon as it becomes available. This may be after it has been loaded from a remote server, or immediately if it is stored locally, for example in the browser's cache. Starting the applet as soon as the start tag of the form is encountered may be problematic: the applet may need to be informed of the contents of the form for it to function properly. However, the Tcl Applet API provides a means for the applet to be notified when the form end tag has been reached, and so the applet may choose to delay performing any processing until that notification has been received.

SurfIt! provides no extensions to HTML to support client-side scripting, nor are any required due to the availability of Tk for managing the form. This is in contrast to JavaScript [6], which extends HTML to embed applet directives by introducing new, arbitrary, attributes for the <FORM> and <INPUT> elements.

3.2 Tcl/Tk Applets

SurfIt! includes a definition of the environment in which Tcl applets are executed. Briefly, this environment is defined as:

evaluation in a safe slave interpreter, which provides an isolated namespace for the applet and protects the browser and other applets from interference.
access to windowing functions via Safe-Tk, with a top level window automatically created for the applet.
access to the browser window in which the applet was loaded.

Applet Scope and Longevity

The browser creates objects of four different types: the browser itself, hyperwindows, hyperpages and forms. Applets are attached at one of these levels, and how the applet is managed by the browser differs depending on which level it is attached at. Normally, applets start their execution attached at the hyperpage level, and subsequently may change the level at which they are attached. However applets associated with a form are automatically attached at the form level and cannot change the level at which they are attached.

An applet attached at the form level is terminated when a new hyperpage is loaded. The browser also provides other mechanisms for the user to control any applet, including the means to explicitly terminate an applet.

3.3 The Tcl Applet API

In order to be able to perform useful functions an applet must be able to interact with the browser. Tcl applets and the browser may interact in two ways: via the applet command and by means of browser call-ins.

The `applet` Command

Applets use the applet command to retrieve information and perform applet-related functions. Some of the commands which are useful to an applets attached to a form include:

applet browserversion: returns a string which identifies the browser.
applet embedwindow: returns an indentifier which may be used to refer to the Tk Text widget within which the form is rendered.
applet embedindex: returns the index of the start of the form.
applet level: returns the level at which the applet is attached. This always returns "form" when executed by an applet attached to a form.

The applet command also includes functions related specifically to forms, such as:

applet formendindex: returns the index of the end of the form.
applet formitems: returns a list of the input items contained within the form. This list will only be complete once the form element has been completely parsed. Each element of the list describes an input item, and each item is itself a list of four elements: {type name value widget} The meaning of these elements corresponds to the arguments passed to the HMapplet_item procedure (see below).

Browser Call-ins

The browser also communicates with the applet by means of "call-ins". Call-ins are well-known procedure names that the browser invokes in response to certain meta-events. Below are some of the currently defined call-ins that may be used by an applet attached to a form:

terminate: invoked immediately prior to the applet being destroyed.
anchor_activation url: invoked whenever a source anchor within the applet's form is activated. This functionality is not yet implemented.
HMend_form: invoked when the parser has encountered the end tag of the form element.
HMreset_form: invoked when the form is reset to its default values.
HMsubmit_form method query: invoked when the form is submitted. method is the HTTP method to use for form submission: get or put. Currently only the GET method is supported. query is a Tcl list representing the query to be submitted. The list contains name/value pairs.
HMapplet_item type name value widget: invoked for each input element in the form. type gives the type of item which has been added to the form, for example text, password, checkbox, radio, hidden, image, select, textarea, submit or reset. name is the value of the NAME attribute as given in the <INPUT> element. value is the default value for the input item as given by the VALUE attribute in the <INPUT> element. widget is the Tk pathname of the user interface widget that the user manipulates to enter a value for this item. Figure 2 illustrates how an applet may gather information on the input items in a form.

Information gathered about a form by an applet

Figure 2: A sample form, with the applet displaying information gathered for the input items.

The applet is free to manipulate the widget associated with an input item using the facilities provided by Safe-Tk. For example, if the applet wishes to monitor the keystrokes being entered into the Entry widget for all text items then it would use the bind command, for example:


proc HMapplet_item {type name value win} {
	switch $type {
	text {
		bind applet  {monitor_keystrokes %W %A}
		bindtags $win "applet [bindtags $win]"
	}
	.
	.
	.
}

One use of a script bound to a widget in this fashion may be to implement field types and constraints. The problem of handling computed field values is solved since a change in the value for one field may cause the (re-)computation of values for other fields. The value displayed by the appropriate widgets may then be updated.

3.5 Writing Form Applets

When writing a Tcl script to manage a form it is useful to split its execution into three phases: applet startup, form management and form submission.

Applet Startup

Some applications will employ an applet written specifically to manage a particular form. An applet of this nature will expect certain input items to be present in the form. If the required input items are present then their details should be stored (such as the default value, widget pathname, and so on) and bindings set as appropriate. For example, a procedure may simply set a variable to store all of these values, such as:


proc HMapplet_item {type name value win} {
    global item_$name
    set item_$name [list $type $name $value $win]
    .
    .
    .
}

If the required input items have not been parsed then the applet should take appropriate action. For example, the applet could disable all submit buttons in the form and display an error message to the user suggesting that they reload the document.

Other applications may have simpler requirements and will want to employ general-purpose applets to achieve certain common effects, for example to constrain a certain text type input item to have only a numeric value. In this case the applet will wait until the input item with the correct name parameter is parsed and will store that item's details and bind a script only on that item. For example:


proc HMapplet_item {type name value win} {
    switch $type {
	text {
	    bind $win <KeyPress> procedure that enforces field type
	}
    .
    .
    .
    }
}

A current limitation of the HTML v3.0 specification is that only one script may be attached to a form. This may be overcome by simply specifying an applet that loads the scripts for all of the other required applets and executes them.

Form Management

The problems of handling field labels, entry instructions, field zones and sequencing are solved by managing the state of the on-line form in response to user actions. An example which possesses these problems is an on-line form for tax assessment. To provide varying levels of explanatory material the form may include a field that the user may set as needed. The applet may then add or remove text from the Tk Text widget as appropriate.

The tax return form will be much easier for the user to fill out if the form is organised into field zones which are sequenced. For example, a dependent spouse rebate will only be applicable if the user has indicated that they are married. Another example (see Figure 3 and Figure 4) shows that fields, with accompanying text, may be displayed only if the user indicates that they must fill out that part of the form.

Figure 3: The form before the user has pressed the "Yes" button.

Figure 4: The form after the user has pressed the "Yes" button.

Note that the "No, I didn't" button will return the form to its original state.

To insert new text or widgets the script needs to know an index in the Tk Text widget at which to insert the new items. The Tcl applet API allows the script to find the beginning and end index of the form by using the applet embedindex and applet formendindex commands respectively. These two indices give a bound within which new items must be placed. The current implementation of Safe-Tk cannot prevent the applet from inserting characters outside these bounds, but a future version may enforce that restriction. It is also useful to use the indices of the input items themselves as "anchors" around which to place text. The Tk Text widget allows the widget pathname to be used as an index, which is very convenient for this purpose. For example:


    [applet embedwindow] insert [applet embedwindow].input,1 "Some extra text"

Form Submission

When the form is submitted for processing the browser first invokes the applet's HMsubmit_form procedure with the details of the form being submitted. The browser uses the return result of this procedure call for the actual form submission. The return result must comply to the format of the query input parameter, ie. it must be a list which has an even number of elements. The applet is free to return whatever string it desires - it is in no way limited to the query presented as input. Presumably the applet will maintain its own state and will return a string based upon its internal state.

If the execution of the applet's HMsubmit_form procedure results in a Tcl error being generated then the form submission is cancelled. This may be used to implement certain effects such as retrieving two button clicks on an input image and then submitting the form with parameters that include the locations of the two clicks. For example,


proc HMsubmit_form {method query} {
    global hasclicked

    if {the form submission is due to an image click} {
	if {$hasclicked} {
	    # This is the second mouse click
	    return $query
	} else {
	    # This is only the first mouse click
	    set hasclicked 1
	    error ""
	}
    }
}

This facility, along with event processing provided by Tk, solves the problems related to field input semantics for on-line forms.

Upon form submission the script should first check the state of the form to make sure it is consistent. If there are any problems then the user must be notified and an error returned, which will cancel the form submission. User feedback may consist of a popup dialog, a visual indication of the offending field (for example, by flashing it's background a different colour), an audible indication (that is, a beep), or any combination of these.

If the applet's internal representation of the state of the form is acceptable then a new form submission string may be constructed. If the applet does not manage all of the input items in the form then the string supplied by the browser must be modified or added to. If the applet has changed the form's user interface by either adding or removing input items then the query passed to the applet's HMsubmit_form procedure by the browser may simply be ignored and a new query constructed based upon the applet's own internal state. The string passed back to the browser must consist of pairs of values, ie. it should contain an even number of elements. The first element in each pair is the name of the parameter and the second element is the value of the parameter. The browser will then generate an appropriate URL to send to the server. For example, if the applet returns name {Steve Ball} age {None of your business} as the result of the HMsubmit_form procedure then the browser will send the URL

http://host.name/cgi-bin/path?name=Steve%20Ball&age=None%20of%20your%20business

4. Future Work

4.1 Form State

The current implementation of the SurfIt! browser only maintains the state of a form's input fields as they existed when the form was initially parsed. The browser makes no attempt to detect any changes made to the form by an applet. If the applet subsequently changes the form, by adding or removing input fields, then the browser will have invalid information on the state of the form. For this reason it is vital that the applet maintain the state of the form internally so that it can intercept form submissions and present a URL that is a true representation of the current state of the form. Unfortunately, if the applet is terminated for some reason then any form submission will not be intercepted and an incorrect URL will be sent as the form action.

To address this problem some additions must be made to the Tcl applet API to allow an applet attached to a form to add new input items and remove existing input items, rather than allowing the applet to directly manipulate the user interface as is currently done now. This will allow the browser to maintain the state of the form and will ensure that the browser's state information is consistent with the user interface for the form at all times, even when the applet has been terminated. This will make the management of forms more robust and will simplify applet development.

4.2 Reloading Forms

When a new hyperpage is loaded any form and all applets on the existing hyperpage are destroyed. This means that any data entered into the form by the user is lost. If the user subsequently reloads that hyperpage from the history list then they will have to reenter all of the data that they had previously entered into the form. Also, any applet attached to the form will be restarted from scratch. It would be better if the state of a form was saved in memory and the applet's execution was suspended. Then when the hyperpage is reloaded any data entered into the form may be restored and the execution of the applet attached to the form may be resumed, thus restoring the hyperpage to the state it was in before any other documents were loaded. The Tcl Applet API may be extended to include a reload meta-event call-in.

4.3 Stylesheets

A common complaint made by authors of on-line forms is that it is not possible using HTML to influence the presentation of a form so that it corresponds to the appearance of a paper version of the same form. This problem is by no means restricted to on-line forms, and is a well-recognised problem for Web documents in general. To address this issue Style sheets [7] will be implemented for SurfIt!. Style sheets separate the logical markup of the form's contents from the form's presentation. In this way the form author will be able to specify the correct fonts, colours, and so on, for the on-line form that are used for the paper version of the form.

Although an applet is quite at liberty to modify the hyperpage in any way, when adding text to the document it would be desirable for the text to take on the characteristics of an HTML element or of the HTML element(s) surrounding the new text - that is, the applet should add text at the HTML level rather than at the raw widget level. Moreover, although it may be possible to deduce the necessary Tk Text widget tags to achieve the desired effects, such details are browser implementation dependent.

The Tcl applet API will be extended to support adding text and elements to the hyperpage, hiding implementation details on how stylesheets are applied to the document text. If an applet adds new text to an existing HTML element then the styles and subclass of that element will also apply to the newly added text. For new HTML elements there are two modes in which an applet may add new elements, either with subclassing of the element or with no subclassing. The latter case poses no problems, but in the former case for privacy reasons it would be unwise to allow an untrusted applet access to the stylesheets active in the browser, so there would be no way for an applet to discover the defined subclasses. However, many applets will be written specifically to manage a certain form and in that case the subclasses will be known in advance. The extension to the applet API should cater for both these cases.

References

[1]: E. Yourdon, Modern Structured Analysis, 672 p, Prentice-Hall International, 1989.
[2]: J. K. Ousterhout. The TCL Language and Tk Toolkit, Addison-Wesley, 1994.
http://www.smli.com/research/tcl
[3]: S. Ball. SurfIt! WWW Browser.
http://pastime.anu.edu.au/SurfIt/
See also Applet execution enviroment.
[4]: Dave Raggett. HTML v3.0 Specification, 28th March 1995 draft. WWW Consortium.
http://www.w3.org/pub/WWW/MarkUp/html3/Contents.html
[5]: J.G. Burch, Systems Analysis, Design and Implementation, 854 p, Boyd and Fraser Publishing, Boston, 1992.
[6]: JavaScript Specification. Netscape Corporation.
http://www.mcom.com/comprod/products/navigator/version_2.0/script/index.html
[7]: H. W. Lie, Web Style Sheets, WWW Consortium.
http://www.w3.org/pub/WWW/Style/

About the authors

Dr Paul Thistlewaite is the Project Leader for the PASTIME Project, of the Cooperative Research Centre for Advanced Computational Systems, which is responsible for making the proceedings of the Australian Parliament available on Web. His principal research interests include the authoring and management of large hyperbases, information retrieval, digital video, parallel processing, and document analysis.

Steve Ball is a Software Engineer with the PASTIME Demonstrator Project of the Cooperative Research Centre for Advanced Computational Systems. He is author of the SurfIt! Tk-based WWW browser. Steve is an application developer for advanced user interfaces and digital video-on-demand systems. He specialises in network applications and X Window application development, particularly with the Tcl/Tk programming language and toolkit. Steve Ball holds a Bachelor of Science (ANU).