Israel Ben-Shaul Shlomit Ifergan
Dept. of Electrical Engineering
Technion, Israel Institute of Technology
Haifa 32000, ISRAEL
issy@ee.technion.ac.il,
shlomiti@tx.technion.ac.il
Motivated by the potential opportunities that lie in providing cooperation among Web-Servers, we propose a generic and scalable approach for dynamic server-interconnectivity, and present the WebRule framework which embodies this approach. WebRule is based on the Event Condition Action rule formalism, which enables Webmasters to specify declaratively various modes of cooperation, among them synchronus vs. asynchronus interaction, on-line vs. off-line, and automatic chaining of cooperative actions. WebRule requires no centralized encoding, control or administration, thereby enabling one to utilize and deploy Web resources in a truly decentralized manner. WebRule has been implemented as a Java server plug-in and an S2S/HTTP protocol, seamlessly integrated with standard Web server technology and protocols.
Keywords: Server cooperation, server interconnectivity, event-based integration, Rule-based systems.
A major goal of network-centric computing is to enable a large number of geographically-dispersed and autonomous peer nodes to engage in a collaborative computing effort that enhances the functionality of the individual nodes as well as the network's overall services and utilization. While the WWW has been both driving and driven-by forces of global network computing, its client-server architecture seriously limits its capabilities, particularly with respect to scalable inter-server cooperation. Despite the incredible growth in the number of Web-servers, (from a few thousand in the early days of 1993, to 225,000 in November of 1995, to an astounding 488,000 in July of 1996; see [NW]), they are conceptually isolated from each other. Currently, one server cannot easily leverage the information and other services provided by other servers in order to enhance its own value and productivity. The only connectivity which is provided in the Web is at the (passive) document level, via static hyperlinks, which can "come to life" only by a client explicitly following them. We are interested in shifting some of the "activism" to the server side, enabling active interconnectivity, for both data- and control- flow.
There is a vast number of potential applications that could benefit
from a generic server-interconnectivity technology.
Example 1: Consider a server which, upon a client
request, contacts several other distributed servers (with which a prior
agreement has been reached regarding access to their services); it collects
from them pertinent and timely information which can be sensitive to client
and server parameters; it then filters, and synthesizes the collected data
from multiple resources, and eventually returns the dynamic page to the
client. This application would be attractive for sites that provide correlated
services. For example, browsing for "Sting" in a CD music store
may result in a page enriched with information from a ticket office site
(and/or from Sting's home page) about his upcoming concerts, perhaps only
showing the local ones (with respect to the querying person).
Example 2: Assume an HTML document, H1,
which originates at site S1 and is replicated in site S2
(e.g., for caching purposes). Suppose that S2 wants to be asynchronously
notified when H1 has changed, and optionally get a fresh copy of
it. Alternatively, suppose S2 does not hold a local copy of H1,
but instead wants to be notified whenever H1 is referenced
by a client.
Example 3: Consider an Intranet with a large set
of distributed hosts, each of which providing certain services that are
accessible through an HTTP server installed in each host. Upon request
for an invocation of a service (e.g., sending mail outside the firewall),
one of the servers may query its peer servers to find which server performs
the desired service and redirect the request to it. Or, in case of duplicated
services, the least loaded server is selected, based on some (distributed)
load balancing method. And if the service function itself can be distributed,
a set of peer servers for execution is selected.
None of the above examples can be implemented on the Web without an extensive programming effort. In order to collaborate with one or more remote servers, either statically or dynamically (i.e., while serving a client request), a site's Web administrator would have to build an ad-hoc application-specific solution, mostly from scratch. Moreover, it would typically be based on a server that acts as a client to another server, limiting both the communication, to basic HTTP, and the interaction, to synchronous client-server interaction, which is often unsuitable for the kind of interactions that may be required by Web servers.
One possible approach to server interconnectivity, which has been explored in several projects (e.g., [SK96], [AD95]), is based on a special client which is capable of coordinating multi-server activities. This client-centered approach has several inherent limitations. First, as seen in some of the examples above, inter-server collaboration may require asynchronous interaction and reactive control, as opposed to the proactive and synchronous client-server interaction. Second, it implies that the coordinating client has to know a priori exactly which servers will be used, thereby significantly limiting scalability and dynamicity. We would like to be able to maintain only partial knowledge at each node regarding connectivity to other nodes, instead of having to encode all information in the client node. Third, this approach might require the explicit invocation of a human-user to initiate this coordination. Our aim, in contrast, is to facilitate automatic collaboration, based on knowledge which is stored in the servers and directs their cooperation. As shown in Example1 we often want to support server collaboration in a manner that is transparent to clients, as opposed to initiated by them. Finally, since clients typically operate with lower available bandwidth than servers, the client-oriented approach wastes expensive bandwidth since all relevant data must be transferred to the client before any filtering decision takes place.
The alternative approach, proposed in this paper, is to encode and execute the collaboration in the servers by extending them with cooperation capabilities. This is the approach embodied in the WebRule system, aimed at providing a generic framework that enables to flexibly and dynamically specify and execute various collaborative services between large number of loosely coupled distributed Web servers.
Three main requirements are involved in designing a server-interconnectivity framework:
The rest of the paper is organized as follows: In Section 2 we present the collaboration framework, covering general concepts and their realization using a distributed rule-based language designed especially for operation over the Web. In Section 3 we present an example that illustrates how collaboration is achieved in WebRule. Section 4 overviews the realization of the framework, covering the integration with Web servers and the S2S/HTTP protocol for server-server interaction. Section 5 compares to related work, and Section 6 summarizes the contributions and points to future work.
In general, WebRule is based on an event-based integration model, or more specifically on the multicast approach of implicit invocation [EBI96]. That is, collaboration is expressed by means of servers registering interest in certain events and associating certain actions with them, which are triggered upon the occurrence of those events. Actions may themselves generate new events which in turn trigger further actions, recursively.
Events and actions are discussed in detail in Section 2.3. For the time being it is only important to distinguish between local events and actions, that occur in and affect only the site in which they operate, and remote events and actions which occur in and affect remote sites. Cross-site collaboration is realized by invocation of remote actions and by the occurrence of remote events. We identify four basic modes of interaction for collaboration purposes.
By combining local and remote activation facilities and interaction modes, arbitrarily rich forms of multi-site collaboration can be formed, in various flexible ways, while still retaining their decentralized nature. The collaboration model is realized by three major components:
We now elaborate on the first two components, and briefly discuss the third in Section 4.
Rules have been shown to be a natural computing paradigm for programming event-based integration (e.g., [BSK93]). There are several rule models. In Event-action rules, a rule is fired if a matching event occurs. In condition-action rules, a rule is fired if its condition is logically satisfied. In event-condition-action (ECA) rules [MD89] [UD90], a rule is fired if a matching event occurs and its guarding condition is logically satisfied.
We adopted ECA rules because of the clear and inherent distinction in the Web between events, which specify when to check if a rule should fire, and conditions, which specify what to check for. Examples of asynchronous events include a client-request for a URI, a change in a file contents, etc. Such events can be modeled directly, instead of mapping them to some data element and checking its value as in condition-action rules. On the other hand, it is equally important to preserve the condition construct for guarding the execution of actions, to be able to enforce security policies and in general protect the privacy of sites against unwanted interactions with remote servers. For example, an administrator may want to limit the number of requests which can be concurrently handled by the server, to avoid server overloading. This can be naturally specified in a rule's condition.
| Rule rule-name; Permissions permissions-attribute-list; On event-expression; Condition condition-expression; Begin action-block End |
Figure 1. Basic rule structure
The general structure of a WebRule rule is depicted in Figure 1.
The name of the rule does not play a major role during its execution, since it is triggered by an event. Rule names are used mainly for (automatic and manual) management and administration of rules. The Permissions block lists attributes-value pairs which apply to accessing and manipulating the rule as a whole. They are analogous to access permissions in file systems, but at the rule granularity (there is a similar server-level permission block which applies to the entire knowledge-base). Two common built-in attributes are Owner, which specifies the server that can manipulate the rule (e.g., remove, modify, disable) and ACL, which denotes which servers can invoke the rule. Note that the owner of the rule may not necessarily be the local host in which the rule is defined. This situation occurs when a rule is uploaded from a remote host, as part of the advanced reactive mode; in this case, the owner is the server that sent the rule.
A rule is triggered if its event-expression becomes enabled. Each built-in (primitive) event has a signature, i.e., a list of formal parameters, which are bound to actual parameters by the event signaling mechanism (unlike the heuristic binding in [HEIN92]). An event expression can specify a subset of the formal parameters with a restricted range, which are used for the signaled event with the expression. An interesting aspect of passing event parameters concerns support for CGI Forms in Request events, which are events that are generated as a result of a client request. Accessing CGI variables is easily done by specifying the variable names in the parameter list of the Request event. This mechanism for passing parameters from forms to rules, to possibly other forms sent to remote servers, has been extremely useful. Finally, composite event-expressions can be formed from primitive ones by means of disjunction, conjunction, and sequencing.
Once an event has been signaled in a rule, its condition is checked. A condition is a collection of queries over parameters of the event signal or the server's global variables, where composite conditions can also by formed by means of logical operators (AND, OR, and NOT).
If the condition is satisfied, the action block is invoked. An action block consists of a set of actions, each of which representing a basic unit of execution of local or remote operations. An action block is enclosed with a control-flow directive that specifies whether to execute all actions in the block in parallel or in serial. Parallel execution is particularly useful when each action in the block interacts with a different remote server and the different threads are independent computationally, increasing concurrency as well as robustness, because a failure in one remote server does not affect the operation of other servers. Actions also take parameters, which are bound in run-time to actual parameters drawn from local or global variables, event parameters, or literals. Finally, each action may be followed by a post-condition, which is a single query on the value returned by the system after invoking the action (e.g., for indicating whether the action succeeded or failed), followed by an action-block.
Although rules in general fire as a result of a triggering event, it is useful to distinguish between cases where they are triggered by an external event, and cases where they are triggered from other rules, i.e., as part of rule forward chaining . We distinguish between synchronous and asynchronous chaining. In the former chaining, an action is not committed until all rules that have an event matching the signaled event are checked, and if satisfied, fired, recursively. This mode is often used when the original rule is fired with the intention to collect information from multiple resources, process it, and return it. Note that in case of a parallel block still all actions execute concurrently. Asynchronous chaining behaves like normal event triggering, i.e., an action is committed once the events have been generated, without waiting.
As mentioned earlier, the richness of the built-in events and actions provided by the system is crucial for its effectiveness. Although mostly anything can be created with extended actions (see below), the whole purpose of WebRule is to facilitate easy and (semi-)automatic construction of collaboration. The set of actions can be classified in a type hierarchy, as shown in Figure 2.
Figure 2. Classification of action/event types.
Remote actions are divided into two main categories. HTTP actions generate
conventional HTTP requests. For example, SendHTTP10GETReq
builds a GET requests of HTTP 1.0, using the actual parameters of the event
(e.g., URI, host, form variables). S2S-requests include WebRule-specific
actions which generate remote WebRule events that can be interpreted by
a peer WebRule server. Two important actions are RegisterToEvent,
which allows to register for events that occur at remote sites, implementing
the reactive interaction mode; and UploadRule
which allows to upload rules at remote sites, for the implementation of
advanced reactive mode.
Local-extended actions are the extensibility mechanism for interfacing
to external applications. The main extended action is RunServlet
which allows to run a Java class, in a similar fashion to Jeeves' servlets
[JEEVES]. There are additional local actions for
file manipulation, system control and utility actions Several utility actions
are used for synthesizing HTML files, fully or semi-automatically. MergeFilesUsingFrames,
for example, merges a set of files into frames following size and location
specifications.
The classification of events is shown in Figure 2. Local events correspond to events which occur in the local site either external to the rule system or within it. Examples of the former are periodic/absolute temporal events and modifications to document root files. Events that occur within the rule system are generated by either actions of rules, or by the runtime system. An example of the former is modification of global variables. An example of the latter is the ShutDown event, which is generated just before the server shuts down and may be used to notify remote servers about the shut down (section 3 shows an example of its use). Event extensibility is provided through the customized event. To define such event, the administrator defines the name and parameters of the event. This event can then be signaled by the SignalCustomizeEvent action, linking it to the rest of the system.
Finally, remote events are generated by either remote WebRule actions or by remote HTTP requests. For example, UploadRuleEvent is generated as a result of the UploadRule action. Putting it all together, a single client request may trigger a massive distributed invocation of chained activities, all of which are contributing to the fulfillment of the request, as shown in Figure 3.
For further details on the event- and action- set, see [WebRuleDesign].
Figure 3. A chaining reaction caused by a client HTTP.
We now turn to a concrete example that captures most of the concepts discussed in the previous section. The example involves a single HTTP server (Internet-Provider), two WebRule servers, Bank-Server and Deal-Server, and one SQL-server. The Bank server allows users to issue queries about the state of their account. Upon such a query, the bank server creates a dynamic document containing: (1) The status of the account, retrieved from the SQL-server by invocation of an extended action. (2) The current deals that the bank offers, provided by a separate Deal server which resides in central location, away from local bank. The Deal server also collects a file from a separate Internet provider server that is involved with the specific deal offered, and merges it with a local document. In addition, the Bank-Server wants to be notified when the Deal-Server is about to shut down, in order to avoid initiation of request to retrieve the deal document. See Figure 4.
Figure 4. Description of the multi-server communication.
The following tables (Figure 5, Figure 6) describe how the example presented above can be expresses by simple rules.
Figure 5. Deal server rules
Figure 6. Bank server rules
The example demonstrates both the direct and reactive collaboration modes. Direct collaboration results from a Request event which triggers the bank server's rules, Bank-Deal-Info and Bank-Account-Info. Notice that Bank-Deal-Info invokes the action SendHTTP10GETReq which triggers the deal server's rule, Deal-Info, launching synchronous rule chaining, as described in Figure 7.
Figure 7. Synchronous rule chaining.
Reactive collaboration results from bank server's internal StartUp event which triggers the bank server's rule, BankServerDoOnStartUp. Notice that this rule registers interest in the occurrence of event ShutDown in the deal server. The notification on the occurrence of that event is sent to the bank server by the deal server's rule DealServerDoOnShutDown only when the deal server is about to shut down. This notification triggers the bank server's rule DoOnDealServerShutDown (which disables the rule Bank-Deal-Info), launching asynchronous rule chaining.
There are several alternatives to integrating Web servers with additional functionality. The simplest and most common method is the CGI [CGI], allowing to attach a completely separate program which is invoked as a result of a specific client request. Besides the low performance of CGIs and the fact that (basic) CGI scripts need to be (re)invoked for each request (these limitations have been partially addressed by FastCGI [Fast-CGI]), their main drawback is that as an external program, they cannot change the generic behavior of the server, so that every client request can be checked. Moreover, they require the client to specify a certain program to invoke, eliminating the client-transparency property. Thus, we used a server API [API], which allows tight integration with the server control. In particular, we replaced the original request handler with our own.
Regarding the implementation language, we chose Java for several reasons: support for security and platform independence were both necessary for the rule-uploading mechanism. Multi-threading and synchronization were important for enabling concurrent execution of rules triggered by the same event (and even among actions in the same parallel action block) and for synchronous chaining (see 2.2.1). Thus, we implemented WebRule on top of Netscape's Enterprise Server, which, at the time of this writing, is the only server that supports Java server plug-ins under WindowsNT as well as Solaris.
Note, however, that by using the extensibility mechanism mentioned in 2.3, WebRule supports any language that can be linked with the native interface of Java (currently C and C++).
To achieve server to server communication we designed S2S/HTTP, a simple protocol built on top of HTTP [HTTP]. S2S/HTTP enables message/request passing and acknowledgment by initiating HTTP connections between the servers and sending unique information encoded in the HTTP message-body. The S2S/HTTP utilizes unique URIs (such as "/WebRule/UploadRule"), hence the receiving WebRule-extended server can identify the HTTP request as a S2S/HTTP message and correspond accordingly. A conventional server accepting a WebRule message will not respond with a proper S2S/HTTP response, enabling the WebRule server to ignore the returned results.
There are two main reasons for implementing the protocol on top of the currently used HTTP, instead of devising a new proprietary protocol. First, using HTTP enables WebRule servers to interoperate while passing through intermediate proxies, firewalls and other standard mediator Web components. Second, it enables the WebRule servers to keep up with any extensions to HTTP (such as SHTTP) without making major changes. For more details on this and other technical aspects of WebRule, see [WebRuleDesign].
Most of the functionality as described in the paper has been implemented, including a full implementation of the example shown in Section 3. A public demo is available, contact the authors for more information. WebRule runs on Solaris and WindowsNT, as an API extension of Netscape's Enterprise server. The code contains approximately 10,000 lines of Java and C code and about 50 Java classes.
CORBA [CORBA] is a prominent framework for interconnecting components by means of mediation. It provides a communication infrastructure for a heterogeneous, distributed collection of general collaborating objects. Unlike the brokering approach of CORBA, WebRule's interaction is performed on a pairwise basis. In addition, standard CORBA supports "quasi-asynchronous" communication style while WebRule supports asynchronous communication style enabled by the event-registration mechanism (see 2.3). Most of all, the main distinction between WebRule and most other middleware frameworks is that they provide a passive integration framework. In contrast, WebRule provides an active Web-server collaboration framework in which occurrence of an event in a server causes occurrence of other events in remote servers as well as the local server itself. The occurrence of those events is based on the knowledge-based stored in the server and directs the cooperation. CORBA is, however, a framework with which WebRule should be able to interact. Thus, we are working on interfacing to ORBs through JOE [MAH96].
When considering Web-specific solutions to server collaboration, we find that most of them are characterized by being application-specific. That is, they extend Web-server, typically in a proprietary manner, to solve specific problems. This leads to a profusion of specialized non compatible servers. One such application-specific and server-oriented cooperation tool is described in [RJD96]. There, the cooperation between caching servers is used to solve the bottleneck problem due to existence of a single proxy. This is done by creating a protocol that allows multiple caching proxies to cooperate and share their cache. Another example is Atlas [ATLAS] which was developed to solve problems related to manageability of the Web. It mainly contributes to the automatic maintenance of link information across distributed servers. In this framework, Web-servers communicate with each other indirectly, through Atlas servers using CGI-bins to communicate with them. Our approach, in contrast, extends Web-servers in a way that still allows them to act as conventional Web-servers (for example, a WebRule server with an empty knowledge base acts exactly as a conventional HTTP server), but is also tightly integrated with the Web.
Sun's Jevees [JEEVES] provides a generic platform for server extensibility, where servlets, the analog of client's Applets, can be attached to servers and invoked by clients using the conventional URL request. While powerful, the explicit procedure-call invocation mechanism dictates that only a single servlet can be invoked as a result of a single client request. As shown in [BSK95], the event-based approach is likely to be more scalable and more effective for control integration among multiple distributed components. In particular, a single event in WebRule may result in an invocation of several rules, both in local and remote servers, concurrently. Jeeves per-se, however, may be viewed as complementray to WebRule. In particular, we are considering implementation of WebRule on Jeeves servers as well as incorporating WebRule as a Jeeves' servlet API.
A different approach for extending server functionality for collaboration purposes is agent-based. There are numerous agent technologies. Telescript [TELE], for example, allows to create a server that hosts "intelligent" agents. Clients can log on to change the profile of their agents and can be notified of results. One limitation in tools like Telescript is that they are proprietary and can be deployed only on their own server [Tabriz]. Kinetoscope [Kin] is another agent technology that allows users to create a variety of agents for different tasks. The environment is extensible and is not restricted to a certain server. Still, in order for two agents to communicate, they, in general, need to be explicitly programmed in that manner, whereas rules, our "agents", need not; they interact implicitly, as a result of a matching between actions and events. Our major strength is in connecting components and allowing them to interoperate. In particular, our rules can encapsulate agents and interconnect them.
The two major contributions of this work are:
We are currently working on several extensions to this work, among them a high-level programmable interface for generating rules, client-injected rules in addition to server-oriented rules, and strengthening the notion of a pairwise (re)negotiable agreement as a basis on which all other collaboration rests. Further extensions to this work would be developing built-in connectivity to databases and CORBA by enriching the action library to work with JDBC and JOE [MAH96].
This research is supported in part by the Israeli Consortium for Multimedia Technologies. We would like to thank Gal Shachor, for numerous technical discussions and for implementing several WebRule applications (including a variant of the example shown in the paper), and Eran Katz, for long and stimulating application-oriented discussions. We also thank George Heineman and Peter Skopp who worked on an early version of WebRule.
[AGENTS11]
D. Miller. Three types of intelligent agents, their uses, and their future.
http://www.netscapeworld.com/netscapeworld/nw-11-1996/nw-11-agents.html
[API]
The Netscape Server API. http://home.netscape.com/newsref/std/server_api.html
[ATLAS]
J.E.Pitkow and R.K.Jones. Supporting the Web: A Distributed Hyperlink
Database system.In Proceedings of the fifth
International Word Wide Web Conference, May 1996, Paris.
[BSK93]
I. Ben-Shaul and G. E. Kaiser, and G. T. Heineman. An Architecture for
Multi-User Software Development Environments. Computing Systems, The Journal
of the USENIX Association, 6(2):65-103, 1993.
[CGI]
The Common Gateway Interface. http://hoohoo.ncsa.uiuc.edu:80/cgi/overview.html
[CORBA]
Object Management Group. The common object request broker: Architecture
and specification, July 1995.
[DE94]
D. Eichmann. Ethical Web Agents. In Proceedings of the First International
World Wide Web Conference, Geneva, 1994. http://www.ncsa.uiuc.edu/SDG/IT94/Proceedings/Agents/eichmann.ethical/eichmann.html.
[EBI96]
D. J. Barrett, L. A Clarke, P. L. Tarr and A. E. Wise. A Framework for
Event-Based Software Integration. ACM Transactions on Software Engineering
and Methodology, 5(4):378-421, October 1996.
[Enterprise]
Netscape Enterprise Server 2.0 Data Sheet. http://home.netscape.com/comprod/server_central/product/enterprise/enterprise_data.html.
[Fast-CGI]
The FastCGI Specification. http://www.fastcgi.com/.
[HEIN92]
G. T. Heineman, G. E. Kaiser, N. S. Barghouti, and I. Ben-Shaul. Rule
Chaining in Marvel: Dynamic Binding of Parameters. IEEE Expert, 7(6):26-32,
December 1992.
[HTTP]
HTTP - Hypertext Transfer Protocol. http://www.w3.org/pub/WWW/Protocols/.
[JEEVES]
Java Servlet Application Programming Interface White Paper. http://java.sun.com:80/products/java-server/alpha2/doc/api.html.
[Kin]
kinetoscope, inc home page. http://www.kinetoscope.com/.
[MAH96]
M. A. Hamilton. Java and the Shift to Net-Centric Computing, IEEE Computer
Society Press, Computer, 29(8):31-39, August 1996.
[MD89]
D. R. MeCartny and U. Dayal. The Architecture Of An Active Data Base Management
System., Proc. ACM SIGMOD, Portland, 1989.
[NW]
Network Wizards. http://www.nw.com/.
[RJD96]
R. Malpani, J. Lorch, and D. Berger. Making World Wide Web Caching Servers
Cooperate. In Proceedings of the fourth International Word Wide Web Conference,
1995.
[SK96]
S.E.Dossick and G.E. Kaiser. WWW Access to Legacy Client/Server Applications.
In Proceedings of the fifth International Word Wide Web Conference, May
1996, Paris.
[Tabriz]
G. Lawton. Agents to roam the Internet. http://www.sun.com/sunworldonline/swol-10-1996/swol-10-agent.html.
[TELE]
Genmagic home page. http://www.genmagic.com.
[UD90]
U. Dayal, M. Hsu and R. Ladin. Organizing Long-Running Activities with
Triggers and Transactions. Proc. ACM SIGMOD Conf. on Management of data,
1990.
[WebRuleDesign]
I. Ben-Shaul and S. Ifergan. The Design of WebRule. Technical report, Technion
- Israel Institute of Technology, Departments of Electrical Engineering,
1997.