1. INTRODUCTION

The Language Observatory was planned primarily to provide means for assessing the usage level of each language in cyberspace. Using UbiCrawler [2], a scalable fullydistributed web crawler developed for use in the project, we have collected a total of 84.6 million web pages from more than 142 thousand sub-domains of 60 country code Top Level Domains (ccTLDs) of Africa. The information downloaded by the crawler is enormous, and only a small portion of it is presented in this paper. This endeavor has proven to be beneficial to individuals and organizations conducting research and developing solutions on bridging the digital divide, such as UNESCO and ACALAN (African Academy of Languages).

2. SURVEY STATISTICS

During our most recent crawling, in December 2005, we have collected 84.6 million pages from the whole of Africa, among them South Africa alone occupies more than 80%. The growth of the pages during the last three years reached as high as 251.17% per year. Due to the space limitation, the paper picks up only a few findings from the survey. For more detailed survey statistics data, refer to the project website [5].

Note: 2002 data are taken from [1]

2.1 Accessibility of the Web Pages

We have analyzed responses to HTTP requests to get an insight into African web servers' accessibility and possible network-related problems. Top Level Domains with network-related problems are Somalia and Guinea-Bissau, from which all pages were unavailable during the experiment. On the other hand, pages stored on the web servers registered in Benin, Gambia, and Libyan Arab Jamahiriya maintained a high success rate of 95-99% even as their number of pages stored in the web server increased by almost 50% in a year. Coincidentally, responses to requests of web pages from Mali and Djibouti web spaces had similar high success rates, their number of pages with even more increases at 75% and 137%, respectively. There was almost no problem in downloading pages from the web space of the following African countries: Democratic Republic of Congo, Reunion Island, Eritrea, Burundi, Seychelles, and Rwanda.

2.2 Web Tree Growth

There is a complete freedom in how Africa's ccTLDs allocate their sub domains. We are analyzing how the African web tree is organized. As an example, during our experiment, Morocco topped the list with 801 second-level sub-domains, followed by Sao Tome & Principe and Senegal with 353 and 315 second-level sub-domains, respectively. On the other hand, Mozambique and Nigeria had no change in their number of second-level sub-domains from 2004 to 2005.

Out of the 142 thousand domains which our crawler has visited in the African web space, Libyan Arab Jamahiriya's number of domains multiplied 5 times in a period of 1 year, which indicates an exponential growth of the internet in the region. At the same time, the current domain count for Gambia almost doubled compared to the previous year's count. Almost all countries had increases in their number of domains, except for some special cases like Somalia and Guinea Bissau.

2.3 Open Source vs Proprietary Technology

A report about the African web in 2002 [1] showed that 56% of the websites use Microsoft technology. However, in the current survey, Microsoft served in only 22.82% of the pages. With the decrease, it became second only to Apache (71.47%), which formerly occupied 37.95% of the African market.

Table 2 shows that the global trend of preference to Apache-like open source server software applies to whole Africa. A large number of pages were created by freely available PHP technology (55.96%) as compared to Microsoft's Active Server Pages (ASP, 15.36%). On the contrary, the 2002 report indicated ASP as the most common file type.

2.4 Persistent and Session Cookie Usage

Monitoring the use of cookies and pointing to their inappropriate use can help protect the privacy of users from abuse [3]. The survey showed that Africa had a low rate of 18.73% of persistent cookies and a relatively high rate of session cookies (42.91%). One possible interpretation of this contradictory fact could be that there are not many e-commerce servers that tend to intensively use persistent cookies to monitor web surfing patterns of the users (typically shown in the case of Saint Helena, one of web hosting server domain). There could also be a few commercial servers that don't use session cookies.

2.5 Interconnection

The interconnection of the countries of Africa was analyzed through the number of outgoing links from each downloaded web page, where the total number of outgoing links is 3.9 billion. We are interested not only in the internal links (links to the web pages within the African ccTLDs), but also the external links (links to web pages outside the African ccTLDs). All the countries, except British Indian Ocean Territories and Zambia, had a high rate of internal interconnection. British Indian Ocean Territories had more links to .com (53.04%) domain, while Zambia had 42.62% of its links distributed among the generic TLDs. For all TLDs of Africa, internal links is 95.19%, while external links is 4.81%. Figure 1 shows the distribution of the external links of all the African ccTLDs. Interested readers can find much more detailed web graph data on our website [5].

3. CONCLUSION

In this paper, we outlined the status of the African web, how it is growing and adapting to the WWW bandwagon, and which countries are experiencing difficulties. In terms of server technologies, the global trend from proprietary technologies to open source technologies is observed in Africa too. Also, we have indicated the profile of servers and users through the analysis of cookie usage of the servers. The analysis of interconnection of African web has shown that African web is mostly connected within its country domain, and only a few outgoing links goes outside of Africa. The African web is not well connected with outside world.

On our website [5] any interested reader can find additional data which were not possible to include in this paper due to the space limitations.

4. ACKNOWLEDGMENTS

The study was made possible by the financial support of the Japan Science and Technology Agency (JST) under the RISTEX program. We also thank UNESCO for giving official support to the project since its inception.