Bibliography

1

Z. Bar-Yossef, A. Z. Broder, R. Kumar, and A. Tomkins.
Sic Transit Gloria Telae: Towards an Understanding of the Web's Decay.
In Proc. WWW, 2004.

2

A. Z. Broder, S. C. Glassman, and M. S. Manasse.
Syntactic clustering of the web.
In Proc. WWW, 1997.

3

J. Cho and H. Garcia-Molina.
The Evolution of the Web and Implications for an Incremental Crawler.
In Proc. VLDB, 2000.

4

J. Cho and H. Garcia-Molina.
Effective Page Refresh Policies for Web Crawlers.
ACM Transactions on Database Systems, 28(4), 2003.

5

J. Cho and H. Garcia-Molina.
Estimating frequency of change.
ACM Transcations on Internet Technology, 3(3), 2003.

6

E. Coffman, Z. Liu, and R. R. Weber.
Optimal robot scheduling for web search engines.
Journal of Scheduling, 1, 1998.

7

J. Edwards, K. S. McCurley, and J. A. Tomlin.
An Adaptive Model for Optimizing Performance of an Incremental Web Crawler.
In Proc. WWW, 2001.

8

D. Fetterly, M. Manasse, M. Najork, and J. L. Wiener.
A large-scale study of the evolution of web pages.
In Proc. WWW, 2003.

9

A. Ntoulas, J. Cho, and C. Olston.
What's New on the Web? The Evolution of the Web from a Search Engine Perspective.
In Proc. WWW, 2004.

10

C. Olston and J. Widom.
Best-effort cache synchronization with source cooperation.
In Proc. ACM SIGMOD, 2002.

11

The Open Directory Project.
http://dmoz.org.

12

S. Pandey and C. Olston.
User-centric web crawling.
In Proc. WWW, 2005.

13

J. Wolf, M. Squillante, P.S.Yu, J.Sethuraman, and L. Ozsen.
Optimal Crawling Strategies for Web Search Engines.
In Proc. WWW, 2002.