edu.usfca.cs272.WebCrawler Class Reference

Public Member Functions
	WebCrawler (ThreadSafeInvertedIndex index, WorkQueue queue, String seed, int max)
void	processLink (URI seed)
void	processLink ()
String	toString ()

Static Public Attributes
static final Logger	log = LogManager.getLogger()

Detailed Description

Class responsible for crawling the web in the search engine.

Author

Ravneet Singh Bhatia

Version

Spring 2024

Constructor & Destructor Documentation

WebCrawler()

edu.usfca.cs272.WebCrawler.WebCrawler	(	ThreadSafeInvertedIndex	index,
		WorkQueue	queue,
		String	seed,
		int	max )

Creates a new WebCrawler.

Parameters

index	the invertedIndex to build.
queue	the workqueue.
seed	the seed uri.
max	the maximum number of webpages to crawl.

Member Function Documentation

processLink() [1/2]

void edu.usfca.cs272.WebCrawler.processLink

(

)

Process the seed uri

processLink() [2/2]

void edu.usfca.cs272.WebCrawler.processLink

(

URI

seed

)

Processes links. Recursively processes any links on the webpage if they haven't already been processed.

Parameters

seed	the base uri

Member Data Documentation

log

final Logger edu.usfca.cs272.WebCrawler.log = LogManager.getLogger()

static

Logger for this class.

---

The documentation for this class was generated from the following file:

• C:/USFCS/CS272/projects/project-Ravneetsb/src/main/java/edu/usfca/cs272/WebCrawler.java