edu.usfca.cs272.HtmlFetcher Class Reference

Static Public Member Functions
static boolean	isHtml (Map< String, List< String > > headers)
static int	getStatusCode (Map< String, List< String > > headers)
static String	getRedirect (Map< String, List< String > > headers)
static boolean	isRedirect (Map< String, List< String > > headers)
static String	fetch (URI uri, int redirects)
static String	fetch (String uri, int redirects)
static String	fetch (String uri)
static String	fetch (URI uri)

Static Public Attributes
static final String	statusRegex = "\\s(\\d{3})\\s"

Detailed Description

A specialized version of HttpsFetcher that follows redirects and returns HTML content if possible.

See also

HttpsFetcher

Author

CS 272 Software Development (University of San Francisco)

Version

Spring 2024

Member Function Documentation

fetch() [1/4]

static String edu.usfca.cs272.HtmlFetcher.fetch ( String uri )

static

Converts the String url into a URL object and then calls fetch(URI, int) with 0 redirects.

Parameters

uri	the URI to fetch

Returns

the HTML or null if unable to fetch valid HTML

See also

#fetch(URI, int)

fetch() [2/4]

static String edu.usfca.cs272.HtmlFetcher.fetch ( String uri, int redirects )

static

Converts the String into a URI object and then calls fetch(URI, int).

Parameters

uri	the URI to fetch
redirects	the number of times to follow redirects

Returns

the HTML or null if unable to fetch valid HTML

See also

#fetch(URI, int)

fetch() [3/4]

static String edu.usfca.cs272.HtmlFetcher.fetch ( URI uri )

static

Calls fetch(URI, int) with 0 redirects.

Parameters

uri	the URI to fetch

Returns

the HTML or null if unable to fetch valid HTML

fetch() [4/4]

static String edu.usfca.cs272.HtmlFetcher.fetch ( URI uri, int redirects )

static

Efficiently fetches HTML using HTTP/1.1 and sockets.

The HTTP body will only be fetched and processed if the status code is 200 and the content-type is HTML. In that case, the HTML will be returned as a single joined String using the .

Otherwise, the HTTP body will not be fetched. However, if the status code is a redirect, then the location of the redirect will be recursively followed up to the specified number of times. Once the number of redirects falls to 0 or lower, then redirects will no longer be followed.

If valid HTML cannot be fetched within the specified number of redirects, then null is returned.

Parameters

uri	the URI to fetch
redirects	the number of times to follow redirects

Returns

the HTML or null if unable to fetch valid HTML

See also

HttpsFetcher::openConnection(URI)

HttpsFetcher::printGetRequest(PrintWriter, URI)

HttpsFetcher::processHttpHeaders(BufferedReader)

String::join(CharSequence, CharSequence...)

System::lineSeparator()

#isHtml(Map)

#getRedirect(Map)

getRedirect()

static String edu.usfca.cs272.HtmlFetcher.getRedirect ( Map< String, List< String > > headers )

static

If the HTTP status code is between 300 and 399 (inclusive) indicating a redirect, returns the first redirect location if it is provided. Otherwise returns null.

Parameters

headers

the HTTP/1.1 headers to parse

Returns

the first redirected location if the headers indicate a redirect

See also

HttpsFetcher::processHttpHeaders(BufferedReader)

getStatusCode()

static int edu.usfca.cs272.HtmlFetcher.getStatusCode ( Map< String, List< String > > headers )

static

Parses the HTTP status code from the provided HTTP headers, assuming the status line is stored under the null key.

Parameters

headers

the HTTP/1.1 headers to parse

Returns

the HTTP status code or -1 if unable to parse for any reasons

See also

HttpsFetcher::processHttpHeaders(BufferedReader)

isHtml()

static boolean edu.usfca.cs272.HtmlFetcher.isHtml ( Map< String, List< String > > headers )

static

Returns true if and only if there is a "content-type" header (assume lowercase) and the first value of that header starts with the value "text/html" (case-insensitive).

Parameters

headers

the HTTP/1.1 headers to parse

Returns

true if the headers indicate the content type is HTML

See also

HttpsFetcher::processHttpHeaders(BufferedReader)

isRedirect()

static boolean edu.usfca.cs272.HtmlFetcher.isRedirect ( Map< String, List< String > > headers )

static

Returns true if the status code indicates that there is a redirect.

Parameters

headers

html headers.

Returns

true if the status code indicates that there is a redirect.

Member Data Documentation

statusRegex

final String edu.usfca.cs272.HtmlFetcher.statusRegex = "\\s(\\d{3})\\s"

static

Regex to find the status code in html.

---

The documentation for this class was generated from the following file:

• C:/USFCS/CS272/projects/project-Ravneetsb/src/main/java/edu/usfca/cs272/HtmlFetcher.java