public interface URLFilter
Modifier and Type | Method and Description |
---|---|
void |
configure(Map stormConf,
com.fasterxml.jackson.databind.JsonNode filterParams)
Called when this filter is being initialized
|
String |
filter(URL sourceUrl,
Metadata sourceMetadata,
String urlToFilter)
Returns null if the URL is to be removed or a normalised representation
which can correspond to the input URL
|
void configure(Map stormConf, com.fasterxml.jackson.databind.JsonNode filterParams)
stormConf
- The Storm configuration used for the ParserBoltfilterParams
- the filter specific configuration. Never nullString filter(URL sourceUrl, Metadata sourceMetadata, String urlToFilter)
sourceUrl
- the URL of the page where the URL was found. Can be null.sourceMetadata
- the metadata collected for the pageurlToFilter
- the URL to be filteredCopyright © 2018 DigitalPebble Ltd. All rights reserved.