Unified · Split
| @@ -57,7 +57,7 @@ Configuration is done via class attributes and divided into three main categorie · 56 unmodified lines | ||
| 57 | 57 | * `version` [String] **(required)** |
| 58 | 58 | The version of the software at the time the scraper was last run. This is only informational and doesn't affect the scraper's behavior. |
| 59 | 59 | |
| 60 | −* `base_url` [String] **(required)** | |
| 60 | +* `base_url` [String] **(required in `UrlScraper`)** | |
| 61 | 61 | The documents' location. Only URLs _inside_ the `base_url` will be scraped. "inside" more or less means "starting with" except that `/docs` is outside `/doc` (but `/doc/` is inside). |
| 62 | 62 | Unless `root_path` is set, the root/initial URL is equal to `base_url`. |
| 63 | 63 | |
| @@ -100,6 +100,7 @@ Default `html_filters`: · 36 unmodified lines | ||
| 100 | 100 | * [`NormalizeUrlsFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/normalize_urls.rb) — replaces all URLs with their fully qualified counterpart |
| 101 | 101 | * [`InternalUrlsFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/internal_urls.rb) — detects internal URLs (the ones to scrape) and replaces them with their unqualified, relative counterpart |
| 102 | 102 | * [`NormalizePathsFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/normalize_paths.rb) — makes the internal paths consistent (e.g. always end with `.html`) |
| 103 | +* [`CleanLocalUrlsFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/clean_local_urls.rb) — remove links, iframes and images pointing to localhost (`FileScraper` only) | |
| 103 | 104 | |
| 104 | 105 | Default `text_filters`: |
| 105 | 106 | |