History

Implement working crawling and section building for salt

Boris Bera committed 8 years ago

Commit b7075dd1 parent 9d60cc83 files · +25 −3Browse files

Unified · Split

3 files
  1. lib/docs/filters/salt_stack/clean_html.rb +2 −0
  2. lib/docs/filters/salt_stack/entries.rb +17 −2
  3. lib/docs/scrapers/salt_stack.rb +6 −1
lib/docs/filters/salt_stack/clean_html.rb+2−0
@@ -2,6 +2,8 @@ module Docs · 1 unmodified line
22 class SaltStack
33 class CleanHtmlFilter < Filter
44 def call
5+ css('.headerlink').remove
6+
57 doc
68 end
79 end
lib/docs/filters/salt_stack/entries.rb+17−2
@@ -1,12 +1,27 @@
11 module Docs
22 class SaltStack
33 class EntriesFilter < Docs::EntriesFilter
4+ SALT_REF_RGX = /salt\.([^\.]+)\.([^\s]+)/
5+
46 def get_name
5− at_css('h1').content
7+ header = at_css('h1').content
8+
9+ ref_match = SALT_REF_RGX.match(header)
10+ if ref_match
11+ ns, mod = ref_match.captures
12+ "#{ns}.#{mod}"
13+ else
14+ header
15+ end
616 end
717
818 def get_type
9− 'TODO'
19+ type, _ = slug.split('/', 2)
20+ type
21+ end
22+
23+ def include_default_entry?
24+ !subpath.end_with?('index.html')
1025 end
1126 end
1227 end
lib/docs/scrapers/salt_stack.rb+6−1
@@ -1,9 +1,14 @@
11 module Docs
22 class SaltStack < UrlScraper
3+ self.type = 'salt_stack'
34 self.release = '2018.3.2'
45 self.base_url = 'https://docs.saltstack.com/en/latest/ref/'
56
6− html_filters.push 'salt_stack/entries', 'salt_stack/clean_html'
7+ html_filters.push 'salt_stack/clean_html', 'salt_stack/entries'
8+
9+ options[:only_patterns] = [
10+ %r{[^/]+/all/}
11+ ]
712
813 options[:container] = '.body-content'
914

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t