History

basic scraping for Reference

Gergely Gombos committed 1 year ago

Commit 75cce721 parent 5e973d83 files · +37 −1Browse files

Unified · Split

3 files
  1. lib/docs/filters/react/clean_html_react_dev.rb +11 −0
  2. lib/docs/filters/react/entries_react_dev.rb +13 −0
  3. lib/docs/scrapers/react.rb +13 −1
lib/docs/filters/react/clean_html_react_dev.rb+11−0
@@ -0,0 +1,11 @@
1module Docs
2 class React
3 class CleanHtmlReactDevFilter < Filter
4 def call
5 @doc = at_css('article')
6
7 doc
8 end
9 end
10 end
11end
lib/docs/filters/react/entries_react_dev.rb+13−0
@@ -0,0 +1,13 @@
1module Docs
2 class React
3 class EntriesReactDevFilter < Docs::EntriesFilter
4 def get_name
5 at_css('article h1').content
6 end
7
8 def get_type
9 return 'TODO add types'
10 end
11 end
12 end
13end
lib/docs/scrapers/react.rb+13−1
@@ -7,6 +7,19 @@ module Docs · 6 unmodified lines
7 code: 'https://github.com/facebook/react'7 code: 'https://github.com/facebook/react'
8 }8 }
9 9
10 version do
11 self.release = '18.3.1'
12 # TODO add /learn
13 self.base_url = 'https://react.dev/reference'
14
15 html_filters.push 'react/entries_react_dev', 'react/clean_html_react_dev'
16
17 options[:attribution] = <<-HTML
18 &copy; 2013&ndash;present Facebook Inc.<br>
19 Licensed under the Creative Commons Attribution 4.0 International Public License.
20 HTML
21 end
22
10 version '17' do23 version '17' do
11 self.release = '17.0.2'24 self.release = '17.0.2'
12 self.base_url = 'https://17.reactjs.org/docs/'25 self.base_url = 'https://17.reactjs.org/docs/'
@@ -33,7 +46,6 @@ module Docs · 20 unmodified lines
33 HTML46 HTML
34 end47 end
35 48
36
37 def get_latest_version(opts)49 def get_latest_version(opts)
38 doc = fetch_doc('https://react.dev/', opts)50 doc = fetch_doc('https://react.dev/', opts)
39 doc.at_css('a[href="/versions"]').content.strip[1..-1]51 doc.at_css('a[href="/versions"]').content.strip[1..-1]

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t