Automatically check for documentation updates
Unified · Split
| @@ -61,7 +61,7 @@ In addition to the [guidelines for contributing code](#contributing-code-and-fea · 60 unmodified lines | ||
| 61 | 61 | |
| 62 | 62 | Please don't submit a pull request updating the version number of a documentation, unless a change is required in the scraper and you've verified that it works. |
| 63 | 63 | |
| 64 | −To ask that an existing documentation be updated, please use the [Trello board](https://trello.com/c/2B0hmW7M/52-request-updates-here). | |
| 64 | +To ask that an existing documentation be updated, first check the last two [Documentation versions reports](https://github.com/freeCodeCamp/devdocs/issues?utf8=%E2%9C%93&q=Documentation+versions+report+is%3Aissue+author%3Adevdocs-bot+sort%3Aupdated-desc). Only create an issue if the documentation has been wrongly marked as up-to-date. | |
| 65 | 65 | |
| 66 | 66 | ## Coding conventions |
| 67 | 67 | |
| @@ -6,6 +6,10 @@ before_script: · 5 unmodified lines | ||
| 6 | 6 | - gem update --system |
| 7 | 7 | - gem install bundler |
| 8 | 8 | |
| 9 | +script: | |
| 10 | + - if [ "$TRAVIS_EVENT_TYPE" != "cron" ]; then bundle exec rake; fi | |
| 11 | + - if [ "$TRAVIS_EVENT_TYPE" = "cron" ]; then bundle exec thor updates:check --github-token $GH_TOKEN --upload; fi | |
| 12 | + | |
| 9 | 13 | deploy: |
| 10 | 14 | provider: heroku |
| 11 | 15 | app: devdocs |
| @@ -42,6 +42,7 @@ group :docs do · 41 unmodified lines | ||
| 42 | 42 | gem 'unix_utils', require: false |
| 43 | 43 | gem 'tty-pager', require: false |
| 44 | 44 | gem 'net-sftp', '>= 2.1.3.rc2', require: false |
| 45 | + gem 'terminal-table', require: false | |
| 45 | 46 | end |
| 46 | 47 | |
| 47 | 48 | group :test do |
| @@ -104,6 +104,8 @@ GEM · 103 unmodified lines | ||
| 104 | 104 | unicode-display_width (~> 1.4.0) |
| 105 | 105 | unicode_utils (~> 1.4.0) |
| 106 | 106 | strings-ansi (0.1.0) |
| 107 | + terminal-table (1.8.0) | |
| 108 | + unicode-display_width (~> 1.1, >= 1.1.1) | |
| 107 | 109 | thin (1.7.2) |
| 108 | 110 | daemons (~> 1.0, >= 1.0.9) |
| 109 | 111 | eventmachine (~> 1.0, >= 1.0.4) |
| @@ -157,6 +159,7 @@ DEPENDENCIES · 47 unmodified lines | ||
| 157 | 159 | sinatra-contrib |
| 158 | 160 | sprockets |
| 159 | 161 | sprockets-helpers |
| 162 | + terminal-table | |
| 160 | 163 | sprockets-sass |
| 161 | 164 | thin |
| 162 | 165 | thor |
| @@ -17,6 +17,7 @@ Adding a documentation may look like a daunting task but once you get the hang o · 16 unmodified lines | ||
| 17 | 17 | 10. To add syntax highlighting or execute custom JavaScript on the pages, create a file in the `assets/javascripts/views/pages/` directory (take a look at the other files to see how it works). |
| 18 | 18 | 11. Add the documentation's icon in the `public/icons/docs/[my_doc]/` directory, in both 16x16 and 32x32-pixels formats. It'll be added to the icon spritesheet after your pull request is merged. |
| 19 | 19 | 12. Add the documentation's copyright details to the list in `assets/javascripts/templates/pages/about_tmpl.coffee`. This is the data shown in the table on the [about](https://devdocs.io/about) page, and is ordered alphabetically. Simply copying an existing item, placing it in the right slot and updating the values to match the new scraper will do the job. |
| 20 | +13. Ensure `thor updates:check [my_doc]` shows the correct latest version. | |
| 20 | 21 | |
| 21 | 22 | If the documentation includes more than a few hundreds pages and is available for download, try to scrape it locally (e.g. using `FileScraper`). It'll make the development process much faster and avoids putting too much load on the source site. (It's not a problem if your scraper is coupled to your local setup, just explain how it works in your pull request.) |
| 22 | 23 | |
| @@ -184,3 +184,44 @@ More information about how filters work is available on the [Filter Reference](. · 183 unmodified lines | ||
| 184 | 184 | Overrides the `:title` option for the root page only. |
| 185 | 185 | |
| 186 | 186 | _Note: this filter is disabled by default._ |
| 187 | + | |
| 188 | +## Keeping scrapers up-to-date | |
| 189 | + | |
| 190 | +In order to keep scrapers up-to-date the `get_latest_version(opts)` method should be overridden. If `self.release` is defined, this should return the latest version of the documentation. If `self.release` is not defined, it should return the Epoch time when the documentation was last modified. If the documentation will never change, simply return `1.0.0`. The result of this method is periodically reported in a "Documentation versions report" issue which helps maintainers keep track of outdated documentations. | |
| 191 | + | |
| 192 | +To make life easier, there are a few utility methods that you can use in `get_latest_version`: | |
| 193 | +* `fetch(url, opts)` | |
| 194 | + | |
| 195 | + Makes a GET request to the url and returns the response body. | |
| 196 | + | |
| 197 | + Example: [lib/docs/scrapers/bash.rb](../lib/docs/scrapers/bash.rb) | |
| 198 | +* `fetch_doc(url, opts)` | |
| 199 | + | |
| 200 | + Makes a GET request to the url and returns the HTML body converted to a Nokogiri document. | |
| 201 | + | |
| 202 | + Example: [lib/docs/scrapers/git.rb](../lib/docs/scrapers/git.rb) | |
| 203 | +* `fetch_json(url, opts)` | |
| 204 | + | |
| 205 | + Makes a GET request to the url and returns the JSON body converted to a dictionary. | |
| 206 | + | |
| 207 | + Example: [lib/docs/scrapers/mdn/mdn.rb](../lib/docs/scrapers/mdn/mdn.rb) | |
| 208 | +* `get_npm_version(package, opts)` | |
| 209 | + | |
| 210 | + Returns the latest version of the given npm package. | |
| 211 | + | |
| 212 | + Example: [lib/docs/scrapers/bower.rb](../lib/docs/scrapers/bower.rb) | |
| 213 | +* `get_latest_github_release(owner, repo, opts)` | |
| 214 | + | |
| 215 | + Returns the tag name of the latest GitHub release of the given repository. If the tag name is preceded by a "v", the "v" will be removed. | |
| 216 | + | |
| 217 | + Example: [lib/docs/scrapers/jsdoc.rb](../lib/docs/scrapers/jsdoc.rb) | |
| 218 | +* `get_github_tags(owner, repo, opts)` | |
| 219 | + | |
| 220 | + Returns the list of tags on the given repository ([format](https://developer.github.com/v3/repos/#list-tags)). | |
| 221 | + | |
| 222 | + Example: [lib/docs/scrapers/liquid.rb](../lib/docs/scrapers/liquid.rb) | |
| 223 | +* `get_github_file_contents(owner, repo, path, opts)` | |
| 224 | + | |
| 225 | + Returns the contents of the requested file in the default branch of the given repository. | |
| 226 | + | |
| 227 | + Example: [lib/docs/scrapers/minitest.rb](../lib/docs/scrapers/minitest.rb) | |
| @@ -152,7 +152,6 @@ module Docs · 151 unmodified lines | ||
| 152 | 152 | end |
| 153 | 153 | end |
| 154 | 154 | |
| 155 | − | |
| 156 | 155 | def initialize |
| 157 | 156 | raise NotImplementedError, "#{self.class} is an abstract class and cannot be instantiated." if self.class.abstract |
| 158 | 157 | end |
| @@ -164,5 +163,104 @@ module Docs · 5 unmodified lines | ||
| 164 | 163 | def build_pages(&block) |
| 165 | 164 | raise NotImplementedError |
| 166 | 165 | end |
| 166 | + | |
| 167 | + def get_scraper_version(opts) | |
| 168 | + if self.class.method_defined?(:options) and !options[:release].nil? | |
| 169 | + options[:release] | |
| 170 | + else | |
| 171 | + # If options[:release] does not exist, we return the Epoch timestamp of when the doc was last modified in DevDocs production | |
| 172 | + json = fetch_json('https://devdocs.io/docs.json', opts) | |
| 173 | + items = json.select {|item| item['name'] == self.class.name} | |
| 174 | + items = items.map {|item| item['mtime']} | |
| 175 | + items.max | |
| 176 | + end | |
| 177 | + end | |
| 178 | + | |
| 179 | + # Should return the latest version of this documentation | |
| 180 | + # If options[:release] is defined, it should be in the same format | |
| 181 | + # If options[:release] is not defined, it should return the Epoch timestamp of when the documentation was last updated | |
| 182 | + # If the docs will never change, simply return '1.0.0' | |
| 183 | + def get_latest_version(opts) | |
| 184 | + raise NotImplementedError | |
| 185 | + end | |
| 186 | + | |
| 187 | + # Returns whether or not this scraper is outdated. | |
| 188 | + # | |
| 189 | + # The default implementation assumes the documentation uses a semver(-like) approach when it comes to versions. | |
| 190 | + # Patch updates are ignored because there are usually little to no documentation changes in bug-fix-only releases. | |
| 191 | + # | |
| 192 | + # Scrapers of documentations that do not use this versioning approach should override this method. | |
| 193 | + # | |
| 194 | + # Examples of the default implementation: | |
| 195 | + # 1 -> 2 = outdated | |
| 196 | + # 1.1 -> 1.2 = outdated | |
| 197 | + # 1.1.1 -> 1.1.2 = not outdated | |
| 198 | + def is_outdated(scraper_version, latest_version) | |
| 199 | + scraper_parts = scraper_version.to_s.split(/\./).map(&:to_i) | |
| 200 | + latest_parts = latest_version.to_s.split(/\./).map(&:to_i) | |
| 201 | + | |
| 202 | + # Only check the first two parts, the third part is for patch updates | |
| 203 | + [0, 1].each do |i| | |
| 204 | + break if i >= scraper_parts.length or i >= latest_parts.length | |
| 205 | + return true if latest_parts[i] > scraper_parts[i] | |
| 206 | + return false if latest_parts[i] < scraper_parts[i] | |
| 207 | + end | |
| 208 | + | |
| 209 | + false | |
| 210 | + end | |
| 211 | + | |
| 212 | + private | |
| 213 | + | |
| 214 | + # | |
| 215 | + # Utility methods for get_latest_version | |
| 216 | + # | |
| 217 | + | |
| 218 | + def fetch(url, opts) | |
| 219 | + headers = {} | |
| 220 | + | |
| 221 | + if opts.key?(:github_token) and url.start_with?('https://api.github.com/') | |
| 222 | + headers['Authorization'] = "token #{opts[:github_token]}" | |
| 223 | + end | |
| 224 | + | |
| 225 | + opts[:logger].debug("Fetching #{url}") | |
| 226 | + response = Request.run(url, { connecttimeout: 15, headers: headers }) | |
| 227 | + | |
| 228 | + if response.success? | |
| 229 | + response.body | |
| 230 | + else | |
| 231 | + reason = response.timed_out? ? "Timed out while connecting to #{url}" : "Couldn't fetch #{url} (response code #{response.code})" | |
| 232 | + opts[:logger].error(reason) | |
| 233 | + raise reason | |
| 234 | + end | |
| 235 | + end | |
| 236 | + | |
| 237 | + def fetch_doc(url, opts) | |
| 238 | + body = fetch(url, opts) | |
| 239 | + Nokogiri::HTML.parse(body, nil, 'UTF-8') | |
| 240 | + end | |
| 241 | + | |
| 242 | + def fetch_json(url, opts) | |
| 243 | + JSON.parse fetch(url, opts) | |
| 244 | + end | |
| 245 | + | |
| 246 | + def get_npm_version(package, opts) | |
| 247 | + json = fetch_json("https://registry.npmjs.com/#{package}", opts) | |
| 248 | + json['dist-tags']['latest'] | |
| 249 | + end | |
| 250 | + | |
| 251 | + def get_latest_github_release(owner, repo, opts) | |
| 252 | + release = fetch_json("https://api.github.com/repos/#{owner}/#{repo}/releases/latest", opts) | |
| 253 | + tag_name = release['tag_name'] | |
| 254 | + tag_name.start_with?('v') ? tag_name[1..-1] : tag_name | |
| 255 | + end | |
| 256 | + | |
| 257 | + def get_github_tags(owner, repo, opts) | |
| 258 | + fetch_json("https://api.github.com/repos/#{owner}/#{repo}/tags", opts) | |
| 259 | + end | |
| 260 | + | |
| 261 | + def get_github_file_contents(owner, repo, path, opts) | |
| 262 | + json = fetch_json("https://api.github.com/repos/#{owner}/#{repo}/contents/#{path}", opts) | |
| 263 | + Base64.decode64(json['content']) | |
| 264 | + end | |
| 167 | 265 | end |
| 168 | 266 | end |
| @@ -155,6 +155,10 @@ module Docs · 154 unmodified lines | ||
| 155 | 155 | end |
| 156 | 156 | end |
| 157 | 157 | |
| 158 | + def get_latest_version(opts) | |
| 159 | + get_npm_version('@angular/core', opts) | |
| 160 | + end | |
| 161 | + | |
| 158 | 162 | private |
| 159 | 163 | |
| 160 | 164 | def parse(response) |
| @@ -69,5 +69,9 @@ module Docs · 68 unmodified lines | ||
| 69 | 69 | self.release = '1.2.32' |
| 70 | 70 | self.base_url = "https://code.angularjs.org/#{release}/docs/partials/" |
| 71 | 71 | end |
| 72 | + | |
| 73 | + def get_latest_version(opts) | |
| 74 | + get_npm_version('angular', opts) | |
| 75 | + end | |
| 72 | 76 | end |
| 73 | 77 | end |
| @@ -87,5 +87,10 @@ module Docs · 86 unmodified lines | ||
| 87 | 87 | quickstart.html |
| 88 | 88 | list_of_all_modules.html) |
| 89 | 89 | end |
| 90 | + | |
| 91 | + def get_latest_version(opts) | |
| 92 | + doc = fetch_doc('https://docs.ansible.com/ansible/latest/index.html', opts) | |
| 93 | + doc.at_css('.DocSiteProduct-CurrentVersion').content.strip | |
| 94 | + end | |
| 90 | 95 | end |
| 91 | 96 | end |
| @@ -33,5 +33,10 @@ module Docs · 32 unmodified lines | ||
| 33 | 33 | © 2018 The Apache Software Foundation<br> |
| 34 | 34 | Licensed under the Apache License, Version 2.0. |
| 35 | 35 | HTML |
| 36 | + | |
| 37 | + def get_latest_version(opts) | |
| 38 | + doc = fetch_doc('http://httpd.apache.org/docs/', opts) | |
| 39 | + doc.at_css('#apcontents > ul a')['href'][0...-1] | |
| 40 | + end | |
| 36 | 41 | end |
| 37 | 42 | end |
| @@ -43,5 +43,10 @@ module Docs · 42 unmodified lines | ||
| 43 | 43 | self.base_url = "https://pig.apache.org/docs/r#{release}/" |
| 44 | 44 | end |
| 45 | 45 | |
| 46 | + def get_latest_version(opts) | |
| 47 | + doc = fetch_doc('https://pig.apache.org/', opts) | |
| 48 | + item = doc.at_css('div[id="menu_1.2"] > .menuitem:last-child') | |
| 49 | + item.content.strip.sub(/Release /, '') | |
| 50 | + end | |
| 46 | 51 | end |
| 47 | 52 | end |
| @@ -17,5 +17,10 @@ module Docs · 16 unmodified lines | ||
| 17 | 17 | © 2010–2018 Caolan McMahon<br> |
| 18 | 18 | Licensed under the MIT License. |
| 19 | 19 | HTML |
| 20 | + | |
| 21 | + def get_latest_version(opts) | |
| 22 | + doc = fetch_doc('https://caolan.github.io/async/v3/', opts) | |
| 23 | + doc.at_css('#version-dropdown > a').content.strip[1..-1] | |
| 24 | + end | |
| 20 | 25 | end |
| 21 | 26 | end |
| @@ -22,5 +22,10 @@ module Docs · 21 unmodified lines | ||
| 22 | 22 | stub '' do |
| 23 | 23 | '<div></div>' |
| 24 | 24 | end |
| 25 | + | |
| 26 | + def get_latest_version(opts) | |
| 27 | + doc = fetch_doc('https://babeljs.io/docs/en/', opts) | |
| 28 | + doc.at_css('a[href="/versions"] > h3').content | |
| 29 | + end | |
| 25 | 30 | end |
| 26 | 31 | end |
| @@ -20,5 +20,10 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | © 2010–2016 Jeremy Ashkenas, DocumentCloud<br> |
| 21 | 21 | Licensed under the MIT License. |
| 22 | 22 | HTML |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + doc = fetch_doc('https://backbonejs.org/', opts) | |
| 26 | + doc.at_css('.version').content[1...-1] | |
| 27 | + end | |
| 23 | 28 | end |
| 24 | 29 | end |
| @@ -17,5 +17,10 @@ module Docs · 16 unmodified lines | ||
| 17 | 17 | Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc.<br> |
| 18 | 18 | Licensed under the GNU Free Documentation License. |
| 19 | 19 | HTML |
| 20 | + | |
| 21 | + def get_latest_version(opts) | |
| 22 | + body = fetch('https://www.gnu.org/software/bash/manual/html_node/index.html', opts) | |
| 23 | + body.scan(/, Version ([0-9.]+)/)[0][0][0...-1] | |
| 24 | + end | |
| 20 | 25 | end |
| 21 | 26 | end |
| @@ -18,5 +18,9 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2013–2017 Petka Antonov<br> |
| 19 | 19 | Licensed under the MIT License. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + get_npm_version('bluebird', opts) | |
| 24 | + end | |
| 21 | 25 | end |
| 22 | 26 | end |
| @@ -34,5 +34,10 @@ module Docs · 33 unmodified lines | ||
| 34 | 34 | |
| 35 | 35 | options[:only] = %w(getting-started/ css/ components/ javascript/) |
| 36 | 36 | end |
| 37 | + | |
| 38 | + def get_latest_version(opts) | |
| 39 | + doc = fetch_doc('https://getbootstrap.com/', opts) | |
| 40 | + doc.at_css('#bd-versions').content.strip[1..-1] | |
| 41 | + end | |
| 37 | 42 | end |
| 38 | 43 | end |
| @@ -27,5 +27,11 @@ module Docs · 26 unmodified lines | ||
| 27 | 27 | self.release = '0.11.7' |
| 28 | 28 | self.base_url = "https://bottlepy.org/docs/#{self.version}/" |
| 29 | 29 | end |
| 30 | + | |
| 31 | + def get_latest_version(opts) | |
| 32 | + doc = fetch_doc('https://bottlepy.org/docs/stable/', opts) | |
| 33 | + label = doc.at_css('.sphinxsidebarwrapper > ul > li > b') | |
| 34 | + label.content.sub(/Bottle /, '') | |
| 35 | + end | |
| 30 | 36 | end |
| 31 | 37 | end |
| @@ -19,5 +19,9 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | © 2018 Bower contributors<br> |
| 20 | 20 | Licensed under the MIT License. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + get_npm_version('bower', opts) | |
| 25 | + end | |
| 22 | 26 | end |
| 23 | 27 | end |
| @@ -26,6 +26,13 @@ module Docs · 25 unmodified lines | ||
| 26 | 26 | Licensed under the Creative Commons Attribution-ShareAlike Unported License v3.0. |
| 27 | 27 | HTML |
| 28 | 28 | |
| 29 | + def get_latest_version(opts) | |
| 30 | + doc = fetch_doc('https://en.cppreference.com/w/Cppreference:Archives', opts) | |
| 31 | + link = doc.at_css('a[title^="File:"]') | |
| 32 | + date = link.content.scan(/(\d+)\./)[0][0] | |
| 33 | + DateTime.strptime(date, '%Y%m%d').to_time.to_i | |
| 34 | + end | |
| 35 | + | |
| 29 | 36 | private |
| 30 | 37 | |
| 31 | 38 | def file_path_for(*) |
| @@ -71,6 +71,11 @@ module Docs · 70 unmodified lines | ||
| 71 | 71 | self.base_url = 'https://api.cakephp.org/2.7/' |
| 72 | 72 | end |
| 73 | 73 | |
| 74 | + def get_latest_version(opts) | |
| 75 | + doc = fetch_doc('https://api.cakephp.org/3.7/', opts) | |
| 76 | + doc.at_css('.version-picker .dropdown-toggle').content.strip | |
| 77 | + end | |
| 78 | + | |
| 74 | 79 | private |
| 75 | 80 | |
| 76 | 81 | def parse(response) |
| @@ -23,5 +23,9 @@ module Docs · 22 unmodified lines | ||
| 23 | 23 | © 2016 Chai.js Assertion Library<br> |
| 24 | 24 | Licensed under the MIT License. |
| 25 | 25 | HTML |
| 26 | + | |
| 27 | + def get_latest_version(opts) | |
| 28 | + get_npm_version('chai', opts) | |
| 29 | + end | |
| 26 | 30 | end |
| 27 | 31 | end |
| @@ -47,5 +47,10 @@ module Docs · 46 unmodified lines | ||
| 47 | 47 | |
| 48 | 48 | options[:only_patterns] = [/\A#{client_path}\//, /\A#{server_path}\//] |
| 49 | 49 | end |
| 50 | + | |
| 51 | + def get_latest_version(opts) | |
| 52 | + doc = fetch_doc('https://downloads.chef.io/chef', opts) | |
| 53 | + doc.at_css('h1.product-heading > span').content.strip | |
| 54 | + end | |
| 50 | 55 | end |
| 51 | 56 | end |
| @@ -27,5 +27,10 @@ module Docs · 26 unmodified lines | ||
| 27 | 27 | self.release = '1.7' |
| 28 | 28 | self.base_url = 'https://clojure.github.io/clojure/branch-clojure-1.7.0/' |
| 29 | 29 | end |
| 30 | + | |
| 31 | + def get_latest_version(opts) | |
| 32 | + doc = fetch_doc('http://clojure.github.io/clojure/index.html', opts) | |
| 33 | + doc.at_css('#header-version').content[1..-1] | |
| 34 | + end | |
| 30 | 35 | end |
| 31 | 36 | end |
| @@ -59,5 +59,11 @@ module Docs · 58 unmodified lines | ||
| 59 | 59 | self.release = '3.5.2' |
| 60 | 60 | self.base_url = 'https://cmake.org/cmake/help/v3.5/' |
| 61 | 61 | end |
| 62 | + | |
| 63 | + def get_latest_version(opts) | |
| 64 | + doc = fetch_doc('https://cmake.org/documentation/', opts) | |
| 65 | + link = doc.at_css('.entry-content ul > li > strong > a > big') | |
| 66 | + link.content.scan(/([0-9.]+)/)[0][0] | |
| 67 | + end | |
| 62 | 68 | end |
| 63 | 69 | end |
| @@ -18,5 +18,10 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2011 Michael Bodnarchuk and contributors<br> |
| 19 | 19 | Licensed under the MIT License. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + doc = fetch_doc('https://codeception.com/changelog', opts) | |
| 24 | + doc.at_css('#page > h4').content | |
| 25 | + end | |
| 21 | 26 | end |
| 22 | 27 | end |
| @@ -21,5 +21,9 @@ module Docs · 20 unmodified lines | ||
| 21 | 21 | © 2015 DavertMik <davert@codegyre.com> (http://codegyre.com)<br> |
| 22 | 22 | Licensed under the MIT License. |
| 23 | 23 | HTML |
| 24 | + | |
| 25 | + def get_latest_version(opts) | |
| 26 | + get_npm_version('codeceptjs', opts) | |
| 27 | + end | |
| 24 | 28 | end |
| 25 | 29 | end |
| @@ -38,5 +38,11 @@ module Docs · 37 unmodified lines | ||
| 38 | 38 | version '3' do |
| 39 | 39 | self.release = '3.1.8' |
| 40 | 40 | end |
| 41 | + | |
| 42 | + def get_latest_version(opts) | |
| 43 | + doc = fetch_doc('https://codeigniter.com/user_guide/changelog.html', opts) | |
| 44 | + header = doc.at_css('#change-log h2') | |
| 45 | + header.content.scan(/([0-9.]+)/)[0][0] | |
| 46 | + end | |
| 41 | 47 | end |
| 42 | 48 | end |
| @@ -30,5 +30,9 @@ module Docs · 29 unmodified lines | ||
| 30 | 30 | |
| 31 | 31 | options[:container] = '.container' |
| 32 | 32 | end |
| 33 | + | |
| 34 | + def get_latest_version(opts) | |
| 35 | + get_npm_version('coffeescript', opts) | |
| 36 | + end | |
| 33 | 37 | end |
| 34 | 38 | end |
| @@ -42,5 +42,15 @@ module Docs · 41 unmodified lines | ||
| 42 | 42 | self.release = '6.5.0' |
| 43 | 43 | self.base_url = 'https://cordova.apache.org/docs/en/6.x/' |
| 44 | 44 | end |
| 45 | + | |
| 46 | + def get_latest_version(opts) | |
| 47 | + doc = fetch_doc('https://cordova.apache.org/docs/en/latest/', opts) | |
| 48 | + | |
| 49 | + label = doc.at_css('#versionDropdown').content.strip | |
| 50 | + version = label.scan(/([0-9.]+)/)[0][0] | |
| 51 | + version = version[0...-1] if version.end_with?('.') | |
| 52 | + | |
| 53 | + version | |
| 54 | + end | |
| 45 | 55 | end |
| 46 | 56 | end |
| @@ -34,6 +34,14 @@ module Docs · 33 unmodified lines | ||
| 34 | 34 | Licensed under the Creative Commons Attribution-ShareAlike Unported License v3.0. |
| 35 | 35 | HTML |
| 36 | 36 | |
| 37 | + # Same as get_latest_version in lib/docs/scrapers/c.rb | |
| 38 | + def get_latest_version(opts) | |
| 39 | + doc = fetch_doc('https://en.cppreference.com/w/Cppreference:Archives', opts) | |
| 40 | + link = doc.at_css('a[title^="File:"]') | |
| 41 | + date = link.content.scan(/(\d+)\./)[0][0] | |
| 42 | + DateTime.strptime(date, '%Y%m%d').to_time.to_i | |
| 43 | + end | |
| 44 | + | |
| 37 | 45 | private |
| 38 | 46 | |
| 39 | 47 | def file_path_for(*) |
| @@ -34,5 +34,10 @@ module Docs · 33 unmodified lines | ||
| 34 | 34 | HTML |
| 35 | 35 | end |
| 36 | 36 | } |
| 37 | + | |
| 38 | + def get_latest_version(opts) | |
| 39 | + body = fetch('https://crystal-lang.org/api', opts) | |
| 40 | + body.scan(/Crystal Docs ([0-9.]+)/)[0][0] | |
| 41 | + end | |
| 37 | 42 | end |
| 38 | 43 | end |
| @@ -26,5 +26,10 @@ module Docs · 25 unmodified lines | ||
| 26 | 26 | def initial_urls |
| 27 | 27 | %w(https://dlang.org/phobos/index.html https://dlang.org/spec/intro.html) |
| 28 | 28 | end |
| 29 | + | |
| 30 | + def get_latest_version(opts) | |
| 31 | + doc = fetch_doc('https://dlang.org/changelog/', opts) | |
| 32 | + doc.at_css('#content > ul > li:nth-child(2) > a')['id'] | |
| 33 | + end | |
| 29 | 34 | end |
| 30 | 35 | end |
| @@ -58,5 +58,9 @@ module Docs · 57 unmodified lines | ||
| 58 | 58 | options[:root_title] = 'D3.js' |
| 59 | 59 | options[:only_patterns] = [/\.md\z/] |
| 60 | 60 | end |
| 61 | + | |
| 62 | + def get_latest_version(opts) | |
| 63 | + get_npm_version('d3', opts) | |
| 64 | + end | |
| 61 | 65 | end |
| 62 | 66 | end |
| @@ -31,5 +31,11 @@ module Docs · 30 unmodified lines | ||
| 31 | 31 | self.release = '1.24.3' |
| 32 | 32 | self.base_url = "https://api.dartlang.org/stable/#{release}/" |
| 33 | 33 | end |
| 34 | + | |
| 35 | + def get_latest_version(opts) | |
| 36 | + doc = fetch_doc('https://api.dartlang.org/', opts) | |
| 37 | + label = doc.at_css('footer > span').content.strip | |
| 38 | + label.sub(/Dart /, '') | |
| 39 | + end | |
| 34 | 40 | end |
| 35 | 41 | end |
| @@ -63,5 +63,10 @@ module Docs · 62 unmodified lines | ||
| 63 | 63 | self.release = '1.8.18' |
| 64 | 64 | self.base_url = 'https://docs.djangoproject.com/en/1.8/' |
| 65 | 65 | end |
| 66 | + | |
| 67 | + def get_latest_version(opts) | |
| 68 | + doc = fetch_doc('https://docs.djangoproject.com/', opts) | |
| 69 | + doc.at_css('#doc-versions > li.current > span > strong').content | |
| 70 | + end | |
| 66 | 71 | end |
| 67 | 72 | end |
| @@ -137,5 +137,11 @@ module Docs · 136 unmodified lines | ||
| 137 | 137 | options[:container] = '#docs' |
| 138 | 138 | options[:only_patterns] << /\Aswarm\// |
| 139 | 139 | end |
| 140 | + | |
| 141 | + def get_latest_version(opts) | |
| 142 | + doc = fetch_doc('https://docs.docker.com/', opts) | |
| 143 | + label = doc.at_css('.nav-container button.dropdown-toggle').content.strip | |
| 144 | + label.scan(/([0-9.]+)/)[0][0] | |
| 145 | + end | |
| 140 | 146 | end |
| 141 | 147 | end |
| @@ -36,6 +36,11 @@ module Docs · 35 unmodified lines | ||
| 36 | 36 | urls.map { |url| "<a href='#{url}'>#{url}</a>" }.join |
| 37 | 37 | end |
| 38 | 38 | |
| 39 | + def get_latest_version(opts) | |
| 40 | + doc = fetch_doc('https://dojotoolkit.org/api/', opts) | |
| 41 | + doc.at_css('#versionSelector > option[selected]').content | |
| 42 | + end | |
| 43 | + | |
| 39 | 44 | private |
| 40 | 45 | |
| 41 | 46 | def get_url_list(json, set = Set.new) |
| @@ -98,5 +98,10 @@ module Docs · 97 unmodified lines | ||
| 98 | 98 | /\A[\w\-\.]+\.php\/7\.x\z/ |
| 99 | 99 | ] |
| 100 | 100 | end |
| 101 | + | |
| 102 | + def get_latest_version(opts) | |
| 103 | + json = fetch_json('https://packagist.org/packages/drupal/drupal.json', opts) | |
| 104 | + json['package']['versions'].keys.find {|version| !version.end_with?('-dev')} | |
| 105 | + end | |
| 101 | 106 | end |
| 102 | 107 | end |
| @@ -22,5 +22,10 @@ module Docs · 21 unmodified lines | ||
| 22 | 22 | © 2013–2018 GitHub Inc.<br> |
| 23 | 23 | Licensed under the MIT license. |
| 24 | 24 | HTML |
| 25 | + | |
| 26 | + def get_latest_version(opts) | |
| 27 | + doc = fetch_doc('https://electronjs.org/docs', opts) | |
| 28 | + doc.at_css('.docs-version').content | |
| 29 | + end | |
| 25 | 30 | end |
| 26 | 31 | end |
| @@ -97,5 +97,10 @@ module Docs · 96 unmodified lines | ||
| 97 | 97 | 'https://elixir-lang.org/getting-started/' |
| 98 | 98 | ] |
| 99 | 99 | end |
| 100 | + | |
| 101 | + def get_latest_version(opts) | |
| 102 | + doc = fetch_doc('https://hexdocs.pm/elixir/api-reference.html', opts) | |
| 103 | + doc.at_css('h2.sidebar-projectVersion').content.strip[1..-1] | |
| 104 | + end | |
| 100 | 105 | end |
| 101 | 106 | end |
| @@ -56,5 +56,10 @@ module Docs · 55 unmodified lines | ||
| 56 | 56 | https://emberjs.com/api/ember-data/2.14/classes/DS |
| 57 | 57 | ) |
| 58 | 58 | end |
| 59 | + | |
| 60 | + def get_latest_version(opts) | |
| 61 | + doc = fetch_doc('https://emberjs.com/api/ember/release', opts) | |
| 62 | + doc.at_css('.sidebar > .select-container .ember-power-select-selected-item').content.strip | |
| 63 | + end | |
| 59 | 64 | end |
| 60 | 65 | end |
| @@ -55,5 +55,10 @@ module Docs · 54 unmodified lines | ||
| 55 | 55 | version '18' do |
| 56 | 56 | self.release = '18.3' |
| 57 | 57 | end |
| 58 | + | |
| 59 | + def get_latest_version(opts) | |
| 60 | + doc = fetch_doc('https://www.erlang.org/downloads', opts) | |
| 61 | + doc.at_css('.col-lg-3 > ul > li').content.strip.sub(/OTP /, '') | |
| 62 | + end | |
| 58 | 63 | end |
| 59 | 64 | end |
| @@ -20,5 +20,9 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | © JS Foundation and other contributors<br> |
| 21 | 21 | Licensed under the MIT License. |
| 22 | 22 | HTML |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + get_npm_version('eslint', opts) | |
| 26 | + end | |
| 23 | 27 | end |
| 24 | 28 | end |
| @@ -28,5 +28,9 @@ module Docs · 27 unmodified lines | ||
| 28 | 28 | © 2017 StrongLoop, IBM, and other expressjs.com contributors.<br> |
| 29 | 29 | Licensed under the Creative Commons Attribution-ShareAlike License v3.0. |
| 30 | 30 | HTML |
| 31 | + | |
| 32 | + def get_latest_version(opts) | |
| 33 | + get_npm_version('express', opts) | |
| 34 | + end | |
| 31 | 35 | end |
| 32 | 36 | end |
| @@ -33,5 +33,10 @@ module Docs · 32 unmodified lines | ||
| 33 | 33 | self.release = '1.2.0' |
| 34 | 34 | self.base_url = "https://falcon.readthedocs.io/en/#{self.release}/" |
| 35 | 35 | end |
| 36 | + | |
| 37 | + def get_latest_version(opts) | |
| 38 | + doc = fetch_doc('https://falcon.readthedocs.io/en/stable/changes/index.html', opts) | |
| 39 | + doc.at_css('#changelogs ul > li > a').content | |
| 40 | + end | |
| 36 | 41 | end |
| 37 | 42 | end |
| @@ -46,5 +46,10 @@ module Docs · 45 unmodified lines | ||
| 46 | 46 | self.release = '2.2.0' |
| 47 | 47 | self.base_url = "https://fishshell.com/docs/#{version}/" |
| 48 | 48 | end |
| 49 | + | |
| 50 | + def get_latest_version(opts) | |
| 51 | + doc = fetch_doc('http://fishshell.com/docs/current/index.html', opts) | |
| 52 | + doc.at_css('#toc-index').content.scan(/([0-9.]+)/)[0][0] | |
| 53 | + end | |
| 49 | 54 | end |
| 50 | 55 | end |
| @@ -18,5 +18,9 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2013–present Facebook Inc.<br> |
| 19 | 19 | Licensed under the MIT License. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + get_npm_version('flow-bin', opts) | |
| 24 | + end | |
| 21 | 25 | end |
| 22 | 26 | end |
| @@ -19,5 +19,10 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | © 2005–2018 Linus Torvalds and others<br> |
| 20 | 20 | Licensed under the GNU General Public License version 2. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + doc = fetch_doc('https://git-scm.com/', opts) | |
| 25 | + doc.at_css('.version').content.strip | |
| 26 | + end | |
| 22 | 27 | end |
| 23 | 28 | end |
| @@ -99,5 +99,11 @@ module Docs · 98 unmodified lines | ||
| 99 | 99 | |
| 100 | 100 | options[:replace_paths] = CPP_PATHS |
| 101 | 101 | end |
| 102 | + | |
| 103 | + def get_latest_version(opts) | |
| 104 | + doc = fetch_doc('https://gcc.gnu.org/onlinedocs/', opts) | |
| 105 | + label = doc.at_css('ul > li > ul > li > a').content.strip | |
| 106 | + label.scan(/([0-9.]+)/)[0][0] | |
| 107 | + end | |
| 102 | 108 | end |
| 103 | 109 | end |
| @@ -25,5 +25,11 @@ module Docs · 24 unmodified lines | ||
| 25 | 25 | self.release = '4.9.3' |
| 26 | 26 | self.base_url = "https://gcc.gnu.org/onlinedocs/gcc-#{release}/gfortran/" |
| 27 | 27 | end |
| 28 | + | |
| 29 | + def get_latest_version(opts) | |
| 30 | + doc = fetch_doc('https://gcc.gnu.org/onlinedocs/', opts) | |
| 31 | + label = doc.at_css('ul > li > ul > li > a').content.strip | |
| 32 | + label.scan(/([0-9.]+)/)[0][0] | |
| 33 | + end | |
| 28 | 34 | end |
| 29 | 35 | end |
| @@ -24,6 +24,11 @@ module Docs · 23 unmodified lines | ||
| 24 | 24 | Licensed under the Creative Commons Attribution License 3.0. |
| 25 | 25 | HTML |
| 26 | 26 | |
| 27 | + def get_latest_version(opts) | |
| 28 | + doc = fetch_doc('https://golang.org/project/', opts) | |
| 29 | + doc.at_css('#page ul > li > a').text[3..-1] | |
| 30 | + end | |
| 31 | + | |
| 27 | 32 | private |
| 28 | 33 | |
| 29 | 34 | def parse(response) # Hook here because Nokogori removes whitespace from textareas |
| @@ -37,5 +37,10 @@ module Docs · 36 unmodified lines | ||
| 37 | 37 | self.release = '2.1' |
| 38 | 38 | self.base_url = "http://docs.godotengine.org/en/#{self.version}/" |
| 39 | 39 | end |
| 40 | + | |
| 41 | + def get_latest_version(opts) | |
| 42 | + doc = fetch_doc('https://docs.godotengine.org/', opts) | |
| 43 | + doc.at_css('.version').content.strip | |
| 44 | + end | |
| 40 | 45 | end |
| 41 | 46 | end |
| @@ -17,5 +17,10 @@ module Docs · 16 unmodified lines | ||
| 17 | 17 | © 2011–2016 The Graphite Project<br> |
| 18 | 18 | Licensed under the Apache License, Version 2.0. |
| 19 | 19 | HTML |
| 20 | + | |
| 21 | + def get_latest_version(opts) | |
| 22 | + doc = fetch_doc('https://graphite.readthedocs.io/en/latest/releases.html', opts) | |
| 23 | + doc.at_css('#release-notes li > a').content | |
| 24 | + end | |
| 20 | 25 | end |
| 21 | 26 | end |
| @@ -26,5 +26,9 @@ module Docs · 25 unmodified lines | ||
| 26 | 26 | © GruntJS Team<br> |
| 27 | 27 | Licensed under the MIT License. |
| 28 | 28 | HTML |
| 29 | + | |
| 30 | + def get_latest_version(opts) | |
| 31 | + get_npm_version('grunt-cli', opts) | |
| 32 | + end | |
| 29 | 33 | end |
| 30 | 34 | end |
| @@ -19,5 +19,9 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | © 2011–2017 by Yehuda Katz<br> |
| 20 | 20 | Licensed under the MIT License. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + get_npm_version('handlebars', opts) | |
| 25 | + end | |
| 22 | 26 | end |
| 23 | 27 | end |
| @@ -10,7 +10,7 @@ module Docs · 9 unmodified lines | ||
| 10 | 10 | |
| 11 | 11 | html_filters.push 'haskell/entries', 'haskell/clean_html' |
| 12 | 12 | |
| 13 | − options[:container] = ->(filter) { filter.subpath.start_with?('users_guide') ? '.body' : '#content' } | |
| 13 | + options[:container] = ->(filter) {filter.subpath.start_with?('users_guide') ? '.body' : '#content'} | |
| 14 | 14 | |
| 15 | 15 | options[:only_patterns] = [/\Alibraries\//, /\Ausers_guide\//] |
| 16 | 16 | options[:skip_patterns] = [ |
| @@ -68,5 +68,12 @@ module Docs · 51 unmodified lines | ||
| 68 | 68 | |
| 69 | 69 | options[:only_patterns] = [/\Alibraries\//] |
| 70 | 70 | end |
| 71 | + | |
| 72 | + def get_latest_version(opts) | |
| 73 | + doc = fetch_doc('https://downloads.haskell.org/~ghc/latest/docs/html/', opts) | |
| 74 | + links = doc.css('a').to_a | |
| 75 | + versions = links.map {|link| link['href'].scan(/ghc-([0-9.]+)/)} | |
| 76 | + versions.find {|version| !version.empty?}[0][0] | |
| 77 | + end | |
| 71 | 78 | end |
| 72 | 79 | end |
| @@ -66,5 +66,11 @@ module Docs · 65 unmodified lines | ||
| 66 | 66 | version 'Python' do |
| 67 | 67 | self.base_url = 'https://api.haxe.org/python/' |
| 68 | 68 | end |
| 69 | + | |
| 70 | + def get_latest_version(opts) | |
| 71 | + doc = fetch_doc('https://api.haxe.org/', opts) | |
| 72 | + label = doc.at_css('.container.main-content h1 > small').content | |
| 73 | + label.sub(/version /, '') | |
| 74 | + end | |
| 69 | 75 | end |
| 70 | 76 | end |
| @@ -19,5 +19,9 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | © 2009–present Homebrew contributors<br> |
| 20 | 20 | Licensed under the BSD 2-Clause License. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + get_latest_github_release('Homebrew', 'brew', opts) | |
| 25 | + end | |
| 22 | 26 | end |
| 23 | 27 | end |
| @@ -7,6 +7,8 @@ module Docs · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | html_filters.push 'http/clean_html', 'http/entries', 'title' |
| 9 | 9 | |
| 10 | + options[:mdn_tag] = 'HTTP' | |
| 11 | + | |
| 10 | 12 | options[:root_title] = 'HTTP' |
| 11 | 13 | options[:title] = ->(filter) { filter.current_url.host == 'tools.ietf.org' ? false : filter.default_title } |
| 12 | 14 | options[:container] = ->(filter) { filter.current_url.host == 'tools.ietf.org' ? '.content' : nil } |
| @@ -54,5 +54,9 @@ module Docs · 53 unmodified lines | ||
| 54 | 54 | JS |
| 55 | 55 | capybara.html |
| 56 | 56 | end |
| 57 | + | |
| 58 | + def get_latest_version(opts) | |
| 59 | + get_npm_version('immutable', opts) | |
| 60 | + end | |
| 57 | 61 | end |
| 58 | 62 | end |
| @@ -46,5 +46,11 @@ module Docs · 45 unmodified lines | ||
| 46 | 46 | © 2015 InfluxData, Inc.<br> |
| 47 | 47 | Licensed under the MIT license. |
| 48 | 48 | HTML |
| 49 | + | |
| 50 | + def get_latest_version(opts) | |
| 51 | + doc = fetch_doc('https://docs.influxdata.com/influxdb/', opts) | |
| 52 | + label = doc.at_css('.navbar--current-product').content.strip | |
| 53 | + label.scan(/([0-9.]+)/)[0][0] | |
| 54 | + end | |
| 49 | 55 | end |
| 50 | 56 | end |
| @@ -17,5 +17,9 @@ module Docs · 16 unmodified lines | ||
| 17 | 17 | © 2008–2017 Pivotal Labs<br> |
| 18 | 18 | Licensed under the MIT License. |
| 19 | 19 | HTML |
| 20 | + | |
| 21 | + def get_latest_version(opts) | |
| 22 | + get_latest_github_release('jasmine', 'jasmine', opts) | |
| 23 | + end | |
| 20 | 24 | end |
| 21 | 25 | end |
| @@ -28,5 +28,10 @@ module Docs · 27 unmodified lines | ||
| 28 | 28 | © 2008–2018 Tom Preston-Werner and Jekyll contributors<br> |
| 29 | 29 | Licensed under the MIT license. |
| 30 | 30 | HTML |
| 31 | + | |
| 32 | + def get_latest_version(opts) | |
| 33 | + doc = fetch_doc('https://jekyllrb.com/docs/', opts) | |
| 34 | + doc.at_css('.meta a').content[1..-1] | |
| 35 | + end | |
| 31 | 36 | end |
| 32 | 37 | end |
| @@ -17,5 +17,10 @@ module Docs · 16 unmodified lines | ||
| 17 | 17 | © 2014–present Facebook Inc.<br> |
| 18 | 18 | Licensed under the BSD License. |
| 19 | 19 | HTML |
| 20 | + | |
| 21 | + def get_latest_version(opts) | |
| 22 | + doc = fetch_doc('https://jestjs.io/docs/en/getting-started', opts) | |
| 23 | + doc.at_css('header > a > h3').content | |
| 24 | + end | |
| 20 | 25 | end |
| 21 | 26 | end |
| @@ -22,5 +22,9 @@ module Docs · 21 unmodified lines | ||
| 22 | 22 | /Selectors\/odd/i, |
| 23 | 23 | /index/i |
| 24 | 24 | ] |
| 25 | + | |
| 26 | + def get_latest_version(opts) | |
| 27 | + get_npm_version('jquery', opts) | |
| 28 | + end | |
| 25 | 29 | end |
| 26 | 30 | end |
| @@ -16,5 +16,10 @@ module Docs · 15 unmodified lines | ||
| 16 | 16 | options[:fix_urls] = ->(url) do |
| 17 | 17 | url.sub! 'http://api.jquerymobile.com/', 'https://api.jquerymobile.com/' |
| 18 | 18 | end |
| 19 | + | |
| 20 | + def get_latest_version(opts) | |
| 21 | + doc = fetch_doc('https://jquerymobile.com/', opts) | |
| 22 | + doc.at_css('.download-box > .download-option:last-child > span').content.sub(/Version /, '') | |
| 23 | + end | |
| 19 | 24 | end |
| 20 | 25 | end |
| @@ -15,5 +15,9 @@ module Docs · 14 unmodified lines | ||
| 15 | 15 | options[:fix_urls] = ->(url) do |
| 16 | 16 | url.sub! 'http://api.jqueryui.com/', 'https://api.jqueryui.com/' |
| 17 | 17 | end |
| 18 | + | |
| 19 | + def get_latest_version(opts) | |
| 20 | + get_npm_version('jquery-ui', opts) | |
| 21 | + end | |
| 18 | 22 | end |
| 19 | 23 | end |
| @@ -21,5 +21,9 @@ module Docs · 20 unmodified lines | ||
| 21 | 21 | © 2011–2017 the contributors to the JSDoc 3 documentation project<br> |
| 22 | 22 | Licensed under the Creative Commons Attribution-ShareAlike Unported License v3.0. |
| 23 | 23 | HTML |
| 24 | + | |
| 25 | + def get_latest_version(opts) | |
| 26 | + get_latest_github_release('jsdoc3', 'jsdoc', opts) | |
| 27 | + end | |
| 24 | 28 | end |
| 25 | 29 | end |
| @@ -49,5 +49,9 @@ module Docs · 48 unmodified lines | ||
| 49 | 49 | |
| 50 | 50 | html_filters.push 'julia/entries_sphinx', 'julia/clean_html_sphinx', 'sphinx/clean_html' |
| 51 | 51 | end |
| 52 | + | |
| 53 | + def get_latest_version(opts) | |
| 54 | + get_latest_github_release('JuliaLang', 'julia', opts) | |
| 55 | + end | |
| 52 | 56 | end |
| 53 | 57 | end |
| @@ -33,5 +33,9 @@ module Docs · 32 unmodified lines | ||
| 33 | 33 | © Steven Sanderson, the Knockout.js team, and other contributors<br> |
| 34 | 34 | Licensed under the MIT License. |
| 35 | 35 | HTML |
| 36 | + | |
| 37 | + def get_latest_version(opts) | |
| 38 | + get_latest_github_release('knockout', 'knockout', opts) | |
| 39 | + end | |
| 36 | 40 | end |
| 37 | 41 | end |
| @@ -34,5 +34,9 @@ module Docs · 33 unmodified lines | ||
| 34 | 34 | © 2018 Koa contributors<br> |
| 35 | 35 | Licensed under the MIT License. |
| 36 | 36 | HTML |
| 37 | + | |
| 38 | + def get_latest_version(opts) | |
| 39 | + get_npm_version('koa', opts) | |
| 40 | + end | |
| 37 | 41 | end |
| 38 | 42 | end |
| @@ -28,5 +28,9 @@ module Docs · 27 unmodified lines | ||
| 28 | 28 | © 2010–2018 JetBrains s.r.o.<br> |
| 29 | 29 | Licensed under the Apache License, Version 2.0. |
| 30 | 30 | HTML |
| 31 | + | |
| 32 | + def get_latest_version(opts) | |
| 33 | + get_latest_github_release('JetBrains', 'kotlin', opts) | |
| 34 | + end | |
| 31 | 35 | end |
| 32 | 36 | end |
| @@ -133,5 +133,9 @@ module Docs · 132 unmodified lines | ||
| 133 | 133 | url |
| 134 | 134 | end |
| 135 | 135 | end |
| 136 | + | |
| 137 | + def get_latest_version(opts) | |
| 138 | + get_latest_github_release('laravel', 'laravel', opts) | |
| 139 | + end | |
| 136 | 140 | end |
| 137 | 141 | end |
| @@ -39,5 +39,10 @@ module Docs · 38 unmodified lines | ||
| 39 | 39 | self.base_url = "https://leafletjs.com/reference-#{release}.html" |
| 40 | 40 | end |
| 41 | 41 | |
| 42 | + def get_latest_version(opts) | |
| 43 | + doc = fetch_doc('https://leafletjs.com/index.html', opts) | |
| 44 | + link = doc.css('ul > li > a').to_a.select {|node| node.content == 'Docs'}.first | |
| 45 | + link['href'].scan(/reference-([0-9.]+)\.html/)[0][0] | |
| 46 | + end | |
| 42 | 47 | end |
| 43 | 48 | end |
| @@ -21,5 +21,11 @@ module Docs · 20 unmodified lines | ||
| 21 | 21 | © 2009–2016 The Core Less Team<br> |
| 22 | 22 | Licensed under the Creative Commons Attribution License 3.0. |
| 23 | 23 | HTML |
| 24 | + | |
| 25 | + def get_latest_version(opts) | |
| 26 | + doc = fetch_doc('http://lesscss.org/features/', opts) | |
| 27 | + label = doc.at_css('.footer-links > li').content | |
| 28 | + label.scan(/([0-9.]+)/)[0][0] | |
| 29 | + end | |
| 24 | 30 | end |
| 25 | 31 | end |
| @@ -19,5 +19,10 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | © 2005, 2006 Tobias Luetke<br> |
| 20 | 20 | Licensed under the MIT License. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + tags = get_github_tags('Shopify', 'liquid', opts) | |
| 25 | + tags[0]['name'][1..-1] | |
| 26 | + end | |
| 22 | 27 | end |
| 23 | 28 | end |
| @@ -32,5 +32,10 @@ module Docs · 31 unmodified lines | ||
| 32 | 32 | self.release = '2.4.2' |
| 33 | 33 | self.base_url = "https://lodash.com/docs/#{release}" |
| 34 | 34 | end |
| 35 | + | |
| 36 | + def get_latest_version(opts) | |
| 37 | + doc = fetch_doc('https://lodash.com/docs/', opts) | |
| 38 | + doc.at_css('#version > option[selected]').content | |
| 39 | + end | |
| 35 | 40 | end |
| 36 | 41 | end |
| @@ -39,5 +39,10 @@ module Docs · 38 unmodified lines | ||
| 39 | 39 | © 2006–2016 LÖVE Development Team<br> |
| 40 | 40 | Licensed under the GNU Free Documentation License, Version 1.3. |
| 41 | 41 | HTML |
| 42 | + | |
| 43 | + def get_latest_version(opts) | |
| 44 | + doc = fetch_doc('https://love2d.org/wiki/Version_History', opts) | |
| 45 | + doc.at_css('#mw-content-text table a').content | |
| 46 | + end | |
| 42 | 47 | end |
| 43 | 48 | end |
| @@ -26,5 +26,10 @@ module Docs · 25 unmodified lines | ||
| 26 | 26 | self.release = '5.1.5' |
| 27 | 27 | self.base_url = 'https://www.lua.org/manual/5.1/' |
| 28 | 28 | end |
| 29 | + | |
| 30 | + def get_latest_version(opts) | |
| 31 | + doc = fetch_doc('https://www.lua.org/manual/', opts) | |
| 32 | + doc.at_css('p.menubar > a').content | |
| 33 | + end | |
| 29 | 34 | end |
| 30 | 35 | end |
| @@ -38,5 +38,9 @@ module Docs · 37 unmodified lines | ||
| 38 | 38 | |
| 39 | 39 | html_filters.push 'marionette/entries_v2' |
| 40 | 40 | end |
| 41 | + | |
| 42 | + def get_latest_version(opts) | |
| 43 | + get_npm_version('backbone.marionette', opts) | |
| 44 | + end | |
| 41 | 45 | end |
| 42 | 46 | end |
| @@ -13,5 +13,9 @@ module Docs · 12 unmodified lines | ||
| 13 | 13 | © 2004 John Gruber<br> |
| 14 | 14 | Licensed under the BSD License. |
| 15 | 15 | HTML |
| 16 | + | |
| 17 | + def get_latest_version(opts) | |
| 18 | + '1.0.0' | |
| 19 | + end | |
| 16 | 20 | end |
| 17 | 21 | end |
| @@ -64,5 +64,9 @@ module Docs · 63 unmodified lines | ||
| 64 | 64 | "https://matplotlib.org/#{release}/mpl_toolkits/axes_grid/api/" |
| 65 | 65 | ] |
| 66 | 66 | end |
| 67 | + | |
| 68 | + def get_latest_version(opts) | |
| 69 | + get_latest_github_release('matplotlib', 'matplotlib', opts) | |
| 70 | + end | |
| 67 | 71 | end |
| 68 | 72 | end |
| @@ -6,6 +6,8 @@ module Docs · 5 unmodified lines | ||
| 6 | 6 | |
| 7 | 7 | html_filters.push 'css/clean_html', 'css/entries', 'title' |
| 8 | 8 | |
| 9 | + options[:mdn_tag] = 'CSS' | |
| 10 | + | |
| 9 | 11 | options[:root_title] = 'CSS' |
| 10 | 12 | |
| 11 | 13 | options[:skip] = %w(/CSS3 /Media/Visual /paged_media /Media/TV /Media/Tactile) |
| @@ -8,6 +8,8 @@ module Docs · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | html_filters.push 'dom/clean_html', 'dom/entries', 'title' |
| 10 | 10 | |
| 11 | + options[:mdn_tag] = 'XSLT_Reference' | |
| 12 | + | |
| 11 | 13 | options[:root_title] = 'DOM' |
| 12 | 14 | |
| 13 | 15 | options[:skip] = %w( |
| @@ -9,6 +9,8 @@ module Docs · 8 unmodified lines | ||
| 9 | 9 | html_filters.insert_after 'clean_html', 'dom_events/clean_html' |
| 10 | 10 | html_filters.push 'dom_events/entries', 'title' |
| 11 | 11 | |
| 12 | + options[:mdn_tag] = 'events' | |
| 13 | + | |
| 12 | 14 | options[:root_title] = 'DOM Events' |
| 13 | 15 | |
| 14 | 16 | options[:skip] = %w(/MozOrientation) |
| @@ -7,6 +7,8 @@ module Docs · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | html_filters.push 'html/clean_html', 'html/entries', 'title' |
| 9 | 9 | |
| 10 | + options[:mdn_tag] = 'HTML' | |
| 11 | + | |
| 10 | 12 | options[:root_title] = 'HTML' |
| 11 | 13 | |
| 12 | 14 | options[:title] = ->(filter) do |
| @@ -8,6 +8,8 @@ module Docs · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | html_filters.push 'javascript/clean_html', 'javascript/entries', 'title' |
| 10 | 10 | |
| 11 | + options[:mdn_tag] = 'JavaScript' | |
| 12 | + | |
| 11 | 13 | options[:root_title] = 'JavaScript' |
| 12 | 14 | |
| 13 | 15 | # Don't want |
| @@ -21,6 +21,11 @@ module Docs · 20 unmodified lines | ||
| 21 | 21 | Licensed under the Creative Commons Attribution-ShareAlike License v2.5 or later. |
| 22 | 22 | HTML |
| 23 | 23 | |
| 24 | + def get_latest_version(opts) | |
| 25 | + json = fetch_json("https://developer.mozilla.org/en-US/docs/feeds/json/tag/#{options[:mdn_tag]}", opts) | |
| 26 | + DateTime.parse(json[0]['pubdate']).to_time.to_i | |
| 27 | + end | |
| 28 | + | |
| 24 | 29 | private |
| 25 | 30 | |
| 26 | 31 | def process_response?(response) |
| @@ -8,6 +8,8 @@ module Docs · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | html_filters.push 'svg/clean_html', 'svg/entries', 'title' |
| 10 | 10 | |
| 11 | + options[:mdn_tag] = 'XSLT_Reference' | |
| 12 | + | |
| 11 | 13 | options[:root_title] = 'SVG' |
| 12 | 14 | |
| 13 | 15 | options[:title] = ->(filter) do |
| @@ -8,6 +8,8 @@ module Docs · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | html_filters.push 'xslt_xpath/clean_html', 'xslt_xpath/entries', 'title' |
| 10 | 10 | |
| 11 | + options[:mdn_tag] = 'XSLT_Reference' | |
| 12 | + | |
| 11 | 13 | options[:root_title] = 'XSLT' |
| 12 | 14 | |
| 13 | 15 | options[:only_patterns] = [/\A\/XSLT/, /\A\/XPath/] |
| @@ -45,5 +45,10 @@ module Docs · 44 unmodified lines | ||
| 45 | 45 | self.base_urls = ['https://guide.meteor.com/v1.3/', "https://docs.meteor.com/v#{self.release}/"] |
| 46 | 46 | options[:fix_urls] = nil |
| 47 | 47 | end |
| 48 | + | |
| 49 | + def get_latest_version(opts) | |
| 50 | + doc = fetch_doc('https://docs.meteor.com/#/full/', opts) | |
| 51 | + doc.at_css('select.version-select > option').content | |
| 52 | + end | |
| 48 | 53 | end |
| 49 | 54 | end |
| @@ -18,5 +18,9 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2011–2018 JS Foundation and contributors<br> |
| 19 | 19 | Licensed under the Creative Commons Attribution 4.0 International License. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + get_npm_version('mocha', opts) | |
| 24 | + end | |
| 21 | 25 | end |
| 22 | 26 | end |
| @@ -15,5 +15,9 @@ module Docs · 14 unmodified lines | ||
| 15 | 15 | © 2009–2017 The Modernizr team<br> |
| 16 | 16 | Licensed under the MIT License. |
| 17 | 17 | HTML |
| 18 | + | |
| 19 | + def get_latest_version(opts) | |
| 20 | + get_npm_version('modernizr', opts) | |
| 21 | + end | |
| 18 | 22 | end |
| 19 | 23 | end |
| @@ -22,5 +22,10 @@ module Docs · 21 unmodified lines | ||
| 22 | 22 | © JS Foundation and other contributors<br> |
| 23 | 23 | Licensed under the MIT License. |
| 24 | 24 | HTML |
| 25 | + | |
| 26 | + def get_latest_version(opts) | |
| 27 | + doc = fetch_doc('http://momentjs.com/', opts) | |
| 28 | + doc.at_css('.hero-title > h1 > span').content | |
| 29 | + end | |
| 25 | 30 | end |
| 26 | 31 | end |
| @@ -26,5 +26,11 @@ module Docs · 25 unmodified lines | ||
| 26 | 26 | © 2010 LearnBoost<br> |
| 27 | 27 | Licensed under the MIT License. |
| 28 | 28 | HTML |
| 29 | + | |
| 30 | + def get_latest_version(opts) | |
| 31 | + doc = fetch_doc('https://mongoosejs.com/docs/', opts) | |
| 32 | + label = doc.at_css('.pure-menu-link').content.strip | |
| 33 | + label.sub(/Version /, '') | |
| 34 | + end | |
| 29 | 35 | end |
| 30 | 36 | end |
| @@ -25,5 +25,11 @@ module Docs · 24 unmodified lines | ||
| 25 | 25 | © 2011-2018 Nginx, Inc.<br> |
| 26 | 26 | Licensed under the BSD License. |
| 27 | 27 | HTML |
| 28 | + | |
| 29 | + def get_latest_version(opts) | |
| 30 | + doc = fetch_doc('https://nginx.org/en/download.html', opts) | |
| 31 | + table = doc.at_css('#content > table').inner_html | |
| 32 | + table.scan(/nginx-([0-9.]+)</)[0][0] | |
| 33 | + end | |
| 28 | 34 | end |
| 29 | 35 | end |
| @@ -4,6 +4,9 @@ module Docs · 3 unmodified lines | ||
| 4 | 4 | self.slug = 'nginx_lua_module' |
| 5 | 5 | self.release = '0.10.13' |
| 6 | 6 | self.base_url = "https://github.com/openresty/lua-nginx-module/tree/v#{self.release}/" |
| 7 | + self.links = { | |
| 8 | + code: 'https://github.com/openresty/lua-nginx-module' | |
| 9 | + } | |
| 7 | 10 | |
| 8 | 11 | html_filters.push 'nginx_lua_module/clean_html', 'nginx_lua_module/entries', 'title' |
| 9 | 12 | |
| @@ -15,5 +18,11 @@ module Docs · 5 unmodified lines | ||
| 15 | 18 | © 2009–2018 Yichun "agentzh" Zhang (章亦春), OpenResty Inc.<br> |
| 16 | 19 | Licensed under the BSD License. |
| 17 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + tags = get_github_tags('openresty', 'lua-nginx-module', opts) | |
| 24 | + tag = tags.find {|tag| !tag['name'].include?('rc')} | |
| 25 | + tag['name'][1..-1] | |
| 26 | + end | |
| 18 | 27 | end |
| 19 | 28 | end |
| @@ -17,5 +17,10 @@ module Docs · 16 unmodified lines | ||
| 17 | 17 | © 2006–2018 Andreas Rumpf<br> |
| 18 | 18 | Licensed under the MIT License. |
| 19 | 19 | HTML |
| 20 | + | |
| 21 | + def get_latest_version(opts) | |
| 22 | + doc = fetch_doc('https://nim-lang.org/docs/overview.html', opts) | |
| 23 | + doc.at_css('.container > .docinfo > tbody > tr:last-child > td').content.strip | |
| 24 | + end | |
| 20 | 25 | end |
| 21 | 26 | end |
| @@ -46,5 +46,10 @@ module Docs · 45 unmodified lines | ||
| 46 | 46 | self.release = '4.9.1' |
| 47 | 47 | self.base_url = 'https://nodejs.org/dist/latest-v4.x/docs/api/' |
| 48 | 48 | end |
| 49 | + | |
| 50 | + def get_latest_version(opts) | |
| 51 | + doc = fetch_doc('https://nodejs.org/en/', opts) | |
| 52 | + doc.at_css('#home-intro > .home-downloadblock:last-of-type > a')['data-version'][1..-1] | |
| 53 | + end | |
| 49 | 54 | end |
| 50 | 55 | end |
| @@ -19,5 +19,9 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | Patrick Mahoney, Yoko Harada, Akinori Musha, John Shahid, Lars Kanis<br> |
| 20 | 20 | Licensed under the MIT License. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + get_latest_github_release('sparklemotion', 'nokogiri', opts) | |
| 25 | + end | |
| 22 | 26 | end |
| 23 | 27 | end |
| @@ -29,5 +29,9 @@ module Docs · 28 unmodified lines | ||
| 29 | 29 | Licensed under the npm License.<br> |
| 30 | 30 | npm is a trademark of npm, Inc. |
| 31 | 31 | HTML |
| 32 | + | |
| 33 | + def get_latest_version(opts) | |
| 34 | + get_latest_github_release('npm', 'cli', opts) | |
| 35 | + end | |
| 32 | 36 | end |
| 33 | 37 | end |
| @@ -49,5 +49,9 @@ module Docs · 48 unmodified lines | ||
| 49 | 49 | self.release = '1.10.4' |
| 50 | 50 | self.base_url = "https://docs.scipy.org/doc/numpy-#{self.release}/reference/" |
| 51 | 51 | end |
| 52 | + | |
| 53 | + def get_latest_version(opts) | |
| 54 | + get_latest_github_release('numpy', 'numpy', opts) | |
| 55 | + end | |
| 52 | 56 | end |
| 53 | 57 | end |
| @@ -86,5 +86,25 @@ module Docs · 85 unmodified lines | ||
| 86 | 86 | def read_file(path) |
| 87 | 87 | File.read(path).force_encoding('iso-8859-1').encode('utf-8') rescue nil |
| 88 | 88 | end |
| 89 | + | |
| 90 | + def get_latest_version(opts) | |
| 91 | + latest_version = 8 | |
| 92 | + current_attempt = latest_version | |
| 93 | + attempts = 0 | |
| 94 | + | |
| 95 | + while attempts < 3 | |
| 96 | + current_attempt += 1 | |
| 97 | + | |
| 98 | + doc = fetch_doc("https://packages.debian.org/sid/openjdk-#{current_attempt}-doc", opts) | |
| 99 | + if doc.at_css('.perror').nil? | |
| 100 | + latest_version = current_attempt | |
| 101 | + attempts = 0 | |
| 102 | + else | |
| 103 | + attempts += 1 | |
| 104 | + end | |
| 105 | + end | |
| 106 | + | |
| 107 | + latest_version | |
| 108 | + end | |
| 89 | 109 | end |
| 90 | 110 | end |
| @@ -18,5 +18,9 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2010–2016 The OpenTSDB Authors<br> |
| 19 | 19 | Licensed under the GNU LGPLv2.1+ and GPLv3+ licenses. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + get_latest_github_release('OpenTSDB', 'opentsdb', opts) | |
| 24 | + end | |
| 21 | 25 | end |
| 22 | 26 | end |
| @@ -23,5 +23,9 @@ module Docs · 22 unmodified lines | ||
| 23 | 23 | stub 'index2' do |
| 24 | 24 | request_one(url_for('index')).body |
| 25 | 25 | end |
| 26 | + | |
| 27 | + def get_latest_version(opts) | |
| 28 | + get_github_tags('padrino', 'padrino-framework', opts)[0]['name'] | |
| 29 | + end | |
| 26 | 30 | end |
| 27 | 31 | end |
| @@ -49,5 +49,11 @@ module Docs · 48 unmodified lines | ||
| 49 | 49 | self.release = '0.18.1' |
| 50 | 50 | self.base_url = "http://pandas.pydata.org/pandas-docs/version/#{self.release}/" |
| 51 | 51 | end |
| 52 | + | |
| 53 | + def get_latest_version(opts) | |
| 54 | + doc = fetch_doc('http://pandas.pydata.org/pandas-docs/stable/', opts) | |
| 55 | + label = doc.at_css('.body > .section > p').content | |
| 56 | + label.scan(/Version: ([0-9.]+)/)[0][0] | |
| 57 | + end | |
| 52 | 58 | end |
| 53 | 59 | end |
| @@ -43,5 +43,11 @@ module Docs · 42 unmodified lines | ||
| 43 | 43 | self.release = '5.20.2' |
| 44 | 44 | self.base_url = "https://perldoc.perl.org/#{self.release}/" |
| 45 | 45 | end |
| 46 | + | |
| 47 | + def get_latest_version(opts) | |
| 48 | + doc = fetch_doc('https://perldoc.perl.org/', opts) | |
| 49 | + header = doc.at_css('h2.h1').content | |
| 50 | + header.scan(/Perl ([0-9.]+)/)[0][0] | |
| 51 | + end | |
| 46 | 52 | end |
| 47 | 53 | end |
| @@ -29,5 +29,10 @@ module Docs · 28 unmodified lines | ||
| 29 | 29 | self.release = '2.0.13' |
| 30 | 30 | self.base_url = 'https://docs.phalconphp.com/en/2.0.0/' |
| 31 | 31 | end |
| 32 | + | |
| 33 | + def get_latest_version(opts) | |
| 34 | + doc = fetch_doc('https://docs.phalconphp.com/', opts) | |
| 35 | + doc.at_css('.header__lang.expand > div > ul > li > a').content | |
| 36 | + end | |
| 32 | 37 | end |
| 33 | 38 | end |
| @@ -25,5 +25,9 @@ module Docs · 24 unmodified lines | ||
| 25 | 25 | © 2016 Richard Davey, Photon Storm Ltd.<br> |
| 26 | 26 | Licensed under the MIT License. |
| 27 | 27 | HTML |
| 28 | + | |
| 29 | + def get_latest_version(opts) | |
| 30 | + get_latest_github_release('photonstorm', 'phaser', opts) | |
| 31 | + end | |
| 28 | 32 | end |
| 29 | 33 | end |
| @@ -46,5 +46,10 @@ module Docs · 45 unmodified lines | ||
| 46 | 46 | HTML |
| 47 | 47 | end |
| 48 | 48 | } |
| 49 | + | |
| 50 | + def get_latest_version(opts) | |
| 51 | + doc = fetch_doc('https://hexdocs.pm/phoenix/Phoenix.html', opts) | |
| 52 | + doc.at_css('.sidebar-projectVersion').content.strip[1..-1] | |
| 53 | + end | |
| 49 | 54 | end |
| 50 | 55 | end |
| @@ -66,5 +66,11 @@ module Docs · 65 unmodified lines | ||
| 66 | 66 | © 1997–2018 The PHP Documentation Group<br> |
| 67 | 67 | Licensed under the Creative Commons Attribution License v3.0 or later. |
| 68 | 68 | HTML |
| 69 | + | |
| 70 | + def get_latest_version(opts) | |
| 71 | + doc = fetch_doc('https://secure.php.net/manual/en/doc.changelog.php', opts) | |
| 72 | + label = doc.at_css('tbody.gen-changelog > tr > td').content | |
| 73 | + label.split(',').last.strip | |
| 74 | + end | |
| 69 | 75 | end |
| 70 | 76 | end |
| @@ -37,5 +37,11 @@ module Docs · 36 unmodified lines | ||
| 37 | 37 | self.release = '4.8' |
| 38 | 38 | self.base_url = "https://phpunit.de/manual/#{release}/en/" |
| 39 | 39 | end |
| 40 | + | |
| 41 | + def get_latest_version(opts) | |
| 42 | + doc = fetch_doc('https://phpunit.readthedocs.io/', opts) | |
| 43 | + label = doc.at_css('.rst-current-version').content.strip | |
| 44 | + label.scan(/v: ([0-9.]+)/)[0][0] | |
| 45 | + end | |
| 40 | 46 | end |
| 41 | 47 | end |
| @@ -80,5 +80,11 @@ module Docs · 79 unmodified lines | ||
| 80 | 80 | |
| 81 | 81 | html_filters.insert_before 'postgresql/extract_metadata', 'postgresql/normalize_class_names' |
| 82 | 82 | end |
| 83 | + | |
| 84 | + def get_latest_version(opts) | |
| 85 | + doc = fetch_doc('https://www.postgresql.org/docs/current/index.html', opts) | |
| 86 | + label = doc.at_css('#pgContentWrap h1.title').content | |
| 87 | + label.scan(/([0-9.]+)/)[0][0] | |
| 88 | + end | |
| 83 | 89 | end |
| 84 | 90 | end |
| @@ -18,6 +18,10 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | Licensed under the MIT license. |
| 19 | 19 | HTML |
| 20 | 20 | |
| 21 | + def get_latest_version(opts) | |
| 22 | + get_npm_version('pug', opts) | |
| 23 | + end | |
| 24 | + | |
| 21 | 25 | private |
| 22 | 26 | |
| 23 | 27 | def parse(response) # Hook here because Nokogori removes whitespace from textareas |
| @@ -14,5 +14,10 @@ module Docs · 13 unmodified lines | ||
| 14 | 14 | © 2017 Google Inc<br> |
| 15 | 15 | Licensed under the Apache License 2.0. |
| 16 | 16 | HTML |
| 17 | + | |
| 18 | + def get_latest_version(opts) | |
| 19 | + contents = get_github_file_contents('GoogleChrome', 'puppeteer', 'README.md', opts) | |
| 20 | + contents.scan(/\/v([0-9.]+)\/docs\/api\.md/)[0][0] | |
| 21 | + end | |
| 17 | 22 | end |
| 18 | 23 | end |
| @@ -2,6 +2,7 @@ module Docs · 1 unmodified line | ||
| 2 | 2 | class Pygame < UrlScraper |
| 3 | 3 | self.type = 'simple' |
| 4 | 4 | self.release = '1.9.4' |
| 5 | + self.base_url = 'https://www.pygame.org/docs/' | |
| 5 | 6 | self.root_path = 'py-modindex.html' |
| 6 | 7 | self.links = { |
| 7 | 8 | home: 'https://www.pygame.org/', |
| @@ -16,5 +17,9 @@ module Docs · 8 unmodified lines | ||
| 16 | 17 | © Pygame Developpers.<br> |
| 17 | 18 | Licensed under the GNU LGPL License version 2.1. |
| 18 | 19 | HTML |
| 20 | + | |
| 21 | + def get_latest_version(opts) | |
| 22 | + get_latest_github_release('pygame', 'pygame', opts) | |
| 23 | + end | |
| 19 | 24 | end |
| 20 | 25 | end |
| @@ -50,5 +50,10 @@ module Docs · 49 unmodified lines | ||
| 50 | 50 | |
| 51 | 51 | html_filters.push 'python/entries_v2', 'sphinx/clean_html', 'python/clean_html' |
| 52 | 52 | end |
| 53 | + | |
| 54 | + def get_latest_version(opts) | |
| 55 | + doc = fetch_doc('https://docs.python.org/', opts) | |
| 56 | + doc.at_css('.version_switcher_placeholder').content | |
| 57 | + end | |
| 53 | 58 | end |
| 54 | 59 | end |
| @@ -19,5 +19,9 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | © 2009–2017 Kristopher Michael Kowal<br> |
| 20 | 20 | Licensed under the MIT License. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + get_npm_version('q', opts) | |
| 25 | + end | |
| 22 | 26 | end |
| 23 | 27 | end |
| @@ -117,5 +117,10 @@ module Docs · 116 unmodified lines | ||
| 117 | 117 | self.release = '5.6' |
| 118 | 118 | self.base_url = 'https://doc.qt.io/qt-5.6/' |
| 119 | 119 | end |
| 120 | + | |
| 121 | + def get_latest_version(opts) | |
| 122 | + doc = fetch_doc('https://doc.qt.io/qt-5/index.html', opts) | |
| 123 | + doc.at_css('.mainContent h1.title').content.sub(/Qt /, '') | |
| 124 | + end | |
| 120 | 125 | end |
| 121 | 126 | end |
| @@ -15,6 +15,11 @@ module Docs · 14 unmodified lines | ||
| 15 | 15 | © 2013–2016 Scott Sauyet and Michael Hurley<br> |
| 16 | 16 | Licensed under the MIT License. |
| 17 | 17 | HTML |
| 18 | + | |
| 19 | + def get_latest_version(opts) | |
| 20 | + doc = fetch_doc('https://ramdajs.com/docs/', opts) | |
| 21 | + doc.at_css('.navbar-brand > .version').content[1..-1] | |
| 22 | + end | |
| 18 | 23 | end |
| 19 | 24 | end |
| 20 | 25 | |
| @@ -21,5 +21,10 @@ module Docs · 20 unmodified lines | ||
| 21 | 21 | © Ryan Davis, seattle.rb<br> |
| 22 | 22 | Licensed under the MIT License. |
| 23 | 23 | HTML |
| 24 | + | |
| 25 | + def get_latest_version(opts) | |
| 26 | + contents = get_github_file_contents('seattlerb', 'minitest', 'History.rdoc', opts) | |
| 27 | + contents.scan(/([0-9.]+)/)[0][0] | |
| 28 | + end | |
| 24 | 29 | end |
| 25 | 30 | end |
| @@ -93,5 +93,9 @@ module Docs · 92 unmodified lines | ||
| 93 | 93 | version '4.1' do |
| 94 | 94 | self.release = '4.1.16' |
| 95 | 95 | end |
| 96 | + | |
| 97 | + def get_latest_version(opts) | |
| 98 | + get_latest_github_release('rails', 'rails', opts) | |
| 99 | + end | |
| 96 | 100 | end |
| 97 | 101 | end |
| @@ -84,5 +84,16 @@ module Docs · 83 unmodified lines | ||
| 84 | 84 | version '2.2' do |
| 85 | 85 | self.release = '2.2.10' |
| 86 | 86 | end |
| 87 | + | |
| 88 | + def get_latest_version(opts) | |
| 89 | + tags = get_github_tags('ruby', 'ruby', opts) | |
| 90 | + tags.each do |tag| | |
| 91 | + version = tag['name'].gsub(/_/, '.')[1..-1] | |
| 92 | + | |
| 93 | + if !/^([0-9.]+)$/.match(version).nil? && version.count('.') == 2 | |
| 94 | + return version | |
| 95 | + end | |
| 96 | + end | |
| 97 | + end | |
| 87 | 98 | end |
| 88 | 99 | end |
| @@ -30,5 +30,10 @@ module Docs · 29 unmodified lines | ||
| 30 | 30 | © 2013–present Facebook Inc.<br> |
| 31 | 31 | Licensed under the Creative Commons Attribution 4.0 International Public License. |
| 32 | 32 | HTML |
| 33 | + | |
| 34 | + def get_latest_version(opts) | |
| 35 | + doc = fetch_doc('https://reactjs.org/docs/getting-started.html', opts) | |
| 36 | + doc.at_css('a[href="/versions"]').content.strip[1..-1] | |
| 37 | + end | |
| 33 | 38 | end |
| 34 | 39 | end |
| @@ -30,5 +30,10 @@ module Docs · 29 unmodified lines | ||
| 30 | 30 | © 2015–2018 Facebook Inc.<br> |
| 31 | 31 | Licensed under the Creative Commons Attribution 4.0 International Public License. |
| 32 | 32 | HTML |
| 33 | + | |
| 34 | + def get_latest_version(opts) | |
| 35 | + doc = fetch_doc('https://facebook.github.io/react-native/docs/getting-started.html', opts) | |
| 36 | + doc.at_css('header > a > h3').content | |
| 37 | + end | |
| 33 | 38 | end |
| 34 | 39 | end |
| @@ -19,5 +19,11 @@ module Docs · 18 unmodified lines | ||
| 19 | 19 | © 2009–2018 Salvatore Sanfilippo<br> |
| 20 | 20 | Licensed under the Creative Commons Attribution-ShareAlike License 4.0. |
| 21 | 21 | HTML |
| 22 | + | |
| 23 | + def get_latest_version(opts) | |
| 24 | + body = fetch('http://download.redis.io/redis-stable/00-RELEASENOTES', opts) | |
| 25 | + body = body.lines[1..-1].join | |
| 26 | + body.scan(/Redis ([0-9.]+)/)[0][0] | |
| 27 | + end | |
| 22 | 28 | end |
| 23 | 29 | end |
| @@ -20,5 +20,9 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | stub '' do |
| 21 | 21 | request_one('http://redux.js.org/index.html').body |
| 22 | 22 | end |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + get_npm_version('redux', opts) | |
| 26 | + end | |
| 23 | 27 | end |
| 24 | 28 | end |
| @@ -18,5 +18,10 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2013–present Facebook Inc.<br> |
| 19 | 19 | Licensed under the BSD License. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + doc = fetch_doc('http://facebook.github.io/relay/en/', opts) | |
| 24 | + doc.at_css('header > a > h3').content[1..-1] | |
| 25 | + end | |
| 21 | 26 | end |
| 22 | 27 | end |
| @@ -30,5 +30,9 @@ module Docs · 29 unmodified lines | ||
| 30 | 30 | © jQuery Foundation and other contributors<br> |
| 31 | 31 | Licensed under the MIT License. |
| 32 | 32 | HTML |
| 33 | + | |
| 34 | + def get_latest_version(opts) | |
| 35 | + get_npm_version('requirejs', opts) | |
| 36 | + end | |
| 33 | 37 | end |
| 34 | 38 | end |
| @@ -58,6 +58,10 @@ module Docs · 57 unmodified lines | ||
| 58 | 58 | CODE |
| 59 | 59 | end |
| 60 | 60 | |
| 61 | + def get_latest_version(opts) | |
| 62 | + get_latest_github_release('rethinkdb', 'rethinkdb', opts) | |
| 63 | + end | |
| 64 | + | |
| 61 | 65 | private |
| 62 | 66 | |
| 63 | 67 | def process_response?(response) |
| @@ -39,6 +39,12 @@ module Docs · 38 unmodified lines | ||
| 39 | 39 | Licensed under the Apache License, Version 2.0 or the MIT license, at your option. |
| 40 | 40 | HTML |
| 41 | 41 | |
| 42 | + def get_latest_version(opts) | |
| 43 | + doc = fetch_doc('https://www.rust-lang.org/', opts) | |
| 44 | + label = doc.at_css('.button-download + p > a').content | |
| 45 | + label.sub(/Version /, '') | |
| 46 | + end | |
| 47 | + | |
| 42 | 48 | private |
| 43 | 49 | |
| 44 | 50 | REDIRECT_RGX = /http-equiv="refresh"/i |
| @@ -23,5 +23,9 @@ module Docs · 22 unmodified lines | ||
| 23 | 23 | © 2006–2016 Hampton Catlin, Nathan Weizenbaum, and Chris Eppstein<br> |
| 24 | 24 | Licensed under the MIT License. |
| 25 | 25 | HTML |
| 26 | + | |
| 27 | + def get_latest_version(opts) | |
| 28 | + get_latest_github_release('sass', 'libsass', opts) | |
| 29 | + end | |
| 26 | 30 | end |
| 27 | 31 | end |
| @@ -20,5 +20,10 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | © 2011 the scikit-image team<br> |
| 21 | 21 | Licensed under the BSD 3-clause License. |
| 22 | 22 | HTML |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + tags = get_github_tags('scikit-image', 'scikit-image', opts) | |
| 26 | + tags[0]['name'][1..-1] | |
| 27 | + end | |
| 23 | 28 | end |
| 24 | 29 | end |
| @@ -25,5 +25,9 @@ module Docs · 24 unmodified lines | ||
| 25 | 25 | Licensed under the 3-clause BSD License. |
| 26 | 26 | HTML |
| 27 | 27 | |
| 28 | + def get_latest_version(opts) | |
| 29 | + doc = fetch_doc('https://scikit-learn.org/stable/documentation.html', opts) | |
| 30 | + doc.at_css('.body h1').content.scan(/([0-9.]+)/)[0][0] | |
| 31 | + end | |
| 28 | 32 | end |
| 29 | 33 | end |
| @@ -52,5 +52,10 @@ module Docs · 51 unmodified lines | ||
| 52 | 52 | self.release = '1.17.7' |
| 53 | 53 | self.base_url = "https://sinonjs.org/releases/v#{release}/" |
| 54 | 54 | end |
| 55 | + | |
| 56 | + def get_latest_version(opts) | |
| 57 | + body = fetch('https://sinonjs.org/', opts) | |
| 58 | + body.scan(/\/releases\/v([0-9.]+)/)[0][0] | |
| 59 | + end | |
| 55 | 60 | end |
| 56 | 61 | end |
| @@ -20,5 +20,9 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | © 2014–2015 Automattic<br> |
| 21 | 21 | Licensed under the MIT License. |
| 22 | 22 | HTML |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + get_npm_version('socket.io', opts) | |
| 26 | + end | |
| 23 | 27 | end |
| 24 | 28 | end |
| @@ -41,6 +41,11 @@ module Docs · 40 unmodified lines | ||
| 41 | 41 | |
| 42 | 42 | options[:attribution] = 'SQLite is in the Public Domain.' |
| 43 | 43 | |
| 44 | + def get_latest_version(opts) | |
| 45 | + doc = fetch_doc('https://sqlite.org/chronology.html', opts) | |
| 46 | + doc.at_css('#chrontab > tbody > tr > td:last-child > a').content | |
| 47 | + end | |
| 48 | + | |
| 44 | 49 | private |
| 45 | 50 | |
| 46 | 51 | def parse(response) |
| @@ -21,5 +21,9 @@ module Docs · 20 unmodified lines | ||
| 21 | 21 | Licensed under the 3-clause BSD License. |
| 22 | 22 | HTML |
| 23 | 23 | |
| 24 | + def get_latest_version(opts) | |
| 25 | + doc = fetch_doc('http://www.statsmodels.org/stable/', opts) | |
| 26 | + doc.at_css('.sphinxsidebarwrapper h3 + p > b').content[1..-1] | |
| 27 | + end | |
| 24 | 28 | end |
| 25 | 29 | end |
| @@ -178,5 +178,11 @@ module Docs · 177 unmodified lines | ||
| 178 | 178 | </p> |
| 179 | 179 | </div> |
| 180 | 180 | HTML |
| 181 | + | |
| 182 | + def get_latest_version(opts) | |
| 183 | + body = fetch('https://feeds.feedburner.com/WhenCanIUse?format=xml', opts) | |
| 184 | + timestamp = body.scan(/<updated>([^<]+)<\/updated>/)[0][0] | |
| 185 | + DateTime.parse(timestamp).to_time.to_i | |
| 186 | + end | |
| 181 | 187 | end |
| 182 | 188 | end |
| @@ -70,5 +70,9 @@ module Docs · 69 unmodified lines | ||
| 70 | 70 | self.release = '2.7.35' |
| 71 | 71 | self.base_url = "https://api.symfony.com/#{version}/" |
| 72 | 72 | end |
| 73 | + | |
| 74 | + def get_latest_version(opts) | |
| 75 | + get_latest_github_release('symfony', 'symfony', opts) | |
| 76 | + end | |
| 73 | 77 | end |
| 74 | 78 | end |
| @@ -25,5 +25,10 @@ module Docs · 24 unmodified lines | ||
| 25 | 25 | options[:attribution] = <<-HTML |
| 26 | 26 | Licensed under <a href="http://tcl.tk/software/tcltk/license.html">Tcl/Tk terms</a> |
| 27 | 27 | HTML |
| 28 | + | |
| 29 | + def get_latest_version(opts) | |
| 30 | + doc = fetch_doc('https://www.tcl.tk/man/tcl/contents.htm', opts) | |
| 31 | + doc.at_css('h2').content.scan(/Tk([0-9.]+)/)[0][0] | |
| 32 | + end | |
| 28 | 33 | end |
| 29 | 34 | end |
| @@ -56,6 +56,10 @@ module Docs · 55 unmodified lines | ||
| 56 | 56 | /\Aextend/] |
| 57 | 57 | end |
| 58 | 58 | |
| 59 | + def get_latest_version(opts) | |
| 60 | + get_latest_github_release('tensorflow', 'tensorflow', opts) | |
| 61 | + end | |
| 62 | + | |
| 59 | 63 | private |
| 60 | 64 | |
| 61 | 65 | def parse(response) |
| @@ -18,5 +18,10 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2018 HashiCorp</br> |
| 19 | 19 | Licensed under the MPL 2.0 License. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + contents = get_github_file_contents('hashicorp', 'terraform-website', 'content/config.rb', opts) | |
| 24 | + contents.scan(/version\s+=\s+"([0-9.]+)"/)[0][0] | |
| 25 | + end | |
| 21 | 26 | end |
| 22 | 27 | end |
| @@ -28,5 +28,10 @@ module Docs · 27 unmodified lines | ||
| 28 | 28 | self.release = '1.34.3' |
| 29 | 29 | self.base_url = 'https://twig.symfony.com/doc/1.x/' |
| 30 | 30 | end |
| 31 | + | |
| 32 | + def get_latest_version(opts) | |
| 33 | + tags = get_github_tags('twigphp', 'Twig', opts) | |
| 34 | + tags[0]['name'][1..-1] | |
| 35 | + end | |
| 31 | 36 | end |
| 32 | 37 | end |
| @@ -24,6 +24,10 @@ module Docs · 23 unmodified lines | ||
| 24 | 24 | © Microsoft and other contributors<br> |
| 25 | 25 | Licensed under the Apache License, Version 2.0. |
| 26 | 26 | HTML |
| 27 | + | |
| 28 | + def get_latest_version(opts) | |
| 29 | + get_latest_github_release('Microsoft', 'TypeScript', opts) | |
| 30 | + end | |
| 27 | 31 | end |
| 28 | 32 | end |
| 29 | 33 | |
| @@ -20,5 +20,10 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | © 2009–2018 Jeremy Ashkenas, DocumentCloud and Investigative Reporters & Editors<br> |
| 21 | 21 | Licensed under the MIT License. |
| 22 | 22 | HTML |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + doc = fetch_doc('https://underscorejs.org/', opts) | |
| 26 | + doc.at_css('.version').content[1...-1] | |
| 27 | + end | |
| 23 | 28 | end |
| 24 | 29 | end |
| @@ -18,5 +18,10 @@ module Docs · 17 unmodified lines | ||
| 18 | 18 | © 2010–2018 Mitchell Hashimoto<br> |
| 19 | 19 | Licensed under the MPL 2.0 License. |
| 20 | 20 | HTML |
| 21 | + | |
| 22 | + def get_latest_version(opts) | |
| 23 | + contents = get_github_file_contents('hashicorp', 'vagrant', 'website/config.rb', opts) | |
| 24 | + contents.scan(/version\s+=\s+"([0-9.]+)"/)[0][0] | |
| 25 | + end | |
| 21 | 26 | end |
| 22 | 27 | end |
| @@ -32,5 +32,9 @@ module Docs · 31 unmodified lines | ||
| 32 | 32 | self.root_path = '/guide/index.html' |
| 33 | 33 | self.initial_paths = %w(/api/index.html) |
| 34 | 34 | end |
| 35 | + | |
| 36 | + def get_latest_version(opts) | |
| 37 | + get_latest_github_release('vuejs', 'vue', opts) | |
| 38 | + end | |
| 35 | 39 | end |
| 36 | 40 | end |
| @@ -20,5 +20,10 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | Licensed under the Creative Commons Attribution 4.0 International License.<br> |
| 21 | 21 | Vulkan and the Vulkan logo are registered trademarks of the Khronos Group Inc. |
| 22 | 22 | HTML |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + tags = get_github_tags('KhronosGroup', 'Vulkan-Docs', opts) | |
| 26 | + tags[0]['name'][1..-1] | |
| 27 | + end | |
| 23 | 28 | end |
| 24 | 29 | end |
| @@ -68,5 +68,9 @@ module Docs · 67 unmodified lines | ||
| 68 | 68 | Licensed under the MIT License. |
| 69 | 69 | HTML |
| 70 | 70 | end |
| 71 | + | |
| 72 | + def get_latest_version(opts) | |
| 73 | + get_npm_version('webpack', opts) | |
| 74 | + end | |
| 71 | 75 | end |
| 72 | 76 | end |
| @@ -20,5 +20,9 @@ module Docs · 19 unmodified lines | ||
| 20 | 20 | © 2016–present Yarn Contributors<br> |
| 21 | 21 | Licensed under the BSD License. |
| 22 | 22 | HTML |
| 23 | + | |
| 24 | + def get_latest_version(opts) | |
| 25 | + get_latest_github_release('yarnpkg', 'yarn', opts) | |
| 26 | + end | |
| 23 | 27 | end |
| 24 | 28 | end |
| @@ -34,5 +34,9 @@ module Docs · 33 unmodified lines | ||
| 34 | 34 | |
| 35 | 35 | options[:container] = '.grid_9' |
| 36 | 36 | end |
| 37 | + | |
| 38 | + def get_latest_version(opts) | |
| 39 | + get_latest_github_release('yiisoft', 'yii2', opts) | |
| 40 | + end | |
| 37 | 41 | end |
| 38 | 42 | end |
| @@ -0,0 +1,315 @@ | ||
| 1 | +class UpdatesCLI < Thor | |
| 2 | + # The GitHub user that is allowed to upload reports | |
| 3 | + UPLOAD_USER = 'devdocs-bot' | |
| 4 | + | |
| 5 | + # The repository to create an issue in when uploading the results | |
| 6 | + UPLOAD_REPO = 'freeCodeCamp/devdocs' | |
| 7 | + | |
| 8 | + def self.to_s | |
| 9 | + 'Updates' | |
| 10 | + end | |
| 11 | + | |
| 12 | + def initialize(*args) | |
| 13 | + require 'docs' | |
| 14 | + require 'progress_bar' | |
| 15 | + require 'terminal-table' | |
| 16 | + require 'date' | |
| 17 | + super | |
| 18 | + end | |
| 19 | + | |
| 20 | + desc 'check [--github-token] [--upload] [--verbose] [doc]...', 'Check for outdated documentations' | |
| 21 | + option :github_token, :type => :string | |
| 22 | + option :upload, :type => :boolean | |
| 23 | + option :verbose, :type => :boolean | |
| 24 | + def check(*names) | |
| 25 | + # Convert names to a list of Scraper instances | |
| 26 | + # Versions are omitted, if v10 is outdated than v8 is aswell | |
| 27 | + docs = names.map {|name| Docs.find(name.split(/@|~/)[0], false)}.uniq | |
| 28 | + | |
| 29 | + # Check all documentations for updates when no arguments are given | |
| 30 | + docs = Docs.all if docs.empty? | |
| 31 | + | |
| 32 | + opts = { | |
| 33 | + logger: logger | |
| 34 | + } | |
| 35 | + | |
| 36 | + if options.key?(:github_token) | |
| 37 | + opts[:github_token] = options[:github_token] | |
| 38 | + end | |
| 39 | + | |
| 40 | + with_progress_bar do |bar| | |
| 41 | + bar.max = docs.length | |
| 42 | + bar.write | |
| 43 | + end | |
| 44 | + | |
| 45 | + results = docs.map do |doc| | |
| 46 | + result = check_doc(doc, opts) | |
| 47 | + with_progress_bar(&:increment!) | |
| 48 | + result | |
| 49 | + end | |
| 50 | + | |
| 51 | + process_results(results) | |
| 52 | + rescue Docs::DocNotFound => error | |
| 53 | + logger.error(error) | |
| 54 | + logger.info('Run "thor docs:list" to see the list of docs.') | |
| 55 | + end | |
| 56 | + | |
| 57 | + private | |
| 58 | + | |
| 59 | + def check_doc(doc, opts) | |
| 60 | + logger.debug("Checking #{doc.name}") | |
| 61 | + | |
| 62 | + instance = doc.versions.first.new | |
| 63 | + scraper_version = instance.get_scraper_version(opts) | |
| 64 | + latest_version = instance.get_latest_version(opts) | |
| 65 | + | |
| 66 | + { | |
| 67 | + name: doc.name, | |
| 68 | + scraper_version: format_version(scraper_version), | |
| 69 | + latest_version: format_version(latest_version), | |
| 70 | + is_outdated: instance.is_outdated(scraper_version, latest_version) | |
| 71 | + } | |
| 72 | + rescue NotImplementedError | |
| 73 | + logger.warn("Couldn't check #{doc.name}, get_latest_version is not implemented") | |
| 74 | + error_result(doc, '`get_latest_version` is not implemented') | |
| 75 | + rescue => error | |
| 76 | + logger.error("Error while checking #{doc.name}\n#{error.full_message.strip}") | |
| 77 | + error_result(doc, error.message.gsub(/'/, '`')) | |
| 78 | + end | |
| 79 | + | |
| 80 | + def format_version(version) | |
| 81 | + str = version.to_s | |
| 82 | + | |
| 83 | + # If the version is numeric and greater than or equal to 1e9 it's probably a timestamp | |
| 84 | + return str if str.match(/^(\d)+$/).nil? or str.to_i < 1e9 | |
| 85 | + | |
| 86 | + DateTime.strptime(str, '%s').strftime('%F') | |
| 87 | + end | |
| 88 | + | |
| 89 | + def error_result(doc, reason) | |
| 90 | + { | |
| 91 | + name: doc.name, | |
| 92 | + error: reason | |
| 93 | + } | |
| 94 | + end | |
| 95 | + | |
| 96 | + def process_results(results) | |
| 97 | + successful_results = results.select {|result| result.key?(:is_outdated)} | |
| 98 | + failed_results = results.select {|result| result.key?(:error)} | |
| 99 | + | |
| 100 | + up_to_date_results = successful_results.select {|result| !result[:is_outdated]} | |
| 101 | + outdated_results = successful_results.select {|result| result[:is_outdated]} | |
| 102 | + | |
| 103 | + log_results(outdated_results, up_to_date_results, failed_results) | |
| 104 | + upload_results(outdated_results, up_to_date_results, failed_results) if options[:upload] | |
| 105 | + end | |
| 106 | + | |
| 107 | + # | |
| 108 | + # Result logging methods | |
| 109 | + # | |
| 110 | + | |
| 111 | + def log_results(outdated_results, up_to_date_results, failed_results) | |
| 112 | + log_failed_results(failed_results) unless failed_results.empty? | |
| 113 | + log_successful_results('Up-to-date', up_to_date_results) unless up_to_date_results.empty? | |
| 114 | + log_successful_results('Outdated', outdated_results) unless outdated_results.empty? | |
| 115 | + end | |
| 116 | + | |
| 117 | + def log_successful_results(label, results) | |
| 118 | + title = "#{label} documentations (#{results.length})" | |
| 119 | + headings = ['Documentation', 'Scraper version', 'Latest version'] | |
| 120 | + rows = results.map {|result| [result[:name], result[:scraper_version], result[:latest_version]]} | |
| 121 | + | |
| 122 | + table = Terminal::Table.new :title => title, :headings => headings, :rows => rows | |
| 123 | + puts table | |
| 124 | + end | |
| 125 | + | |
| 126 | + def log_failed_results(results) | |
| 127 | + title = "Documentations that could not be checked (#{results.length})" | |
| 128 | + headings = %w(Documentation Reason) | |
| 129 | + rows = results.map {|result| [result[:name], result[:error]]} | |
| 130 | + | |
| 131 | + table = Terminal::Table.new :title => title, :headings => headings, :rows => rows | |
| 132 | + puts table | |
| 133 | + end | |
| 134 | + | |
| 135 | + # | |
| 136 | + # Upload methods | |
| 137 | + # | |
| 138 | + | |
| 139 | + def upload_results(outdated_results, up_to_date_results, failed_results) | |
| 140 | + # We can't create issues without a GitHub token | |
| 141 | + unless options.key?(:github_token) | |
| 142 | + logger.error("Please specify a GitHub token with the public_repo permission for #{UPLOAD_USER} with the --github-token parameter") | |
| 143 | + return | |
| 144 | + end | |
| 145 | + | |
| 146 | + logger.info('Uploading the results to a new GitHub issue') | |
| 147 | + | |
| 148 | + logger.info('Checking if the GitHub token belongs to the correct user') | |
| 149 | + user = github_get('/user') | |
| 150 | + | |
| 151 | + # Only allow the DevDocs bot to upload reports | |
| 152 | + unless user['login'] == UPLOAD_USER | |
| 153 | + logger.error("Only #{UPLOAD_USER} is supposed to upload the results to a new issue. The specified github token is not for #{UPLOAD_USER}.") | |
| 154 | + return | |
| 155 | + end | |
| 156 | + | |
| 157 | + logger.info('Creating a new GitHub issue') | |
| 158 | + | |
| 159 | + issue = results_to_issue(outdated_results, up_to_date_results, failed_results) | |
| 160 | + created_issue = github_post("/repos/#{UPLOAD_REPO}/issues", issue) | |
| 161 | + | |
| 162 | + logger.info('Checking if the previous issue is still open') | |
| 163 | + | |
| 164 | + search_params = { | |
| 165 | + q: "Documentation versions report in:title author:#{UPLOAD_USER} is:issue repo:#{UPLOAD_REPO}", | |
| 166 | + sort: 'created', | |
| 167 | + order: 'desc' | |
| 168 | + } | |
| 169 | + | |
| 170 | + matching_issues = github_get('/search/issues', search_params) | |
| 171 | + previous_issue = matching_issues['items'].find {|item| item['number'] != created_issue['number']} | |
| 172 | + | |
| 173 | + if previous_issue.nil? | |
| 174 | + logger.info('No previous issue found') | |
| 175 | + log_upload_success(created_issue) | |
| 176 | + else | |
| 177 | + logger.info('Commenting on the previous issue') | |
| 178 | + | |
| 179 | + comment = "This report was superseded by ##{created_issue['number']}." | |
| 180 | + github_post("/repos/#{UPLOAD_REPO}/issues/#{previous_issue['number']}/comments", {body: comment}) | |
| 181 | + if previous_issue['closed_at'].nil? | |
| 182 | + logger.info('Closing the previous issue') | |
| 183 | + github_patch("/repos/#{UPLOAD_REPO}/issues/#{previous_issue['number']}", {state: 'closed'}) | |
| 184 | + log_upload_success(created_issue) | |
| 185 | + else | |
| 186 | + logger.info('The previous issue has already been closed') | |
| 187 | + log_upload_success(created_issue) | |
| 188 | + end | |
| 189 | + end | |
| 190 | + end | |
| 191 | + | |
| 192 | + def results_to_issue(outdated_results, up_to_date_results, failed_results) | |
| 193 | + results = [ | |
| 194 | + successful_results_to_markdown('Outdated', outdated_results), | |
| 195 | + successful_results_to_markdown('Up-to-date', up_to_date_results), | |
| 196 | + failed_results_to_markdown(failed_results) | |
| 197 | + ] | |
| 198 | + | |
| 199 | + results_str = results.select {|result| !result.nil?}.join("\n\n") | |
| 200 | + travis_str = ENV['TRAVIS'].nil? ? '' : "\n\nThis issue was created by Travis CI build [##{ENV['TRAVIS_BUILD_NUMBER']}](#{ENV['TRAVIS_BUILD_WEB_URL']})." | |
| 201 | + | |
| 202 | + title = "Documentation versions report for #{Date.today.strftime('%B %Y')}" | |
| 203 | + body = <<-MARKDOWN | |
| 204 | +## What is this? | |
| 205 | + | |
| 206 | +This is an automatically created issue which contains information about the version status of the documentations available on DevDocs. The results of this report can be used by maintainers when updating outdated documentations. | |
| 207 | + | |
| 208 | +Maintainers can close this issue when all documentations are up-to-date. The issue is also automatically closed when the next report is created.#{travis_str} | |
| 209 | + | |
| 210 | +## Results | |
| 211 | + | |
| 212 | +The #{outdated_results.length + up_to_date_results.length + failed_results.length} documentations are divided as follows: | |
| 213 | +- #{outdated_results.length} that #{outdated_results.length == 1 ? 'is' : 'are'} outdated | |
| 214 | +- #{up_to_date_results.length} that #{up_to_date_results.length == 1 ? 'is' : 'are'} up-to-date (patch updates are ignored) | |
| 215 | +- #{failed_results.length} that could not be checked | |
| 216 | + MARKDOWN | |
| 217 | + | |
| 218 | + { | |
| 219 | + title: title, | |
| 220 | + body: body.strip + "\n\n" + results_str | |
| 221 | + } | |
| 222 | + end | |
| 223 | + | |
| 224 | + def successful_results_to_markdown(label, results) | |
| 225 | + return nil if results.empty? | |
| 226 | + | |
| 227 | + title = "#{label} documentations (#{results.length})" | |
| 228 | + headings = ['Documentation', 'Scraper version', 'Latest version'] | |
| 229 | + rows = results.map {|result| [result[:name], result[:scraper_version], result[:latest_version]]} | |
| 230 | + | |
| 231 | + results_to_markdown(title, headings, rows) | |
| 232 | + end | |
| 233 | + | |
| 234 | + def failed_results_to_markdown(results) | |
| 235 | + return nil if results.empty? | |
| 236 | + | |
| 237 | + title = "Documentations that could not be checked (#{results.length})" | |
| 238 | + headings = %w(Documentation Reason) | |
| 239 | + rows = results.map {|result| [result[:name], result[:error]]} | |
| 240 | + | |
| 241 | + results_to_markdown(title, headings, rows) | |
| 242 | + end | |
| 243 | + | |
| 244 | + def results_to_markdown(title, headings, rows) | |
| 245 | + "<details>\n<summary>#{title}</summary>\n\n#{create_markdown_table(headings, rows)}\n</details>" | |
| 246 | + end | |
| 247 | + | |
| 248 | + def create_markdown_table(headings, rows) | |
| 249 | + header = headings.join(' | ') | |
| 250 | + separator = '-|' * headings.length | |
| 251 | + body = rows.map {|row| row.join(' | ')} | |
| 252 | + | |
| 253 | + header + "\n" + separator[0...-1] + "\n" + body.join("\n") | |
| 254 | + end | |
| 255 | + | |
| 256 | + def log_upload_success(created_issue) | |
| 257 | + logger.info("Successfully uploaded the results to #{created_issue['html_url']}") | |
| 258 | + end | |
| 259 | + | |
| 260 | + # | |
| 261 | + # HTTP utilities | |
| 262 | + # | |
| 263 | + | |
| 264 | + def github_get(endpoint, params = {}) | |
| 265 | + github_request(endpoint, {method: :get, params: params}) | |
| 266 | + end | |
| 267 | + | |
| 268 | + def github_post(endpoint, params) | |
| 269 | + github_request(endpoint, {method: :post, body: params.to_json}) | |
| 270 | + end | |
| 271 | + | |
| 272 | + def github_patch(endpoint, params) | |
| 273 | + github_request(endpoint, {method: :patch, body: params.to_json}) | |
| 274 | + end | |
| 275 | + | |
| 276 | + def github_request(endpoint, opts) | |
| 277 | + url = "https://api.github.com#{endpoint}" | |
| 278 | + | |
| 279 | + # GitHub token authentication | |
| 280 | + opts[:headers] = { | |
| 281 | + Authorization: "token #{options[:github_token]}" | |
| 282 | + } | |
| 283 | + | |
| 284 | + # GitHub requires the Content-Type to be application/json when a body is passed | |
| 285 | + if opts.key?(:body) | |
| 286 | + opts[:headers]['Content-Type'] = 'application/json' | |
| 287 | + end | |
| 288 | + | |
| 289 | + logger.debug("Making a #{opts[:method]} request to #{url}") | |
| 290 | + response = Docs::Request.run(url, opts) | |
| 291 | + | |
| 292 | + # response.success? is false if the response code is 201 | |
| 293 | + # GitHub returns 201 Created after an issue is created | |
| 294 | + if response.success? || response.code == 201 | |
| 295 | + JSON.parse(response.body) | |
| 296 | + else | |
| 297 | + logger.error("Couldn't make a #{opts[:method]} request to #{url} (response code #{response.code})") | |
| 298 | + nil | |
| 299 | + end | |
| 300 | + end | |
| 301 | + | |
| 302 | + # A utility method which ensures no progress bar is shown when stdout is not a tty | |
| 303 | + def with_progress_bar(&block) | |
| 304 | + return unless $stdout.tty? | |
| 305 | + @progress_bar ||= ::ProgressBar.new | |
| 306 | + block.call @progress_bar | |
| 307 | + end | |
| 308 | + | |
| 309 | + def logger | |
| 310 | + @logger ||= Logger.new($stdout).tap do |logger| | |
| 311 | + logger.level = options[:verbose] ? Logger::DEBUG : Logger::INFO | |
| 312 | + logger.formatter = proc {|severity, datetime, progname, msg| "[#{severity}] #{msg}\n"} | |
| 313 | + end | |
| 314 | + end | |
| 315 | +end | |