Code

History

Commits on main to apps/api/src/scraper/scrapeURL35+
  1. build is failing. Getting back tsl config for engines. Keeping the fallbacklist without itrafaelmmiller committed 1 year ago69f0cc7
  2. Temporarily comment out TLS client engine options in the scraperrafaelmmiller committed 1 year ago1da56fe
  3. Remove TLS client engine options from the scraper temporarilyrafaelmmiller committed 1 year ago0a60985
  4. removing tlsclient from fallback list for nowrafaelmmiller committed 1 year ago2005a31
  5. Format api and add pre-commit hooks (ENG-3408) (#2088)tom committed 1 year agoc527796
  6. Fix sitemap parsing (ENG-3361) (#2085)tom committed 1 year ago688bf93
  7. Replace custom address validation functions with ipaddr.js (ENG-3404) (#2084)tom committed 1 year ago52a8fad
  8. fix(api/scrapeURL/index): re-sign expired screenshot URLs (#2080)Gergő Móricz committed 1 year ago4291af2
  9. fix(scrapeURL): title in metadataGergő Móricz committed 1 year ago309b1a8
  10. Update index.tsNicolas committed 1 year ago50d2335
  11. Update index.tsNicolas committed 1 year agoc1d6137
  12. Update index.tsNicolas committed 1 year ago4eacdb0
  13. Update index.tsNicolas committed 1 year ago0e8fd92
  14. Update index.tsNicolas committed 1 year agofd2a170
  15. feat: add maxPages parameter to PDF parser in v2 scrape API (#2047)devin-ai-integration[bot] committed 1 year ago7bea613
  16. feat(pdf-parser): get PDF title and show in metadata (#2062)Gergő Móricz committed 1 year agob3607eb
  17. fix(scrapeURL/fire-engine): turn off instantReturnGergő Móricz committed 1 year agod1418c8
  18. fix(scrapeURL/fire-engine): lower checkStatus frequencyGergő Móricz committed 1 year agob153a66
  19. feat(scrapeURL/rewrite): scrape Google Drive TXT/PDF files and sheets (#2053)Gergő Móricz committed 1 year ago822b95f
  20. feat(api): support extraction of data-* attributes in scrape endpoints (#2006)Vishnu Krishnan committed 1 year ago0589994
  21. feat(api): add image extraction support to v2 scrape endpoint (#2008)Vishnu Krishnan committed 1 year ago30c6bdd
  22. fix(map): use tlsclient to fetch sitemap for better reliability (#2034)Gergő Móricz committed 1 year ago879908a
  23. fix(scrapeURL): incorrect log messageGergő Móricz committed 1 year agod95d09c
  24. fix(scrapeURL): invalid Add/RemoveFeatureError handlingGergő Móricz committed 1 year ago34214ff
  25. fix(scrapeURL): fix HTML transforming logic (#1999)Gergő Móricz committed 1 year agoa4a2fcf
  26. fix(scrapeURL): odd timeout on large HTML (#1994)Gergő Móricz committed 1 year ago6af184d
  27. feat(api/abtest): send off AB requests at a higher levelGergő Móricz committed 1 year ago71218e7
  28. feat(scrapeURL): AI maxAgeGergő Móricz committed 1 year agof0665fa
  29. feat(scrapeURL): improve waterfalling MRT logicGergő Móricz committed 1 year agofacba27
  30. Merge branch 'main' into nsc/v2Gergő Móricz committed 1 year ago17008ac
  31. feat(v2/crawl): change sitemap paramGergő Móricz committed 1 year ago548f19a
  32. Update mu (#1959)Thomas Kosmas committed 1 year agocfb09fd
  33. Test new mu version (#1958)Thomas Kosmas committed 1 year ago8237be0
  34. Update mu (#1959)Thomas Kosmas committed 1 year agoefede5e
  35. Test new mu version (#1958)Thomas Kosmas committed 1 year agob2c6425

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t