aboutsummaryrefslogtreecommitdiff
path: root/pyaggr3g470r/lib/crawler.py
Commit message (Collapse)AuthorAge
* limiting the feed editing request by removing the dull updating of etag and ↵François Schmidts2015-10-21
| | | | last modified of no meaning
* fixing stupid, stupid bug where we instanciated multiple threadpoolFrançois Schmidts2015-10-20
|
* fixing bug preventing from bumping error count from api crawler for none ↵François Schmidts2015-08-05
| | | | admin user
* redoing the way the crawler wait on itselfFrançois Schmidts2015-08-05
|
* making feeds/fetchable and articles/challenge returns 204 on emptyFrançois Schmidts2015-08-05
|
* sqlalchemy was requesting icons everytime feed where listedFrançois Schmidts2015-08-03
| | | | so i choosed to move the icons into their own table
* ensuring the icon isn't empty and redoing a bit of loggingFrançois Schmidts2015-07-31
|
* making the crawler getting the feed with high traffic earlierFrançois Schmidts2015-07-07
|
* making it possible for super admin to update other's feedFrançois Schmidts2015-07-06
|
* light refactFrançois Schmidts2015-07-06
| | | | | * both crawlers now use the same utils methods * the original crawler use now more of the controllers, enabling the filters feature
* fixing bug on reset link for feeds and hidding feed title for eXtraSmall deviceFrançois Schmidts2015-07-06
|
* the icon featureFrançois Schmidts2015-07-03
| | | | | | * icon of feeds is now an url retrieved from the feed or the site link * the icon is displayed in the home page making it visually easier to read * the http crawler is in charge of keeping it up to date
* registering an strftime even if headers don't provide oneFrançois Schmidts2015-07-02
|
* correcting log level for error in feed retrievingFrançois Schmidts2015-05-13
|
* enforcing better user of user_id in controllersFrançois Schmidts2015-04-23
| | | | | | thus enhancing rights limitations between users wider_controller are a way to say "I was the maximum rights my role allows me"
* making admin able to update all other users feedFrançois Schmidts2015-04-22
|
* Merge remote-tracking branch 'upstream/master'François Schmidts2015-04-06
|\ | | | | | | | | | | Conflicts: pyaggr3g470r/controllers/feed.py pyaggr3g470r/templates/home.html
| * Capure logging for InsecureRequestWarning (https).Cédric Bonhomme2015-03-05
| |
* | misc updateFrançois Schmidts2015-04-06
| | | | | | | | | | | | updating the way we maintain feed up to date in the database fixing the counter bumping the minimum error count
* | implementing cache construction on crawler side (limiting useless pushes)François Schmidts2015-03-10
| |
* | failover for bad counter manipulationFrançois Schmidts2015-03-10
| |
* | better crawling crontrolFrançois Schmidts2015-03-06
| |
* | correcting wait counter and reactivating last_retrievedFrançois Schmidts2015-03-05
|/
* last fixesFrançois Schmidts2015-03-03
|
* adding some docstringFrançois Schmidts2015-03-03
|
* cleaning feed on okay download resulting in 304François Schmidts2015-03-03
|
* displaying feed errors in uiFrançois Schmidts2015-03-03
|
* multi crawler updateFrançois Schmidts2015-03-03
| | | | | | * adding action in the manager * removing debug code and adapting logging level * adding a wait method so it can be ran through cli
* fixing/restoring logging levelFrançois Schmidts2015-03-03
|
* new crawler with cache control and error handlingFrançois Schmidts2015-03-03
|
* continuing refactoFrançois Schmidts2015-03-03
bgstack15