diff options
| author | François Schmidts <francois.schmidts@gmail.com> | 2015-04-06 10:37:13 +0200 |
|---|---|---|
| committer | François Schmidts <francois.schmidts@gmail.com> | 2015-04-06 10:37:13 +0200 |
| commit | 29bb2a36f0b5d1781ab05f1976aa0c5017351807 (patch) | |
| tree | 52b2dd87f4d36f6a9c518cc14f96e523b1dea045 | |
| parent | misc update (diff) | |
| parent | Minor changes to the CSS. (diff) | |
| download | newspipe-29bb2a36f0b5d1781ab05f1976aa0c5017351807.tar.gz newspipe-29bb2a36f0b5d1781ab05f1976aa0c5017351807.tar.bz2 newspipe-29bb2a36f0b5d1781ab05f1976aa0c5017351807.zip | |
Merge remote-tracking branch 'upstream/master'
Conflicts:
pyaggr3g470r/controllers/feed.py
pyaggr3g470r/templates/home.html
53 files changed, 1249 insertions, 1004 deletions
@@ -1,5 +1,22 @@ pyAggr3g470r project news +6.4: 2015-03-17 + New: + * a new page 'history' to explore your database of articles. + Changes: + * updated documentation; + * minor improvements. + Fixes: + * changed the type of the column 'last_modified' to string. + +6.3: 2015-03-08 + New: + * a new architecture with base for controllers; + * new, more complete RESTful API; + * a crawler handling errors and cache control; + * the new crawler takes advantage of the RESTful API + (can be run on the client side). + 6.2: 2015-02-26 The system of email notifications for new articles has been removed. This feature was hardly used. diff --git a/bootstrap.py b/bootstrap.py index 5cfd2250..3b76d0ac 100644 --- a/bootstrap.py +++ b/bootstrap.py @@ -1,3 +1,6 @@ +#! /usr/bin/env python +# -*- coding: utf-8 - + # required imports and code exection for basic functionning import os diff --git a/conf/conf.cfg-sample b/conf/conf.cfg-sample index 4f1e06c6..e5b2683b 100644 --- a/conf/conf.cfg-sample +++ b/conf/conf.cfg-sample @@ -1,5 +1,5 @@ [misc] -platform_url = https://pyaggr3g470r.herokuapp.com/ +platform_url = http://127.0.0.1:5000/ admin_email = recaptcha_public_key = recaptcha_private_key = diff --git a/documentation/deployment.rst b/documentation/deployment.rst index 8a7f098d..59080b89 100644 --- a/documentation/deployment.rst +++ b/documentation/deployment.rst @@ -3,7 +3,8 @@ Deployment This application can be deployed on Heroku or on a traditional server. -After installation, you will be able to connect with the email *root@pyAggr3g470r.localhost* and the password *password*. +After installation, you will be able to connect with the email +*root@pyAggr3g470r.localhost* and the password *password*. Deploying the application with Vagrant -------------------------------------- @@ -13,8 +14,8 @@ Installation of VirtualBox and Vagrant .. code-block:: bash $ sudo apt-get install virtualbox - $ wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.6.5_x86_64.deb - $ sudo dpkg -i vagrant_1.6.5_x86_64.deb + $ wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.2_x86_64.deb + $ sudo dpkg -i vagrant_1.7.2_x86_64.deb Deployment of pyAggr3g470r @@ -45,7 +46,8 @@ The geek way $ heroku run init $ heroku ps:scale web=1 -To enable account creation for users, you have to set some environment variables: +To enable account creation for users, you have to set some environment +variables: .. code-block:: bash @@ -79,50 +81,27 @@ Deploying the application on a traditional server .. code-block:: bash - $ sudo apt-get install python libpq-dev python-dev python-pip build-essential git - $ sudo apt-get install libxml2-dev libxslt1-dev # for lxml $ git clone https://bitbucket.org/cedricbonhomme/pyaggr3g470r.git - $ cd pyaggr3g470r - $ sudo pip install --upgrade -r requirements.txt - $ cp conf/conf.cfg-sample conf/conf.cfg + $ cd pyaggr3g470r/ If you want to use PostgreSQL ''''''''''''''''''''''''''''' - .. code-block:: bash - $ sudo apt-get install postgresql postgresql-server-dev-9.3 postgresql-client - $ pip install psycopg2 - $ echo "127.0.0.1:5432:aggregator:pgsqluser:pgsqlpwd" > ~/.pgpass - $ chmod 700 ~/.pgpass - $ sudo -u postgres createuser pgsqluser --no-superuser --createdb --no-createrole - $ createdb aggregator --no-password - $ echo "ALTER USER pgsqluser WITH ENCRYPTED PASSWORD 'pgsqlpwd';" | sudo -u postgres psql - $ echo "GRANT ALL PRIVILEGES ON DATABASE aggregator TO pgsqluser;" | sudo -u postgres psql - -Edit the configuration file with the line: - -.. code-block:: cfg - - [database] - uri = postgres://pgsqluser:pgsqlpwd@127.0.0.1:5433/aggregator + $ ./install.sh postgres If you want to use SQLite ''''''''''''''''''''''''' -Just edit the configuration file with the line: - -.. code-block:: cfg +.. code-block:: bash - [database] - uri = sqlite+pysqlite:///pyAggr3g470r.db + $ ./install.sh sqlite Finally: .. code-block:: bash - $ python manager.py db_create $ python runserver.py * Running on http://0.0.0.0:5000/ * Restarting with reloader @@ -131,20 +110,22 @@ Finally: Configuration ============= -Configuration (database url, email, proxy, user agent, etc.) is done via the file *conf/conf.cfg*. -Check these configuration before executing *db_create.py*. -If you want to use pyAggr3g470r with Tor/Privoxy, you just have to set the value of -*http_proxy* (most of the time: *http_proxy = 127.0.0.1:8118**). Else leave the value blank. +Configuration (database url, email, proxy, user agent, etc.) is done via the +file `conf/conf.cfg`. +Check this file before initializing the database (with `manager.py`). + +If you want to use pyAggr3g470r with Tor/Privoxy, you just have to set the value +of `http_proxy` (most of the time: `http_proxy = 127.0.0.1:8118`). Else leave +the value blank. Automatic updates ================= -You can fetch new articles with `cron <https://en.wikipedia.org/wiki/Cron>`_ and the script *fetch.py*. -For example if you want to check for updates every 30 minutes, add this line to your cron rules (*crontab -e*): +You can fetch new articles with `cron <https://en.wikipedia.org/wiki/Cron>`_. +For example if you want to check for updates every 30 minutes, add this line to +your cron rules (*crontab -e*): .. code-block:: bash */30 * * * * cd ~/.pyaggr3g470r/ ; python manager.py fetch_asyncio None None - -You must give the email address you use to login to pyAggr3g470r. diff --git a/documentation/migrations.rst b/documentation/migrations.rst index 69f6c974..97d2c0f9 100644 --- a/documentation/migrations.rst +++ b/documentation/migrations.rst @@ -14,11 +14,11 @@ Local migrations .. code-block:: bash - $ python manage.py db upgrade + $ python manager.py db upgrade Remote migrations ----------------- .. code-block:: bash - $ heroku run python manage.py db upgrade + $ heroku run python manager.py db upgrade diff --git a/install.sh b/install.sh new file mode 100755 index 00000000..101e388c --- /dev/null +++ b/install.sh @@ -0,0 +1,42 @@ +#! /usr/bin/env bash + +# +# This script install all dependencies and configure pyAggr3g470r +# for Python 3. +# + +sudo apt-get install -y python libpq-dev python-dev python-pip build-essential git +sudo apt-get install -y libxml2-dev libxslt1-dev # for lxml + +sed -i '/psycopg2/d' requirements.txt +sudo pip install --upgrade -r requirements.txt + +# Initializes the configuration file +cp conf/conf.cfg-sample conf/conf.cfg + +# Delete default database configuration +sed -i '/database/d' conf/conf.cfg +sed -i '/uri/d' conf/conf.cfg + +if [ "$1" == postgres ]; then + sudo apt-get install -y postgresql postgresql-server-dev-9.3 postgresql-client + sudo pip install psycopg2 + echo "127.0.0.1:5432:aggregator:pgsqluser:pgsqlpwd" > ~/.pgpass + chmod 700 ~/.pgpass + sudo -u postgres createuser pgsqluser --no-superuser --createdb --no-createrole + createdb aggregator --no-password + echo "ALTER USER pgsqluser WITH ENCRYPTED PASSWORD 'pgsqlpwd';" | sudo -u postgres psql + echo "GRANT ALL PRIVILEGES ON DATABASE aggregator TO pgsqluser;" | sudo -u postgres psql + + # Add configuration lines for PostgreSQL + echo '[database]' >> conf/conf.cfg + echo 'uri = postgres://pgsqluser:pgsqlpwd@127.0.0.1:5432/aggregator' >> conf/conf.cfg +elif [ "$1" == sqlite ]; then + sudo pip install pysqlite # not working with Python 3! + # Add configuration lines for SQLite + echo '[database]' >> conf/conf.cfg + echo 'uri = sqlite+pysqlite:///pyAggr3g470r.db' >> conf/conf.cfg +fi + +python manager.py db_empty +python manager.py db_create @@ -1,20 +1,10 @@ -#!/usr/bin/env python -import os +#! /usr/bin/env python +# -*- coding: utf-8 -*- + from bootstrap import application, db, populate_g from flask.ext.script import Manager from flask.ext.migrate import Migrate, MigrateCommand -from werkzeug import generate_password_hash - -from sqlalchemy.engine import reflection -from sqlalchemy.schema import ( - MetaData, - Table, - DropTable, - ForeignKeyConstraint, - DropConstraint) - - Migrate(application, db) manager = Manager(application) @@ -23,62 +13,18 @@ manager.add_command('db', MigrateCommand) @manager.command def db_empty(): "Will drop every datas stocked in db." - # From http://www.sqlalchemy.org/trac/wiki/UsageRecipes/DropEverything - conn = db.engine.connect() - - # the transaction only applies if the DB supports - # transactional DDL, i.e. Postgresql, MS SQL Server - trans = conn.begin() - - inspector = reflection.Inspector.from_engine(db.engine) - - # gather all data first before dropping anything. - # some DBs lock after things have been dropped in - # a transaction. - metadata = MetaData() - - tbs = [] - all_fks = [] - - for table_name in inspector.get_table_names(): - fks = [] - for fk in inspector.get_foreign_keys(table_name): - if not fk['name']: - continue - fks.append(ForeignKeyConstraint((), (), name=fk['name'])) - t = Table(table_name, metadata, *fks) - tbs.append(t) - all_fks.extend(fks) - - for fkc in all_fks: - conn.execute(DropConstraint(fkc)) - - for table in tbs: - conn.execute(DropTable(table)) - - trans.commit() + with application.app_context(): + populate_g() + import pyaggr3g470r.models + pyaggr3g470r.models.db_empty(db) @manager.command def db_create(): "Will create the database from conf parameters." with application.app_context(): populate_g() - from pyaggr3g470r.models import User, Role - db.create_all() - - role_admin = Role(name="admin") - role_user = Role(name="user") - - user1 = User(nickname="admin", - email=os.environ.get("ADMIN_EMAIL", - "root@pyAggr3g470r.localhost"), - pwdhash=generate_password_hash( - os.environ.get("ADMIN_PASSWORD", "password")), - activation_key="") - user1.roles.extend([role_admin, role_user]) - - db.session.add(user1) - db.session.commit() + import pyaggr3g470r.models + pyaggr3g470r.models.db_create(db) @manager.command def fetch(user, password, limit=100): @@ -114,6 +60,5 @@ def fetch_asyncio(user_id, feed_id): print("Fetching articles for " + user.nickname) feed_getter = crawler.retrieve_feed(user, feed_id) - if __name__ == '__main__': - manager.run() + manager.run()
\ No newline at end of file diff --git a/messages.pot b/messages.pot index da07cc28..a8bdf457 100644 --- a/messages.pot +++ b/messages.pot @@ -8,7 +8,7 @@ msgid "" msgstr "" "Project-Id-Version: PROJECT VERSION\n" "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n" -"POT-Creation-Date: 2015-01-09 22:12+0100\n" +"POT-Creation-Date: 2015-03-28 11:26+0100\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" "Language-Team: LANGUAGE <LL@li.org>\n" @@ -17,308 +17,124 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 1.3\n" -#: pyaggr3g470r/forms.py:39 pyaggr3g470r/forms.py:99 +#: pyaggr3g470r/forms.py:40 pyaggr3g470r/forms.py:97 #: pyaggr3g470r/templates/admin/dashboard.html:12 msgid "Nickname" msgstr "" -#: pyaggr3g470r/forms.py:39 pyaggr3g470r/forms.py:99 +#: pyaggr3g470r/forms.py:41 pyaggr3g470r/forms.py:98 msgid "Please enter your nickname." msgstr "" -#: pyaggr3g470r/forms.py:40 pyaggr3g470r/forms.py:100 pyaggr3g470r/forms.py:121 +#: pyaggr3g470r/forms.py:42 pyaggr3g470r/forms.py:99 pyaggr3g470r/forms.py:133 #: pyaggr3g470r/templates/admin/dashboard.html:13 msgid "Email" msgstr "" -#: pyaggr3g470r/forms.py:40 pyaggr3g470r/forms.py:60 pyaggr3g470r/forms.py:121 +#: pyaggr3g470r/forms.py:45 pyaggr3g470r/forms.py:65 pyaggr3g470r/forms.py:136 msgid "Please enter your email address." msgstr "" -#: pyaggr3g470r/forms.py:41 pyaggr3g470r/forms.py:61 pyaggr3g470r/forms.py:101 +#: pyaggr3g470r/forms.py:46 pyaggr3g470r/forms.py:66 pyaggr3g470r/forms.py:102 msgid "Password" msgstr "" -#: pyaggr3g470r/forms.py:41 pyaggr3g470r/forms.py:61 +#: pyaggr3g470r/forms.py:47 pyaggr3g470r/forms.py:67 msgid "Please enter a password." msgstr "" -#: pyaggr3g470r/forms.py:43 pyaggr3g470r/templates/login.html:26 +#: pyaggr3g470r/forms.py:50 pyaggr3g470r/templates/login.html:26 msgid "Sign up" msgstr "" -#: pyaggr3g470r/forms.py:52 pyaggr3g470r/forms.py:111 +#: pyaggr3g470r/forms.py:55 pyaggr3g470r/forms.py:117 msgid "" "This nickname has invalid characters. Please use letters, numbers, dots " "and underscores only." msgstr "" -#: pyaggr3g470r/forms.py:62 pyaggr3g470r/templates/login.html:5 +#: pyaggr3g470r/forms.py:69 pyaggr3g470r/templates/login.html:5 msgid "Log In" msgstr "" -#: pyaggr3g470r/forms.py:75 +#: pyaggr3g470r/forms.py:80 msgid "Account not confirmed" msgstr "" -#: pyaggr3g470r/forms.py:78 +#: pyaggr3g470r/forms.py:83 msgid "Invalid email or password" msgstr "" -#: pyaggr3g470r/forms.py:83 pyaggr3g470r/templates/feeds.html:11 +#: pyaggr3g470r/forms.py:89 pyaggr3g470r/templates/feeds.html:11 msgid "Title" msgstr "" -#: pyaggr3g470r/forms.py:84 pyaggr3g470r/templates/admin/user.html:27 +#: pyaggr3g470r/forms.py:90 pyaggr3g470r/templates/admin/user.html:27 msgid "Feed link" msgstr "" -#: pyaggr3g470r/forms.py:85 pyaggr3g470r/templates/admin/user.html:28 +#: pyaggr3g470r/forms.py:91 pyaggr3g470r/templates/admin/user.html:28 msgid "Site link" msgstr "" -#: pyaggr3g470r/forms.py:86 -msgid "Email notification" -msgstr "" - -#: pyaggr3g470r/forms.py:87 +#: pyaggr3g470r/forms.py:92 msgid "Check for updates" msgstr "" -#: pyaggr3g470r/forms.py:88 pyaggr3g470r/forms.py:102 +#: pyaggr3g470r/forms.py:93 pyaggr3g470r/forms.py:107 msgid "Save" msgstr "" -#: pyaggr3g470r/forms.py:100 +#: pyaggr3g470r/forms.py:101 msgid "Please enter your email." msgstr "" -#: pyaggr3g470r/forms.py:116 +#: pyaggr3g470r/forms.py:103 +msgid "Password Confirmation" +msgstr "" + +#: pyaggr3g470r/forms.py:104 +msgid "Feeds refresh frequency (in minutes)" +msgstr "" + +#: pyaggr3g470r/forms.py:112 +msgid "Passwords aren't the same." +msgstr "" + +#: pyaggr3g470r/forms.py:125 msgid "Subject" msgstr "" -#: pyaggr3g470r/forms.py:116 +#: pyaggr3g470r/forms.py:126 msgid "Please enter a subject." msgstr "" -#: pyaggr3g470r/forms.py:117 +#: pyaggr3g470r/forms.py:127 msgid "Message" msgstr "" -#: pyaggr3g470r/forms.py:117 +#: pyaggr3g470r/forms.py:128 msgid "Please enter a content." msgstr "" -#: pyaggr3g470r/forms.py:118 +#: pyaggr3g470r/forms.py:129 msgid "Send" msgstr "" -#: pyaggr3g470r/forms.py:122 +#: pyaggr3g470r/forms.py:137 msgid "Recover" msgstr "" -#: pyaggr3g470r/forms.py:135 +#: pyaggr3g470r/forms.py:147 msgid "Account not confirmed." msgstr "" -#: pyaggr3g470r/forms.py:138 +#: pyaggr3g470r/forms.py:150 msgid "Invalid email." msgstr "" -#: pyaggr3g470r/views.py:99 -msgid "Authentication required." -msgstr "" - -#: pyaggr3g470r/views.py:104 -msgid "Forbidden." -msgstr "" - -#: pyaggr3g470r/views.py:156 -msgid "Logged in successfully." -msgstr "" - -#: pyaggr3g470r/views.py:178 -msgid "Logged out successfully." -msgstr "" - -#: pyaggr3g470r/views.py:187 -msgid "Self-registration is disabled." -msgstr "" - -#: pyaggr3g470r/views.py:204 -msgid "Email already used." -msgstr "" - -#: pyaggr3g470r/views.py:211 pyaggr3g470r/views.py:921 -msgid "Problem while sending activation email" -msgstr "" - -#: pyaggr3g470r/views.py:214 -msgid "Your account has been created. Check your mail to confirm it." -msgstr "" - -#: pyaggr3g470r/views.py:271 pyaggr3g470r/views.py:601 -msgid "Downloading articles..." -msgstr "" - -#: pyaggr3g470r/views.py:343 pyaggr3g470r/views.py:403 -msgid "This article do not exist." -msgstr "" - -#: pyaggr3g470r/views.py:400 pyaggr3g470r/templates/home.html:75 -msgid "Article" -msgstr "" - -#: pyaggr3g470r/views.py:400 -msgid "deleted." -msgstr "" - -#: pyaggr3g470r/views.py:499 -msgid "Indexing database..." -msgstr "" - -#: pyaggr3g470r/views.py:501 pyaggr3g470r/views.py:570 -msgid "An error occured" -msgstr "" - -#: pyaggr3g470r/views.py:504 -msgid "Option not available on Heroku." -msgstr "" - -#: pyaggr3g470r/views.py:519 pyaggr3g470r/views.py:529 -msgid "Error when exporting articles." -msgstr "" - -#: pyaggr3g470r/views.py:535 -msgid "Export format not supported." -msgstr "" - -#: pyaggr3g470r/views.py:558 -msgid "Full text search is not yet implemented for Heroku." -msgstr "" - -#: pyaggr3g470r/views.py:595 pyaggr3g470r/views.py:608 -#: pyaggr3g470r/views.py:616 -msgid "File not allowed." -msgstr "" - -#: pyaggr3g470r/views.py:600 -msgid "feeds imported." -msgstr "" - -#: pyaggr3g470r/views.py:603 -msgid "Impossible to import the new feeds." -msgstr "" - -#: pyaggr3g470r/views.py:612 -msgid "Account imported." -msgstr "" - -#: pyaggr3g470r/views.py:614 -msgid "Impossible to import the account." -msgstr "" - -#: pyaggr3g470r/views.py:652 -msgid "Feed successfully updated." -msgstr "" - -#: pyaggr3g470r/views.py:664 -msgid "Feed successfully created." -msgstr "" - -#: pyaggr3g470r/views.py:667 -msgid "Downloading articles for the new feed..." -msgstr "" - -#: pyaggr3g470r/views.py:671 -msgid "Feed already in the database." -msgstr "" - -#: pyaggr3g470r/views.py:677 -msgid "Edit the feed" -msgstr "" - -#: pyaggr3g470r/views.py:690 pyaggr3g470r/templates/layout.html:109 -msgid "Add a feed" -msgstr "" - -#: pyaggr3g470r/views.py:703 pyaggr3g470r/templates/home.html:74 -#: pyaggr3g470r/templates/admin/user.html:42 -msgid "Feed" -msgstr "" - -#: pyaggr3g470r/views.py:703 pyaggr3g470r/views.py:898 -msgid "successfully deleted." -msgstr "" - -#: pyaggr3g470r/views.py:721 pyaggr3g470r/views.py:848 -#: pyaggr3g470r/views.py:858 pyaggr3g470r/views.py:898 -msgid "User" -msgstr "" - -#: pyaggr3g470r/views.py:721 pyaggr3g470r/views.py:848 -msgid "successfully updated." -msgstr "" - -#: pyaggr3g470r/views.py:740 -msgid "Your account has been deleted." -msgstr "" - -#: pyaggr3g470r/views.py:742 pyaggr3g470r/views.py:884 -#: pyaggr3g470r/views.py:900 pyaggr3g470r/views.py:929 -msgid "This user does not exist." -msgstr "" - -#: pyaggr3g470r/views.py:756 -msgid "Articles deleted." -msgstr "" - -#: pyaggr3g470r/views.py:770 -msgid "Your account has been confirmed." -msgstr "" - -#: pyaggr3g470r/views.py:772 -msgid "Impossible to confirm this account." -msgstr "" - -#: pyaggr3g470r/views.py:796 -msgid "New password sent to your address." -msgstr "" - -#: pyaggr3g470r/views.py:798 -msgid "Problem while sending your new password." -msgstr "" - -#: pyaggr3g470r/views.py:823 -msgid "Problem while sending email" -msgstr "" - -#: pyaggr3g470r/views.py:858 -msgid "successfully created." -msgstr "" - -#: pyaggr3g470r/views.py:867 -msgid "Edit the user" -msgstr "" - -#: pyaggr3g470r/views.py:870 pyaggr3g470r/templates/admin/dashboard.html:45 -msgid "Add a new user" -msgstr "" - -#: pyaggr3g470r/views.py:919 pyaggr3g470r/views.py:926 -msgid "Account of the user" -msgstr "" - -#: pyaggr3g470r/views.py:919 -msgid "successfully activated." -msgstr "" - -#: pyaggr3g470r/views.py:926 -msgid "successfully disabled." -msgstr "" - -#: pyaggr3g470r/templates/about.html:5 pyaggr3g470r/templates/layout.html:126 -#: pyaggr3g470r/templates/layout.html:143 +#: pyaggr3g470r/templates/about.html:5 pyaggr3g470r/templates/layout.html:117 +#: pyaggr3g470r/templates/layout.html:134 msgid "About" msgstr "" @@ -368,8 +184,8 @@ msgstr "" #: pyaggr3g470r/templates/about.html:19 msgid "" "The documentation of the RESTful API is <a " -"href=\"https://bitbucket.org/cedricbonhomme/pyaggr3g470r#rst-header-web-" -"service\">here</a>." +"href=\"https://pyaggr3g470r.readthedocs.org/en/latest/web-" +"services.html\">here</a>." msgstr "" #: pyaggr3g470r/templates/about.html:20 @@ -395,25 +211,25 @@ msgid "from" msgstr "" #: pyaggr3g470r/templates/article.html:11 -#: pyaggr3g470r/templates/duplicates.html:19 -#: pyaggr3g470r/templates/duplicates.html:20 -#: pyaggr3g470r/templates/home.html:89 +#: pyaggr3g470r/templates/duplicates.html:23 +#: pyaggr3g470r/templates/duplicates.html:24 +#: pyaggr3g470r/templates/home.html:90 msgid "Delete this article" msgstr "" -#: pyaggr3g470r/templates/article.html:13 pyaggr3g470r/templates/home.html:91 +#: pyaggr3g470r/templates/article.html:13 pyaggr3g470r/templates/home.html:92 msgid "One of your favorites" msgstr "" -#: pyaggr3g470r/templates/article.html:15 pyaggr3g470r/templates/home.html:93 +#: pyaggr3g470r/templates/article.html:15 pyaggr3g470r/templates/home.html:94 msgid "Click if you like this article" msgstr "" -#: pyaggr3g470r/templates/article.html:18 pyaggr3g470r/templates/home.html:96 +#: pyaggr3g470r/templates/article.html:18 pyaggr3g470r/templates/home.html:97 msgid "Mark this article as unread" msgstr "" -#: pyaggr3g470r/templates/article.html:20 pyaggr3g470r/templates/home.html:98 +#: pyaggr3g470r/templates/article.html:20 pyaggr3g470r/templates/home.html:99 msgid "Mark this article as read" msgstr "" @@ -434,7 +250,12 @@ msgstr "" msgid "Duplicates in the feed" msgstr "" -#: pyaggr3g470r/templates/duplicates.html:27 +#: pyaggr3g470r/templates/duplicates.html:12 +#: pyaggr3g470r/templates/duplicates.html:15 +msgid "Delete all in this column" +msgstr "" + +#: pyaggr3g470r/templates/duplicates.html:31 msgid "No duplicates in the feed" msgstr "" @@ -451,26 +272,26 @@ msgstr "" msgid "More articles" msgstr "" -#: pyaggr3g470r/templates/favorites.html:17 pyaggr3g470r/templates/home.html:29 -#: pyaggr3g470r/templates/home.html:44 pyaggr3g470r/templates/unread.html:17 +#: pyaggr3g470r/templates/favorites.html:17 pyaggr3g470r/templates/home.html:32 +#: pyaggr3g470r/templates/home.html:50 pyaggr3g470r/templates/unread.html:17 msgid "Details" msgstr "" #: pyaggr3g470r/templates/favorites.html:18 pyaggr3g470r/templates/feed.html:8 -#: pyaggr3g470r/templates/feeds.html:33 pyaggr3g470r/templates/home.html:31 -#: pyaggr3g470r/templates/home.html:46 pyaggr3g470r/templates/unread.html:18 +#: pyaggr3g470r/templates/feeds.html:33 pyaggr3g470r/templates/home.html:34 +#: pyaggr3g470r/templates/home.html:52 pyaggr3g470r/templates/unread.html:18 #: pyaggr3g470r/templates/admin/user.html:43 msgid "Edit this feed" msgstr "" #: pyaggr3g470r/templates/feed.html:7 pyaggr3g470r/templates/feeds.html:35 -#: pyaggr3g470r/templates/home.html:32 pyaggr3g470r/templates/home.html:47 +#: pyaggr3g470r/templates/home.html:35 pyaggr3g470r/templates/home.html:53 #: pyaggr3g470r/templates/admin/user.html:44 msgid "Delete this feed" msgstr "" #: pyaggr3g470r/templates/feed.html:7 pyaggr3g470r/templates/feeds.html:35 -#: pyaggr3g470r/templates/home.html:32 pyaggr3g470r/templates/home.html:47 +#: pyaggr3g470r/templates/home.html:35 pyaggr3g470r/templates/home.html:53 #: pyaggr3g470r/templates/admin/user.html:44 msgid "You are going to delete this feed." msgstr "" @@ -483,35 +304,51 @@ msgstr "" msgid "articles" msgstr "" -#: pyaggr3g470r/templates/feed.html:14 -msgid "of the database" -msgstr "" - -#: pyaggr3g470r/templates/feed.html:17 +#: pyaggr3g470r/templates/feed.html:13 msgid "Address of the feed" msgstr "" -#: pyaggr3g470r/templates/feed.html:19 +#: pyaggr3g470r/templates/feed.html:15 msgid "Address of the site" msgstr "" -#: pyaggr3g470r/templates/feed.html:23 +#: pyaggr3g470r/templates/feed.html:21 +msgid "Last download:" +msgstr "" + +#: pyaggr3g470r/templates/feed.html:25 +msgid "" +"That feed has encountered too much consecutive errors and won't be " +"retrieved anymore." +msgstr "" + +#: pyaggr3g470r/templates/feed.html:27 +msgid "" +"The download of this feed has encountered some problems. However its " +"error counter will be reinitialized at the next successful retrieving." +msgstr "" + +#: pyaggr3g470r/templates/feed.html:31 +msgid "Here's the last error encountered while retrieving this feed:" +msgstr "" + +#: pyaggr3g470r/templates/feed.html:35 msgid "The last article was posted" msgstr "" -#: pyaggr3g470r/templates/feed.html:23 +#: pyaggr3g470r/templates/feed.html:35 msgid "day(s) ago." msgstr "" -#: pyaggr3g470r/templates/feed.html:24 +#: pyaggr3g470r/templates/feed.html:36 msgid "Daily average" msgstr "" -#: pyaggr3g470r/templates/feed.html:24 +#: pyaggr3g470r/templates/feed.html:36 msgid "between the" msgstr "" -#: pyaggr3g470r/templates/feed.html:24 +#: pyaggr3g470r/templates/feed.html:36 msgid "and the" msgstr "" @@ -540,8 +377,8 @@ msgid "Site" msgstr "" #: pyaggr3g470r/templates/feeds.html:13 pyaggr3g470r/templates/feeds.html:32 -#: pyaggr3g470r/templates/home.html:30 pyaggr3g470r/templates/home.html:45 -#: pyaggr3g470r/templates/layout.html:104 +#: pyaggr3g470r/templates/home.html:33 pyaggr3g470r/templates/home.html:51 +#: pyaggr3g470r/templates/layout.html:95 msgid "Articles" msgstr "" @@ -563,6 +400,14 @@ msgstr "" msgid "Duplicate articles" msgstr "" +#: pyaggr3g470r/templates/history.html:4 pyaggr3g470r/templates/layout.html:110 +msgid "History" +msgstr "" + +#: pyaggr3g470r/templates/history.html:9 +msgid "all years" +msgstr "" + #: pyaggr3g470r/templates/home.html:10 msgid "You don't have any feeds." msgstr "" @@ -580,31 +425,44 @@ msgstr "" msgid "upload an OPML file." msgstr "" -#: pyaggr3g470r/templates/home.html:18 pyaggr3g470r/templates/layout.html:118 +#: pyaggr3g470r/templates/home.html:18 pyaggr3g470r/templates/layout.html:109 msgid "All feeds" msgstr "" -#: pyaggr3g470r/templates/home.html:33 pyaggr3g470r/templates/home.html:48 +#: pyaggr3g470r/templates/home.html:25 pyaggr3g470r/templates/home.html:43 +msgid "error" +msgstr "" + +#: pyaggr3g470r/templates/home.html:36 pyaggr3g470r/templates/home.html:54 msgid "Mark this feed as read" msgstr "" -#: pyaggr3g470r/templates/home.html:34 pyaggr3g470r/templates/home.html:49 +#: pyaggr3g470r/templates/home.html:37 pyaggr3g470r/templates/home.html:55 msgid "Mark this feed as unread" msgstr "" -#: pyaggr3g470r/templates/home.html:57 pyaggr3g470r/templates/home.html:65 +#: pyaggr3g470r/templates/home.html:63 pyaggr3g470r/templates/home.html:71 msgid "All" msgstr "" -#: pyaggr3g470r/templates/home.html:58 +#: pyaggr3g470r/templates/home.html:64 msgid "Read" msgstr "" -#: pyaggr3g470r/templates/home.html:59 pyaggr3g470r/templates/layout.html:115 +#: pyaggr3g470r/templates/home.html:65 pyaggr3g470r/templates/layout.html:106 msgid "Unread" msgstr "" -#: pyaggr3g470r/templates/home.html:76 +#: pyaggr3g470r/templates/home.html:81 +#: pyaggr3g470r/templates/admin/user.html:42 pyaggr3g470r/views/views.py:630 +msgid "Feed" +msgstr "" + +#: pyaggr3g470r/templates/home.html:82 pyaggr3g470r/views/views.py:336 +msgid "Article" +msgstr "" + +#: pyaggr3g470r/templates/home.html:83 msgid "Date" msgstr "" @@ -620,39 +478,39 @@ msgstr "" msgid "No inactive feeds." msgstr "" -#: pyaggr3g470r/templates/layout.html:106 +#: pyaggr3g470r/templates/layout.html:97 msgid "Fetch" msgstr "" -#: pyaggr3g470r/templates/layout.html:107 +#: pyaggr3g470r/templates/layout.html:98 msgid "Mark all as read" msgstr "" -#: pyaggr3g470r/templates/layout.html:113 +#: pyaggr3g470r/templates/layout.html:100 pyaggr3g470r/views/views.py:617 +msgid "Add a feed" +msgstr "" + +#: pyaggr3g470r/templates/layout.html:104 msgid "Filter" msgstr "" -#: pyaggr3g470r/templates/layout.html:116 +#: pyaggr3g470r/templates/layout.html:107 msgid "Favorites" msgstr "" -#: pyaggr3g470r/templates/layout.html:117 +#: pyaggr3g470r/templates/layout.html:108 msgid "Inactive feeds" msgstr "" -#: pyaggr3g470r/templates/layout.html:119 -msgid "History" -msgstr "" - -#: pyaggr3g470r/templates/layout.html:122 +#: pyaggr3g470r/templates/layout.html:113 msgid "Management" msgstr "" -#: pyaggr3g470r/templates/layout.html:124 +#: pyaggr3g470r/templates/layout.html:115 msgid "Dashboard" msgstr "" -#: pyaggr3g470r/templates/layout.html:127 +#: pyaggr3g470r/templates/layout.html:118 msgid "Logout" msgstr "" @@ -804,6 +662,11 @@ msgstr "" msgid "You are going to delete this account." msgstr "" +#: pyaggr3g470r/templates/admin/dashboard.html:45 +#: pyaggr3g470r/views/views.py:797 +msgid "Add a new user" +msgstr "" + #: pyaggr3g470r/templates/admin/dashboard.html:46 msgid "Send notification messages" msgstr "" @@ -828,3 +691,184 @@ msgstr "" msgid "Number of articles" msgstr "" +#: pyaggr3g470r/views/views.py:103 +msgid "Authentication required." +msgstr "" + +#: pyaggr3g470r/views/views.py:108 +msgid "Forbidden." +msgstr "" + +#: pyaggr3g470r/views/views.py:161 +msgid "Logged in successfully." +msgstr "" + +#: pyaggr3g470r/views/views.py:184 +msgid "Logged out successfully." +msgstr "" + +#: pyaggr3g470r/views/views.py:193 +msgid "Self-registration is disabled." +msgstr "" + +#: pyaggr3g470r/views/views.py:210 +msgid "Email already used." +msgstr "" + +#: pyaggr3g470r/views/views.py:217 pyaggr3g470r/views/views.py:848 +msgid "Problem while sending activation email" +msgstr "" + +#: pyaggr3g470r/views/views.py:220 +msgid "Your account has been created. Check your mail to confirm it." +msgstr "" + +#: pyaggr3g470r/views/views.py:269 pyaggr3g470r/views/views.py:519 +msgid "Downloading articles..." +msgstr "" + +#: pyaggr3g470r/views/views.py:271 +msgid "" +"The manual retrieving of news is only available for administrator, on the" +" Heroku platform." +msgstr "" + +#: pyaggr3g470r/views/views.py:336 +msgid "deleted." +msgstr "" + +#: pyaggr3g470r/views/views.py:339 +msgid "This article do not exist." +msgstr "" + +#: pyaggr3g470r/views/views.py:417 +msgid "Indexing database..." +msgstr "" + +#: pyaggr3g470r/views/views.py:419 pyaggr3g470r/views/views.py:488 +msgid "An error occured" +msgstr "" + +#: pyaggr3g470r/views/views.py:422 +msgid "Option not available on Heroku." +msgstr "" + +#: pyaggr3g470r/views/views.py:437 pyaggr3g470r/views/views.py:447 +msgid "Error when exporting articles." +msgstr "" + +#: pyaggr3g470r/views/views.py:453 +msgid "Export format not supported." +msgstr "" + +#: pyaggr3g470r/views/views.py:476 +msgid "Full text search is not yet implemented for Heroku." +msgstr "" + +#: pyaggr3g470r/views/views.py:513 pyaggr3g470r/views/views.py:526 +#: pyaggr3g470r/views/views.py:534 +msgid "File not allowed." +msgstr "" + +#: pyaggr3g470r/views/views.py:518 +msgid "feeds imported." +msgstr "" + +#: pyaggr3g470r/views/views.py:521 +msgid "Impossible to import the new feeds." +msgstr "" + +#: pyaggr3g470r/views/views.py:530 +msgid "Account imported." +msgstr "" + +#: pyaggr3g470r/views/views.py:532 +msgid "Impossible to import the account." +msgstr "" + +#: pyaggr3g470r/views/views.py:579 +msgid "Feed successfully updated." +msgstr "" + +#: pyaggr3g470r/views/views.py:590 +msgid "Feed successfully created." +msgstr "" + +#: pyaggr3g470r/views/views.py:593 +msgid "Downloading articles for the new feed..." +msgstr "" + +#: pyaggr3g470r/views/views.py:597 +msgid "Feed already in the database." +msgstr "" + +#: pyaggr3g470r/views/views.py:604 +msgid "Edit the feed" +msgstr "" + +#: pyaggr3g470r/views/views.py:630 pyaggr3g470r/views/views.py:825 +msgid "successfully deleted." +msgstr "" + +#: pyaggr3g470r/views/views.py:648 pyaggr3g470r/views/views.py:775 +#: pyaggr3g470r/views/views.py:785 pyaggr3g470r/views/views.py:825 +msgid "User" +msgstr "" + +#: pyaggr3g470r/views/views.py:649 pyaggr3g470r/views/views.py:775 +msgid "successfully updated." +msgstr "" + +#: pyaggr3g470r/views/views.py:669 +msgid "Your account has been deleted." +msgstr "" + +#: pyaggr3g470r/views/views.py:671 pyaggr3g470r/views/views.py:811 +#: pyaggr3g470r/views/views.py:827 pyaggr3g470r/views/views.py:855 +msgid "This user does not exist." +msgstr "" + +#: pyaggr3g470r/views/views.py:685 +msgid "Articles deleted." +msgstr "" + +#: pyaggr3g470r/views/views.py:699 +msgid "Your account has been confirmed." +msgstr "" + +#: pyaggr3g470r/views/views.py:701 +msgid "Impossible to confirm this account." +msgstr "" + +#: pyaggr3g470r/views/views.py:723 +msgid "New password sent to your address." +msgstr "" + +#: pyaggr3g470r/views/views.py:725 +msgid "Problem while sending your new password." +msgstr "" + +#: pyaggr3g470r/views/views.py:750 +msgid "Problem while sending email" +msgstr "" + +#: pyaggr3g470r/views/views.py:785 +msgid "successfully created." +msgstr "" + +#: pyaggr3g470r/views/views.py:794 +msgid "Edit the user" +msgstr "" + +#: pyaggr3g470r/views/views.py:846 pyaggr3g470r/views/views.py:852 +msgid "Account of the user" +msgstr "" + +#: pyaggr3g470r/views/views.py:846 +msgid "successfully activated." +msgstr "" + +#: pyaggr3g470r/views/views.py:852 +msgid "successfully disabled." +msgstr "" + diff --git a/migrations/versions/17dcb75f3fe_changed_the_type_of_the_column_last_.py b/migrations/versions/17dcb75f3fe_changed_the_type_of_the_column_last_.py new file mode 100644 index 00000000..e7790b3f --- /dev/null +++ b/migrations/versions/17dcb75f3fe_changed_the_type_of_the_column_last_.py @@ -0,0 +1,29 @@ +"""changed the type of the column 'last_modified' to string. + +Revision ID: 17dcb75f3fe +Revises: cde34831ea +Create Date: 2015-03-10 14:20:53.676344 + +""" + +# revision identifiers, used by Alembic. +revision = '17dcb75f3fe' +down_revision = 'cde34831ea' + +from alembic import op +import sqlalchemy as sa + +from datetime import datetime + +def upgrade(): + unix_start = datetime(1970, 1, 1) + op.drop_column('feed', 'last_modified') + op.add_column('feed', sa.Column('last_modified', sa.String(), + nullable=True, default=unix_start, server_default=str(unix_start))) + + +def downgrade(): + unix_start = datetime(1970, 1, 1) + op.drop_column('feed', 'last_modified') + op.add_column('feed', sa.Column('last_modified', sa.DateTime(), + nullable=True, default=unix_start, server_default=unix_start)) diff --git a/pyaggr3g470r/controllers/abstract.py b/pyaggr3g470r/controllers/abstract.py index a99e67f3..c084deb9 100644 --- a/pyaggr3g470r/controllers/abstract.py +++ b/pyaggr3g470r/controllers/abstract.py @@ -1,6 +1,5 @@ import logging from bootstrap import db -from sqlalchemy import update from werkzeug.exceptions import Forbidden, NotFound logger = logging.getLogger(__name__) diff --git a/pyaggr3g470r/controllers/user.py b/pyaggr3g470r/controllers/user.py index c6c1d545..ed46e1e7 100644 --- a/pyaggr3g470r/controllers/user.py +++ b/pyaggr3g470r/controllers/user.py @@ -4,4 +4,4 @@ from pyaggr3g470r.models import User class UserController(AbstractController): _db_cls = User - _user_id_key = 'id' + _user_id_key = 'email' diff --git a/pyaggr3g470r/crawler.py b/pyaggr3g470r/crawler.py index ebcb8ce4..ded9df6f 100644 --- a/pyaggr3g470r/crawler.py +++ b/pyaggr3g470r/crawler.py @@ -56,7 +56,7 @@ def get(*args, **kwargs): return (yield from response.read_and_close(decode=False)) except Exception as e: #print(e) - return None + raise e @asyncio.coroutine def parse_feed(user, feed): @@ -66,14 +66,17 @@ def parse_feed(user, feed): data = None with (yield from sem): - data = yield from get(feed.link) - - if data is None: - feed.error_count += 1 - if feed.error_count > 2: - feed.enabled = False - db.session.commit() - return + try: + data = yield from get(feed.link) + except Exception as e: + feed.last_error = str(e) + finally: + if data is None: + feed.error_count += 1 + if feed.error_count > 2: + feed.enabled = False + db.session.commit() + return a_feed = feedparser.parse(data) if a_feed['bozo'] == 1: @@ -88,6 +91,7 @@ def parse_feed(user, feed): feed.last_retrieved = datetime.now(dateutil.tz.tzlocal()) feed.error_count = 0 + feed.last_error = "" # Feed informations if feed.title == "": diff --git a/pyaggr3g470r/duplicate.py b/pyaggr3g470r/duplicate.py deleted file mode 100644 index d4c6e31a..00000000 --- a/pyaggr3g470r/duplicate.py +++ /dev/null @@ -1,20 +0,0 @@ -#! /usr/bin/env python -#-*- coding: utf-8 -*- - -import itertools -from datetime import timedelta - -from pyaggr3g470r import utils - -def compare_documents(feed): - """ - Compare a list of documents by pair. - """ - duplicates = [] - for pair in itertools.combinations(feed.articles, 2): - date1 = pair[0].date - date2 = pair[1].date - if utils.clear_string(pair[0].title) == utils.clear_string(pair[1].title) and \ - (date1 - date2) < timedelta(days = 1): - duplicates.append(pair) - return duplicates
\ No newline at end of file diff --git a/pyaggr3g470r/lib/crawler.py b/pyaggr3g470r/lib/crawler.py index 1ac6029a..1cb61973 100644 --- a/pyaggr3g470r/lib/crawler.py +++ b/pyaggr3g470r/lib/crawler.py @@ -27,6 +27,7 @@ from requests_futures.sessions import FuturesSession from pyaggr3g470r.lib.utils import default_handler logger = logging.getLogger(__name__) +logging.captureWarnings(True) API_ROOT = "api/v2.0/" diff --git a/pyaggr3g470r/models/__init__.py b/pyaggr3g470r/models/__init__.py index 9584d1f2..42903f4e 100644 --- a/pyaggr3g470r/models/__init__.py +++ b/pyaggr3g470r/models/__init__.py @@ -31,5 +31,71 @@ from .role import Role from .user import User from .article import Article - __all__ = ['Feed', 'Role', 'User', 'Article'] + +import os + +from werkzeug import generate_password_hash + +from sqlalchemy.engine import reflection +from sqlalchemy.schema import ( + MetaData, + Table, + DropTable, + ForeignKeyConstraint, + DropConstraint) + +def db_empty(db): + "Will drop every datas stocked in db." + # From http://www.sqlalchemy.org/trac/wiki/UsageRecipes/DropEverything + conn = db.engine.connect() + + # the transaction only applies if the DB supports + # transactional DDL, i.e. Postgresql, MS SQL Server + trans = conn.begin() + + inspector = reflection.Inspector.from_engine(db.engine) + + # gather all data first before dropping anything. + # some DBs lock after things have been dropped in + # a transaction. + metadata = MetaData() + + tbs = [] + all_fks = [] + + for table_name in inspector.get_table_names(): + fks = [] + for fk in inspector.get_foreign_keys(table_name): + if not fk['name']: + continue + fks.append(ForeignKeyConstraint((), (), name=fk['name'])) + t = Table(table_name, metadata, *fks) + tbs.append(t) + all_fks.extend(fks) + + for fkc in all_fks: + conn.execute(DropConstraint(fkc)) + + for table in tbs: + conn.execute(DropTable(table)) + + trans.commit() + +def db_create(db): + "Will create the database from conf parameters." + db.create_all() + + role_admin = Role(name="admin") + role_user = Role(name="user") + + user1 = User(nickname="admin", + email=os.environ.get("ADMIN_EMAIL", + "root@pyAggr3g470r.localhost"), + pwdhash=generate_password_hash( + os.environ.get("ADMIN_PASSWORD", "password")), + activation_key="") + user1.roles.extend([role_admin, role_user]) + + db.session.add(user1) + db.session.commit()
\ No newline at end of file diff --git a/pyaggr3g470r/models/article.py b/pyaggr3g470r/models/article.py index f8f9d2d8..58cd0384 100644 --- a/pyaggr3g470r/models/article.py +++ b/pyaggr3g470r/models/article.py @@ -4,7 +4,7 @@ # pyAggr3g470r - A Web based news aggregator. # Copyright (C) 2010-2015 Cédric Bonhomme - https://www.cedricbonhomme.org # -# For more information : https://bitbucket.org/cedricbonhomme/pyaggr3g470r/ +# For more information : https://bitbucket.org/cedricbonhomme/pyaggr3g470r # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as diff --git a/pyaggr3g470r/models/feed.py b/pyaggr3g470r/models/feed.py index a37744d6..a36d9573 100644 --- a/pyaggr3g470r/models/feed.py +++ b/pyaggr3g470r/models/feed.py @@ -33,7 +33,7 @@ from sqlalchemy import desc class Feed(db.Model): """ - Represent a station. + Represent a feed. """ id = db.Column(db.Integer, primary_key=True) title = db.Column(db.String(), default="No title") diff --git a/pyaggr3g470r/notifications.py b/pyaggr3g470r/notifications.py index cf8fb723..006aa594 100644 --- a/pyaggr3g470r/notifications.py +++ b/pyaggr3g470r/notifications.py @@ -30,10 +30,12 @@ def information_message(subject, plaintext): from pyaggr3g470r.models import User users = User.query.all() # Only send email for activated accounts. - emails = [user.email for user in users if user.activation_key == ""] + user_emails = [user.email for user in users if user.activation_key == ""] # Postmark has a limit of twenty recipients per message in total. - for i in xrange(0, len(emails), 19): - emails.send(to=conf.NOTIFICATION_EMAIL, bcc=", ".join(emails[i:i+19]), subject=subject, plaintext=plaintext) + for i in xrange(0, len(user_emails), 19): + emails.send(to=conf.NOTIFICATION_EMAIL, + bcc=", ".join(user_emails[i:i+19]), + subject=subject, plaintext=plaintext) def new_account_notification(user): """ @@ -41,7 +43,8 @@ def new_account_notification(user): """ plaintext = """Hello,\n\nYour account has been created. Click on the following link to confirm it:\n%s\n\nSee you,""" % \ (conf.PLATFORM_URL + 'confirm_account/' + user.activation_key) - emails.send(to=user.email, bcc=conf.NOTIFICATION_EMAIL, subject="[pyAggr3g470r] Account creation", plaintext=plaintext) + emails.send(to=user.email, bcc=conf.NOTIFICATION_EMAIL, + subject="[pyAggr3g470r] Account creation", plaintext=plaintext) def new_account_activation(user): """ @@ -49,7 +52,8 @@ def new_account_activation(user): """ plaintext = """Hello,\n\nYour account has been activated. You can now connect to the platform:\n%s\n\nSee you,""" % \ (conf.PLATFORM_URL) - emails.send(to=user.email, bcc=conf.NOTIFICATION_EMAIL, subject="[pyAggr3g470r] Account activated", plaintext=plaintext) + emails.send(to=user.email, bcc=conf.NOTIFICATION_EMAIL, + subject="[pyAggr3g470r] Account activated", plaintext=plaintext) def new_password_notification(user, password): """ @@ -58,4 +62,6 @@ def new_password_notification(user, password): plaintext = """Hello,\n\nA new password has been generated at your request:\n\n%s""" % \ (password, ) plaintext += "\n\nIt is advised to replace it as soon as connected to pyAggr3g470r.\n\nSee you," - emails.send(to=user.email, bcc=conf.NOTIFICATION_EMAIL, subject="[pyAggr3g470r] New password", plaintext=plaintext) + emails.send(to=user.email, + bcc=conf.NOTIFICATION_EMAIL, + subject="[pyAggr3g470r] New password", plaintext=plaintext) diff --git a/pyaggr3g470r/static/css/customized-bootstrap.css b/pyaggr3g470r/static/css/customized-bootstrap.css new file mode 100644 index 00000000..58a9d182 --- /dev/null +++ b/pyaggr3g470r/static/css/customized-bootstrap.css @@ -0,0 +1,44 @@ +body { + margin-top: 50px; +} +div.top { + position: relative; + top: -50px; + display: block; + height: 0; +} + +.navbar-custom { + background-color: #205081; + border: #205081; + color: #FFFFFF; + border-radius: 0; +} + +.navbar-custom .navbar-nav > li > a { + color: #FFFFFF; +} + +.navbar-custom .navbar-nav > li > a:hover { + background-color: #3572B0; +} + +.navbar-custom .navbar-nav > .active > a, +.navbar-nav > .active > a:hover, +.navbar-nav > .active > a:focus { + color: #FFFFFF; + background-color: #3572B0; +} +.navbar-custom .navbar-brand { + color: #FFFFFF; +} + +.navbar-custom .navbar-nav > .open > a, +.navbar-custom .navbar-nav > .open > a:hover, +.navbar-custom .navbar-nav > .open > a:focus { + color: #FFFFFF; + background-color: #3572B0; +} +a { + color: #3572B0; +} diff --git a/pyaggr3g470r/static/css/side-nav.css b/pyaggr3g470r/static/css/side-nav.css new file mode 100644 index 00000000..5786bffa --- /dev/null +++ b/pyaggr3g470r/static/css/side-nav.css @@ -0,0 +1,50 @@ +/* First level of nav */ +.sidenav { + margin-top: 0px; + margin-bottom: 0px; + padding-top: 10px; + padding-bottom: 0px; + overflow-y: auto; + height: 90%; + z-index: 1000; + background-color: #FFFFFF; + border-radius: 2px; + font-size: 100%; +} +/* All levels of nav */ +.sidebar .nav > li > a { + display: block; + color: #3572B0; + padding: 5px 20px; +} +.sidebar .nav > li > a:hover, +.sidebar .nav > li > a:focus { + text-decoration: none; + background-color: #F0FFFF; +} + +.sidebar .nav > .active > a, +.sidebar .nav > .active:hover > a, +.sidebar .nav > .active:focus > a { + font-weight: bold; + color: #3572B0; + background-color: transparent; +} + +.badge { + background-color: #3572B0; +} + +/* Nav: second level */ +.sidebar .nav .nav { + margin-bottom: 8px; +} +.sidebar .nav .nav > li > a { + padding-top: 3px; + padding-bottom: 3px; + font-size: 80%; +} + +li.feed-commands {display: none; text-align: right;} +li.feed-commands > span > a {margin-right: 10px;} +li.feed-menu:hover + li.feed-commands, li.feed-commands:hover {display: block;} diff --git a/pyaggr3g470r/static/js/articles.js b/pyaggr3g470r/static/js/articles.js index 312a5cb6..a5ac82d0 100644 --- a/pyaggr3g470r/static/js/articles.js +++ b/pyaggr3g470r/static/js/articles.js @@ -18,7 +18,7 @@ * along with this program. If not, see <http://www.gnu.org/licenses/>. */ -API_ROOT = 'api/v2.0/' +API_ROOT = '/api/v2.0/' if (typeof jQuery === 'undefined') { throw new Error('Requires jQuery') } @@ -131,7 +131,7 @@ if (typeof jQuery === 'undefined') { throw new Error('Requires jQuery') } var article_id = $(this).parent().parent().parent().attr("data-article"); $(this).parent().parent().parent().remove(); - // sends the updates to the server + // sends the updates to the server $.ajax({ type: 'DELETE', url: API_ROOT + "article/" + article_id, @@ -144,4 +144,35 @@ if (typeof jQuery === 'undefined') { throw new Error('Requires jQuery') } }); }); + + // Delete all duplicate articles (used in the page /duplicates) + $('.delete-all').click(function(){ + var data = []; + + var columnNo = $(this).parent().index(); + $(this).closest("table") + .find("tr td:nth-child(" + (columnNo+1) + ")") + .each(function(line, column) { + data.push(parseInt(column.id)); + }).remove(); + + data = JSON.stringify(data); + + // sends the updates to the server + $.ajax({ + type: 'DELETE', + // Provide correct Content-Type, so that Flask will know how to process it. + contentType: 'application/json', + data: data, + url: API_ROOT + "articles", + success: function (result) { + //console.log(result); + }, + error: function(XMLHttpRequest, textStatus, errorThrown){ + console.log(XMLHttpRequest.responseText); + } + }); + + }); + }(jQuery); diff --git a/pyaggr3g470r/templates/about.html b/pyaggr3g470r/templates/about.html index 84315271..08b80fbb 100644 --- a/pyaggr3g470r/templates/about.html +++ b/pyaggr3g470r/templates/about.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h1>{{ _('About') }}</h1> <p> {{ _('pyAggr3g470r is a news aggregator platform and can be shared between several users.') }} @@ -13,13 +13,13 @@ according to the <a href="https://www.gnu.org/licenses/agpl-3.0.html">Affero GPL</a> license.') }}</p> <p>{{ _('Found a bug? Report it <a href="https://bitbucket.org/cedricbonhomme/pyaggr3g470r/issues">here</a>.') }}</p> </div> - <div class="jumbotron"> + <div class="well"> <h1>{{ _('Help') }}</h1> <p>{{ _('If you have any problem, <a href="http://wiki.cedricbonhomme.org/contact">contact</a> the administrator.') }}</p> <p>{{ _('The documentation of the RESTful API is <a href="https://pyaggr3g470r.readthedocs.org/en/latest/web-services.html">here</a>.') }}</p> <p>{{ _('You can subscribe to new feeds with a bookmarklet. Drag <a href="%(bookmarklet)s">this link</a> to your browser bookmarks.', bookmarklet='javascript:window.location="https://pyaggr3g470r.herokuapp.com/bookmarklet?url="+encodeURIComponent(document.location)') }}</p> </div> - <div class="jumbotron"> + <div class="well"> <h1>{{ _('Donation') }}</h1> <p>{{ _('If you wish and if you like pyAggr3g470r, you can donate via bitcoin <a href="https://blockexplorer.com/address/1GVmhR9fbBeEh7rP1qNq76jWArDdDQ3otZ">1GVmhR9fbBeEh7rP1qNq76jWArDdDQ3otZ</a>. Thank you!') }}</p> </div> diff --git a/pyaggr3g470r/templates/admin/create_user.html b/pyaggr3g470r/templates/admin/create_user.html index 833ae601..1d6d6c11 100644 --- a/pyaggr3g470r/templates/admin/create_user.html +++ b/pyaggr3g470r/templates/admin/create_user.html @@ -4,7 +4,7 @@ {% endblock %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h2>{{ message | safe }}</h2> <form action="" method="post" name="saveprofileform" id="profileform"> {{ form.hidden_tag() }} diff --git a/pyaggr3g470r/templates/admin/user.html b/pyaggr3g470r/templates/admin/user.html index 046093e1..f20d53dd 100644 --- a/pyaggr3g470r/templates/admin/user.html +++ b/pyaggr3g470r/templates/admin/user.html @@ -4,7 +4,7 @@ {% endblock %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <a href="/admin/edit_user/{{ user.id }}" class="btn btn-default">{{ _('Edit this user') }}</a> <h2>{{ _('Membership') }}</h2> <div class="row"> @@ -14,7 +14,7 @@ </div> </div> </div> - <div class="jumbotron"> + <div class="well"> {% if user.feeds.all()|count == 0 %} <h1>{{ _('This user is not subscribed to any feed.') }}</h1> {% else %} diff --git a/pyaggr3g470r/templates/article.html b/pyaggr3g470r/templates/article.html index 101cf628..92014599 100644 --- a/pyaggr3g470r/templates/article.html +++ b/pyaggr3g470r/templates/article.html @@ -5,7 +5,7 @@ {% endblock %} {% block content %} <div class="container" data-article="{{ article.id }}"> - <div class="jumbotron"> + <div class="well"> <h2><a href="{{ article.link }}" target="_blank">{{ article.title|safe }}</a></h2> <h3>{{ _('from') }} <a href="/feed/{{ article.source.id }}">{{ article.source.title }}</a></h3> <a href="/delete/{{ article.id }}"><i class="glyphicon glyphicon-remove" title="{{ _('Delete this article') }}"></i></a> @@ -21,10 +21,10 @@ {% endif %} <h6>{{ article.date | datetime }}</h6> </div> - <div class="jumbotron"> + <div class="well"> {{ article.content | safe }} </div> - <div class="jumbotron"> + <div class="well"> <div class="row"> <div class="col-md-6"> {{ _('Next post:') }} <a href="/article/{{ next_article.id }}">{{ next_article.title }}</a> @@ -34,7 +34,7 @@ </div> </div> </div> - <div class="jumbotron"> + <div class="well"> <a href="https://api.pinboard.in/v1/posts/add?url={{ article.link }}&description={{ article.title }}" rel="noreferrer" target="_blank"> <img src="/static/img/pinboard.png" title="{{ _('Share on') }} Pinboard" /> </a> diff --git a/pyaggr3g470r/templates/articles.html b/pyaggr3g470r/templates/articles.html index eecb9579..383c28a4 100644 --- a/pyaggr3g470r/templates/articles.html +++ b/pyaggr3g470r/templates/articles.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h2><a href="{{ feed.site_link }}">{{ feed.title|safe }}</a></h2> <a href="/feed/{{ feed.id }}"><i class="glyphicon glyphicon-info-sign" title="Details"></i></a> <a href="/edit_feed/{{ feed.id }}"><i class="glyphicon glyphicon-edit" title="Edit this feed"></i></a> diff --git a/pyaggr3g470r/templates/duplicates.html b/pyaggr3g470r/templates/duplicates.html index a7eff2d0..4d7ac650 100644 --- a/pyaggr3g470r/templates/duplicates.html +++ b/pyaggr3g470r/templates/duplicates.html @@ -8,16 +8,20 @@ <thead> <tr> <th>#</th> - <th></th> - <th></th> + <th align="center"> + <span class="delete-all btn btn-default">{{ _('Delete all in this column') }}</span> + </th> + <th align="center"> + <span class="delete-all btn btn-default">{{ _('Delete all in this column') }}</span> + </th> </tr> </thead> <tbody> {% for pair in duplicates %} <tr> <td>{{ loop.index }}</td> - <td><a href="/delete/{{ pair[0].id }}"><i class="glyphicon glyphicon-remove" title="{{ _('Delete this article') }}"></i></a> <a href="/article/{{ pair[0].id }}">{{ pair[0].title }}</a></td> - <td><a href="/delete/{{ pair[1].id }}"><i class="glyphicon glyphicon-remove" title="{{ _('Delete this article') }}"></i></a> <a href="/article/{{ pair[1].id }}">{{ pair[1].title }}</a></td> + <td id="{{ pair[0].id }}"><a href="/delete/{{ pair[0].id }}"><i class="glyphicon glyphicon-remove" title="{{ _('Delete this article') }}"></i></a> <a href="/article/{{ pair[0].id }}">{{ pair[0].title }}</a> ({{ pair[0].retrieved_date }})</td> + <td id="{{ pair[1].id }}"><a href="/delete/{{ pair[1].id }}"><i class="glyphicon glyphicon-remove" title="{{ _('Delete this article') }}"></i></a> <a href="/article/{{ pair[1].id }}">{{ pair[1].title }}</a> ({{ pair[1].retrieved_date }})</td> </tr> {% endfor %} </tobdy> diff --git a/pyaggr3g470r/templates/edit_feed.html b/pyaggr3g470r/templates/edit_feed.html index a6b28ded..e9a90960 100644 --- a/pyaggr3g470r/templates/edit_feed.html +++ b/pyaggr3g470r/templates/edit_feed.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h1>{{ action }}</h1> <form action="" method="post" name="save"> {{ form.hidden_tag() }} diff --git a/pyaggr3g470r/templates/errors/404.html b/pyaggr3g470r/templates/errors/404.html index 49c9ef5a..c64a2be8 100644 --- a/pyaggr3g470r/templates/errors/404.html +++ b/pyaggr3g470r/templates/errors/404.html @@ -4,7 +4,7 @@ {% endblock %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h1>Page Not Found</h1> <p>What you were looking for is just not there, go to the <a href="{{ url_for('home') }}">home page</a>.</p> </div> diff --git a/pyaggr3g470r/templates/errors/500.html b/pyaggr3g470r/templates/errors/500.html index 1fa6acc9..417fc0c7 100644 --- a/pyaggr3g470r/templates/errors/500.html +++ b/pyaggr3g470r/templates/errors/500.html @@ -4,7 +4,7 @@ {% endblock %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h1>Internal Server Error</h1> <p>Something bad just happened! Go to the <a href="{{ url_for('home') }}">home page</a>.</p> </div> diff --git a/pyaggr3g470r/templates/feed.html b/pyaggr3g470r/templates/feed.html index 4b050573..268cbf7d 100644 --- a/pyaggr3g470r/templates/feed.html +++ b/pyaggr3g470r/templates/feed.html @@ -1,19 +1,22 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h2>{{ feed.title }}</h2> {% if feed.description %} <p>{{ feed.description }}</p> {% endif %} <a href="/delete_feed/{{ feed.id }}"><i class="glyphicon glyphicon-remove" title="{{ _('Delete this feed') }}" onclick="return confirm('{{ _('You are going to delete this feed.') }}');"></i></a> <a href="/edit_feed/{{ feed.id }}"><i class="glyphicon glyphicon-edit" title="{{ _('Edit this feed') }}"></i></a> </div> - <div class="jumbotron"> + <div class="well"> <p> - {{ _('This feed contains') }} {{ feed.articles.all()|count }} <a href="/articles/{{ feed.id }}/100">{{ _('articles') }}</a> - {% if nb_articles != 0 %} - ({{ ((feed.articles.all()|count * 100 ) / nb_articles) | round(2, 'floor') }}% {{ _('of the database') }}).<br /> + {{ _('This feed contains') }} {{ feed.articles.all()|count }} <a href="/articles/{{ feed.id }}/100">{{ _('articles') }}</a>.<br /> + {{ _('Address of the feed') }}: <a href="{{ feed.link }}" target="_blank">{{ feed.link }}</a><br /> + {% if feed.site_link != "" %} + {{ _('Address of the site') }}: <a href="{{ feed.site_link }}" target="_blank">{{ feed.site_link }}</a><br /> {% endif %} + <br /> + {% if feed.last_retrieved %} {{ _("Last download:") }} {{ feed.last_retrieved | datetime }}<br /> {% endif %} @@ -28,18 +31,13 @@ {{ _("Here's the last error encountered while retrieving this feed:") }} <pre>{{ feed.last_error }}</pre><br /> {% endif %} - {{ _('Address of the feed') }}: <a href="{{ feed.link }}">{{ feed.link }}</a><br /> - {% if feed.site_link != "" %} - {{ _('Address of the site') }}: <a href="{{ feed.site_link }}">{{ feed.site_link }}</a><br /> - {% endif %} - {% if feed.articles.all()|count != 0 %} {{ _('The last article was posted') }} {{ elapsed.days }} {{ _('day(s) ago.') }}<br /> {{ _('Daily average') }}: {{ average }}, {{ _('between the') }} {{ first_post_date | datetime }} {{ _('and the') }} {{ end_post_date | datetime }}. {% endif %} </p> </div> - <div class="jumbotron"> + <div class="well"> {% if feed.articles.all()|count != 0 %} <div>{{ tag_cloud|safe }}</div> {% endif %} diff --git a/pyaggr3g470r/templates/history.html b/pyaggr3g470r/templates/history.html index 0194cb89..6be54d71 100644 --- a/pyaggr3g470r/templates/history.html +++ b/pyaggr3g470r/templates/history.html @@ -1,74 +1,26 @@ - {% extends "layout.html" %} - {% block head %} -{{ super() }} -<script type="text/javascript" src="http://d3js.org/d3.v3.min.js"></script> -<style type="text/css"> -rect { - fill: purple ; - padding: 7px; - margin: 2px; - color: white; - } -rect:hover - { - opacity : 0.5; - } -</style> -{% endblock %} - +{% extends "layout.html" %} {% block content %} <div class="container"> -<h1>History</h1> -<div align="center" id="pie_chart"> - <div class="chart"> - </div> - - -<script type="text/javascript"> -function createGraph(dataset, w, h) { - var barPadding = 1; - - //Create SVG element - var svg = d3.select("#pie_chart .chart") - .append("svg") - .attr("width", w) - .attr("height", h); - - svg.selectAll("rect") - .data(dataset) - .enter() - .append("rect") - .on("click", clickEvent) - .attr("x", function(d, i) { - return i * (w / dataset.length); - }) - .attr("y", function(d) { - return h - (d * 4); - }) - .attr("width", w / dataset.length - barPadding) - .attr("height", function(d) { - return d * 4; - }); -} - -function clickEvent(d){ - //var op = prompt("Please enter the value", ""); - alert(d); - -}; - - - -var w = 800; -var h = 100; -var dataset = [ 12, 10, 13, 19, 21, 25, 22, 18, 15, 13, - 11, 12, 15, 20, 18, 17, 16, 18, 23, 25 ]; - -createGraph(dataset, w, h); - - -</script> - - -</div> -{% endblock %}
\ No newline at end of file + <h1>{{ _('History') }}</h1> + {% if month != None %} + <h2><a href="/history/{{ year }}"><span class="glyphicon glyphicon-chevron-left"></span> {{ year }}</a></h2> + <h3>{{ month | month_name }}</h3> + {% elif year != None %} + <h2><a href="/history"><span class="glyphicon glyphicon-chevron-left"></span> {{ _('all years') }}</a></h2> + <h3>{{ year }}</h3> + {% endif %} + <ul class="list-group"> + {% for article in articles_counter %} + {% if year == None %} + <li class="list-group-item"><a href="/history/{{ article }}">{{ article }}</a> : {{ articles_counter[article] }} articles</li> + {% elif month == None %} + <li class="list-group-item"><a href="/history/{{ year }}/{{ article }}">{{ article | month_name }}</a> : {{ articles_counter[article] }} articles</li> + {% else %} + {% for article in articles | sort(attribute="date", reverse = True) %} + <li class="list-group-item">{{ article.date | datetime }} - <a href="/article/{{ article.id }}">{{ article.title | safe }}</a></li> + {% endfor %} + {% endif %} + {% endfor %} + </ul> +</div><!-- /.container --> +{% endblock %} diff --git a/pyaggr3g470r/templates/home.html b/pyaggr3g470r/templates/home.html index d2a961ab..69ca582b 100644 --- a/pyaggr3g470r/templates/home.html +++ b/pyaggr3g470r/templates/home.html @@ -22,7 +22,7 @@ <li class="feed-menu"><a href="{{ gen_url(feed=fid) }}"> {% if feed_id == fid %}<b>{% endif %} {% if in_error.get(fid, 0) > 0 %} - <span style="background-color: {{ "red" if in_error[fid] > 5 else "orange" }} ;" class="badge pull-right" title="Some errors occured while trying to retrieve that feed.">{{ in_error[fid] }}</span> + <span style="background-color: {{ "red" if in_error[fid] > 5 else "orange" }} ;" class="badge pull-right" title="Some errors occured while trying to retrieve that feed.">{{ in_error[fid] }} {{ _("error") }}{% if in_error[fid] > 1 %}s{% endif %}</span> {% endif %} <span id="unread-{{ fid }}" class="badge pull-right">{{ nbunread }}</span> {{ feeds[fid]|safe }} @@ -40,7 +40,7 @@ {% for fid, ftitle in feeds|dictsort(case_sensitive=False, by='value') if not fid in unread %} <li class="feed-menu"><a href="{{ gen_url(feed=fid) }}"> {% if in_error.get(fid, 0) > 0 %} - <span style="background-color: {{ "red" if in_error[fid] > 5 else "orange" }} ;" class="badge pull-right" title="Some errors occured while trying to retrieve that feed.">{{ in_error[fid] }}</span> + <span style="background-color: {{ "red" if in_error[fid] > 5 else "orange" }} ;" class="badge pull-right" title="Some errors occured while trying to retrieve that feed.">{{ in_error[fid] }} {{ _("error") }}{% if in_error[fid] > 1 %}s{% endif %}</span> {% endif %} {% if feed_id == fid %}<b>{% endif %} {{ ftitle|safe }} diff --git a/pyaggr3g470r/templates/inactives.html b/pyaggr3g470r/templates/inactives.html index e6897281..6a4ff055 100644 --- a/pyaggr3g470r/templates/inactives.html +++ b/pyaggr3g470r/templates/inactives.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <form method=get action="/inactives"> <p>{{ _('Days of inactivity') }}:</p> <input type="number" name="nb_days" class="form-control" value="{{ nb_days }}" min="0" max="1000000" step="1" size="4" style="text-align: center" /> diff --git a/pyaggr3g470r/templates/layout.html b/pyaggr3g470r/templates/layout.html index 6b929bf3..60efa69e 100644 --- a/pyaggr3g470r/templates/layout.html +++ b/pyaggr3g470r/templates/layout.html @@ -11,71 +11,12 @@ <!-- Bootstrap core CSS --> <link href="{{ url_for('static', filename = 'css/bootstrap.css') }}" rel="stylesheet" media="screen" /> <!-- Add custom CSS here --> - <style> - body { - margin-top: 60px; - } - div.top { - position: relative; - top:-60px; - display: block; - height: 0; - } - ul.affix { - position: fixed; - top: 0px; - } - ul.affix-top { - position: static; - } - ul.affix-bottom { - position: absolute; - } - /* First level of nav */ - .sidenav { - margin-top: 10px; - margin-bottom: 0px; - padding-top: 10px; - padding-bottom: 0px; - overflow-y: auto; - height: 90%; - z-index: 1000; - background-color: #ffffff; - border-radius: 2px; - font-size: 100%; - } - /* All levels of nav */ - .sidebar .nav > li > a { - display: block; - color: #716b7a; - padding: 5px 20px; - } - .sidebar .nav > li > a:hover, - .sidebar .nav > li > a:focus { - text-decoration: none; - background-color: #e5e3e9; - } - .sidebar .nav > .active > a, - .sidebar .nav > .active:hover > a, - .sidebar .nav > .active:focus > a { - font-weight: bold; - color: #563d7c; - background-color: transparent; - } - /* Nav: second level */ - .sidebar .nav .nav { - margin-bottom: 8px; - } - .sidebar .nav .nav > li > a { - padding-top: 3px; - padding-bottom: 3px; - font-size: 80%; - } - </style> + <link href="{{ url_for('static', filename = 'css/customized-bootstrap.css') }}" rel="stylesheet" media="screen" /> + <link href="{{ url_for('static', filename = 'css/side-nav.css') }}" rel="stylesheet" media="screen" /> {% endblock %} </head> - <body data-spy="scroll" data-target="#affix-nav"> - <nav class="navbar navbar-inverse navbar-fixed-top" role="navigation"> + <body> + <nav class="navbar navbar-inverse navbar-fixed-top navbar-custom" role="navigation"> <div class="container-fluid"> <div class="navbar-header"> <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-ex1-collapse"> @@ -118,7 +59,7 @@ <li><a href="{{ url_for('logout') }}"><span class="glyphicon glyphicon-log-out"></span> {{ _('Logout') }}</a></li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown"> - <button class="btn btn-default btn-xs" type="submit"><span class="glyphicon glyphicon-search"></span></button> <b class="caret"></b> + <div><span class="glyphicon glyphicon-search"></span> <b class="caret"></b></div> </a> <ul class="dropdown-menu"> <li> @@ -138,6 +79,8 @@ </div><!-- /.container --> </nav> + <br /> + <div class="container"> {% with messages = get_flashed_messages(with_categories=true) %} {% if messages %} diff --git a/pyaggr3g470r/templates/login.html b/pyaggr3g470r/templates/login.html index ae0797a5..c37d6937 100644 --- a/pyaggr3g470r/templates/login.html +++ b/pyaggr3g470r/templates/login.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h2>{{ _('Log In') }}</h2> <form action="{{ url_for('login') }}" method=post> {{ form.hidden_tag() }} @@ -27,4 +27,4 @@ <a href="/recover" class="btn btn-default">{{ _('Forgot password') }}</a> </div><!-- /.container --> -{% endblock %}
\ No newline at end of file +{% endblock %} diff --git a/pyaggr3g470r/templates/management.html b/pyaggr3g470r/templates/management.html index 4a2de617..722300af 100644 --- a/pyaggr3g470r/templates/management.html +++ b/pyaggr3g470r/templates/management.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h1>{{ _('Your subscriptions') }}</h1> <p>{{ _('You are subscribed to') }} {{ nb_feeds }} <a href="/feeds">{{ _('feeds') }}</a>. {{ _('Add a') }} <a href="/create_feed">{{ _('feed') }}</a>.</p> <p>{{ nb_articles }} {{ _('articles are stored in the database with') }} {{ nb_unread_articles }} <a href="/unread">{{ _('unread articles') }}</a>.</p> @@ -10,7 +10,7 @@ {% endif %} <a href="/expire_articles?weeks=10" class="btn btn-default" onclick="return confirm('{{ _('You are going to delete old articles.') }}');">{{ _('Delete articles older than 10 weeks') }}</a> </div> - <div class="jumbotron"> + <div class="well"> <h1>{{ _('Your Profile') }}</h1> <div class="row"> <div class="col-md-6"> @@ -25,7 +25,7 @@ </div> </div> </div> - <div class="jumbotron"> + <div class="well"> <h1 id="import">{{ _('OPML import/export') }}</h1> <form action="" method="post" id="formImportOPML" enctype="multipart/form-data"> <span class="btn btn-default btn-file">{{ _('Batch import feeds from OPML') }} (<span class="text-info">*.xml {{ _('or') }} *.opml</span>)<input type="file" name="opmlfile" /></span> @@ -41,7 +41,7 @@ <br /> <a href="/export?format=JSON" class="btn btn-default">{{ _('Export account to JSON') }}</a> </div> - <div class="jumbotron"> + <div class="well"> <h1>{{ _('Export articles') }}</h1> <a href="/export?format=HTML" class="btn btn-default">HTML</a> </div> diff --git a/pyaggr3g470r/templates/profile.html b/pyaggr3g470r/templates/profile.html index f879b054..971c4e64 100644 --- a/pyaggr3g470r/templates/profile.html +++ b/pyaggr3g470r/templates/profile.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h2>Edit your profile</h2> <form action="" method="post" name="save"> {{ form.hidden_tag() }} diff --git a/pyaggr3g470r/templates/recover.html b/pyaggr3g470r/templates/recover.html index 1098ffef..c1176d55 100644 --- a/pyaggr3g470r/templates/recover.html +++ b/pyaggr3g470r/templates/recover.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="jumbotron"> + <div class="well"> <h2>{{ _('Recover your account') }}</h2> {% for message in form.email.errors %} <div class="flash">{{ message }}</div> @@ -15,4 +15,4 @@ </form> </div> </div><!-- /.container --> -{% endblock %}
\ No newline at end of file +{% endblock %} diff --git a/pyaggr3g470r/templates/signup.html b/pyaggr3g470r/templates/signup.html index dd8154e2..3962c42a 100644 --- a/pyaggr3g470r/templates/signup.html +++ b/pyaggr3g470r/templates/signup.html @@ -1,7 +1,7 @@ {% extends "layout.html" %} {% block content %} <div class="container"> - <div class="form jumbotron"> + <div class="form well"> <form action="" method="post" name="save"> {{ form.hidden_tag() }} <div class="form-group"> diff --git a/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.mo b/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.mo Binary files differindex e3b7f0ff..6ecf87bd 100644 --- a/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.mo +++ b/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.mo diff --git a/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.po b/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.po index 3bb25cb9..8a0a5ca8 100644 --- a/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.po +++ b/pyaggr3g470r/translations/fr/LC_MESSAGES/messages.po @@ -7,8 +7,8 @@ msgid "" msgstr "" "Project-Id-Version: PROJECT VERSION\n" "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n" -"POT-Creation-Date: 2015-01-09 22:12+0100\n" -"PO-Revision-Date: 2015-01-09 22:13+0100\n" +"POT-Creation-Date: 2015-03-28 11:26+0100\n" +"PO-Revision-Date: 2015-03-28 11:27+0100\n" "Last-Translator: Cédric Bonhomme <cedric@cedricbonhomme.org>\n" "Language-Team: fr <LL@li.org>\n" "Language: fr\n" @@ -19,37 +19,37 @@ msgstr "" "Generated-By: Babel 1.3\n" "X-Generator: Poedit 1.5.4\n" -#: pyaggr3g470r/forms.py:39 pyaggr3g470r/forms.py:99 +#: pyaggr3g470r/forms.py:40 pyaggr3g470r/forms.py:97 #: pyaggr3g470r/templates/admin/dashboard.html:12 msgid "Nickname" msgstr "Pseudonyme" -#: pyaggr3g470r/forms.py:39 pyaggr3g470r/forms.py:99 +#: pyaggr3g470r/forms.py:41 pyaggr3g470r/forms.py:98 msgid "Please enter your nickname." msgstr "S'il vous plaît, entrez votre pseudonyme." -#: pyaggr3g470r/forms.py:40 pyaggr3g470r/forms.py:100 -#: pyaggr3g470r/forms.py:121 pyaggr3g470r/templates/admin/dashboard.html:13 +#: pyaggr3g470r/forms.py:42 pyaggr3g470r/forms.py:99 pyaggr3g470r/forms.py:133 +#: pyaggr3g470r/templates/admin/dashboard.html:13 msgid "Email" msgstr "Email" -#: pyaggr3g470r/forms.py:40 pyaggr3g470r/forms.py:60 pyaggr3g470r/forms.py:121 +#: pyaggr3g470r/forms.py:45 pyaggr3g470r/forms.py:65 pyaggr3g470r/forms.py:136 msgid "Please enter your email address." msgstr "S'il vous plaît, entrez votre adresse email." -#: pyaggr3g470r/forms.py:41 pyaggr3g470r/forms.py:61 pyaggr3g470r/forms.py:101 +#: pyaggr3g470r/forms.py:46 pyaggr3g470r/forms.py:66 pyaggr3g470r/forms.py:102 msgid "Password" msgstr "Mot de passe" -#: pyaggr3g470r/forms.py:41 pyaggr3g470r/forms.py:61 +#: pyaggr3g470r/forms.py:47 pyaggr3g470r/forms.py:67 msgid "Please enter a password." msgstr "S'il vous plaît entrer un mot de passe." -#: pyaggr3g470r/forms.py:43 pyaggr3g470r/templates/login.html:26 +#: pyaggr3g470r/forms.py:50 pyaggr3g470r/templates/login.html:26 msgid "Sign up" msgstr "S'inscrire" -#: pyaggr3g470r/forms.py:52 pyaggr3g470r/forms.py:111 +#: pyaggr3g470r/forms.py:55 pyaggr3g470r/forms.py:117 msgid "" "This nickname has invalid characters. Please use letters, numbers, dots and " "underscores only." @@ -57,272 +57,88 @@ msgstr "" "Ce pseudonyme a des caractères non valides. Utilisez seulement des lettres, " "des chiffres, des points et '_'." -#: pyaggr3g470r/forms.py:62 pyaggr3g470r/templates/login.html:5 +#: pyaggr3g470r/forms.py:69 pyaggr3g470r/templates/login.html:5 msgid "Log In" msgstr "Connexion" -#: pyaggr3g470r/forms.py:75 +#: pyaggr3g470r/forms.py:80 msgid "Account not confirmed" msgstr "Compte non confirmé" -#: pyaggr3g470r/forms.py:78 +#: pyaggr3g470r/forms.py:83 msgid "Invalid email or password" msgstr "E-mail ou mot de passe invalide" -#: pyaggr3g470r/forms.py:83 pyaggr3g470r/templates/feeds.html:11 +#: pyaggr3g470r/forms.py:89 pyaggr3g470r/templates/feeds.html:11 msgid "Title" msgstr "Titre" -#: pyaggr3g470r/forms.py:84 pyaggr3g470r/templates/admin/user.html:27 +#: pyaggr3g470r/forms.py:90 pyaggr3g470r/templates/admin/user.html:27 msgid "Feed link" msgstr "Lien du flux" -#: pyaggr3g470r/forms.py:85 pyaggr3g470r/templates/admin/user.html:28 +#: pyaggr3g470r/forms.py:91 pyaggr3g470r/templates/admin/user.html:28 msgid "Site link" msgstr "Lien du site" -#: pyaggr3g470r/forms.py:86 -msgid "Email notification" -msgstr "Notification par email" - -#: pyaggr3g470r/forms.py:87 +#: pyaggr3g470r/forms.py:92 msgid "Check for updates" msgstr "Vérifier les mises à jour" -#: pyaggr3g470r/forms.py:88 pyaggr3g470r/forms.py:102 +#: pyaggr3g470r/forms.py:93 pyaggr3g470r/forms.py:107 msgid "Save" msgstr "Sauver" -#: pyaggr3g470r/forms.py:100 +#: pyaggr3g470r/forms.py:101 msgid "Please enter your email." msgstr "S'il vous plaît, entrez votre email." -#: pyaggr3g470r/forms.py:116 +#: pyaggr3g470r/forms.py:103 +msgid "Password Confirmation" +msgstr "Confirmation du mot de passe" + +#: pyaggr3g470r/forms.py:104 +msgid "Feeds refresh frequency (in minutes)" +msgstr "Fréquence de rafraîchissement du flux (en minutes)" + +#: pyaggr3g470r/forms.py:112 +msgid "Passwords aren't the same." +msgstr "Les mots de passe ne sont pas identiques." + +#: pyaggr3g470r/forms.py:125 msgid "Subject" msgstr "Objet" -#: pyaggr3g470r/forms.py:116 +#: pyaggr3g470r/forms.py:126 msgid "Please enter a subject." msgstr "S'il vous plaît entrer un objet." -#: pyaggr3g470r/forms.py:117 +#: pyaggr3g470r/forms.py:127 msgid "Message" msgstr "Message" -#: pyaggr3g470r/forms.py:117 +#: pyaggr3g470r/forms.py:128 msgid "Please enter a content." msgstr "S'il vous plaît entrer un contenu." -#: pyaggr3g470r/forms.py:118 +#: pyaggr3g470r/forms.py:129 msgid "Send" msgstr "Envoyer" -#: pyaggr3g470r/forms.py:122 +#: pyaggr3g470r/forms.py:137 msgid "Recover" msgstr "Récupérer" -#: pyaggr3g470r/forms.py:135 +#: pyaggr3g470r/forms.py:147 msgid "Account not confirmed." msgstr "Compte non confirmé." -#: pyaggr3g470r/forms.py:138 +#: pyaggr3g470r/forms.py:150 msgid "Invalid email." msgstr "Email invalide." -#: pyaggr3g470r/views.py:99 -msgid "Authentication required." -msgstr "Authentification requise." - -#: pyaggr3g470r/views.py:104 -msgid "Forbidden." -msgstr "Interdit." - -#: pyaggr3g470r/views.py:156 -msgid "Logged in successfully." -msgstr "Connecté avec succès." - -#: pyaggr3g470r/views.py:178 -msgid "Logged out successfully." -msgstr "Déconnecté avec succès." - -#: pyaggr3g470r/views.py:187 -msgid "Self-registration is disabled." -msgstr "L'auto-enregistrement est désactivé." - -#: pyaggr3g470r/views.py:204 -msgid "Email already used." -msgstr "Email déjà utilisé." - -#: pyaggr3g470r/views.py:211 pyaggr3g470r/views.py:921 -msgid "Problem while sending activation email" -msgstr "Problème lors de l'envoi d'email d'activation" - -#: pyaggr3g470r/views.py:214 -msgid "Your account has been created. Check your mail to confirm it." -msgstr "Votre compte a été créé. Vérifiez votre courrier pour le confirmer." - -#: pyaggr3g470r/views.py:271 pyaggr3g470r/views.py:601 -msgid "Downloading articles..." -msgstr "Téléchargement des articles." - -#: pyaggr3g470r/views.py:343 pyaggr3g470r/views.py:403 -msgid "This article do not exist." -msgstr "Cet article n'existe pas." - -#: pyaggr3g470r/views.py:400 pyaggr3g470r/templates/home.html:75 -msgid "Article" -msgstr "Article" - -#: pyaggr3g470r/views.py:400 -msgid "deleted." -msgstr "supprimé." - -#: pyaggr3g470r/views.py:499 -msgid "Indexing database..." -msgstr "Indexation la base de données..." - -#: pyaggr3g470r/views.py:501 pyaggr3g470r/views.py:570 -msgid "An error occured" -msgstr "Une erreur est survenue." - -#: pyaggr3g470r/views.py:504 -msgid "Option not available on Heroku." -msgstr "Option non disponible sur Heroku." - -#: pyaggr3g470r/views.py:519 pyaggr3g470r/views.py:529 -msgid "Error when exporting articles." -msgstr "Erreur lors de l'export des articles." - -#: pyaggr3g470r/views.py:535 -msgid "Export format not supported." -msgstr "Ce format d'export n'est pas supporté." - -#: pyaggr3g470r/views.py:558 -msgid "Full text search is not yet implemented for Heroku." -msgstr "La recherche rapide n'est pas supporté sur Heroku." - -#: pyaggr3g470r/views.py:595 pyaggr3g470r/views.py:608 -#: pyaggr3g470r/views.py:616 -msgid "File not allowed." -msgstr "Fichier non autorisé." - -#: pyaggr3g470r/views.py:600 -msgid "feeds imported." -msgstr "flux importés." - -#: pyaggr3g470r/views.py:603 -msgid "Impossible to import the new feeds." -msgstr "Impossible d'importer les nouveaux flux." - -#: pyaggr3g470r/views.py:612 -msgid "Account imported." -msgstr "Compte importé." - -#: pyaggr3g470r/views.py:614 -msgid "Impossible to import the account." -msgstr "Impossible d'importer le compte." - -#: pyaggr3g470r/views.py:652 -msgid "Feed successfully updated." -msgstr "Flux mis à jour avec succès." - -#: pyaggr3g470r/views.py:664 -msgid "Feed successfully created." -msgstr "Flux créé avec succès." - -#: pyaggr3g470r/views.py:667 -msgid "Downloading articles for the new feed..." -msgstr "Téléchargement des articles du nouveau flux..." - -#: pyaggr3g470r/views.py:671 -msgid "Feed already in the database." -msgstr "Flux déjà dans la base de données." - -#: pyaggr3g470r/views.py:677 -msgid "Edit the feed" -msgstr "Éditez ce flux" - -#: pyaggr3g470r/views.py:690 pyaggr3g470r/templates/layout.html:109 -msgid "Add a feed" -msgstr "Ajouter un flux" - -#: pyaggr3g470r/views.py:703 pyaggr3g470r/templates/home.html:74 -#: pyaggr3g470r/templates/admin/user.html:42 -msgid "Feed" -msgstr "Flux" - -#: pyaggr3g470r/views.py:703 pyaggr3g470r/views.py:898 -msgid "successfully deleted." -msgstr "supprimé avec succès." - -#: pyaggr3g470r/views.py:721 pyaggr3g470r/views.py:848 -#: pyaggr3g470r/views.py:858 pyaggr3g470r/views.py:898 -msgid "User" -msgstr "Utilisateur" - -#: pyaggr3g470r/views.py:721 pyaggr3g470r/views.py:848 -msgid "successfully updated." -msgstr "mis à jour avec succès." - -#: pyaggr3g470r/views.py:740 -msgid "Your account has been deleted." -msgstr "Votre compte a été supprimé." - -#: pyaggr3g470r/views.py:742 pyaggr3g470r/views.py:884 -#: pyaggr3g470r/views.py:900 pyaggr3g470r/views.py:929 -msgid "This user does not exist." -msgstr "Cet utilisateur n'existe pas." - -#: pyaggr3g470r/views.py:756 -msgid "Articles deleted." -msgstr "Articles supprimés." - -#: pyaggr3g470r/views.py:770 -msgid "Your account has been confirmed." -msgstr "Votre compte a été confirmé." - -#: pyaggr3g470r/views.py:772 -msgid "Impossible to confirm this account." -msgstr "Impossible de confirmer ce compte." - -#: pyaggr3g470r/views.py:796 -msgid "New password sent to your address." -msgstr "Nouveau mot de passe envoyé à votre adresse." - -#: pyaggr3g470r/views.py:798 -msgid "Problem while sending your new password." -msgstr "Problème lors de l'envoi de votre nouveau mot de passe." - -#: pyaggr3g470r/views.py:823 -msgid "Problem while sending email" -msgstr "Problème lors de l'envoi de l'email" - -#: pyaggr3g470r/views.py:858 -msgid "successfully created." -msgstr "créé avec succès." - -#: pyaggr3g470r/views.py:867 -msgid "Edit the user" -msgstr "Éditer cet utilisateur" - -#: pyaggr3g470r/views.py:870 pyaggr3g470r/templates/admin/dashboard.html:45 -msgid "Add a new user" -msgstr "Ajouter un nouvel utilisateur" - -#: pyaggr3g470r/views.py:919 pyaggr3g470r/views.py:926 -msgid "Account of the user" -msgstr "Compte de l'utilisateur" - -#: pyaggr3g470r/views.py:919 -msgid "successfully activated." -msgstr "activé avec succès." - -#: pyaggr3g470r/views.py:926 -msgid "successfully disabled." -msgstr "désactivé avec succès." - -#: pyaggr3g470r/templates/about.html:5 pyaggr3g470r/templates/layout.html:126 -#: pyaggr3g470r/templates/layout.html:143 +#: pyaggr3g470r/templates/about.html:5 pyaggr3g470r/templates/layout.html:117 +#: pyaggr3g470r/templates/layout.html:134 msgid "About" msgstr "À propos" @@ -379,11 +195,11 @@ msgstr "" #: pyaggr3g470r/templates/about.html:19 msgid "" -"The documentation of the RESTful API is <a href=\"https://bitbucket.org/" -"cedricbonhomme/pyaggr3g470r#rst-header-web-service\">here</a>." +"The documentation of the RESTful API is <a href=\"https://pyaggr3g470r." +"readthedocs.org/en/latest/web-services.html\">here</a>." msgstr "" -"La documentation de l'API RESTful est <a href=\"https://bitbucket.org/" -"cedricbonhomme/pyaggr3g470r#rst-header-web-service\">ici</a>." +"La documentation de l'API RESTful est <a href=\"https://pyaggr3g470r." +"readthedocs.org/en/latest/web-services.html\">ici</a>." #: pyaggr3g470r/templates/about.html:20 #, python-format @@ -414,25 +230,25 @@ msgid "from" msgstr "de" #: pyaggr3g470r/templates/article.html:11 -#: pyaggr3g470r/templates/duplicates.html:19 -#: pyaggr3g470r/templates/duplicates.html:20 -#: pyaggr3g470r/templates/home.html:89 +#: pyaggr3g470r/templates/duplicates.html:23 +#: pyaggr3g470r/templates/duplicates.html:24 +#: pyaggr3g470r/templates/home.html:90 msgid "Delete this article" msgstr "Supprimer cet article" -#: pyaggr3g470r/templates/article.html:13 pyaggr3g470r/templates/home.html:91 +#: pyaggr3g470r/templates/article.html:13 pyaggr3g470r/templates/home.html:92 msgid "One of your favorites" msgstr "Un de vos favoris" -#: pyaggr3g470r/templates/article.html:15 pyaggr3g470r/templates/home.html:93 +#: pyaggr3g470r/templates/article.html:15 pyaggr3g470r/templates/home.html:94 msgid "Click if you like this article" msgstr "Cliquez si vous aimez cet article" -#: pyaggr3g470r/templates/article.html:18 pyaggr3g470r/templates/home.html:96 +#: pyaggr3g470r/templates/article.html:18 pyaggr3g470r/templates/home.html:97 msgid "Mark this article as unread" msgstr "Marquer cet article comme non lu" -#: pyaggr3g470r/templates/article.html:20 pyaggr3g470r/templates/home.html:98 +#: pyaggr3g470r/templates/article.html:20 pyaggr3g470r/templates/home.html:99 msgid "Mark this article as read" msgstr "Marquer cet article comme lu" @@ -453,7 +269,12 @@ msgstr "Partager sur" msgid "Duplicates in the feed" msgstr "Doublons dans le flux" -#: pyaggr3g470r/templates/duplicates.html:27 +#: pyaggr3g470r/templates/duplicates.html:12 +#: pyaggr3g470r/templates/duplicates.html:15 +msgid "Delete all in this column" +msgstr "Supprimer tout dans cette colonne" + +#: pyaggr3g470r/templates/duplicates.html:31 msgid "No duplicates in the feed" msgstr "Pas de doublon dans ce flux" @@ -471,26 +292,26 @@ msgid "More articles" msgstr "Plus d'articles" #: pyaggr3g470r/templates/favorites.html:17 -#: pyaggr3g470r/templates/home.html:29 pyaggr3g470r/templates/home.html:44 +#: pyaggr3g470r/templates/home.html:32 pyaggr3g470r/templates/home.html:50 #: pyaggr3g470r/templates/unread.html:17 msgid "Details" msgstr "Détails" #: pyaggr3g470r/templates/favorites.html:18 pyaggr3g470r/templates/feed.html:8 -#: pyaggr3g470r/templates/feeds.html:33 pyaggr3g470r/templates/home.html:31 -#: pyaggr3g470r/templates/home.html:46 pyaggr3g470r/templates/unread.html:18 +#: pyaggr3g470r/templates/feeds.html:33 pyaggr3g470r/templates/home.html:34 +#: pyaggr3g470r/templates/home.html:52 pyaggr3g470r/templates/unread.html:18 #: pyaggr3g470r/templates/admin/user.html:43 msgid "Edit this feed" msgstr "Éditer ce flux" #: pyaggr3g470r/templates/feed.html:7 pyaggr3g470r/templates/feeds.html:35 -#: pyaggr3g470r/templates/home.html:32 pyaggr3g470r/templates/home.html:47 +#: pyaggr3g470r/templates/home.html:35 pyaggr3g470r/templates/home.html:53 #: pyaggr3g470r/templates/admin/user.html:44 msgid "Delete this feed" msgstr "Supprimer ce flux" #: pyaggr3g470r/templates/feed.html:7 pyaggr3g470r/templates/feeds.html:35 -#: pyaggr3g470r/templates/home.html:32 pyaggr3g470r/templates/home.html:47 +#: pyaggr3g470r/templates/home.html:35 pyaggr3g470r/templates/home.html:53 #: pyaggr3g470r/templates/admin/user.html:44 msgid "You are going to delete this feed." msgstr "Vous allez supprimer ce flux." @@ -503,35 +324,55 @@ msgstr "Ce flux contient" msgid "articles" msgstr "articles" -#: pyaggr3g470r/templates/feed.html:14 -msgid "of the database" -msgstr "de la base de données" - -#: pyaggr3g470r/templates/feed.html:17 +#: pyaggr3g470r/templates/feed.html:13 msgid "Address of the feed" msgstr "Adresse du flux" -#: pyaggr3g470r/templates/feed.html:19 +#: pyaggr3g470r/templates/feed.html:15 msgid "Address of the site" msgstr "Adresse du site" -#: pyaggr3g470r/templates/feed.html:23 +#: pyaggr3g470r/templates/feed.html:21 +msgid "Last download:" +msgstr "Dernier téléchargement:" + +#: pyaggr3g470r/templates/feed.html:25 +msgid "" +"That feed has encountered too much consecutive errors and won't be retrieved " +"anymore." +msgstr "" +"Ce flux a rencontré trop d'erreurs consécutives et ne sera plus récupéré." + +#: pyaggr3g470r/templates/feed.html:27 +msgid "" +"The download of this feed has encountered some problems. However its error " +"counter will be reinitialized at the next successful retrieving." +msgstr "" +"Le téléchargement de ce flux a rencontré quelques problèmes. Cependant, son " +"compteur d'erreurs sera réinitialisé lors de la prochaine récupération " +"réussie." + +#: pyaggr3g470r/templates/feed.html:31 +msgid "Here's the last error encountered while retrieving this feed:" +msgstr "Voici la dernière erreur survenue lors de la récupération ce flux:" + +#: pyaggr3g470r/templates/feed.html:35 msgid "The last article was posted" msgstr "Le dernier article a été posté il y a" -#: pyaggr3g470r/templates/feed.html:23 +#: pyaggr3g470r/templates/feed.html:35 msgid "day(s) ago." msgstr "jours." -#: pyaggr3g470r/templates/feed.html:24 +#: pyaggr3g470r/templates/feed.html:36 msgid "Daily average" msgstr "Moyenne journalière" -#: pyaggr3g470r/templates/feed.html:24 +#: pyaggr3g470r/templates/feed.html:36 msgid "between the" msgstr "entre le" -#: pyaggr3g470r/templates/feed.html:24 +#: pyaggr3g470r/templates/feed.html:36 msgid "and the" msgstr "et le" @@ -564,8 +405,8 @@ msgid "Site" msgstr "Site" #: pyaggr3g470r/templates/feeds.html:13 pyaggr3g470r/templates/feeds.html:32 -#: pyaggr3g470r/templates/home.html:30 pyaggr3g470r/templates/home.html:45 -#: pyaggr3g470r/templates/layout.html:104 +#: pyaggr3g470r/templates/home.html:33 pyaggr3g470r/templates/home.html:51 +#: pyaggr3g470r/templates/layout.html:95 msgid "Articles" msgstr "Articles" @@ -587,6 +428,15 @@ msgstr "Flux désactivé" msgid "Duplicate articles" msgstr "Articles doublon" +#: pyaggr3g470r/templates/history.html:4 +#: pyaggr3g470r/templates/layout.html:110 +msgid "History" +msgstr "Historique" + +#: pyaggr3g470r/templates/history.html:9 +msgid "all years" +msgstr "toutes les années" + #: pyaggr3g470r/templates/home.html:10 msgid "You don't have any feeds." msgstr "Vous n'avez pas de flux." @@ -604,31 +454,44 @@ msgstr "ou" msgid "upload an OPML file." msgstr "téléchargez un fichier OPML." -#: pyaggr3g470r/templates/home.html:18 pyaggr3g470r/templates/layout.html:118 +#: pyaggr3g470r/templates/home.html:18 pyaggr3g470r/templates/layout.html:109 msgid "All feeds" msgstr "Tous les flux" -#: pyaggr3g470r/templates/home.html:33 pyaggr3g470r/templates/home.html:48 +#: pyaggr3g470r/templates/home.html:25 pyaggr3g470r/templates/home.html:43 +msgid "error" +msgstr "erreur" + +#: pyaggr3g470r/templates/home.html:36 pyaggr3g470r/templates/home.html:54 msgid "Mark this feed as read" msgstr "Marquer ce flux comme lu" -#: pyaggr3g470r/templates/home.html:34 pyaggr3g470r/templates/home.html:49 +#: pyaggr3g470r/templates/home.html:37 pyaggr3g470r/templates/home.html:55 msgid "Mark this feed as unread" msgstr "Marquer ce flux comme non lu" -#: pyaggr3g470r/templates/home.html:57 pyaggr3g470r/templates/home.html:65 +#: pyaggr3g470r/templates/home.html:63 pyaggr3g470r/templates/home.html:71 msgid "All" msgstr "Tout" -#: pyaggr3g470r/templates/home.html:58 +#: pyaggr3g470r/templates/home.html:64 msgid "Read" msgstr "Lus" -#: pyaggr3g470r/templates/home.html:59 pyaggr3g470r/templates/layout.html:115 +#: pyaggr3g470r/templates/home.html:65 pyaggr3g470r/templates/layout.html:106 msgid "Unread" msgstr "Non lus" -#: pyaggr3g470r/templates/home.html:76 +#: pyaggr3g470r/templates/home.html:81 +#: pyaggr3g470r/templates/admin/user.html:42 pyaggr3g470r/views/views.py:630 +msgid "Feed" +msgstr "Flux" + +#: pyaggr3g470r/templates/home.html:82 pyaggr3g470r/views/views.py:336 +msgid "Article" +msgstr "Article" + +#: pyaggr3g470r/templates/home.html:83 msgid "Date" msgstr "Date" @@ -644,39 +507,39 @@ msgstr "jours" msgid "No inactive feeds." msgstr "Aucun flux inactifs." -#: pyaggr3g470r/templates/layout.html:106 +#: pyaggr3g470r/templates/layout.html:97 msgid "Fetch" msgstr "Télécharger" -#: pyaggr3g470r/templates/layout.html:107 +#: pyaggr3g470r/templates/layout.html:98 msgid "Mark all as read" msgstr "Marquer tout comme lu" -#: pyaggr3g470r/templates/layout.html:113 +#: pyaggr3g470r/templates/layout.html:100 pyaggr3g470r/views/views.py:617 +msgid "Add a feed" +msgstr "Ajouter un flux" + +#: pyaggr3g470r/templates/layout.html:104 msgid "Filter" msgstr "Filtrer" -#: pyaggr3g470r/templates/layout.html:116 +#: pyaggr3g470r/templates/layout.html:107 msgid "Favorites" msgstr "Favoris" -#: pyaggr3g470r/templates/layout.html:117 +#: pyaggr3g470r/templates/layout.html:108 msgid "Inactive feeds" msgstr "Flux inactifs" -#: pyaggr3g470r/templates/layout.html:119 -msgid "History" -msgstr "Historique" - -#: pyaggr3g470r/templates/layout.html:122 +#: pyaggr3g470r/templates/layout.html:113 msgid "Management" msgstr "Gestion" -#: pyaggr3g470r/templates/layout.html:124 +#: pyaggr3g470r/templates/layout.html:115 msgid "Dashboard" msgstr "Tableau de bord" -#: pyaggr3g470r/templates/layout.html:127 +#: pyaggr3g470r/templates/layout.html:118 msgid "Logout" msgstr "Déconnexion" @@ -828,6 +691,11 @@ msgstr "Supprimer cet utilisateur" msgid "You are going to delete this account." msgstr "Vous allez supprimer ce compte." +#: pyaggr3g470r/templates/admin/dashboard.html:45 +#: pyaggr3g470r/views/views.py:797 +msgid "Add a new user" +msgstr "Ajouter un nouvel utilisateur" + #: pyaggr3g470r/templates/admin/dashboard.html:46 msgid "Send notification messages" msgstr "Envoyer des messages de notification" @@ -852,6 +720,195 @@ msgstr "Nom de famille" msgid "Number of articles" msgstr "Nombre d'articles" +#: pyaggr3g470r/views/views.py:103 +msgid "Authentication required." +msgstr "Authentification requise." + +#: pyaggr3g470r/views/views.py:108 +msgid "Forbidden." +msgstr "Interdit." + +#: pyaggr3g470r/views/views.py:161 +msgid "Logged in successfully." +msgstr "Connecté avec succès." + +#: pyaggr3g470r/views/views.py:184 +msgid "Logged out successfully." +msgstr "Déconnecté avec succès." + +#: pyaggr3g470r/views/views.py:193 +msgid "Self-registration is disabled." +msgstr "L'auto-enregistrement est désactivé." + +#: pyaggr3g470r/views/views.py:210 +msgid "Email already used." +msgstr "Email déjà utilisé." + +#: pyaggr3g470r/views/views.py:217 pyaggr3g470r/views/views.py:848 +msgid "Problem while sending activation email" +msgstr "Problème lors de l'envoi d'email d'activation" + +#: pyaggr3g470r/views/views.py:220 +msgid "Your account has been created. Check your mail to confirm it." +msgstr "Votre compte a été créé. Vérifiez votre courrier pour le confirmer." + +#: pyaggr3g470r/views/views.py:269 pyaggr3g470r/views/views.py:519 +msgid "Downloading articles..." +msgstr "Téléchargement des articles." + +#: pyaggr3g470r/views/views.py:271 +msgid "" +"The manual retrieving of news is only available for administrator, on the " +"Heroku platform." +msgstr "" +"La récupération manuelle de nouvelles est disponible uniquement pour " +"l'administrateur, sur la plate-forme Heroku." + +#: pyaggr3g470r/views/views.py:336 +msgid "deleted." +msgstr "supprimé." + +#: pyaggr3g470r/views/views.py:339 +msgid "This article do not exist." +msgstr "Cet article n'existe pas." + +#: pyaggr3g470r/views/views.py:417 +msgid "Indexing database..." +msgstr "Indexation la base de données..." + +#: pyaggr3g470r/views/views.py:419 pyaggr3g470r/views/views.py:488 +msgid "An error occured" +msgstr "Une erreur est survenue." + +#: pyaggr3g470r/views/views.py:422 +msgid "Option not available on Heroku." +msgstr "Option non disponible sur Heroku." + +#: pyaggr3g470r/views/views.py:437 pyaggr3g470r/views/views.py:447 +msgid "Error when exporting articles." +msgstr "Erreur lors de l'export des articles." + +#: pyaggr3g470r/views/views.py:453 +msgid "Export format not supported." +msgstr "Ce format d'export n'est pas supporté." + +#: pyaggr3g470r/views/views.py:476 +msgid "Full text search is not yet implemented for Heroku." +msgstr "La recherche rapide n'est pas supporté sur Heroku." + +#: pyaggr3g470r/views/views.py:513 pyaggr3g470r/views/views.py:526 +#: pyaggr3g470r/views/views.py:534 +msgid "File not allowed." +msgstr "Fichier non autorisé." + +#: pyaggr3g470r/views/views.py:518 +msgid "feeds imported." +msgstr "flux importés." + +#: pyaggr3g470r/views/views.py:521 +msgid "Impossible to import the new feeds." +msgstr "Impossible d'importer les nouveaux flux." + +#: pyaggr3g470r/views/views.py:530 +msgid "Account imported." +msgstr "Compte importé." + +#: pyaggr3g470r/views/views.py:532 +msgid "Impossible to import the account." +msgstr "Impossible d'importer le compte." + +#: pyaggr3g470r/views/views.py:579 +msgid "Feed successfully updated." +msgstr "Flux mis à jour avec succès." + +#: pyaggr3g470r/views/views.py:590 +msgid "Feed successfully created." +msgstr "Flux créé avec succès." + +#: pyaggr3g470r/views/views.py:593 +msgid "Downloading articles for the new feed..." +msgstr "Téléchargement des articles du nouveau flux..." + +#: pyaggr3g470r/views/views.py:597 +msgid "Feed already in the database." +msgstr "Flux déjà dans la base de données." + +#: pyaggr3g470r/views/views.py:604 +msgid "Edit the feed" +msgstr "Éditez ce flux" + +#: pyaggr3g470r/views/views.py:630 pyaggr3g470r/views/views.py:825 +msgid "successfully deleted." +msgstr "supprimé avec succès." + +#: pyaggr3g470r/views/views.py:648 pyaggr3g470r/views/views.py:775 +#: pyaggr3g470r/views/views.py:785 pyaggr3g470r/views/views.py:825 +msgid "User" +msgstr "Utilisateur" + +#: pyaggr3g470r/views/views.py:649 pyaggr3g470r/views/views.py:775 +msgid "successfully updated." +msgstr "mis à jour avec succès." + +#: pyaggr3g470r/views/views.py:669 +msgid "Your account has been deleted." +msgstr "Votre compte a été supprimé." + +#: pyaggr3g470r/views/views.py:671 pyaggr3g470r/views/views.py:811 +#: pyaggr3g470r/views/views.py:827 pyaggr3g470r/views/views.py:855 +msgid "This user does not exist." +msgstr "Cet utilisateur n'existe pas." + +#: pyaggr3g470r/views/views.py:685 +msgid "Articles deleted." +msgstr "Articles supprimés." + +#: pyaggr3g470r/views/views.py:699 +msgid "Your account has been confirmed." +msgstr "Votre compte a été confirmé." + +#: pyaggr3g470r/views/views.py:701 +msgid "Impossible to confirm this account." +msgstr "Impossible de confirmer ce compte." + +#: pyaggr3g470r/views/views.py:723 +msgid "New password sent to your address." +msgstr "Nouveau mot de passe envoyé à votre adresse." + +#: pyaggr3g470r/views/views.py:725 +msgid "Problem while sending your new password." +msgstr "Problème lors de l'envoi de votre nouveau mot de passe." + +#: pyaggr3g470r/views/views.py:750 +msgid "Problem while sending email" +msgstr "Problème lors de l'envoi de l'email" + +#: pyaggr3g470r/views/views.py:785 +msgid "successfully created." +msgstr "créé avec succès." + +#: pyaggr3g470r/views/views.py:794 +msgid "Edit the user" +msgstr "Éditer cet utilisateur" + +#: pyaggr3g470r/views/views.py:846 pyaggr3g470r/views/views.py:852 +msgid "Account of the user" +msgstr "Compte de l'utilisateur" + +#: pyaggr3g470r/views/views.py:846 +msgid "successfully activated." +msgstr "activé avec succès." + +#: pyaggr3g470r/views/views.py:852 +msgid "successfully disabled." +msgstr "désactivé avec succès." + +#~ msgid "of the database" +#~ msgstr "de la base de données" + +#~ msgid "Email notification" +#~ msgstr "Notification par email" + #~ msgid "Database indexed." #~ msgstr "Base de données indexée." @@ -912,9 +969,6 @@ msgstr "Nombre d'articles" #~ msgid "First name" #~ msgstr "Prénom" -#~ msgid "Last name" -#~ msgstr "Nom de famille" - #~ msgid "Please enter your last name." #~ msgstr "S'il vous plaît, entrez votre nom de famille." diff --git a/pyaggr3g470r/utils.py b/pyaggr3g470r/utils.py index 3ed89f55..ea8a87bf 100755 --- a/pyaggr3g470r/utils.py +++ b/pyaggr3g470r/utils.py @@ -4,7 +4,7 @@ # pyAggr3g470r - A Web based news aggregator. # Copyright (C) 2010-2015 Cédric Bonhomme - https://www.cedricbonhomme.org # -# For more information : https://bitbucket.org/cedricbonhomme/pyaggr3g470r/ +# For more information : https://bitbucket.org/cedricbonhomme/pyaggr3g470r # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as @@ -20,15 +20,16 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. __author__ = "Cedric Bonhomme" -__version__ = "$Revision: 1.6 $" +__version__ = "$Revision: 1.7 $" __date__ = "$Date: 2010/12/07 $" -__revision__ = "$Date: 2013/11/17 $" +__revision__ = "$Date: 2015/03/28 $" __copyright__ = "Copyright (c) Cedric Bonhomme" __license__ = "AGPLv3" # # This file provides functions used for: -# - the database management; +# - detection of duplicate articles; +# - import from a JSON file; # - generation of tags cloud; # - HTML processing. # @@ -41,30 +42,34 @@ import logging import datetime import operator import urllib +import itertools import subprocess +import sqlalchemy try: from urlparse import urlparse, parse_qs, urlunparse except: from urllib.parse import urlparse, parse_qs, urlunparse from bs4 import BeautifulSoup +from datetime import timedelta from collections import Counter from contextlib import contextmanager import conf from flask import g +from bootstrap import application as app, db +from pyaggr3g470r import controllers from pyaggr3g470r.models import User, Feed, Article - -# regular expression to check URL -url_finders = [ - re.compile("([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}|(((news|telnet|nttp|file|http|ftp|https)://)|(www|ftp)[-A-Za-z0-9]*\\.)[-A-Za-z0-9\\.]+)(:[0-9]*)?/[-A-Za-z0-9_\\$\\.\\+\\!\\*\\(\\),;:@&=\\?/~\\#\\%]*[^]'\\.}>\\),\\\"]"), \ - re.compile("([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}|(((news|telnet|nttp|file|http|ftp|https)://)|(www|ftp)[-A-Za-z0-9]*\\.)[-A-Za-z0-9\\.]+)(:[0-9]*)?"), \ - re.compile("(~/|/|\\./)([-A-Za-z0-9_\\$\\.\\+\\!\\*\\(\\),;:@&=\\?/~\\#\\%]|\\\\)+"), \ - re.compile("'\\<((mailto:)|)[-A-Za-z0-9\\.]+@[-A-Za-z0-9\\.]+") \ -] - logger = logging.getLogger(__name__) +ALLOWED_EXTENSIONS = set(['xml', 'opml', 'json']) + +def allowed_file(filename): + """ + Check if the uploaded file is allowed. + """ + return '.' in filename and \ + filename.rsplit('.', 1)[1] in ALLOWED_EXTENSIONS @contextmanager def opened_w_error(filename, mode="r"): @@ -79,9 +84,31 @@ def opened_w_error(filename, mode="r"): f.close() def fetch(id, feed_id=None): - cmd = [conf.PYTHON, conf.basedir+'/manager.py', 'fetch_asyncio', str(id), str(feed_id)] + """ + Fetch the feeds in a new processus. + The "asyncio" crawler is launched with the manager. + """ + cmd = [conf.PYTHON, conf.basedir+'/manager.py', 'fetch_asyncio', str(id), + str(feed_id)] p = subprocess.Popen(cmd, stdout=subprocess.PIPE) +def history(user_id, year=None, month=None): + """ + Sort articles by year and month. + """ + articles_counter = Counter() + articles = controllers.ArticleController(user_id).read() + if None != year: + articles = articles.filter(sqlalchemy.extract('year', Article.date) == year) + if None != month: + articles = articles.filter(sqlalchemy.extract('month', Article.date) == month) + for article in articles.all(): + if None != year: + articles_counter[article.date.month] += 1 + else: + articles_counter[article.date.year] += 1 + return articles_counter, articles + def import_opml(email, opml_content): """ Import new feeds from an OPML file. @@ -98,45 +125,35 @@ def import_opml(email, opml_content): Parse recursively through the categories and sub-categories. """ for subscription in subsubscription: - if len(subscription) != 0: nb = read(subscription, nb) else: - try: title = subscription.text - except: title = "" - try: description = subscription.description except: description = "" - try: link = subscription.xmlUrl except: continue - if None != Feed.query.filter(Feed.user_id == user.id, Feed.link == link).first(): continue - try: site_link = subscription.htmlUrl except: site_link = "" - new_feed = Feed(title=title, description=description, link=link, site_link=site_link, enabled=True) - user.feeds.append(new_feed) nb += 1 return nb - nb = read(subscriptions) - g.db.session.commit() + db.session.commit() return nb def import_json(email, json_content): @@ -146,44 +163,46 @@ def import_json(email, json_content): user = User.query.filter(User.email == email).first() json_account = json.loads(json_content) nb_feeds, nb_articles = 0, 0 - - # Create feeds + # Create feeds: for feed in json_account["result"]: - - if None != Feed.query.filter(Feed.user_id == user.id, Feed.link == feed["link"]).first(): + if None != Feed.query.filter(Feed.user_id == user.id, + Feed.link == feed["link"]).first(): continue - - new_feed = Feed(title=feed["title"], description="", link=feed["link"], \ - site_link=feed["site_link"], \ - created_date=datetime.datetime.fromtimestamp(int(feed["created_date"])), - enabled=feed["enabled"]) + new_feed = Feed(title=feed["title"], + description="", + link=feed["link"], + site_link=feed["site_link"], + created_date=datetime.datetime.\ + fromtimestamp(int(feed["created_date"])), + enabled=feed["enabled"]) user.feeds.append(new_feed) nb_feeds += 1 - g.db.session.commit() - - # Create articles + db.session.commit() + # Create articles: for feed in json_account["result"]: - user_feed = Feed.query.filter(Feed.user_id == user.id, Feed.link == feed["link"]).first() + user_feed = Feed.query.filter(Feed.user_id == user.id, + Feed.link == feed["link"]).first() if None != user_feed: for article in feed["articles"]: - if None == Article.query.filter(Article.user_id == user.id, - Article.feed_id == user_feed.id, - Article.link == article["link"]).first(): - - new_article = Article(link=article["link"], title=article["title"], \ - content=article["content"], readed=article["readed"], like=article["like"], \ - retrieved_date=datetime.datetime.fromtimestamp(int(article["retrieved_date"])), - date=datetime.datetime.fromtimestamp(int(article["date"])), - user_id=user.id, feed_id=user_feed.id) - + Article.feed_id == user_feed.id, + Article.link == article["link"]).first(): + new_article = Article(link=article["link"], + title=article["title"], + content=article["content"], + readed=article["readed"], + like=article["like"], \ + retrieved_date=datetime.datetime.\ + fromtimestamp(int(article["retrieved_date"])), + date=datetime.datetime.\ + fromtimestamp(int(article["date"])), + user_id=user.id, + feed_id=user_feed.id) user_feed.articles.append(new_article) nb_articles += 1 - g.db.session.commit() - + db.session.commit() return nb_feeds, nb_articles - def clean_url(url): """ Remove utm_* parameters @@ -201,7 +220,6 @@ def clean_url(url): parsed_url.fragment ]).rstrip('=') - def open_url(url): """ Open an URL with the proxy and the user-agent @@ -220,7 +238,6 @@ def open_url(url): # server couldn't fulfill the request error = (url, e.code, \ http.server.BaseHTTPRequestHandler.responses[e.code][1]) - #pyaggr3g470r_log.error(url + " " + str(e.code) + " " + http.server.BaseHTTPRequestHandler.responses[e.code][1]) return (False, error) except urllib.error.URLError as e: # failed to reach the server @@ -229,10 +246,8 @@ def open_url(url): #pyaggr3g470r_log.error(url + " " + e.reason) else: error = (url, e.reason.errno, e.reason.strerror) - #pyaggr3g470r_log.error(url + " " + str(e.reason.errno) + " " + e.reason.strerror) return (False, error) - def clear_string(data): """ Clear a string by removing HTML tags, HTML special caracters @@ -242,7 +257,6 @@ def clear_string(data): q = re.compile('\s') # consecutive white spaces return p.sub('', q.sub(' ', data)) - def load_stop_words(): """ Load the stop words and return them in a list. @@ -258,7 +272,6 @@ def load_stop_words(): stop_words += stop_wods_file.read().split(";") return stop_words - def top_words(articles, n=10, size=5): """ Return the n most frequent words in a list. @@ -273,16 +286,30 @@ def top_words(articles, n=10, size=5): words[word] += 1 return words.most_common(n) - def tag_cloud(tags): """ Generates a tags cloud. """ tags.sort(key=operator.itemgetter(0)) - return '\n'.join([('<font size=%d><a href="/search/?query=%s" title="Count: %s">%s</a></font>' % \ + return '\n'.join([('<font size=%d><a href="/search?query=%s" title="Count: %s">%s</a></font>' % \ (min(1 + count * 7 / max([tag[1] for tag in tags]), 7), word, format(count, ',d'), word)) \ for (word, count) in tags]) +def compare_documents(feed): + """ + Compare a list of documents by pair. + Pairs of duplicates are sorted by "retrieved date". + """ + duplicates = [] + for pair in itertools.combinations(feed.articles, 2): + date1, date2 = pair[0].date, pair[1].date + if clear_string(pair[0].title) == clear_string(pair[1].title) and \ + (date1 - date2) < timedelta(days = 1): + if pair[0].retrieved_date < pair[1].retrieved_date: + duplicates.append((pair[0], pair[1])) + else: + duplicates.append((pair[1], pair[0])) + return duplicates def search_feed(url): """ @@ -306,7 +333,6 @@ def search_feed(url): return feed_link['href'] return None - if __name__ == "__main__": import_opml("root@pyAggr3g470r.localhost", "./var/feeds_test.opml") #import_opml("root@pyAggr3g470r.localhost", "./var/pyAggr3g470r.opml") diff --git a/pyaggr3g470r/views/api/article.py b/pyaggr3g470r/views/api/article.py index 17881412..c3ec2d34 100644 --- a/pyaggr3g470r/views/api/article.py +++ b/pyaggr3g470r/views/api/article.py @@ -1,3 +1,6 @@ +#! /usr/bin/env python +# -*- coding: utf-8 - + from flask import g import dateutil.parser diff --git a/pyaggr3g470r/views/api/common.py b/pyaggr3g470r/views/api/common.py index 48a0d0ac..b8477d4b 100644 --- a/pyaggr3g470r/views/api/common.py +++ b/pyaggr3g470r/views/api/common.py @@ -1,7 +1,10 @@ +#! /usr/bin/env python +# -*- coding: utf-8 - + """For a given resources, classes in the module intend to create the following routes : GET resource/<id> - -> to retreive one + -> to retrieve one POST resource -> to create one PUT resource/<id> @@ -10,7 +13,7 @@ routes : -> to delete one GET resources - -> to retreive several + -> to retrieve several POST resources -> to create several PUT resources @@ -21,7 +24,6 @@ routes : import json import logging import dateutil.parser -from copy import deepcopy from functools import wraps from werkzeug.exceptions import Unauthorized, BadRequest from flask import request, g, session, Response @@ -54,13 +56,11 @@ def authenticate(func): and user.activation_key == "": g.user = user logged_in = True - if logged_in: return func(*args, **kwargs) raise Unauthorized({'WWWAuthenticate': 'Basic realm="Login Required"'}) return wrapper - def to_response(func): """Will cast results of func as a result, and try to extract a status_code for the Response object""" @@ -146,12 +146,16 @@ class PyAggResourceExisting(PyAggAbstractResource): class PyAggResourceMulti(PyAggAbstractResource): def get(self): - """retreive several objects. filters can be set in the payload on the + """retrieve several objects. filters can be set in the payload on the different fields of the object, and a limit can be set in there as well """ - if 'application/json' != request.headers.get('Content-Type'): + if 'application/json' not in request.headers.get('Content-Type'): raise BadRequest("Content-Type must be application/json") - limit = request.json.pop('limit', 10) + limit = 10 + try: + limit = request.json.pop('limit', 10) + except: + return [res for res in self.controller.read().limit(limit)] if not limit: return [res for res in self.controller.read(**request.json).all()] return [res for res in self.controller.read(**request.json).limit(limit)] @@ -159,7 +163,7 @@ class PyAggResourceMulti(PyAggAbstractResource): def post(self): """creating several objects. payload should be a list of dict. """ - if 'application/json' != request.headers.get('Content-Type'): + if 'application/json' not in request.headers.get('Content-Type'): raise BadRequest("Content-Type must be application/json") status = 201 results = [] @@ -180,7 +184,7 @@ class PyAggResourceMulti(PyAggAbstractResource): [[obj_id1, {attr1: val1, attr2: val2}] [obj_id2, {attr1: val1, attr2: val2}]] """ - if 'application/json' != request.headers.get('Content-Type'): + if 'application/json' not in request.headers.get('Content-Type'): raise BadRequest("Content-Type must be application/json") status = 200 results = [] @@ -201,7 +205,7 @@ class PyAggResourceMulti(PyAggAbstractResource): def delete(self): """will delete several objects, a list of their ids should be in the payload""" - if 'application/json' != request.headers.get('Content-Type'): + if 'application/json' not in request.headers.get('Content-Type'): raise BadRequest("Content-Type must be application/json") status = 204 results = [] diff --git a/pyaggr3g470r/views/api/feed.py b/pyaggr3g470r/views/api/feed.py index 0d83ea43..7d0e2862 100644 --- a/pyaggr3g470r/views/api/feed.py +++ b/pyaggr3g470r/views/api/feed.py @@ -1,3 +1,6 @@ +#! /usr/bin/env python +# -*- coding: utf-8 - + from flask import g from pyaggr3g470r.controllers.feed import FeedController, \ @@ -8,7 +11,6 @@ from pyaggr3g470r.views.api.common import PyAggAbstractResource, \ PyAggResourceExisting, \ PyAggResourceMulti - FEED_ATTRS = {'title': {'type': str}, 'description': {'type': str}, 'link': {'type': str}, @@ -20,25 +22,21 @@ FEED_ATTRS = {'title': {'type': str}, 'last_error': {'type': str}, 'error_count': {'type': int, 'default': 0}} - class FeedNewAPI(PyAggResourceNew): controller_cls = FeedController attrs = FEED_ATTRS to_date = ['date', 'last_retrieved'] - class FeedAPI(PyAggResourceExisting): controller_cls = FeedController attrs = FEED_ATTRS to_date = ['date', 'last_retrieved'] - class FeedsAPI(PyAggResourceMulti): controller_cls = FeedController attrs = FEED_ATTRS to_date = ['date', 'last_retrieved'] - class FetchableFeedAPI(PyAggAbstractResource): controller_cls = FeedController to_date = ['date', 'last_retrieved'] @@ -49,7 +47,6 @@ class FetchableFeedAPI(PyAggAbstractResource): return [feed for feed in self.controller.list_fetchable( **self.reqparse_args())] - g.api.add_resource(FeedNewAPI, '/feed', endpoint='feed_new.json') g.api.add_resource(FeedAPI, '/feed/<int:obj_id>', endpoint='feed.json') g.api.add_resource(FeedsAPI, '/feeds', endpoint='feeds.json') diff --git a/pyaggr3g470r/views/article.py b/pyaggr3g470r/views/article.py index 66cc0f37..08c92686 100644 --- a/pyaggr3g470r/views/article.py +++ b/pyaggr3g470r/views/article.py @@ -1,3 +1,6 @@ +#! /usr/bin/env python +# -*- coding: utf-8 - + from flask import Blueprint, g, render_template, redirect from sqlalchemy import desc @@ -18,10 +21,9 @@ def articles(feed_id=None, nb_articles=-1): feed.articles = controllers.ArticleController(g.user.id)\ .read(feed_id=feed.id)\ .order_by(desc("Article.date")) - if len(feed.articles.all()) <= nb_articles: - nb_articles = -1 - if nb_articles == -1: - feed.articles = feed.articles.limit(nb_articles) + if len(feed.articles.all()) <= nb_articles or nb_articles == -1: + nb_articles = int(1e9) + feed.articles = feed.articles.limit(nb_articles) return render_template('articles.html', feed=feed, nb_articles=nb_articles) diff --git a/pyaggr3g470r/views/feed.py b/pyaggr3g470r/views/feed.py index 2af502a7..4fe4e5da 100644 --- a/pyaggr3g470r/views/feed.py +++ b/pyaggr3g470r/views/feed.py @@ -1,5 +1,9 @@ +#! /usr/bin/env python +# -*- coding: utf-8 - + from datetime import datetime from flask import Blueprint, g, render_template +from sqlalchemy import desc from pyaggr3g470r import controllers, utils from pyaggr3g470r.decorators import pyagg_default_decorator, \ @@ -22,9 +26,9 @@ def feed(feed_id=None): "Presents detailed information about a feed." feed = controllers.FeedController(g.user.id).get(id=feed_id) word_size = 6 - articles = controllers.ArticleController(g.user.id)\ - .read(feed_id=feed_id).all() - nb_articles = controllers.ArticleController(g.user.id).read().count() + articles = controllers.ArticleController(g.user.id) \ + .read(feed_id=feed_id) \ + .order_by(desc("Article.date")).all() top_words = utils.top_words(articles, n=50, size=int(word_size)) tag_cloud = utils.tag_cloud(top_words) @@ -46,5 +50,4 @@ def feed(feed_id=None): feed=feed, tag_cloud=tag_cloud, first_post_date=first_article, end_post_date=last_article, - nb_articles=nb_articles, average=average, delta=delta, elapsed=elapsed) diff --git a/pyaggr3g470r/views/views.py b/pyaggr3g470r/views/views.py index 7934eef8..e202ad4d 100644 --- a/pyaggr3g470r/views/views.py +++ b/pyaggr3g470r/views/views.py @@ -27,14 +27,13 @@ __copyright__ = "Copyright (c) Cedric Bonhomme" __license__ = "AGPLv3" import os -import json import string import random import hashlib import datetime from collections import namedtuple from bootstrap import application as app, db -from flask import render_template, request, flash, session, Response, \ +from flask import render_template, request, flash, session, \ url_for, redirect, g, current_app, make_response, jsonify from flask.ext.login import LoginManager, login_user, logout_user, \ login_required, current_user, AnonymousUserMixin @@ -47,7 +46,8 @@ from sqlalchemy.exc import IntegrityError from werkzeug import generate_password_hash import conf -from pyaggr3g470r import utils, notifications, export, duplicate +from pyaggr3g470r import utils, notifications, export +from pyaggr3g470r import controllers from pyaggr3g470r.models import User, Feed, Article, Role from pyaggr3g470r.decorators import feed_access_required from pyaggr3g470r.forms import SignupForm, SigninForm, AddFeedForm, \ @@ -93,7 +93,7 @@ def before_request(): @login_manager.user_loader def load_user(email): # Return an instance of the User model - return User.query.filter(User.email == email).first() + return controllers.UserController(email).get(email=email) # @@ -153,7 +153,7 @@ def login(): form = SigninForm() if form.validate_on_submit(): - user = User.query.filter(User.email == form.email.data).first() + user = controllers.UserController(form.email.data).get(email=form.email.data) login_user(user) g.user = user session['email'] = form.email.data @@ -265,8 +265,12 @@ def fetch(feed_id=None): Triggers the download of news. News are downloaded in a separated process, mandatory for Heroku. """ - utils.fetch(g.user.id, feed_id) - flash(gettext("Downloading articles..."), 'info') + if not conf.ON_HEROKU or g.user.is_admin(): + utils.fetch(g.user.id, feed_id) + flash(gettext("Downloading articles..."), "info") + else: + flash(gettext("The manual retrieving of news is only available " + + "for administrator, on the Heroku platform."), "info") return redirect(redirect_url()) @app.route('/about', methods=['GET']) @@ -378,7 +382,7 @@ def inactives(): List of inactive feeds. """ nb_days = int(request.args.get('nb_days', 365)) - user = User.query.filter(User.id == g.user.id).first() + user = controllers.UserController(g.user.email).get(email=g.user.email) today = datetime.datetime.now() inactives = [] for feed in user.feeds: @@ -399,7 +403,7 @@ def duplicates(feed_id=None): """ feed = Feed.query.filter(Feed.user_id == g.user.id, Feed.id == feed_id).first() duplicates = [] - duplicates = duplicate.compare_documents(feed) + duplicates = utils.compare_documents(feed) return render_template('duplicates.html', duplicates=duplicates, feed=feed) @app.route('/index_database', methods=['GET']) @@ -425,7 +429,7 @@ def export_articles(): """ Export all articles to HTML or JSON. """ - user = User.query.filter(User.id == g.user.id).first() + user = controllers.UserController(g.user.email).get(id=g.user.id) if request.args.get('format') == "HTML": # Export to HTML try: @@ -457,7 +461,7 @@ def export_opml(): """ Export all feeds to OPML. """ - user = User.query.filter(User.id == g.user.id).first() + user = controllers.UserController(g.user.email).get(id=g.user.id) response = make_response(render_template('opml.xml', user=user, now=datetime.datetime.now())) response.headers['Content-Type'] = 'application/xml' response.headers['Content-Disposition'] = 'attachment; filename=feeds.opml' @@ -506,7 +510,7 @@ def management(): if None != request.files.get('opmlfile', None): # Import an OPML file data = request.files.get('opmlfile', None) - if not g.allowed_file(data.filename): + if not utils.allowed_file(data.filename): flash(gettext('File not allowed.'), 'danger') else: try: @@ -519,7 +523,7 @@ def management(): elif None != request.files.get('jsonfile', None): # Import an account data = request.files.get('jsonfile', None) - if not g.allowed_file(data.filename): + if not utils.allowed_file(data.filename): flash(gettext('File not allowed.'), 'danger') else: try: @@ -540,10 +544,15 @@ def management(): not_on_heroku = not conf.ON_HEROKU) @app.route('/history', methods=['GET']) +@app.route('/history/<int:year>', methods=['GET']) +@app.route('/history/<int:year>/<int:month>', methods=['GET']) @login_required -def history(): - #user = User.query.filter(User.id == g.user.id).first() - return render_template('history.html') +def history(year=None, month=None): + articles_counter, articles = utils.history(g.user.id, year, month) + return render_template('history.html', + articles_counter=articles_counter, + articles=articles, + year=year, month=month) @app.route('/bookmarklet', methods=['GET']) @app.route('/create_feed', methods=['GET', 'POST']) @@ -628,7 +637,7 @@ def profile(): """ Edit the profile of the currently logged user. """ - user = User.query.filter(User.email == g.user.email).first() + user = controllers.UserController(g.user.email).get(id=g.user.id) form = ProfileForm() if request.method == 'POST': @@ -654,7 +663,7 @@ def delete_account(): """ Delete the account of the user (with all its data). """ - user = User.query.filter(User.email == g.user.email).first() + user = controllers.UserController(g.user.email).get(id=g.user.id) if user is not None: db.session.delete(user) db.session.commit() diff --git a/requirements.txt b/requirements.txt index 3469aef9..8dc23d8a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,10 +1,14 @@ aiohttp -feedparser -opml requests +requests-futures +feedparser beautifulsoup4 lxml +opml +psycopg2 SQLAlchemy +alembic +whoosh Flask Flask-SQLAlchemy Flask-Login @@ -14,11 +18,7 @@ Flask-RESTful Flask-Babel Flask-SSLify Flask-Migrate -flask-Script +Flask-Script WTForms python-postmark -whoosh python-dateutil -alembic -requests-futures==0.9.5 -psycopg2 diff --git a/runserver.py b/runserver.py index 2ced409f..db1c4410 100755 --- a/runserver.py +++ b/runserver.py @@ -18,7 +18,7 @@ # # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. - +import calendar from bootstrap import conf, application, db, populate_g from flask.ext.babel import Babel from flask.ext.babel import format_datetime @@ -27,19 +27,16 @@ if conf.ON_HEROKU: from flask_sslify import SSLify SSLify(application) -ALLOWED_EXTENSIONS = set(['xml', 'opml', 'json']) - -def allowed_file(filename): - """ - Check if the uploaded file is allowed. - """ - return '.' in filename and \ - filename.rsplit('.', 1)[1] in ALLOWED_EXTENSIONS - babel = Babel(application) +# Jinja filters application.jinja_env.filters['datetime'] = format_datetime +#@register.filter +def month_name(month_number): + return calendar.month_name[month_number] +application.jinja_env.filters['month_name'] = month_name + # Views from flask.ext.restful import Api from flask import g @@ -48,7 +45,6 @@ with application.app_context(): populate_g() g.api = Api(application, prefix='/api/v2.0') g.babel = babel - g.allowed_file = allowed_file from pyaggr3g470r import views application.register_blueprint(views.articles_bp) diff --git a/vagrant/bootstrap.sh b/vagrant/bootstrap.sh index 1d931cf8..62747f64 100644 --- a/vagrant/bootstrap.sh +++ b/vagrant/bootstrap.sh @@ -11,6 +11,9 @@ if [ $? -ne 0 ]; then exit 1;
fi
+# Installation of PostgreSQL
+apt-get install -y postgresql postgresql-server-dev-9.4 postgresql-client
+
# Install all Python requierements
cd pyaggr3g470r
# For lxml
@@ -19,15 +22,13 @@ apt-get install -y libxml2-dev libxslt1-dev sudo pip3 install --upgrade -r requirements.txt
wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
sudo python3 get-pip.py
-sudo pip3 uninstall feedparser
+rm get-pip.py
+sudo pip3 uninstall -y feedparser
sudo pip3 install feedparser==5.1.2
# copy of the default configuration files for vagrant
cp vagrant/conf.cfg-sample conf/conf.cfg
cd ..
-# Installation of PostgreSQL
-apt-get install -y postgresql postgresql-server-dev-9.4 postgresql-client
-
# Configuration of the database
echo "127.0.0.1:5432:aggregator:vagrant:xxYzToW42" > .pgpass
chmod 700 .pgpass
@@ -39,7 +40,8 @@ echo "GRANT ALL PRIVILEGES ON DATABASE aggregator TO vagrant;" | sudo -u postgre # Initializes the database
cd pyaggr3g470r
chown -R vagrant:vagrant .
-sudo -u vagrant python3 db_create.py
+sudo -u vagrant python3 manager.py db_empty
+sudo -u vagrant python3 manager.py db_create
# start pyAggr3g470r at startup
echo "#!/bin/sh -e" > /etc/rc.local
@@ -50,3 +52,12 @@ chmod 755 /etc/rc.local # Start the application.
/etc/init.d/rc.local start
+
+
+#write out current crontab
+sudo -u vagrant crontab -l > mycron
+#echo new cron into cron file
+sudo -u vagrant echo "*/30 * * * * cd /home/vagrant/pyaggr3g470r/ ; python3 manager.py fetch_asyncio None None" >> mycron
+#install new cron file
+sudo -u vagrant crontab mycron
+sudo -u vagrant rm mycron
diff --git a/vagrant/conf.cfg-sample b/vagrant/conf.cfg-sample index 93a0b89c..fa76caad 100644 --- a/vagrant/conf.cfg-sample +++ b/vagrant/conf.cfg-sample @@ -1,10 +1,11 @@ [misc] -platform_url = https://pyaggr3g470r.herokuapp.com/ +platform_url = http://127.0.0.1:5000/ admin_email = recaptcha_public_key = recaptcha_private_key = log_path = ./pyaggr3g470r/var/pyaggr3g470r.log python = python3 +nb_worker = 5 [database] uri = postgres://vagrant:xxYzToW42@127.0.0.1:5432/aggregator [feedparser] |