1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
|
++++++++++++
pyAggr3g470r
++++++++++++
Presentation
============
`pyAggr3g470r <https://bitbucket.org/cedricbonhomme/pyaggr3g470r/>`_ is a news aggregator with a web interface
based on `Flask <http://flask.pocoo.org/>`_.
Features
========
* can be deployed on Heroku or on a traditional server;
* multiple users can use the platform;
* articles are accessible via a JSON Web service;
* data liberation: export and import all your account with a JSON file;
* export and import feeds with OPML files;
* export articles to HTML;
* e-mail notification;
* favorite articles;
* detection of inactive feeds;
* share articles with Google +, Pinboard and reddit;
* HTTP proxy support.
Usage
=====
Deployment
----------
This application can be deployed on Heroku or on a traditional server.
After installation, you will be able to connect with the email *root@pyAggr3g470r.localhost* and the password *password*.
Deploying the application on Heroku
'''''''''''''''''''''''''''''''''''
An instance of pyAggr3g470r is running `here <https://pyaggr3g470r.herokuapp.com/>`_.
.. code:: bash
$ git clone https://bitbucket.org/cedricbonhomme/pyaggr3g470r.git
$ cd pyaggr3g470r
$ heroku create
$ heroku addons:add heroku-postgresql:dev
$ heroku config:set HEROKU=1
$ git push heroku master
$ heroku run init
$ heroku ps:scale web=1
To enable account creation for users, you have to set some environment variables:
.. code:: bash
$ heroku config:set PLATFORM_URL=<URL-of-your-platform>
$ heroku config:set RECAPTCHA_PUBLIC_KEY=<your-recaptcha-public-key>
$ heroku config:set RECAPTCHA_PRIVATE_KEY=<your-recaptcha-private-key>
$ heroku config:set ADMIN_EMAIL=<administrator-email>
$ heroku addons:add postmark:10k
`Postmark <https://postmarkapp.com/>`_ is used to send account confirmation links.
If you don't want to open your platform you will be still able to create accounts via the administration page.
Deploying the application on a traditional server
'''''''''''''''''''''''''''''''''''''''''''''''''
.. code:: bash
$ git clone https://bitbucket.org/cedricbonhomme/pyaggr3g470r.git
$ cd pyaggr3g470r
$ sudo apt-get install libxml2-dev libxslt1-dev
$ sudo pip install --upgrade -r requirements.txt
$ cp conf/conf.cfg-sample conf/conf.cfg
If you want to use PostgreSQL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: bash
$ sudo apt-get install postgresql postgresql-server-dev-9.3 postgresql-client
$ sudo -u postgres createuser
Enter name of role to add: username
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) y
Shall the new role be allowed to create more new roles? (y/n) n
$ createdb pyAggr3g470r
$ sudo -u postgres psql
postgres=# ALTER USER username WITH ENCRYPTED PASSWORD 'password';
postgres=# GRANT ALL PRIVILEGES ON DATABASE pyAggr3g470r TO username;
postgres=# \q
Edit the configuration file with the line:
.. code:: cfg
[database]
uri = postgres://username:password@127.0.0.1:5433/pyAggr3g470r
If you want to use SQLite
~~~~~~~~~~~~~~~~~~~~~~~~~
Just edit the configuration file with the line:
.. code:: cfg
[database]
uri = sqlite+pysqlite:///pyAggr3g470r.db
Finally:
.. code:: bash
$ python db_create.py
$ python runserver.py
* Running on http://0.0.0.0:5000/
* Restarting with reloader
Configuration
-------------
Configuration (database url, email, proxy, user agent, etc.) is done via the file *conf/conf.cfg*.
Check these configuration before executing *db_create.py*.
If you want to use pyAggr3g470r with Tor/Privoxy, you just have to set the value of
*http_proxy* (most of the time: *http_proxy = 127.0.0.1:8118**). Else leave the value blank.
Automatic updates
-----------------
You can fetch new articles with `cron <https://en.wikipedia.org/wiki/Cron>`_ and the script *fetch.py*.
For example if you want to check for updates every 30 minutes, add this line to your cron rules (*crontab -e*):
.. code:: bash
*/30 * * * * cd ~/.pyaggr3g470r/ ; python fetch.py firstname.lastname@mail.com
You must give the email address you use to login to pyAggr3g470r.
Web service
-----------
It is possible to get your articles *via* a dedicated Web service. See the example below.
.. code:: python
>>> import requests, json
>>> r = requests.get("https://pyaggr3g470r.herokuapp.com/api/v1.0/articles", auth=("your-email", "your-password"))
>>> r.status_code
200
>>> rjson = json.loads(r.text)
>>> rjson["result"][0]["title"]
u'Sponsors required for KDE code sprint in Randa'
>>> rjson["result"][0]["date"]
u'Wed, 18 Jun 2014 14:25:18 GMT'
Possible parameters:
.. code:: bash
$ curl --user your-email:your-password "https://pyaggr3g470r.herokuapp.com/api/v1.0/articles?filter_=unread&feed=24"
$ curl --user your-email:your-password "https://pyaggr3g470r.herokuapp.com/api/v1.0/articles?filter_=read&feed=24&limit=20"
$ curl --user your-email:your-password "https://pyaggr3g470r.herokuapp.com/api/v1.0/articles?filter_=all&feed=24&limit=20"
To get an article:
.. code:: bash
$ curl --user your-email:your-password "https://pyaggr3g470r.herokuapp.com/api/v1.0/articles/84566"
Add an article:
.. code:: python
>>> import requests, json
>>> headers = {'Content-type': 'application/json', 'Accept': 'application/json'}
>>> payload = {'link': 'http://blog.cedricbonhomme.org/2014/05/24/sortie-de-pyaggr3g470r-5-3/', 'title': 'Sortie de pyAggr3g470r 5.3', 'content':'La page principale de pyAggr3g470r a été améliorée...', 'date':str(datetime.datetime.now()), 'feed_id':'42'}
>>> r = requests.post("https://pyaggr3g470r.herokuapp.com/api/v1.0/articles/0", headers=headers, auth=("your-email", "your-password"), data=json.dumps(payload))
>>> print r.content
{
"message": "ok"
}
>>> r = requests.get("https://pyaggr3g470r.herokuapp.com/api/v1.0/articles?feed=42&limit=1", auth=("your-email", "your-password"))
>>> print json.loads(r.content)["result"][0]["title"]
Sortie de pyAggr3g470r 5.3
>>>
For example you can use this in order to connect pyAggr3g470r to your own crawler.
Donation
========
If you wish and if you like *pyAggr3g470r*, you can donate via bitcoin
`1GVmhR9fbBeEh7rP1qNq76jWArDdDQ3otZ <https://blockexplorer.com/address/1GVmhR9fbBeEh7rP1qNq76jWArDdDQ3otZ>`_.
Thank you!
Internationalization
====================
pyAggr3g470r is translated into English and French.
License
=======
`pyAggr3g470r <https://bitbucket.org/cedricbonhomme/pyaggr3g470r>`_
is under the `GNU Affero General Public License version 3 <https://www.gnu.org/licenses/agpl-3.0.html>`_.
Contact
=======
`My home page <http://cedricbonhomme.org/>`_.
|
