 Achim D. Brucker
|
6892a614a6
|
Converted get_existing_ids to new tar-based file layout.
|
2017-03-01 12:48:29 +00:00 |
|
Achim D. Brucker
|
2d97cdb6df
|
Ignore archive directory (locale test data).
|
2017-02-06 17:54:59 +00:00 |
|
Achim D. Brucker
|
ad53411d55
|
Code reformatting.
|
2017-02-05 14:11:09 +00:00 |
|
Achim D. Brucker
|
9bc898812b
|
Reformatting.
|
2017-02-04 00:07:48 +00:00 |
|
Achim D. Brucker
|
75e54d274c
|
Fixed re-dowload bug wrt forum downloads.
|
2017-02-04 00:07:23 +00:00 |
|
Achim D. Brucker
|
2548940e0e
|
Moved to full isodate/time for log file names.
|
2017-02-03 09:46:55 +00:00 |
|
Achim D. Brucker
|
b1666aade0
|
Fixed datetime imports.
|
2017-02-03 09:45:51 +00:00 |
|
Achim D. Brucker
|
a6d8d6d068
|
Fixed empty list check resulting in complete re-download.
|
2017-02-03 08:41:20 +00:00 |
|
Achim D. Brucker
|
51c7bf064f
|
Retry to download extensions where an exception was thrown.
|
2017-02-02 01:51:29 +00:00 |
|
Achim D. Brucker
|
f6a7a6466c
|
Catch exception during discovery.
|
2017-02-01 07:32:52 +00:00 |
|
Achim D. Brucker
|
58f0e2cb1f
|
Fixed non initialised variable in exception handling.
|
2017-01-31 22:35:05 +00:00 |
|
Achim D. Brucker
|
1802350dbb
|
Added timeout for requests.
|
2017-01-31 09:52:21 +00:00 |
|
Achim D. Brucker
|
c73b1e5ac2
|
Compute new extensions by checking if directory already exists.
|
2017-01-29 13:29:46 +00:00 |
|
Achim D. Brucker
|
3dd16d1a95
|
Log IDs with anomalies to log files.
|
2017-01-29 12:27:26 +00:00 |
|
Achim D. Brucker
|
828c6c95da
|
Reformatting.
|
2017-01-28 17:22:28 +00:00 |
|
Achim D. Brucker
|
111c5394c3
|
Increased DDOS protection delay.
|
2017-01-28 17:21:25 +00:00 |
|
Achim D. Brucker
|
f41a2f67b1
|
Only try to download crx if response status is 200.
|
2017-01-28 17:20:17 +00:00 |
|
Achim D. Brucker
|
18a3519bc5
|
Log messages are now (kind of) thread safe.
|
2017-01-28 17:08:18 +00:00 |
|
Achim D. Brucker
|
0083fd8aab
|
Basic multithreading support.
|
2017-01-28 16:37:44 +00:00 |
|
Achim D. Brucker
|
a301796a9b
|
Ensure that new extensions are indeed unkown/new.
|
2017-01-28 15:27:31 +00:00 |
|
Achim D. Brucker
|
644419ed5d
|
Fixed crx download validation.
|
2017-01-28 15:05:56 +00:00 |
|
Achim D. Brucker
|
2dbf90adc5
|
Flush output.
|
2017-01-28 14:49:29 +00:00 |
|
Achim D. Brucker
|
f99a931f43
|
Create empty dirs.
|
2017-01-28 14:37:35 +00:00 |
|
Achim D. Brucker
|
24dd304a6d
|
Print summary to stderr.
|
2017-01-28 14:20:56 +00:00 |
|
Achim D. Brucker
|
c631c22827
|
Support command line options.
|
2017-01-28 14:01:53 +00:00 |
|
Achim D. Brucker
|
d652edb5aa
|
Removed old crawler implementation.
|
2017-01-28 13:40:42 +00:00 |
|
Achim D. Brucker
|
1951df8a9d
|
Fixed path to env.
|
2017-01-28 13:36:59 +00:00 |
|
Achim D. Brucker
|
5a468578e7
|
Removed outdated description.
|
2017-01-28 13:34:50 +00:00 |
|
Achim D. Brucker
|
41a5eddb24
|
Log runtime.
|
2017-01-28 13:32:47 +00:00 |
|
Achim D. Brucker
|
3cdeba20b4
|
Reformatting.
|
2017-01-28 13:15:05 +00:00 |
|
Achim D. Brucker
|
3ed43f036d
|
Refactoring.
|
2017-01-28 13:12:47 +00:00 |
|
Achim D. Brucker
|
23e1147370
|
Refactoring.
|
2017-01-28 13:03:40 +00:00 |
|
Achim D. Brucker
|
e8f01eae55
|
Refactoring.
|
2017-01-28 12:56:29 +00:00 |
|
Achim D. Brucker
|
9c4ba39558
|
Refactoring.
|
2017-01-28 12:52:18 +00:00 |
|
Achim D. Brucker
|
d5528bfb52
|
Renaming main script for parsing/downloading the sitemap.
|
2017-01-28 12:26:17 +00:00 |
|
Achim D. Brucker
|
0affd45201
|
Improved summary.
|
2017-01-28 12:21:52 +00:00 |
|
Achim D. Brucker
|
d188930f3f
|
Only download crx if modified.
|
2017-01-28 11:59:53 +00:00 |
|
Achim D. Brucker
|
d531d04f1d
|
Refactoring.
|
2017-01-28 01:25:24 +00:00 |
|
Achim D. Brucker
|
9a5dc0c3c9
|
Implemented sanity checks for crx download.
|
2017-01-28 01:18:20 +00:00 |
|
Achim D. Brucker
|
2763133c06
|
Google DDOS detection prevention.
|
2017-01-28 01:12:50 +00:00 |
|
Achim D. Brucker
|
32fcf1fd03
|
Log exceptions to file.
|
2017-01-28 01:00:23 +00:00 |
|
Achim D. Brucker
|
379dfce6ef
|
Improved summary.
|
2017-01-28 00:51:21 +00:00 |
|
Achim D. Brucker
|
84c9781951
|
Basic error handling.
|
2017-01-28 00:21:37 +00:00 |
|
Achim D. Brucker
|
9a61341ae2
|
Refactoring.
|
2017-01-27 23:16:48 +00:00 |
|
Achim D. Brucker
|
bd53b117e5
|
Refactoring.
|
2017-01-27 22:40:07 +00:00 |
|
Achim D. Brucker
|
6c4971ab4e
|
Basic download feature implemented.
|
2017-01-27 22:31:21 +00:00 |
|
Achim D. Brucker
|
63869b56d4
|
Cleanup and improved verbose output.
|
2017-01-26 03:53:16 +00:00 |
|
Achim D. Brucker
|
bd18a73271
|
Refactoring.
|
2017-01-26 03:16:47 +00:00 |
|
Achim D. Brucker
|
a346118c13
|
Renamed basedir to archivedir.
|
2017-01-25 19:43:10 +00:00 |
|
Achim D. Brucker
|
0b96aeb49d
|
Computer forum update id.
|
2017-01-24 22:04:34 +00:00 |