 Achim D. Brucker
|
86ed713b9d
|
Updated last_crx call in update_extension to match new archive layout.
|
2017-04-04 22:51:45 +01:00 |
|
Achim D. Brucker
|
8cfd0d3bb2
|
Fixed folder confusion during tar file creation.
|
2017-04-03 21:32:53 +01:00 |
|
Achim D. Brucker
|
90d58705c4
|
Re-formatting.
|
2017-04-02 09:23:13 +01:00 |
|
Achim D. Brucker
|
308c0d8721
|
Improved log output.
|
2017-04-01 06:57:39 +01:00 |
|
Achim D. Brucker
|
5a8b76ce14
|
Move tar creation to (local) tmp dir.
|
2017-03-31 17:04:51 +01:00 |
|
Achim D. Brucker
|
af1aeb5b80
|
Re-formatting.
|
2017-03-30 07:53:04 +01:00 |
|
Achim D. Brucker
|
6513a5727a
|
Report extensions with file corruption explicitely.
|
2017-03-29 06:07:27 +01:00 |
|
Achim D. Brucker
|
e72da5b3ff
|
Catching even more file i/o-exceptions.
|
2017-03-28 06:38:12 +01:00 |
|
Achim D. Brucker
|
660ebcf34d
|
Sync before executing large file operations.
|
2017-03-27 22:00:06 +01:00 |
|
Achim D. Brucker
|
00b59d8429
|
Ignore exceptions during backup creation.
|
2017-03-26 21:30:07 +01:00 |
|
Achim D. Brucker
|
fe49f56959
|
Re-formatting.
|
2017-03-25 06:31:48 +00:00 |
|
Achim D. Brucker
|
e6e8914004
|
Fixed call of write_text (missing argument to interface changes).
|
2017-03-24 06:19:40 +00:00 |
|
Achim D. Brucker
|
581f2ef10d
|
Catch exception in case of corrupt tar file and rename tar file as mitigation.
|
2017-03-23 20:32:51 +00:00 |
|
Achim D. Brucker
|
26a9c7e8f1
|
Increased pool size to 24 to mitigitate longer downloads due to tar/untar-operations.
|
2017-03-22 05:44:09 +00:00 |
|
Achim D. Brucker
|
7007ddebb6
|
Ensure that dir is defined even if an exception is thrown.
|
2017-03-21 20:47:28 +00:00 |
|
Achim D. Brucker
|
8b38afedc5
|
Fixed file permissions.
|
2017-03-20 21:51:06 +00:00 |
|
Achim D. Brucker
|
19bd4f8f44
|
Renaming.
|
2017-03-19 16:34:45 +00:00 |
|
Achim D. Brucker
|
ad47feae61
|
Moved crx analysis in own module.
|
2017-03-18 09:21:18 +00:00 |
|
Achim D. Brucker
|
95561773dd
|
Removed outdated parse sitemap.
|
2017-03-17 06:48:02 +00:00 |
|
Achim D. Brucker
|
74992388e0
|
Re-formatting.
|
2017-03-16 08:30:33 +00:00 |
|
Achim D. Brucker
|
cee61773ed
|
Removed outdated discovery script.
|
2017-03-15 20:06:24 +00:00 |
|
Achim D. Brucker
|
34a7bf7228
|
Updated log directory.
|
2017-03-14 06:52:58 +00:00 |
|
Achim D. Brucker
|
5007e60c71
|
First working version that creates tar archives.
|
2017-03-13 07:02:17 +00:00 |
|
Achim D. Brucker
|
6892a614a6
|
Converted get_existing_ids to new tar-based file layout.
|
2017-03-01 12:48:29 +00:00 |
|
Achim D. Brucker
|
2d97cdb6df
|
Ignore archive directory (locale test data).
|
2017-02-06 17:54:59 +00:00 |
|
Achim D. Brucker
|
ad53411d55
|
Code reformatting.
|
2017-02-05 14:11:09 +00:00 |
|
Achim D. Brucker
|
9bc898812b
|
Reformatting.
|
2017-02-04 00:07:48 +00:00 |
|
Achim D. Brucker
|
75e54d274c
|
Fixed re-dowload bug wrt forum downloads.
|
2017-02-04 00:07:23 +00:00 |
|
Achim D. Brucker
|
2548940e0e
|
Moved to full isodate/time for log file names.
|
2017-02-03 09:46:55 +00:00 |
|
Achim D. Brucker
|
b1666aade0
|
Fixed datetime imports.
|
2017-02-03 09:45:51 +00:00 |
|
Achim D. Brucker
|
a6d8d6d068
|
Fixed empty list check resulting in complete re-download.
|
2017-02-03 08:41:20 +00:00 |
|
Achim D. Brucker
|
51c7bf064f
|
Retry to download extensions where an exception was thrown.
|
2017-02-02 01:51:29 +00:00 |
|
Achim D. Brucker
|
f6a7a6466c
|
Catch exception during discovery.
|
2017-02-01 07:32:52 +00:00 |
|
Achim D. Brucker
|
58f0e2cb1f
|
Fixed non initialised variable in exception handling.
|
2017-01-31 22:35:05 +00:00 |
|
Achim D. Brucker
|
1802350dbb
|
Added timeout for requests.
|
2017-01-31 09:52:21 +00:00 |
|
Achim D. Brucker
|
c73b1e5ac2
|
Compute new extensions by checking if directory already exists.
|
2017-01-29 13:29:46 +00:00 |
|
Achim D. Brucker
|
3dd16d1a95
|
Log IDs with anomalies to log files.
|
2017-01-29 12:27:26 +00:00 |
|
Achim D. Brucker
|
828c6c95da
|
Reformatting.
|
2017-01-28 17:22:28 +00:00 |
|
Achim D. Brucker
|
111c5394c3
|
Increased DDOS protection delay.
|
2017-01-28 17:21:25 +00:00 |
|
Achim D. Brucker
|
f41a2f67b1
|
Only try to download crx if response status is 200.
|
2017-01-28 17:20:17 +00:00 |
|
Achim D. Brucker
|
18a3519bc5
|
Log messages are now (kind of) thread safe.
|
2017-01-28 17:08:18 +00:00 |
|
Achim D. Brucker
|
0083fd8aab
|
Basic multithreading support.
|
2017-01-28 16:37:44 +00:00 |
|
Achim D. Brucker
|
a301796a9b
|
Ensure that new extensions are indeed unkown/new.
|
2017-01-28 15:27:31 +00:00 |
|
Achim D. Brucker
|
644419ed5d
|
Fixed crx download validation.
|
2017-01-28 15:05:56 +00:00 |
|
Achim D. Brucker
|
2dbf90adc5
|
Flush output.
|
2017-01-28 14:49:29 +00:00 |
|
Achim D. Brucker
|
f99a931f43
|
Create empty dirs.
|
2017-01-28 14:37:35 +00:00 |
|
Achim D. Brucker
|
24dd304a6d
|
Print summary to stderr.
|
2017-01-28 14:20:56 +00:00 |
|
Achim D. Brucker
|
c631c22827
|
Support command line options.
|
2017-01-28 14:01:53 +00:00 |
|
Achim D. Brucker
|
d652edb5aa
|
Removed old crawler implementation.
|
2017-01-28 13:40:42 +00:00 |
|
Achim D. Brucker
|
1951df8a9d
|
Fixed path to env.
|
2017-01-28 13:36:59 +00:00 |