 Achim D. Brucker
|
aa32e8071f
|
Merge branch 'master' into production
|
2018-08-17 22:11:58 +01:00 |
|
Achim D. Brucker
|
3cd1b221fd
|
Added reporting of WorkerExceptions.
|
2018-08-17 22:08:40 +01:00 |
Michael Herzberg
|
43b8d01b46
|
Merge branch 'master' into production
|
2018-08-17 15:20:46 +01:00 |
|
Michael Herzberg
|
2d52091292
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2018-08-17 15:20:09 +01:00 |
|
Michael Herzberg
|
873c249504
|
Build list for simhash lazily to save memory.
|
2018-08-17 15:20:00 +01:00 |
|
Michael Herzberg
|
c0477269d0
|
Merge branch 'master' into production
|
2018-08-09 14:20:29 +01:00 |
|
Michael Herzberg
|
e7b7625453
|
Added database documentation.
|
2018-08-09 14:20:01 +01:00 |
|
Michael Herzberg
|
61840d6a86
|
Merge branch 'master' into production
|
2018-08-02 16:14:02 +01:00 |
|
Michael Herzberg
|
e492f516ac
|
Restrict sharc jobs to 1 hour.
|
2018-08-02 16:13:25 +01:00 |
|
Michael Herzberg
|
66db569d5f
|
Only open DB connection when needed.
|
2018-08-02 12:37:57 +01:00 |
|
Michael Herzberg
|
05c1cbdea5
|
Give MySQL server up to 1 hour to recover.
|
2018-08-02 11:47:38 +01:00 |
|
Michael Herzberg
|
c931f0b683
|
Merge branch 'master' into production
|
2018-07-28 19:03:42 +02:00 |
|
Michael Herzberg
|
c6d3056fee
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2018-07-28 19:02:48 +02:00 |
|
Michael Herzberg
|
947ecf50d4
|
Removed queue length and reduced mysql insert batch size.
|
2018-07-28 19:02:38 +02:00 |
|
Achim D. Brucker
|
1ee699cc0f
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2018-07-28 10:48:09 +01:00 |
|
Achim D. Brucker
|
0401ff914d
|
Merge branch 'production' of logicalhacking.com:BrowserSecurity/ExtensionCrawler into production
|
2018-07-28 10:47:42 +01:00 |
|
Achim D. Brucker
|
5783eb8f27
|
Optimized image size.
|
2018-07-28 10:46:24 +01:00 |
|
Michael Herzberg
|
4592cba9b2
|
Actually return n new ids when discovering.
|
2018-07-28 10:32:16 +02:00 |
|
Michael Herzberg
|
45a8486f69
|
Fixed small bug.
|
2018-07-27 16:39:42 +02:00 |
|
Achim D. Brucker
|
540e45ed4d
|
Added cleanup hook to apt configuration.
|
2018-07-27 12:52:51 +01:00 |
|
Michael Herzberg
|
85a5645763
|
Ignore GPU nodes, use less RAM.
|
2018-07-26 18:38:11 +02:00 |
|
Achim D. Brucker
|
d7160e50db
|
Added SPDX identifier.
|
2018-07-21 12:13:54 +01:00 |
|
Achim D. Brucker
|
86523c840c
|
Added master repository URL.
|
2018-07-21 12:12:59 +01:00 |
|
Michael Herzberg
|
67b7a46543
|
Add support for provided extension ids.
|
2018-07-21 02:15:18 +01:00 |
|
Michael Herzberg
|
eb616b0ac3
|
Fix some encoding issues.
|
2018-07-21 01:50:59 +01:00 |
|
Michael Herzberg
|
250bdd2c6b
|
Bundle mysql inserts.
|
2018-07-19 23:26:25 +01:00 |
|
Michael Herzberg
|
a1d866d0ff
|
Overwrite last_updated on duplicate.
|
2018-07-17 14:06:43 +01:00 |
|
Michael Herzberg
|
a6173fe23e
|
Don't look for etags in the DB anymore.
|
2018-07-16 19:19:26 +01:00 |
|
Michael Herzberg
|
4b5cc276ee
|
Added option to use INSERT DELAYED with create-db.
|
2018-07-16 19:14:24 +01:00 |
|
Michael Herzberg
|
1857ec7b75
|
Don't restart processes (hopefully mitigates semlock rebuilding error).
|
2018-07-16 11:21:42 +01:00 |
|
Michael Herzberg
|
8bc4e8fa37
|
Cache etags in applications.
|
2018-07-16 01:04:27 +01:00 |
|
Michael Herzberg
|
595f0f8759
|
Use 16 threads to discover new extensions.
|
2018-07-15 19:19:26 +01:00 |
|
Michael Herzberg
|
c9e66186ef
|
Log duration of tar append.
|
2018-07-15 19:15:20 +01:00 |
|
Michael Herzberg
|
c2eaa10bcb
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2018-07-15 00:57:27 +01:00 |
|
Michael Herzberg
|
3bef0afe7a
|
Group mysql inserts and don't compress them.
|
2018-07-15 00:08:11 +01:00 |
|
Achim D. Brucker
|
9738fb26cf
|
Improved status messages.
|
2018-07-14 12:59:38 +01:00 |
|
Achim D. Brucker
|
78fb88cbc9
|
Improved logging.
|
2018-07-14 12:22:57 +01:00 |
|
Achim D. Brucker
|
7025c77f63
|
Merge branch 'production'
|
2018-07-14 06:02:46 +01:00 |
|
Achim D. Brucker
|
8308aec962
|
Do not try to install image, if image creation failed.
|
2018-07-14 06:02:11 +01:00 |
|
Achim D. Brucker
|
1884a02c33
|
Merge branch 'production'
|
2018-07-13 01:16:57 +01:00 |
|
Achim D. Brucker
|
f91dba6c27
|
Merge branch 'production' of logicalhacking.com:BrowserSecurity/ExtensionCrawler into production
|
2018-07-13 01:16:27 +01:00 |
|
Achim D. Brucker
|
de81fbd224
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2018-07-13 01:12:54 +01:00 |
|
Achim D. Brucker
|
a9bb0adb17
|
Added g++ as build depedency.
|
2018-07-13 01:07:51 +01:00 |
|
Michael Herzberg
|
cdc607f524
|
Deleted simhash queries.
|
2018-07-13 00:43:06 +01:00 |
|
Michael Herzberg
|
8f0ce7535a
|
Added build requirements for pycryptodome.
|
2018-07-13 00:41:10 +01:00 |
|
Achim D. Brucker
|
29bfc6fd38
|
Switched from pycrypto (2.6.1) to pycryptodomex (3.4.6). This should
mitigate CVE-2018-6594.
|
2018-07-12 23:02:51 +01:00 |
|
Michael Herzberg
|
455b7e02c9
|
Added md5 db benchmark.
|
2018-07-12 13:35:09 +01:00 |
|
Michael Herzberg
|
5d4f321dca
|
Changed way of parallelism for simhashbucket and added comparemd5.
|
2018-07-10 20:47:07 +01:00 |
|
Michael Herzberg
|
da5133b2b1
|
Added missing newline.
|
2018-07-08 18:17:09 +01:00 |
|
Michael Herzberg
|
1d0d0d8437
|
Changed most recent query.
|
2018-07-08 17:22:40 +01:00 |