 Achim D. Brucker
|
58f0e2cb1f
|
Fixed non initialised variable in exception handling.
|
2017-01-31 22:35:05 +00:00 |
|
Achim D. Brucker
|
1802350dbb
|
Added timeout for requests.
|
2017-01-31 09:52:21 +00:00 |
|
Achim D. Brucker
|
c73b1e5ac2
|
Compute new extensions by checking if directory already exists.
|
2017-01-29 13:29:46 +00:00 |
|
Achim D. Brucker
|
3dd16d1a95
|
Log IDs with anomalies to log files.
|
2017-01-29 12:27:26 +00:00 |
|
Achim D. Brucker
|
828c6c95da
|
Reformatting.
|
2017-01-28 17:22:28 +00:00 |
|
Achim D. Brucker
|
111c5394c3
|
Increased DDOS protection delay.
|
2017-01-28 17:21:25 +00:00 |
|
Achim D. Brucker
|
f41a2f67b1
|
Only try to download crx if response status is 200.
|
2017-01-28 17:20:17 +00:00 |
|
Achim D. Brucker
|
18a3519bc5
|
Log messages are now (kind of) thread safe.
|
2017-01-28 17:08:18 +00:00 |
|
Achim D. Brucker
|
0083fd8aab
|
Basic multithreading support.
|
2017-01-28 16:37:44 +00:00 |
|
Achim D. Brucker
|
a301796a9b
|
Ensure that new extensions are indeed unkown/new.
|
2017-01-28 15:27:31 +00:00 |
|
Achim D. Brucker
|
644419ed5d
|
Fixed crx download validation.
|
2017-01-28 15:05:56 +00:00 |
|
Achim D. Brucker
|
2dbf90adc5
|
Flush output.
|
2017-01-28 14:49:29 +00:00 |
|
Achim D. Brucker
|
f99a931f43
|
Create empty dirs.
|
2017-01-28 14:37:35 +00:00 |
|
Achim D. Brucker
|
24dd304a6d
|
Print summary to stderr.
|
2017-01-28 14:20:56 +00:00 |
|
Achim D. Brucker
|
c631c22827
|
Support command line options.
|
2017-01-28 14:01:53 +00:00 |
|
Achim D. Brucker
|
d652edb5aa
|
Removed old crawler implementation.
|
2017-01-28 13:40:42 +00:00 |
|
Achim D. Brucker
|
1951df8a9d
|
Fixed path to env.
|
2017-01-28 13:36:59 +00:00 |
|
Achim D. Brucker
|
5a468578e7
|
Removed outdated description.
|
2017-01-28 13:34:50 +00:00 |
|
Achim D. Brucker
|
41a5eddb24
|
Log runtime.
|
2017-01-28 13:32:47 +00:00 |
|
Achim D. Brucker
|
3cdeba20b4
|
Reformatting.
|
2017-01-28 13:15:05 +00:00 |
|
Achim D. Brucker
|
3ed43f036d
|
Refactoring.
|
2017-01-28 13:12:47 +00:00 |
|
Achim D. Brucker
|
23e1147370
|
Refactoring.
|
2017-01-28 13:03:40 +00:00 |
|
Achim D. Brucker
|
e8f01eae55
|
Refactoring.
|
2017-01-28 12:56:29 +00:00 |
|
Achim D. Brucker
|
9c4ba39558
|
Refactoring.
|
2017-01-28 12:52:18 +00:00 |
|
Achim D. Brucker
|
d5528bfb52
|
Renaming main script for parsing/downloading the sitemap.
|
2017-01-28 12:26:17 +00:00 |
|
Achim D. Brucker
|
0affd45201
|
Improved summary.
|
2017-01-28 12:21:52 +00:00 |
|
Achim D. Brucker
|
d188930f3f
|
Only download crx if modified.
|
2017-01-28 11:59:53 +00:00 |
|
Achim D. Brucker
|
d531d04f1d
|
Refactoring.
|
2017-01-28 01:25:24 +00:00 |
|
Achim D. Brucker
|
9a5dc0c3c9
|
Implemented sanity checks for crx download.
|
2017-01-28 01:18:20 +00:00 |
|
Achim D. Brucker
|
2763133c06
|
Google DDOS detection prevention.
|
2017-01-28 01:12:50 +00:00 |
|
Achim D. Brucker
|
32fcf1fd03
|
Log exceptions to file.
|
2017-01-28 01:00:23 +00:00 |
|
Achim D. Brucker
|
379dfce6ef
|
Improved summary.
|
2017-01-28 00:51:21 +00:00 |
|
Achim D. Brucker
|
84c9781951
|
Basic error handling.
|
2017-01-28 00:21:37 +00:00 |
|
Achim D. Brucker
|
9a61341ae2
|
Refactoring.
|
2017-01-27 23:16:48 +00:00 |
|
Achim D. Brucker
|
bd53b117e5
|
Refactoring.
|
2017-01-27 22:40:07 +00:00 |
|
Achim D. Brucker
|
6c4971ab4e
|
Basic download feature implemented.
|
2017-01-27 22:31:21 +00:00 |
|
Achim D. Brucker
|
63869b56d4
|
Cleanup and improved verbose output.
|
2017-01-26 03:53:16 +00:00 |
|
Achim D. Brucker
|
bd18a73271
|
Refactoring.
|
2017-01-26 03:16:47 +00:00 |
|
Achim D. Brucker
|
a346118c13
|
Renamed basedir to archivedir.
|
2017-01-25 19:43:10 +00:00 |
|
Achim D. Brucker
|
0b96aeb49d
|
Computer forum update id.
|
2017-01-24 22:04:34 +00:00 |
|
Achim D. Brucker
|
be36b7f129
|
Re-discover already archived ids.
|
2017-01-23 18:54:32 +00:00 |
|
Achim D. Brucker
|
3dc7a5d663
|
Added comment on skipping language specific extension URLs.
|
2017-01-22 15:53:00 +00:00 |
|
Achim D. Brucker
|
1077d04322
|
Switched to new library.
|
2017-01-22 15:52:16 +00:00 |
|
Achim D. Brucker
|
c74f36371c
|
Switched to new library.
|
2017-01-21 20:02:31 +00:00 |
|
Achim D. Brucker
|
6b38e2c738
|
Reformatting.
|
2017-01-21 18:47:44 +00:00 |
|
Achim D. Brucker
|
b4756b3350
|
Modulified parse_sitemap.
|
2017-01-21 12:51:55 +00:00 |
|
Achim D. Brucker
|
fac837d016
|
Module setup.
|
2017-01-20 23:02:56 +00:00 |
 Michael Herzberg
|
c75e2f2c97
|
Added script that parses the chrome web store sitemap for ids. This yields around 100k ids.
|
2017-01-19 14:59:34 +00:00 |
|
Achim D. Brucker
|
97a3e16c72
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-01-18 05:05:54 +00:00 |
|
Achim D. Brucker
|
8622869ef5
|
Added optional summary to stderr.
|
2017-01-17 19:29:03 +00:00 |