Commit Graph

377 Commits

Author SHA1 Message Date
Achim D. Brucker f7bcd91a34 Initial commit. 2017-06-27 09:00:44 +01:00
Achim D. Brucker 25dfdc3b90 Fixed typo. 2017-06-25 09:42:54 +01:00
Achim D. Brucker ff331f0465 Documented archive hierarchy. 2017-06-24 08:16:34 +01:00
Achim D. Brucker b23ffd9723 Added create_db. 2017-06-23 18:47:14 +01:00
Achim D. Brucker 221a31bbf0 Removed outdated permissions analysis script. 2017-06-23 18:43:12 +01:00
Michael Herzberg b080543e9b Added script to generate small db. 2017-06-23 14:44:07 +01:00
M.S. Herzberg 5949150b00 Added two scripts. 2017-06-22 19:41:11 +01:00
Michael Herzberg 42161363ce Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-06-22 17:46:28 +01:00
Michael Herzberg 6d5221c5d7 Make db path configurable. 2017-06-22 17:46:18 +01:00
Achim D. Brucker 675bd301cc Added missing parallel parameter to second call of update_extensions. 2017-06-21 15:26:48 +01:00
Michael Herzberg 6833ba6683 Fixed sqlite creation, added missing commit 2017-06-20 23:47:31 +01:00
Michael Herzberg 4220d48d34 Close db when an exception is thrown. 2017-06-20 23:15:15 +01:00
Achim D. Brucker 8dbd535e3e Merge branch 'master' into production 2017-06-20 20:03:10 +01:00
Achim D. Brucker d9ebe265ae Re-formatting 2017-06-20 18:17:44 +01:00
Achim D. Brucker 05227494d6 Re-formatting 2017-06-20 18:17:36 +01:00
Michael Herzberg dae5d4caa9 Fixed creation of empty .crx files. 2017-06-20 18:05:22 +01:00
Michael Herzberg 7ef8ecf3b1 Relax json parsing of manifest. 2017-06-20 17:45:13 +01:00
Michael Herzberg 1cfe1bdab9 Also ignore /* style comments in manifests. 2017-06-20 15:22:09 +01:00
Michael Herzberg f40febdf09 Added -p option for create-db (parallelism). 2017-06-20 15:10:32 +01:00
Michael Herzberg 437a00d256 Don't print warning when crx status is 404. 2017-06-20 15:07:44 +01:00
Michael Herzberg 69cdcd7174 Remove JavaScript-style comments from manifest before parsing. 2017-06-20 11:22:54 +01:00
Michael Herzberg b6bf280d1e Fixed error. 2017-06-20 08:49:01 +01:00
Michael Herzberg 3496e89460 Fixed error. 2017-06-20 08:43:43 +01:00
Michael Herzberg aa259807e2 Catch exceptions due to empty crx header file. 2017-06-20 08:42:30 +01:00
Michael Herzberg c47ba57c97 Changed handing of manifest parsing exceptions. 2017-06-20 08:28:50 +01:00
Michael Herzberg d2dd2aaf81 Moved db path into config file. 2017-06-20 08:10:28 +01:00
Michael Herzberg 3b6662d05a Added .swp file to gitignore. 2017-06-20 08:09:40 +01:00
Michael Herzberg 39d7bf0330 Deal with missing annotation block in reviews. 2017-06-20 08:03:15 +01:00
Michael Herzberg fa82129c2b Deal with a possibly missing overview.html.status file. 2017-06-19 21:34:42 +01:00
Michael Herzberg c1cd41c2e1 Improved logging. 2017-06-19 18:41:29 +01:00
Michael Herzberg 282e2c4e8c Worked on sqlite stuff. 2017-06-19 16:42:35 +01:00
Achim D. Brucker 1f403ff2b5 Merge branch 'master' into production 2017-06-19 06:03:52 +01:00
Achim D. Brucker c5a7f82e13 Fixed getopt setup. 2017-06-19 06:03:36 +01:00
Achim D. Brucker 456bf292c8 Merge branch 'master' into production 2017-06-19 05:57:29 +01:00
Achim D. Brucker be293b1ba8 Fixed import. 2017-06-19 05:57:17 +01:00
Achim D. Brucker f95619670c Merge branch 'master' into production 2017-06-18 15:38:13 +01:00
Achim D. Brucker d9195c8174 Max. number of concurrent download can now be configured via command line. 2017-06-18 15:36:21 +01:00
Achim D. Brucker 66eff6780d Fixed passign is_new. 2017-06-17 18:26:04 +01:00
Achim D. Brucker 85c8f6a546 Pass is_new flag to sqlite update. 2017-06-17 18:19:44 +01:00
Achim D. Brucker 2e6323c8c5 Report number of extensions for which the SQL database was updated. 2017-06-17 18:15:08 +01:00
Michael Herzberg c4ff67a129 Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-06-17 17:10:36 +01:00
Michael Herzberg 7f24a9da7a Split db creation into incremental part and separate full regeneration script. 2017-06-17 17:10:18 +01:00
Achim D. Brucker 760ac171f1 Releaxed supported version to 3.4 or 3.5. 2017-06-17 15:43:18 +01:00
Achim D. Brucker ea71c5b6e3 Removed debugging code raising execptions. 2017-06-17 15:38:17 +01:00
Achim D. Brucker 8fcc7ab99f Fixed logging. 2017-06-17 00:48:34 +01:00
Achim D. Brucker 97460c498f Basic support for logging of errors related to SQL import/update. 2017-06-17 00:43:40 +01:00
Achim D. Brucker c4a5c5a231 Ignore non extensions ids in forums.conf. 2017-06-16 23:32:52 +01:00
Achim D. Brucker 86a608c6a1 Re-formatting. 2017-06-16 23:19:13 +01:00
Achim D. Brucker 1c8d68d495 Moved path utility functions into config module. 2017-06-16 23:09:23 +01:00
Achim D. Brucker 5dd424c06d Removed obsolete permstats.py. 2017-06-16 22:48:37 +01:00