Commit Graph

618 Commits

Author SHA1 Message Date
Michael Herzberg c7a808db3f Don't process replies if there are none. 2017-08-30 15:15:12 +01:00
Michael Herzberg 5f234d8539 Improved logging. 2017-08-30 15:12:54 +01:00
Michael Herzberg 12c111ca11 Once more, make mysql the default. Also, increased timeout. 2017-08-30 12:13:25 +01:00
Michael Herzberg 9b8a693a5f Changed logging a little bit. 2017-08-30 12:12:57 +01:00
Michael Herzberg f8c8382919 Merge. 2017-08-30 11:42:12 +01:00
Michael Herzberg 6a9a1cda63 Moved crx logging to where crx will actually be parsed. 2017-08-30 11:29:30 +01:00
Michael Herzberg c7e36cec81 Exclude archive dir when pushing to sharc. 2017-08-30 10:11:25 +01:00
Michael Herzberg 405ab32ffd zip log files 2017-08-30 10:09:15 +01:00
Michael Herzberg d99142f8d0 Added and changed a few columns. 2017-08-30 10:07:06 +01:00
Achim D. Brucker d6d43cfb0a Bug fix: printing of detection method. 2017-08-30 09:56:35 +01:00
Achim D. Brucker 3269a4900c Bug fix: printing of file name in Javascript mode. 2017-08-30 09:56:19 +01:00
Achim D. Brucker b5b6a17ee5 Support analysis of crx files and plain JavaScript files. 2017-08-30 09:11:55 +01:00
Achim D. Brucker cacdf1f727 Refactoring. 2017-08-30 08:28:39 +01:00
Achim D. Brucker 85d6ec084d Bug fix: missing detection method for empty files. 2017-08-30 08:24:16 +01:00
Achim D. Brucker d7120fad45 Bug fix: update char if loops reads another char (next_char). This avoids missing an escape character or a newline. 2017-08-30 01:43:18 +01:00
Achim D. Brucker 9be998efd1 Ignore vscode workspace configuration. 2017-08-30 00:52:47 +01:00
Achim D. Brucker b4dbfc1f19 Added new tool: crx-jsdecompose. 2017-08-30 00:48:28 +01:00
Achim D. Brucker 66818b2fa6 Renamed hash to md5 in JSON file and added support for sha1 hashes. 2017-08-30 00:38:30 +01:00
Achim D. Brucker e947e69f37 Define type and detection method for all generated entries. 2017-08-30 00:24:19 +01:00
Achim D. Brucker ae3bbd7339 Using values of enumeration to obtain nice and short human readable representations. 2017-08-30 00:12:57 +01:00
Achim D. Brucker fb13925be9 Added new tool: crx-jsdecompose. 2017-08-30 00:04:32 +01:00
Michael Herzberg 47f424cf2f Added more logging. 2017-08-29 23:10:46 +01:00
Achim D. Brucker 4f59c56e73 Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-08-29 22:54:54 +01:00
Michael Herzberg 080f00f17c Added new columns for jsfile table. 2017-08-29 22:40:01 +01:00
Michael Herzberg 95d71a9edc Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-08-29 22:29:49 +01:00
Michael Herzberg 3e24d1f08c Changed logging to use logging library. 2017-08-29 22:29:38 +01:00
Achim D. Brucker b4e00ea4fe Renamed extract-crx to crx-extract. 2017-08-29 22:06:18 +01:00
Achim D. Brucker 39cd03dccc Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-08-29 18:01:42 +01:00
Achim D. Brucker 97f5b14158 Compute sha1 for JavaScript files. 2017-08-29 18:01:28 +01:00
Michael Herzberg bddd80c138 Made removal of manifest.json comments stricter. 2017-08-29 15:43:04 +01:00
Michael Herzberg 7ffdf30545 Push manifest into table crx column manifest. 2017-08-29 15:41:13 +01:00
Michael Herzberg 2b11117b6f Always process crx, regardless whether or not crx_etag is already in db. 2017-08-29 15:24:59 +01:00
Michael Herzberg 8b91957372 Reduced default MySQL timeout. 2017-08-29 15:20:58 +01:00
Michael Herzberg 6a99d41471 Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-08-29 15:11:37 +01:00
Achim D. Brucker d4ad5f96f8 Report empty files as own category/type. 2017-08-28 22:38:06 +01:00
Michael Herzberg f81aac7c61 Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-08-28 22:38:05 +01:00
Achim D. Brucker 422112e537 Fixed JSON syntax. 2017-08-28 22:20:38 +01:00
Achim D. Brucker d853b6e2ab Added md5 hashes for several underscore versions. 2017-08-28 22:16:29 +01:00
Achim D. Brucker 9c871e0560 Added new library identifiers. 2017-08-28 22:09:27 +01:00
Achim D. Brucker c3fb2cd598 Added md5 hashes for bootstrap 3.x. 2017-08-28 21:47:30 +01:00
Achim D. Brucker 2ace19f453 Compute js_info (including md5 hash and character set detection) only once per file. 2017-08-28 21:05:15 +01:00
Achim D. Brucker 91dfe67513 Auto-detect character encoding of JavaScript files using cchardet. 2017-08-28 20:53:55 +01:00
Michael Herzberg c30f0c4147 Removed database and host setting. To be set in ~/.my.cnf file now. 2017-08-28 20:17:11 +01:00
Achim D. Brucker 5cff2bc1b7 New check based on file hash (md5). 2017-08-28 20:09:34 +01:00
Achim D. Brucker 030adb6adc Minor refactoring and cleanup. 2017-08-28 19:20:50 +01:00
Michael Herzberg 5175d28edc Convert some stuff to string for db insert. 2017-08-28 17:12:32 +01:00
Achim D. Brucker 1a09277e70 Added mysqlclient to dependencies. 2017-08-28 14:42:16 +01:00
Michael Herzberg 35ec26e001 Flush guarded output immediatly and remove parallelism for sharc jobs :(. 2017-08-28 11:56:34 +01:00
Michael Herzberg 0a4e8839a1 Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-08-28 11:50:49 +01:00
Michael Herzberg 81077b807c Added mysql retry logic and use time.time() now. 2017-08-28 11:50:41 +01:00