Michael Herzberg
|
080f00f17c
|
Added new columns for jsfile table.
|
2017-08-29 22:40:01 +01:00 |
Michael Herzberg
|
95d71a9edc
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-08-29 22:29:49 +01:00 |
Michael Herzberg
|
3e24d1f08c
|
Changed logging to use logging library.
|
2017-08-29 22:29:38 +01:00 |
Achim D. Brucker
|
39cd03dccc
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-08-29 18:01:42 +01:00 |
Achim D. Brucker
|
97f5b14158
|
Compute sha1 for JavaScript files.
|
2017-08-29 18:01:28 +01:00 |
Michael Herzberg
|
bddd80c138
|
Made removal of manifest.json comments stricter.
|
2017-08-29 15:43:04 +01:00 |
Michael Herzberg
|
7ffdf30545
|
Push manifest into table crx column manifest.
|
2017-08-29 15:41:13 +01:00 |
Michael Herzberg
|
2b11117b6f
|
Always process crx, regardless whether or not crx_etag is already in db.
|
2017-08-29 15:24:59 +01:00 |
Michael Herzberg
|
8b91957372
|
Reduced default MySQL timeout.
|
2017-08-29 15:20:58 +01:00 |
Michael Herzberg
|
6a99d41471
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-08-29 15:11:37 +01:00 |
Achim D. Brucker
|
d4ad5f96f8
|
Report empty files as own category/type.
|
2017-08-28 22:38:06 +01:00 |
Michael Herzberg
|
f81aac7c61
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-08-28 22:38:05 +01:00 |
Achim D. Brucker
|
422112e537
|
Fixed JSON syntax.
|
2017-08-28 22:20:38 +01:00 |
Achim D. Brucker
|
d853b6e2ab
|
Added md5 hashes for several underscore versions.
|
2017-08-28 22:16:29 +01:00 |
Achim D. Brucker
|
9c871e0560
|
Added new library identifiers.
|
2017-08-28 22:09:27 +01:00 |
Achim D. Brucker
|
c3fb2cd598
|
Added md5 hashes for bootstrap 3.x.
|
2017-08-28 21:47:30 +01:00 |
Achim D. Brucker
|
2ace19f453
|
Compute js_info (including md5 hash and character set detection) only once per file.
|
2017-08-28 21:05:15 +01:00 |
Achim D. Brucker
|
91dfe67513
|
Auto-detect character encoding of JavaScript files using cchardet.
|
2017-08-28 20:53:55 +01:00 |
Michael Herzberg
|
c30f0c4147
|
Removed database and host setting. To be set in ~/.my.cnf file now.
|
2017-08-28 20:17:11 +01:00 |
Achim D. Brucker
|
5cff2bc1b7
|
New check based on file hash (md5).
|
2017-08-28 20:09:34 +01:00 |
Achim D. Brucker
|
030adb6adc
|
Minor refactoring and cleanup.
|
2017-08-28 19:20:50 +01:00 |
Michael Herzberg
|
5175d28edc
|
Convert some stuff to string for db insert.
|
2017-08-28 17:12:32 +01:00 |
Achim D. Brucker
|
1a09277e70
|
Added mysqlclient to dependencies.
|
2017-08-28 14:42:16 +01:00 |
Michael Herzberg
|
35ec26e001
|
Flush guarded output immediatly and remove parallelism for sharc jobs :(.
|
2017-08-28 11:56:34 +01:00 |
Michael Herzberg
|
0a4e8839a1
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-08-28 11:50:49 +01:00 |
Michael Herzberg
|
81077b807c
|
Added mysql retry logic and use time.time() now.
|
2017-08-28 11:50:41 +01:00 |
Achim D. Brucker
|
9bf0b47f98
|
Minor improvement of string conversion for JsBlock.
|
2017-08-28 10:50:52 +01:00 |
Achim D. Brucker
|
c721e6fdbf
|
Merge with upstream.
|
2017-08-28 10:49:01 +01:00 |
Achim D. Brucker
|
f10923af03
|
Integreated js_mincer into decomposition analysis to allow, in the future, to check comments, code, and string literals explicitely.
|
2017-08-28 10:40:37 +01:00 |
Achim D. Brucker
|
9ef27f9ac9
|
Added missing return statements.
|
2017-08-28 10:28:21 +01:00 |
Achim D. Brucker
|
90b1db4a25
|
Added additional comment checks.
|
2017-08-28 01:26:13 +01:00 |
Achim D. Brucker
|
9b272c9302
|
Added option to merge subsequent single line comments into a single line comment block.
|
2017-08-28 01:17:00 +01:00 |
Achim D. Brucker
|
111777c821
|
Improved position counting.
|
2017-08-28 00:57:58 +01:00 |
Achim D. Brucker
|
d4de20efc1
|
Bug fix: start position of blocks and omit empty code blocks.
|
2017-08-28 00:19:28 +01:00 |
Achim D. Brucker
|
e2e92594ce
|
Bug fix: catch also last block of file.
|
2017-08-27 23:34:33 +01:00 |
Michael Herzberg
|
e04ec98eca
|
Added logging for create-db.
|
2017-08-27 23:04:38 +01:00 |
Michael Herzberg
|
257afe92f0
|
Use selective insert.
|
2017-08-27 23:00:28 +01:00 |
Achim D. Brucker
|
629f492fa7
|
Added tests for code blocks and comments.
|
2017-08-27 22:58:09 +01:00 |
Achim D. Brucker
|
7ff1623bc6
|
Introduced JavaScript mincer working on file objects.
|
2017-08-27 22:51:55 +01:00 |
Michael Herzberg
|
b98b7bc0f7
|
Fixed column typo.
|
2017-08-27 22:49:07 +01:00 |
Achim D. Brucker
|
e324ab9483
|
Re-formatted and added documentation.
|
2017-08-27 22:41:04 +01:00 |
Achim D. Brucker
|
9376b4056f
|
Collect string literals in code blocks.
|
2017-08-27 22:27:35 +01:00 |
Achim D. Brucker
|
41ca506b9f
|
Return iterator that iterates over JavaScript blocks.
|
2017-08-27 22:17:04 +01:00 |
Achim D. Brucker
|
5add586da3
|
Initial commit.
|
2017-08-27 20:47:24 +01:00 |
Achim D. Brucker
|
f6f0bc0394
|
Renamed jsdecompose.py to js_decomposer.py.
|
2017-08-27 20:45:56 +01:00 |
Michael Herzberg
|
9521240d90
|
Make stuff configurable.
|
2017-08-27 18:28:19 +01:00 |
Michael Herzberg
|
0cff600861
|
Fixed etag keys.
|
2017-08-27 17:35:58 +01:00 |
Michael Herzberg
|
d4b0a6535b
|
Fixed some things.
|
2017-08-27 16:57:23 +01:00 |
Michael Herzberg
|
f075192b44
|
made sqlite default again.
|
2017-08-27 03:26:29 +01:00 |
Michael Herzberg
|
22c90dcb4f
|
Truncate timezone from timestamps for mysql, make mysql default.
|
2017-08-27 03:14:43 +01:00 |