Achim D. Brucker
|
5bd7a70813
|
Added missing help text for -a.
|
2017-09-19 10:11:56 +01:00 |
Achim D. Brucker
|
f642f44ea8
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-19 10:08:35 +01:00 |
Achim D. Brucker
|
6ffa08282f
|
Added archive/basedir command line argument.
|
2017-09-19 10:08:05 +01:00 |
Achim D. Brucker
|
50a7ba8a91
|
Minor refactoring.
|
2017-09-19 10:02:46 +01:00 |
Achim D. Brucker
|
4f84c5626d
|
Minor refactoring.
|
2017-09-19 09:16:32 +01:00 |
Michael Herzberg
|
98a2d69ebb
|
This time actually disable annoying HTTPS log messages.
|
2017-09-18 13:41:54 +01:00 |
Michael Herzberg
|
cee90ececc
|
Moved urllib logger config.
|
2017-09-18 12:59:52 +01:00 |
Achim D. Brucker
|
061622f588
|
Refactoring: stub of new main analysis method.
|
2017-09-18 09:09:00 +01:00 |
Achim D. Brucker
|
aadbc5aa0c
|
Refactoring: removed unused variables.
|
2017-09-18 00:35:35 +01:00 |
Achim D. Brucker
|
50b91d3a35
|
Renaming jsFilename -> filename.
|
2017-09-18 00:30:55 +01:00 |
Achim D. Brucker
|
15704a3c10
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-17 19:06:52 +01:00 |
Achim D. Brucker
|
9b7364b5c2
|
Added requirement: python_magic.
|
2017-09-17 19:06:40 +01:00 |
Michael Herzberg
|
175ebd53b7
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-17 17:45:17 +01:00 |
Michael Herzberg
|
7277e6f76e
|
Fixed log msg bug.
|
2017-09-17 17:45:01 +01:00 |
Michael Herzberg
|
0cb7d6e792
|
Fixed error in exception handling.
|
2017-09-17 17:40:48 +01:00 |
Achim D. Brucker
|
701ef2bef6
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-17 13:40:43 +01:00 |
Achim D. Brucker
|
3626b9fb76
|
Ordered and extended enumeration DetectionType. Order reflects reliability of checks.
|
2017-09-17 13:40:38 +01:00 |
Achim D. Brucker
|
a3346cb95e
|
Use file_identfiers module to compute file identifiers.
|
2017-09-17 13:18:49 +01:00 |
Achim D. Brucker
|
6d69377f28
|
Introduced optional parameter data to compute identifiers without opening a file handle.
|
2017-09-17 13:18:20 +01:00 |
Michael Herzberg
|
1ddb9c1c10
|
Surpress HTTPS connection log messages.
|
2017-09-17 12:26:51 +01:00 |
Michael Herzberg
|
1fab393e56
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-16 17:23:16 +01:00 |
Michael Herzberg
|
c3e295267b
|
Log loglevel and only print stacktrace on first mysql exception.
|
2017-09-16 17:22:57 +01:00 |
Achim D. Brucker
|
205c8836e9
|
Bug fix: do not catch exceptions too aggresively and fix libvers computation for updates.
|
2017-09-16 17:20:23 +01:00 |
Achim D. Brucker
|
4cf41e2e4f
|
Refactoring: moved generic file identifiers into own module.
|
2017-09-16 17:19:36 +01:00 |
Achim D. Brucker
|
e98f58fff8
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-16 13:41:56 +01:00 |
Achim D. Brucker
|
24c65daecf
|
Bug fix: check for dirty missed actual function application.
|
2017-09-16 13:41:47 +01:00 |
Achim D. Brucker
|
c274b96f66
|
Added csv output for debugging.
|
2017-09-16 13:21:49 +01:00 |
Michael Herzberg
|
69e95fdf13
|
Catch json parse extensions for reviews etc. more nicely.
|
2017-09-16 12:53:35 +01:00 |
Achim D. Brucker
|
de6dde5269
|
Updated help text to include taskid/maxtaskid.
|
2017-09-16 12:41:18 +01:00 |
Michael Herzberg
|
58aacef3ff
|
Reopen connection after every exception.
|
2017-09-16 12:31:00 +01:00 |
Michael Herzberg
|
a514c0001e
|
Added check for empty crx files.
|
2017-09-16 12:14:41 +01:00 |
Michael Herzberg
|
b51de8577f
|
Added compression for mysql.
|
2017-09-16 12:04:35 +01:00 |
Achim D. Brucker
|
92e1c4c2e5
|
Skip deleted files.
|
2017-09-16 11:41:21 +01:00 |
Achim D. Brucker
|
082cd2fc65
|
Added hacking pull method that uses the regular git binary. While method will not work well with filenames containg spaces and there mit be other glitches, it allows to pull an update of the cdnjs git reposistory (> 100GB) within a couple of minutes compared to a couple of days that the non hackish solution needs.
|
2017-09-16 11:36:40 +01:00 |
Achim D. Brucker
|
5d3343acf1
|
Refactoring: moved git_repo creation into pull_get_list_changed_files(...).
|
2017-09-16 10:33:11 +01:00 |
Achim D. Brucker
|
7b0e63da10
|
Implemented n/N options for external parallelisation (only for fresh initialization).
|
2017-09-15 22:40:46 +01:00 |
Michael Herzberg
|
a1781b9ff9
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-15 21:32:25 +01:00 |
Michael Herzberg
|
1814b1738a
|
Added email notifications on abort.
|
2017-09-15 21:32:12 +01:00 |
Achim D. Brucker
|
400e74ae3f
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-15 20:21:45 +01:00 |
Achim D. Brucker
|
26678636eb
|
Ignore commits where blobs are None.
|
2017-09-15 20:21:05 +01:00 |
Michael Herzberg
|
85680d360b
|
Automatically reopen database connection on failure.
|
2017-09-15 18:23:25 +01:00 |
Michael Herzberg
|
ddbbc2672d
|
Try to insert also other data if some inserts fail. Use autocommit to prevent data loss on retries.
|
2017-09-15 18:15:03 +01:00 |
Michael Herzberg
|
c57bce2491
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-15 17:42:05 +01:00 |
Achim D. Brucker
|
936f2d3189
|
Log git info before starting pull (update).
|
2017-09-14 22:54:37 +01:00 |
Achim D. Brucker
|
2ff30f7382
|
Parallel execution of git date queries.
|
2017-09-14 15:11:53 +01:00 |
Achim D. Brucker
|
12a1e282aa
|
The method pull_get_updated_lib_files(...) now also returns unique library/version information.
|
2017-09-14 10:44:30 +01:00 |
Achim D. Brucker
|
e3f1202e44
|
Use version dictionary.
|
2017-09-14 10:33:00 +01:00 |
Achim D. Brucker
|
f54f29c9ba
|
Added build_release_date_dic(...).
|
2017-09-14 09:50:09 +01:00 |
Achim D. Brucker
|
3b217922c5
|
Added line count.
|
2017-09-13 16:41:01 +01:00 |
Achim D. Brucker
|
420eec7462
|
Minor memory optimizations.
|
2017-09-13 11:12:33 +01:00 |