Commit Graph

381 Commits

Author SHA1 Message Date
Michael Herzberg c61f19e860 Use INSERT OR IGNORE. 2017-07-31 23:55:21 +01:00
Michael Herzberg b8f57196c7 Changed fts table structure. 2017-07-31 23:23:57 +01:00
Michael Herzberg b34d45c4dc Added md5sum to sqlite. 2017-07-31 20:38:21 +01:00
Michael Herzberg 9f6cfc22a3 Added create-db sge wrapper. 2017-07-31 20:30:35 +01:00
Michael Herzberg 82a41aef18 Added python wrapper for grepper sge stuff. 2017-07-31 20:18:34 +01:00
Michael Herzberg e1aee1446a Fixed grepper. 2017-07-31 19:11:39 +01:00
Michael Herzberg eac88001c3 Use empty string instead of None. 2017-07-31 16:19:54 +01:00
Michael Herzberg 4cc8a2695e Fixed import. 2017-07-31 16:19:20 +01:00
Michael Herzberg 4d0b2b5ec5 Fixed sge_common and merge script. 2017-07-31 14:21:57 +01:00
Michael Herzberg e94bb344d3 Refactored grepper. 2017-07-31 14:19:52 +01:00
Achim D. Brucker 35c133e395 Slightly more aggressive implementation of google_dos_protection. 2017-07-30 14:41:20 +01:00
Achim D. Brucker eace911472 Added description of extract-crx. 2017-07-30 09:59:42 +01:00
Achim D. Brucker 5268f2a732 Refactoring: clean-up of imports and a few other minor improvements. 2017-07-29 16:13:39 +01:00
Achim D. Brucker eb0054b47d Refactoring: Moved default configuration to config module. 2017-07-29 12:36:20 +01:00
Achim D. Brucker 0b24fb15fe Refactoring. 2017-07-29 11:32:06 +01:00
Achim D. Brucker 0ca3476b09 Slightly more aggressive implementation of google_dos_protection. 2017-07-29 11:21:15 +01:00
Achim D. Brucker d05ca9678e Refactoring. 2017-07-29 10:57:35 +01:00
Achim D. Brucker ac663299b3 Refactoring. 2017-07-29 10:17:16 +01:00
Achim D. Brucker b68eb42f96 Refactoring. 2017-07-29 09:46:28 +01:00
Achim D. Brucker 10cce2859d Renamed variable/attribute pk to public_key. 2017-07-29 09:15:22 +01:00
Achim D. Brucker 333bcaa62d Strip path from crx file. 2017-07-29 09:12:01 +01:00
Achim D. Brucker e5d671c7c4 Refactoring. 2017-07-29 09:05:16 +01:00
Achim D. Brucker 659f37c90c Refactoring. 2017-07-28 21:18:10 +01:00
Achim D. Brucker d5d2251de9 Refactoring. 2017-07-28 20:21:38 +01:00
Achim D. Brucker 716e35be00 Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-07-28 19:45:06 +01:00
Achim D. Brucker 22f18ec3c8 Refactoring. 2017-07-28 19:44:51 +01:00
Michael Herzberg fb64499c8f Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-07-28 16:47:33 +01:00
Michael Herzberg 44756e44d6 Added python-based sqlite merger. 2017-07-28 16:47:25 +01:00
Achim D. Brucker 1833dbad90 Refactoring. 2017-07-28 15:57:17 +01:00
Achim D. Brucker 5658facebd Set unused variables to None to help garbage collector. 2017-07-28 15:32:19 +01:00
Michael Herzberg 11604c0fa5 Collect jsfilesize instead of jsloc. 2017-07-26 12:05:54 +01:00
Achim D. Brucker 349adb4b39 Updated README.md to reflect new requirements.txt. 2017-07-26 07:41:37 +01:00
Achim D. Brucker 9c8cedad74 Add hoc generated requirements.txt. 2017-07-26 07:38:17 +01:00
Achim D. Brucker 73eedab07d Log time delta for each extension upate. 2017-07-26 07:29:48 +01:00
Michael Herzberg b3d1ab912e Wait a maximum of 10min before stopping jsbeautifier. 2017-07-25 22:57:55 +01:00
Achim D. Brucker 04d242ce7e Added sqlite3 version to config log. 2017-07-24 11:10:16 +01:00
Achim D. Brucker 9b291d02ae Added local bin/lib-directory to PATH. 2017-07-24 10:47:42 +01:00
Achim D. Brucker d6d1d21d05 Bug fix: Append log to log instead of overwritting it. 2017-07-24 10:26:03 +01:00
Achim D. Brucker 2bf32b7a8f Merge branch 'production' of logicalhacking.com:BrowserSecurity/ExtensionCrawler into production 2017-07-22 21:14:07 +01:00
Achim D. Brucker 052748af07 Create log file for profiling. 2017-07-20 07:51:30 +01:00
Michael Herzberg 072e008fe2 Run the garbage collector manually after using jsbeautify. 2017-07-19 17:25:21 +01:00
Achim D. Brucker c73f4cda3e Fixed log output. 2017-07-19 07:01:38 +01:00
Michael Herzberg cf0478d740 Added sge file for merging the individual dbs into the individual dbs of our archive. 2017-07-18 23:51:11 +01:00
Michael Herzberg 186f6162af Fixed NoneType str conversion exception. 2017-07-17 15:29:00 +01:00
Michael Herzberg 9b1e5db96f Check for attributes key first and use traceback module instead of printing str(e). 2017-07-17 14:00:39 +01:00
Michael Herzberg 50123f14b8 Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-07-16 20:14:59 +01:00
Michael Herzberg eded1ca893 Only attempt to search for replies when we actually have search parameters. 2017-07-16 20:14:50 +01:00
Achim D. Brucker 3346b1b61a Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler 2017-07-16 14:02:40 +01:00
Achim D. Brucker 049db03021 Add user local to binary and library search paths. 2017-07-16 14:02:22 +01:00
Michael Herzberg 5e60715d98 Removed schema creation from scripts as it is now being done by the merge script. 2017-07-13 14:47:42 +01:00