Commit Graph

24 Commits

Author SHA1 Message Date
Michael Herzberg eb616b0ac3 Fix some encoding issues. 2018-07-21 01:50:59 +01:00
Michael Herzberg 3bef0afe7a Group mysql inserts and don't compress them. 2018-07-15 00:08:11 +01:00
Michael Herzberg 630fcba1df Actually write simhash into db... 2018-06-10 01:22:12 +01:00
Michael Herzberg d8d49b1b80 Moved ext_id into logger formatter to make logger output more uniform. 2018-04-21 19:59:02 +01:00
Michael Herzberg a789fe505f Fixed style errors and warnings. 2018-04-21 19:00:07 +01:00
Michael Herzberg a758134c97 Readded mimetype from mimetypes. TODO: add mysql columns 2018-04-11 16:52:22 +01:00
Michael Herzberg f4c0ff56ff Use magic for mimetypes and don't attempt text-based analyses on binary resources. 2018-04-09 14:25:47 +01:00
Michael Herzberg f51bcfbf46 Use con object from db.py. 2017-10-12 16:01:45 +01:00
Michael Herzberg 63ae8ac4a7 Added missing fields for cdnjs and introduced new crxfile and libdet tables. 2017-10-10 18:55:28 +01:00
Michael Herzberg 301ad23d4c Use new review etc. table structure. 2017-10-09 17:18:01 +01:00
Michael Herzberg 6ba73c2ed9 Changed autocommit behaviour. 2017-10-04 20:56:47 +01:00
Michael Herzberg 4712e15249 Fixed autocommit bug. 2017-09-19 17:09:35 +01:00
Achim D. Brucker 50b91d3a35 Renaming jsFilename -> filename. 2017-09-18 00:30:55 +01:00
Michael Herzberg 0cb7d6e792 Fixed error in exception handling. 2017-09-17 17:40:48 +01:00
Michael Herzberg 69e95fdf13 Catch json parse extensions for reviews etc. more nicely. 2017-09-16 12:53:35 +01:00
Michael Herzberg a514c0001e Added check for empty crx files. 2017-09-16 12:14:41 +01:00
Michael Herzberg b51de8577f Added compression for mysql. 2017-09-16 12:04:35 +01:00
Michael Herzberg ddbbc2672d Try to insert also other data if some inserts fail. Use autocommit to prevent data loss on retries. 2017-09-15 18:15:03 +01:00
Michael Herzberg 36d36facfe Relaxed mysql retries. 2017-09-04 15:54:28 +01:00
Michael Herzberg b5fd382ab8 Use utf8mb4 for mysql connections. 2017-09-01 12:11:37 +01:00
Michael Herzberg 05ffdc6e24 Added explicit utf-8 request to mysql connector. 2017-09-01 08:15:22 +01:00
Achim D. Brucker e0db2a5f47 Added detection details. 2017-08-31 08:43:19 +01:00
Michael Herzberg bde59c5040 Fixed crx_etag select bug and some logging. 2017-08-30 16:32:36 +01:00
Michael Herzberg cbd2dea820 Removed everything related to sqlite and updated README. 2017-08-30 15:38:04 +01:00