 Achim D. Brucker
|
247b96db6d
|
Refactoring: moved core functionality in own module.
|
2017-09-02 18:47:41 +01:00 |
|
Achim D. Brucker
|
7bcf9aca8e
|
Removed executable flag.
|
2017-09-02 18:08:20 +01:00 |
|
Achim D. Brucker
|
99028c3763
|
Removed executable flag.
|
2017-09-02 18:08:06 +01:00 |
|
Achim D. Brucker
|
6b3ef921ff
|
Reformatting and minor refactoring.
|
2017-09-02 18:00:24 +01:00 |
 Michael Herzberg
|
45496a0d5d
|
Log parameters.
|
2017-09-02 17:52:51 +01:00 |
|
Michael Herzberg
|
d7dcfdbcbd
|
Use $* instead of $@.
|
2017-09-02 17:50:16 +01:00 |
|
Michael Herzberg
|
54475b97a8
|
Added arg option to sge script.
|
2017-09-02 17:45:12 +01:00 |
|
Achim D. Brucker
|
7e07c6d734
|
Initital commit: tool for crawling cdnjs.com.
|
2017-09-02 17:42:46 +01:00 |
|
Michael Herzberg
|
c94f23dcee
|
Added --from-date option for create-db.
|
2017-09-02 17:42:18 +01:00 |
|
Michael Herzberg
|
c33e8204ea
|
Cleaned up create-db sge script a bit.
|
2017-09-02 17:05:42 +01:00 |
|
Michael Herzberg
|
1647aac086
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-02 16:46:04 +01:00 |
|
Michael Herzberg
|
08fced735c
|
Fixed queries.
|
2017-09-02 16:45:38 +01:00 |
|
Michael Herzberg
|
f94f6140b7
|
Added same query for content script urls.
|
2017-09-02 14:27:01 +01:00 |
|
Michael Herzberg
|
1137421548
|
Added download numbers to query.
|
2017-09-02 14:16:11 +01:00 |
|
Achim D. Brucker
|
8af3c99d26
|
Changed pip to pip3.
|
2017-09-02 00:07:50 +01:00 |
|
Achim D. Brucker
|
9ed8f5f926
|
Improved reporting.
|
2017-09-02 00:05:07 +01:00 |
|
Achim D. Brucker
|
8faba1d00f
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-01 23:45:51 +01:00 |
|
Achim D. Brucker
|
a69c173064
|
Activated preliminary check of regexps for specific libs.
|
2017-09-01 23:41:45 +01:00 |
|
Achim D. Brucker
|
28f6aa5f45
|
Bug fix: indentation
|
2017-09-01 23:24:55 +01:00 |
|
Achim D. Brucker
|
5c987833a4
|
Bug fix: NoneType object is not iterable.
|
2017-09-01 23:23:11 +01:00 |
|
Achim D. Brucker
|
faaa458921
|
Updated regexps.
|
2017-09-01 22:26:30 +01:00 |
|
Michael Herzberg
|
f125f683ec
|
Fixed query.
|
2017-09-01 20:10:50 +01:00 |
|
Michael Herzberg
|
f20da9490f
|
Updated README and setup.py from my experiments.
|
2017-09-01 20:01:53 +01:00 |
|
Michael Herzberg
|
4980bdbe9e
|
Updated query for MySQL.
|
2017-09-01 20:00:28 +01:00 |
|
Michael Herzberg
|
bb03a67a29
|
Deleted ropeproject stuff.
|
2017-09-01 17:04:25 +01:00 |
|
Achim D. Brucker
|
2693fb0fcd
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-01 16:28:18 +01:00 |
|
Achim D. Brucker
|
3fb0d740c0
|
Bug fix: exception due to reading from the wrong dictionary.
|
2017-09-01 16:27:44 +01:00 |
|
Michael Herzberg
|
ab943c87f0
|
Expand user directory for mysql config file.
|
2017-09-01 16:17:51 +01:00 |
|
Michael Herzberg
|
abd9605ebc
|
Use python3.5 for all files.
|
2017-09-01 14:12:05 +01:00 |
|
Michael Herzberg
|
cbcb3bc3b0
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-01 13:30:57 +01:00 |
|
Michael Herzberg
|
5c24608c4d
|
Added --max-discover <N> option to limit the number of new extensions.
|
2017-09-01 13:30:42 +01:00 |
|
Achim D. Brucker
|
258269abb6
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-01 12:14:53 +01:00 |
|
Achim D. Brucker
|
8289d50d38
|
Download all extensions in parallel and later do a second download for a subset including forums/reviews.
|
2017-09-01 12:14:39 +01:00 |
|
Michael Herzberg
|
b5fd382ab8
|
Use utf8mb4 for mysql connections.
|
2017-09-01 12:11:37 +01:00 |
|
Achim D. Brucker
|
53f080ba36
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-01 12:02:17 +01:00 |
|
Achim D. Brucker
|
22264fb9e0
|
Changed download order: first download parallel all extensions without forum/review download, then download extensions with forums.
|
2017-09-01 12:02:12 +01:00 |
|
Michael Herzberg
|
62c353f647
|
Removed crawling restriction to 10 extids.
|
2017-09-01 10:42:21 +01:00 |
|
Michael Herzberg
|
21a7741f0c
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-09-01 08:15:36 +01:00 |
|
Michael Herzberg
|
05ffdc6e24
|
Added explicit utf-8 request to mysql connector.
|
2017-09-01 08:15:22 +01:00 |
|
Achim D. Brucker
|
9446c20d01
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-08-31 23:45:00 +01:00 |
|
Achim D. Brucker
|
883e7ddcd8
|
Report details of matches.
|
2017-08-31 23:44:51 +01:00 |
|
Michael Herzberg
|
e06d3f4ac4
|
Reduced timeout and fixed logging.
|
2017-08-31 23:01:05 +01:00 |
|
Achim D. Brucker
|
b63791bdb4
|
Reformatting.
|
2017-08-31 21:50:12 +01:00 |
|
Achim D. Brucker
|
e0db2a5f47
|
Added detection details.
|
2017-08-31 08:43:19 +01:00 |
|
Achim D. Brucker
|
2831362584
|
JSON prettify.
|
2017-08-31 08:27:53 +01:00 |
|
Achim D. Brucker
|
d52b2bab13
|
Sorted jQuery entries wrt version number.
|
2017-08-31 08:22:24 +01:00 |
|
Achim D. Brucker
|
2437606ab2
|
Fixed markdown.
|
2017-08-30 23:25:49 +01:00 |
|
Achim D. Brucker
|
47b2cb6d9a
|
Fixed merge conflict.
|
2017-08-30 23:22:09 +01:00 |
|
Achim D. Brucker
|
168ec3bf36
|
Merge branch 'master' of logicalhacking.com:BrowserSecurity/ExtensionCrawler
|
2017-08-30 22:57:47 +01:00 |
|
Achim D. Brucker
|
bd944c0b9d
|
jQuery 3.x information added.
|
2017-08-30 22:57:41 +01:00 |