Added instructions for cloning the repository.

.gitignore

@@ -7,6 +7,9 @@
 # Ignore bundler config.
 /.bundle
 
+# ignore generated doc
+doc/rdocs
+
 # Ignore the default SQLite database.
 /db/*.sqlite3
 /db/*.sqlite3-journal

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "doc/solutions"]
+	path = doc/solutions
+	url = git@git.logicalhacking.com:BrowserSecurity/DVGM-solutions.git

README.md

@@ -8,7 +8,7 @@ started with automatic security testing tools.
 
 ## Known Vulnerabilities
 
-VGM contains (at least) the following vulnerabilities:
+DVGM contains (at least) the following vulnerabilities:
 
 * SQL Injection
 * Cross-Site Scripting (XSS)
@@ -55,6 +55,21 @@ You are Peter, a student and you can log in with `peter` as username and
 
 * Ruby 2.5 and [bundler](https://github.com/bundler/bundler)
 
+### Checkout
+
+The repository can be cloned as usual:
+
+``` sh
+git clone https://git.logicalhacking.com/BrowserSecurity/DVGM.git
+```
+
+Note, if you authorized to access the confidential solutions of the
+the exercises for DVGM, you can obtain them by executing
+
+``` sh
+git submodule update --init --recursive
+```
+
 ### Installation
 
 After cloning the repository, install the dependencies; `bundle` will install
@@ -68,7 +83,7 @@ bundle install --path vendor/bundle
 ### Starting the server
 
 To make exploration of the app a bit easier, we run DVGM in development mode.
-This means that 
+This means that
 
 * on errors, rails will return a detailed debug page, and
 * changed source files will automatically be picked up, without needing to

doc/README.md

@@ -0,0 +1,24 @@
+# DVGM -- Usage and Security Analysis
+
+## Introduction / Prerequisites
+
+This exercise sheet is meant to be followed on a recent GNU/Linux installation
+and makes use of the terminal. While all necessary commands are provided, a
+basic understanding if its usage is still required.
+
+In the following, we will use the Damn Vulnerable Grade Management (DVGM) app as
+a training target. Before continuing, please familiarize yourself with the app
+and ensure that it is listening on `http://$(hostname):3000`, where
+`$(hostname)` is the host name of your machine as returned by the `hostname`
+command. This is important because some scanners have problems when scanning
+loopback addresses such as `localhost` and `127.0.0.1`.
+
+If you need to fresh-up your Ruby knowledge, our small [Ruby Primer](ruby-primer.md)
+might be a helpful companion.
+
+## Questions / Challenges
+
+The folder [exercises](exercises/) contains several exercises that illustrate both manual
+exploration of DVGM and the use of tools such as [Brakeman](https://brakemanscanner.org/),
+[Arachni](http://www.arachni-scanner.com/), and [OWASP ZAP](https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project)
+for finding various security vulnerabilities in DVGM.

doc/exercises/01-static-analysis-with-breakman.md

@@ -0,0 +1,49 @@
+# Static Analysis: Brakeman
+
+Brakeman is a static source code analyser for Ruby on Rails applications and can
+be used at any stage of development, as it does not rely on any code being
+executed. It finds many different types of security vulnerabilities. In the
+following exercises, you will run Brakeman on DVGM and inspect three findings
+more closely.
+
+## Running Brakeman
+
+Running Brakeman is simple. Navigate to the source code directory of DVGM and
+call `brakeman` without any arguments.
+
+Brakeman should return withing a few seconds and produce 7 security warnings. We
+will have a closer look at the SQL Injection and Cross-Site Scripting findings.
+
+### SQL Injection
+
+Brakeman will report one possible SQL injection.
+
+1. In which file and line is the possible SQL Injection located?
+2. What action in what part of the app triggers the SQL query?
+3. Is the vulnerability exploitable? If yes, write an exploit and test it.
+4. If it is exploitable, how would a possible fix look like? Try the fix by
+   changing the source code of DVGM (the changes are automatically picked up).
+   See if your exploit still works. Do not forget to revert all changes afterwards,
+   as we will also use other tools.
+
+### Cross-Site Scripting (XSS)
+
+Brakeman will report two possible cross-site scripting vulnerabilities *in DVGM
+itself*. We will look more closely at the one that possibly affects logged-in
+lecturers.
+
+5. In which file and line is the possible XSS vulnerability located?
+6. What action in what part of the app triggers the flagged line?
+7. Is the vulnerability exploitable? If yes, write an exploit and test it.
+8. If it is exploitable, how would a possible fix look like? Try the fix by
+   changing the source code of DVGM (the changes are automatically picked up).
+   See if your exploit still works. Do not forget to revert all changes afterwards,
+   as we will also use other tools.
+
+### Vulnerable Dependencies
+
+Brakeman will also report (at least) two possible Cross-Site Scripting
+vulnerabilities in dependencies.
+
+ 9. Which dependencies are affected?
+10. Is DVGM likely to be affected by the reported CVEs?

doc/exercises/02-dynamic-analysis-with-arachni.md

@@ -0,0 +1,111 @@
+# Dynamic Analysis: Arachni
+
+Arachni is a web application security scanner framework that supports many
+different kinds of security checks, including detection of insecure header,
+cross-site scripting (XSS), SQL injection, and cross-site request forgery
+(CSRF). We will run the tool twice; first on the login page, then we will
+provide the login credentials for a student to the tool -- otherwise, Arachni
+would not see much beside the login screen, similar to an unauthenticated user.
+
+## Running Arachni without credentials
+
+In order to run Arachni, start it with the following options:
+
+```bash
+arachni --report-save-path=/tmp/sign_in_page.report \
+  --check=unencrypted_password_forms,x_frame_options \
+  --scope-exclude-pattern='/assets/|__web_console' \
+  http://$(hostname):3000
+```
+
+To speed up the analysis, we only use the `unencrypted_password_forms` and
+`x_frame_options` check, because this is the only one that will yield a result
+at this point. If we would run Arachni with all checks enabled, it would take
+considerably longer. In addition, by using `--scope-exclude-pattern`, we tell
+Arachni to skip assets and the debug console of Ruby on Rails, which is active
+since we run the app in development mode. Otherwise, Arachni will loop for a
+long time trying to navigate the console.
+
+Once the analysis is finished, inspect the report:
+```bash
+arachni_reporter \
+  --reporter=html:outfile=/tmp/sign_in_page.html.zip \
+  /tmp/sign_in_page.report 
+unzip -d /tmp/sign_in_page /tmp/sign_in_page.html.zip
+chromium /tmp/sign_in_page/index.html
+```
+
+Arachni should have found insecure headers and a possibly unsecured login page.
+Remember that Arachni was acting as an unauthenticated user, so it is not
+surprising that it found considerably less than brakeman.
+
+### Insecure HTTP Headers
+
+1. What is the purpose of the missing headers?
+2. How could the lack of these headers be used to attack DVGM?
+3. Ruby aims to have secure defaults. Can you find which part of DVGM's sorce
+  is responsible for the lack of said headers?
+4. We have now used a static analysis tool, Brakeman, and started to
+  use a dynamic tool, Arachni. What are some benefits and limitations
+  of these different kind of tools?
+
+## Running Arachni with credentials
+
+For the second run of Arachni, you will provide the login credentials for our
+student account, `peter`. This will allow Arachni to scan more parts of DVGM.
+
+```bash
+arachni --report-save-path=/tmp/logged_in.report \
+  --plugin=autologin:url="http://$(hostname):3000/sign_in",\
+    parameters="user_session[login]=peter&user_session[password]=football",\
+    check="| Logout" \
+  --scope-exclude-pattern="sign_out|__web_console|/new" \
+  --check=code_injection,csrf,'xss*',hsts,\
+    unencrypted_password_forms,allowed_methods,x_frame_options \
+  http://$(hostname):3000
+```
+
+Again, the scan might take a few minutes. This time, we use the `autologin`
+plugin, which takes three parameters: `url`, the URL to the sign-in page;
+`parameters`, which contains the POST parameters that a browser sends when we
+type in username and password and submit the form; and `check`, which contains
+text that Arachni uses to check whether the login succeeded (only when we are
+logged in, we will find the text `| Logout` in the top bar).  Additionally, we
+use more checks this time, because Arachni now has access to larger parts of our
+app. We also need to make sure to exclude URLs from the scan that includes
+`sign_out|`  (we do not want Arachni to log out automatically) or `/new|`'
+(we do not want Arachni to end up in an infinite loop creating new grade
+comments).
+
+Once the scan finishes, inspect the report:
+
+```bash
+arachni_reporter \
+  --reporter=html:outfile=/tmp/logged_in.html.zip \
+  /tmp/logged_in.report
+unzip -d /tmp/logged_in /tmp/logged_in.html.zip
+chromium /tmp/logged_in/index.html
+```
+
+Arachni should have found XSS and CSRF vulnerabilites.
+
+5. We ran Arachni with the `sql_injection` plugin. Recall the SQL exploit that
+  you found earlier. What could be a reason for Arachni not finding the SQL
+  injection vulnerability?
+
+### DOM XSS / Client-Side XSS
+
+6. Read through the two types of reported XSS issues. In
+    particular, click on the eye button to inspect the injected seed
+    along with the presented proof. Do you think the found issues
+    are true positives? How does the automatic verification of the
+    found XSS vulnerabilities work? How is the found XSS vulnerability different
+    from the found DOM XSS vulnerability?
+
+### Cross-Site Request Forgery (CSRF)
+
+7. Inspect the found cross-site request forgery attack. The Ruby on Rails
+  [Security Guide](http://guides.rubyonrails.org/security.html#cross-site-request-forgery-csrf)
+  contains a good introduction to this kind of vulnerability. How can this kind of
+  vulnerability be problematic found an app like DVGM? Compare the finding with
+  the CSRF warning from Brakeman.

doc/exercises/03-dynamic-analysis-with-zap.md

@@ -0,0 +1,204 @@
+# (Interactive) Dynamic Analysis: ZAP
+
+In the following, we will use the OWASP Zed Attack Proxy (ZAP) for analyzing our
+web application. First, we will use ZAP interactively, meaning that we will
+manually use DVGM while all traffic is being routed through ZAP's proxy and
+analyzed. Then, we will use the automatic features: *Spider* for discovering all
+URLs in the app, *Active Scan* to get a fully automatic scan similar to
+Arachni's, and the *Fuzzer* as a more manual, but also more powerful tool.
+Finally, we will see how ZAP can be scripted using Python for a completely
+automatic workflow for, e.g., continuous integration (CI) environments.
+
+## Using ZAP as an interactive tool
+
+Oftentimes, the Zed Attack Proxy is used passively with an automated test suite
+to discover additional problems, but the proxy can also be used when browsing
+the app interactively.
+
+To begin, press the windows key, search for `ZAP`, and start the GUI of ZAP.
+  Then, select *No, I do not want to persist this session at this moment in time*.
+
+Now, in the top toolbar, click on the Chromium symbol (the right-most icon) to
+start a built-in version of Chromium that is already configured to use the
+proxy. Once the browser is started, enter `http://127.0.0.1:3000/` and press
+enter (you can also use your hostname again, but ZAP does not require it).
+
+Your previous attempts at hacking DVGM have been successful; you have been able
+to obtain the login credentials of one of the lecturers (username: `achim`,
+password: `sigmisinsecure`). Use them to log into the app, and browse around.
+You will notice that you have more permissions now.
+
+### Insecure Headers
+
+You should notice that ZAP has been logging all visited URLs so far. Most of
+them should have been tagged with an alert level of *Low* or *Medium*. Change
+the current tab from *History* to *Alerts* to inspect the alerts in more detail.
+Most of them should be related to insecure HTTP headers.
+
+### Missing server-side validation
+
+We will now tell ZAP to enable disabled form fields and also show form fields
+that have been set to hidden. To do so, click the white light-bulb symbol in the
+top toolbar.
+
+Now, in the browser tab showing DVGM, go to the *Students* tab and try to create a
+new student. You will notice that the form looks different now.
+
+1. Can you create a new lecturer or even a new admin user? Try and see if you
+  can log in with the new user.
+2. What is the vulnerability here? Does it need to be fixed on the client or
+  server side? Try and fix the vulnerability and test your fix.
+
+Before you continue, make sure that you log out of DVGM. We will not continue with the lecturer account; a lecturer can create multiple kinds of resources in DVGM (grades, students), which leads the automated scanner not necessarily into infinite loops, but it will increase the scan time significantly. Do deal with these kinds of issues, one could limit the depth of the scanning further or even exclude these URLs.
+
+## Using ZAP as an automated tool
+
+We will now use the more automated functions of ZAP. Unlike previously, we will
+directly use them with login credentials. ZAP offers to
+automatically *steal* the authentication cookie of your interactive session and
+use it for the automated functions.
+
+To do so, click on the green plus next to the *Output* tab and open *HTTP
+Sessions*. The table will most likely be empty, as ZAP by default checks all
+cookies that come through the proxy for a small set of entries that resemble
+sessions, which does not contain the one used by Ruby on Rails.
+
+To add the cookie name to this set, click on the wheel in the top right corner
+of the *HTTP Sessions* tab. Click on *Add...*, and add `user_credentials`.
+Confirm the settings dialog with *OK*.
+
+Now, log in as a *student*, using the credentials from *peter* (password:
+*football*). You will notice that ZAP detected a new session (*Session 0*).
+Right-click the session and set it to active. This will make sure that ZAP uses
+the session for the scans.
+
+### Spider
+
+We will now use *Spider* to discover all URLs of our app, which is a
+prerequisite for the automated scanner. In the top left window, right-click on
+the URL and choose *Attack -> Spider...*. Click on *Start Scan* and observe the
+output. Only URLs that are below the entry URL are scanned; all others are
+skipped.
+
+Before we start the scan, we want to exclude the *assets* and *sign_out* pages
+again. *In the top left window*, right-click both entries and choose *Exclude
+From -> Scanner*, and confirm with *OK*.
+
+### Active Scan
+
+To start the active scan, right-click on the URL in the top left window again
+and choose *Active Scan...*. This will take a few minutes. Afterwards, in the
+alert tab, you will see that ZAP now also found the two XSS vulnerabilities.
+
+These alerts can also be exported into various formats. Do to so, click on
+*Report* in the top bar. If you want to, have a look at a few generated reports.
+
+### Fuzzer
+
+We will now try to use ZAP's fuzzer to find the SQL Injection and XSS
+vulnerabilities in the lecturer search bar.
+
+To do so, select the *History* tab in ZAP. In DVGM, navigate to the *Grades* tab
+and search for something that is easily recognizable, e.g., `FOOBAR`. After
+pressing *Filter*, click on the new entry in ZAP's history tab.
+
+In the top right part of the window, select *Request* to show the request that
+you have just sent to the server. In the first line, select your search request,
+e.g., `FOOBAR`, and right-click on the selection. Now, select *Fuzz...*, click
+on *Payloads...* on the right, click *Add...*, select *File Fuzzers*, open the
+*jprofuzz* sub-menu, and select *SQL Injection* and *XSS*. Confirm all dialog
+boxes and click *Start Fuzzer*.
+
+ZAP will now take the original request and replace `FOOBAR` with a number of
+different payloads and are typically used for SQL Injections and XSS attacks.
+Note that this mode is less automated than the *Active Scan*, so we will not get
+alerts and a report from ZAP.
+
+Instead, we will look for two things:
+
+- *Reflected* in the *State* column means that ZAP, after having sent a specific
+  payload for the *lecturer* parameter, found this particular token again in the
+  HTML response from the server, which is an indicator that there *might* be a XSS
+  vulnerability here.
+- The HTTP status code in the *Code* and *Reason* columns can give a hint
+  whether there is a SQL Injection in the app. A response code of 5xx indicates
+  an internal server error, which might be due to a malformed SQL query as a
+  result of the injection. This indicates that one can tamper with the structure
+  of a SQL query, which often allows to arbitrary modification.
+
+Before you proceed, close ZAP.
+
+## Scripting ZAP
+
+ZAP can also be controlled with a powerful python API. Many tasks that we have
+just seen can therefore be automated and used as part of an automated testing
+strategy. The following example shows how the *Spider* and *Active Scan* can be
+started from python (without a user sesssion).
+
+First, install the ZAP API:
+
+```bash
+pip3 install python-owasp-zap-v2.4
+```
+
+Then, start the ZAP in headless mode without a GUI:
+
+```bash
+/usr/share/zaproxy/zap.sh -daemon \
+  -config api.key="dvgmisinsecure" \
+  -port 8080
+```
+
+Finally, save the following python script somewhere and execute it:
+
+```python
+#!/usr/bin/env python
+import time
+from pprint import pprint
+from zapv2 import ZAPv2
+
+target = 'http://127.0.0.1:3000'
+apikey = 'dvgmisinsecure'
+
+# By default ZAP API client will connect to port 8080
+zap = ZAPv2(apikey=apikey)
+
+# Proxy a request to the target so that ZAP has something to deal with
+print('Accessing target {}'.format(target))
+zap.urlopen(target)
+time.sleep(2) # Give the sites tree a chance to get updated
+
+print('Spidering target {}'.format(target))
+scanid = zap.spider.scan(target)
+
+time.sleep(2) # Give the Spider a chance to start
+while (int(zap.spider.status(scanid)) < 100):
+    # Loop until the spider has finished
+    print('Spider progress %: {}'.format(zap.spider.status(scanid)))
+    time.sleep(2)
+
+print ('Spider completed')
+
+while (int(zap.pscan.records_to_scan) > 0):
+      print ('Records to passive scan : {}'.format(zap.pscan.records_to_scan))
+      time.sleep(2)
+
+print ('Passive Scan completed')
+
+print ('Active Scanning target {}'.format(target))
+scanid = zap.ascan.scan(target)
+while (int(zap.ascan.status(scanid)) < 100):
+    # Loop until the scanner has finished
+    print ('Scan progress %: {}'.format(zap.ascan.status(scanid)))
+    time.sleep(5)
+
+print ('Active Scan completed')
+
+# Report the results
+print ('Hosts: {}'.format(', '.join(zap.core.hosts)))
+print ('Alerts: ')
+pprint (zap.core.alerts())
+```
+
+This script is mostly taken from <https://github.com/zaproxy/zap-api-python/>,
+where also a more sophisticated script can be found.

doc/ruby-primer.md

@@ -0,0 +1,67 @@
+# DVGM -- Tips & Tricks
+
+## Helpful Ruby (on Rails) Resources
+
+- The official [Getting Started with Ruby on Rails guide](https://guides.rubyonrails.org/getting_started.html)
+- The Ruby on Rails [Security Guide](http://guides.rubyonrails.org/security.html#cross-site-request-forgery-csrf)
+- The [Ruby API search](https://apidock.com/rails/search)
+
+## Structure of the DVGM Ruby on Rails Project
+
+DVGM, like all Ruby on Rails projects, uses the Model-View-Controller (MVC)
+paradigm, meaning it is decoupled into a view-part (handles the presentation to
+the user), a controller-part (handles the business logic of the application),
+and a model-part (handles the representation of the involved data and its
+storage).
+
+![Diagram of interactions within the MVC pattern.](https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller#/media/File:MVC-Process.svg)
+
+The most important folders in the DVGM project are:
+
+Path                    | Description
+----------------------- | ---
+`app`                     | The main directory of the app
+`app/assets/javascripts`  | This folder contains the CoffeeScript files that get compiled to JavaScript and sent with every page of the app.
+`app/controllers`         | The app controllers that contain the business logic of the app.
+`app/models`              | The model of the app. Ruby on Rails also automatically generates database tables from these files.
+`app/views`               | The HTML template files. Some views exist in multiple versions, one for each role (student/lecturer/admin) in the app.
+`db/development.sqlite3`  | The database that is used by DVGM. If you want to reset the state of the app, restore this file from the repository.
+
+## Working with DVGM
+
+The app can be started by using `bin/rails server`. This will use the development
+mode, which uses a local SQLite3 as a database backend and will automatically
+pick-up any changes you make to the source code, so there is no need to restart
+the server.
+
+If you want to reset the database (maybe because you found a SQL Injection and
+dropped all tables ;)), you can restore `db/development.sqlite3` from the app source
+code repository.
+
+## Lightning-Quick Introduction to SQLite3 and SQL
+
+SQLite3 is a file-base relational database. This means that all data is stored in
+a single file, organized into rows and columns, where one data entry is represented
+by one row. For example, one student grade in DVGM is stored like this:
+
+```csv
+id|lecture_id|student_id|grade|comment|created_at|updated_at
+6|7|6|55|I guess I should have studied more...|2017-04-03 09:15:57.698459|2017-04-03 09:17:05.000330
+```
+
+This table only does not contain the name of the student, but only a reference
+to them, as they are stored in a different table. Using SQL, we can look up the
+student and lecture names, and only display the ones for for students whose names
+start with a *P*:
+
+```SQL
+SELECT login,name,grade,comment FROM users JOIN grades JOIN lectures ON users.id=student_id AND lectures.id=lecture_id WHERE login LIKE 'P% ';
+```
+
+The output then will look like this:
+
+```csv
+login|name|grade|comment
+Peter|Security|55|I guess I should have studied more...
+Peter|Security|80|Doing the homework paid off!
+```

doc/solutions

@@ -0,0 +1 @@
+Subproject commit c6ce58d6e3d306c210de2843c8f080e86a9c2187

lib/tasks/rdoc.rake

@@ -0,0 +1,16 @@
+require 'rdoc/task'
+     
+desc 'generate API documentation to doc/rdocs/index.html'
+     
+Rake::RDocTask.new do |rd|
+  rd.rdoc_dir = 'doc/rdocs'
+  rd.main = 'README.md'
+  rd.rdoc_files.include 'README.md', "config/**/*\.rb", "helpers/**/*\.rb", "script/**/*\.rb" 
+     
+  rd.options << '--inline-source'
+  rd.options << '--line-numbers'
+  rd.options << '--all'
+  rd.options << '--fileboxes'
+  rd.options << '--diagram'
+end
+