Reduce Rack Attack false positives causing 403 errors during HTTP authentication
### What does this MR do?
This MR reduces false positives causing `403 Forbidden` messages after HTTP authentication.
A Git client may attempt to access a repository without a password. If it receives a 401 error, the client often will try again, this time supplying a password. The problem is that `grack_auth.rb` considers a blank password an authentication failure and increases a Redis counter each time this happens. With enough requests, an IP can be banned temporarily even though previous attempts may have been successful. This leads users to see `403 Forbidden` errors until the ban times out (default: 1 hour).
To reduce the chance of a false positive, this MR resets the counter upon a successful authentication from an IP.
In addition, this MR logs when a user has been banned and introduces the ability to disable Rack Attack via a config variable.
### Are there points in the code the reviewer needs to double check?
rack-attack v4.2.0 doesn't support the ability to clear counters out of the box, so `rack_attack_helpers.rb` includes a number of monkey patches to make it work. It looks like this functionality may be added in v4.3.0. I've also sent pull requests to rack-attack to add the functionality necessary to delete a key.
Each time an authentication is successful, the Redis counter for that IP is cleared. I deemed it better to clear the counter than to allow for blank passwords, since the latter seems like a security risk.
### Why was this MR needed?
It was quite difficult to figure out why users were seeing `403 Forbidden`, which is why the log message was added. Users were getting a lot of false positives when accessing repositories with HTTPS. Including the username in the HTTPS URL (e.g. `https://username@mydomain.com/account/repo.git`) caused authentication failures because while the git client provided the username, it left the password blank, leading to an authentication failure.
### What are the relevant issue numbers / [Feature requests](http://feedback.gitlab.com/)?
See Issue #1171https://github.com/kickstarter/rack-attack/issues/113
See merge request !392
Fix nested task lists
When nesting task list items, the parent item is wrapped in a `<p>` tag. Update the task list parser to handle these paragraph wrappers.
cc @sytse
See merge request !413
Branch names that contain `/` return a 405 error when being deleted because the slash is escaped to `%2F`
This patch will unescape the param prior to executing the delete action.
Replace commits calendar with contributions calendar
* count opening of issues and merge requests
* dont trigger git repository - use events from database
* count pushes instead of commits for faster and easier counting
* much-much faster since does not affected by repository size
See merge request !420
Disable reference generation in preformatted/code blocks
### Summary
If a user adds text in code or preformatted text via Markdown or HTML that contains `#XXX`, the system adds a note that issue `XXX` was mentioned. This is particularly annoying because we often list gdb backtrace dumps into our issues, and many issues get mentioned as a result. For example:
```
(gdb) bt
#0 0x00000000004004c4 in second () at main.cc:6
#1 0x00000000004004d2 in first () at main.cc:11
#2 0x00000000004004dd in main () at main.cc:17
(gdb)
```
### Steps to reproduce
1. In an issue, write the above text using Markdown or HTML tags (e.g. `<code>`, `<pre>`).
2. Observe that [issue 1](https://gitlab.com/gitlab-org/gitlab-ce/issues/1) and [issue 2](https://gitlab.com/gitlab-org/gitlab-ce/issues/2) have a note that says they were mentioned.
### Expected behavior
Everything enclosed in the code blocks should be ignored as references.
### Observed behavior
Issues get referenced unnecessarily.
### Fix
I've made `reference_extractor.rb` strip out HTML and Markdown blocks before processing. I considered running the raw text through the entire Markdown processor, but this seems overkill and perhaps could lead to some unintended side effects.
See merge request !365
* count opening of issues and merge requests
* dont trigger git repository - use events from database
* much-much faster since does not affected by repository size
Change permissions on backup files - #2
Use more restrictive permissions for backup tar files and for the db, uploads, and repositories directories inside the tar files. See #1894. Now the backup task recursively `chmod`s the `db/`, `uploads/`, and `repositories/` folders with 0700 permissions, and the tar file is created as 0600.
This is a followup to !1703, which was reverted because it broke Rspec tests. The test failures were due to the rake task changing directories and not changing back, which I fixed with this commit.
cc @sytse
See merge request !1716
The Net::LDAP::Filter.escape function can not be used to escape the DN name because the backslash is required to escape special chars in the DN name. This leads to the error message "Access denied for your LDAP account." and prevents the user from logging in to gitlab.
Example DN:
CN=Test\, User,OU=Organization,DC=Company
CN=Test User,OU=Organization,DC=Company
http://www.ietf.org/rfc/rfc4514.txt
Fixed rake task gitlab:cleanup:block_removed_ldap_users
Maybe not the most elegant solution, but it works for us.
This closes issue gitlab-org/gitlab-ce#955.
See merge request !338
Valery Sizov
Dmitriy Zaporozhets
Robert Schilling
Douwe Maan
Dan Tudor
Stan Hu
Douwe Maan
Dmitriy Zaporozhets
Michael Alt
Vinnie Okada
Kamil Trzcinski
Andrew Tomaka
Valery SizovandValery Sizov
Marin Jankovski
Vinnie Okada
Hannes Rosenögger
