Tuning of metrics data to store
This removes data we don't really need, as well as making sure we don't overload any cache stores or databases.
See merge request !2265
This will be used to store/increment the total query/view rendering
timings on a per transaction basis. This in turn can greatly reduce the
amount of metrics stored.
Attempting to use the /projects/:id API by specifying :id in
"namespace/project" format would always result in a 404 if the namespace
contained a dot.
The reason? From http://guides.rubyonrails.org/routing.html#specifying-constraints:
"By default the :id parameter doesn't accept dots - this is because the dot is
used as a separator for formatted routes. If you need to use a dot within an
:id add a constraint which overrides this - for example id: /[^\/]+/ allows
anything except a slash."
Closes https://github.com/gitlabhq/gitlabhq/issues/9573
This particular setup had 3 problems:
1. Storing SQL queries as tags is very inefficient as InfluxDB ends up
indexing every query (and they can get pretty large). Storing these
as values instead means we can't always display the SQL as easily.
2. We already instrument ActiveRecord query methods, thus we already
have timing information about database queries.
3. SQL obfuscation is difficult to get right and I'd rather not expose
sensitive data by accident.
Nokogiri produces inefficient XPath expressions when given CSS
expressions such as "a.gfm". Luckily these expressions can be optimized
quite easily while still achieving the same results.
In the two cases where this optimization is applied the run time has
been reduced from around 170 ms to around 15 ms.
Around 300 ms (in total) would be spent in these delegated methods due
to the extra stuff ActiveSupport adds to the compiled methods. Because
these delegations are so simple we can just manually define the methods,
saving around 275 milliseconds.
While it's useful to keep track of the different versions (Ruby, GitLab,
etc) doing so for every point wastes disk space and possibly also RAM
(which InfluxDB is all to eager to gobble up). If we want to see the
performance differences between different GitLab versions simply looking
at the performance since the last release date should suffice.
This removes the need for Sidekiq and any overhead/problems introduced
by TCP. There are a few things to take into account:
1. When writing data to InfluxDB you may still get an error if the
server becomes unavailable during the write. Because of this we're
catching all exceptions and just ignore them (for now).
2. Writing via UDP apparently requires the timestamp to be in
nanoseconds. Without this data either isn't written properly.
3. Due to the restrictions on UDP buffer sizes we're writing metrics one
by one, instead of writing all of them at once.
Storing of application metrics in InfluxDB
This adds support for tracking metrics in InfluxDB, which in turn can be visualized using Grafana. For more information see #2936.
See merge request !2042
Fix identity and user retrieval when special characters are used
Fixes#4023
I also added tests to make sure the user with special characters in his name is returned correctly.
@rspeicher this probably should be added to 8.3 as a patch.
See merge request !2176
Automatically fork a project when not allowed to edit a file.
Fixes#3215.
To do:
- [ ] Add tests
-----
## "Edit" button on file in a project the user does NOT have write access to
## Clicking will automatically create a fork
## When the fork has been created, the user is returned to the edit page on the original project with a notice
## The user cannot change the target branch and is informed that editing will start an MR
## Hitting "Commit changes" will commit and start an MR from my fork to the origin project
-----
## "Create file, "Upload file" and "New directory" buttons in a project the user does NOT have write access to
## Clicking any of these options will automatically create a fork
## When the fork has been created, the user is returned to the tree page on the original project with a notice
## Clicking "New directory" again will show the modal. The user cannot change the target branch and is informed that editing will start an MR
## Hitting "Create directory" will commit and start an MR from my fork to the origin project
cc @dzaporozhets @skyruler
See merge request !2145
Add project permissions to all project API endpoints
This standardizes all the project API formats. Also needed to support Huboard.
See merge request !2090
Fix method that ensures authentication token
Until now, `ensure_#{token_filed_name}!` method didn't persist new token in database.
This closes#4235.
See merge request !2185
Robert Speicher
Douwe Maan
Dmitriy Zaporozhets
Yorick Peterse
Stan Hu
Douglas Barbosa Alexandre
Valery Sizov
Gabriel Mazetto
Valery Sizov
Robert Speicher
cafuego
Grzegorz Bizon