Handle external issues in IssueReferenceFilter
Rendering issue references such as `#1` was broken for projects using an external issues tracker.
See gitlab-org/gitlab-ce#19036
See merge request !4988
Allow importing from Github using Personal Access Tokens
_Originally opened at !4005 by @ericidema._
------
## What does this MR do?
* Made changes to `Gitlab::GithubImport::Client` so that it can be used with Github Personal Access Tokens without the need for OAuth.
* Added UI to collect Personal Access Token from user.
* Detect if the user has logged in with GitHub and use OAuth to skip the Personal Access Token form.
## Are there points in the code the reviewer needs to double check?
Twin Omnibus MR: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/774
## What are the relevant issue numbers?
Closes#13970.
## Screenshots
### GitHub import is configured
-----
### GitHub import is not configured
-----
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
- [x] Added for this feature/bug
- [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !4938
Fix restore warning message
## What does this MR do?
Fix the restore Rake task so it properly outputs the database warning. This is a pretty important warning and it was not even being output. After this fix, the output looks like the screenshot below.
See merge request !4980
This stands as an alternative to using OAuth to access a user's Github
repositories. This is setup in such a way that it can be used without OAuth
configuration.
From a UI perspective, the how to import modal has been replaced by a full
page, which includes a form for posting a personal access token back to the
Import::GithubController.
If the user has logged in via GitHub, skip the Personal Access Token and go
directly to Github for an access token via OAuth.
Enable Style/UnneededCapitalW Rubocop cop
## What does this MR do?
This MR enables Rubocop cop that checks if `%W[]`, which supports interpolation, is needed.
## What are the relevant issue numbers?
#17478
See merge request !5010
Remove coveralls as its unused
This gem is unused so could be removed.
@dzaporozhets @connorshea We've discussed this some time ago on Slack
See merge request !5001
IssueReferenceFilter will end up processing internal issue references
when a project uses an external issues tracker while still using
internal issue references (in the form of `#\d+`). This commit ensures
that these links are rendered as external issue links, regardless of
whether the project one currently views uses an internal or external
issues tracker.
Fixesgitlab-org/gitlab-ce#19036, gitlab-com/performance#16
Fix database migrations when Redis is not running
If Redis were not running or USE_DB were set to false, the application settings retrieval would fail completely. This change only attempts to use the cache if the system actually
wants to connect to the DB and rescues any failures in talking to Redis.
Closes#17557
See merge request !4924
If Redis were not running or USE_DB were set to false, the
application settings retrieval would fail completely. This
change only attempts to use the cache if the system actually
wants to connect to the DB and rescues any failures in talking to
Redis.
Closes#17557
Process.clock_gettime allows getting the real time in nanoseconds as
well as allowing one to get a monotonic timestamp. This offers greater
accuracy without the overhead of having to allocate a Time instance. In
general using Time.now/Time.new is about 2x slower than using
Process.clock_gettime(). For example:
require 'benchmark/ips'
Benchmark.ips do |bench|
bench.report 'Time.now' do
Time.now.to_f
end
bench.report 'clock_gettime' do
Process.clock_gettime(Process::CLOCK_MONOTONIC, :millisecond)
end
bench.compare!
end
Running this benchmark gives:
Calculating -------------------------------------
Time.now 108.052k i/100ms
clock_gettime 125.984k i/100ms
-------------------------------------------------
Time.now 2.343M (± 7.1%) i/s - 11.670M
clock_gettime 4.979M (± 0.8%) i/s - 24.945M
Comparison:
clock_gettime: 4979393.8 i/s
Time.now: 2342986.8 i/s - 2.13x slower
Another benefit of using Process.clock_gettime() is that we can simplify
the code a bit since it can give timestamps in nanoseconds out of the
box.
In any case if just want the value which is always ‘gitlab’
require 'benchmark/ips'
Project.first # To load database things
GitlabIssueTrackerService.first # To load database things
Benchmark.ips do |x|
x.config(:time => 5, :warmup => 2)
x.report("current") do
Project.new.default_issue_tracker.to_param
end
x.report("") do
'gitlab'
end
x.compare!
end
Calculating -------------------------------------
current 4.000 i/100ms
30.938k i/100ms
-------------------------------------------------
current 47.298 (±10.6%) i/s - 232.000
4.366M (±20.9%) i/s - 17.202M
Comparison:
: 4366456.0 i/s
current: 47.3 i/s - 92318.26x slower
1. Return 404 if commit is not found (RESTful resource not found)
2. Return an empty array if pipeline is not found (resource present, no
associated builds found)
3. Return an empty array if pipeline found but no builds there (resource
present, no associated builds)
Better debugging for memory killer middleware
This adds more info to the warning messages output by `MemoryKiller`.
Previously only the PID was showed, making it difficult to debug issues like https://gitlab.com/gitlab-org/gitlab-ce/issues/19124
This adds the worker class and job ID to the log messages.
See merge request !4936
Improve validations and error handling in new CI config entries
## What does this MR do?
This MR improves validation in new CI config.
## Why was this MR needed?
With that it will be easier to handle errors during validation and post-processing.
## What are the relevant issue numbers?
This is a continuation of #15060
See merge request !4560
Fix temp file being deleted after the request while importing a GitLab project
Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151
In production, the temporary uploaded file is getting deleted straight after the request so the Sidekiq worker is unable to find it in `/tmp`
Also, improved erroring/logging of this situation.
See merge request !4894
This commit changes the way certain documents are rendered (currently
only Notes) and how documents are redacted. Previously both rendering
and redacting would run on a per document basis. The result of this was
that for every document we'd have to run countless queries just to
figure out if we could display a set of links or not.
This commit changes things around so that redacting Markdown documents
is no longer tied into the html-pipeline Gem. This in turn allows it to
redact multiple documents in a single pass, thus reducing the number of
queries needed.
In turn rendering issue/merge request notes has been adjusted to take
advantage of this new setup. Instead of rendering Markdown somewhere
deep down in a view the Markdown is rendered and redacted in the
controller (taking the current user and all that into account). This has
been done in such a way that the "markdown()" helper method can still be
used on its own.
This particular commit also paves the way for caching rendered HTML on
object level. Right now there's an accessor method Note#note_html which
is used for setting/getting the rendered HTML. Once we cache HTML on row
level we can simply change this field to be a column and call a "save"
whenever needed and we're pretty much done.
Fix subsequent SAML sign ins
Fixes a bug when `auto_link_ldap_user` is `true` that causes SAML users to be unable to sign in a second time.
Fix the problem for https://gitlab.zendesk.com/agent/tickets/22546
See merge request !4718
Handle external issues in IssueReferenceFilter
Handling of external issues was broken when I refactored
`IssueReferenceFilter` to use fewer SQL queries.
Fixes#18827
See merge request !4789
Robert Speicher
Douwe Maan
Rémy Coutable
Eric K IdemaandRémy Coutable
Dmitriy Zaporozhets
Grzegorz Bizon
Yorick Peterse
Z.J. van de Weg
Alejandro Rodríguez
Stan Hu
Drew Blessing
Paco Guzman
http://jneen.net/
Robert Speicher
James Lopez
Connor Shea
Marc Siegfriedt