Add Project.where_paths_in
In https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4410 I'm working on reducing the SQL queries needed to render Markdown. One reason for the large amount of queries is executing a large number of queries needed to find projects and related data. Basically `Project.find_with_namespace` is called in a loop and then any relations have to be retrieved separately.
By using `Project.where_paths_in` we can work around this by doing something like:
```ruby
project_paths = [...] # populated by some method
projects = Project.where_paths_in(project_paths).includes(:namespace, ...)
```
Ref: https://gitlab.com/gitlab-org/gitlab-ce/issues/18042
See merge request !4535
This method can be used to find multiple projects for multiple paths.
For example, take this snippet:
Project.where_paths_in(%w{gitlab-org/gitlab-ce gitlab-org/gitlab-ee})
This will return an ActiveRecord::Relation containing the GitLab CE and
GitLab EE projects.
This method takes care of matching rows both case-sensitively and
case-insensitively where needed.
Project.find_with_namespace in turn has been modified to use
Project.where_paths_in without nuking any scoping (instead it uses
reorder(nil)). This means that any default scopes (e.g. those used for
"pending_delete" stay intact).
The method Project.where_paths_in was added so the various Markdown
filters can use a single query to grab all the projects referenced in a
set of documents, something Project.find_with_namespace didn't allow.
This was not a clear cut n+1 query, given that if you're directly subscribed to all issues
that the API is returning you never really need to check for the notes. However, if you're
subscribed to _all_ of them, then for each issuable you need to go once to `notes`, and
once to `users` (for the authors). By preemtively loading notes and authors, at worst you have
1 extra query, and at best you saved 2n extra queries. We also took advantage of this preloading
of notes when counting user notes.
Notes are awardables
## What does this MR do?
Makes sure we can :thunder_cloud_rain: comments/notes.
## What are the relevant issue numbers?
Follows up upon !2901, depends on !3785Closes#3655
## Screenshots (if relevant)
TODO
See merge request !4291
- To hold registrations from U2F devices, and to authenticate them.
- Previously, `User#two_factor_enabled` was aliased to the
`otp_required_for_login` column on `users`.
- This commit changes things a bit:
- `User#two_factor_enabled` is not a method anymore
- `User#two_factor_enabled?` checks both the
`otp_required_for_login` column, as well as `U2fRegistration`s
- Change all instances of `User#two_factor_enabled` to
`User#two_factor_enabled?`
- Add the `u2f` gem, and implement registration/authentication at the
model level.
This will be useful when you want to ask for the
number of items and later iterate over them, without
needing to ask if the association is load or not.
So you avoid extra database queries
Upgrade devise, devise-two-factor, and attr_encrypted
Devise 4 includes support for Rails 5, working towards #14286. devise-async doesn't support Devise 4.0 and in 4.1 the bug that was blocking using Devise's built-in ActiveJob integration was fixed. So devise-async is removed. devise-two-factor 3.0.0 is required for Devise 4 support.
attr_encrypted and encryptor are optional but recommended upgrades for devise-two-factor 3.0.0. The mode and algorithm will need to be changed in order to update to attr_encrypted 4.x in the future.
See merge request !4216
Separate reference gathering from rendering
This is a required step to allow batch processing when gathering references. This in turn would allow grabbing (for example) all mentioned users of an issue/merge request using a single query.
cc @rspeicher @DouweM
See merge request !3969
There are several changes to this module:
1. The use of an explicit stack in Participable#participants
2. Proc behaviour has been changed
3. Batch permissions checking
== Explicit Stack
Participable#participants no longer uses recursion to process "self" and
all child objects, instead it uses an Array and processes objects in
breadth-first order. This allows us to for example create a single
Gitlab::ReferenceExtractor instance and pass this to any Procs. Re-using
a ReferenceExtractor removes the need for running potentially many SQL
queries every time a Proc is called on a new object.
== Proc Behaviour Changed
Previously a Proc in Participable was expected to return an Array of
User instances. This has been changed and instead it's now expected that
a Proc modifies the Gitlab::ReferenceExtractor passed to it. The return
value of the Proc is ignored.
== Permissions Checking
The method Participable#participants uses
Ability.users_that_can_read_project to check if the returned users have
access to the project of "self" _without_ running multiple SQL queries
for every user.
Douwe Maan
Robert Speicher
Rémy Coutable
Felipe Artur
Paco GuzmanandYorick Peterse
Rui AndersonandRémy Coutable
Kamil Trzcinski
Alejandro Rodríguez
Lin Jen-Shin
ZJ van de Weg
Timothy Andrew
Grzegorz Bizon
Josh Frye
Stan Hu
Douwe Maan