Reduce SQL query counts in IssueReferenceFilter
## What does this MR do?
This MR adds a preparation phase for reference filters that allows them to prepare/create data structures used while iterating over HTML nodes. In this particular case the preparation phase is used for issue references to greatly cut down the amount of queries executed to get projects/issues for Markdown references.
## Are there points in the code the reviewer needs to double check?
No.
## Why was this MR needed?
Rendering Markdown containing issue references would run at most two queries for every issue reference: one to get the project and one to get the issue from said project. When rendering Markdown with lots of issue references this would result in _a lot_ of queries being executed.
## What are the relevant issue numbers?
#18042
See merge request !4410
This reduces the number of queries executed in IssueReferenceFilter by
retrieving the various projects/issues that may be referenced in batches
_before_ iterating over all the HTML nodes.
A chunk of the logic resides in AbstractReferenceFilter so it can be
re-used by other filters in the future.
Improve pipeline status in case that pipeline has no jobs
## What does this MR do?
This MR resolves problem with pipeline status when there are no build in pipeline.
This can happen when builds were skipped - for example - by using `only`/`except` keyword in `.gitlab-ci.yml`.
## What are the relevant issue numbers?
Closes#17977
See merge request !4403
* master: (198 commits)
Set inverse_of for Project/Services relation
Fix admin hooks spec
Prevent default disabled buttons and links.
Add index on `requested_at` to the `members` table
Rearrange order of tabs
Fix admin active tab tests
Show created_at in table column
Nest li elements directly under ul
Move builds tab to admin overview
Add monitoring link with subtabs
Add sub links to overview
Add counter for abuse reports
Remove admin layout-nav counters
Move admin nav to horizontal layout nav
Eager load project relations in IssueParser
Use validate and required for environment and project
Award Emoji can't be awarded on system notes backend
Get rid of Gitlab::ShellEnv
Update CHANGELOG.
Fix project star tooltip on the fly.
...
Conflicts:
app/services/ci/create_builds_service.rb
Add support for Docker Registry manifest v1
## What does this MR do?
Adds support for Manifest V1 generated by older versions of Docker (before 1.10).
## What are the relevant issue numbers?
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18609
See merge request !4669
Eager load project relations in IssueParser
## What does this MR do?
This changes the ReferenceParser class to eager load various associations. This in turn results in the permissions checking code (e.g. the `Ability` model) to _not_ run dozens if not hundreds of extra SQL queries depending on the amount of references involved (in a single document).
## Are there points in the code the reviewer needs to double check?
No.
## Why was this MR needed?
In !4410 it was revealed a _lot_ of a queries came from the `Ability` model and the code it would call. In many cases this was because the code would simply get a project, then get the owners; or get a group, then get some association of that. Eager loading these associations is a fairly simple solution and greatly cuts down the number of queries.
## What are the relevant issue numbers?
None.
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [x] ~~API support added~~
- [ ] Tests
- [x] ~~Added for this feature/bug~~
- [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !4675
Add environments and deployments
This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.
The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.
The `.gitlab-ci.yml`:
```
deploy to production:
stage: deploy
script: dpl travis...
environment: production
```
What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)
See merge request !4605
By eager loading these associations we can greatly cut down the number
of SQL queries executed when processing documents with lots of
references, especially in cases where there are references belonging to
the same project.
Since these associations are so specific to the reference parsing
process and the permissions checking process that follows it I opted to
include them directly in IssueParser instead of using something like a
scope. Once we have a need for it we can move this code to a scope or
method.
We can't do a lot with classes without names as we can't filter by them,
have no idea where they come from, etc. As such it's best to just ignore
these.
Allow users to request access to projects and groups
#### TODO:
- [x] Factorize the projects & groups members list
- [x] Factorize `request_access` and `approve` controller methods into a concern
- [x] Factorize the project & group dropdowns
- [x] Factorize access request routes
- [x] Factorize mailers
#### Will be done in separate MRs
- [ ] Documentation
- [ ] API
#### Access request to project
- [x] Design project access request button to spec (defined in #13948)
- [x] Request access to a project as a logged in user
- [x] Withdraw access request to a project as a logged in user
- [x] Do not treat project member request as actual membership (done partly in Ability)
- [x] Display access requests on project's members page (+ allow to change level, approve, and deny request)
- [x] Email requesting user at notification email address when accepted/denied
- [x] Email all project admins at notification email addresses when new access request
- [x] Unit specs to cover new functionality: `AccessRequestable` is missing specs
- [x] Controller specs to cover new functionality
- [x] Mailer specs to cover new functionality
- [x] Features specs to cover new UI and interactions
- [x] Regression specs still passing
#### Access request to group
- [x] Design group access request button to spec (defined in #13948)
- [x] Request access to a group as a logged in user
- [x] Withdraw access request to a group as a logged in user
- [x] Do not treat group member request as actual membership
- [x] Display access requests on group's members page (+ allow to change level, approve, and deny request)
- [x] Email requesting user at notification email address when accepted/denied
- [x] Email all project admins at notification email addresses when new access request
- [x] Unit specs to cover new functionality
- [x] Controller specs to cover new functionality
- [x] Mailer specs to cover new functionality
- [x] Features specs to cover new UI and interactions
- [x] Regression specs still passing
Closes#13948.
### Requesting access to a project
#### Responsive UI
#### Request access link
#### Flash once access is requested
#### Email sent to project admins on new access requests
##### HTML Email
##### Plain Email
### Withdrawing access request to a project
#### Withdraw access request link
#### Withdraw access request confirmation dialog box
#### Flash once access has been withdrawn
### Access requests management page
#### Deny access request confirmation dialog box
#### Email sent to requester on denied access requests
##### HTML Email
##### Plain Email
#### Email sent to requester on granted access requests
##### HTML Email
##### Plain Email
----------------------------------
----------------------------------
### Requesting access to a group
#### Responsive UI
#### Request access link
#### Flash once access is requested
#### Email sent to project admins on new access requests
##### HTML Email
##### Plain Email
### Withdrawing access request to a project
#### Withdraw access request link
#### Withdraw access request confirmation dialog box
#### Flash once access has been withdrawn
### Access requests management page
#### Deny access request confirmation dialog box
#### Email sent to requester on denied access requests
##### HTML Email
##### Plain Email
#### Email sent to requester on granted access requests
##### HTML Email
##### Plain Email
See merge request !3798
By default instrumentation will instrument public,
protected and private methods, because usually
heavy work is done on private method or at least
that’s what facts is showing
* master: (538 commits)
Fix broken URI joining for `teamcity_url` with suffixes
Factorize duplicated code into a method in BambooService and update specs
Fix broken URI joining for `bamboo_url` with suffixes
Honor credentials on calling Bamboo CI trigger
Update CHANGELOG
Use Issue.visible_to_user in Notes.search to avoid query duplication
Project members with guest role can't access confidential issues
Allow users to create confidential issues in private projects
Update CHANGELOG
Remove deprecated issues_tracker and issues_tracker_id from project
Schema doesn’t reflect the changes of the last 3 migrations
Apply reviewer notes: update CHANGELOG, adjust code formatting
Move issue rendering tests into separate contexts
Move change description to proper release and fix typo
Add more information into RSS fead for issues
Revert CHANGELOG
Also rename "find" in the specs
Change to new Notes styleguide
Add guide on changing a document's location
Change logs.md location in README
...
Conflicts:
app/services/ci/create_builds_service.rb
app/services/ci/create_pipeline_service.rb
app/services/create_commit_builds_service.rb
spec/models/ci/commit_spec.rb
spec/services/ci/create_builds_service_spec.rb
spec/services/create_commit_builds_service_spec.rb
+ Move 'Edit Project/Group' out of membership-related partial
+ Show the access request buttons only to logged-in users
+ Put the request access buttons out of in a more visible button
+ Improve the copy in the #remove_member_message helper
Signed-off-by: Rémy Coutable <remy@rymai.me>
Generating the following tags
Grape#GET /projects/:id/archive
from Grape::Route objects like
{ :path => /:version/projects/:id/archive(.:format)
:version => “v3”,
:method => “GET” }
Use an instance variable to cache raw_path transformations.
This variable is only going to growth to the number of
endpoints of the API, not with exact different requests
We can store this cache as an instance variable because
middleware are initialised only once
Add global entry with before script to new CI config
## What does this MR do?
This MR adds a new entries to a new CI config class. It is next refactoring step after !4462.
See #15060
See merge request !4482
Add whitelisted elements correctly in sanitization
Add whitelisted elements correctly in sanitization
Consider this command:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>', pipeline: :description)
puts markdown('<span>this is a span</span>')"
And the same in the opposite order:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>')
puts markdown('<span>this is a span</span>', pipeline: :description)"
Before this change, they would both output:
<p><span>this is a span</span></p>
<p>this is a span</p>
That's because `span` is added to the list of whitelisted elements in
the `SanitizationFilter`, but this method tries not to make the same
changes multiple times. Unfortunately,
`HTML::Pipeline::SanitizationFilter::LIMITED`, which is used by the
`DescriptionPipeline`, uses the same Ruby objects for all of its hash
values _except_ `:elements`.
That means that whichever of `DescriptionPipeline` and `GfmPipeline` is
called first would have `span` in its whitelisted elements, and the
second wouldn't.
Fix this by adding a special check for modifying `:elements` twice, then
checking `:transformers` as before.
See merge request !4588
Improve Gitlab::Auth method names
Auth.find was a very generic name for a very specific method.
Auth.find_in_gitlab_or_ldap was inaccurate in GitLab EE where it also
looks in Kerberos.
See merge request !4589
Consider this command:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>', pipeline: :description)
puts markdown('<span>this is a span</span>')"
And the same in the opposite order:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>')
puts markdown('<span>this is a span</span>', pipeline: :description)"
Before this change, they would both output:
<p><span>this is a span</span></p>
<p>this is a span</p>
That's because `span` is added to the list of whitelisted elements in
the `SanitizationFilter`, but this method tries not to make the same
changes multiple times. Unfortunately,
`HTML::Pipeline::SanitizationFilter::LIMITED`, which is used by the
`DescriptionPipeline`, uses the same Ruby objects for all of its hash
values _except_ `:elements`.
That means that whichever of `DescriptionPipeline` and `GfmPipeline` is
called first would have `span` in its whitelisted elements, and the
second wouldn't.
Fix this by creating an entirely separate hash, before either pipeline
is invoked.
Douwe Maan
Yorick Peterse
Paco Guzman
Rémy Coutable
Grzegorz Bizon
Jacob Vosmaer
Kamil Trzcinski
Z.J. van de Weg
Douwe Maan
Sean McGivern
