Prefer ref rather than id because id is shadowing database id
## What does this MR do?
Just a local variable renaming.
## Why was this MR needed?
Prefer ref rather than id because id is shadowing database id.
See merge request !5134
Allow specifying protected branches using wildcards
Closes#18627
# Tasks
- [ ] #18627!4665 Allow specifying protected branches using wildcards
- [x] Find existing usages of protected branches
- Protecting branches
- `ProtectedBranchesController` is used to mark a branch protected/unprotected
- `API::Branches` can be used to mark a branch protected/unprotected
- Enforcing branch protection
- `Gitlab::GitAccess` has helpers (`can_push_to_branch?`, `check`) that are used to deny pushes if a branch is protected
- Over SSH: `gitlab-shell` receives a push, and calls `/allowed` on the GitLab API, which calls `GitAccess.check`
- Over HTTP:
- `gitlab-workhorse` receives the request, and forwards it to rails
- Rails (in the `GitHttpController#git-recieve-pack`) runs basic checks (is the user logged in, not protected branch checks) and returns ok with `GL_ID` and `RepoPath`
- `gitlab-workhorse` looks at the response, and calls the relevant `gitlab-shell` action from `git-http/handlePostRPC`
- Rest of this flow is the same as the SSH flow above
- [x] Implementation
- [x] Backend
- [x] Change `project#protected_branch?` to look at wildcard protected branches
- [x] Change `project#developers_can_push_to_protected_branch?`
- [x] Change `project#open_branches`
- [x] Better error message when creating a disallowed branch from the Web UI
- [x] Frontend
- [x] Protected branches page should allow typing out a wildcard pattern
- [x] Add help text explaining the use of wildcards
- [x] Show matching branches for each protected branch
- [x] ~~On the index page~~
- [x] On a show page
- [x] Index?
- [x] Can't have the "last commit" column for wildcard protected branches
- [x] Fix / write tests
- [x] What happens if a hook is missing in dev?
- [x] Refactor
- [x] Test workflows
- Create a branch matching a wildcard pattern
- Push to a branch matching a wildcard pattern
- Force push to a branch matching a wildcard pattern
- Delete a branch matching a wildcard pattern
- [x] Test using Web UI
- [x] Test over SSH
- [x] Test over HTTP
- [x] Test as developer and master
- [x] Investigate performance
- [x] Test with a large number of protected branches / branches
- [x] Paginate list of protected branches
- [x] ~~Possibly rewrite `open_branches`~~
- [x] Add `iid`s to existing `ProtectedBranch`es
- [x] Add documentation
- [x] Add CHANGELOG entry
- [x] Add screenshots
- [x] Make sure [build](https://gitlab.com/gitlab-org/gitlab-ce/commit/2f753e3ed2ce681b4444944d521f4419e8ed37f7/builds) passes
- [x] Assign to endboss for review
- [x] Address @DouweM's comments
- [x] `protected_branch_params`
- [x] `exact_match` instead of `explicit_match`
- [x] When would self.name be blank?
- [x] Move `protected_branches.each` to a partial
- [x] Move `matching_branches.each` to a partial
- [x] If the branch is in @matching_branches, it's not been removed
- [x] move this regex to a method and memoize it
- [x] `commit_sha` directly for exact matches
- [x] Number of matches for wildcard matches, with a link
- [x] Wait for [build](https://gitlab.com/gitlab-org/gitlab-ce/commit/43f9ce0e88194b8f719bb1c1e656b7fc13278d56/builds) to pass
- [x] Respond to @DouweM's comments
- [x] Don't use iid
- [x] Controller should use `@project.protected_branches.new`
- [x] move the memoization to `def wildcard_regex`
- [x] render with `collection: @protected_branches`
- [x] Wait for [build](https://gitlab.com/gitlab-org/gitlab-ce/commit/f7beedf122fa0c7aa89e86181fe7499321fb10ca/builds) to pass
- [x] Wait for @DouweM's review
- [x] Wait for @jschatz1's review
- [x] Respond to @jschatz1's comments
- [x] Use the new dropdown style
- [x] description should be moved to the description section without the styling
- [x] Protect button should be disabled when no branch is selected
- [x] Update screenshots
- [x] Merge conflicts
- [x] Make sure [build](https://gitlab.com/gitlab-org/gitlab-ce/commit/20f3cfe8d5540eab64c2ba548043d600b28c61ba/builds) passes
- [ ] Revisit performance, possibly with staging/production data
- [ ] Get a dump of staging / run against staging live
- [ ] Get SSH access to staging
- [ ] Wait for review/merge
# Screenshots
## Creating wildcard protected branches
### Using the `GLDropdown` component
## Enforcing wildcard protected branches
### From the Web UI
### Over SSH
### Over HTTPS
## Listing matching branches
See merge request !4665
1. The `open_branches` method is used to provide a list of branches
while creating a protected branch.
2. It makes sense to include branches which are matched by one or more
wildcard protected branches, since the user might want to make exact
protected branches from these as well.
3. This also provides a large performance improvement. On my machine, in
a project with 5000 branches and 2000 protected branches, the
`ProtectedBranches#index` page went from a 40 seconds load time to 4
seconds (10x speedup).
Fixing URL validation for import_url on projects
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/17536
This MR fixes problems related to bypassing `import_url` validation on projects. This makes sure the URL is properly validated so we don't enter crap and fail while running workers that handle this URL.
It also adds a migration to fix current invalid `import_url`s
See merge request !4753
`avatar.present?` goes through CarrierWave, and checks that the file
exists on disk and checks its filesize. Because we're hitting the disk,
this adds extra overhead to something where the worst-case scenario is
rendering a broken image.
Instead, we now just check that the _database attribute_ is present,
which is good enough for our purposes.
See https://gitlab.com/gitlab-org/gitlab-ce/issues/19273
1. The main implementation is in the `ProtectedBranch` model. The
wildcard is converted to a Regex and compared. This has been tested
thoroughly.
- While `Project#protected_branch?` is the main entry point,
`project#open_branches` and
`project#developers_can_push_to_protected_branch?`
have also been modified to work with wildcard protected branches.
- The regex is memoized (within the `ProtectedBranch` instance)
2. Improve the performance of `Project#protected_branch?`
- This method is called from `Project#open_branches` once _per branch_
in the project, to check if that branch is protected or not.
- Before, `#protected_branch?` was making a database call every
time it was invoked (in the above case, that amounts to once
per branch), which is expensive.
- This commit caches the list of protected branches in memory, which
reduces the number of database calls down to 1.
- A downside to this approach is that `#protected_branch?` _could_
return a stale value (due to the caching), but this is
an acceptable tradeoff.
3. Remove the (now) unused `Project#protected_branch_names` method.
- This was previously used to check for protected branch status.
Fixed ordering in Project.find_with_namespace
This MR fixes the ordering of `Project.find_with_namespace` to ensure that it returns rows that match literally first.
Closes#18603
See merge request !4682
This ensures that Project.find_with_namespace returns a row matching
literally as the first value, instead of returning a random value.
The ordering here is _only_ applied to Project.find_with_namespace and
_not_ Project.where_paths_in as currently there's no code that requires
Project.where_paths_in to return rows in a certain order. Since this
method also returns all rows that match there's no real harm in not
setting a specific order either. Another reason is that generating all
the "WHEN" arms for multiple values in Project.where_paths_in becomes
really messy.
On MySQL we have to use the "BINARY" operator to turn a "WHERE" into a
case-sensitive WHERE as otherwise MySQL may still end up returning rows
in an unpredictable order.
Fixesgitlab-org/gitlab-ce#18603
This ensures that code such as this don't run needless SQL queries:
project.gitlab_issue_tracker_service.project
This also means that if the root `project` eager loads any associations
the Service object will be able to re-use those.
Add environments and deployments
This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.
The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.
The `.gitlab-ci.yml`:
```
deploy to production:
stage: deploy
script: dpl travis...
environment: production
```
What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)
See merge request !4605
+ Move 'Edit Project/Group' out of membership-related partial
+ Show the access request buttons only to logged-in users
+ Put the request access buttons out of in a more visible button
+ Improve the copy in the #remove_member_message helper
Signed-off-by: Rémy Coutable <remy@rymai.me>
Add Project.where_paths_in
In https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4410 I'm working on reducing the SQL queries needed to render Markdown. One reason for the large amount of queries is executing a large number of queries needed to find projects and related data. Basically `Project.find_with_namespace` is called in a loop and then any relations have to be retrieved separately.
By using `Project.where_paths_in` we can work around this by doing something like:
```ruby
project_paths = [...] # populated by some method
projects = Project.where_paths_in(project_paths).includes(:namespace, ...)
```
Ref: https://gitlab.com/gitlab-org/gitlab-ce/issues/18042
See merge request !4535
This method can be used to find multiple projects for multiple paths.
For example, take this snippet:
Project.where_paths_in(%w{gitlab-org/gitlab-ce gitlab-org/gitlab-ee})
This will return an ActiveRecord::Relation containing the GitLab CE and
GitLab EE projects.
This method takes care of matching rows both case-sensitively and
case-insensitively where needed.
Project.find_with_namespace in turn has been modified to use
Project.where_paths_in without nuking any scoping (instead it uses
reorder(nil)). This means that any default scopes (e.g. those used for
"pending_delete" stay intact).
The method Project.where_paths_in was added so the various Markdown
filters can use a single query to grab all the projects referenced in a
set of documents, something Project.find_with_namespace didn't allow.
Douwe Maan
Lin Jen-Shin
Timothy Andrew
Robert Speicher
Robert Speicher
Rémy Coutable
James Lopez
Alejandro Rodríguez
Tim
Ruben Davila
Yorick Peterse
Kamil Trzcinski
Douglas Barbosa Alexandre
Paco GuzmanandYorick Peterse