Add option to mirror an upstream repository.
Closes internal https://dev.gitlab.org/gitlab/gitlab-ee/issues/279
Depends on gitlab-org/gitlab-shell!29
To do:
- [x] Decide on what user should be the author of the activity feed events. The initial project creator perhaps? That makes sense for personal projects, but less so for group projects, where it's even possible that the creator has since left the team.
- [x] Write specs!
- [x] Write documentation
- [x] Port back relevant commits to CE: gitlab-org/gitlab-ce!1822
----
## Mirror status on project homepage
## Settings navigation item
## Mirroring settings
## Activity feed with new "pushes" (mirror updates)
I have yet to decide on what user should be the author of these events. The initial project creator perhaps? That makes sense for personal projects, but less so for group projects, where it's even possible that the creator has since left the team.
## Warning that branch has diverged from upstream
## Commits heading after failed update
Link send user to the mirroring settings if they are a project admin.
## Mirroring settings after failed update
## New project page
## Import form after failed import
(Screenshot outdated)
Shown when the initial import of a project fails, not when an update of an existing project fails.
cc @dzaporozhets @sytses @JobV
See merge request !51
Show specific runners from projects where user is master or owner
This fix for permission escalation when handling specific runners.
The users were allowed to assign runners from projects where they were guests.
See merge request !1809
ApplicationSetting.current was called in every pages, cache it and expires it after it updated.
This changes will avoid a SQL query in every pages (~0.3 - 0.5ms).
```SQL
SELECT "application_settings".* FROM "application_settings" ORDER BY "application_settings"."id" DESC LIMIT 1
```
These scopes can just sort by the "id" column in descending order to
achieve the same result. An added benefit is being able to perform a
backwards index scan (depending on the rest of the final query) instead
of having to actually sort data.
Implement Build Artifacts
This implements #3028
1. It stores artifacts in shared/artifacts,
1. It adds `artifacts` to `.gitlab-ci.yml`,
1. We use GitLab Workhorse to offload artifacts uploading,
1. To download artifacts it uses GitLab Workhorse X-Sendfile extension,
1. There's one "artifact" per-build. The new upload removes previous one and creates a new one,
1. Default max artifact size is set to 100MB - this can be changed in settings.
Missing things:
1. Support for `.gitlab-ci.yml`: `artifacts: true or git-ls-files` which will upload all non tracked files,
1. Artifacts passing between builds.
GitLab Workhorse changes: https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/5
GitLab Runner changes: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/46
Syntax:
```
artifacts:
untracked: true # default: false
paths: # default: empty
- bin/files
```
See merge request !1584
- Offloads uploading to GitLab Workhorse
- Use /authorize request for fast uploading
- Added backup recipes for artifacts
- Support download acceleration using X-Sendfile
Improve JIRA issue closing
- Zendesk ticket: https://gitlab.zendesk.com/agent/tickets/11034
### Expected behavior
When a merge request has 'Fixes JIRA-1234' in the description the issue should not only be transitioned but also record a comment (regardless of workflow type in JIRA).
### Observed behavior
Depending on the configured workflow a comment may or may not be allowed while transitioning to a new state. In my testing the newest JIRA default workflow did not allow comments while transitioning to 'Done', for example. The 'classic' workflow does, though.
This merge request splits the transition and comment in to two API calls. The advantage is that the comment will be recorded 100% of the time instead of it being dependent on the workflow configuration.
Also, JIRA returns a '204' response code when the transition is successful. I added '204' to the success conditional because we were generating an error in the current configuration even though the transition was successful.
See merge request !45
- Add page titles to CI settings.
- Fix CI admin navigation.
- Remove duplicated scope.
- Use monospace font for commit sha.
- Add page title and header title to build page.
- Proper authorization for cancel/retry builds.
- Use gitlab pagination theme for builds and group members.
- Don't paginate builds widget on build page.
- Add badges to commit page Changes/Builds tabs.
- Add "Builds" to commit Builds tab page title.
- Add and use Ci::Build#retryable? method.
- Add CI::Build#retried? method.
- Allow all failed commit builds to be retried.
- Proper authorization for cancel/retry all builds.
- Remove unused param.
- Use time_ago_with_tooltip where appropriate.
- Tweak builds index text
- Remove duplication between builds/build and commit_statuses/commit_status.
- Use POST rather than GET for canceling and retrying builds.
- Remove redundant URL helpers.
- Add build ID to build page.
- Link branch name on build page.
- Move commit/:sha/ci to commit/:sha/builds.
Remove duplicate orders
This changes the default order from `created_at DESC, id DESC` to just `id DESC` as this achieves the same results without the overhead of having to sort data twice (we've seen queries go from 200ms to just a few ms by removing the double sort).
cc @jacobvosmaer @dzaporozhets @rspeicher @DouweM
See merge request !1735
Extend yml syntax for only and except to support specifying repository path
This allows to limit execution of jobs to specific repository.
For example:
```yaml
job:
only:
- branches@gitlab-org/gitlab-ce
except:
- master@gitlab-org/gitlab-ce
```
The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master.
@dzaporozhets @JobV @vsizov Please review.
See merge request !1720
Sorting by both "created_at" and "id" in descending order is not needed
as simply sorting by "id" in descending order will already sort rows
from new to old. Depending on the query and data involved sorting twice
can also introduce significant overhead.
This further improves performance of User.find_by_any_email and is
roughly twice as fast as the previous UNION setup.
Thanks again to @dlemstra for suggesting this.
This is significantly faster than using a sub-query, at least when run
on the GitLab.com production database. The benchmarks are a lot slower
now with these changes, most likely due to PostgreSQL choosing a
different (and less efficient) plan based on the amount of data present
in the test database.
Thanks to @dlemstra for suggesting the use of a UNION.
This query used to rely on a JOIN, effectively producing the following
SQL:
SELECT users.*
FROM users
LEFT OUTER JOIN emails ON emails.user_id = users.id
WHERE (users.email = X OR emails.email = X)
LIMIT 1;
The use of a JOIN means having to scan over all Emails and users, join
them together and then filter out the rows that don't match the criteria
(though this step may be taken into account already when joining).
In the new setup this query instead uses a sub-query, producing the
following SQL:
SELECT *
FROM users
WHERE id IN (select user_id FROM emails WHERE email = X)
OR email = X
LIMIT 1;
This query has the benefit that it:
1. Doesn't have to JOIN any rows
2. Only has to operate on a relatively small set of rows from the
"emails" table.
Since most users will only have a handful of Emails associated
(certainly not hundreds or even thousands) the size of the set returned
by the sub-query is small enough that it should not become problematic.
Performance of the old versus new version can be measured using the
following benchmark:
# Save this in ./bench.rb
require 'benchmark/ips'
email = 'yorick@gitlab.com'
def User.find_by_any_email_old(email)
user_table = arel_table
email_table = Email.arel_table
query = user_table.
project(user_table[Arel.star]).
join(email_table, Arel::Nodes::OuterJoin).
on(user_table[:id].eq(email_table[:user_id])).
where(user_table[:email].eq(email).or(email_table[:email].eq(email)))
find_by_sql(query.to_sql).first
end
Benchmark.ips do |bench|
bench.report 'original' do
User.find_by_any_email_old(email)
end
bench.report 'optimized' do
User.find_by_any_email(email)
end
bench.compare!
end
Running this locally using "bundle exec rails r bench.rb" produces the
following output:
Calculating -------------------------------------
original 1.000 i/100ms
optimized 93.000 i/100ms
-------------------------------------------------
original 11.103 (± 0.0%) i/s - 56.000
optimized 948.713 (± 5.3%) i/s - 4.743k
Comparison:
optimized: 948.7 i/s
original: 11.1 i/s - 85.45x slower
In other words, the new setup is 85x faster compared to the old setup,
at least when running this benchmark locally.
For GitLab.com these improvements result in User.find_by_any_email
taking only ~170 ms to run, instead of around 800 ms. While this is
"only" an improvement of about 4.5 times (instead of 85x) it's still
significantly better than before.
Fixes#3242
Douwe Maan
Robert Speicher
Valery Sizov
Dmitriy Zaporozhets
Kamil Trzcinski
Marin Jankovski
Minsik Yoonand윤민식
Jason Lee
Yorick Peterse
Stan Hu
Robert Speicher
Drew Blessing
Valery Sizov
Alex Lossent
Michael Chmielewski