Remove JiraIssue model and replace references with ExternalIssue
This MR Removes unused JiraIssue class and replaces references with ExternalIssue
Closes#18203
See merge request !4659
Added source branch text to dropdown toggle
## What does this MR do?
Previously, the dropdown toggle would default to "Select source branch", this changes that so that it defaults to the branch name and if that doesn't exist, it defaults to "Select source branch"
## Screenshots (if relevant)
See merge request !4504
Fixed alignment of download dropdown
## What does this MR do?
Correctly aligns the download dropdown on merge requests
## Screenshots (if relevant)
See merge request !4646
Fix images in emails
Closes#18474. The confirmation email is actually the only one that uses the `image_tag` helper at the moment, but this will work for all of them:
```shell
$ ag image_tag -G mailer
app/views/layouts/devise_mailer.html.haml
13: = image_tag('mailers/gitlab_header_logo.png', id: 'logo', alt: 'GitLab Wordmark')
24: = image_tag('mailers/gitlab_tanuki_2x.png', alt: 'GitLab Logo')
```
See merge request !4705
Lighten each logo path color instead of white
## What does this MR do?
Changes highlight/loading colors of tanuki from white to a lighter version of each path color
## Why was this MR needed?
Since the logo is no longer on the dark sidebar, the highlight color was getting lost against the light gray background
## What are the relevant issue numbers?
Closes#18521
## Screenshots (if relevant)
cc @jschatz1 @lbennett @dzaporozhets
See merge request !4690
Improve pipeline status in case that pipeline has no jobs
## What does this MR do?
This MR resolves problem with pipeline status when there are no build in pipeline.
This can happen when builds were skipped - for example - by using `only`/`except` keyword in `.gitlab-ci.yml`.
## What are the relevant issue numbers?
Closes#17977
See merge request !4403
Previously, the dropdown toggle would default to "Select source branch",
this changes that so that it defaults to the branch name and if that
doesn't exist, it defaults to "Select source branch"
Fix long commit message scroll issue.
Fixes horizontal scrollbar for long commit description.
## Before
## After
See merge request !4643
Due date can be removed from milestones
## What does this MR do?
Adds a button to milestone form to allow user to remove due date
## What are the relevant issue numbers?
Closes#15063
## Screenshots (if relevant)
See merge request !4428
Add shortcut to copy file content hash link
## What does this MR do?
* Allows for multiple shortcut handlers to be initialised
* Adds `y` shortcut to file pages to copy permalink URL.
## Are there points in the code the reviewer needs to double check?
Ney.
## Why was this MR needed?
UX
## What are the relevant issue numbers?
Closes#14470.
## Screenshots (if relevant)
See merge request !4496
New issue shortcut doesnt work on some pages
## What does this MR do?
1. Slightly tidies dispatcher switch case by utilising 'fall through' cases.
2. Adds shortcuts to project pipelines, milestones, builds and forks pages
## Are there points in the code the reviewer needs to double check?
I have a feeling that these may have been missed out of the case statements on purpose?
## Why was this MR needed?
Possible UX issue highlighted in #17929.
## What are the relevant issue numbers?
Closes#17929.
Closes#12616. (dup)
Closes#18123. (dup)
Closes#13688. (dup)
## Screenshots (if relevant)
See merge request !4393
Fair usage of Shared Runners
## What does this MR do?
Introduces a fair usage scheduler for shared runners.
It tries to assign builds to shared runner from projects that have the lowest number of builds currently running on shared runners.
**Example 1**:
```
We have following builds in queue:
build 1 for project 1
build 2 for project 1
build 3 for project 1
build 4 for project 2
build 5 for project 2
build 6 for project 3
With the new algorithm we will assign builds in following order:
- We choose build 1, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 4, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 6, because project 3 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 2, because project 1 as other it runs 1 build,
- We choose build 5, because project 2 runs 1 build, where project 1 runs 2 builds now,
- We choose build 3, because project 1 and runs 2 builds.
```
**Example 2**:
```
We have following builds in queue:
build 1 for project 1
build 2 for project 1
build 3 for project 1
build 4 for project 2
build 5 for project 2
build 6 for project 3
With the new algorithm we will assign builds in following order:
- We choose build 1, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We finish build 1,
- We choose build 2, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 4, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We finish build 4,
- We choose build 5, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 6, because project 3 doesn't run currently any builds,
- We choose build 3, because project 1, 2 and 3 runs exactly one build now,
```
## Why was this MR needed?
Currently, we are scheduling builds using FIFO. This is catastrophic if there are projects that create a 100-300 jobs, this basically eats most of available shared runners.
## Performance
All this logic is implemented with the help of SQL queries, because this is the fastest way to process 1k-2k pending builds in queue.
It's not the fastest SQL query, because it sorts based on number of running_builds, and this forces to calculate a number of running builds for all dependent projects. However, since we have one/two shared runners that asks every few seconds for builds this should have minimal impact on DB performance.
```
explain analyze SELECT "ci_builds".* FROM "ci_builds" JOIN (SELECT "ci_builds"."gl_project_id", count(case when status = 'running' AND runner_id = (SELECT "ci_runners"."id" FROM "ci_runners" WHERE "ci_runners"."is_shared" = 't') then 1 end) as running_builds FROM "ci_builds" INNER JOIN "projects" ON "projects"."id" = "ci_builds"."gl_project_id" AND "projects"."pending_delete" = 'f' WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" IN ('running', 'pending') AND "projects"."builds_enabled" = 't' AND "projects"."shared_runners_enabled" = 't' GROUP BY "ci_builds"."gl_project_id") AS projects ON ci_builds.gl_project_id=projects.gl_project_id WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" = 'pending' AND "ci_builds"."runner_id" IS NULL ORDER BY projects.running_builds ASC, ci_builds.id ASC;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------
Sort (cost=64777.28..64777.29 rows=1 width=1010) (actual time=301.794..302.535 rows=1537 loops=1)
Sort Key: (count(CASE WHEN (((public.ci_builds.status)::text = 'running'::text) AND (public.ci_builds.runner_id = $0)) THEN 1 ELSE NULL::integer END)), public.ci
_builds.id
Sort Method: quicksort Memory: 1423kB
-> Nested Loop (cost=63279.78..64777.27 rows=1 width=1010) (actual time=66.384..298.724 rows=1537 loops=1)
-> HashAggregate (cost=63177.15..63177.30 rows=15 width=15) (actual time=65.641..65.851 rows=187 loops=1)
InitPlan 1 (returns $0)
-> Seq Scan on ci_runners (cost=0.00..26963.66 rows=1 width=4) (actual time=1.145..34.381 rows=1 loops=1)
Filter: is_shared
Rows Removed by Filter: 6965
-> Nested Loop (cost=0.00..36186.34 rows=2715 width=15) (actual time=0.065..29.717 rows=1710 loops=1)
-> Index Scan using index_ci_builds_on_status on ci_builds (cost=0.00..8913.95 rows=3577 width=15) (actual time=0.051..12.012 rows=2583 loops
=1)
Index Cond: ((status)::text = ANY ('{running,pending}'::text[]))
Filter: ((type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 1219
-> Index Scan using projects_pkey on projects (cost=0.00..7.61 rows=1 width=4) (actual time=0.003..0.004 rows=1 loops=2583)
Index Cond: (id = public.ci_builds.gl_project_id)
Filter: ((NOT pending_delete) AND builds_enabled AND shared_runners_enabled)
Rows Removed by Filter: 0
-> Bitmap Heap Scan on ci_builds (cost=102.63..106.64 rows=1 width=1002) (actual time=1.216..1.231 rows=8 loops=187)
Recheck Cond: ((gl_project_id = public.ci_builds.gl_project_id) AND ((status)::text = 'pending'::text))
Filter: ((runner_id IS NULL) AND ((type)::text = 'Ci::Build'::text))
-> BitmapAnd (cost=102.63..102.63 rows=1 width=0) (actual time=1.201..1.201 rows=0 loops=187)
-> Bitmap Index Scan on index_ci_builds_on_gl_project_id (cost=0.00..10.52 rows=241 width=0) (actual time=0.406..0.406 rows=1944 loops=187)
Index Cond: (gl_project_id = public.ci_builds.gl_project_id)
-> Bitmap Index Scan on index_ci_builds_on_status (cost=0.00..91.78 rows=3089 width=0) (actual time=0.652..0.652 rows=3362 loops=187)
Index Cond: ((status)::text = 'pending'::text)
Total runtime: 303.832 ms
```
## Specific runners
It doesn't affect the specific runners which still serve builds FIFO.
@stanhu @markpundsack @yorickpeterse What do you think?
See merge request !4634
* master: (198 commits)
Set inverse_of for Project/Services relation
Fix admin hooks spec
Prevent default disabled buttons and links.
Add index on `requested_at` to the `members` table
Rearrange order of tabs
Fix admin active tab tests
Show created_at in table column
Nest li elements directly under ul
Move builds tab to admin overview
Add monitoring link with subtabs
Add sub links to overview
Add counter for abuse reports
Remove admin layout-nav counters
Move admin nav to horizontal layout nav
Eager load project relations in IssueParser
Use validate and required for environment and project
Award Emoji can't be awarded on system notes backend
Get rid of Gitlab::ShellEnv
Update CHANGELOG.
Fix project star tooltip on the fly.
...
Conflicts:
app/services/ci/create_builds_service.rb
This ensures that code such as this don't run needless SQL queries:
project.gitlab_issue_tracker_service.project
This also means that if the root `project` eager loads any associations
the Service object will be able to re-use those.
Add support for Docker Registry manifest v1
## What does this MR do?
Adds support for Manifest V1 generated by older versions of Docker (before 1.10).
## What are the relevant issue numbers?
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18609
See merge request !4669
Update project star/unstar tooltip
## What does this MR do?
Fixes project toggle star button tooltip to show actual tooltip for its state.
## Are there points in the code the reviewer needs to double check?
No.
## Why was this MR needed?
- When you star a project tooltip wasn't changing.
- When you star a project and refresh the page, tooltip was still showing `Star Project`
## What are the relevant issue numbers?
Fixes#18143
## Screenshots (if relevant)
### Before
### After
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- [ ] Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !4672
Eager load project relations in IssueParser
## What does this MR do?
This changes the ReferenceParser class to eager load various associations. This in turn results in the permissions checking code (e.g. the `Ability` model) to _not_ run dozens if not hundreds of extra SQL queries depending on the amount of references involved (in a single document).
## Are there points in the code the reviewer needs to double check?
No.
## Why was this MR needed?
In !4410 it was revealed a _lot_ of a queries came from the `Ability` model and the code it would call. In many cases this was because the code would simply get a project, then get the owners; or get a group, then get some association of that. Eager loading these associations is a fairly simple solution and greatly cuts down the number of queries.
## What are the relevant issue numbers?
None.
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [x] ~~API support added~~
- [ ] Tests
- [x] ~~Added for this feature/bug~~
- [ ] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !4675
Add environments and deployments
This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.
The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.
The `.gitlab-ci.yml`:
```
deploy to production:
stage: deploy
script: dpl travis...
environment: production
```
What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)
See merge request !4605
By eager loading these associations we can greatly cut down the number
of SQL queries executed when processing documents with lots of
references, especially in cases where there are references belonging to
the same project.
Since these associations are so specific to the reference parsing
process and the permissions checking process that follows it I opted to
include them directly in IssueParser instead of using something like a
scope. Once we have a need for it we can move this code to a scope or
method.
Robert Speicher
Robert Speicher
Jacob Schatz
Douwe Maan
Yorick Peterse
Paco Guzman
Rémy Coutable
Sean McGivern
Phil Hughes
Fatih Acet
Ilan Shamir
Douglas Barbosa Alexandre
Annabel Dunstone
Stan Hu
Grzegorz Bizon
Z.J. van de Weg
Douwe Maan
Kamil Trzcinski