* upstream/master: (353 commits)
Put some admin settings in dropdown
Add styleguide on configuration settings documentation
Remove Duplicated keys add UNIQUE index to fingerprint
Avoid autoload issue such as 'Mail::Parsers::AddressStruct'
Move appearance settings as sub tab to application settings
use rails root join
fixed a couple of errors spotted in production
Fix RangeError exceptions when referring to issues or merge requests outside of max database values
Fix bug in `WikiLinkFilter`.
Grammar and typographic changes to artifacts documentation
Tweak grammar
Small frontend code fixes and restore 8a2d88f commit
Warn about admin privilege to disable GitHub Webhooks
Listing GH Webhooks doesn't stop import process for non GH admin users
fixup! updated docs for api endpoint award emoji
Update CHANGELOG
Ensure Todos counters doesn't count Todos for projects pending delete
Add endpoints for award emoji on notes
Sort API endpoints and implement feedback
Add endpoints for Award Emoji
...
Implement custom notification level options
part of #12758
See merge request !4389
- As todos are created/updated inside the TodoService
we repopulate the cache just there for both pending/done todos
- Todos as mark as done from the TodosController we update cache
there too
- All the added methods are kept in the User class for cohesion
* upstream/master: (337 commits)
Update CHANGELOG for !4659
Center the header logo for all Devise emails
Add previews for all customized Devise emails
Customize the Devise `unlock_instructions` email
Customize the Devise `reset_password_instructions` email
Customize the Devise `password_change` emails
Use gitlab-git 10.2.0
Use Git cached counters on project show page
Fix indentation scss-lint errors
Added title attribute to enties in tree view Closes#18353
Banzai::Filter::ExternalLinkFilter use XPath
Reduce queries in IssueReferenceFilter
Use gitlab_git 10.1.4
Fixed ordering in Project.find_with_namespace
Fix images in emails
Banzai::Filter::UploadLinkFilter use XPath
Turn Group#owners into a has_many association
Make project_id nullable
CHANGELOG [ci skip]
CHANGELOG [ci skip]
...
Improve pipeline status in case that pipeline has no jobs
## What does this MR do?
This MR resolves problem with pipeline status when there are no build in pipeline.
This can happen when builds were skipped - for example - by using `only`/`except` keyword in `.gitlab-ci.yml`.
## What are the relevant issue numbers?
Closes#17977
See merge request !4403
Fair usage of Shared Runners
## What does this MR do?
Introduces a fair usage scheduler for shared runners.
It tries to assign builds to shared runner from projects that have the lowest number of builds currently running on shared runners.
**Example 1**:
```
We have following builds in queue:
build 1 for project 1
build 2 for project 1
build 3 for project 1
build 4 for project 2
build 5 for project 2
build 6 for project 3
With the new algorithm we will assign builds in following order:
- We choose build 1, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 4, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 6, because project 3 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 2, because project 1 as other it runs 1 build,
- We choose build 5, because project 2 runs 1 build, where project 1 runs 2 builds now,
- We choose build 3, because project 1 and runs 2 builds.
```
**Example 2**:
```
We have following builds in queue:
build 1 for project 1
build 2 for project 1
build 3 for project 1
build 4 for project 2
build 5 for project 2
build 6 for project 3
With the new algorithm we will assign builds in following order:
- We choose build 1, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We finish build 1,
- We choose build 2, because project 1 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 4, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We finish build 4,
- We choose build 5, because project 2 doesn't run currently any builds and has the lowest build number from projects that doesn't run builds,
- We choose build 6, because project 3 doesn't run currently any builds,
- We choose build 3, because project 1, 2 and 3 runs exactly one build now,
```
## Why was this MR needed?
Currently, we are scheduling builds using FIFO. This is catastrophic if there are projects that create a 100-300 jobs, this basically eats most of available shared runners.
## Performance
All this logic is implemented with the help of SQL queries, because this is the fastest way to process 1k-2k pending builds in queue.
It's not the fastest SQL query, because it sorts based on number of running_builds, and this forces to calculate a number of running builds for all dependent projects. However, since we have one/two shared runners that asks every few seconds for builds this should have minimal impact on DB performance.
```
explain analyze SELECT "ci_builds".* FROM "ci_builds" JOIN (SELECT "ci_builds"."gl_project_id", count(case when status = 'running' AND runner_id = (SELECT "ci_runners"."id" FROM "ci_runners" WHERE "ci_runners"."is_shared" = 't') then 1 end) as running_builds FROM "ci_builds" INNER JOIN "projects" ON "projects"."id" = "ci_builds"."gl_project_id" AND "projects"."pending_delete" = 'f' WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" IN ('running', 'pending') AND "projects"."builds_enabled" = 't' AND "projects"."shared_runners_enabled" = 't' GROUP BY "ci_builds"."gl_project_id") AS projects ON ci_builds.gl_project_id=projects.gl_project_id WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" = 'pending' AND "ci_builds"."runner_id" IS NULL ORDER BY projects.running_builds ASC, ci_builds.id ASC;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------
Sort (cost=64777.28..64777.29 rows=1 width=1010) (actual time=301.794..302.535 rows=1537 loops=1)
Sort Key: (count(CASE WHEN (((public.ci_builds.status)::text = 'running'::text) AND (public.ci_builds.runner_id = $0)) THEN 1 ELSE NULL::integer END)), public.ci
_builds.id
Sort Method: quicksort Memory: 1423kB
-> Nested Loop (cost=63279.78..64777.27 rows=1 width=1010) (actual time=66.384..298.724 rows=1537 loops=1)
-> HashAggregate (cost=63177.15..63177.30 rows=15 width=15) (actual time=65.641..65.851 rows=187 loops=1)
InitPlan 1 (returns $0)
-> Seq Scan on ci_runners (cost=0.00..26963.66 rows=1 width=4) (actual time=1.145..34.381 rows=1 loops=1)
Filter: is_shared
Rows Removed by Filter: 6965
-> Nested Loop (cost=0.00..36186.34 rows=2715 width=15) (actual time=0.065..29.717 rows=1710 loops=1)
-> Index Scan using index_ci_builds_on_status on ci_builds (cost=0.00..8913.95 rows=3577 width=15) (actual time=0.051..12.012 rows=2583 loops
=1)
Index Cond: ((status)::text = ANY ('{running,pending}'::text[]))
Filter: ((type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 1219
-> Index Scan using projects_pkey on projects (cost=0.00..7.61 rows=1 width=4) (actual time=0.003..0.004 rows=1 loops=2583)
Index Cond: (id = public.ci_builds.gl_project_id)
Filter: ((NOT pending_delete) AND builds_enabled AND shared_runners_enabled)
Rows Removed by Filter: 0
-> Bitmap Heap Scan on ci_builds (cost=102.63..106.64 rows=1 width=1002) (actual time=1.216..1.231 rows=8 loops=187)
Recheck Cond: ((gl_project_id = public.ci_builds.gl_project_id) AND ((status)::text = 'pending'::text))
Filter: ((runner_id IS NULL) AND ((type)::text = 'Ci::Build'::text))
-> BitmapAnd (cost=102.63..102.63 rows=1 width=0) (actual time=1.201..1.201 rows=0 loops=187)
-> Bitmap Index Scan on index_ci_builds_on_gl_project_id (cost=0.00..10.52 rows=241 width=0) (actual time=0.406..0.406 rows=1944 loops=187)
Index Cond: (gl_project_id = public.ci_builds.gl_project_id)
-> Bitmap Index Scan on index_ci_builds_on_status (cost=0.00..91.78 rows=3089 width=0) (actual time=0.652..0.652 rows=3362 loops=187)
Index Cond: ((status)::text = 'pending'::text)
Total runtime: 303.832 ms
```
## Specific runners
It doesn't affect the specific runners which still serve builds FIFO.
@stanhu @markpundsack @yorickpeterse What do you think?
See merge request !4634
* master: (198 commits)
Set inverse_of for Project/Services relation
Fix admin hooks spec
Prevent default disabled buttons and links.
Add index on `requested_at` to the `members` table
Rearrange order of tabs
Fix admin active tab tests
Show created_at in table column
Nest li elements directly under ul
Move builds tab to admin overview
Add monitoring link with subtabs
Add sub links to overview
Add counter for abuse reports
Remove admin layout-nav counters
Move admin nav to horizontal layout nav
Eager load project relations in IssueParser
Use validate and required for environment and project
Award Emoji can't be awarded on system notes backend
Get rid of Gitlab::ShellEnv
Update CHANGELOG.
Fix project star tooltip on the fly.
...
Conflicts:
app/services/ci/create_builds_service.rb
Add environments and deployments
This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.
The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.
The `.gitlab-ci.yml`:
```
deploy to production:
stage: deploy
script: dpl travis...
environment: production
```
What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)
See merge request !4605
Manually create todo for issuable
## What does this MR do?
Adds a button to the sidebar in issues & merge requests to allow users to manually create a todo item themselves.
## What are the relevant issue numbers?
Closes#15045
## Screenshots (if relevant)
See merge request !4502
* master: (147 commits)
Update CHANGELOG
Remove deprecated issues_tracker and issues_tracker_id from project
Schema doesn’t reflect the changes of the last 3 migrations
Revert CHANGELOG
Also rename "find" in the specs
Change to new Notes styleguide
Add guide on changing a document's location
Change logs.md location in README
Move logs/logs.md to administration/logs.md
Make "four phase test"
Only show branches for revert / cherry-pick
Instrument all Banzai::ReferenceParser classes
Removed old comment from update_column_in_batches
Update columns in batches until no rows are left
Remove counters from Pipeline navigation
Handle NULL migration errors in migration helpers
Fix typo causing related branches to Error 500
Improved SVG sanitizer specs to include smoke tests for clean.
Refactored SVG sanitizer
Added SVG sanitizer fix to the changelog
...
* master: (538 commits)
Fix broken URI joining for `teamcity_url` with suffixes
Factorize duplicated code into a method in BambooService and update specs
Fix broken URI joining for `bamboo_url` with suffixes
Honor credentials on calling Bamboo CI trigger
Update CHANGELOG
Use Issue.visible_to_user in Notes.search to avoid query duplication
Project members with guest role can't access confidential issues
Allow users to create confidential issues in private projects
Update CHANGELOG
Remove deprecated issues_tracker and issues_tracker_id from project
Schema doesn’t reflect the changes of the last 3 migrations
Apply reviewer notes: update CHANGELOG, adjust code formatting
Move issue rendering tests into separate contexts
Move change description to proper release and fix typo
Add more information into RSS fead for issues
Revert CHANGELOG
Also rename "find" in the specs
Change to new Notes styleguide
Add guide on changing a document's location
Change logs.md location in README
...
Conflicts:
app/services/ci/create_builds_service.rb
app/services/ci/create_pipeline_service.rb
app/services/create_commit_builds_service.rb
spec/models/ci/commit_spec.rb
spec/services/ci/create_builds_service_spec.rb
spec/services/create_commit_builds_service_spec.rb
+ Move 'Edit Project/Group' out of membership-related partial
+ Show the access request buttons only to logged-in users
+ Put the request access buttons out of in a more visible button
+ Improve the copy in the #remove_member_message helper
Signed-off-by: Rémy Coutable <remy@rymai.me>
Lin Jen-Shin
James Lopez
Douwe Maan
Yorick Peterse
Felipe Artur
Paco Guzman
Rémy Coutable
Robert Speicher
Douglas Barbosa Alexandre
Stan Hu
Grzegorz Bizon
Jacob Vosmaer
Kamil Trzcinski
David AlexanderandRémy Coutable
Phil Hughes
Fatih Acet