199 Commits
Author SHA1 Message Date
Rémy Coutable 06c7d6f3a8 Merge branch 'refactor/ci-config-move-global-entries' into 'master'
Move global ci entries handling from legacy to new config

## What does this MR do?

This MR moves responsibility of handling global CI config entries (like `image`, `services`), from legacy `GitlabCiYamlProcessor` to new CI Config

## Why was this MR needed?

This is the next iteration of CI configuration refactoring

## What are the relevant issue numbers?

#15060

## Does this MR meet the acceptance criteria?

- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)

See merge request !4820
2016-07-05 08:37:16 +00:00
Lin Jen-Shin 596f46881f Merge remote-tracking branch 'upstream/master' into save-artifacts_sizes
* upstream/master: (175 commits)
  Document Repository#keep_around
  Don't garbage collect commits that have related DB records like comments
  Update CHANGELOG
  Update RedCloth to 4.3.2 for CVE-2012-6684
  Fix typo in Merge Requests API documentation
  Downgrade to Redis 3.2.2 due to massive memory leak with Sidekiq
  Enable Style/EmptyLines cop, remove redundant ones
  Update CHANGELOG
  Cache results from jQuery selectors to retrieve namespace name
  Fix import button when import fail due the namespace already been taken
  Fix snippets comments not displayed
  Fix emoji paths in relative root configurations
  Exclude requesters from Project#members, Group#members and User#members
  Upgrade Thin from 1.6.1 to 1.7.0.
  Many squashed commits
  Cache autocomplete results
  Upgrade Sidekiq from 4.1.2 to 4.1.4.
  Upgrade seed-fu from 2.3.5 to 2.3.6
  use has_many relationship with events
  Support creating a todo on issuables via API
  ...
2016-07-04 19:14:54 +08:00
Grzegorz Bizon 9e211091a8 Enable Style/EmptyLines cop, remove redundant ones 2016-07-01 21:56:17 +02:00
Lin Jen-Shin 9d8dca08e4 Use AR callbacks as suggested by:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4964#note_12744656
2016-06-29 17:44:39 +08:00
Grzegorz Bizon 7759242ae5 Move global CI cache configuration to new CI classes 2016-06-29 09:49:46 +02:00
Grzegorz Bizon 2846f95d2a Merge branch 'master' into refactor/ci-config-move-global-entries
* master: (352 commits)
  Display last commit of deleted branch in push events (!4699)
  add changelog
  add missing attribute to attr_encrypted so it is fully backwards-compatible
  Add "GitLab team members only" to diagram link
  doc: note that .gitattributes uses default branch
  use the conf lexer so we have highlighted comments
  first draft of docs
  support cgi style options, such as erb?parent=json
  move the path alias to a more appropriate location
  make #custom_language private
  appease rubocop
  add an alias for Snippet#path
  appease rubocop
  check the tag so that an instance will pass too
  fix the spec, using project.change_head
  Revert "bump the master sha for gitlab-test!9"
  bump the master sha for gitlab-test!9
  add custom highlighting via .gitattributes
  Rename Licenses API to License Templates API
  Check for conflict with wiki projects when creating a new project.
  ...
2016-06-28 14:38:05 +02:00
Lin Jen-Shin de54335958 Prefer Ci::Build#erase_artifacts! 2016-06-28 20:32:32 +08:00
Lin Jen-Shin 0a294698db Just save the size in total rather than individual files
Feedback:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4964#note_12741046
2016-06-28 11:16:48 +00:00
Lin Jen-Shin fe0c59d2e6 Introduce ci_builds.artifacts_sizes as JSON:
We store the sizes as a hash from path to bytes like:

``` ruby
{'ci_artifacts.txt' => 27,
 'other_artifacts_0.1.2/another-subdirectory/banana_sample.gif' =>
71759,
 'other_artifacts_0.1.2/doc_sample.txt' => 1314,
 'rails_sample.jpg' => 35255,
 'tests_encoding/utf8 test dir ✓/regular_file_2' => 7}
```

So that it's easier to access than reading gzip file again.
2016-06-28 18:14:21 +08:00
Rémy Coutable a9dbd394a6 Merge branch 'refactor/ci-config-add-entry-error' into 'master'
Improve validations and error handling in new CI config entries

## What does this MR do?

This MR improves validation in new CI config.

## Why was this MR needed?

With that it will be easier to handle errors during validation and post-processing.

## What are the relevant issue numbers?

This is a continuation of #15060

See merge request !4560
2016-06-27 08:24:17 +00:00
Grzegorz Bizon 823970b570 Fix ci config cache validation in legacy processor 2016-06-24 08:25:10 +02:00
Grzegorz Bizon 1f320edb77 Minor refactorings in new CI configuration classes 2016-06-23 13:51:34 +02:00
Grzegorz Bizon 29b96d92c1 Move CI stages configuration to new CI config 2016-06-23 13:51:07 +02:00
Grzegorz Bizon 05ce8a1187 Handle CI environment variables in a new CI config 2016-06-22 11:22:53 +02:00
Grzegorz Bizon d399128955 Handle after script CI config in new classes
This also makes Script to return an array of commands instead of
concatented command, which is our current direction.
2016-06-21 13:02:14 +02:00
Grzegorz Bizon fc00c545b2 Handle CI services config in new CI config classes 2016-06-21 12:40:52 +02:00
Grzegorz Bizon cd6a2afbbb Move CI image configuration entry to new CI config 2016-06-21 12:10:13 +02:00
Lin Jen-Shin aeb24ee872 Merge remote-tracking branch 'upstream/master' into feature/runner-lock-on-project
* upstream/master: (353 commits)
  Put some admin settings in dropdown
  Add styleguide on configuration settings documentation
  Remove Duplicated keys add UNIQUE index to fingerprint
  Avoid autoload issue such as 'Mail::Parsers::AddressStruct'
  Move appearance settings as sub tab to application settings
  use rails root join
  fixed a couple of errors spotted in production
  Fix RangeError exceptions when referring to issues or merge requests outside of max database values
  Fix bug in `WikiLinkFilter`.
  Grammar and typographic changes to artifacts documentation
  Tweak grammar
  Small frontend code fixes and restore 8a2d88f commit
  Warn about admin privilege to disable GitHub Webhooks
  Listing GH Webhooks doesn't stop import process for non GH admin users
  fixup! updated docs for api endpoint award emoji
  Update CHANGELOG
  Ensure Todos counters doesn't count Todos for projects pending delete
  Add endpoints for award emoji on notes
  Sort API endpoints and implement feedback
  Add endpoints for Award Emoji
  ...
2016-06-20 14:04:14 +00:00
Grzegorz Bizon 9510d31b4d Merge branch 'master' into refactor/ci-config-add-entry-error
* master: (345 commits)
  use rails root join
  fixed a couple of errors spotted in production
  Fix RangeError exceptions when referring to issues or merge requests outside of max database values
  Fix bug in `WikiLinkFilter`.
  Small frontend code fixes and restore 8a2d88f commit
  Warn about admin privilege to disable GitHub Webhooks
  Listing GH Webhooks doesn't stop import process for non GH admin users
  fixup! updated docs for api endpoint award emoji
  Update CHANGELOG
  Ensure Todos counters doesn't count Todos for projects pending delete
  Add endpoints for award emoji on notes
  Sort API endpoints and implement feedback
  Add endpoints for Award Emoji
  Fixed issue with assignee dropdown not selecting correctly
  Removed update method Re-structured controller spec Renamed issuable param to issuable_id
  Fix clibpoard buttons on "Check out branch" modal.
  Track method call times/counts as a single metric
  Cache todo counters (pending/done)
  Fix a 'wrong number of arguments' error
  Added missing mount point for Sidekiq Metrics API, after it got lost on rebase.
  ...
2016-06-20 10:38:46 +02:00
Rémy Coutable ae4491b421 Merge branch 'fix/error-when-job-variables-not-defined-but-specified' into 'master'
Fix error when CI job variables key used but not specified

## What does this MR do?

This MR fixes a an error when CI job variables specified, but not defined:

```yaml
image: ruby:2.2

test:
  variables:
  script:
     - rspec
```

## What are the relevant issue numbers?

Closes #18764  
Follow up discussion in: #18775 

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4745
2016-06-17 15:10:06 +00:00
Grzegorz Bizon eaa91cbe12 Fix error when CI job variables not specified 2016-06-17 14:14:55 +02:00
Lin Jen-Shin 5174d6c146 Merge remote-tracking branch 'upstream/master' into feature/runner-lock-on-project
* upstream/master: (337 commits)
  Update CHANGELOG for !4659
  Center the header logo for all Devise emails
  Add previews for all customized Devise emails
  Customize the Devise `unlock_instructions` email
  Customize the Devise `reset_password_instructions` email
  Customize the Devise `password_change` emails
  Use gitlab-git 10.2.0
  Use Git cached counters on project show page
  Fix indentation scss-lint errors
  Added title attribute to enties in tree view Closes #18353
  Banzai::Filter::ExternalLinkFilter use XPath
  Reduce queries in IssueReferenceFilter
  Use gitlab_git 10.1.4
  Fixed ordering in Project.find_with_namespace
  Fix images in emails
  Banzai::Filter::UploadLinkFilter use XPath
  Turn Group#owners into a has_many association
  Make project_id nullable
  CHANGELOG [ci skip]
  CHANGELOG [ci skip]
  ...
2016-06-17 19:52:55 +08:00
Grzegorz Bizon 2a87a55f87 Merge branch 'master' into refactor/ci-config-add-entry-error
* master: (189 commits)
  Update CHANGELOG for !4659
  Center the header logo for all Devise emails
  Add previews for all customized Devise emails
  Customize the Devise `unlock_instructions` email
  Customize the Devise `reset_password_instructions` email
  Customize the Devise `password_change` emails
  Use gitlab-git 10.2.0
  Use Git cached counters on project show page
  Fix indentation scss-lint errors
  Added title attribute to enties in tree view Closes #18353
  Banzai::Filter::ExternalLinkFilter use XPath
  Reduce queries in IssueReferenceFilter
  Use gitlab_git 10.1.4
  Fixed ordering in Project.find_with_namespace
  Fix images in emails
  Banzai::Filter::UploadLinkFilter use XPath
  Turn Group#owners into a has_many association
  Make project_id nullable
  ...
2016-06-17 12:25:43 +02:00
Grzegorz Bizon a9bd16bd0a Rename legacy validation helpers for new ci config 2016-06-17 11:55:55 +02:00
Kamil Trzcinski aef6214c42 Validate only and except regexp
Currently the RegexpError can be raised when processing next stage which leads to 500 in different places of code base.
This adds early check that regexps used in only and except are valid.
2016-06-17 11:52:22 +02:00
Grzegorz Bizon d8670e114a Merge branch 'master' into fix/status-of-pipeline-without-builds
* master: (198 commits)
  Set inverse_of for Project/Services relation
  Fix admin hooks spec
  Prevent default disabled buttons and links.
  Add index on `requested_at` to the `members` table
  Rearrange order of tabs
  Fix admin active tab tests
  Show created_at in table column
  Nest li elements directly under ul
  Move builds tab to admin overview
  Add monitoring link with subtabs
  Add sub links to overview
  Add counter for abuse reports
  Remove admin layout-nav counters
  Move admin nav to horizontal layout nav
  Eager load project relations in IssueParser
  Use validate and required for environment and project
  Award Emoji can't be awarded on system notes backend
  Get rid of Gitlab::ShellEnv
  Update CHANGELOG.
  Fix project star tooltip on the fly.
  ...

Conflicts:
	app/services/ci/create_builds_service.rb
2016-06-15 20:14:25 +02:00
Lin Jen-Shin fd285f71d8 Merge branch 'master' into feature/runner-lock-on-project
* master: (147 commits)
  Update CHANGELOG
  Remove deprecated issues_tracker and issues_tracker_id from project
  Schema doesn’t reflect the changes of the last 3 migrations
  Revert CHANGELOG
  Also rename "find" in the specs
  Change to new Notes styleguide
  Add guide on changing a document's location
  Change logs.md location in README
  Move logs/logs.md to administration/logs.md
  Make "four phase test"
  Only show branches for revert / cherry-pick
  Instrument all Banzai::ReferenceParser classes
  Removed old comment from update_column_in_batches
  Update columns in batches until no rows are left
  Remove counters from Pipeline navigation
  Handle NULL migration errors in migration helpers
  Fix typo causing related branches to Error 500
  Improved SVG sanitizer specs to include smoke tests for clean.
  Refactored SVG sanitizer
  Added SVG sanitizer fix to the changelog
  ...
2016-06-14 23:05:26 +08:00
Kamil Trzcinski 006b650988 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	db/schema.rb
2016-06-14 16:19:29 +02:00
Kamil Trzcinski 975e388751 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	lib/ci/gitlab_ci_yaml_processor.rb
2016-06-14 14:53:10 +02:00
Kamil Trzcinski 3f5819fb7a Merge remote-tracking branch 'origin/master' into artifacts-expire-date 2016-06-14 14:49:59 +02:00
Grzegorz Bizon ab9a8643d8 Merge branch 'master' into fix/status-of-pipeline-without-builds
* master: (538 commits)
  Fix broken URI joining for `teamcity_url` with suffixes
  Factorize duplicated code into a method in BambooService and update specs
  Fix broken URI joining for `bamboo_url` with suffixes
  Honor credentials on calling Bamboo CI trigger
  Update CHANGELOG
  Use Issue.visible_to_user in Notes.search to avoid query duplication
  Project members with guest role can't access confidential issues
  Allow users to create confidential issues in private projects
  Update CHANGELOG
  Remove deprecated issues_tracker and issues_tracker_id from project
  Schema doesn’t reflect the changes of the last 3 migrations
  Apply reviewer notes: update CHANGELOG, adjust code formatting
  Move issue rendering tests into separate contexts
  Move change description to proper release and fix typo
  Add more information into RSS fead for issues
  Revert CHANGELOG
  Also rename "find" in the specs
  Change to new Notes styleguide
  Add guide on changing a document's location
  Change logs.md location in README
  ...

Conflicts:
	app/services/ci/create_builds_service.rb
	app/services/ci/create_pipeline_service.rb
	app/services/create_commit_builds_service.rb
	spec/models/ci/commit_spec.rb
	spec/services/ci/create_builds_service_spec.rb
	spec/services/create_commit_builds_service_spec.rb
2016-06-14 13:44:03 +02:00
Kamil Trzcinski e8f09f02bf Validate environment name with regex 2016-06-14 13:04:21 +02:00
Rémy Coutable 47cdb69921 Merge branch 'refactor/ci-config-add-global-entry' into 'master'
Add global entry with before script to new CI config

## What does this MR do?

This MR adds a new entries to a new CI config class. It is next refactoring step after !4462.

See #15060

See merge request !4482
2016-06-14 10:53:26 +00:00
Kamil Trzcinski 7d9e8b04c0 Merge remote-tracking branch 'origin/master' into environments-and-deployments 2016-06-14 12:41:18 +02:00
Kamil Trzcinski 0fab7b11ba Merge remote-tracking branch 'origin/master' into artifacts-when 2016-06-13 16:05:43 +02:00
Kamil TrzcinskiandPhil Hughes 421be01dab Improve design based on review 2016-06-13 11:09:19 +01:00
Kamil TrzcinskiandPhil Hughes d23b91b0d9 Improve after review 2016-06-13 11:08:15 +01:00
Kamil TrzcinskiandPhil Hughes 86800bf51a Support expiration date in CI API when uploading artifacts 2016-06-13 11:03:30 +01:00
Kamil TrzcinskiandPhil Hughes 1501940ee0 Validate artifacts:expire_in in yaml processor 2016-06-13 11:03:30 +01:00
Kamil TrzcinskiandPhil Hughes aea4041ce9 Allow to expire build artifacts 2016-06-13 11:01:19 +01:00
Kamil TrzcinskiandPhil Hughes 7a1b2e4f94 Added when to artifacts 2016-06-13 10:45:24 +01:00
Grzegorz Bizon ebe14437e9 Merge branch 'master' into refactor/ci-config-add-global-entry
* master: (59 commits)
  Improved SVG sanitizer specs to include smoke tests for clean.
  Refactored SVG sanitizer
  Added SVG sanitizer fix to the changelog
  Refactor SVG sanitizer and prevent `xlink:href` to refer to external resources
  Fix SVG whitelisting to allow namespaced attributes
  Fix Error 500 when using closes_issues API with an external issue tracker
  Center layout navigation and remove icons
  Fix preferences_spec test
  Add back sidebar counters and username
  Only create the backup directory if it is local
  Fix safari logo loading animation safari bug
  Revert side nav to full width; remove border under nav; remove tooltips on nav links; stop page content shifting with side nav; put project nav in container
  Update media queries
  Fix profile test
  Fix logo at all screen widths, update sidebar text
  Move tanuki icon to center of nav bar; keep nav closed by default; remove collapsed nav cookie
  Remove unused MergeRequest#gitlab_merge_status method
  Add CHANGELOG item for labels/milestones navigation change
  Render issues link on issues subnav unless you visit merge request controller
  Render only issues/mr in subnav depends on context
  ...
2016-06-13 11:14:38 +02:00
Kamil Trzcinski 907c0e6796 Added initial version of deployments 2016-06-10 23:36:54 +02:00
Kamil Trzcinski 3714e1914b Improve after review 2016-06-10 21:25:48 +02:00
Kamil Trzcinski c43279a8d9 Fix expose of TriggerRequest 2016-06-10 13:58:33 +02:00
Kamil Trzcinski 4663ae064d Fix CI TriggerRequest entity 2016-06-09 17:57:07 +02:00
Lin Jen-Shin 781d35c191 Prefer attributes_for_keys so that it ignores nils
Also add a test for setting locked.
2016-06-09 16:00:24 +08:00
Lin Jen-Shin 1c302d566b WIP, try to add views for locked runners 2016-06-09 16:00:24 +08:00
Kamil Trzcinski 81dfabad39 Added when to artifacts 2016-06-08 17:10:58 +02:00
Grzegorz Bizon cba266aabc Remove old before_script from legacy Ci config 2016-06-08 11:44:07 +02:00