Commit Graph
1949 Commits
Author SHA1 Message Date
Rémy CoutableandRobert Speicher ac7cea3084 Merge branch 'fix/gitlab-import-project-file-fix' into 'master'
Fix temp file being deleted after the request while importing a GitLab project

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151

In production, the temporary uploaded file is getting deleted straight after the request so the Sidekiq worker is unable to find it in `/tmp`

Also, improved erroring/logging of this situation.

See merge request !4894
2016-06-24 11:31:41 -04:00
Rémy CoutableandRobert Speicher 735d5cbc7d Merge branch 'fix/import-export-gitlab-errors' into 'master'
Fix errors found on importing GitLab CE repo

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18968

See merge request !4855
2016-06-23 14:36:40 -04:00
Robert SpeicherandRobert Speicher a767bb4690 Merge branch 'fix_saml_signin' into 'master'
Fix subsequent SAML sign ins

Fixes a bug when `auto_link_ldap_user` is `true` that causes SAML users to be unable to sign in a second time.

Fix the problem for https://gitlab.zendesk.com/agent/tickets/22546

See merge request !4718
2016-06-23 14:28:43 -04:00
Douwe MaanandRémy Coutable 9af7d71bd6 Merge branch '18755-fix-destroy-project-causes-post_decline_request-to-be-executed' into 'master'
Resolve "Destroying a project causes post_decline_request to be executed"

## What does this MR do?

Ensure we don't send "access request declined" to access requesters when a project is deleted.

## Are there points in the code the reviewer needs to double check?

I've created a service to decouple the notification sending from the AR model.

## Why was this MR needed?

Because there was an issue.

## What are the relevant issue numbers?

Fixes #18755, #18750.

## Does this MR meet the acceptance criteria?

- [x] No CHANGELOG needed.
- [x] Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4744

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-06-22 15:50:04 +02:00
Jacob SchatzandRobert Speicher 355310ba98 Merge branch '17521-gitlab-ci-yml-templates' into 'master'
GitLab CI Yaml template dropdown

## What does this MR do?
Make it possible to select a dropdown for an easy start with GitLab CI.

## What are the relevant issue numbers?
Closes #17521

## TODO
- [ ] Backend
  - [x] CHANGELOG item
  - [x] Fix rubocop failure
  - [x] API Support
  - [x] New tests
  - [x] Add disclaimer to the top of the gitlab-ci.yml
- [ ] Frontend
  - [x] New tests

See merge request !4411
2016-06-21 12:17:47 -04:00
Rémy CoutableandRobert Speicher 6d0f9bbc6b Merge branch 'ci-lfs-fetch' into 'master'
Allow to fetch LFS from CI

## What does this MR do?

This adds support for fetching LFS object from CI jobs (mostly it's made for supporting GitLab CI).

## What is left?

- [x] Write tests covering a new authorization mechanism

cc @grzesiek @marin

See merge request !4465
2016-06-21 11:50:51 -04:00
Rémy CoutableandRobert Speicher 95aaf66ec3 Merge branch 'fix/import-export-prod' into 'master'
Fixed a couple of errors on import/export spotted in production

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18828 and https://gitlab.com/gitlab-org/gitlab-ce/issues/18829

A couple of tiny bugs spotted after deploying in prod

See merge request !4782
2016-06-20 10:16:14 -04:00
Robert SpeicherandRobert Speicher 7716c7e84a Merge branch 'fix-out-of-bounds-markdown-refs' into 'master'
Fix RangeError exceptions when referring to issues or merge requests outside of max database values

When using #XYZ in Markdown text, if XYZ exceeds the maximum value of a signed 32-bit integer, we get an exception when the Markdown render attempts to run `where(iids: XYZ)`. Introduce a method that will throw out out-of-bounds values.
    
Closes #18777

See merge request !4777
2016-06-20 10:16:00 -04:00
Robert SpeicherandRobert Speicher f279ce631d Merge branch 'limit-email-diff-size' into 'master'
Limit push email diff size

Restrict the size of diffs in push emails to 30 lines / 150 KB (whichever is smaller),
following https://gitlab.com/gitlab-org/gitlab_git/merge_requests/85.

Emails on push can get very large if they contain a lot of files that sneak under the single-file limit.

https://gitlab.com/gitlab-org/gitlab-ee/issues/490

See merge request !4566
2016-06-20 10:14:20 -04:00
Robert SpeicherandRobert Speicher a41481b85d Merge branch 'gh-webhooks' into 'master'
Avoid that GitHub import fails when retrieving Webhooks for non GitHub admins

Closes #18729 

See merge request !4723
2016-06-17 22:10:08 -04:00
Robert SpeicherandRobert Speicher c6b9925335 Merge branch 'track-total-method-call-time' into 'master'
Track method call times/counts as a single metric

This changes method call tracking so only a single metric is emitted regardless of the number of calls. This allows us to more accurately measure the total execution time of a method as well as the number of times a method is called. See 851e3ff7578973c2206628424eac3b951a3c656d for more details.

Method call tracking tracked calls individually meaning the end statistics may not always be accurate enough to get a good understanding of where time is spent.

See merge request !4754
2016-06-17 14:23:05 -04:00
Yorick PeterseandRobert Speicher 5e3f9ea1b8 Merge branch 'secure-request-uris' into 'master'
Filter out sensitive parameters of metrics data

See merge request !4748
2016-06-17 14:22:43 -04:00
Douwe MaanandRobert Speicher bf0294378e Merge branch 'feature/project-export' into 'master'
Export project functionality

This is a MR for the export functionality of https://gitlab.com/gitlab-org/gitlab-ce/issues/3050, which adds the ability to export single projects.

- [x] members
- DB data
  - [x] issues
  - [x] issue comments
  - [x] merge requests
  - [x] merge request diff
  - [x] merge request comments
  - [x] labels
  - [x] milestones
  - [x] snippets
  - [x] releases
  - [x] events
  - [x] commit statuses
  - [x] CI builds
- File system data
  - [x] Git repository
  - [x] wiki
  - [x] uploads
  - [ ] ~~CI build traces~~
  - [ ] ~~CI build artifacts~~
  - [ ] ~~LFS objects~~
- DB configuration
  - [x] services
  - [x] web hooks
  - [x] protected branches
  - [x] deploy keys
  - [x] CI variables
  - [x] CI triggers

See merge request !3114
2016-06-17 14:18:15 -04:00
Rémy CoutableandRobert Speicher 4a6eb00729 Merge branch 'validate-only-except-regexp' into 'master'
Validate only and except regexp

## What does this MR do?
Adds a better validation for only and except which can contain regexps.

## Why was this MR needed?
Currently the RegexpError can be raised when processing next stage which leads to 500 in different places of code base.
This adds early check that regexps used in only and except are valid.

cc @grzesiek 

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4736
2016-06-17 14:15:14 -04:00
Robert SpeicherandRobert Speicher 432347426c Merge branch 'update-column-in-batches-where' into 'master'
Allow customising of queries used for `update_column_in_batches`

This MR makes two changes to `add_column_with_default` and `update_column_in_batches`:

1. `add_column_with_default` no longer wraps the entire set of updates in a single transaction, preventing any locks from sticking around for the duration of the entire transaction
2. `update_column_in_batches` now takes a block which can be used to customise the queries. This uses Arel as messing with raw SQL strings is a total pain

In !4381 there's a need for updating existing rows/columns in a table in batches using a custom `WHERE` condition. Without the changes in this MR this would not be possible.

See merge request !4680
2016-06-17 14:14:25 -04:00
Douwe Maan af493ccf36 Merge branch 'remove-shell-env' into 'master'
Get rid of Gitlab::ShellEnv

Remove obsolete code we once needed for Grack and satellites.

See merge request !4673
2016-06-15 16:41:13 +00:00
Rémy Coutable be09845914 Merge branch 'environments-and-deployments' into 'master'
Add environments and deployments

This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.

The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.

The `.gitlab-ci.yml`:
```
deploy to production:
  stage: deploy
  script: dpl travis...
  environment: production
```

What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill 
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)

See merge request !4605
2016-06-15 13:48:09 +00:00
Jacob Vosmaer c32e61251e Get rid of Gitlab::ShellEnv 2016-06-15 14:59:37 +02:00
Z.J. van de Weg b2df118561 Random selection now also works for MySQL 2016-06-15 12:10:41 +02:00
Yorick Peterse ab91f1226f Filter out classes without names in the sampler
We can't do a lot with classes without names as we can't filter by them,
have no idea where they come from, etc. As such it's best to just ignore
these.
2016-06-14 18:09:06 +02:00
Yorick Peterse 0ca7b3ba37 Merge branch '18449-instrument-grape-endpoints' into 'master'
Instrument Grape API endpoints

See merge request !4587
2016-06-14 14:29:55 +00:00
Yorick Peterse f558bf0de4 Merge branch '18527-instrument-private-methods' into 'master'
Instrument private methods and instance private methods

See merge request !4639
2016-06-14 14:28:40 +00:00
Kamil Trzcinski 006b650988 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	db/schema.rb
2016-06-14 16:19:29 +02:00
Paco Guzman dadc531353 Instrument private/protected methods
By default instrumentation will instrument public,
protected and private methods, because usually
heavy work is done on private method or at least
that’s what facts is showing
2016-06-14 15:17:51 +02:00
Kamil Trzcinski 975e388751 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	lib/ci/gitlab_ci_yaml_processor.rb
2016-06-14 14:53:10 +02:00
Kamil Trzcinski 3f5819fb7a Merge remote-tracking branch 'origin/master' into artifacts-expire-date 2016-06-14 14:49:59 +02:00
Yorick Peterse 4b964011cf Merge branch '18528-cpu-time-instrumentation' into 'master'
Measure CPU time for instrumented methods

See merge request !4640
2016-06-14 11:39:19 +00:00
Paco Guzman 509082bafb Instrument Grape Endpoint with Metrics::RackMiddleware
Generating the following tags

Grape#GET /projects/:id/archive

from Grape::Route objects like

{ :path => /:version/projects/:id/archive(.:format)
  :version => “v3”,
  :method => “GET” }

Use an instance variable to cache raw_path transformations.
This variable is only going to growth to the number of 
endpoints of the API, not with exact different requests

We can store this cache as an instance variable because 
middleware are initialised only once
2016-06-14 13:06:46 +02:00
Kamil Trzcinski e8f09f02bf Validate environment name with regex 2016-06-14 13:04:21 +02:00
Rémy Coutable 47cdb69921 Merge branch 'refactor/ci-config-add-global-entry' into 'master'
Add global entry with before script to new CI config

## What does this MR do?

This MR adds a new entries to a new CI config class. It is next refactoring step after !4462.

See #15060

See merge request !4482
2016-06-14 10:53:26 +00:00
Paco Guzman 120fbbd487 Measure CPU time for instrumented methods 2016-06-14 12:49:31 +02:00
Grzegorz Bizon 30e946ce8a Validate ci config entry value before processing nodes 2016-06-14 11:28:20 +02:00
Grzegorz Bizon 59eeec3ff8 Make method that composes ci config entry private 2016-06-14 10:48:51 +02:00
Douwe Maan 1cbd5c6686 Merge branch 'gitlab-auth-method-names' into 'master'
Improve Gitlab::Auth method names

Auth.find was a very generic name for a very specific method.
Auth.find_in_gitlab_or_ldap was inaccurate in GitLab EE where it also
looks in Kerberos.


See merge request !4589
2016-06-13 18:05:53 +00:00
Yorick Peterse ea7ff13410 Removed old comment from update_column_in_batches 2016-06-13 11:50:27 +02:00
Yorick Peterse 9c238dc970 Update columns in batches until no rows are left
Instead of updating a fixed number of rows (based on the amount of rows
available at the start of the update) the method
"update_column_in_batches" will now continue updating rows until it runs
out of rows to process.

For a table with a high rate of inserts this may result in the migration
taking quite some time. However, the alternative is not all rows being
updated or the "change_column_null" method raising an error due to there
being NULL values.
2016-06-13 11:38:57 +02:00
Yorick Peterse b33b7be53e Handle NULL migration errors in migration helpers
This ensures that whenever changing the NULL constraint of a column
fails we still drop the column.
2016-06-13 11:22:58 +02:00
Grzegorz Bizon ebe14437e9 Merge branch 'master' into refactor/ci-config-add-global-entry
* master: (59 commits)
  Improved SVG sanitizer specs to include smoke tests for clean.
  Refactored SVG sanitizer
  Added SVG sanitizer fix to the changelog
  Refactor SVG sanitizer and prevent `xlink:href` to refer to external resources
  Fix SVG whitelisting to allow namespaced attributes
  Fix Error 500 when using closes_issues API with an external issue tracker
  Center layout navigation and remove icons
  Fix preferences_spec test
  Add back sidebar counters and username
  Only create the backup directory if it is local
  Fix safari logo loading animation safari bug
  Revert side nav to full width; remove border under nav; remove tooltips on nav links; stop page content shifting with side nav; put project nav in container
  Update media queries
  Fix profile test
  Fix logo at all screen widths, update sidebar text
  Move tanuki icon to center of nav bar; keep nav closed by default; remove collapsed nav cookie
  Remove unused MergeRequest#gitlab_merge_status method
  Add CHANGELOG item for labels/milestones navigation change
  Render issues link on issues subnav unless you visit merge request controller
  Render only issues/mr in subnav depends on context
  ...
2016-06-13 11:14:38 +02:00
Grzegorz Bizon 11c0d02283 Simplify ci config node factory 2016-06-13 09:58:35 +02:00
Grzegorz Bizon 7c8f3b0cfc Duplicate CI config node factory on class level 2016-06-13 08:50:12 +02:00
Gabriel MazettoandRobert Speicher a9eaa20dcb Refactored SVG sanitizer 2016-06-12 21:05:58 -04:00
Gabriel MazettoandRobert Speicher 13791c6704 Refactor SVG sanitizer and prevent xlink:href to refer to external resources 2016-06-12 21:05:58 -04:00
Gabriel MazettoandRobert Speicher 02b882418a Fix SVG whitelisting to allow namespaced attributes 2016-06-12 21:05:57 -04:00
Robert Speicher 37be2007f9 Merge branch 'workhorse-helpers' into 'master'
Add workhorse controller and API helpers

Adds `send_git_blob` and `send_git_archive` controller and API helpers to reduce duplication and make Workhorse easier for a developer to work with.

See merge request !4486
2016-06-10 16:21:26 +00:00
Jacob Vosmaer 0e896ffe4e Improve Gitlab::Auth method names
Auth.find was a very generic name for a very specific method.
Auth.find_in_gitlab_or_ldap was inaccurate in GitLab EE where it also
looks in Kerberos.
2016-06-10 14:51:16 +02:00
Grzegorz Bizon cc373a3550 Add factory for fabricating new ci config nodes 2016-06-10 14:01:07 +02:00
Douwe Maan a9a9f19b0b Merge branch 'gh-rate-limit' into 'master'
Wrap all rate limiting logic inside GitHub API client

## What does this MR do?

Move the actual rate limiting logic to GitHub API to clean the code inside the GitHub importer, and avoid code duplication.

## Are there points in the code the reviewer needs to double check?

No there aren't.

## Why was this MR needed?

Avoid code duplication to handle API rate limit in every call to the GitHub API.

## What are the relevant issue numbers?

There are none.

## Screenshots (if relevant)

Not relevant.

See merge request !4552
2016-06-10 10:34:19 +00:00
Douwe Maan 16bd4e5668 Merge branch 'master' into workhorse-helpers 2016-06-10 12:28:04 +02:00
Grzegorz Bizon 5abfc7fa71 Define ci entry accessor instead of method_missing 2016-06-10 11:32:49 +02:00
Grzegorz Bizon 12080ba150 Simplify new ci config entry class interface 2016-06-10 11:26:03 +02:00