Commit Graph
1077 Commits
Author SHA1 Message Date
Robert Speicher cef021917f Merge branch 'fix_saml_signin' into 'master'
Fix subsequent SAML sign ins

Fixes a bug when `auto_link_ldap_user` is `true` that causes SAML users to be unable to sign in a second time.

Fix the problem for https://gitlab.zendesk.com/agent/tickets/22546

See merge request !4718
2016-06-22 22:12:09 +00:00
Robert Speicher 69479d3615 Merge branch 'fix-external-issue-links' into 'master'
Handle external issues in IssueReferenceFilter

Handling of external issues was broken when I refactored
`IssueReferenceFilter` to use fewer SQL queries.

Fixes #18827

See merge request !4789
2016-06-22 21:40:17 +00:00
Drew Blessing 938f2b9979 Fix subsequent SAML sign ins 2016-06-22 16:10:55 -05:00
Yorick Peterse ceeba75c76 Handle external issues in IssueReferenceFilter
In the past this class would use Project#get_issue to retrieve an issue
by its ID. This method would automatically determine whether to return
an Issue or ExternalIssue.

This commit changes IssueReferenceFilter to handle external issues again
and in a somewhat more explicit manner than before.

Fixes gitlab-org/gitlab-ce#18827
2016-06-21 20:01:56 +02:00
Yorick Peterse 15a87f3129 Merge branch '18590-banzai-filter-relativelinkfilter-is-slow' into 'master'
Optimize Banzai::Filter::RelativeLinkFilter

See merge request !4813
2016-06-21 15:28:59 +00:00
Alejandro Rodríguez ca696175db Optimize Banzai::Filter::RelativeLinkFilter
A lot of git operations were being repeated, for example, to build a url
    you would ask if the path was a Tree, which would call a recursive routine
    in Gitlab::Git::Tree#where, then ask if the path was a Blob, which would
    call a recursive routine at Gitlab::Git::Blob#find, making reference to
    the same git objects several times. Now we call Rugged::Tree#path, which
    allows us to determine the type of the path in one pass.

    Some other minor improvement added, like saving commonly used references
    instead of calculating them each time.
2016-06-21 10:53:16 -04:00
Rémy Coutable ed519c161b Merge branch 'ci-lfs-fetch' into 'master'
Allow to fetch LFS from CI

## What does this MR do?

This adds support for fetching LFS object from CI jobs (mostly it's made for supporting GitLab CI).

## What is left?

- [x] Write tests covering a new authorization mechanism

cc @grzesiek @marin

See merge request !4465
2016-06-21 13:05:35 +00:00
Kamil Trzcinski 6bc22d95b8 Add test coverage to LFS fetching 2016-06-21 11:26:44 +02:00
Kamil Trzcinski 2f545a15c0 Merge remote-tracking branch 'origin/master' into ci-lfs-fetch 2016-06-21 10:11:14 +02:00
ZJ van de WegandAlfredo Sumaran 27bf7ae59e Refactor Gitlab::Gitignores 2016-06-20 14:48:28 -05:00
Robert Speicher 98cede7ebe Merge branch 'fix-out-of-bounds-markdown-refs' into 'master'
Fix RangeError exceptions when referring to issues or merge requests outside of max database values

When using #XYZ in Markdown text, if XYZ exceeds the maximum value of a signed 32-bit integer, we get an exception when the Markdown render attempts to run `where(iids: XYZ)`. Introduce a method that will throw out out-of-bounds values.
    
Closes #18777

See merge request !4777
2016-06-18 20:48:58 +00:00
Stan Hu 7c9eba8919 Fix RangeError exceptions when referring to issues or merge requests outside of max database values
When using #XYZ in Markdown text, if XYZ exceeds the maximum value of a signed 32-bit integer, we
get an exception when the Markdown render attempts to run `where(iids: XYZ)`. Introduce a method
that will throw out out-of-bounds values.

Closes #18777
2016-06-18 13:07:38 -07:00
Timothy Andrew 6d169d36ca Fix bug in WikiLinkFilter.
1. An exception would be raised if the filter was called with an invalid
   URI. Mainly because we weren't catching the `Addressable` exception.

2. This commit fixes it and adds a spec for the filter.
2016-06-18 14:10:40 +05:30
Yorick PeterseandRobert Speicher be3b878443 Track method call times/counts as a single metric
Previously we'd create a separate Metric instance for every method call
that would exceed the method call threshold. This is problematic because
it doesn't provide us with information to accurately get the _total_
execution time of a particular method. For example, if the method
"Foo#bar" was called 4 times with a runtime of ~10 milliseconds we'd end
up with 4 different Metric instances. If we were to then get the
average/95th percentile/etc of the timings this would be roughly 10
milliseconds. However, the _actual_ total time spent in this method
would be around 40 milliseconds.

To solve this problem we now create a single Metric instance per method.
This Metric instance contains the _total_ real/CPU time and the call
count for every instrumented method.
2016-06-17 13:09:55 -04:00
Rémy Coutable 1ca1ebc09b Merge branch 'registry-500-fix' into 'master'
Properly support application/json in Container Registry

## What does this MR do?
When requesting tags a `application/json` is used by `docker/distribution`.

## Why was this MR needed?
Fixes regression introduced by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4669

## What are the relevant issue numbers?
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18736


See merge request !4742
2016-06-17 17:06:55 +00:00
Yorick Peterse ed5f17cc73 Merge branch 'secure-request-uris' into 'master'
Filter out sensitive parameters of metrics data

See merge request !4748
2016-06-17 16:43:14 +00:00
Robert Speicher 8dccfb4a9c Merge branch 'update-column-in-batches-where' into 'master'
Allow customising of queries used for `update_column_in_batches`

This MR makes two changes to `add_column_with_default` and `update_column_in_batches`:

1. `add_column_with_default` no longer wraps the entire set of updates in a single transaction, preventing any locks from sticking around for the duration of the entire transaction
2. `update_column_in_batches` now takes a block which can be used to customise the queries. This uses Arel as messing with raw SQL strings is a total pain

In !4381 there's a need for updating existing rows/columns in a table in batches using a custom `WHERE` condition. Without the changes in this MR this would not be possible.

See merge request !4680
2016-06-17 16:36:22 +00:00
Paco Guzman 2e552c6bf0 Filter out sensitive parameters of metrics data 2016-06-17 18:14:25 +02:00
Douwe Maan 9b7fd74805 Merge branch 'feature/project-export' into 'master'
Export project functionality

This is a MR for the export functionality of https://gitlab.com/gitlab-org/gitlab-ce/issues/3050, which adds the ability to export single projects.

- [x] members
- DB data
  - [x] issues
  - [x] issue comments
  - [x] merge requests
  - [x] merge request diff
  - [x] merge request comments
  - [x] labels
  - [x] milestones
  - [x] snippets
  - [x] releases
  - [x] events
  - [x] commit statuses
  - [x] CI builds
- File system data
  - [x] Git repository
  - [x] wiki
  - [x] uploads
  - [ ] ~~CI build traces~~
  - [ ] ~~CI build artifacts~~
  - [ ] ~~LFS objects~~
- DB configuration
  - [x] services
  - [x] web hooks
  - [x] protected branches
  - [x] deploy keys
  - [x] CI variables
  - [x] CI triggers

See merge request !3114
2016-06-17 15:35:27 +00:00
Rémy Coutable ae4491b421 Merge branch 'fix/error-when-job-variables-not-defined-but-specified' into 'master'
Fix error when CI job variables key used but not specified

## What does this MR do?

This MR fixes a an error when CI job variables specified, but not defined:

```yaml
image: ruby:2.2

test:
  variables:
  script:
     - rspec
```

## What are the relevant issue numbers?

Closes #18764  
Follow up discussion in: #18775 

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4745
2016-06-17 15:10:06 +00:00
Grzegorz Bizon eaa91cbe12 Fix error when CI job variables not specified 2016-06-17 14:14:55 +02:00
Kamil Trzcinski 04ffd71e53 Fix regression introduced by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4669
When requesting tags a `application/json` is used.
2016-06-17 13:24:50 +02:00
Kamil Trzcinski aef6214c42 Validate only and except regexp
Currently the RegexpError can be raised when processing next stage which leads to 500 in different places of code base.
This adds early check that regexps used in only and except are valid.
2016-06-17 11:52:22 +02:00
Robert Speicher 79c28e1057 Merge branch 'remove_jiraissue' into 'master'
Remove JiraIssue model and replace references with ExternalIssue

This MR Removes unused JiraIssue class and replaces references with ExternalIssue

Closes #18203 

See merge request !4659
2016-06-17 00:24:10 +00:00
Douwe Maan 84632f0aff Merge branch 'banzai-issue-filter-queries' into 'master'
Reduce SQL query counts in IssueReferenceFilter

## What does this MR do?

This MR adds a preparation phase for reference filters that allows them to prepare/create data structures used while iterating over HTML nodes. In this particular case the preparation phase is used for issue references to greatly cut down the amount of queries executed to get projects/issues for Markdown references.

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

Rendering Markdown containing issue references would run at most two queries for every issue reference: one to get the project and one to get the issue from said project. When rendering Markdown with lots of issue references this would result in _a lot_ of queries being executed.

## What are the relevant issue numbers?

#18042

See merge request !4410
2016-06-16 20:49:13 +00:00
James Lopez b3a0a1c315 fixed merge conflicts on UI branch 2016-06-16 15:01:27 +02:00
Paco Guzman ae6a54f73c Banzai::Filter::ExternalLinkFilter use XPath 2016-06-16 14:13:29 +02:00
James Lopez 2a747d386d fixed merge conflicts 2016-06-16 14:07:49 +02:00
Yorick Peterse 19a290e7bf Reduce queries in IssueReferenceFilter
This reduces the number of queries executed in IssueReferenceFilter by
retrieving the various projects/issues that may be referenced in batches
_before_ iterating over all the HTML nodes.

A chunk of the logic resides in AbstractReferenceFilter so it can be
re-used by other filters in the future.
2016-06-16 14:04:45 +02:00
James Lopez 778d72664f Merge branch 'master' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export
# Conflicts:
#	app/models/ci/pipeline.rb
2016-06-16 13:04:00 +02:00
James Lopez 452c076a34 Revert "squashed merge and fixed conflicts"
This reverts commit 13e37a3ee5.
2016-06-16 12:59:07 +02:00
James Lopez 13e37a3ee5 squashed merge and fixed conflicts 2016-06-16 12:12:28 +02:00
Paco Guzman 46696bde83 Banzai::Filter::UploadLinkFilter use XPath 2016-06-16 11:26:56 +02:00
Ilan Shamir 94135e6275 Remove JiraIssue model and replace references with ExternalIssue 2016-06-16 01:25:18 +03:00
Rémy Coutable a8b4630190 Merge branch 'docker-registry-manifest-v1' into 'master'
Add support for Docker Registry manifest v1

## What does this MR do?

Adds support for Manifest V1 generated by older versions of Docker (before 1.10).

## What are the relevant issue numbers?

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18609


See merge request !4669
2016-06-15 15:07:39 +00:00
Yorick Peterse 816c453558 Don't update columns in batches in a transaction
This ensures that whatever locks are acquired aren't held onto until the
end of the transaction (= after _all_ rows have been updated). Timing
wise there's also no difference between using a transaction and not
using one.
2016-06-15 17:04:07 +02:00
Yorick Peterse 8966263e0c Customizing of update_column_in_batches queries
By passing a block to update_column_in_batches() one can now customize
the queries executed. This in turn can be used to only update a specific
set of rows instead of simply all the rows in the table.
2016-06-15 16:44:44 +02:00
Rémy Coutable be09845914 Merge branch 'environments-and-deployments' into 'master'
Add environments and deployments

This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.

The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.

The `.gitlab-ci.yml`:
```
deploy to production:
  stage: deploy
  script: dpl travis...
  environment: production
```

What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill 
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)

See merge request !4605
2016-06-15 13:48:09 +00:00
Kamil Trzcinski f30d1fdf94 Add support for Docker Registry manifest v1 2016-06-15 12:53:23 +02:00
James Lopez f6896f9381 fix annoying spec 2016-06-15 12:15:42 +02:00
James Lopez cbd0201fdb fixing specs and some code cleanup 2016-06-15 10:37:17 +02:00
Yorick Peterse ab91f1226f Filter out classes without names in the sampler
We can't do a lot with classes without names as we can't filter by them,
have no idea where they come from, etc. As such it's best to just ignore
these.
2016-06-14 18:09:06 +02:00
Yorick Peterse 0ca7b3ba37 Merge branch '18449-instrument-grape-endpoints' into 'master'
Instrument Grape API endpoints

See merge request !4587
2016-06-14 14:29:55 +00:00
Yorick Peterse f558bf0de4 Merge branch '18527-instrument-private-methods' into 'master'
Instrument private methods and instance private methods

See merge request !4639
2016-06-14 14:28:40 +00:00
Kamil Trzcinski 006b650988 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	db/schema.rb
2016-06-14 16:19:29 +02:00
Paco Guzman dadc531353 Instrument private/protected methods
By default instrumentation will instrument public,
protected and private methods, because usually
heavy work is done on private method or at least
that’s what facts is showing
2016-06-14 15:17:51 +02:00
Kamil Trzcinski 975e388751 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	lib/ci/gitlab_ci_yaml_processor.rb
2016-06-14 14:53:10 +02:00
Kamil Trzcinski 3f5819fb7a Merge remote-tracking branch 'origin/master' into artifacts-expire-date 2016-06-14 14:49:59 +02:00
Yorick Peterse 4b964011cf Merge branch '18528-cpu-time-instrumentation' into 'master'
Measure CPU time for instrumented methods

See merge request !4640
2016-06-14 11:39:19 +00:00
Paco Guzman 509082bafb Instrument Grape Endpoint with Metrics::RackMiddleware
Generating the following tags

Grape#GET /projects/:id/archive

from Grape::Route objects like

{ :path => /:version/projects/:id/archive(.:format)
  :version => “v3”,
  :method => “GET” }

Use an instance variable to cache raw_path transformations.
This variable is only going to growth to the number of 
endpoints of the API, not with exact different requests

We can store this cache as an instance variable because 
middleware are initialised only once
2016-06-14 13:06:46 +02:00