Commit Graph
1066 Commits
Author SHA1 Message Date
Yorick Peterse ceeba75c76 Handle external issues in IssueReferenceFilter
In the past this class would use Project#get_issue to retrieve an issue
by its ID. This method would automatically determine whether to return
an Issue or ExternalIssue.

This commit changes IssueReferenceFilter to handle external issues again
and in a somewhat more explicit manner than before.

Fixes gitlab-org/gitlab-ce#18827
2016-06-21 20:01:56 +02:00
Robert Speicher 98cede7ebe Merge branch 'fix-out-of-bounds-markdown-refs' into 'master'
Fix RangeError exceptions when referring to issues or merge requests outside of max database values

When using #XYZ in Markdown text, if XYZ exceeds the maximum value of a signed 32-bit integer, we get an exception when the Markdown render attempts to run `where(iids: XYZ)`. Introduce a method that will throw out out-of-bounds values.
    
Closes #18777

See merge request !4777
2016-06-18 20:48:58 +00:00
Stan Hu 7c9eba8919 Fix RangeError exceptions when referring to issues or merge requests outside of max database values
When using #XYZ in Markdown text, if XYZ exceeds the maximum value of a signed 32-bit integer, we
get an exception when the Markdown render attempts to run `where(iids: XYZ)`. Introduce a method
that will throw out out-of-bounds values.

Closes #18777
2016-06-18 13:07:38 -07:00
Timothy Andrew 6d169d36ca Fix bug in WikiLinkFilter.
1. An exception would be raised if the filter was called with an invalid
   URI. Mainly because we weren't catching the `Addressable` exception.

2. This commit fixes it and adds a spec for the filter.
2016-06-18 14:10:40 +05:30
Yorick PeterseandRobert Speicher be3b878443 Track method call times/counts as a single metric
Previously we'd create a separate Metric instance for every method call
that would exceed the method call threshold. This is problematic because
it doesn't provide us with information to accurately get the _total_
execution time of a particular method. For example, if the method
"Foo#bar" was called 4 times with a runtime of ~10 milliseconds we'd end
up with 4 different Metric instances. If we were to then get the
average/95th percentile/etc of the timings this would be roughly 10
milliseconds. However, the _actual_ total time spent in this method
would be around 40 milliseconds.

To solve this problem we now create a single Metric instance per method.
This Metric instance contains the _total_ real/CPU time and the call
count for every instrumented method.
2016-06-17 13:09:55 -04:00
Rémy Coutable 1ca1ebc09b Merge branch 'registry-500-fix' into 'master'
Properly support application/json in Container Registry

## What does this MR do?
When requesting tags a `application/json` is used by `docker/distribution`.

## Why was this MR needed?
Fixes regression introduced by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4669

## What are the relevant issue numbers?
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18736


See merge request !4742
2016-06-17 17:06:55 +00:00
Yorick Peterse ed5f17cc73 Merge branch 'secure-request-uris' into 'master'
Filter out sensitive parameters of metrics data

See merge request !4748
2016-06-17 16:43:14 +00:00
Robert Speicher 8dccfb4a9c Merge branch 'update-column-in-batches-where' into 'master'
Allow customising of queries used for `update_column_in_batches`

This MR makes two changes to `add_column_with_default` and `update_column_in_batches`:

1. `add_column_with_default` no longer wraps the entire set of updates in a single transaction, preventing any locks from sticking around for the duration of the entire transaction
2. `update_column_in_batches` now takes a block which can be used to customise the queries. This uses Arel as messing with raw SQL strings is a total pain

In !4381 there's a need for updating existing rows/columns in a table in batches using a custom `WHERE` condition. Without the changes in this MR this would not be possible.

See merge request !4680
2016-06-17 16:36:22 +00:00
Paco Guzman 2e552c6bf0 Filter out sensitive parameters of metrics data 2016-06-17 18:14:25 +02:00
Douwe Maan 9b7fd74805 Merge branch 'feature/project-export' into 'master'
Export project functionality

This is a MR for the export functionality of https://gitlab.com/gitlab-org/gitlab-ce/issues/3050, which adds the ability to export single projects.

- [x] members
- DB data
  - [x] issues
  - [x] issue comments
  - [x] merge requests
  - [x] merge request diff
  - [x] merge request comments
  - [x] labels
  - [x] milestones
  - [x] snippets
  - [x] releases
  - [x] events
  - [x] commit statuses
  - [x] CI builds
- File system data
  - [x] Git repository
  - [x] wiki
  - [x] uploads
  - [ ] ~~CI build traces~~
  - [ ] ~~CI build artifacts~~
  - [ ] ~~LFS objects~~
- DB configuration
  - [x] services
  - [x] web hooks
  - [x] protected branches
  - [x] deploy keys
  - [x] CI variables
  - [x] CI triggers

See merge request !3114
2016-06-17 15:35:27 +00:00
Rémy Coutable ae4491b421 Merge branch 'fix/error-when-job-variables-not-defined-but-specified' into 'master'
Fix error when CI job variables key used but not specified

## What does this MR do?

This MR fixes a an error when CI job variables specified, but not defined:

```yaml
image: ruby:2.2

test:
  variables:
  script:
     - rspec
```

## What are the relevant issue numbers?

Closes #18764  
Follow up discussion in: #18775 

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !4745
2016-06-17 15:10:06 +00:00
Grzegorz Bizon eaa91cbe12 Fix error when CI job variables not specified 2016-06-17 14:14:55 +02:00
Kamil Trzcinski 04ffd71e53 Fix regression introduced by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4669
When requesting tags a `application/json` is used.
2016-06-17 13:24:50 +02:00
Kamil Trzcinski aef6214c42 Validate only and except regexp
Currently the RegexpError can be raised when processing next stage which leads to 500 in different places of code base.
This adds early check that regexps used in only and except are valid.
2016-06-17 11:52:22 +02:00
Robert Speicher 79c28e1057 Merge branch 'remove_jiraissue' into 'master'
Remove JiraIssue model and replace references with ExternalIssue

This MR Removes unused JiraIssue class and replaces references with ExternalIssue

Closes #18203 

See merge request !4659
2016-06-17 00:24:10 +00:00
Douwe Maan 84632f0aff Merge branch 'banzai-issue-filter-queries' into 'master'
Reduce SQL query counts in IssueReferenceFilter

## What does this MR do?

This MR adds a preparation phase for reference filters that allows them to prepare/create data structures used while iterating over HTML nodes. In this particular case the preparation phase is used for issue references to greatly cut down the amount of queries executed to get projects/issues for Markdown references.

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

Rendering Markdown containing issue references would run at most two queries for every issue reference: one to get the project and one to get the issue from said project. When rendering Markdown with lots of issue references this would result in _a lot_ of queries being executed.

## What are the relevant issue numbers?

#18042

See merge request !4410
2016-06-16 20:49:13 +00:00
James Lopez b3a0a1c315 fixed merge conflicts on UI branch 2016-06-16 15:01:27 +02:00
Paco Guzman ae6a54f73c Banzai::Filter::ExternalLinkFilter use XPath 2016-06-16 14:13:29 +02:00
James Lopez 2a747d386d fixed merge conflicts 2016-06-16 14:07:49 +02:00
Yorick Peterse 19a290e7bf Reduce queries in IssueReferenceFilter
This reduces the number of queries executed in IssueReferenceFilter by
retrieving the various projects/issues that may be referenced in batches
_before_ iterating over all the HTML nodes.

A chunk of the logic resides in AbstractReferenceFilter so it can be
re-used by other filters in the future.
2016-06-16 14:04:45 +02:00
James Lopez 778d72664f Merge branch 'master' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export
# Conflicts:
#	app/models/ci/pipeline.rb
2016-06-16 13:04:00 +02:00
James Lopez 452c076a34 Revert "squashed merge and fixed conflicts"
This reverts commit 13e37a3ee5.
2016-06-16 12:59:07 +02:00
James Lopez 13e37a3ee5 squashed merge and fixed conflicts 2016-06-16 12:12:28 +02:00
Paco Guzman 46696bde83 Banzai::Filter::UploadLinkFilter use XPath 2016-06-16 11:26:56 +02:00
Ilan Shamir 94135e6275 Remove JiraIssue model and replace references with ExternalIssue 2016-06-16 01:25:18 +03:00
Rémy Coutable a8b4630190 Merge branch 'docker-registry-manifest-v1' into 'master'
Add support for Docker Registry manifest v1

## What does this MR do?

Adds support for Manifest V1 generated by older versions of Docker (before 1.10).

## What are the relevant issue numbers?

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/18609


See merge request !4669
2016-06-15 15:07:39 +00:00
Yorick Peterse 816c453558 Don't update columns in batches in a transaction
This ensures that whatever locks are acquired aren't held onto until the
end of the transaction (= after _all_ rows have been updated). Timing
wise there's also no difference between using a transaction and not
using one.
2016-06-15 17:04:07 +02:00
Yorick Peterse 8966263e0c Customizing of update_column_in_batches queries
By passing a block to update_column_in_batches() one can now customize
the queries executed. This in turn can be used to only update a specific
set of rows instead of simply all the rows in the table.
2016-06-15 16:44:44 +02:00
Rémy Coutable be09845914 Merge branch 'environments-and-deployments' into 'master'
Add environments and deployments

This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.

The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.

The `.gitlab-ci.yml`:
```
deploy to production:
  stage: deploy
  script: dpl travis...
  environment: production
```

What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill 
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)

See merge request !4605
2016-06-15 13:48:09 +00:00
Kamil Trzcinski f30d1fdf94 Add support for Docker Registry manifest v1 2016-06-15 12:53:23 +02:00
James Lopez f6896f9381 fix annoying spec 2016-06-15 12:15:42 +02:00
James Lopez cbd0201fdb fixing specs and some code cleanup 2016-06-15 10:37:17 +02:00
Yorick Peterse ab91f1226f Filter out classes without names in the sampler
We can't do a lot with classes without names as we can't filter by them,
have no idea where they come from, etc. As such it's best to just ignore
these.
2016-06-14 18:09:06 +02:00
Yorick Peterse 0ca7b3ba37 Merge branch '18449-instrument-grape-endpoints' into 'master'
Instrument Grape API endpoints

See merge request !4587
2016-06-14 14:29:55 +00:00
Yorick Peterse f558bf0de4 Merge branch '18527-instrument-private-methods' into 'master'
Instrument private methods and instance private methods

See merge request !4639
2016-06-14 14:28:40 +00:00
Kamil Trzcinski 006b650988 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	db/schema.rb
2016-06-14 16:19:29 +02:00
Paco Guzman dadc531353 Instrument private/protected methods
By default instrumentation will instrument public,
protected and private methods, because usually
heavy work is done on private method or at least
that’s what facts is showing
2016-06-14 15:17:51 +02:00
Kamil Trzcinski 975e388751 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	lib/ci/gitlab_ci_yaml_processor.rb
2016-06-14 14:53:10 +02:00
Kamil Trzcinski 3f5819fb7a Merge remote-tracking branch 'origin/master' into artifacts-expire-date 2016-06-14 14:49:59 +02:00
Yorick Peterse 4b964011cf Merge branch '18528-cpu-time-instrumentation' into 'master'
Measure CPU time for instrumented methods

See merge request !4640
2016-06-14 11:39:19 +00:00
Paco Guzman 509082bafb Instrument Grape Endpoint with Metrics::RackMiddleware
Generating the following tags

Grape#GET /projects/:id/archive

from Grape::Route objects like

{ :path => /:version/projects/:id/archive(.:format)
  :version => “v3”,
  :method => “GET” }

Use an instance variable to cache raw_path transformations.
This variable is only going to growth to the number of 
endpoints of the API, not with exact different requests

We can store this cache as an instance variable because 
middleware are initialised only once
2016-06-14 13:06:46 +02:00
Kamil Trzcinski e8f09f02bf Validate environment name with regex 2016-06-14 13:04:21 +02:00
Rémy Coutable 47cdb69921 Merge branch 'refactor/ci-config-add-global-entry' into 'master'
Add global entry with before script to new CI config

## What does this MR do?

This MR adds a new entries to a new CI config class. It is next refactoring step after !4462.

See #15060

See merge request !4482
2016-06-14 10:53:26 +00:00
Paco Guzman 120fbbd487 Measure CPU time for instrumented methods 2016-06-14 12:49:31 +02:00
Kamil Trzcinski 1b62b86fdd Merge remote-tracking branch 'origin/master' into artifacts-expire-date 2016-06-14 11:38:44 +02:00
Grzegorz Bizon 59eeec3ff8 Make method that composes ci config entry private 2016-06-14 10:48:51 +02:00
James Lopez 9be06bbbb4 fix indents 2016-06-14 10:23:00 +02:00
James Lopez 7779457907 fix merge 2016-06-14 10:20:47 +02:00
James Lopez 9ff1990422 Merge branches 'feature/project-export' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-import 2016-06-14 10:18:03 +02:00
James Lopez fde90c3e7f better coverage for Reader 2016-06-14 10:12:41 +02:00