Commit Graph
772 Commits
Author SHA1 Message Date
Yorick PeterseandRobert Speicher be3b878443 Track method call times/counts as a single metric
Previously we'd create a separate Metric instance for every method call
that would exceed the method call threshold. This is problematic because
it doesn't provide us with information to accurately get the _total_
execution time of a particular method. For example, if the method
"Foo#bar" was called 4 times with a runtime of ~10 milliseconds we'd end
up with 4 different Metric instances. If we were to then get the
average/95th percentile/etc of the timings this would be roughly 10
milliseconds. However, the _actual_ total time spent in this method
would be around 40 milliseconds.

To solve this problem we now create a single Metric instance per method.
This Metric instance contains the _total_ real/CPU time and the call
count for every instrumented method.
2016-06-17 13:09:55 -04:00
Yorick Peterse ed5f17cc73 Merge branch 'secure-request-uris' into 'master'
Filter out sensitive parameters of metrics data

See merge request !4748
2016-06-17 16:43:14 +00:00
Robert Speicher 8dccfb4a9c Merge branch 'update-column-in-batches-where' into 'master'
Allow customising of queries used for `update_column_in_batches`

This MR makes two changes to `add_column_with_default` and `update_column_in_batches`:

1. `add_column_with_default` no longer wraps the entire set of updates in a single transaction, preventing any locks from sticking around for the duration of the entire transaction
2. `update_column_in_batches` now takes a block which can be used to customise the queries. This uses Arel as messing with raw SQL strings is a total pain

In !4381 there's a need for updating existing rows/columns in a table in batches using a custom `WHERE` condition. Without the changes in this MR this would not be possible.

See merge request !4680
2016-06-17 16:36:22 +00:00
Paco Guzman 2e552c6bf0 Filter out sensitive parameters of metrics data 2016-06-17 18:14:25 +02:00
Douwe Maan 9b7fd74805 Merge branch 'feature/project-export' into 'master'
Export project functionality

This is a MR for the export functionality of https://gitlab.com/gitlab-org/gitlab-ce/issues/3050, which adds the ability to export single projects.

- [x] members
- DB data
  - [x] issues
  - [x] issue comments
  - [x] merge requests
  - [x] merge request diff
  - [x] merge request comments
  - [x] labels
  - [x] milestones
  - [x] snippets
  - [x] releases
  - [x] events
  - [x] commit statuses
  - [x] CI builds
- File system data
  - [x] Git repository
  - [x] wiki
  - [x] uploads
  - [ ] ~~CI build traces~~
  - [ ] ~~CI build artifacts~~
  - [ ] ~~LFS objects~~
- DB configuration
  - [x] services
  - [x] web hooks
  - [x] protected branches
  - [x] deploy keys
  - [x] CI variables
  - [x] CI triggers

See merge request !3114
2016-06-17 15:35:27 +00:00
James Lopez b3a0a1c315 fixed merge conflicts on UI branch 2016-06-16 15:01:27 +02:00
James Lopez 2a747d386d fixed merge conflicts 2016-06-16 14:07:49 +02:00
James Lopez 778d72664f Merge branch 'master' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export
# Conflicts:
#	app/models/ci/pipeline.rb
2016-06-16 13:04:00 +02:00
James Lopez 452c076a34 Revert "squashed merge and fixed conflicts"
This reverts commit 13e37a3ee5.
2016-06-16 12:59:07 +02:00
James Lopez 13e37a3ee5 squashed merge and fixed conflicts 2016-06-16 12:12:28 +02:00
Ilan Shamir 94135e6275 Remove JiraIssue model and replace references with ExternalIssue 2016-06-16 01:25:18 +03:00
Yorick Peterse 816c453558 Don't update columns in batches in a transaction
This ensures that whatever locks are acquired aren't held onto until the
end of the transaction (= after _all_ rows have been updated). Timing
wise there's also no difference between using a transaction and not
using one.
2016-06-15 17:04:07 +02:00
Yorick Peterse 8966263e0c Customizing of update_column_in_batches queries
By passing a block to update_column_in_batches() one can now customize
the queries executed. This in turn can be used to only update a specific
set of rows instead of simply all the rows in the table.
2016-06-15 16:44:44 +02:00
James Lopez f6896f9381 fix annoying spec 2016-06-15 12:15:42 +02:00
James Lopez cbd0201fdb fixing specs and some code cleanup 2016-06-15 10:37:17 +02:00
Yorick Peterse ab91f1226f Filter out classes without names in the sampler
We can't do a lot with classes without names as we can't filter by them,
have no idea where they come from, etc. As such it's best to just ignore
these.
2016-06-14 18:09:06 +02:00
Yorick Peterse 0ca7b3ba37 Merge branch '18449-instrument-grape-endpoints' into 'master'
Instrument Grape API endpoints

See merge request !4587
2016-06-14 14:29:55 +00:00
Paco Guzman dadc531353 Instrument private/protected methods
By default instrumentation will instrument public,
protected and private methods, because usually
heavy work is done on private method or at least
that’s what facts is showing
2016-06-14 15:17:51 +02:00
Yorick Peterse 4b964011cf Merge branch '18528-cpu-time-instrumentation' into 'master'
Measure CPU time for instrumented methods

See merge request !4640
2016-06-14 11:39:19 +00:00
Paco Guzman 509082bafb Instrument Grape Endpoint with Metrics::RackMiddleware
Generating the following tags

Grape#GET /projects/:id/archive

from Grape::Route objects like

{ :path => /:version/projects/:id/archive(.:format)
  :version => “v3”,
  :method => “GET” }

Use an instance variable to cache raw_path transformations.
This variable is only going to growth to the number of 
endpoints of the API, not with exact different requests

We can store this cache as an instance variable because 
middleware are initialised only once
2016-06-14 13:06:46 +02:00
Rémy Coutable 47cdb69921 Merge branch 'refactor/ci-config-add-global-entry' into 'master'
Add global entry with before script to new CI config

## What does this MR do?

This MR adds a new entries to a new CI config class. It is next refactoring step after !4462.

See #15060

See merge request !4482
2016-06-14 10:53:26 +00:00
Paco Guzman 120fbbd487 Measure CPU time for instrumented methods 2016-06-14 12:49:31 +02:00
Grzegorz Bizon 59eeec3ff8 Make method that composes ci config entry private 2016-06-14 10:48:51 +02:00
James Lopez 9be06bbbb4 fix indents 2016-06-14 10:23:00 +02:00
James Lopez 7779457907 fix merge 2016-06-14 10:20:47 +02:00
James Lopez 9ff1990422 Merge branches 'feature/project-export' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-import 2016-06-14 10:18:03 +02:00
James Lopez fde90c3e7f better coverage for Reader 2016-06-14 10:12:41 +02:00
Douglas Barbosa Alexandre b56c456750 Project members with guest role can't access confidential issues 2016-06-13 19:32:00 -03:00
James Lopez 4020b0f55f few changes based on MR feedback 2016-06-13 21:18:26 +02:00
Douwe Maan 1cbd5c6686 Merge branch 'gitlab-auth-method-names' into 'master'
Improve Gitlab::Auth method names

Auth.find was a very generic name for a very specific method.
Auth.find_in_gitlab_or_ldap was inaccurate in GitLab EE where it also
looks in Kerberos.


See merge request !4589
2016-06-13 18:05:53 +00:00
James Lopez 833dc3204d few more changes based on feedback 2016-06-13 16:55:51 +02:00
Jacob Vosmaer f73cf3e937 Also rename "find" in the specs 2016-06-13 15:38:25 +02:00
James Lopez b07dc938b9 fixed specs and refactored a few things due to recent model changes and merge conflicts 2016-06-13 13:34:36 +02:00
James Lopez ad68bc63b5 Merge branches 'feature/project-export' and 'feature/project-import' of gitlab.com:gitlab-org/gitlab-ce into feature/project-import
# Conflicts:
#	app/models/project.rb
#	db/schema.rb
#	lib/gitlab/import_export/import_export_reader.rb
2016-06-13 12:43:25 +02:00
Yorick Peterse b33b7be53e Handle NULL migration errors in migration helpers
This ensures that whenever changing the NULL constraint of a column
fails we still drop the column.
2016-06-13 11:22:58 +02:00
Grzegorz Bizon ebe14437e9 Merge branch 'master' into refactor/ci-config-add-global-entry
* master: (59 commits)
  Improved SVG sanitizer specs to include smoke tests for clean.
  Refactored SVG sanitizer
  Added SVG sanitizer fix to the changelog
  Refactor SVG sanitizer and prevent `xlink:href` to refer to external resources
  Fix SVG whitelisting to allow namespaced attributes
  Fix Error 500 when using closes_issues API with an external issue tracker
  Center layout navigation and remove icons
  Fix preferences_spec test
  Add back sidebar counters and username
  Only create the backup directory if it is local
  Fix safari logo loading animation safari bug
  Revert side nav to full width; remove border under nav; remove tooltips on nav links; stop page content shifting with side nav; put project nav in container
  Update media queries
  Fix profile test
  Fix logo at all screen widths, update sidebar text
  Move tanuki icon to center of nav bar; keep nav closed by default; remove collapsed nav cookie
  Remove unused MergeRequest#gitlab_merge_status method
  Add CHANGELOG item for labels/milestones navigation change
  Render issues link on issues subnav unless you visit merge request controller
  Render only issues/mr in subnav depends on context
  ...
2016-06-13 11:14:38 +02:00
James Lopez 069bc26418 refactored loads of things due to commits to pipeline change 2016-06-13 10:55:54 +02:00
Grzegorz Bizon 11c0d02283 Simplify ci config node factory 2016-06-13 09:58:35 +02:00
James Lopez e2c95c07a4 Merge branches 'feature/project-export' and 'master' of gitlab.com:gitlab-org/gitlab-ce into feature/project-export 2016-06-13 09:22:33 +02:00
Grzegorz Bizon 7c8f3b0cfc Duplicate CI config node factory on class level 2016-06-13 08:50:12 +02:00
Gabriel MazettoandRobert Speicher 7c87dac5f1 Improved SVG sanitizer specs to include smoke tests for clean. 2016-06-12 21:05:58 -04:00
Gabriel MazettoandRobert Speicher a9eaa20dcb Refactored SVG sanitizer 2016-06-12 21:05:58 -04:00
Gabriel MazettoandRobert Speicher 13791c6704 Refactor SVG sanitizer and prevent xlink:href to refer to external resources 2016-06-12 21:05:58 -04:00
Gabriel MazettoandRobert Speicher 02b882418a Fix SVG whitelisting to allow namespaced attributes 2016-06-12 21:05:57 -04:00
Jacob Vosmaer 0e896ffe4e Improve Gitlab::Auth method names
Auth.find was a very generic name for a very specific method.
Auth.find_in_gitlab_or_ldap was inaccurate in GitLab EE where it also
looks in Kerberos.
2016-06-10 14:51:16 +02:00
Grzegorz Bizon cc373a3550 Add factory for fabricating new ci config nodes 2016-06-10 14:01:07 +02:00
Douwe Maan 16bd4e5668 Merge branch 'master' into workhorse-helpers 2016-06-10 12:28:04 +02:00
Grzegorz Bizon 12080ba150 Simplify new ci config entry class interface 2016-06-10 11:26:03 +02:00
Grzegorz Bizon 51046dd226 Merge branch 'master' into refactor/ci-config-add-global-entry
* master: (147 commits)
  Minor MR comment fixes.
  Update CHANGELOG for 8.8.4 and 8.8.5
  Properly quote table name in Rake task for MySQL and PostgreSQL compatibility
  Checks based on whether data is loaded not undefined
  Checks for undefined when inserting autocomplete into textarea
  Ignore frequent emojis in search.
  Fixed tests
  CHANGELOG
  Improved the UX of issue & milestone date picker
  Change date format to be non zero padded in order to fix failing test
  Update method name for better understanding
  Add tests for dates on tooltips
  Fix local timeago on user dashboard
  Update CHANGELOG
  Toggling a task in a description with mentions doesn't creates a Todo
  Update CHANGELOG
  Fixed failing label subscribe test
  Tests update
  Updated subscribe icon
  Fixed failing tests
  ...
2016-06-10 10:49:07 +02:00
Grzegorz Bizon d7e1251161 Rename method that returns allowed nodes in Ci config 2016-06-09 14:59:59 +02:00