From 2b25297e1724f1284e28ecd8c782d948082cc679 Mon Sep 17 00:00:00 2001 From: Karen Carias Date: Mon, 27 Apr 2015 22:58:55 +0000 Subject: [PATCH 001/158] Added text "How to reset your root password to menu" --- doc/security/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/security/README.md b/doc/security/README.md index 49dfa6eec7..9f7e3a5059 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -1,6 +1,6 @@ # Security -- [Password length limits](password_length_limits.md) +- [Password length limits and how to reset your root password](password_length_limits.md) - [Rack attack](rack_attack.md) - [Web Hooks and insecure internal web services](webhooks.md) -- [Information exclusivity](information_exclusivity.md) +- [Information exclusivity](information_exclusivity.md) \ No newline at end of file From 7594a7501a1fe006abb62d1eb41e7547bd7bfe10 Mon Sep 17 00:00:00 2001 From: Karen Carias Date: Mon, 27 Apr 2015 22:59:21 +0000 Subject: [PATCH 002/158] updated one letter --- doc/security/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/security/README.md b/doc/security/README.md index 9f7e3a5059..c67f297df1 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -1,6 +1,6 @@ # Security -- [Password length limits and how to reset your root password](password_length_limits.md) +- [Password length limits and How to reset your root password](password_length_limits.md) - [Rack attack](rack_attack.md) - [Web Hooks and insecure internal web services](webhooks.md) - [Information exclusivity](information_exclusivity.md) \ No newline at end of file From 0b9363245ecf7125f58f1b0757bce25a56566060 Mon Sep 17 00:00:00 2001 From: Karen Carias Date: Tue, 28 Apr 2015 22:42:19 +0000 Subject: [PATCH 003/158] New file to add to security documentation, "How to reset your root password" --- doc/security/reset_root_password.md | 36 +++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 doc/security/reset_root_password.md diff --git a/doc/security/reset_root_password.md b/doc/security/reset_root_password.md new file mode 100644 index 0000000000..58c7b0e5a7 --- /dev/null +++ b/doc/security/reset_root_password.md @@ -0,0 +1,36 @@ +# How to reset your root password: + +Log into your server with root privileges. Then start a Ruby on Rails console. + +Start the console with this command: + +```bash +> gitlab-rails console production +``` + +Wait until the console has loaded. + +There are multiple ways to find your user. You can search for email or username. + +```bash +irb(main):001:0> u = User.where(id: 1).first +or +user = User.find_by(email: 'admin@local.host') +``` + +Now you can change your password: + +```bash +> u.password = 'secret_pass' +> u.password_confirmation = 'secret_pass' +``` + +It's important that you change both password and password_confirmation to make it work. + +Don't forget to save the changes. + +```bash +> u.save! +``` + +Exit the console and try to login with your new password. From 5820c00805b7be2db4abfafbe9c7bd78b1866522 Mon Sep 17 00:00:00 2001 From: Karen Carias Date: Tue, 28 Apr 2015 23:02:14 +0000 Subject: [PATCH 004/158] Added link to separate file about how to reset root password --- doc/security/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/security/README.md b/doc/security/README.md index c67f297df1..d35f09c7a3 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -1,6 +1,7 @@ # Security -- [Password length limits and How to reset your root password](password_length_limits.md) +- [Password length limits](password_length_limits.md) - [Rack attack](rack_attack.md) - [Web Hooks and insecure internal web services](webhooks.md) -- [Information exclusivity](information_exclusivity.md) \ No newline at end of file +- [Information exclusivity](information_exclusivity.md) +- [Reset your root password] (reset_root_password.md) \ No newline at end of file From 72aa9dd76840f74aa42fe88b7d09722577b2418b Mon Sep 17 00:00:00 2001 From: Karen Carias Date: Fri, 1 May 2015 15:26:00 +0000 Subject: [PATCH 005/158] Added link to document "Reset root password" --- doc/security/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/security/README.md b/doc/security/README.md index d35f09c7a3..1ca45fd0c0 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -4,4 +4,4 @@ - [Rack attack](rack_attack.md) - [Web Hooks and insecure internal web services](webhooks.md) - [Information exclusivity](information_exclusivity.md) -- [Reset your root password] (reset_root_password.md) \ No newline at end of file +- [Reset your root password] (https://dev.gitlab.org/gitlab/gitlabhq/blob/How_to_Reset_your_root_password/doc/security/reset_root_password.md) \ No newline at end of file From e27bc577af8d9e8f3bbf4d41c8d7835c417bfb51 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Mon, 4 May 2015 07:39:59 -0700 Subject: [PATCH 006/158] Bump gitlab_git to 7.1.11 to fix bug where viewing a tag on a project with submodules would return a 500 error Closes https://github.com/gitlabhq/gitlabhq/issues/8411 --- CHANGELOG | 1 + Gemfile | 2 +- Gemfile.lock | 2 +- 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 72ae0b4f24..06310a3174 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,6 +1,7 @@ Please view this file on the master branch, on stable branches it's out of date. v 7.11.0 (unreleased) + - Fix broken view when using a tag to display a tree that contains git submodules (Stan Hu) - Make Reply-To config apply to change e-mail confirmation and other Devise notifications (Stan Hu) - Add application setting to restrict user signups to e-mail domains (Stan Hu) - Don't allow a merge request to be merged when its title starts with "WIP". diff --git a/Gemfile b/Gemfile index ea62396d8a..4b8b683a66 100644 --- a/Gemfile +++ b/Gemfile @@ -39,7 +39,7 @@ gem "browser" # Extracting information from a git repository # Provide access to Gitlab::Git library -gem "gitlab_git", '~> 7.1.10' +gem "gitlab_git", '~> 7.1.11' # Ruby/Rack Git Smart-HTTP Server Handler gem 'gitlab-grack', '~> 2.0.2', require: 'grack' diff --git a/Gemfile.lock b/Gemfile.lock index 676a519790..5c613fdb55 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -215,7 +215,7 @@ GEM mime-types (~> 1.19) gitlab_emoji (0.1.0) gemojione (~> 2.0) - gitlab_git (7.1.10) + gitlab_git (7.1.11) activesupport (~> 4.0) charlock_holmes (~> 0.6) gitlab-linguist (~> 3.0) From affd049dc4427d749b97eaee37a5d54873016657 Mon Sep 17 00:00:00 2001 From: Alex Lossent Date: Mon, 27 Apr 2015 18:06:51 +0200 Subject: [PATCH 007/158] Improve handling of large diffs Diffs with a large number of changed lines time out (504 HTTP error) or generate a HTML page that's so heavy web browsers struggle with it. https://github.com/gitlabhq/gitlabhq/pull/5014 introduced limits on commit line count so that only a safe portion is rendered. This was later undone by code refactoring in be5b6db8, e0eb4803 and c741fcab. This patch re-introduces a safe limit on number of lines. --- CHANGELOG | 1 + app/helpers/diff_helper.rb | 19 ++++++-- app/views/projects/diffs/_diffs.html.haml | 8 +-- app/views/projects/diffs/_warning.html.haml | 2 +- spec/helpers/diff_helper_spec.rb | 54 ++++++++++++++++++++- 5 files changed, 74 insertions(+), 10 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 88678b087c..0995133d21 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -20,6 +20,7 @@ v 7.11.0 (unreleased) - Add "Reply quoting selected text" shortcut key (`r`) - Fix bug causing `@whatever` inside an issue's first code block to be picked up as a user mention. - Fix bug causing `@whatever` inside an inline code snippet (backtick-style) to be picked up as a user mention. + - Improve handling of large diffs - - Show Atom feed buttons everywhere where applicable. - Add project activity atom feed. diff --git a/app/helpers/diff_helper.rb b/app/helpers/diff_helper.rb index 162778ade5..1b10795bb7 100644 --- a/app/helpers/diff_helper.rb +++ b/app/helpers/diff_helper.rb @@ -7,14 +7,23 @@ module DiffHelper end end - def safe_diff_files(diffs) - diffs.first(allowed_diff_size).map do |diff| - Gitlab::Diff::File.new(diff) + def allowed_diff_lines + if diff_hard_limit_enabled? + Commit::DIFF_HARD_LIMIT_LINES + else + Commit::DIFF_SAFE_LINES end end - def show_diff_size_warning?(diffs) - diffs.size > allowed_diff_size + def safe_diff_files(diffs) + lines = 0 + safe_files = [] + diffs.first(allowed_diff_size).each do |diff| + lines += diff.diff.lines.count + break if lines > allowed_diff_lines + safe_files << Gitlab::Diff::File.new(diff) + end + safe_files end def diff_hard_limit_enabled? diff --git a/app/views/projects/diffs/_diffs.html.haml b/app/views/projects/diffs/_diffs.html.haml index b49aee504f..ec8974c547 100644 --- a/app/views/projects/diffs/_diffs.html.haml +++ b/app/views/projects/diffs/_diffs.html.haml @@ -5,11 +5,13 @@ = parallel_diff_btn = render 'projects/diffs/stats', diffs: diffs -- if show_diff_size_warning?(diffs) - = render 'projects/diffs/warning', diffs: diffs +- diff_files = safe_diff_files(diffs) + +- if diff_files.count < diffs.size + = render 'projects/diffs/warning', diffs: diffs, shown_files_count: diff_files.count .files - - safe_diff_files(diffs).each_with_index do |diff_file, index| + - diff_files.each_with_index do |diff_file, index| = render 'projects/diffs/file', diff_file: diff_file, i: index, project: project - if @diff_timeout diff --git a/app/views/projects/diffs/_warning.html.haml b/app/views/projects/diffs/_warning.html.haml index 47abbba2eb..bd0b7376ba 100644 --- a/app/views/projects/diffs/_warning.html.haml +++ b/app/views/projects/diffs/_warning.html.haml @@ -14,6 +14,6 @@ = link_to "Email patch", merge_request_path(@merge_request, format: :patch), class: "btn btn-warning btn-sm" %p To preserve performance only - %strong #{allowed_diff_size} of #{diffs.size} + %strong #{shown_files_count} of #{diffs.size} files are displayed. diff --git a/spec/helpers/diff_helper_spec.rb b/spec/helpers/diff_helper_spec.rb index 95719b4b49..dd4c1d645e 100644 --- a/spec/helpers/diff_helper_spec.rb +++ b/spec/helpers/diff_helper_spec.rb @@ -5,7 +5,8 @@ describe DiffHelper do let(:project) { create(:project) } let(:commit) { project.commit(sample_commit.id) } - let(:diff) { commit.diffs.first } + let(:diffs) { commit.diffs } + let(:diff) { diffs.first } let(:diff_file) { Gitlab::Diff::File.new(diff) } describe 'diff_hard_limit_enabled?' do @@ -30,6 +31,57 @@ describe DiffHelper do end end + describe 'allowed_diff_lines' do + it 'should return hard limit for number of lines in a diff if force diff is true' do + allow(controller).to receive(:params) { { force_show_diff: true } } + expect(allowed_diff_lines).to eq(50000) + end + + it 'should return safe limit for numbers of lines a diff if force diff is false' do + expect(allowed_diff_lines).to eq(5000) + end + end + + describe 'safe_diff_files' do + it 'should return all files from a commit that is smaller than safe limits' do + expect(safe_diff_files(diffs).length).to eq(2) + end + + it 'should return only the first file if the diff line count in the 2nd file takes the total beyond safe limits' do + diffs[1].diff.stub(lines: [""] * 4999) #simulate 4999 lines + expect(safe_diff_files(diffs).length).to eq(1) + end + + it 'should return all files from a commit that is beyond safe limit for numbers of lines if force diff is true' do + allow(controller).to receive(:params) { { force_show_diff: true } } + diffs[1].diff.stub(lines: [""] * 4999) #simulate 4999 lines + expect(safe_diff_files(diffs).length).to eq(2) + end + + it 'should return only the first file if the diff line count in the 2nd file takes the total beyond hard limits' do + allow(controller).to receive(:params) { { force_show_diff: true } } + diffs[1].diff.stub(lines: [""] * 49999) #simulate 49999 lines + expect(safe_diff_files(diffs).length).to eq(1) + end + + it 'should return only a safe number of file diffs if a commit touches more files than the safe limits' do + large_diffs = diffs * 100 #simulate 200 diffs + expect(safe_diff_files(large_diffs).length).to eq(100) + end + + it 'should return all file diffs if a commit touches more files than the safe limits but force diff is true' do + allow(controller).to receive(:params) { { force_show_diff: true } } + large_diffs = diffs * 100 #simulate 200 diffs + expect(safe_diff_files(large_diffs).length).to eq(200) + end + + it 'should return a limited file diffs if a commit touches more files than the hard limits and force diff is true' do + allow(controller).to receive(:params) { { force_show_diff: true } } + very_large_diffs = diffs * 1000 #simulate 2000 diffs + expect(safe_diff_files(very_large_diffs).length).to eq(1000) + end + end + describe 'parallel_diff' do it 'should return an array of arrays containing the parsed diff' do expect(parallel_diff(diff_file, 0)). From a1152e19d130d153223e00ee4f35d68408f89689 Mon Sep 17 00:00:00 2001 From: dreis Date: Tue, 5 May 2015 21:52:06 +0200 Subject: [PATCH 008/158] Fix Firefox problems when it wants to render issuable component --- app/assets/javascripts/dispatcher.js.coffee | 7 +++++++ app/views/shared/_issuable_filter.html.haml | 2 -- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/app/assets/javascripts/dispatcher.js.coffee b/app/assets/javascripts/dispatcher.js.coffee index 06787ddf87..2baaf430d9 100644 --- a/app/assets/javascripts/dispatcher.js.coffee +++ b/app/assets/javascripts/dispatcher.js.coffee @@ -112,6 +112,13 @@ class Dispatcher new NamespaceSelect() when 'dashboard' shortcut_handler = new ShortcutsDashboardNavigation() + switch path[1] + when 'issues', 'merge_requests' + new UsersSelect() + when 'groups' + switch path[1] + when 'issues', 'merge_requests' + new UsersSelect() when 'profiles' new Profile() when 'projects' diff --git a/app/views/shared/_issuable_filter.html.haml b/app/views/shared/_issuable_filter.html.haml index fa8b4eae31..4ab9421f01 100644 --- a/app/views/shared/_issuable_filter.html.haml +++ b/app/views/shared/_issuable_filter.html.haml @@ -51,8 +51,6 @@ = button_tag "Update issues", class: "btn update_selected_issues btn-save" :coffeescript - new UsersSelect() - $('form.filter-form').on 'submit', (event) -> event.preventDefault() Turbolinks.visit @.action + '&' + $(@).serialize() From 03138a3c72d9ccdfda6398564c6183de70c68df5 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Tue, 5 May 2015 23:49:30 -0700 Subject: [PATCH 009/158] Make the first branch pushed to an empty repository the default HEAD. In an empty repository, pushing a new branch not called "master" would leave HEAD in an unknown state, causing ambiguity if another branch were pushed. This could in turn cause a new protected branch to be created and cause the default branch to change. * Closes #1561 * Closes #1576 * Closes https://github.com/gitlabhq/gitlabhq/issues/8883 --- CHANGELOG | 1 + app/services/git_push_service.rb | 4 ++++ spec/services/git_push_service_spec.rb | 15 ++++++++++++++- 3 files changed, 19 insertions(+), 1 deletion(-) diff --git a/CHANGELOG b/CHANGELOG index 88678b087c..3774d64f4e 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,6 +1,7 @@ Please view this file on the master branch, on stable branches it's out of date. v 7.11.0 (unreleased) + - Make the first branch pushed to an empty repository the default HEAD (Stan Hu) - Make Reply-To config apply to change e-mail confirmation and other Devise notifications (Stan Hu) - Add application setting to restrict user signups to e-mail domains (Stan Hu) - Don't allow a merge request to be merged when its title starts with "WIP". diff --git a/app/services/git_push_service.rb b/app/services/git_push_service.rb index 1b889e0da8..bdf36af02f 100644 --- a/app/services/git_push_service.rb +++ b/app/services/git_push_service.rb @@ -31,6 +31,10 @@ class GitPushService # Initial push to the default branch. Take the full history of that branch as "newly pushed". @push_commits = project.repository.commits(newrev) + # Ensure HEAD points to the default branch in case it is not master + branch_name = Gitlab::Git.ref_name(ref) + project.change_head(branch_name) + # Set protection on the default branch if configured if (current_application_settings.default_branch_protection != PROTECTION_NONE) developers_can_push = current_application_settings.default_branch_protection == PROTECTION_DEV_CAN_PUSH ? true : false diff --git a/spec/services/git_push_service_spec.rb b/spec/services/git_push_service_spec.rb index af37e8319a..e7558f2876 100644 --- a/spec/services/git_push_service_spec.rb +++ b/spec/services/git_push_service_spec.rb @@ -234,5 +234,18 @@ describe GitPushService do expect(Issue.find(issue.id)).to be_opened end end -end + describe "empty project" do + let(:project) { create(:project_empty_repo) } + let(:new_ref) { 'refs/heads/feature'} + + before do + allow(project).to receive(:default_branch).and_return('feature') + expect(project).to receive(:change_head) { 'feature'} + end + + it 'push to first branch updates HEAD' do + service.execute(project, user, @blankrev, @newrev, new_ref) + end + end +end From b4be7aed7711e51becb3be29ed82d6753cc73cc4 Mon Sep 17 00:00:00 2001 From: Jakub Jirutka Date: Sun, 15 Feb 2015 01:28:50 +0100 Subject: [PATCH 010/158] Allow to configure smtp and sendmail in gitlab.yml --- app/mailers/devise_mailer.rb | 4 +- app/mailers/emails/projects.rb | 2 +- app/mailers/notify.rb | 8 +-- config/gitlab.teatro.yml | 7 +- config/gitlab.yml.example | 67 ++++++++++++++++--- config/initializers/1_settings.rb | 24 +++++-- .../initializers/disable_email_interceptor.rb | 4 +- config/initializers/email_settings.rb | 5 ++ config/initializers/smtp_settings.rb.sample | 22 ------ lib/tasks/gitlab/check.rake | 2 +- spec/mailers/notify_spec.rb | 6 +- 11 files changed, 100 insertions(+), 51 deletions(-) create mode 100644 config/initializers/email_settings.rb delete mode 100644 config/initializers/smtp_settings.rb.sample diff --git a/app/mailers/devise_mailer.rb b/app/mailers/devise_mailer.rb index b616add283..e4b65c19b3 100644 --- a/app/mailers/devise_mailer.rb +++ b/app/mailers/devise_mailer.rb @@ -1,4 +1,4 @@ class DeviseMailer < Devise::Mailer - default from: "#{Gitlab.config.gitlab.email_display_name} <#{Gitlab.config.gitlab.email_from}>" - default reply_to: Gitlab.config.gitlab.email_reply_to + default from: "#{Gitlab.config.outgoing_emails.display_name} <#{Gitlab.config.outgoing_emails.from}>" + default reply_to: Gitlab.config.outgoing_emails.reply_to end diff --git a/app/mailers/emails/projects.rb b/app/mailers/emails/projects.rb index 9cb7077e59..c6bc716930 100644 --- a/app/mailers/emails/projects.rb +++ b/app/mailers/emails/projects.rb @@ -129,7 +129,7 @@ module Emails if send_from_committer_email && can_send_from_user_email?(@author) @author.email else - Gitlab.config.gitlab.email_reply_to + Gitlab.config.outgoing_emails.reply_to end mail(from: sender(author_id, send_from_committer_email), diff --git a/app/mailers/notify.rb b/app/mailers/notify.rb index 79fb48b00d..a9800f7f10 100644 --- a/app/mailers/notify.rb +++ b/app/mailers/notify.rb @@ -17,7 +17,7 @@ class Notify < ActionMailer::Base helper_method :current_user, :can? default from: Proc.new { default_sender_address.format } - default reply_to: Gitlab.config.gitlab.email_reply_to + default reply_to: Gitlab.config.outgoing_emails.reply_to # Just send email with 2 seconds delay def self.delay @@ -50,9 +50,9 @@ class Notify < ActionMailer::Base # The default email address to send emails from def default_sender_address - address = Mail::Address.new(Gitlab.config.gitlab.email_from) - address.display_name = Gitlab.config.gitlab.email_display_name - address + Mail::Address.new(Gitlab.config.outgoing_emails.from).tap do |address| + address.display_name = Gitlab.config.outgoing_emails.display_name + end end def can_send_from_user_email?(sender) diff --git a/config/gitlab.teatro.yml b/config/gitlab.teatro.yml index f0656400be..0713350384 100644 --- a/config/gitlab.teatro.yml +++ b/config/gitlab.teatro.yml @@ -7,8 +7,6 @@ production: &base user: root - email_from: example@example.com - support_email: support@example.com default_projects_features: @@ -19,6 +17,9 @@ production: &base snippets: false visibility_level: "private" # can be "private" | "internal" | "public" + outgoing_emails: + from: example@example.com + issues_tracker: gravatar: @@ -75,6 +76,8 @@ test: gitlab: host: localhost port: 80 + outgoing_emails: + delivery_method: :test issues_tracker: redmine: title: "Redmine" diff --git a/config/gitlab.yml.example b/config/gitlab.yml.example index bd2081688d..f0b65e1e90 100644 --- a/config/gitlab.yml.example +++ b/config/gitlab.yml.example @@ -10,7 +10,7 @@ # How to use: # 1. Copy file as gitlab.yml # 2. Update gitlab -> host with your fully qualified domain name -# 3. Update gitlab -> email_from +# 3. Update outgoing_emails -> from # 4. If you installed Git from source, change git -> bin_path to /usr/local/bin/git # IMPORTANT: If Git was installed in a different location use that instead. # You can check with `which git`. If a wrong path of Git is specified, it will @@ -46,16 +46,6 @@ production: &base # To see all available zones, run `bundle exec rake time:zones:all RAILS_ENV=production` # time_zone: 'UTC' - ## Email settings - # Uncomment and set to false if you need to disable email sending from GitLab (default: true) - # email_enabled: true - # Email address used in the "From" field in mails sent by GitLab - email_from: example@example.com - email_display_name: GitLab - email_reply_to: noreply@example.com - - # Email server smtp settings are in config/initializers/smtp_settings.rb.sample - # default_can_create_group: false # default: true # username_changing_enabled: false # default: true - User can change her username/namespace ## Default theme @@ -89,6 +79,57 @@ production: &base # The default is 'tmp/repositories' relative to the root of the Rails app. # repository_downloads_path: tmp/repositories + ## Settings for sending emails + outgoing_emails: + + # Set to false if you need to disable email sending from GitLab (default: true). + enabled: true + + # Email address and name used in the "From" field in mails sent by GitLab. + from: example@example.com + display_name: GitLab + + # Email address used in "Reply-To" field in mails sent by GitLab. + reply_to: noreply@example.com + + # Defines a delivery method. Possible values are: + # :sendmail - send emails using sendmail command (default). + # :smtp - send emails using SMTP server. + # :letter_opener - open sent emails in browser. + # :test - save emails to ActionMailer::Base.deliveries array. + delivery_method: :sendmail + + # Settings for the :sendmail delivery method. + sendmail_settings: + # The location of the sendmail executable (default: /usr/sbin/sendmail). + location: /usr/sbin/sendmail + + # The command line arguments to be passed to sendmail (default: -i -t). + arguments: '-i -t' + + # Settings for the :smtp delivery method. + smtp_settings: + # Hostname of the SMTP server used to send emails (default: localhost). + address: localhost + + # Port of the SMTP server used to send emails (default: 25). + port: 25 + + # If your mail server requires authentication, you need to specify the + # authentication type here. Possible values are: + # :plain - send the password in the clear text. + # :login - send password Base64 encoded. + # :cram_md5 - combines a Challenge/Response mechanism to exchange information and + # a cryptographic Message Digest 5 algorithm to hash important information. + # authentication: :plain + # user_name: 'gitlab' + # password: '123456' + + # Detects if STARTTLS is enabled in your SMTP server and starts to use it. + # Set this to false if there is a problem with your server certificate + # that you cannot resolve. It should be true for Gmail SMTP. + # enable_starttls_auto: true + ## Gravatar ## For Libravatar see: http://doc.gitlab.com/ce/customization/libravatar.html gravatar: @@ -295,6 +336,8 @@ production: &base development: <<: *base + outgoing_emails: + delivery_method: :letter_opener test: <<: *base @@ -303,6 +346,8 @@ test: gitlab: host: localhost port: 80 + outgoing_emails: + delivery_method: :test # When you run tests we clone and setup gitlab-shell # In order to setup it correctly you need to specify diff --git a/config/initializers/1_settings.rb b/config/initializers/1_settings.rb index e5ac66a232..ce0023e079 100644 --- a/config/initializers/1_settings.rb +++ b/config/initializers/1_settings.rb @@ -105,10 +105,6 @@ Settings.gitlab['https'] = false if Settings.gitlab['https'].nil? Settings.gitlab['port'] ||= Settings.gitlab.https ? 443 : 80 Settings.gitlab['relative_url_root'] ||= ENV['RAILS_RELATIVE_URL_ROOT'] || '' Settings.gitlab['protocol'] ||= Settings.gitlab.https ? "https" : "http" -Settings.gitlab['email_enabled'] ||= true if Settings.gitlab['email_enabled'].nil? -Settings.gitlab['email_from'] ||= "gitlab@#{Settings.gitlab.host}" -Settings.gitlab['email_display_name'] ||= "GitLab" -Settings.gitlab['email_reply_to'] ||= "noreply@#{Settings.gitlab.host}" Settings.gitlab['url'] ||= Settings.send(:build_gitlab_url) Settings.gitlab['user'] ||= 'git' Settings.gitlab['user_home'] ||= begin @@ -134,6 +130,26 @@ Settings.gitlab.default_projects_features['visibility_level'] = Settings.send Settings.gitlab['repository_downloads_path'] = File.absolute_path(Settings.gitlab['repository_downloads_path'] || 'tmp/repositories', Rails.root) Settings.gitlab['restricted_signup_domains'] ||= [] +# +# Outgoing emails +# +Settings['outgoing_emails'] ||= Settingslogic.new({}) +Settings['outgoing_emails'].tap do |opts| + # For backward compatibility. TODO remove in next major release. + opts['enabled'] ||= Settings.gitlab['email_enabled'] + opts['from'] ||= Settings.gitlab['email_from'] + opts['display_name'] ||= Settings.gitlab['display_name'] + opts['reply_to'] ||= Settings.gitlab['email_reply_to'] + + opts['enabled'] ||= opts['enabled'].nil? + opts['display_name'] ||= "GitLab" + opts['from'] ||= "gitlab@#{Settings.gitlab.host}" + opts['reply_to'] ||= "noreply@#{Settings.gitlab.host}" + opts['delivery_method'] ||= :sendmail + opts['sendmail_settings'] ||= {} + opts['smtp_settings'] ||= {} +end + # # Gravatar # diff --git a/config/initializers/disable_email_interceptor.rb b/config/initializers/disable_email_interceptor.rb index c76a6b8b19..122a582991 100644 --- a/config/initializers/disable_email_interceptor.rb +++ b/config/initializers/disable_email_interceptor.rb @@ -1,2 +1,4 @@ # Interceptor in lib/disable_email_interceptor.rb -ActionMailer::Base.register_interceptor(DisableEmailInterceptor) unless Gitlab.config.gitlab.email_enabled +unless Gitlab.config.outgoing_emails.enabled + ActionMailer::Base.register_interceptor(DisableEmailInterceptor) +end diff --git a/config/initializers/email_settings.rb b/config/initializers/email_settings.rb new file mode 100644 index 0000000000..99cb09ebfd --- /dev/null +++ b/config/initializers/email_settings.rb @@ -0,0 +1,5 @@ +Gitlab.config.outgoing_emails.tap do |c| + Gitlab::Application.config.action_mailer.delivery_method = c.delivery_method + ActionMailer::Base.smtp_settings = c.smtp_settings.symbolize_keys + ActionMailer::Base.sendmail_settings = c.sendmail_settings.symbolize_keys +end diff --git a/config/initializers/smtp_settings.rb.sample b/config/initializers/smtp_settings.rb.sample deleted file mode 100644 index f0fe2fdfa4..0000000000 --- a/config/initializers/smtp_settings.rb.sample +++ /dev/null @@ -1,22 +0,0 @@ -# To enable smtp email delivery for your GitLab instance do the following: -# 1. Rename this file to smtp_settings.rb -# 2. Edit settings inside this file -# 3. Restart GitLab instance -# -# For full list of options and their values see http://api.rubyonrails.org/classes/ActionMailer/Base.html -# - -if Rails.env.production? - Gitlab::Application.config.action_mailer.delivery_method = :smtp - - ActionMailer::Base.smtp_settings = { - address: "email.server.com", - port: 456, - user_name: "smtp", - password: "123456", - domain: "gitlab.company.com", - authentication: :login, - enable_starttls_auto: true, - openssl_verify_mode: 'peer' # See ActionMailer documentation for other possible options - } -end diff --git a/lib/tasks/gitlab/check.rake b/lib/tasks/gitlab/check.rake index 1a6303b6c8..f967fddf56 100644 --- a/lib/tasks/gitlab/check.rake +++ b/lib/tasks/gitlab/check.rake @@ -320,7 +320,7 @@ namespace :gitlab do options = { "user.name" => "GitLab", - "user.email" => Gitlab.config.gitlab.email_from, + "user.email" => Gitlab.config.outgoing_emails.from, "core.autocrlf" => "input" } correct_options = options.map do |name, value| diff --git a/spec/mailers/notify_spec.rb b/spec/mailers/notify_spec.rb index dbcf7286e4..f626efe1f7 100644 --- a/spec/mailers/notify_spec.rb +++ b/spec/mailers/notify_spec.rb @@ -5,9 +5,9 @@ describe Notify do include EmailSpec::Matchers include RepoHelpers - let(:gitlab_sender_display_name) { Gitlab.config.gitlab.email_display_name } - let(:gitlab_sender) { Gitlab.config.gitlab.email_from } - let(:gitlab_sender_reply_to) { Gitlab.config.gitlab.email_reply_to } + let(:gitlab_sender_display_name) { Gitlab.config.outgoing_emails.display_name } + let(:gitlab_sender) { Gitlab.config.outgoing_emails.from } + let(:gitlab_sender_reply_to) { Gitlab.config.outgoing_emails.reply_to } let(:recipient) { create(:user, email: 'recipient@example.com') } let(:project) { create(:project) } From 22d5e9855c776b32a79853548839211bba07f792 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 15:13:11 +0300 Subject: [PATCH 011/158] When use change branches link at MR form - save source branch selection instead of target one Signed-off-by: Dmitriy Zaporozhets --- CHANGELOG | 2 +- app/helpers/merge_requests_helper.rb | 12 ++++++++++++ .../projects/merge_requests/_new_submit.html.haml | 2 +- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 433ea27a31..c7fb49dc18 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -20,7 +20,7 @@ v 7.11.0 (unreleased) - Add "Reply quoting selected text" shortcut key (`r`) - Fix bug causing `@whatever` inside an issue's first code block to be picked up as a user mention. - Fix bug causing `@whatever` inside an inline code snippet (backtick-style) to be picked up as a user mention. - - + - When use change branches link at MR form - save source branch selection instead of target one - Show Atom feed buttons everywhere where applicable. - Add project activity atom feed. - Don't crash when an MR from a fork has a cross-reference comment from the target project on one of its commits. diff --git a/app/helpers/merge_requests_helper.rb b/app/helpers/merge_requests_helper.rb index 54462fd00e..45ee4fe413 100644 --- a/app/helpers/merge_requests_helper.rb +++ b/app/helpers/merge_requests_helper.rb @@ -49,4 +49,16 @@ module MergeRequestsHelper def issues_sentence(issues) issues.map { |i| "##{i.iid}" }.to_sentence end + + def mr_change_branches_path(merge_request) + new_namespace_project_merge_request_path( + @project.namespace, @project, + merge_request: { + source_project_id: @merge_request.source_project_id, + target_project_id: @merge_request.target_project_id, + source_branch: @merge_request.source_branch, + target_branch: nil + } + ) + end end diff --git a/app/views/projects/merge_requests/_new_submit.html.haml b/app/views/projects/merge_requests/_new_submit.html.haml index 4e72458932..6dc883ba22 100644 --- a/app/views/projects/merge_requests/_new_submit.html.haml +++ b/app/views/projects/merge_requests/_new_submit.html.haml @@ -7,7 +7,7 @@ %strong.label-branch #{@merge_request.target_project_namespace}:#{@merge_request.target_branch} %span.pull-right - = link_to 'Change branches', new_namespace_project_merge_request_path(@project.namespace, @project) + = link_to 'Change branches', mr_change_branches_path(@merge_request) = form_for [@project.namespace.becomes(Namespace), @project, @merge_request], html: { class: "merge-request-form form-horizontal gfm-form" } do |f| .merge-request-form-info From cd108a828ba8ec18392409de021d0309989ba8a4 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 16:11:47 +0300 Subject: [PATCH 012/158] Re-use issuable_form for new merge request Signed-off-by: Dmitriy Zaporozhets --- .../merge_requests/_new_submit.html.haml | 75 ++----------------- 1 file changed, 6 insertions(+), 69 deletions(-) diff --git a/app/views/projects/merge_requests/_new_submit.html.haml b/app/views/projects/merge_requests/_new_submit.html.haml index 6dc883ba22..c38da76ba2 100644 --- a/app/views/projects/merge_requests/_new_submit.html.haml +++ b/app/views/projects/merge_requests/_new_submit.html.haml @@ -9,76 +9,13 @@ %span.pull-right = link_to 'Change branches', mr_change_branches_path(@merge_request) -= form_for [@project.namespace.becomes(Namespace), @project, @merge_request], html: { class: "merge-request-form form-horizontal gfm-form" } do |f| += form_for [@project.namespace.becomes(Namespace), @project, @merge_request], html: { class: 'merge-request-form form-horizontal gfm-form' } do |f| .merge-request-form-info - .form-group - = f.label :title, class: 'control-label' do - %strong Title * - .col-sm-10 - = f.text_field :title, maxlength: 255, autofocus: true, class: 'form-control pad js-gfm-input', required: true - .form-group.issuable-description - = f.label :description, 'Description', class: 'control-label' - .col-sm-10 - = render layout: 'projects/md_preview', locals: { preview_class: "wiki" } do - = render 'projects/zen', f: f, attr: :description, classes: 'description form-control' - - .col-sm-12-hint - .pull-left - Parsed with - #{link_to 'Gitlab Flavored Markdown', help_page_path('markdown', 'markdown'), target: '_blank'}. - .pull-right - Attach files by dragging & dropping - or #{link_to 'selecting them', '#', class: 'markdown-selector'}. - - .clearfix - .error-alert - %hr - .form-group - .issue-assignee - = f.label :assignee_id, class: 'control-label' do - %i.fa.fa-user - Assign to - .col-sm-10 - = users_select_tag('merge_request[assignee_id]', placeholder: 'Select a user', class: 'custom-form-control', selected: @merge_request.assignee_id, project_id: @merge_request.target_project_id) -   - = link_to 'Assign to me', '#', class: 'btn assign-to-me-link' - .form-group - .issue-milestone - = f.label :milestone_id, class: 'control-label' do - %i.fa.fa-clock-o - Milestone - .col-sm-10 - - if milestone_options(@merge_request).present? - = f.select(:milestone_id, milestone_options(@merge_request), {include_blank: 'Select milestone'}, {class: 'select2'}) - - else - %span.light No open milestones available. -   - - if can? current_user, :admin_milestone, @merge_request.target_project - = link_to 'Create new milestone', new_namespace_project_milestone_path(@merge_request.target_project.namespace, @merge_request.target_project), target: :blank - .form-group - = f.label :label_ids, class: 'control-label' do - %i.fa.fa-tag - Labels - .col-sm-10 - - if @merge_request.target_project.labels.any? - = f.collection_select :label_ids, @merge_request.target_project.labels.all, :id, :name, {selected: @merge_request.label_ids}, multiple: true, class: 'select2' - - else - %span.light No labels yet. -   - - if can? current_user, :admin_label, @merge_request.target_project - = link_to 'Create new label', new_namespace_project_label_path(@merge_request.target_project.namespace, @merge_request.target_project), target: :blank - - .form-actions - - if guide_url = contribution_guide_url(@target_project) - %p - Please review the - %strong #{link_to 'guidelines for contribution', guide_url} - to this repository. - = f.hidden_field :source_project_id - = f.hidden_field :source_branch - = f.hidden_field :target_project_id - = f.hidden_field :target_branch - = f.submit 'Submit merge request', class: 'btn btn-create' + = render 'projects/issuable_form', f: f, issuable: @merge_request + = f.hidden_field :source_project_id + = f.hidden_field :source_branch + = f.hidden_field :target_project_id + = f.hidden_field :target_branch .mr-compare.merge-request %ul.nav.nav-tabs.merge-request-tabs From d49b993929f21585ece74fbca3ed8ab8163f8737 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 16:19:09 +0300 Subject: [PATCH 013/158] Small UI improvements to MR form Signed-off-by: Dmitriy Zaporozhets --- app/views/projects/_aside.html.haml | 5 +++++ app/views/projects/merge_requests/_new_compare.html.haml | 7 +++---- app/views/projects/merge_requests/_new_submit.html.haml | 2 +- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/app/views/projects/_aside.html.haml b/app/views/projects/_aside.html.haml index 1a7ab74c93..1e7d2adcb8 100644 --- a/app/views/projects/_aside.html.haml +++ b/app/views/projects/_aside.html.haml @@ -39,6 +39,11 @@ = link_to url_for_new_issue(@project, only_path: true), title: "New Issue", class: 'btn btn-sm' do %i.fa.fa-fw.fa-exclamation-circle New issue + - if can? current_user, :write_merge_request, @project +   + = link_to new_namespace_project_merge_request_path(@project.namespace, @project), class: "btn btn-sm", title: "New Merge Request" do + %i.fa.fa-plus + New Merge Request diff --git a/app/views/projects/merge_requests/_new_compare.html.haml b/app/views/projects/merge_requests/_new_compare.html.haml index 17e76059fd..e611b23bca 100644 --- a/app/views/projects/merge_requests/_new_compare.html.haml +++ b/app/views/projects/merge_requests/_new_compare.html.haml @@ -1,5 +1,4 @@ -%h3.page-title Compare branches for new Merge Request -%hr +%p.lead Compare branches for new Merge Request = form_for [@project.namespace.becomes(Namespace), @project, @merge_request], url: new_namespace_project_merge_request_path(@project.namespace, @project), method: :get, html: { class: "merge-request-form form-inline" } do |f| .hide.alert.alert-danger.mr-compare-errors @@ -52,8 +51,8 @@ are the same. - %hr - = f.submit 'Compare branches', class: "btn btn-primary mr-compare-btn" + %div + = f.submit 'Compare branches', class: "btn btn-new mr-compare-btn" :javascript var source_branch = $("#merge_request_source_branch") diff --git a/app/views/projects/merge_requests/_new_submit.html.haml b/app/views/projects/merge_requests/_new_submit.html.haml index c38da76ba2..24a9563dd4 100644 --- a/app/views/projects/merge_requests/_new_submit.html.haml +++ b/app/views/projects/merge_requests/_new_submit.html.haml @@ -8,7 +8,7 @@ %span.pull-right = link_to 'Change branches', mr_change_branches_path(@merge_request) - +%hr = form_for [@project.namespace.becomes(Namespace), @project, @merge_request], html: { class: 'merge-request-form form-horizontal gfm-form' } do |f| .merge-request-form-info = render 'projects/issuable_form', f: f, issuable: @merge_request From e8edd20bfd4f27a3dc644a86b667e1d2900b729e Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Wed, 6 May 2015 15:32:29 +0200 Subject: [PATCH 014/158] Made a single docker file. --- docker/README.md | 134 ++++++++++++++++++++++---------- docker/{ => app}/Dockerfile | 0 docker/{ => app}/assets/wrapper | 0 docker/single/Dockerfile | 35 +++++++++ docker/single/assets/wrapper | 16 ++++ docker/single/marathon.json | 31 ++++++++ docker/troubleshooting.md | 8 ++ 7 files changed, 183 insertions(+), 41 deletions(-) rename docker/{ => app}/Dockerfile (100%) rename docker/{ => app}/assets/wrapper (100%) create mode 100644 docker/single/Dockerfile create mode 100755 docker/single/assets/wrapper create mode 100644 docker/single/marathon.json diff --git a/docker/README.md b/docker/README.md index a54739ae3d..b88d570f08 100644 --- a/docker/README.md +++ b/docker/README.md @@ -5,21 +5,74 @@ GitLab offers git repository management, code reviews, issue tracking, activity Learn more on [https://about.gitlab.com](https://about.gitlab.com) +Single and app and data images +=================== -How to build and use images yourself +Normally docker uses a single image for one applications. +But GitLab stores repositories and uploads in the filesystem. +This means that upgrades of a single image are hard. +That is why we recommend using separate app and data images. +We'll first describe how to use a single image. +After that we'll describe how to use the app and data images. + +Single image +================= + +Run the below commands from the GitLab repo root directory. +People using boot2docker should run the commands without sudo. + +Build the image with: + +``` +sudo docker build --tag gitlab-ce docker/single/ +``` + +Run the image with: + +``` +sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 gitlab-ce +``` + +You can then go to [http://localhost:8080/](http://localhost:8080/) or [http://192.168.59.103:8080/](http://192.168.59.103:8080/) if you use boot2docker. + +You can login with username `root` and password `5iveL!fe`. +Next time, you can just use `sudo docker start gitlab-ce` and `sudo docker stop gitlab-ce`. + +Publish the image with: + +```bash +sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab-ce sytse/gitlab-ce:7.10.1 +sudo docker push sytse/gitlab-ce:7.10.1 +``` + +Use the published image with: + +```bash +sudo docker pull sytse/gitlab-ce:7.10.1 +sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 +``` + +Troubleshoot with: + +``` +sudo docker run -ti -e TERM=linux --name gitlab-ce --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash +/opt/gitlab/embedded/bin/runsvdir-start & gitlab-ctl reconfigure +gitlab-ctl start +``` + +Build and use app and data images ====================== At this moment GitLab doesn't have official Docker images. There are unofficial images at the bottom of this document. -But in this section we'll build our own. -For convinience we will use suffix _xy where xy is current version of GitLab. Build your own based on the Omnibus packages with the following commands. -Run these from the GitLab repo root directory. -People using boot2docker should run it without sudo. +For convinience we will use suffix _xy where xy is current version of GitLab. +Run the below commands from the GitLab repo root directory. +People using boot2docker should run the commands without sudo. ```bash -sudo docker build --tag gitlab_data_image docker/data/ -sudo docker build --tag gitlab_app_image_xy docker/ +sudo docker build --tag gitlab-data docker/data/ +sudo docker build --tag gitlab-app:7.10.1 docker/app/ ``` We assume using a data volume container, this will simplify migrations and backups. @@ -34,13 +87,13 @@ The directories on data container are: Create the data container with: ```bash -sudo docker run --name gitlab_data gitlab_data_image /bin/true +sudo docker run --name gitlab-data gitlab-data /bin/true ``` After creating data container run GitLab container: ```bash -sudo docker run --detach --name gitlab_app_xy --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data gitlab_app_image_xy +sudo docker run --detach --name gitlab-app::7.10.1 --publish 8080:80 --publish 2222:22 --volumes-from gitlab-data gitlab-app:7.10.1 ``` It might take a while before the docker container is responding to queries. You can follow the configuration process with `sudo docker logs -f gitlab_app_xy`. @@ -50,7 +103,7 @@ You can then go to [http://localhost:8080/](http://localhost:8080/) or [http://1 You can login with username `root` and password `5iveL!fe`. Next time, you can just use `sudo docker start gitlab_app` and `sudo docker stop gitlab_app`. -How to configure GitLab +Configure GitLab ======================== This container uses the official Omnibus GitLab distribution, so all configuration is done in the unique configuration file `/etc/gitlab/gitlab.rb`. @@ -58,7 +111,7 @@ This container uses the official Omnibus GitLab distribution, so all configurati To access GitLab configuration, you can start an interactive command line in a new container using the shared data volume container, you will be able to browse the 3 directories and use your favorite text editor: ```bash -sudo docker run -ti -e TERM=linux --rm --volumes-from gitlab_data ubuntu +sudo docker run -ti -e TERM=linux --rm --volumes-from gitlab-data ubuntu vi /etc/gitlab/gitlab.rb ``` @@ -66,7 +119,7 @@ vi /etc/gitlab/gitlab.rb You can find all available options in [Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#configuration). -How to upgrade GitLab +Upgrade GitLab with app and data images ======================== To updgrade GitLab to new versions, stop running container, create new docker image and container from that image. @@ -74,41 +127,40 @@ To updgrade GitLab to new versions, stop running container, create new docker im It Assumes that you're upgrading from 7.8 to 7.9 and you're in the updated GitLab repo root directory: ```bash -sudo docker stop gitlab_app_78 -sudo docker build --tag gitlab_app_image_79 docker/ -sudo docker run --detach --name gitlab_app_79 --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data gitlab_app_image_79 +sudo docker stop gitlab-app +sudo docker rm gitlab-app +sudo docker build --tag gitlab-app:7.10.1 docker/app/ +sudo docker run --detach --name gitlab-app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data gitlab-app:7.10.1 ``` -On the first run GitLab will reconfigure and update itself. If everything runs OK don't forget to cleanup old container and image: +On the first run GitLab will reconfigure and update itself. If everything runs OK don't forget to cleanup image: ```bash -sudo docker rm gitlab_app_78 -sudo docker rmi gitlab_app_image_78 +sudo docker rmi gitlab-app:7.8.1 +``` + +Publish app and data images to Dockerhub +========================= +Login to Dockerhub with `sudo docker login` and run the following (replace '7.9.2' with the version you're using and 'Sytse Sijbrandij' with your name): + +```bash +sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab-app:7.10.1 sytse/gitlab-app:7.10.1 +sudo docker push sytse/gitlab-app:7.10.1 +sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab_data sytse/gitlab_data +sudo docker push sytse/gitlab_data +``` + +Use app and data images published to Dockerhub +================================ +This examples uses the unofficial images made by GitLab CEO Sytse. + +```bash +sudo docker pull sytse/gitlab-data +sudo docker pull sytse/gitlab-app:7.10.1 +sudo docker run --name gitlab-data sytse/gitlab-data /bin/true +sudo docker run --detach --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data sytse/gitlab-app:7.10.1 ``` Troubleshooting ========================= Please see the [troubleshooting](troubleshooting.md) file in this directory. - - -Publish the images to Dockerhub -========================= -Login to Dockerhub with `sudo docker login` and run the following (replace '7.9.2' with the version you're using and 'Sytse Sijbrandij' with your name): - -```bash -sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab_app_xy sytse/gitlab-ce:7.9.2 -sudo docker push sytse/gitlab-ce:7.9.2 -sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab_data sytse/gitlab_data -sudo docker push sytse/gitlab_data -``` - -Use images published to Dockerhub -================================ -This examples uses the unofficial images made by GitLab CEO Sytse. - -```bash -sudo docker pull sytse/gitlab_data -sudo docker pull sytse/gitlab-ce:7.9.2 -sudo docker run --name gitlab_data_volume sytse/gitlab_data /bin/true -sudo docker run --detach --name gitlab_app_7_9_2 --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data_volume sytse/gitlab-ce:7.9.2 -``` diff --git a/docker/Dockerfile b/docker/app/Dockerfile similarity index 100% rename from docker/Dockerfile rename to docker/app/Dockerfile diff --git a/docker/assets/wrapper b/docker/app/assets/wrapper similarity index 100% rename from docker/assets/wrapper rename to docker/app/assets/wrapper diff --git a/docker/single/Dockerfile b/docker/single/Dockerfile new file mode 100644 index 0000000000..8cdc24cf04 --- /dev/null +++ b/docker/single/Dockerfile @@ -0,0 +1,35 @@ +FROM ubuntu:14.04 +MAINTAINER Sytse Sijbrandij + +# Install required packages +RUN apt-get update +ENV DEBIAN_FRONTEND noninteractive +RUN apt-get install -yq --no-install-recommends \ + ca-certificates \ + openssh-server \ + wget + +# Download & Install GitLab +# If the Omnibus package version below is outdated please contribute a merge request to update it. +# If you run GitLab Enterprise Edition point it to a location where you have downloaded it. +RUN TMP_FILE=$(mktemp); \ + wget -q -O $TMP_FILE https://downloads-packages.s3.amazonaws.com/ubuntu-14.04/gitlab-ce_7.10.1~omnibus.2-1_amd64.deb \ + && dpkg -i $TMP_FILE \ + && rm -f $TMP_FILE + +# Manage SSHD through runit +RUN mkdir -p /opt/gitlab/sv/sshd/supervise \ + && mkfifo /opt/gitlab/sv/sshd/supervise/ok \ + && printf "#!/bin/sh\nexec 2>&1\numask 077\nexec /usr/sbin/sshd -D" > /opt/gitlab/sv/sshd/run \ + && chmod a+x /opt/gitlab/sv/sshd/run \ + && ln -s /opt/gitlab/sv/sshd /opt/gitlab/service \ + && mkdir -p /var/run/sshd + +# Expose web & ssh +EXPOSE 80 22 + +# Copy assets +COPY assets/wrapper /usr/local/bin/ + +# Wrapper to handle signal, trigger runit and reconfigure GitLab +CMD ["/usr/local/bin/wrapper"] diff --git a/docker/single/assets/wrapper b/docker/single/assets/wrapper new file mode 100755 index 0000000000..966b2cab4a --- /dev/null +++ b/docker/single/assets/wrapper @@ -0,0 +1,16 @@ +#!/bin/bash + +function sigterm_handler() { + echo "SIGTERM signal received, try to gracefully shutdown all services..." + gitlab-ctl stop +} + +trap "sigterm_handler; exit" TERM + +function entrypoint() { + /opt/gitlab/embedded/bin/runsvdir-start & + gitlab-ctl reconfigure # will also start everything + gitlab-ctl tail # tail all logs +} + +entrypoint diff --git a/docker/single/marathon.json b/docker/single/marathon.json new file mode 100644 index 0000000000..2c15315e7a --- /dev/null +++ b/docker/single/marathon.json @@ -0,0 +1,31 @@ +{ + "id": "/gitlab", + "ports": [0,0], + "cpus": 2, + "mem": 2048.0, + "disk": 10240.0, + "container": { + "type": "DOCKER", + "docker": { + "network": "HOST", + "image": "sytse/gitlab-ce:7.10.1" + }, + "volumes": [ + { + "containerPath": "/var/opt/gitlab", + "hostPath": "/var/opt/gitlab", + "mode": "RW" + }, + { + "containerPath": "/var/log/gitlab", + "hostPath": "/var/log/gitlab", + "mode": "RW" + }, + { + "containerPath": "/var/opt/gitlab", + "hostPath": "/var/opt/gitlab", + "mode": "RW" + } + ] + } +} \ No newline at end of file diff --git a/docker/troubleshooting.md b/docker/troubleshooting.md index b1b70de599..1b137922bf 100644 --- a/docker/troubleshooting.md +++ b/docker/troubleshooting.md @@ -61,3 +61,11 @@ head /proc/sys/kernel/shmmax /proc/sys/kernel/shmall free -m ``` + +Remove ALL docker containers and images (also non GitLab ones): + +``` +docker rm $(docker ps -a -q) +docker rmi $(docker images -q) +``` + From 342078a76dd7af51ecdf7f3b2e14527f76d9d167 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 16:38:45 +0300 Subject: [PATCH 015/158] Small UI improvements to make project sidebar more neat Signed-off-by: Dmitriy Zaporozhets --- app/assets/stylesheets/pages/projects.scss | 3 +++ app/views/projects/_aside.html.haml | 8 ++++---- app/views/shared/_clone_panel.html.haml | 6 +++--- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/app/assets/stylesheets/pages/projects.scss b/app/assets/stylesheets/pages/projects.scss index afa8aa24ee..e6a4bc62c5 100644 --- a/app/assets/stylesheets/pages/projects.scss +++ b/app/assets/stylesheets/pages/projects.scss @@ -201,9 +201,12 @@ ul.nav.nav-projects-tabs { } .well { + padding: 14px; + h4 { font-weight: normal; margin: 0; + color: #555; } .nav-pills a { diff --git a/app/views/projects/_aside.html.haml b/app/views/projects/_aside.html.haml index 1a7ab74c93..b3ae256949 100644 --- a/app/views/projects/_aside.html.haml +++ b/app/views/projects/_aside.html.haml @@ -51,10 +51,10 @@ - if @project.forked_from_project .well - %i.fa.fa-code-fork.project-fork-icon - Forked from: - %br - = link_to @project.forked_from_project.name_with_namespace, project_path(@project.forked_from_project) + %h4 + Forked from + .pull-right + = link_to @project.forked_from_project.namespace.try(:name), project_path(@project.forked_from_project) - if version = @repository.version diff --git a/app/views/shared/_clone_panel.html.haml b/app/views/shared/_clone_panel.html.haml index 96357cac59..aaab0d35f5 100644 --- a/app/views/shared/_clone_panel.html.haml +++ b/app/views/shared/_clone_panel.html.haml @@ -3,7 +3,7 @@ .input-group-btn %button{ | :type => 'button', | - class: "btn #{ 'active' if default_clone_protocol == 'ssh' }#{ ' has_tooltip' if current_user && current_user.require_ssh_key? }", | + class: "btn btn-sm #{ 'active' if default_clone_protocol == 'ssh' }#{ ' has_tooltip' if current_user && current_user.require_ssh_key? }", | :"data-clone" => project.ssh_url_to_repo, | :"data-title" => "Add an SSH key to your profile
to pull or push via SSH", :"data-html" => "true", @@ -11,13 +11,13 @@ SSH %button{ | :type => 'button', | - class: "btn #{ 'active' if default_clone_protocol == 'http' }#{ ' has_tooltip' if current_user && current_user.require_password? }", | + class: "btn btn-sm #{ 'active' if default_clone_protocol == 'http' }#{ ' has_tooltip' if current_user && current_user.require_password? }", | :"data-clone" => project.http_url_to_repo, | :"data-title" => "Set a password on your account
to pull or push via #{gitlab_config.protocol.upcase}", :"data-html" => "true", :"data-container" => "body"} = gitlab_config.protocol.upcase - = text_field_tag :project_clone, default_url_to_repo(project), class: "js-select-on-focus form-control", readonly: true + = text_field_tag :project_clone, default_url_to_repo(project), class: "js-select-on-focus form-control input-sm", readonly: true - if project.kind_of?(Project) .input-group-addon .visibility-level-label.has_tooltip{'data-title' => "#{visibility_level_label(project.visibility_level)} project" } From fb00ed9983dd515142352f06a1d9a0dd4751d167 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 16:52:04 +0300 Subject: [PATCH 016/158] Make README link on project home look nicer Signed-off-by: Dmitriy Zaporozhets --- app/views/projects/_section.html.haml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/app/views/projects/_section.html.haml b/app/views/projects/_section.html.haml index e6b04c5a34..0b7f4cb780 100644 --- a/app/views/projects/_section.html.haml +++ b/app/views/projects/_section.html.haml @@ -26,9 +26,10 @@ - if readme = @repository.readme .tab-pane#tab-readme %article.readme-holder#README - = link_to namespace_project_blob_path(@project.namespace, @project, tree_join(@repository.root_ref, readme.name)) do - %h4.readme-file-title - %i.fa.fa-file - = readme.name + .clearfix + %small.pull-right + = link_to namespace_project_blob_path(@project.namespace, @project, tree_join(@repository.root_ref, readme.name)), class: 'light' do + %i.fa.fa-file + = readme.name .wiki = render_readme(readme) From c66e5751796adc979fad684ce9b71286dd43c22f Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 17:00:19 +0300 Subject: [PATCH 017/158] Fix tests Signed-off-by: Dmitriy Zaporozhets --- features/steps/project/forked_merge_requests.rb | 2 +- features/steps/project/merge_requests.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/features/steps/project/forked_merge_requests.rb b/features/steps/project/forked_merge_requests.rb index 63ad90e124..94d21d28a0 100644 --- a/features/steps/project/forked_merge_requests.rb +++ b/features/steps/project/forked_merge_requests.rb @@ -46,7 +46,7 @@ class Spinach::Features::ProjectForkedMergeRequests < Spinach::FeatureSteps end step 'I submit the merge request' do - click_button "Submit merge request" + click_button "Submit new merge request" end step 'I follow the target commit link' do diff --git a/features/steps/project/merge_requests.rb b/features/steps/project/merge_requests.rb index bb1f9f129c..12d4d81159 100644 --- a/features/steps/project/merge_requests.rb +++ b/features/steps/project/merge_requests.rb @@ -77,7 +77,7 @@ class Spinach::Features::ProjectMergeRequests < Spinach::FeatureSteps select "feature", from: "merge_request_target_branch" click_button "Compare branches" fill_in "merge_request_title", with: "Wiki Feature" - click_button "Submit merge request" + click_button "Submit new merge request" end step 'project "Shop" have "Bug NS-04" open merge request' do From 317ed1fa9034eba91af95b2e98a3576497cbaf3a Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 17:39:18 +0300 Subject: [PATCH 018/158] Revert "Allow to configure smtp and sendmail in gitlab.yml" --- app/mailers/devise_mailer.rb | 4 +- app/mailers/emails/projects.rb | 2 +- app/mailers/notify.rb | 8 +-- config/gitlab.teatro.yml | 7 +- config/gitlab.yml.example | 67 +++---------------- config/initializers/1_settings.rb | 24 ++----- .../initializers/disable_email_interceptor.rb | 4 +- config/initializers/email_settings.rb | 5 -- config/initializers/smtp_settings.rb.sample | 22 ++++++ lib/tasks/gitlab/check.rake | 2 +- spec/mailers/notify_spec.rb | 6 +- 11 files changed, 51 insertions(+), 100 deletions(-) delete mode 100644 config/initializers/email_settings.rb create mode 100644 config/initializers/smtp_settings.rb.sample diff --git a/app/mailers/devise_mailer.rb b/app/mailers/devise_mailer.rb index e4b65c19b3..b616add283 100644 --- a/app/mailers/devise_mailer.rb +++ b/app/mailers/devise_mailer.rb @@ -1,4 +1,4 @@ class DeviseMailer < Devise::Mailer - default from: "#{Gitlab.config.outgoing_emails.display_name} <#{Gitlab.config.outgoing_emails.from}>" - default reply_to: Gitlab.config.outgoing_emails.reply_to + default from: "#{Gitlab.config.gitlab.email_display_name} <#{Gitlab.config.gitlab.email_from}>" + default reply_to: Gitlab.config.gitlab.email_reply_to end diff --git a/app/mailers/emails/projects.rb b/app/mailers/emails/projects.rb index c6bc716930..9cb7077e59 100644 --- a/app/mailers/emails/projects.rb +++ b/app/mailers/emails/projects.rb @@ -129,7 +129,7 @@ module Emails if send_from_committer_email && can_send_from_user_email?(@author) @author.email else - Gitlab.config.outgoing_emails.reply_to + Gitlab.config.gitlab.email_reply_to end mail(from: sender(author_id, send_from_committer_email), diff --git a/app/mailers/notify.rb b/app/mailers/notify.rb index a9800f7f10..79fb48b00d 100644 --- a/app/mailers/notify.rb +++ b/app/mailers/notify.rb @@ -17,7 +17,7 @@ class Notify < ActionMailer::Base helper_method :current_user, :can? default from: Proc.new { default_sender_address.format } - default reply_to: Gitlab.config.outgoing_emails.reply_to + default reply_to: Gitlab.config.gitlab.email_reply_to # Just send email with 2 seconds delay def self.delay @@ -50,9 +50,9 @@ class Notify < ActionMailer::Base # The default email address to send emails from def default_sender_address - Mail::Address.new(Gitlab.config.outgoing_emails.from).tap do |address| - address.display_name = Gitlab.config.outgoing_emails.display_name - end + address = Mail::Address.new(Gitlab.config.gitlab.email_from) + address.display_name = Gitlab.config.gitlab.email_display_name + address end def can_send_from_user_email?(sender) diff --git a/config/gitlab.teatro.yml b/config/gitlab.teatro.yml index 0713350384..f0656400be 100644 --- a/config/gitlab.teatro.yml +++ b/config/gitlab.teatro.yml @@ -7,6 +7,8 @@ production: &base user: root + email_from: example@example.com + support_email: support@example.com default_projects_features: @@ -17,9 +19,6 @@ production: &base snippets: false visibility_level: "private" # can be "private" | "internal" | "public" - outgoing_emails: - from: example@example.com - issues_tracker: gravatar: @@ -76,8 +75,6 @@ test: gitlab: host: localhost port: 80 - outgoing_emails: - delivery_method: :test issues_tracker: redmine: title: "Redmine" diff --git a/config/gitlab.yml.example b/config/gitlab.yml.example index f0b65e1e90..bd2081688d 100644 --- a/config/gitlab.yml.example +++ b/config/gitlab.yml.example @@ -10,7 +10,7 @@ # How to use: # 1. Copy file as gitlab.yml # 2. Update gitlab -> host with your fully qualified domain name -# 3. Update outgoing_emails -> from +# 3. Update gitlab -> email_from # 4. If you installed Git from source, change git -> bin_path to /usr/local/bin/git # IMPORTANT: If Git was installed in a different location use that instead. # You can check with `which git`. If a wrong path of Git is specified, it will @@ -46,6 +46,16 @@ production: &base # To see all available zones, run `bundle exec rake time:zones:all RAILS_ENV=production` # time_zone: 'UTC' + ## Email settings + # Uncomment and set to false if you need to disable email sending from GitLab (default: true) + # email_enabled: true + # Email address used in the "From" field in mails sent by GitLab + email_from: example@example.com + email_display_name: GitLab + email_reply_to: noreply@example.com + + # Email server smtp settings are in config/initializers/smtp_settings.rb.sample + # default_can_create_group: false # default: true # username_changing_enabled: false # default: true - User can change her username/namespace ## Default theme @@ -79,57 +89,6 @@ production: &base # The default is 'tmp/repositories' relative to the root of the Rails app. # repository_downloads_path: tmp/repositories - ## Settings for sending emails - outgoing_emails: - - # Set to false if you need to disable email sending from GitLab (default: true). - enabled: true - - # Email address and name used in the "From" field in mails sent by GitLab. - from: example@example.com - display_name: GitLab - - # Email address used in "Reply-To" field in mails sent by GitLab. - reply_to: noreply@example.com - - # Defines a delivery method. Possible values are: - # :sendmail - send emails using sendmail command (default). - # :smtp - send emails using SMTP server. - # :letter_opener - open sent emails in browser. - # :test - save emails to ActionMailer::Base.deliveries array. - delivery_method: :sendmail - - # Settings for the :sendmail delivery method. - sendmail_settings: - # The location of the sendmail executable (default: /usr/sbin/sendmail). - location: /usr/sbin/sendmail - - # The command line arguments to be passed to sendmail (default: -i -t). - arguments: '-i -t' - - # Settings for the :smtp delivery method. - smtp_settings: - # Hostname of the SMTP server used to send emails (default: localhost). - address: localhost - - # Port of the SMTP server used to send emails (default: 25). - port: 25 - - # If your mail server requires authentication, you need to specify the - # authentication type here. Possible values are: - # :plain - send the password in the clear text. - # :login - send password Base64 encoded. - # :cram_md5 - combines a Challenge/Response mechanism to exchange information and - # a cryptographic Message Digest 5 algorithm to hash important information. - # authentication: :plain - # user_name: 'gitlab' - # password: '123456' - - # Detects if STARTTLS is enabled in your SMTP server and starts to use it. - # Set this to false if there is a problem with your server certificate - # that you cannot resolve. It should be true for Gmail SMTP. - # enable_starttls_auto: true - ## Gravatar ## For Libravatar see: http://doc.gitlab.com/ce/customization/libravatar.html gravatar: @@ -336,8 +295,6 @@ production: &base development: <<: *base - outgoing_emails: - delivery_method: :letter_opener test: <<: *base @@ -346,8 +303,6 @@ test: gitlab: host: localhost port: 80 - outgoing_emails: - delivery_method: :test # When you run tests we clone and setup gitlab-shell # In order to setup it correctly you need to specify diff --git a/config/initializers/1_settings.rb b/config/initializers/1_settings.rb index ce0023e079..e5ac66a232 100644 --- a/config/initializers/1_settings.rb +++ b/config/initializers/1_settings.rb @@ -105,6 +105,10 @@ Settings.gitlab['https'] = false if Settings.gitlab['https'].nil? Settings.gitlab['port'] ||= Settings.gitlab.https ? 443 : 80 Settings.gitlab['relative_url_root'] ||= ENV['RAILS_RELATIVE_URL_ROOT'] || '' Settings.gitlab['protocol'] ||= Settings.gitlab.https ? "https" : "http" +Settings.gitlab['email_enabled'] ||= true if Settings.gitlab['email_enabled'].nil? +Settings.gitlab['email_from'] ||= "gitlab@#{Settings.gitlab.host}" +Settings.gitlab['email_display_name'] ||= "GitLab" +Settings.gitlab['email_reply_to'] ||= "noreply@#{Settings.gitlab.host}" Settings.gitlab['url'] ||= Settings.send(:build_gitlab_url) Settings.gitlab['user'] ||= 'git' Settings.gitlab['user_home'] ||= begin @@ -130,26 +134,6 @@ Settings.gitlab.default_projects_features['visibility_level'] = Settings.send Settings.gitlab['repository_downloads_path'] = File.absolute_path(Settings.gitlab['repository_downloads_path'] || 'tmp/repositories', Rails.root) Settings.gitlab['restricted_signup_domains'] ||= [] -# -# Outgoing emails -# -Settings['outgoing_emails'] ||= Settingslogic.new({}) -Settings['outgoing_emails'].tap do |opts| - # For backward compatibility. TODO remove in next major release. - opts['enabled'] ||= Settings.gitlab['email_enabled'] - opts['from'] ||= Settings.gitlab['email_from'] - opts['display_name'] ||= Settings.gitlab['display_name'] - opts['reply_to'] ||= Settings.gitlab['email_reply_to'] - - opts['enabled'] ||= opts['enabled'].nil? - opts['display_name'] ||= "GitLab" - opts['from'] ||= "gitlab@#{Settings.gitlab.host}" - opts['reply_to'] ||= "noreply@#{Settings.gitlab.host}" - opts['delivery_method'] ||= :sendmail - opts['sendmail_settings'] ||= {} - opts['smtp_settings'] ||= {} -end - # # Gravatar # diff --git a/config/initializers/disable_email_interceptor.rb b/config/initializers/disable_email_interceptor.rb index 122a582991..c76a6b8b19 100644 --- a/config/initializers/disable_email_interceptor.rb +++ b/config/initializers/disable_email_interceptor.rb @@ -1,4 +1,2 @@ # Interceptor in lib/disable_email_interceptor.rb -unless Gitlab.config.outgoing_emails.enabled - ActionMailer::Base.register_interceptor(DisableEmailInterceptor) -end +ActionMailer::Base.register_interceptor(DisableEmailInterceptor) unless Gitlab.config.gitlab.email_enabled diff --git a/config/initializers/email_settings.rb b/config/initializers/email_settings.rb deleted file mode 100644 index 99cb09ebfd..0000000000 --- a/config/initializers/email_settings.rb +++ /dev/null @@ -1,5 +0,0 @@ -Gitlab.config.outgoing_emails.tap do |c| - Gitlab::Application.config.action_mailer.delivery_method = c.delivery_method - ActionMailer::Base.smtp_settings = c.smtp_settings.symbolize_keys - ActionMailer::Base.sendmail_settings = c.sendmail_settings.symbolize_keys -end diff --git a/config/initializers/smtp_settings.rb.sample b/config/initializers/smtp_settings.rb.sample new file mode 100644 index 0000000000..f0fe2fdfa4 --- /dev/null +++ b/config/initializers/smtp_settings.rb.sample @@ -0,0 +1,22 @@ +# To enable smtp email delivery for your GitLab instance do the following: +# 1. Rename this file to smtp_settings.rb +# 2. Edit settings inside this file +# 3. Restart GitLab instance +# +# For full list of options and their values see http://api.rubyonrails.org/classes/ActionMailer/Base.html +# + +if Rails.env.production? + Gitlab::Application.config.action_mailer.delivery_method = :smtp + + ActionMailer::Base.smtp_settings = { + address: "email.server.com", + port: 456, + user_name: "smtp", + password: "123456", + domain: "gitlab.company.com", + authentication: :login, + enable_starttls_auto: true, + openssl_verify_mode: 'peer' # See ActionMailer documentation for other possible options + } +end diff --git a/lib/tasks/gitlab/check.rake b/lib/tasks/gitlab/check.rake index f967fddf56..1a6303b6c8 100644 --- a/lib/tasks/gitlab/check.rake +++ b/lib/tasks/gitlab/check.rake @@ -320,7 +320,7 @@ namespace :gitlab do options = { "user.name" => "GitLab", - "user.email" => Gitlab.config.outgoing_emails.from, + "user.email" => Gitlab.config.gitlab.email_from, "core.autocrlf" => "input" } correct_options = options.map do |name, value| diff --git a/spec/mailers/notify_spec.rb b/spec/mailers/notify_spec.rb index f626efe1f7..dbcf7286e4 100644 --- a/spec/mailers/notify_spec.rb +++ b/spec/mailers/notify_spec.rb @@ -5,9 +5,9 @@ describe Notify do include EmailSpec::Matchers include RepoHelpers - let(:gitlab_sender_display_name) { Gitlab.config.outgoing_emails.display_name } - let(:gitlab_sender) { Gitlab.config.outgoing_emails.from } - let(:gitlab_sender_reply_to) { Gitlab.config.outgoing_emails.reply_to } + let(:gitlab_sender_display_name) { Gitlab.config.gitlab.email_display_name } + let(:gitlab_sender) { Gitlab.config.gitlab.email_from } + let(:gitlab_sender_reply_to) { Gitlab.config.gitlab.email_reply_to } let(:recipient) { create(:user, email: 'recipient@example.com') } let(:project) { create(:project) } From 6518c728acca65732f89a22a5cbbb75fe3f238f6 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Wed, 6 May 2015 16:39:30 +0200 Subject: [PATCH 019/158] Small improvements to the dockerfile. --- docker/README.md | 4 +--- docker/troubleshooting.md | 2 ++ 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docker/README.md b/docker/README.md index b88d570f08..537855624a 100644 --- a/docker/README.md +++ b/docker/README.md @@ -55,9 +55,7 @@ sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 sy Troubleshoot with: ``` -sudo docker run -ti -e TERM=linux --name gitlab-ce --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash -/opt/gitlab/embedded/bin/runsvdir-start & gitlab-ctl reconfigure -gitlab-ctl start +sudo docker run -ti -e TERM=linux --name gitlab-ce-troubleshoot --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash /usr/local/bin/wrapper ``` Build and use app and data images diff --git a/docker/troubleshooting.md b/docker/troubleshooting.md index 1b137922bf..5827f2185d 100644 --- a/docker/troubleshooting.md +++ b/docker/troubleshooting.md @@ -62,6 +62,8 @@ free -m ``` +# Cleanup + Remove ALL docker containers and images (also non GitLab ones): ``` From 48a398328f39a00a2e690901ca6a2da2b492a11c Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 15:11:55 +0000 Subject: [PATCH 020/158] Replace outdated gif in README with link to website --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 130351b15b..1ceac4621d 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ The source of GitLab Community Edition is [hosted on GitLab.com](https://gitlab. ## Open source software to collaborate on code -![Animated screenshots](https://about.gitlab.com/images/animated/compiled.gif) +To see how GitLab looks please see the [features page on our website](https://about.gitlab.com/features/). - Manage Git repositories with fine grained access controls that keep your code secure - Perform code reviews and enhance collaboration with merge requests @@ -101,4 +101,4 @@ Please see [Getting help for GitLab](https://about.gitlab.com/getting-help/) on ## Is it awesome? Thanks for [asking this question](https://twitter.com/supersloth/status/489462789384056832) Joshua. -[These people](https://twitter.com/gitlab/favorites) seem to like it. +[These people](https://twitter.com/gitlab/favorites) seem to like it. \ No newline at end of file From 5b3c823fc9fe6788184a29dec7393158738ef584 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Wed, 6 May 2015 18:27:44 +0300 Subject: [PATCH 021/158] Make header title bold and line between header lighter Signed-off-by: Dmitriy Zaporozhets --- app/assets/stylesheets/generic/header.scss | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/app/assets/stylesheets/generic/header.scss b/app/assets/stylesheets/generic/header.scss index df1553d859..fcd62373bf 100644 --- a/app/assets/stylesheets/generic/header.scss +++ b/app/assets/stylesheets/generic/header.scss @@ -15,7 +15,7 @@ header { padding: 0; background: #FFF; - border-bottom: 1px solid #DDD; + border-bottom: 1px solid #EEE; filter: none; .title { @@ -25,13 +25,13 @@ header { margin-left: 25px; font-size: 18px; line-height: 44px; - font-weight: normal; - color: #555; + font-weight: bold; + color: #444; @include str-truncated(37%); a { - color: #555; + color: #444; &:hover { text-decoration: underline; } From 5ed7a39dc7aeff97bd7dceb9e125b352223e29a3 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 12:25:23 -0400 Subject: [PATCH 022/158] Fix tooltips for event filter links --- app/helpers/events_helper.rb | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/app/helpers/events_helper.rb b/app/helpers/events_helper.rb index c9fd0f0362..18c75a8726 100644 --- a/app/helpers/events_helper.rb +++ b/app/helpers/events_helper.rb @@ -25,12 +25,16 @@ module EventsHelper def event_filter_link(key, tooltip) key = key.to_s - active = if @event_filter.active? key - 'active' - end + active = 'active' if @event_filter.active?(key) + link_opts = { + class: 'event_filter_link', + id: "#{key}_event_filter", + title: "Filter by #{tooltip.downcase}", + data: { toggle: 'tooltip', placement: 'top' } + } content_tag :li, class: "filter_icon #{active}" do - link_to request.path, class: 'has_tooltip event_filter_link', id: "#{key}_event_filter", 'data-original-title' => 'Filter by ' + tooltip.downcase do + link_to request.path, link_opts do icon(icon_for_event[key]) + content_tag(:span, ' ' + tooltip) end end From bc14d223a781322d672ca54d7afa946532b66d56 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 15:02:41 -0400 Subject: [PATCH 023/158] Add task_list gem --- Gemfile | 21 +++++++++------------ Gemfile.lock | 3 +++ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/Gemfile b/Gemfile index ea62396d8a..05e26f5b72 100644 --- a/Gemfile +++ b/Gemfile @@ -87,20 +87,17 @@ gem "six" # Seed data gem "seed-fu" -# Markup pipeline for GitLab +# Markdown and HTML processing gem 'html-pipeline', '~> 1.11.0' - -# Markdown to HTML -gem "github-markup" - -# Required markup gems by github-markdown -gem 'redcarpet', '~> 3.2.3' +gem 'task_list', '~> 1.0.0', require: 'task_list/railtie' +gem 'github-markup' +gem 'redcarpet', '~> 3.2.3' gem 'RedCloth' -gem 'rdoc', '~>3.6' -gem 'org-ruby', '= 0.9.12' -gem 'creole', '~>0.3.6' -gem 'wikicloth', '=0.8.1' -gem 'asciidoctor', '= 0.1.4' +gem 'rdoc', '~>3.6' +gem 'org-ruby', '= 0.9.12' +gem 'creole', '~>0.3.6' +gem 'wikicloth', '=0.8.1' +gem 'asciidoctor', '= 0.1.4' # Diffs gem 'diffy', '~> 3.0.3' diff --git a/Gemfile.lock b/Gemfile.lock index 676a519790..3334917797 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -597,6 +597,8 @@ GEM stamp (0.5.0) state_machine (1.2.0) stringex (2.5.2) + task_list (1.0.2) + html-pipeline temple (0.6.7) term-ansicolor (1.2.2) tins (~> 0.8) @@ -789,6 +791,7 @@ DEPENDENCIES spring-commands-spinach (= 1.0.0) stamp state_machine + task_list (~> 1.0.0) test_after_commit thin tinder (~> 1.9.2) From 5ef8bd33d03947528979540c6f7013c2b4c2a4ce Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 15:50:22 -0400 Subject: [PATCH 024/158] Remove taskable.js --- .../javascripts/behaviors/taskable.js.coffee | 21 ------------------- 1 file changed, 21 deletions(-) delete mode 100644 app/assets/javascripts/behaviors/taskable.js.coffee diff --git a/app/assets/javascripts/behaviors/taskable.js.coffee b/app/assets/javascripts/behaviors/taskable.js.coffee deleted file mode 100644 index ddce71c188..0000000000 --- a/app/assets/javascripts/behaviors/taskable.js.coffee +++ /dev/null @@ -1,21 +0,0 @@ -window.updateTaskState = (taskableType) -> - objType = taskableType.data - isChecked = $(this).prop("checked") - if $(this).is(":checked") - stateEvent = "task_check" - else - stateEvent = "task_uncheck" - - taskableUrl = $("form.edit-" + objType).first().attr("action") - taskableNum = taskableUrl.match(/\d+$/) - taskNum = 0 - $("li.task-list-item input:checkbox").each( (index, e) => - if e == this - taskNum = index + 1 - ) - - $.ajax - type: "PATCH" - url: taskableUrl - data: objType + "[state_event]=" + stateEvent + - "&" + objType + "[task_num]=" + taskNum From 61c73575c348b3fd06047af130680f665f8ea724 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 15:51:32 -0400 Subject: [PATCH 025/158] Add TaskList::Filter to pipeline --- lib/gitlab/markdown.rb | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/lib/gitlab/markdown.rb b/lib/gitlab/markdown.rb index 8348e28d0f..bbae6ae3e7 100644 --- a/lib/gitlab/markdown.rb +++ b/lib/gitlab/markdown.rb @@ -1,4 +1,5 @@ require 'html/pipeline' +require 'task_list/filter' module Gitlab # Custom parser for GitLab-flavored Markdown @@ -76,9 +77,9 @@ module Gitlab text = result[:output].to_html(save_with: save_options) - if options[:parse_tasks] - text = parse_tasks(text) - end + # if options[:parse_tasks] + # text = parse_tasks(text) + # end text.html_safe end @@ -106,7 +107,9 @@ module Gitlab Gitlab::Markdown::SnippetReferenceFilter, Gitlab::Markdown::CommitRangeReferenceFilter, Gitlab::Markdown::CommitReferenceFilter, - Gitlab::Markdown::LabelReferenceFilter + Gitlab::Markdown::LabelReferenceFilter, + + TaskList::Filter ] end From 18baf550e1ecfd0a8cd356490c3cae4e4566fb8f Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 15:51:54 -0400 Subject: [PATCH 026/158] Add task_list JS --- app/assets/javascripts/application.js.coffee | 1 + 1 file changed, 1 insertion(+) diff --git a/app/assets/javascripts/application.js.coffee b/app/assets/javascripts/application.js.coffee index bb9da14701..3b8eaa143e 100644 --- a/app/assets/javascripts/application.js.coffee +++ b/app/assets/javascripts/application.js.coffee @@ -41,6 +41,7 @@ #= require shortcuts_issuable #= require shortcuts_network #= require cal-heatmap +#= require task_list #= require_tree . window.slugify = (text) -> From a3d55db91053955a36c6a48c9ad596752a3aba40 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 16:47:31 -0400 Subject: [PATCH 027/158] Update task list behavior for Issues --- app/assets/javascripts/issue.js.coffee | 27 ++++++++++++++++--- .../projects/issues/_issue_context.html.haml | 2 +- app/views/projects/issues/show.html.haml | 4 ++- 3 files changed, 27 insertions(+), 6 deletions(-) diff --git a/app/assets/javascripts/issue.js.coffee b/app/assets/javascripts/issue.js.coffee index 4e2e6550eb..63d38594d2 100644 --- a/app/assets/javascripts/issue.js.coffee +++ b/app/assets/javascripts/issue.js.coffee @@ -6,11 +6,11 @@ class @Issue $(".context .inline-update").on "change", "#issue_assignee_id", -> $(this).submit() - if $("a.btn-close").length - $("li.task-list-item input:checkbox").prop("disabled", false) + # Prevent duplicate event bindings + @disableTaskList() - $('.task-list-item input:checkbox').off('change') - $('.task-list-item input:checkbox').change('issue', updateTaskState) + if $("a.btn-close").length + @initTaskList() $('.issue-details').waitForImages -> $('.issuable-affix').affix offset: @@ -22,3 +22,22 @@ class @Issue $(@).width($(@).outerWidth()) .on 'affixed-top.bs.affix affixed-bottom.bs.affix', -> $(@).width('') + + initTaskList: -> + $('.issue-details .js-task-list-container').taskList('enable') + $(document).on 'tasklist:changed', '.issue-details .js-task-list-container', @updateTaskList + + disableTaskList: -> + $('.issue-details .js-task-list-container').taskList('disable') + $(document).off 'tasklist:changed', '.issue-details .js-task-list-container' + + # TODO (rspeicher): Make the issue description inline-editable like a note so + # that we can re-use its form here + updateTaskList: -> + patchData = {} + patchData['issue'] = {'description': $('.js-task-list-field', this).val()} + + $.ajax + type: 'PATCH' + url: $('form.js-issue-update').attr('action') + data: patchData diff --git a/app/views/projects/issues/_issue_context.html.haml b/app/views/projects/issues/_issue_context.html.haml index 7cadb9983f..323f5c84a8 100644 --- a/app/views/projects/issues/_issue_context.html.haml +++ b/app/views/projects/issues/_issue_context.html.haml @@ -1,4 +1,4 @@ -= form_for [@project.namespace.becomes(Namespace), @project, @issue], remote: true, html: {class: 'edit-issue inline-update'} do |f| += form_for [@project.namespace.becomes(Namespace), @project, @issue], remote: true, html: {class: 'edit-issue inline-update js-issue-update'} do |f| %div.prepend-top-20 .issuable-context-title %label diff --git a/app/views/projects/issues/show.html.haml b/app/views/projects/issues/show.html.haml index 81478dfe56..2d248c15f4 100644 --- a/app/views/projects/issues/show.html.haml +++ b/app/views/projects/issues/show.html.haml @@ -31,10 +31,12 @@ = gfm escape_once(@issue.title) %div - if @issue.description.present? - .description + .description.js-task-list-container .wiki = preserve do = markdown(@issue.description, parse_tasks: true) + %textarea.hidden.js-task-list-field + = @issue.description %hr .issue-discussion From 922497560eb7d619a6064d7b1d44c831ccd64ea9 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 16:56:50 -0400 Subject: [PATCH 028/158] Update task list behavior for Notes --- app/assets/javascripts/notes.js.coffee | 17 +++++++++++++++++ app/views/projects/notes/_edit_form.html.haml | 5 ++--- app/views/projects/notes/_note.html.haml | 2 +- 3 files changed, 20 insertions(+), 4 deletions(-) diff --git a/app/assets/javascripts/notes.js.coffee b/app/assets/javascripts/notes.js.coffee index 6dfe10f000..416218d2e0 100644 --- a/app/assets/javascripts/notes.js.coffee +++ b/app/assets/javascripts/notes.js.coffee @@ -11,6 +11,7 @@ class @Notes @setupMainTargetNoteForm() @cleanBinding() @addBinding() + @initTaskList() addBinding: -> # add note to UI after creation @@ -81,6 +82,9 @@ class @Notes $(document).off "click", ".js-note-target-reopen" $(document).off "click", ".js-note-target-close" + $('.note .js-task-list-container').taskList('disable') + $(document).off 'tasklist:changed', '.note .js-task-list-container' + initRefresh: -> clearInterval(Notes.interval) Notes.interval = setInterval => @@ -114,6 +118,7 @@ class @Notes if @isNewNote(note) @note_ids.push(note.id) $('ul.main-notes-list').append(note.html) + @initTaskList() ### Check if note does not exists on page @@ -268,6 +273,8 @@ class @Notes note_li.replaceWith(note.html) note_li.find('.note-edit-form').hide() note_li.find('.note-body > .note-text').show() + note_li.find('js-task-list-container').taskList('enable') + @enableTaskList() ### Called in response to clicking the edit note link @@ -479,3 +486,13 @@ class @Notes else form.find('.js-note-target-reopen').text('Reopen') form.find('.js-note-target-close').text('Close') + + enableTaskList: -> + $('.note .js-task-list-container').taskList('enable') + + initTaskList: -> + @enableTaskList() + $(document).on 'tasklist:changed', '.note .js-task-list-container', @updateTaskList + + updateTaskList: -> + $('form', this).submit() diff --git a/app/views/projects/notes/_edit_form.html.haml b/app/views/projects/notes/_edit_form.html.haml index acb3991d29..d2c0933f8b 100644 --- a/app/views/projects/notes/_edit_form.html.haml +++ b/app/views/projects/notes/_edit_form.html.haml @@ -1,9 +1,8 @@ .note-edit-form = form_for note, url: namespace_project_note_path(@project.namespace, @project, note), method: :put, remote: true, authenticity_token: true do |f| = note_target_fields(note) - = render layout: 'projects/md_preview', locals: { preview_class: "note-text" } do - = render 'projects/zen', f: f, attr: :note, - classes: 'note_text js-note-text' + = render layout: 'projects/md_preview', locals: { preview_class: 'note-text' } do + = render 'projects/zen', f: f, attr: :note, classes: 'note_text js-note-text js-task-list-field' .comment-hints.clearfix .pull-left Comments are parsed with #{link_to "GitLab Flavored Markdown", help_page_path("markdown", "markdown"),{ target: '_blank', tabindex: -1 }} diff --git a/app/views/projects/notes/_note.html.haml b/app/views/projects/notes/_note.html.haml index 0728f8fa42..203ab816e4 100644 --- a/app/views/projects/notes/_note.html.haml +++ b/app/views/projects/notes/_note.html.haml @@ -50,7 +50,7 @@ \-1 - .note-body + .note-body.js-task-list-container .note-text = preserve do = markdown(note.note, {no_header_anchors: true}) From ce29e5cd8f09212afd3274326917c364272a6e39 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 17:00:30 -0400 Subject: [PATCH 029/158] Update task list behavior for Merge Requests --- .../javascripts/merge_request.js.coffee | 27 ++++++++++++++++--- .../merge_requests/show/_context.html.haml | 2 +- .../merge_requests/show/_mr_box.html.haml | 4 ++- 3 files changed, 27 insertions(+), 6 deletions(-) diff --git a/app/assets/javascripts/merge_request.js.coffee b/app/assets/javascripts/merge_request.js.coffee index ae5d088d59..c8affba5b2 100644 --- a/app/assets/javascripts/merge_request.js.coffee +++ b/app/assets/javascripts/merge_request.js.coffee @@ -17,8 +17,11 @@ class @MergeRequest disableButtonIfEmptyField '#commit_message', '.accept_merge_request' + # Prevent duplicate event bindings + @disableTaskList() + if $("a.btn-close").length - $("li.task-list-item input:checkbox").prop("disabled", false) + @initTaskList() $('.merge-request-details').waitForImages -> $('.issuable-affix').affix offset: @@ -77,9 +80,6 @@ class @MergeRequest this.$('.remove_source_branch_in_progress').hide() this.$('.remove_source_branch_widget.failed').show() - $('.task-list-item input:checkbox').off('change') - $('.task-list-item input:checkbox').change('merge_request', updateTaskState) - activateTab: (action) -> this.$('.merge-request-tabs li').removeClass 'active' this.$('.tab-content').hide() @@ -156,3 +156,22 @@ class @MergeRequest else setTimeout(merge_request.mergeInProgress, 3000) dataType: 'json' + + initTaskList: -> + $('.merge-request-details .js-task-list-container').taskList('enable') + $(document).on 'tasklist:changed', '.merge-request-details .js-task-list-container', @updateTaskList + + disableTaskList: -> + $('.merge-request-details .js-task-list-container').taskList('disable') + $(document).off 'tasklist:changed', '.merge-request-details .js-task-list-container' + + # TODO (rspeicher): Make the merge request description inline-editable like a + # note so that we can re-use its form here + updateTaskList: -> + patchData = {} + patchData['merge_request'] = {'description': $('.js-task-list-field', this).val()} + + $.ajax + type: 'PATCH' + url: $('form.js-merge-request-update').attr('action') + data: patchData diff --git a/app/views/projects/merge_requests/show/_context.html.haml b/app/views/projects/merge_requests/show/_context.html.haml index eb80391ebc..7b9ad15a84 100644 --- a/app/views/projects/merge_requests/show/_context.html.haml +++ b/app/views/projects/merge_requests/show/_context.html.haml @@ -1,4 +1,4 @@ -= form_for [@project.namespace.becomes(Namespace), @project, @merge_request], remote: true, html: {class: 'edit-merge_request inline-update'} do |f| += form_for [@project.namespace.becomes(Namespace), @project, @merge_request], remote: true, html: {class: 'edit-merge_request inline-update js-merge-request-update'} do |f| %div.prepend-top-20 .issuable-context-title %label diff --git a/app/views/projects/merge_requests/show/_mr_box.html.haml b/app/views/projects/merge_requests/show/_mr_box.html.haml index ada9ae58b8..80cfeb1c88 100644 --- a/app/views/projects/merge_requests/show/_mr_box.html.haml +++ b/app/views/projects/merge_requests/show/_mr_box.html.haml @@ -3,7 +3,9 @@ %div - if @merge_request.description.present? - .description + .description.js-task-list-container .wiki = preserve do = markdown(@merge_request.description, parse_tasks: true) + %textarea.hidden.js-task-list-field + = @merge_request.description From e167f28549b80141165f42b567073c5dd55f80c5 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 17:47:55 -0400 Subject: [PATCH 030/158] Update Taskable to use TaskList --- app/models/concerns/taskable.rb | 45 ++++++++---------------- spec/support/taskable_shared_examples.rb | 30 ++++++---------- 2 files changed, 25 insertions(+), 50 deletions(-) diff --git a/app/models/concerns/taskable.rb b/app/models/concerns/taskable.rb index bbb3b301a9..6e04578a7d 100644 --- a/app/models/concerns/taskable.rb +++ b/app/models/concerns/taskable.rb @@ -1,51 +1,36 @@ +require 'task_list' + # Contains functionality for objects that can have task lists in their # descriptions. Task list items can be added with Markdown like "* [x] Fix # bugs". # # Used by MergeRequest and Issue module Taskable - TASK_PATTERN_MD = /^(? *[*-] *)\[(?[ xX])\]/.freeze - TASK_PATTERN_HTML = /^
  • (?\s*

    )?\[(?[ xX])\]/.freeze + # Called by `TaskList::Summary` + def task_list_items + return [] if description.blank? - # Change the state of a task list item for this Taskable. Edit the object's - # description by finding the nth task item and changing its checkbox - # placeholder to "[x]" if +checked+ is true, or "[ ]" if it's false. - # Note: task numbering starts with 1 - def update_nth_task(n, checked) - index = 0 - check_char = checked ? 'x' : ' ' - - # Do this instead of using #gsub! so that ActiveRecord detects that a field - # has changed. - self.description = self.description.gsub(TASK_PATTERN_MD) do |match| - index += 1 - case index - when n then "#{$LAST_MATCH_INFO[:bullet]}[#{check_char}]" - else match - end + @task_list_items ||= description.scan(TaskList::Filter::ItemPattern).collect do |item| + # ItemPattern strips out the hyphen, but Item requires it. Rabble rabble. + TaskList::Item.new("- #{item}") end + end - save + def tasks + @tasks ||= TaskList.new(self) end # Return true if this object's description has any task list items. def tasks? - description && description.match(TASK_PATTERN_MD) + tasks.summary.items? end # Return a string that describes the current state of this Taskable's task # list items, e.g. "20 tasks (12 done, 8 unfinished)" def task_status - return nil unless description + return '' if description.blank? - num_tasks = 0 - num_done = 0 - - description.scan(TASK_PATTERN_MD) do - num_tasks += 1 - num_done += 1 unless $LAST_MATCH_INFO[:checked] == ' ' - end - - "#{num_tasks} tasks (#{num_done} done, #{num_tasks - num_done} unfinished)" + sum = tasks.summary + "#{sum.item_count} tasks (#{sum.complete_count} done, #{sum.incomplete_count} unfinished)" end end diff --git a/spec/support/taskable_shared_examples.rb b/spec/support/taskable_shared_examples.rb index 8e5e3a8aaf..77e0ed5f57 100644 --- a/spec/support/taskable_shared_examples.rb +++ b/spec/support/taskable_shared_examples.rb @@ -4,27 +4,13 @@ # subject { Issue or MergeRequest } shared_examples 'a Taskable' do before do - subject.description = < Date: Wed, 29 Apr 2015 17:53:32 -0400 Subject: [PATCH 031/158] Remove all references to `parse_tasks` --- app/views/projects/issues/show.html.haml | 2 +- .../merge_requests/show/_mr_box.html.haml | 2 +- lib/gitlab/markdown.rb | 31 +---- spec/helpers/gitlab_markdown_helper_spec.rb | 109 ------------------ 4 files changed, 5 insertions(+), 139 deletions(-) diff --git a/app/views/projects/issues/show.html.haml b/app/views/projects/issues/show.html.haml index 2d248c15f4..c1438dd718 100644 --- a/app/views/projects/issues/show.html.haml +++ b/app/views/projects/issues/show.html.haml @@ -34,7 +34,7 @@ .description.js-task-list-container .wiki = preserve do - = markdown(@issue.description, parse_tasks: true) + = markdown(@issue.description) %textarea.hidden.js-task-list-field = @issue.description diff --git a/app/views/projects/merge_requests/show/_mr_box.html.haml b/app/views/projects/merge_requests/show/_mr_box.html.haml index 80cfeb1c88..d5aab122b7 100644 --- a/app/views/projects/merge_requests/show/_mr_box.html.haml +++ b/app/views/projects/merge_requests/show/_mr_box.html.haml @@ -6,6 +6,6 @@ .description.js-task-list-container .wiki = preserve do - = markdown(@merge_request.description, parse_tasks: true) + = markdown(@merge_request.description) %textarea.hidden.js-task-list-field = @merge_request.description diff --git a/lib/gitlab/markdown.rb b/lib/gitlab/markdown.rb index bbae6ae3e7..63294aa54c 100644 --- a/lib/gitlab/markdown.rb +++ b/lib/gitlab/markdown.rb @@ -32,9 +32,9 @@ module Gitlab # Public: Parse the provided text with GitLab-Flavored Markdown # # text - the source text - # options - parse_tasks - render tasks - # - xhtml - output XHTML instead of HTML - # - reference_only_path - Use relative path for reference links + # options - A Hash of options used to customize output (default: {}): + # :xhtml - output XHTML instead of HTML + # :reference_only_path - Use relative path for reference links # project - the project # html_options - extra options for the reference links as given to link_to def gfm_with_options(text, options = {}, project = @project, html_options = {}) @@ -46,7 +46,6 @@ module Gitlab text = text.dup.to_str options.reverse_merge!( - parse_tasks: false, xhtml: false, reference_only_path: true ) @@ -77,10 +76,6 @@ module Gitlab text = result[:output].to_html(save_with: save_options) - # if options[:parse_tasks] - # text = parse_tasks(text) - # end - text.html_safe end @@ -112,25 +107,5 @@ module Gitlab TaskList::Filter ] end - - # Turn list items that start with "[ ]" into HTML checkbox inputs. - def parse_tasks(text) - li_tag = '

  • ' - unchecked_box = '' - checked_box = unchecked_box.sub(/\/>$/, 'checked="checked" />') - - # Regexp captures don't seem to work when +text+ is an - # ActiveSupport::SafeBuffer, hence the `String.new` - String.new(text).gsub(Taskable::TASK_PATTERN_HTML) do - checked = $LAST_MATCH_INFO[:checked].downcase == 'x' - p_tag = $LAST_MATCH_INFO[:p_tag] - - if checked - "#{li_tag}#{p_tag}#{checked_box}" - else - "#{li_tag}#{p_tag}#{unchecked_box}" - end - end - end end end diff --git a/spec/helpers/gitlab_markdown_helper_spec.rb b/spec/helpers/gitlab_markdown_helper_spec.rb index b6be82e410..2f67879efd 100644 --- a/spec/helpers/gitlab_markdown_helper_spec.rb +++ b/spec/helpers/gitlab_markdown_helper_spec.rb @@ -43,115 +43,6 @@ describe GitlabMarkdownHelper do expect(gfm(actual)).to match(expected) end end - - context 'parse_tasks: true' do - before(:all) do - @source_text_asterisk = <<-EOT.strip_heredoc - * [ ] valid unchecked task - * [x] valid lowercase checked task - * [X] valid uppercase checked task - * [ ] valid unchecked nested task - * [x] valid checked nested task - - [ ] not an unchecked task - no list item - [x] not a checked task - no list item - - * [ ] not an unchecked task - too many spaces - * [x ] not a checked task - too many spaces - * [] not an unchecked task - no spaces - * Not a task [ ] - not at beginning - EOT - - @source_text_dash = <<-EOT.strip_heredoc - - [ ] valid unchecked task - - [x] valid lowercase checked task - - [X] valid uppercase checked task - - [ ] valid unchecked nested task - - [x] valid checked nested task - EOT - end - - it 'should render checkboxes at beginning of asterisk list items' do - rendered_text = markdown(@source_text_asterisk, parse_tasks: true) - - expect(rendered_text).to match(/ Date: Wed, 29 Apr 2015 16:11:48 -0400 Subject: [PATCH 032/158] Single quotes/icon helper in views --- .../projects/issues/_discussion.html.haml | 8 ++--- app/views/projects/issues/show.html.haml | 14 ++++---- .../merge_requests/show/_context.html.haml | 16 +++++----- app/views/projects/notes/_edit_form.html.haml | 8 ++--- app/views/projects/notes/_note.html.haml | 32 +++++++++---------- 5 files changed, 39 insertions(+), 39 deletions(-) diff --git a/app/views/projects/issues/_discussion.html.haml b/app/views/projects/issues/_discussion.html.haml index ceca265301..2016f5c709 100644 --- a/app/views/projects/issues/_discussion.html.haml +++ b/app/views/projects/issues/_discussion.html.haml @@ -1,9 +1,9 @@ - content_for :note_actions do - if can?(current_user, :modify_issue, @issue) - if @issue.closed? - = link_to 'Reopen Issue', issue_path(@issue, issue: {state_event: :reopen }, status_only: true), method: :put, class: "btn btn-grouped btn-reopen js-note-target-reopen", title: 'Reopen Issue' + = link_to 'Reopen Issue', issue_path(@issue, issue: {state_event: :reopen}, status_only: true), method: :put, class: 'btn btn-grouped btn-reopen js-note-target-reopen', title: 'Reopen Issue' - else - = link_to 'Close Issue', issue_path(@issue, issue: {state_event: :close }, status_only: true), method: :put, class: "btn btn-grouped btn-close js-note-target-close", title: "Close Issue" + = link_to 'Close Issue', issue_path(@issue, issue: {state_event: :close}, status_only: true), method: :put, class: 'btn btn-grouped btn-close js-note-target-close', title: 'Close Issue' = render 'shared/show_aside' @@ -15,11 +15,11 @@ %span= pluralize(@issue.participants(current_user).count, 'participant') - @issue.participants(current_user).each do |participant| = link_to_member(@project, participant, name: false, size: 24) - .voting_notes#notes= render "projects/notes/notes_with_form" + .voting_notes#notes= render 'projects/notes/notes_with_form' %aside.col-md-3 .issuable-affix .clearfix - %span.slead.has_tooltip{:"data-original-title" => 'Cross-project reference'} + %span.slead.has_tooltip{title: 'Cross-project reference'} = cross_project_reference(@project, @issue) %hr .context diff --git a/app/views/projects/issues/show.html.haml b/app/views/projects/issues/show.html.haml index c1438dd718..4c92823503 100644 --- a/app/views/projects/issues/show.html.haml +++ b/app/views/projects/issues/show.html.haml @@ -13,17 +13,17 @@ .pull-right - if can?(current_user, :write_issue, @project) - = link_to new_namespace_project_issue_path(@project.namespace, @project), class: "btn btn-grouped new-issue-link", title: "New Issue", id: "new_issue_link" do - %i.fa.fa-plus + = link_to new_namespace_project_issue_path(@project.namespace, @project), class: 'btn btn-grouped new-issue-link', title: 'New Issue', id: 'new_issue_link' do + = icon('plus') New Issue - if can?(current_user, :modify_issue, @issue) - if @issue.closed? - = link_to 'Reopen', issue_path(@issue, issue: {state_event: :reopen }, status_only: true), method: :put, class: "btn btn-grouped btn-reopen" + = link_to 'Reopen', issue_path(@issue, issue: {state_event: :reopen}, status_only: true), method: :put, class: 'btn btn-grouped btn-reopen' - else - = link_to 'Close', issue_path(@issue, issue: {state_event: :close }, status_only: true), method: :put, class: "btn btn-grouped btn-close", title: "Close Issue" + = link_to 'Close', issue_path(@issue, issue: {state_event: :close}, status_only: true), method: :put, class: 'btn btn-grouped btn-close', title: 'Close Issue' - = link_to edit_namespace_project_issue_path(@project.namespace, @project, @issue), class: "btn btn-grouped issuable-edit" do - %i.fa.fa-pencil-square-o + = link_to edit_namespace_project_issue_path(@project.namespace, @project, @issue), class: 'btn btn-grouped issuable-edit' do + = icon('pencil-square-o') Edit %hr @@ -40,4 +40,4 @@ %hr .issue-discussion - = render "projects/issues/discussion" + = render 'projects/issues/discussion' diff --git a/app/views/projects/merge_requests/show/_context.html.haml b/app/views/projects/merge_requests/show/_context.html.haml index 7b9ad15a84..a5a821c184 100644 --- a/app/views/projects/merge_requests/show/_context.html.haml +++ b/app/views/projects/merge_requests/show/_context.html.haml @@ -19,13 +19,13 @@ %span.back-to-milestone = link_to namespace_project_milestone_path(@project.namespace, @project, @merge_request.milestone) do %strong - %i.fa.fa-clock-o + = icon('clock-o') = @merge_request.milestone.title - else none .issuable-context-selectbox - if can?(current_user, :modify_merge_request, @merge_request) - = f.select(:milestone_id, milestone_options(@merge_request), { include_blank: "Select milestone" }, {class: 'select2 select2-compact js-select2 js-milestone'}) + = f.select(:milestone_id, milestone_options(@merge_request), { include_blank: 'Select milestone' }, {class: 'select2 select2-compact js-select2 js-milestone'}) = hidden_field_tag :merge_request_context = f.submit class: 'btn' @@ -35,13 +35,13 @@ %label Subscription: %button.btn.btn-block.subscribe-button{:type => 'button'} - %i.fa.fa-eye - %span= @merge_request.subscribed?(current_user) ? "Unsubscribe" : "Subscribe" - - subscribtion_status = @merge_request.subscribed?(current_user) ? "subscribed" : "unsubscribed" - .subscription-status{"data-status" => subscribtion_status} - .description-block.unsubscribed{class: ( "hidden" if @merge_request.subscribed?(current_user) )} + = icon('eye') + %span= @merge_request.subscribed?(current_user) ? 'Unsubscribe' : 'Subscribe' + - subscribtion_status = @merge_request.subscribed?(current_user) ? 'subscribed' : 'unsubscribed' + .subscription-status{data: {status: subscribtion_status}} + .description-block.unsubscribed{class: ( 'hidden' if @merge_request.subscribed?(current_user) )} You're not receiving notifications from this thread. - .description-block.subscribed{class: ( "hidden" unless @merge_request.subscribed?(current_user) )} + .description-block.subscribed{class: ( 'hidden' unless @merge_request.subscribed?(current_user) )} You're receiving notifications because you're subscribed to this thread. :coffeescript diff --git a/app/views/projects/notes/_edit_form.html.haml b/app/views/projects/notes/_edit_form.html.haml index d2c0933f8b..a663950f03 100644 --- a/app/views/projects/notes/_edit_form.html.haml +++ b/app/views/projects/notes/_edit_form.html.haml @@ -5,10 +5,10 @@ = render 'projects/zen', f: f, attr: :note, classes: 'note_text js-note-text js-task-list-field' .comment-hints.clearfix - .pull-left Comments are parsed with #{link_to "GitLab Flavored Markdown", help_page_path("markdown", "markdown"),{ target: '_blank', tabindex: -1 }} - .pull-right Attach files by dragging & dropping or #{link_to "selecting them", '#', class: 'markdown-selector', tabindex: -1 }. + .pull-left Comments are parsed with #{link_to 'GitLab Flavored Markdown', help_page_path('markdown', 'markdown'),{ target: '_blank', tabindex: -1 }} + .pull-right Attach files by dragging & dropping or #{link_to 'selecting them', '#', class: 'markdown-selector', tabindex: -1 }. .note-form-actions .buttons - = f.submit 'Save Comment', class: "btn btn-primary btn-save btn-grouped js-comment-button" - = link_to 'Cancel', "#", class: "btn btn-cancel note-edit-cancel" \ No newline at end of file + = f.submit 'Save Comment', class: 'btn btn-primary btn-save btn-grouped js-comment-button' + = link_to 'Cancel', '#', class: 'btn btn-cancel note-edit-cancel' diff --git a/app/views/projects/notes/_note.html.haml b/app/views/projects/notes/_note.html.haml index 203ab816e4..d582b247a4 100644 --- a/app/views/projects/notes/_note.html.haml +++ b/app/views/projects/notes/_note.html.haml @@ -2,28 +2,28 @@ .timeline-entry-inner .timeline-icon - if note.system - %span.fa.fa-circle + %span= icon('circle') - else = link_to user_path(note.author) do - = image_tag avatar_icon(note.author_email), class: "avatar s40", alt: '' + = image_tag avatar_icon(note.author_email), class: 'avatar s40', alt: '' .timeline-content .note-header .note-actions = link_to "##{dom_id(note)}", name: dom_id(note) do - %i.fa.fa-link + = icon('link') Link here   - if can?(current_user, :admin_note, note) && note.editable? - = link_to "#", title: "Edit comment", class: "js-note-edit" do - %i.fa.fa-pencil-square-o + = link_to '#', title: 'Edit comment', class: 'js-note-edit' do + = icon('pencil-square-o') Edit   - = link_to namespace_project_note_path(@project.namespace, @project, note), title: "Remove comment", method: :delete, data: { confirm: 'Are you sure you want to remove this comment?' }, remote: true, class: "danger js-note-delete" do - %i.fa.fa-trash-o.cred + = link_to namespace_project_note_path(@project.namespace, @project, note), title: 'Remove comment', method: :delete, data: { confirm: 'Are you sure you want to remove this comment?' }, remote: true, class: 'danger js-note-delete' do + = icon('trash-o', class: 'cred') Remove - if note.system = link_to user_path(note.author) do - = image_tag avatar_icon(note.author_email), class: "avatar s16", alt: '' + = image_tag avatar_icon(note.author_email), class: 'avatar s16', alt: '' = link_to_member(@project, note.author, avatar: false) %span.author-username = '@' + note.author.username @@ -33,20 +33,20 @@ - if note.superceded?(@notes) - if note.upvote? %span.vote.upvote.label.label-gray.strikethrough - %i.fa.fa-thumbs-up + = icon('thumbs-up') \+1 - if note.downvote? %span.vote.downvote.label.label-gray.strikethrough - %i.fa.fa-thumbs-down + = icon('thumbs-down') \-1 - else - if note.upvote? %span.vote.upvote.label.label-success - %i.fa.fa-thumbs-up + = icon('thumbs-up') \+1 - if note.downvote? %span.vote.downvote.label.label-danger - %i.fa.fa-thumbs-down + = icon('thumbs-down') \-1 @@ -62,10 +62,10 @@ = link_to note.attachment.url, target: '_blank' do = image_tag note.attachment.url, class: 'note-image-attach' .attachment - = link_to note.attachment.url, target: "_blank" do - %i.fa.fa-paperclip + = link_to note.attachment.url, target: '_blank' do + = icon('paperclip') = note.attachment_identifier = link_to delete_attachment_namespace_project_note_path(@project.namespace, @project, note), - title: "Delete this attachment", method: :delete, remote: true, data: { confirm: 'Are you sure you want to remove the attachment?' }, class: "danger js-note-attachment-delete" do - %i.fa.fa-trash-o.cred + title: 'Delete this attachment', method: :delete, remote: true, data: { confirm: 'Are you sure you want to remove the attachment?' }, class: 'danger js-note-attachment-delete' do + = icon('trash-o', class: 'cred') .clear From 725ff58bc891bfdad60f55db6b0dacdba11b1b20 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 20:34:43 -0400 Subject: [PATCH 033/158] Don't add js-task-list-container to things the user can't edit --- app/helpers/notes_helper.rb | 4 ++++ app/views/projects/issues/show.html.haml | 2 +- app/views/projects/merge_requests/show/_mr_box.html.haml | 2 +- app/views/projects/notes/_note.html.haml | 4 ++-- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/app/helpers/notes_helper.rb b/app/helpers/notes_helper.rb index ab44fa6ee4..271b53aa2b 100644 --- a/app/helpers/notes_helper.rb +++ b/app/helpers/notes_helper.rb @@ -9,6 +9,10 @@ module NotesHelper hidden_field_tag(:target_id, note.noteable.id) end + def note_editable?(note) + note.editable? && can?(current_user, :admin_note, note) + end + def link_to_commit_diff_line_note(note) if note.for_commit_diff_line? link_to( diff --git a/app/views/projects/issues/show.html.haml b/app/views/projects/issues/show.html.haml index 4c92823503..ee1b2a08bc 100644 --- a/app/views/projects/issues/show.html.haml +++ b/app/views/projects/issues/show.html.haml @@ -31,7 +31,7 @@ = gfm escape_once(@issue.title) %div - if @issue.description.present? - .description.js-task-list-container + .description{class: can?(current_user, :modify_issue, @issue) ? 'js-task-list-container' : ''} .wiki = preserve do = markdown(@issue.description) diff --git a/app/views/projects/merge_requests/show/_mr_box.html.haml b/app/views/projects/merge_requests/show/_mr_box.html.haml index d5aab122b7..b3470ba37d 100644 --- a/app/views/projects/merge_requests/show/_mr_box.html.haml +++ b/app/views/projects/merge_requests/show/_mr_box.html.haml @@ -3,7 +3,7 @@ %div - if @merge_request.description.present? - .description.js-task-list-container + .description{class: can?(current_user, :modify_merge_request, @merge_request) ? 'js-task-list-container' : ''} .wiki = preserve do = markdown(@merge_request.description) diff --git a/app/views/projects/notes/_note.html.haml b/app/views/projects/notes/_note.html.haml index d582b247a4..4d26b52df0 100644 --- a/app/views/projects/notes/_note.html.haml +++ b/app/views/projects/notes/_note.html.haml @@ -13,7 +13,7 @@ = icon('link') Link here   - - if can?(current_user, :admin_note, note) && note.editable? + - if note_editable?(note) = link_to '#', title: 'Edit comment', class: 'js-note-edit' do = icon('pencil-square-o') Edit @@ -50,7 +50,7 @@ \-1 - .note-body.js-task-list-container + .note-body{class: note_editable?(note) ? 'js-task-list-container' : ''} .note-text = preserve do = markdown(note.note, {no_header_anchors: true}) From 421f3f1551fee86cac163b06404c407b5ac34604 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 29 Apr 2015 20:36:12 -0400 Subject: [PATCH 034/158] Change padding for task lists in notes --- app/assets/stylesheets/pages/notes.scss | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/app/assets/stylesheets/pages/notes.scss b/app/assets/stylesheets/pages/notes.scss index 97b19deb3e..589a43c426 100644 --- a/app/assets/stylesheets/pages/notes.scss +++ b/app/assets/stylesheets/pages/notes.scss @@ -62,6 +62,16 @@ ul.notes { word-wrap: break-word; @include md-typography; + // Reduce left padding of first ul element + ul.task-list:first-child { + padding-left: 10px; + + // sub-lists should be padded normally + ul { + padding-left: 20px; + } + } + hr { margin: 10px 0; } From c915c7b0010d5f9a76092622eff5f17afa072c61 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 17:37:37 -0400 Subject: [PATCH 035/158] Add TaskList to Markdown feature spec --- spec/features/markdown_spec.rb | 15 ++++++++++++--- spec/fixtures/markdown.md.erb | 10 ++++++++++ 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/spec/features/markdown_spec.rb b/spec/features/markdown_spec.rb index 3528200e12..296055e89f 100644 --- a/spec/features/markdown_spec.rb +++ b/spec/features/markdown_spec.rb @@ -24,6 +24,7 @@ require 'erb' # -> Rinku (http, https, ftp) # -> Other schemes # -> References +# -> TaskList # -> `html_safe` # -> Template # @@ -266,6 +267,15 @@ describe 'GitLab Markdown' do expect(body).to have_selector('a.gfm.gfm-label', count: 3) end end + + describe 'Task Lists' do + it 'generates task lists' do + body = get_section('task-lists') + expect(body).to have_selector('ul.task-list', count: 2) + expect(body).to have_selector('li.task-list-item', count: 7) + expect(body).to have_selector('input[checked]', count: 3) + end + end end end @@ -276,9 +286,8 @@ end # once. Unfortunately RSpec will not let you access `let`s in a `before(:all)` # block, so we fake it by encapsulating all the shared setup in this class. # -# The class contains the raw Markup used in the test, dynamically substituting -# real objects, created from factories and setup on-demand, when referenced in -# the Markdown. +# The class renders `spec/fixtures/markdown.md.erb` using ERB, allowing for +# reference to the factory-created objects. class MarkdownFeature include FactoryGirl::Syntax::Methods diff --git a/spec/fixtures/markdown.md.erb b/spec/fixtures/markdown.md.erb index 0c14075855..09ee83ba77 100644 --- a/spec/fixtures/markdown.md.erb +++ b/spec/fixtures/markdown.md.erb @@ -176,3 +176,13 @@ References should be parseable even inside _!<%= merge_request.iid %>_ emphasis. - Label by name in quotes: ~"<%= label.name %>" - Ignored in code: `~<%= simple_label.name %>` - Ignored in links: [Link to ~<%= simple_label.id %>](#label-link) + +### Task Lists + +- [ ] Incomplete task 1 +- [x] Complete task 1 +- [ ] Incomplete task 2 + - [ ] Incomplete sub-task 1 + - [ ] Incomplete sub-task 2 + - [x] Complete sub-task 1 +- [X] Complete task 2 From f3d63eeb181db1c089bf9e4f5c0e296f6ca6230b Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 17:46:21 -0400 Subject: [PATCH 036/158] Remove feature tests for old task lists --- features/project/issues/issues.feature | 41 +++------------------ features/project/merge_requests.feature | 32 ----------------- features/steps/project/issues/issues.rb | 8 ----- features/steps/project/merge_requests.rb | 4 --- features/steps/shared/markdown.rb | 45 ------------------------ features/steps/shared/note.rb | 14 -------- features/steps/shared/paths.rb | 20 ----------- 7 files changed, 4 insertions(+), 160 deletions(-) diff --git a/features/project/issues/issues.feature b/features/project/issues/issues.feature index eb813884d1..bf84e2f8e8 100644 --- a/features/project/issues/issues.feature +++ b/features/project/issues/issues.feature @@ -134,48 +134,15 @@ Feature: Project Issues And I should see "Release 0.4" in issues And I should not see "Tweet control" in issues - Scenario: Issue description should render task checkboxes - Given project "Shop" has "Tasks-open" open issue with task markdown - When I visit issue page "Tasks-open" - Then I should see task checkboxes in the description - - @javascript - Scenario: Issue notes should not render task checkboxes - Given project "Shop" has "Tasks-open" open issue with task markdown - When I visit issue page "Tasks-open" - And I leave a comment with task markdown - Then I should not see task checkboxes in the comment - @javascript Scenario: Issue notes should be editable with +1 - Given project "Shop" has "Tasks-open" open issue with task markdown - When I visit issue page "Tasks-open" + Given project "Shop" have "Release 0.4" open issue + When I visit issue page "Release 0.4" And I leave a comment with a header containing "Comment with a header" Then The comment with the header should not have an ID And I edit the last comment with a +1 Then I should see +1 in the description - # Task status in issues list - - Scenario: Issues list should display task status - Given project "Shop" has "Tasks-open" open issue with task markdown - When I visit project "Shop" issues page - Then I should see the task status for the Taskable - - # Toggling task items - - @javascript - Scenario: Task checkboxes should be enabled for an open issue - Given project "Shop" has "Tasks-open" open issue with task markdown - When I visit issue page "Tasks-open" - Then Task checkboxes should be enabled - - @javascript - Scenario: Task checkboxes should be disabled for a closed issue - Given project "Shop" has "Tasks-closed" closed issue with task markdown - When I visit issue page "Tasks-closed" - Then Task checkboxes should be disabled - # Issue description preview @javascript @@ -212,8 +179,8 @@ Feature: Project Issues @javascript Scenario: I can unsubscribe from issue - Given project "Shop" has "Tasks-open" open issue with task markdown - When I visit issue page "Tasks-open" + Given project "Shop" have "Release 0.4" open issue + When I visit issue page "Release 0.4" Then I should see that I am subscribed When I click button "Unsubscribe" Then I should see that I am unsubscribed diff --git a/features/project/merge_requests.feature b/features/project/merge_requests.feature index cbb5c8eb39..60caf783fe 100644 --- a/features/project/merge_requests.feature +++ b/features/project/merge_requests.feature @@ -96,16 +96,6 @@ Feature: Project Merge Requests And I leave a comment with a header containing "Comment with a header" Then The comment with the header should not have an ID - Scenario: Merge request description should render task checkboxes - Given project "Shop" has "MR-task-open" open MR with task markdown - When I visit merge request page "MR-task-open" - Then I should see task checkboxes in the description - - Scenario: Merge request notes should not render task checkboxes - Given project "Shop" has "MR-task-open" open MR with task markdown - When I visit merge request page "MR-task-open" - Then I should not see task checkboxes in the comment - # Toggling inline comments @javascript @@ -173,28 +163,6 @@ Feature: Project Merge Requests And I click on the Changes tab via Javascript Then I should see the proper Inline and Side-by-side links - # Task status in issues list - - Scenario: Merge requests list should display task status - Given project "Shop" has "MR-task-open" open MR with task markdown - When I visit project "Shop" merge requests page - Then I should see the task status for the Taskable - - # Toggling task items - - @javascript - Scenario: Task checkboxes should be enabled for an open merge request - Given project "Shop" has "MR-task-open" open MR with task markdown - When I visit merge request page "MR-task-open" - Then Task checkboxes should be enabled - - @javascript - Scenario: Task checkboxes should be disabled for a closed merge request - Given project "Shop" has "MR-task-open" open MR with task markdown - And I visit merge request page "MR-task-open" - And I click link "Close" - Then Task checkboxes should be disabled - # Description preview @javascript diff --git a/features/steps/project/issues/issues.rb b/features/steps/project/issues/issues.rb index b8e282b202..504f0cff72 100644 --- a/features/steps/project/issues/issues.rb +++ b/features/steps/project/issues/issues.rb @@ -179,14 +179,6 @@ class Spinach::Features::ProjectIssues < Spinach::FeatureSteps author: project.users.first) end - step 'project "Shop" has "Tasks-open" open issue with task markdown' do - create_taskable(:issue, 'Tasks-open') - end - - step 'project "Shop" has "Tasks-closed" closed issue with task markdown' do - create_taskable(:closed_issue, 'Tasks-closed') - end - step 'empty project "Empty Project"' do create :empty_project, name: 'Empty Project', namespace: @user.namespace end diff --git a/features/steps/project/merge_requests.rb b/features/steps/project/merge_requests.rb index 12d4d81159..f67e6e3d8c 100644 --- a/features/steps/project/merge_requests.rb +++ b/features/steps/project/merge_requests.rb @@ -108,10 +108,6 @@ class Spinach::Features::ProjectMergeRequests < Spinach::FeatureSteps author: project.users.first) end - step 'project "Shop" has "MR-task-open" open MR with task markdown' do - create_taskable(:merge_request, 'MR-task-open') - end - step 'I switch to the diff tab' do visit diffs_namespace_project_merge_request_path(project.namespace, project, merge_request) end diff --git a/features/steps/shared/markdown.rb b/features/steps/shared/markdown.rb index a7231c47d1..943640007a 100644 --- a/features/steps/shared/markdown.rb +++ b/features/steps/shared/markdown.rb @@ -10,55 +10,10 @@ module SharedMarkdown find(:xpath, "#{node.path}/..").text.should == text end - def create_taskable(type, title) - desc_text = < Date: Mon, 4 May 2015 23:01:24 -0400 Subject: [PATCH 037/158] Update jasmine-fixture --- vendor/assets/javascripts/jasmine-fixture.js | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) mode change 100644 => 100755 vendor/assets/javascripts/jasmine-fixture.js diff --git a/vendor/assets/javascripts/jasmine-fixture.js b/vendor/assets/javascripts/jasmine-fixture.js old mode 100644 new mode 100755 index 3815731fd2..9980aec6dd --- a/vendor/assets/javascripts/jasmine-fixture.js +++ b/vendor/assets/javascripts/jasmine-fixture.js @@ -1,4 +1,4 @@ -/* jasmine-fixture - 1.2.2 +/* jasmine-fixture - 1.3.1 * Makes injecting HTML snippets into the DOM easy & clean! * https://github.com/searls/jasmine-fixture */ @@ -8,7 +8,7 @@ (function($) { var ewwSideEffects, jasmineFixture, originalAffix, originalJasmineDotFixture, originalJasmineFixture, root, _, _ref; - root = this; + root = (1, eval)('this'); originalJasmineFixture = root.jasmineFixture; originalJasmineDotFixture = (_ref = root.jasmine) != null ? _ref.fixture : void 0; originalAffix = root.affix; @@ -33,7 +33,7 @@ create = function(selectorOptions, attach) { var $top; $top = null; - _(selectorOptions.split(/[ ](?=[^\]]*?(?:\[|$))/)).inject(function($parent, elementSelector) { + _(selectorOptions.split(/[ ](?![^\{]*\})(?=[^\]]*?(?:\[|$))/)).inject(function($parent, elementSelector) { var $el; if (elementSelector === ">") { return $parent; From 968c281f403d59909ac6fcd54659ccc036c1775f Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 23:03:35 -0400 Subject: [PATCH 038/158] Make notes JS require its dependencies --- app/assets/javascripts/notes.js.coffee | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/app/assets/javascripts/notes.js.coffee b/app/assets/javascripts/notes.js.coffee index 416218d2e0..72e7492abc 100644 --- a/app/assets/javascripts/notes.js.coffee +++ b/app/assets/javascripts/notes.js.coffee @@ -1,3 +1,11 @@ +#= require autosave +#= require bootstrap +#= require dropzone +#= require dropzone_input +#= require gfm_auto_complete +#= require jquery.atwho +#= require task_list + class @Notes @interval: null @@ -487,12 +495,12 @@ class @Notes form.find('.js-note-target-reopen').text('Reopen') form.find('.js-note-target-close').text('Close') - enableTaskList: -> - $('.note .js-task-list-container').taskList('enable') - initTaskList: -> @enableTaskList() $(document).on 'tasklist:changed', '.note .js-task-list-container', @updateTaskList + enableTaskList: -> + $('.note .js-task-list-container').taskList('enable') + updateTaskList: -> $('form', this).submit() From c96c7cbfb15f3fae50b21862fdc5a863929ef0cd Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 12:57:12 -0400 Subject: [PATCH 039/158] Make issue JS require its dependencies --- app/assets/javascripts/issue.js.coffee | 2 ++ 1 file changed, 2 insertions(+) diff --git a/app/assets/javascripts/issue.js.coffee b/app/assets/javascripts/issue.js.coffee index 63d38594d2..ab2435e28b 100644 --- a/app/assets/javascripts/issue.js.coffee +++ b/app/assets/javascripts/issue.js.coffee @@ -1,3 +1,5 @@ +#= require jquery.waitforimages + class @Issue constructor: -> $('.edit-issue.inline-update input[type="submit"]').hide() From 51c3393974c0702f56f22122006606d3e03b668d Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 23:06:57 -0400 Subject: [PATCH 040/158] Task List feature and JS specs --- spec/features/task_lists_spec.rb | 151 ++++++++++++++++++ spec/javascripts/issue_spec.js.coffee | 36 +++++ spec/javascripts/merge_request_spec.js.coffee | 36 +++++ spec/javascripts/notes_spec.js.coffee | 30 ++++ 4 files changed, 253 insertions(+) create mode 100644 spec/features/task_lists_spec.rb create mode 100644 spec/javascripts/issue_spec.js.coffee create mode 100644 spec/javascripts/merge_request_spec.js.coffee create mode 100644 spec/javascripts/notes_spec.js.coffee diff --git a/spec/features/task_lists_spec.rb b/spec/features/task_lists_spec.rb new file mode 100644 index 0000000000..651792601e --- /dev/null +++ b/spec/features/task_lists_spec.rb @@ -0,0 +1,151 @@ +require 'spec_helper' + +feature 'Task Lists' do + include Warden::Test::Helpers + + let(:project) { create(:project) } + let(:user) { create(:user) } + let(:user2) { create(:user) } + + let(:markdown) do + <<-MARKDOWN.strip_heredoc + This is a task list: + + - [ ] Incomplete entry 1 + - [x] Complete entry 1 + - [ ] Incomplete entry 2 + - [x] Complete entry 2 + - [ ] Incomplete entry 3 + - [ ] Incomplete entry 4 + MARKDOWN + end + + before do + Warden.test_mode! + + project.team << [user, :master] + project.team << [user2, :guest] + + login_as(user) + end + + def visit_issue(project, issue) + visit namespace_project_issue_path(project.namespace, project, issue) + end + + describe 'for Issues' do + let!(:issue) { create(:issue, description: markdown, author: user, project: project) } + + it 'renders' do + visit_issue(project, issue) + + expect(page).to have_selector('ul.task-list', count: 1) + expect(page).to have_selector('li.task-list-item', count: 6) + expect(page).to have_selector('ul input[checked]', count: 2) + end + + it 'contains the required selectors' do + visit_issue(project, issue) + + container = '.issue-details .description.js-task-list-container' + + expect(page).to have_selector(container) + expect(page).to have_selector("#{container} .wiki .task-list .task-list-item .task-list-item-checkbox") + expect(page).to have_selector("#{container} .js-task-list-field") + expect(page).to have_selector('form.js-issue-update') + expect(page).to have_selector('a.btn-close') + end + + it 'is only editable by author' do + visit_issue(project, issue) + expect(page).to have_selector('.js-task-list-container') + + logout(:user) + + login_as(user2) + visit current_path + expect(page).not_to have_selector('.js-task-list-container') + end + + it 'provides a summary on Issues#index' do + visit namespace_project_issues_path(project.namespace, project) + expect(page).to have_content("6 tasks (2 done, 4 unfinished)") + end + end + + describe 'for Notes' do + let!(:issue) { create(:issue, author: user, project: project) } + let!(:note) { create(:note, note: markdown, noteable: issue, author: user) } + + it 'renders for note body' do + visit_issue(project, issue) + + expect(page).to have_selector('.note ul.task-list', count: 1) + expect(page).to have_selector('.note li.task-list-item', count: 6) + expect(page).to have_selector('.note ul input[checked]', count: 2) + end + + it 'contains the required selectors' do + visit_issue(project, issue) + + expect(page).to have_selector('.note .js-task-list-container') + expect(page).to have_selector('.note .js-task-list-container .task-list .task-list-item .task-list-item-checkbox') + expect(page).to have_selector('.note .js-task-list-container .js-task-list-field') + end + + it 'is only editable by author' do + visit_issue(project, issue) + expect(page).to have_selector('.js-task-list-container') + + logout(:user) + + login_as(user2) + visit current_path + expect(page).not_to have_selector('.js-task-list-container') + end + end + + describe 'for Merge Requests' do + def visit_merge_request(project, merge) + visit namespace_project_merge_request_path(project.namespace, project, merge) + end + + let!(:merge) { create(:merge_request, :simple, description: markdown, author: user, source_project: project) } + + it 'renders for description' do + visit_merge_request(project, merge) + + expect(page).to have_selector('ul.task-list', count: 1) + expect(page).to have_selector('li.task-list-item', count: 6) + expect(page).to have_selector('ul input[checked]', count: 2) + end + + it 'contains the required selectors' do + visit_merge_request(project, merge) + + container = '.merge-request-details .description.js-task-list-container' + + expect(page).to have_selector(container) + expect(page).to have_selector("#{container} .wiki .task-list .task-list-item .task-list-item-checkbox") + expect(page).to have_selector("#{container} .js-task-list-field") + expect(page).to have_selector('form.js-merge-request-update') + expect(page).to have_selector('a.btn-close') + end + + it 'is only editable by author' do + visit_merge_request(project, merge) + expect(page).to have_selector('.js-task-list-container') + + logout(:user) + + login_as(user2) + visit current_path + expect(page).not_to have_selector('.js-task-list-container') + end + + it 'provides a summary on MergeRequests#index' do + visit namespace_project_merge_requests_path(project.namespace, project) + expect(page).to have_content("6 tasks (2 done, 4 unfinished)") + end + end +end diff --git a/spec/javascripts/issue_spec.js.coffee b/spec/javascripts/issue_spec.js.coffee new file mode 100644 index 0000000000..13b25862f5 --- /dev/null +++ b/spec/javascripts/issue_spec.js.coffee @@ -0,0 +1,36 @@ +#= require jquery +#= require jasmine-fixture +#= require issue + +describe 'Issue', -> + describe 'task lists', -> + selectors = { + container: '.issue-details .description.js-task-list-container' + item: '.wiki ul.task-list li.task-list-item input.task-list-item-checkbox[type=checkbox] {Task List Item}' + textarea: '.wiki textarea.js-task-list-field{- [ ] Task List Item}' + form: 'form.js-issue-update[action="/foo"]' + close: 'a.btn-close' + } + + beforeEach -> + $container = affix(selectors.container) + + # # These two elements are siblings inside the container + $container.find('.js-task-list-container').append(affix(selectors.item)) + $container.find('.js-task-list-container').append(affix(selectors.textarea)) + + # Task lists don't get initialized unless this button exists. Not ideal. + $container.append(affix(selectors.close)) + + # This form is used to get the `update` URL. Not ideal. + $container.append(affix(selectors.form)) + + @issue = new Issue() + + it 'submits an ajax request on tasklist:changed', -> + spyOn($, 'ajax').and.callFake (req) -> + expect(req.type).toBe('PATCH') + expect(req.url).toBe('/foo') + expect(req.data.issue.description).not.toBe(null) + + $('.js-task-list-field').trigger('tasklist:changed') diff --git a/spec/javascripts/merge_request_spec.js.coffee b/spec/javascripts/merge_request_spec.js.coffee new file mode 100644 index 0000000000..3ebc4a4eed --- /dev/null +++ b/spec/javascripts/merge_request_spec.js.coffee @@ -0,0 +1,36 @@ +#= require jquery +#= require jasmine-fixture +#= require merge_request + +describe 'MergeRequest', -> + describe 'task lists', -> + selectors = { + container: '.merge-request-details .description.js-task-list-container' + item: '.wiki ul.task-list li.task-list-item input.task-list-item-checkbox[type=checkbox] {Task List Item}' + textarea: '.wiki textarea.js-task-list-field{- [ ] Task List Item}' + form: 'form.js-merge-request-update[action="/foo"]' + close: 'a.btn-close' + } + + beforeEach -> + $container = affix(selectors.container) + + # # These two elements are siblings inside the container + $container.find('.js-task-list-container').append(affix(selectors.item)) + $container.find('.js-task-list-container').append(affix(selectors.textarea)) + + # Task lists don't get initialized unless this button exists. Not ideal. + $container.append(affix(selectors.close)) + + # This form is used to get the `update` URL. Not ideal. + $container.append(affix(selectors.form)) + + @merge = new MergeRequest({}) + + it 'submits an ajax request on tasklist:changed', -> + spyOn($, 'ajax').and.callFake (req) -> + expect(req.type).toBe('PATCH') + expect(req.url).toBe('/foo') + expect(req.data.merge_request.description).not.toBe(null) + + $('.js-task-list-field').trigger('tasklist:changed') diff --git a/spec/javascripts/notes_spec.js.coffee b/spec/javascripts/notes_spec.js.coffee new file mode 100644 index 0000000000..de2e8e7f6c --- /dev/null +++ b/spec/javascripts/notes_spec.js.coffee @@ -0,0 +1,30 @@ +#= require jquery +#= require jasmine-fixture +#= require notes + +window.gon = {} +window.disableButtonIfEmptyField = -> null + +describe 'Notes', -> + describe 'task lists', -> + selectors = { + container: 'li.note .js-task-list-container' + item: '.note-text ul.task-list li.task-list-item input.task-list-item-checkbox[type=checkbox] {Task List Item}' + textarea: '.note-edit-form form textarea.js-task-list-field{- [ ] Task List Item}' + } + + beforeEach -> + $container = affix(selectors.container) + + # These two elements are siblings inside the container + $container.find('.js-task-list-container').append(affix(selectors.item)) + $container.find('.js-task-list-container').append(affix(selectors.textarea)) + + @notes = new Notes() + + it 'submits the form on tasklist:changed', -> + submitted = false + $('form').on 'submit', (e) -> submitted = true; e.preventDefault() + + $('.js-task-list-field').trigger('tasklist:changed') + expect(submitted).toBe(true) From b519e2312fc52c5065846c67d8e9ca1ce195ddc2 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 15:12:57 -0400 Subject: [PATCH 041/158] Disable Rack::MiniProfiler for Jasmine's specs route --- config/initializers/6_rack_profiler.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/config/initializers/6_rack_profiler.rb b/config/initializers/6_rack_profiler.rb index bdad6202b1..38a5fa98dc 100644 --- a/config/initializers/6_rack_profiler.rb +++ b/config/initializers/6_rack_profiler.rb @@ -5,4 +5,5 @@ if Rails.env.development? Rack::MiniProfilerRails.initialize!(Rails.application) Rack::MiniProfiler.config.position = 'right' Rack::MiniProfiler.config.start_hidden = true + Rack::MiniProfiler.config.skip_paths << '/specs' end From 33836ae02c160657c4320e41916f8162ebfab635 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 15:29:46 -0400 Subject: [PATCH 042/158] Remove unneeded jasmine setup --- spec/javascripts/helpers/.gitkeep | 0 spec/javascripts/support/jasmine.yml | 5 ----- 2 files changed, 5 deletions(-) delete mode 100644 spec/javascripts/helpers/.gitkeep diff --git a/spec/javascripts/helpers/.gitkeep b/spec/javascripts/helpers/.gitkeep deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/spec/javascripts/support/jasmine.yml b/spec/javascripts/support/jasmine.yml index f4b01c9f27..168c961864 100644 --- a/spec/javascripts/support/jasmine.yml +++ b/spec/javascripts/support/jasmine.yml @@ -9,11 +9,6 @@ # defaults to spec/javascripts spec_dir: spec/javascripts -# list of file expressions to include as helpers into spec runner -# relative path from spec_dir -helpers: - - "helpers/**/*.{js.coffee,js,coffee}" - # list of file expressions to include as specs into spec runner # relative path from spec_dir spec_files: From d5ee54df163e0b2e613701ba107d6163be656930 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 15:31:08 -0400 Subject: [PATCH 043/158] More JS dependency requirements --- app/assets/javascripts/application.js.coffee | 1 - app/assets/javascripts/issue.js.coffee | 2 ++ app/assets/javascripts/merge_request.js.coffee | 4 ++++ app/assets/javascripts/notes.js.coffee | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) diff --git a/app/assets/javascripts/application.js.coffee b/app/assets/javascripts/application.js.coffee index 3b8eaa143e..bb9da14701 100644 --- a/app/assets/javascripts/application.js.coffee +++ b/app/assets/javascripts/application.js.coffee @@ -41,7 +41,6 @@ #= require shortcuts_issuable #= require shortcuts_network #= require cal-heatmap -#= require task_list #= require_tree . window.slugify = (text) -> diff --git a/app/assets/javascripts/issue.js.coffee b/app/assets/javascripts/issue.js.coffee index ab2435e28b..86ad3d03ba 100644 --- a/app/assets/javascripts/issue.js.coffee +++ b/app/assets/javascripts/issue.js.coffee @@ -1,4 +1,6 @@ +#= require jquery #= require jquery.waitforimages +#= require task_list class @Issue constructor: -> diff --git a/app/assets/javascripts/merge_request.js.coffee b/app/assets/javascripts/merge_request.js.coffee index c8affba5b2..7c1e2b822d 100644 --- a/app/assets/javascripts/merge_request.js.coffee +++ b/app/assets/javascripts/merge_request.js.coffee @@ -1,3 +1,7 @@ +#= require jquery +#= require bootstrap +#= require task_list + class @MergeRequest constructor: (@opts) -> @initContextWidget() diff --git a/app/assets/javascripts/notes.js.coffee b/app/assets/javascripts/notes.js.coffee index 72e7492abc..c25b1ddb06 100644 --- a/app/assets/javascripts/notes.js.coffee +++ b/app/assets/javascripts/notes.js.coffee @@ -1,3 +1,4 @@ +#= require jquery #= require autosave #= require bootstrap #= require dropzone From 0a99e6e2facee4126509c41b987aa88f1520626e Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 15:31:24 -0400 Subject: [PATCH 044/158] Remove 'jasmine' gem jasmine-rails still requires it as a dependency, but for some reason running `rake jasmine:ci` produced test failures but `rake spec:javascript` didn't. --- Gemfile | 1 - Gemfile.lock | 6 ------ lib/tasks/jasmine.rake | 12 ++++++++++++ 3 files changed, 12 insertions(+), 7 deletions(-) create mode 100644 lib/tasks/jasmine.rake diff --git a/Gemfile b/Gemfile index 05e26f5b72..f950c5be15 100644 --- a/Gemfile +++ b/Gemfile @@ -248,7 +248,6 @@ group :development, :test do # PhantomJS driver for Capybara gem 'poltergeist', '~> 1.5.1' - gem 'jasmine', '~> 2.2.0' gem 'jasmine-rails' gem "spring", '~> 1.3.1' diff --git a/Gemfile.lock b/Gemfile.lock index 3334917797..6f58c4f4fd 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -290,11 +290,6 @@ GEM i18n (0.7.0) ice_cube (0.11.1) ice_nine (0.10.0) - jasmine (2.2.0) - jasmine-core (~> 2.2) - phantomjs - rack (>= 1.2.1) - rake jasmine-core (2.2.0) jasmine-rails (0.10.8) jasmine-core (>= 1.3, < 3.0) @@ -726,7 +721,6 @@ DEPENDENCIES hipchat (~> 1.5.0) html-pipeline (~> 1.11.0) httparty - jasmine (~> 2.2.0) jasmine-rails jquery-atwho-rails (~> 1.0.0) jquery-rails diff --git a/lib/tasks/jasmine.rake b/lib/tasks/jasmine.rake new file mode 100644 index 0000000000..9e2cceffa1 --- /dev/null +++ b/lib/tasks/jasmine.rake @@ -0,0 +1,12 @@ +# Since we no longer explicitly require the 'jasmine' gem, we lost the +# `jasmine:ci` task used by GitLab CI jobs. +# +# This provides a simple alias to run the `spec:javascript` task from the +# 'jasmine-rails' gem. +task jasmine: ['jasmine:ci'] + +namespace :jasmine do + task :ci do + Rake::Task['spec:javascript'].invoke + end +end From fdff815a4cddd9e956d1bf4cc8f01eeb3384b9ee Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 17:21:19 -0400 Subject: [PATCH 045/158] Add Markdown feature specs for markup inside links It was already working, just want to make sure we don't break it in the future. --- spec/features/markdown_spec.rb | 13 +++++++++++++ spec/fixtures/markdown.md.erb | 8 ++++++++ 2 files changed, 21 insertions(+) diff --git a/spec/features/markdown_spec.rb b/spec/features/markdown_spec.rb index 3528200e12..3fe593c47b 100644 --- a/spec/features/markdown_spec.rb +++ b/spec/features/markdown_spec.rb @@ -161,6 +161,19 @@ describe 'GitLab Markdown' do end end + describe 'Edge Cases' do + it 'allows markup inside link elements' do + expect(@doc.at_css('a[href="#link-emphasis"]').to_html). + to eq %{text} + + expect(@doc.at_css('a[href="#link-strong"]').to_html). + to eq %{text} + + expect(@doc.at_css('a[href="#link-code"]').to_html). + to eq %{text} + end + end + describe 'EmojiFilter' do it 'parses Emoji' do expect(@doc).to have_selector('img.emoji', count: 10) diff --git a/spec/fixtures/markdown.md.erb b/spec/fixtures/markdown.md.erb index 0c14075855..6b79aad8f8 100644 --- a/spec/fixtures/markdown.md.erb +++ b/spec/fixtures/markdown.md.erb @@ -94,6 +94,14 @@ The problem with SanitizationFilter is that it can be too aggressive. | `1 < 3 & 5` | 1 < 3 & 5 | 1 < 3 & 5 | | `` | <foo> | | +### Edge Cases + +Markdown should be usable inside a link. Let's try! + +- [_text_](#link-emphasis) +- [**text**](#link-strong) +- [`text`](#link-code) + ### EmojiFilter Because life would be :zzz: without Emoji, right? :rocket: From 1108915f758db254187d47d6a2e626ed6abcbbf3 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 17:27:10 -0400 Subject: [PATCH 046/158] Change wording for task list summaries --- app/models/concerns/taskable.rb | 4 ++-- spec/features/task_lists_spec.rb | 4 ++-- spec/support/taskable_shared_examples.rb | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/app/models/concerns/taskable.rb b/app/models/concerns/taskable.rb index 6e04578a7d..33b4814d7e 100644 --- a/app/models/concerns/taskable.rb +++ b/app/models/concerns/taskable.rb @@ -26,11 +26,11 @@ module Taskable end # Return a string that describes the current state of this Taskable's task - # list items, e.g. "20 tasks (12 done, 8 unfinished)" + # list items, e.g. "20 tasks (12 completed, 8 remaining)" def task_status return '' if description.blank? sum = tasks.summary - "#{sum.item_count} tasks (#{sum.complete_count} done, #{sum.incomplete_count} unfinished)" + "#{sum.item_count} tasks (#{sum.complete_count} completed, #{sum.incomplete_count} remaining)" end end diff --git a/spec/features/task_lists_spec.rb b/spec/features/task_lists_spec.rb index 651792601e..2099fc40cc 100644 --- a/spec/features/task_lists_spec.rb +++ b/spec/features/task_lists_spec.rb @@ -69,7 +69,7 @@ feature 'Task Lists' do it 'provides a summary on Issues#index' do visit namespace_project_issues_path(project.namespace, project) - expect(page).to have_content("6 tasks (2 done, 4 unfinished)") + expect(page).to have_content("6 tasks (2 completed, 4 remaining)") end end @@ -145,7 +145,7 @@ feature 'Task Lists' do it 'provides a summary on MergeRequests#index' do visit namespace_project_merge_requests_path(project.namespace, project) - expect(page).to have_content("6 tasks (2 done, 4 unfinished)") + expect(page).to have_content("6 tasks (2 completed, 4 remaining)") end end end diff --git a/spec/support/taskable_shared_examples.rb b/spec/support/taskable_shared_examples.rb index 77e0ed5f57..927c72c740 100644 --- a/spec/support/taskable_shared_examples.rb +++ b/spec/support/taskable_shared_examples.rb @@ -15,8 +15,8 @@ shared_examples 'a Taskable' do it 'returns the correct task status' do expect(subject.task_status).to match('5 tasks') - expect(subject.task_status).to match('2 done') - expect(subject.task_status).to match('3 unfinished') + expect(subject.task_status).to match('2 completed') + expect(subject.task_status).to match('3 remaining') end describe '#tasks?' do From 0cc75363415a4f60100a230d78945465f9cefdcc Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 18:38:43 -0400 Subject: [PATCH 047/158] Fix header link anchors --- app/assets/stylesheets/generic/typography.scss | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/app/assets/stylesheets/generic/typography.scss b/app/assets/stylesheets/generic/typography.scss index 36a9a54074..e559089794 100644 --- a/app/assets/stylesheets/generic/typography.scss +++ b/app/assets/stylesheets/generic/typography.scss @@ -37,7 +37,9 @@ pre { position: relative; a.anchor { - display: none; + // Setting `display: none` would prevent the anchor being scrolled to, so + // instead we set the height to 0 and it gets updated on hover. + height: 0; } &:hover > a.anchor { From cb079c8208b386425d2d94377a20804a53df6489 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 20:43:12 -0400 Subject: [PATCH 048/158] Fix Markdown feature spec --- spec/features/markdown_spec.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/features/markdown_spec.rb b/spec/features/markdown_spec.rb index 3fe593c47b..57d0123650 100644 --- a/spec/features/markdown_spec.rb +++ b/spec/features/markdown_spec.rb @@ -62,7 +62,7 @@ describe 'GitLab Markdown' do # Given a header ID, goes to that element's parent (the header), then to its # second sibling (the body). def get_section(id) - @doc.at_css("##{id}").parent.next.next + @doc.at_css("##{id}").parent.next_element end # it 'writes to a file' do @@ -189,7 +189,7 @@ describe 'GitLab Markdown' do end describe 'AutolinkFilter' do - let(:list) { get_section('autolinkfilter').parent.search('ul') } + let(:list) { get_section('autolinkfilter').next_element } def item(index) list.at_css("li:nth-child(#{index})") From cca28c79201aea3703c46289e22d5ff75faf10d6 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 18:55:57 -0400 Subject: [PATCH 049/158] Update Markdown help docs for latest changes --- doc/markdown/markdown.md | 93 ++++++++++--------- .../markdown/table_of_contents_filter.rb | 2 +- 2 files changed, 50 insertions(+), 45 deletions(-) diff --git a/doc/markdown/markdown.md b/doc/markdown/markdown.md index 8ec5a20035..e95ddbb757 100644 --- a/doc/markdown/markdown.md +++ b/doc/markdown/markdown.md @@ -172,7 +172,7 @@ GFM will turn that reference into a link so you can navigate between them easily GFM will recognize the following: | input | references | -|-----------------------:|:---------------------------| +|:-----------------------|:---------------------------| | `@user_name` | specific user | | `@group_name` | specific group | | `@all` | entire team | @@ -189,7 +189,7 @@ GFM will recognize the following: GFM also recognizes certain cross-project references: | input | references | -|----------------------------------------:|:------------------------| +|:----------------------------------------|:------------------------| | `namespace/project#123` | issue | | `namespace/project!123` | merge request | | `namespace/project$123` | snippet | @@ -198,15 +198,23 @@ GFM also recognizes certain cross-project references: ## Task Lists -You can add task lists to merge request and issue descriptions to keep track of to-do items. To create a task, add an unordered list to the description in an issue or merge request, formatted like so: +You can add task lists to issues, merge requests and comments. To create a task list, add a specially-formatted Markdown list, like so: ```no-highlight -* [x] Completed task -* [ ] Unfinished task - * [x] Nested task +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 ``` -Task lists can only be created in descriptions, not in titles or comments. Task item state can be managed by editing the description's Markdown or by clicking the rendered checkboxes. +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 + +Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes. # Standard Markdown @@ -246,51 +254,38 @@ Alt-H2 ### Header IDs and links -All markdown rendered headers automatically get IDs, except for comments. +All Markdown-rendered headers automatically get IDs, except in comments. On hover a link to those IDs becomes visible to make it easier to copy the link to the header to give it to someone else. The IDs are generated from the content of the header according to the following rules: -1. remove the heading hashes `#` and process the rest of the line as it would be processed if it were not a header -2. from the result, remove all HTML tags, but keep their inner content -3. convert all characters to lowercase -4. convert all characters except `[a-z0-9_-]` into hyphens `-` -5. transform multiple adjacent hyphens into a single hyphen -6. remove trailing and heading hyphens +1. All text is converted to lowercase +1. All non-word text (e.g., punctuation, HTML) is removed +1. All spaces are converted to hyphens +1. Two or more hyphens in a row are converted to one +1. If a header with the same ID has already been generated, a unique + incrementing number is appended. For example: ``` -###### ..Ab_c-d. e [anchor](URL) ![alt text](URL).. +# This header has spaces in it +## This header has a :thumbsup: in it +# This header has Unicode in it: 한글 +## This header has spaces in it +### This header has spaces in it ``` -which renders as: +Would generate the following link IDs: -###### ..Ab_c-d. e [anchor](URL) ![alt text](URL).. +1. `this-header-has-spaces-in-it` +1. `this-header-has-a-in-it` +1. `this-header-has-unicode-in-it-한글` +1. `this-header-has-spaces-in-it-1` +1. `this-header-has-spaces-in-it-2` -will first be converted by step 1) into a string like: - -``` -..Ab_c-d. e <a href="URL">anchor</a> <img src="URL" alt="alt text"/>.. -``` - -After removing the tags in step 2) we get: - -``` -..Ab_c-d. e anchor .. -``` - -And applying all the other steps gives the id: - -``` -ab_c-d-e-anchor -``` - -Note in particular how: - -- for markdown anchors `[text](URL)`, only the `text` is used -- markdown images `![alt](URL)` are completely ignored +Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. ## Emphasis @@ -322,8 +317,6 @@ Strikethrough uses two tildes. ~~Scratch this.~~ 1. Ordered sub-list 4. And another item. - Some text that should be aligned with the above item. - * Unordered list can use asterisks - Or minuses + Or pluses @@ -336,8 +329,6 @@ Strikethrough uses two tildes. ~~Scratch this.~~ 1. Ordered sub-list 4. And another item. - Some text that should be aligned with the above item. - * Unordered list can use asterisks - Or minuses + Or pluses @@ -432,7 +423,7 @@ Quote break. You can also use raw HTML in your Markdown, and it'll mostly work pretty well. -See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows the `class`, `id`, and `style` attributes. +See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span` elements, as well as the `class`, and `id` attributes on all elements. ```no-highlight
    @@ -536,6 +527,20 @@ Code above produces next output: The row of dashes between the table header and body must have at least three dashes in each column. +By including colons in the header row, you can align the text within that column: + +``` +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :----------- | :------: | ------------: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +``` + +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :----------- | :------: | ------------: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | + ## References - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). diff --git a/lib/gitlab/markdown/table_of_contents_filter.rb b/lib/gitlab/markdown/table_of_contents_filter.rb index c97612dafb..38887c9778 100644 --- a/lib/gitlab/markdown/table_of_contents_filter.rb +++ b/lib/gitlab/markdown/table_of_contents_filter.rb @@ -31,7 +31,7 @@ module Gitlab id = text.downcase id.gsub!(PUNCTUATION_REGEXP, '') # remove punctuation id.gsub!(' ', '-') # replace spaces with dash - id.squeeze!(' -') # replace multiple spaces or dashes with one + id.squeeze!('-') # replace multiple dashes with one uniq = (headers[id] > 0) ? "-#{headers[id]}" : '' headers[id] += 1 From da8d6feb2c9a273a32b072fbed30f958435c2eeb Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 21:06:18 -0400 Subject: [PATCH 050/158] Add CHANGELOG entry for latest task lists [ci skip] --- CHANGELOG | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG b/CHANGELOG index c7fb49dc18..f471d46de1 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -32,7 +32,7 @@ v 7.11.0 (unreleased) - Show incompatible projects in Google Code import status (Stan Hu) - Fix bug where commit data would not appear in some subdirectories (Stan Hu) - Unescape branch names in compare commit (Stan Hu) - - + - Task lists are now usable in comments, and will show up in Markdown previews. - Fix bug where Slack service channel was not saved in admin template settings. (Stan Hu) - Move snippets UI to fluid layout - Improve UI for sidebar. Increase separation between navigation and content From 2576971f39de0884bd3e6464c3f0611c6590f69e Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 09:10:35 +0200 Subject: [PATCH 051/158] Forgot to link the third directory, thanks Ben. --- docker/single/marathon.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/single/marathon.json b/docker/single/marathon.json index 2c15315e7a..b22b9f6274 100644 --- a/docker/single/marathon.json +++ b/docker/single/marathon.json @@ -22,8 +22,8 @@ "mode": "RW" }, { - "containerPath": "/var/opt/gitlab", - "hostPath": "/var/opt/gitlab", + "containerPath": "/etc/gitlab", + "hostPath": "/etc/gitlab", "mode": "RW" } ] From 661142ca4a2dee6a98fde45a1f6f0ceb8c804e13 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 09:21:32 +0200 Subject: [PATCH 052/158] Add headers and reduce duplication. --- docker/README.md | 72 +++++++++++++++++++++--------------------------- 1 file changed, 32 insertions(+), 40 deletions(-) diff --git a/docker/README.md b/docker/README.md index 537855624a..748640d92d 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,12 +1,24 @@ -What is GitLab? -=============== +# GitLab Docker images -GitLab offers git repository management, code reviews, issue tracking, activity feeds, wikis. It has LDAP/AD integration, handles 25,000 users on a single server but can also run on a highly available active/active cluster. A subscription gives you access to our support team and to GitLab Enterprise Edition that contains extra features aimed at larger organizations. +## What is GitLab? +GitLab offers git repository management, code reviews, issue tracking, activity feeds, wikis. It has LDAP/AD integration, handles 25,000 users on a single server but can also run on a highly available active/active cluster. Learn more on [https://about.gitlab.com](https://about.gitlab.com) -Single and app and data images -=================== +## How to build and use the docker images + +This guide will let you know how to build docker images yourself. +Run the below commands from the GitLab repo root directory. +People using boot2docker on OSX should run all the commands without sudo. + +After starting a container you can then go to [http://localhost:8080/](http://localhost:8080/) or [http://192.168.59.103:8080/](http://192.168.59.103:8080/) if you use boot2docker. +It might take a while before the docker container is responding to queries. + +You can login to the web interface with username `root` and password `5iveL!fe`. + +Next time, you can just use docker start and stop to run the container. + +## Choosing between the single and the app and data images Normally docker uses a single image for one applications. But GitLab stores repositories and uploads in the filesystem. @@ -15,11 +27,7 @@ That is why we recommend using separate app and data images. We'll first describe how to use a single image. After that we'll describe how to use the app and data images. -Single image -================= - -Run the below commands from the GitLab repo root directory. -People using boot2docker should run the commands without sudo. +## Single image Build the image with: @@ -33,11 +41,6 @@ Run the image with: sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 gitlab-ce ``` -You can then go to [http://localhost:8080/](http://localhost:8080/) or [http://192.168.59.103:8080/](http://192.168.59.103:8080/) if you use boot2docker. - -You can login with username `root` and password `5iveL!fe`. -Next time, you can just use `sudo docker start gitlab-ce` and `sudo docker stop gitlab-ce`. - Publish the image with: ```bash @@ -55,18 +58,14 @@ sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 sy Troubleshoot with: ``` +sudo docker logs -f gitlab-ce sudo docker run -ti -e TERM=linux --name gitlab-ce-troubleshoot --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash /usr/local/bin/wrapper ``` -Build and use app and data images -====================== +## Build and use app and data images -At this moment GitLab doesn't have official Docker images. -There are unofficial images at the bottom of this document. Build your own based on the Omnibus packages with the following commands. -For convinience we will use suffix _xy where xy is current version of GitLab. -Run the below commands from the GitLab repo root directory. -People using boot2docker should run the commands without sudo. + ```bash sudo docker build --tag gitlab-data docker/data/ @@ -94,17 +93,11 @@ After creating data container run GitLab container: sudo docker run --detach --name gitlab-app::7.10.1 --publish 8080:80 --publish 2222:22 --volumes-from gitlab-data gitlab-app:7.10.1 ``` -It might take a while before the docker container is responding to queries. You can follow the configuration process with `sudo docker logs -f gitlab_app_xy`. +You can follow the configuration process with `sudo docker logs -f gitlab-app`. -You can then go to [http://localhost:8080/](http://localhost:8080/) or [http://192.168.59.103:8080/](http://192.168.59.103:8080/) if you use boot2docker. +### Configure GitLab -You can login with username `root` and password `5iveL!fe`. -Next time, you can just use `sudo docker start gitlab_app` and `sudo docker stop gitlab_app`. - -Configure GitLab -======================== - -This container uses the official Omnibus GitLab distribution, so all configuration is done in the unique configuration file `/etc/gitlab/gitlab.rb`. +These container uses the official Omnibus GitLab distribution, so all configuration is done in the unique configuration file `/etc/gitlab/gitlab.rb`. To access GitLab configuration, you can start an interactive command line in a new container using the shared data volume container, you will be able to browse the 3 directories and use your favorite text editor: @@ -117,12 +110,11 @@ vi /etc/gitlab/gitlab.rb You can find all available options in [Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#configuration). -Upgrade GitLab with app and data images -======================== +### Upgrade GitLab with app and data images To updgrade GitLab to new versions, stop running container, create new docker image and container from that image. -It Assumes that you're upgrading from 7.8 to 7.9 and you're in the updated GitLab repo root directory: +It Assumes that you're upgrading from 7.8.1 to 7.10.1 and you're in the updated GitLab repo root directory: ```bash sudo docker stop gitlab-app @@ -137,8 +129,8 @@ On the first run GitLab will reconfigure and update itself. If everything runs O sudo docker rmi gitlab-app:7.8.1 ``` -Publish app and data images to Dockerhub -========================= +### Publish app and data images to Dockerhub + Login to Dockerhub with `sudo docker login` and run the following (replace '7.9.2' with the version you're using and 'Sytse Sijbrandij' with your name): ```bash @@ -148,8 +140,8 @@ sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab_data sytse/g sudo docker push sytse/gitlab_data ``` -Use app and data images published to Dockerhub -================================ +### Use app and data images published to Dockerhub + This examples uses the unofficial images made by GitLab CEO Sytse. ```bash @@ -159,6 +151,6 @@ sudo docker run --name gitlab-data sytse/gitlab-data /bin/true sudo docker run --detach --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data sytse/gitlab-app:7.10.1 ``` -Troubleshooting -========================= +## Troubleshooting + Please see the [troubleshooting](troubleshooting.md) file in this directory. From 9edabac522f72970e0b28152a7d1d727440941cc Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 09:33:13 +0200 Subject: [PATCH 053/158] Put the retrieval of published images first. --- docker/README.md | 85 ++++++++++++++++++++++-------------------------- 1 file changed, 39 insertions(+), 46 deletions(-) diff --git a/docker/README.md b/docker/README.md index 748640d92d..a3b3c0e66b 100644 --- a/docker/README.md +++ b/docker/README.md @@ -7,17 +7,17 @@ Learn more on [https://about.gitlab.com](https://about.gitlab.com) ## How to build and use the docker images -This guide will let you know how to build docker images yourself. -Run the below commands from the GitLab repo root directory. -People using boot2docker on OSX should run all the commands without sudo. - -After starting a container you can then go to [http://localhost:8080/](http://localhost:8080/) or [http://192.168.59.103:8080/](http://192.168.59.103:8080/) if you use boot2docker. +After starting a container you can go to [http://localhost:8080/](http://localhost:8080/) or [http://192.168.59.103:8080/](http://192.168.59.103:8080/) if you use boot2docker. It might take a while before the docker container is responding to queries. You can login to the web interface with username `root` and password `5iveL!fe`. Next time, you can just use docker start and stop to run the container. +This guide will also let you know how to build docker images yourself. +Please run all the commands from the GitLab repo root directory. +People using boot2docker should run all the commands without sudo. + ## Choosing between the single and the app and data images Normally docker uses a single image for one applications. @@ -29,44 +29,60 @@ After that we'll describe how to use the app and data images. ## Single image -Build the image with: +Get a published image from Dockerhub: -``` -sudo docker build --tag gitlab-ce docker/single/ +```bash +sudo docker pull sytse/gitlab-ce:7.10.1 ``` -Run the image with: +Run the image: -``` +```bash sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 gitlab-ce ``` -Publish the image with: +Build the image: + +```bash +sudo docker build --tag gitlab-ce docker/single/ +``` + +Publish the image to Dockerhub: ```bash sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab-ce sytse/gitlab-ce:7.10.1 sudo docker push sytse/gitlab-ce:7.10.1 ``` -Use the published image with: +Troubleshoot: ```bash -sudo docker pull sytse/gitlab-ce:7.10.1 -sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 -``` - -Troubleshoot with: - -``` sudo docker logs -f gitlab-ce sudo docker run -ti -e TERM=linux --name gitlab-ce-troubleshoot --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash /usr/local/bin/wrapper ``` -## Build and use app and data images +## App and data images + +### Get published images from Dockerhub + +```bash +sudo docker pull sytse/gitlab-data +sudo docker pull sytse/gitlab-app:7.10.1 +``` + +### Run images + +```bash +sudo docker run --name gitlab-data sytse/gitlab-data /bin/true +sudo docker run --detach --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data sytse/gitlab-app:7.10.1 +``` + +You can follow the configuration process with `sudo docker logs -f gitlab-app`. + +### Build images Build your own based on the Omnibus packages with the following commands. - ```bash sudo docker build --tag gitlab-data docker/data/ sudo docker build --tag gitlab-app:7.10.1 docker/app/ @@ -81,19 +97,7 @@ The directories on data container are: - `/var/log/gitlab` for logs - `/etc/gitlab` for configuration -Create the data container with: - -```bash -sudo docker run --name gitlab-data gitlab-data /bin/true -``` - -After creating data container run GitLab container: - -```bash -sudo docker run --detach --name gitlab-app::7.10.1 --publish 8080:80 --publish 2222:22 --volumes-from gitlab-data gitlab-app:7.10.1 -``` - -You can follow the configuration process with `sudo docker logs -f gitlab-app`. +After this run the images. ### Configure GitLab @@ -129,7 +133,7 @@ On the first run GitLab will reconfigure and update itself. If everything runs O sudo docker rmi gitlab-app:7.8.1 ``` -### Publish app and data images to Dockerhub +### Publish images to Dockerhub Login to Dockerhub with `sudo docker login` and run the following (replace '7.9.2' with the version you're using and 'Sytse Sijbrandij' with your name): @@ -140,17 +144,6 @@ sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab_data sytse/g sudo docker push sytse/gitlab_data ``` -### Use app and data images published to Dockerhub - -This examples uses the unofficial images made by GitLab CEO Sytse. - -```bash -sudo docker pull sytse/gitlab-data -sudo docker pull sytse/gitlab-app:7.10.1 -sudo docker run --name gitlab-data sytse/gitlab-data /bin/true -sudo docker run --detach --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data sytse/gitlab-app:7.10.1 -``` - ## Troubleshooting Please see the [troubleshooting](troubleshooting.md) file in this directory. From 66f7cdeb0af70100b0c60163c4685ba671413ddd Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 09:38:54 +0200 Subject: [PATCH 054/158] Cleanup the order of the docker manual a bit more. --- docker/README.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/docker/README.md b/docker/README.md index a3b3c0e66b..5561e56263 100644 --- a/docker/README.md +++ b/docker/README.md @@ -5,15 +5,20 @@ GitLab offers git repository management, code reviews, issue tracking, activity feeds, wikis. It has LDAP/AD integration, handles 25,000 users on a single server but can also run on a highly available active/active cluster. Learn more on [https://about.gitlab.com](https://about.gitlab.com) -## How to build and use the docker images +## After starting a container After starting a container you can go to [http://localhost:8080/](http://localhost:8080/) or [http://192.168.59.103:8080/](http://192.168.59.103:8080/) if you use boot2docker. + It might take a while before the docker container is responding to queries. +You can check the status with `sudo docker logs -f gitlab-ce`. + You can login to the web interface with username `root` and password `5iveL!fe`. Next time, you can just use docker start and stop to run the container. +## How to build the docker images + This guide will also let you know how to build docker images yourself. Please run all the commands from the GitLab repo root directory. People using boot2docker should run all the commands without sudo. @@ -41,6 +46,8 @@ Run the image: sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 gitlab-ce ``` +After this you can login to the web interface as explained in 'After starting a container' + Build the image: ```bash @@ -57,7 +64,6 @@ sudo docker push sytse/gitlab-ce:7.10.1 Troubleshoot: ```bash -sudo docker logs -f gitlab-ce sudo docker run -ti -e TERM=linux --name gitlab-ce-troubleshoot --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash /usr/local/bin/wrapper ``` @@ -70,14 +76,14 @@ sudo docker pull sytse/gitlab-data sudo docker pull sytse/gitlab-app:7.10.1 ``` -### Run images +### Run the images ```bash sudo docker run --name gitlab-data sytse/gitlab-data /bin/true sudo docker run --detach --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data sytse/gitlab-app:7.10.1 ``` -You can follow the configuration process with `sudo docker logs -f gitlab-app`. +Now please refer to the section above 'After starting a container'. ### Build images @@ -88,6 +94,8 @@ sudo docker build --tag gitlab-data docker/data/ sudo docker build --tag gitlab-app:7.10.1 docker/app/ ``` +After this run the images as described in the prepivous section. + We assume using a data volume container, this will simplify migrations and backups. This empty container will exist to persist as volumes the 3 directories used by GitLab, so remember not to delete it. @@ -97,8 +105,6 @@ The directories on data container are: - `/var/log/gitlab` for logs - `/etc/gitlab` for configuration -After this run the images. - ### Configure GitLab These container uses the official Omnibus GitLab distribution, so all configuration is done in the unique configuration file `/etc/gitlab/gitlab.rb`. @@ -127,7 +133,7 @@ sudo docker build --tag gitlab-app:7.10.1 docker/app/ sudo docker run --detach --name gitlab-app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data gitlab-app:7.10.1 ``` -On the first run GitLab will reconfigure and update itself. If everything runs OK don't forget to cleanup image: +On the first run GitLab will reconfigure and update itself. If everything runs OK don't forget to cleanup the app image: ```bash sudo docker rmi gitlab-app:7.8.1 From cb4e16b7a39b25676a93a27b46cfc5659c628971 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 10:24:38 +0200 Subject: [PATCH 055/158] Ignore mounted Docker directories. --- .gitignore | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitignore b/.gitignore index 3e30fb8cf7..efd84a7dcc 100644 --- a/.gitignore +++ b/.gitignore @@ -40,3 +40,6 @@ rails_best_practices_output.html /tags tmp/ vendor/bundle/* +var-log-gitlab/ +var-opt-gitlab/ +etc-gitlab/ From b18dfc8d40269d3770a432ecf28416736f7bf8d7 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 10:24:55 +0200 Subject: [PATCH 056/158] No need to commit when using a dockerfile. --- docker/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docker/README.md b/docker/README.md index 5561e56263..84e4655640 100644 --- a/docker/README.md +++ b/docker/README.md @@ -51,19 +51,19 @@ After this you can login to the web interface as explained in 'After starting a Build the image: ```bash -sudo docker build --tag gitlab-ce docker/single/ +sudo docker build --tag sytse/gitlab-ce:7.10.1 docker/single/ ``` Publish the image to Dockerhub: ```bash -sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab-ce sytse/gitlab-ce:7.10.1 -sudo docker push sytse/gitlab-ce:7.10.1 +sudo docker push sytse/gitlab-ce ``` -Troubleshoot: +Diagnosing commands: ```bash +docker run -i -t -v `pwd`/var-opt-gitlab:/var/opt/gitlab -v `pwd`/var-log-gitlab:/var/log/gitlab -v `pwd`/etc-gitlab:/etc/gitlab sytse/gitlab-ce:7.10.1 sudo docker run -ti -e TERM=linux --name gitlab-ce-troubleshoot --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash /usr/local/bin/wrapper ``` From 5b534a287aa4c277b7f0dbb8b5c7be3ee508175b Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Thu, 7 May 2015 10:42:17 +0200 Subject: [PATCH 057/158] revert bad link in documentation --- doc/security/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/security/README.md b/doc/security/README.md index 1ca45fd0c0..49dfa6eec7 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -4,4 +4,3 @@ - [Rack attack](rack_attack.md) - [Web Hooks and insecure internal web services](webhooks.md) - [Information exclusivity](information_exclusivity.md) -- [Reset your root password] (https://dev.gitlab.org/gitlab/gitlabhq/blob/How_to_Reset_your_root_password/doc/security/reset_root_password.md) \ No newline at end of file From 60887cf94904e10cd26246301d09a49be49ef5fa Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 10:47:24 +0200 Subject: [PATCH 058/158] Remove the volumes from the single image to keep it simple. --- .gitignore | 3 --- docker/README.md | 10 +++++----- docker/single/marathon.json | 19 +------------------ 3 files changed, 6 insertions(+), 26 deletions(-) diff --git a/.gitignore b/.gitignore index efd84a7dcc..3e30fb8cf7 100644 --- a/.gitignore +++ b/.gitignore @@ -40,6 +40,3 @@ rails_best_practices_output.html /tags tmp/ vendor/bundle/* -var-log-gitlab/ -var-opt-gitlab/ -etc-gitlab/ diff --git a/docker/README.md b/docker/README.md index 84e4655640..82eac9cd0d 100644 --- a/docker/README.md +++ b/docker/README.md @@ -11,7 +11,7 @@ After starting a container you can go to [http://localhost:8080/](http://localho It might take a while before the docker container is responding to queries. -You can check the status with `sudo docker logs -f gitlab-ce`. +You can check the status with something like `sudo docker logs -f 7c10172d7705`. You can login to the web interface with username `root` and password `5iveL!fe`. @@ -43,10 +43,10 @@ sudo docker pull sytse/gitlab-ce:7.10.1 Run the image: ```bash -sudo docker run --detach --name gitlab-ce --publish 8080:80 --publish 2222:22 gitlab-ce +sudo docker run --detach --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 ``` -After this you can login to the web interface as explained in 'After starting a container' +After this you can login to the web interface as explained above in 'After starting a container'. Build the image: @@ -63,7 +63,7 @@ sudo docker push sytse/gitlab-ce Diagnosing commands: ```bash -docker run -i -t -v `pwd`/var-opt-gitlab:/var/opt/gitlab -v `pwd`/var-log-gitlab:/var/log/gitlab -v `pwd`/etc-gitlab:/etc/gitlab sytse/gitlab-ce:7.10.1 +sudo docker run -i -t sytse/gitlab-ce:7.10.1 sudo docker run -ti -e TERM=linux --name gitlab-ce-troubleshoot --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash /usr/local/bin/wrapper ``` @@ -83,7 +83,7 @@ sudo docker run --name gitlab-data sytse/gitlab-data /bin/true sudo docker run --detach --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data sytse/gitlab-app:7.10.1 ``` -Now please refer to the section above 'After starting a container'. +After this you can login to the web interface as explained above in 'After starting a container'. ### Build images diff --git a/docker/single/marathon.json b/docker/single/marathon.json index b22b9f6274..d23c2b84e0 100644 --- a/docker/single/marathon.json +++ b/docker/single/marathon.json @@ -9,23 +9,6 @@ "docker": { "network": "HOST", "image": "sytse/gitlab-ce:7.10.1" - }, - "volumes": [ - { - "containerPath": "/var/opt/gitlab", - "hostPath": "/var/opt/gitlab", - "mode": "RW" - }, - { - "containerPath": "/var/log/gitlab", - "hostPath": "/var/log/gitlab", - "mode": "RW" - }, - { - "containerPath": "/etc/gitlab", - "hostPath": "/etc/gitlab", - "mode": "RW" - } - ] + } } } \ No newline at end of file From bc45469b6a93009279118d40cfdb90e63d128196 Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Thu, 7 May 2015 13:13:57 +0300 Subject: [PATCH 059/158] Link to CI backup documentation --- doc/raketasks/backup_restore.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index bfef975024..b83a713012 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -9,6 +9,8 @@ This archive will be saved in backup_path (see `config/gitlab.yml`). The filename will be `[TIMESTAMP]_gitlab_backup.tar`. This timestamp can be used to restore an specific backup. You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. +If you are interested in GitLab CI backup please follow to the [CI backup documentation](https://gitlab.com/gitlab-org/gitlab-ci/blob/master/doc/backup.md)* + ``` # use this command if you've installed GitLab with the Omnibus package sudo gitlab-rake gitlab:backup:create From deddad6025afcbd7279c483f1388149e2fcca2e2 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 14:22:14 +0200 Subject: [PATCH 060/158] Document sidekiq restarting functionality. --- doc/install/installation.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/install/installation.md b/doc/install/installation.md index a61a40ebd1..ca25eaea79 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -4,6 +4,12 @@ Since an installation from source is a lot of work and error prone we strongly recommend the fast and reliable [Omnibus package installation](https://about.gitlab.com/downloads/) (deb/rpm). +One reason the Omnibus package is more reliable is its use of Runit to restart any of the GitLab processes in case one crashes. +On heavily used GitLab instances the memory usage of the Sidekiq background worker will grow over time. +Omnibus packages solve this by [letting the Sidekiq terminate gracefully](http://doc.gitlab.com/ce/operations/sidekiq_memory_killer.html) if it uses too much memory. +After this termination Runit will detect Sidekiq is not running and will start it. +Since installations from source don't have Runit, Sidekiq can't be terminated and its memory usage will grow over time. + ## Select Version to Install Make sure you view [this installation guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md) from the tag (version) of GitLab you would like to install. From 895bc33fef53b8ffd9eab0a9c31e3c81af8ec6a9 Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Thu, 7 May 2015 12:24:18 +0000 Subject: [PATCH 061/158] Update upgrader doc to point to 7.10.2 --- doc/update/upgrader.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/update/upgrader.md b/doc/update/upgrader.md index 2cca46c86a..b228708328 100644 --- a/doc/update/upgrader.md +++ b/doc/update/upgrader.md @@ -27,10 +27,10 @@ If you have local changes to your GitLab repository the script will stash them a Note: GitLab 7.9 adds `nodejs` as a dependency. GitLab 7.6 adds `libkrb5-dev` as a dependency (installed by default on Ubuntu and OSX). GitLab 7.2 adds `pkg-config` and `cmake` as dependency. Please check the dependencies in the [installation guide.](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies) cd /home/git/gitlab - sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.1"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' + sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.2"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' # to perform a non-interactive install (no user input required) you can add -y - # sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.1"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y + # sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.2"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y ## 3. Start application @@ -65,7 +65,7 @@ Here is a one line command with step 1 to 5 for the next time you upgrade: cd /home/git/gitlab; \ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production; \ sudo service gitlab stop; \ - sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.1"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y; \ + sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.2"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y; \ cd /home/git/gitlab-shell; \ sudo -u git -H git fetch; \ sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION`; \ @@ -73,4 +73,4 @@ cd /home/git/gitlab; \ sudo service gitlab start; \ sudo service nginx restart; \ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production -``` +``` \ No newline at end of file From 93a2cc095bea039c92751673c4d81c04bab0b3a2 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Thu, 7 May 2015 12:41:05 +0000 Subject: [PATCH 062/158] Fix spelling error, thanks Mark --- docker/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/README.md b/docker/README.md index 82eac9cd0d..2e533ae9dd 100644 --- a/docker/README.md +++ b/docker/README.md @@ -94,7 +94,7 @@ sudo docker build --tag gitlab-data docker/data/ sudo docker build --tag gitlab-app:7.10.1 docker/app/ ``` -After this run the images as described in the prepivous section. +After this run the images as described in the previous section. We assume using a data volume container, this will simplify migrations and backups. This empty container will exist to persist as volumes the 3 directories used by GitLab, so remember not to delete it. @@ -152,4 +152,4 @@ sudo docker push sytse/gitlab_data ## Troubleshooting -Please see the [troubleshooting](troubleshooting.md) file in this directory. +Please see the [troubleshooting](troubleshooting.md) file in this directory. \ No newline at end of file From 415648e2555e25d30f64f4c2642cf149f6965859 Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Thu, 7 May 2015 13:39:10 +0000 Subject: [PATCH 063/158] revert to 7.10.1 --- doc/update/upgrader.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/update/upgrader.md b/doc/update/upgrader.md index b228708328..d23534d58b 100644 --- a/doc/update/upgrader.md +++ b/doc/update/upgrader.md @@ -27,10 +27,10 @@ If you have local changes to your GitLab repository the script will stash them a Note: GitLab 7.9 adds `nodejs` as a dependency. GitLab 7.6 adds `libkrb5-dev` as a dependency (installed by default on Ubuntu and OSX). GitLab 7.2 adds `pkg-config` and `cmake` as dependency. Please check the dependencies in the [installation guide.](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies) cd /home/git/gitlab - sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.2"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' + sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.1"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' # to perform a non-interactive install (no user input required) you can add -y - # sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.2"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y + # sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.1"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y ## 3. Start application @@ -65,7 +65,7 @@ Here is a one line command with step 1 to 5 for the next time you upgrade: cd /home/git/gitlab; \ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production; \ sudo service gitlab stop; \ - sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.2"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y; \ + sudo -u git -H ruby -Ilib -e 'require "gitlab/upgrader"' -e 'class Gitlab::Upgrader' -e 'def latest_version_raw' -e '"v7.10.1"' -e 'end' -e 'end' -e 'Gitlab::Upgrader.new.execute' -- -y; \ cd /home/git/gitlab-shell; \ sudo -u git -H git fetch; \ sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION`; \ From e7dc9ca2d026a4d95aac74a8a040b605d3055ae3 Mon Sep 17 00:00:00 2001 From: Marin Jankovski Date: Thu, 7 May 2015 16:15:58 +0200 Subject: [PATCH 064/158] Add yanking pkg example to the release documentation. --- doc/release/master.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/doc/release/master.md b/doc/release/master.md index 19070b46a0..9163e65200 100644 --- a/doc/release/master.md +++ b/doc/release/master.md @@ -31,3 +31,32 @@ git remote add gl git@gitlab.com:gitlab-org/gitlab-ce.git gpa ``` +# Yanking packages from packages.gitlab.com + +In case something went wrong with the release and there is a need to remove the packages you can yank the packages by following the +procedure described in [package cloud documentation](https://packagecloud.io/docs#yank_pkg). + +You need to have: + +1. `package_cloud` gem installed (sudo gem install package_cloud) +1. Email and password for packages.gitlab.com +1. Make sure that you are supplying the url to packages.gitlab.com (default is packagecloud.io) + +Example of yanking a package: + +```bash +package_cloud yank --url https://packages.gitlab.com gitlab/gitlab-ce/el/6 gitlab-ce-7.10.2~omnibus-1.x86_64.rpm +``` + +If you are attempting this for the first time the output will look something like: + +```bash +Looking for repository at gitlab/gitlab-ce... No config file exists at /Users/marin/.packagecloud. Login to create one. +Email: +marin@gitlab.com +Password: + +Got your token. Writing a config file to /Users/marin/.packagecloud... success! +success! +Attempting to yank package at gitlab/gitlab-ce/el/6/gitlab-ce-7.10.2~omnibus-1.x86_64.rpm...done! +``` From 4164f6155ab91fe64d3b8cd66f5db23ba6005018 Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Thu, 7 May 2015 14:26:21 +0000 Subject: [PATCH 065/158] new issue after release --- doc/release/patch.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/release/patch.md b/doc/release/patch.md index 4c7b471785..a569bb3da8 100644 --- a/doc/release/patch.md +++ b/doc/release/patch.md @@ -44,7 +44,7 @@ Create release tag and push to remotes: bundle exec rake release["x.x.x"] ``` -### Release +## Release 1. [Build new packages with the latest version](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/release.md) 1. Apply the patch to GitLab.com and the private GitLab development server @@ -53,3 +53,4 @@ bundle exec rake release["x.x.x"] 1. Send tweets about the release from `@gitlab`, tweet should include the most important feature that the release is addressing and link to the blog post 1. Note in the 'GitLab X.X regressions' issue that the patch was published (CE only) 1. [Create new AMIs](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md) +1. Create a new patch release issue for the next potential release \ No newline at end of file From 3e6181db143e62c471da98d6474730c512a591c3 Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Thu, 7 May 2015 17:29:15 +0300 Subject: [PATCH 066/158] Make migrations reversible --- ...d_default_project_visibililty_to_application_settings.rb | 6 +++++- ...20150429002313_remove_abandoned_group_members_records.rb | 5 ++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/db/migrate/20150423033240_add_default_project_visibililty_to_application_settings.rb b/db/migrate/20150423033240_add_default_project_visibililty_to_application_settings.rb index 9b0f13f3fa..50a9b2439e 100644 --- a/db/migrate/20150423033240_add_default_project_visibililty_to_application_settings.rb +++ b/db/migrate/20150423033240_add_default_project_visibililty_to_application_settings.rb @@ -1,7 +1,11 @@ class AddDefaultProjectVisibililtyToApplicationSettings < ActiveRecord::Migration - def change + def up add_column :application_settings, :default_project_visibility, :integer visibility = Settings.gitlab.default_projects_features['visibility_level'] execute("update application_settings set default_project_visibility = #{visibility}") end + + def down + remove_column :application_settings, :default_project_visibility + end end diff --git a/db/migrate/20150429002313_remove_abandoned_group_members_records.rb b/db/migrate/20150429002313_remove_abandoned_group_members_records.rb index 6013605bb3..244637e1c4 100644 --- a/db/migrate/20150429002313_remove_abandoned_group_members_records.rb +++ b/db/migrate/20150429002313_remove_abandoned_group_members_records.rb @@ -1,6 +1,9 @@ class RemoveAbandonedGroupMembersRecords < ActiveRecord::Migration - def change + def up execute("DELETE FROM members WHERE type = 'GroupMember' AND source_id NOT IN(\ SELECT id FROM namespaces WHERE type='Group')") end + + def down + end end From b60717604c8f7b2cb089d4fd30cc282d4f77a85f Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Thu, 7 May 2015 18:17:58 +0300 Subject: [PATCH 067/158] remove tag duplicates --- .../20150425164647_remove_duplicate_tags.rb | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 db/migrate/20150425164647_remove_duplicate_tags.rb diff --git a/db/migrate/20150425164647_remove_duplicate_tags.rb b/db/migrate/20150425164647_remove_duplicate_tags.rb new file mode 100644 index 0000000000..1a9152cb96 --- /dev/null +++ b/db/migrate/20150425164647_remove_duplicate_tags.rb @@ -0,0 +1,16 @@ +class RemoveDuplicateTags < ActiveRecord::Migration + def up + select_all("SELECT name, COUNT(id) as cnt FROM tags GROUP BY name HAVING COUNT(id) > 1").each do |tag| + duplicate_ids = select_all("SELECT id FROM tags WHERE name = '#{tag["name"]}'").map{|tag| tag["id"]} + origin_tag_id = duplicate_ids.first + duplicate_ids.delete origin_tag_id + + execute("UPDATE taggings SET tag_id = #{origin_tag_id} WHERE tag_id IN(#{duplicate_ids.join(",")})") + execute("DELETE FROM tags WHERE id IN(#{duplicate_ids.join(",")})") + end + end + + def down + + end +end From 94a426d2e66d68ef9b93075f3984040c3a22e693 Mon Sep 17 00:00:00 2001 From: Karen Carias Date: Thu, 7 May 2015 16:11:18 +0000 Subject: [PATCH 068/158] Added link to root password doc correctly --- doc/security/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/security/README.md b/doc/security/README.md index 49dfa6eec7..473f3632dc 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -4,3 +4,4 @@ - [Rack attack](rack_attack.md) - [Web Hooks and insecure internal web services](webhooks.md) - [Information exclusivity](information_exclusivity.md) +- [Reset your root password](reset_root_password.md) \ No newline at end of file From 2a3c1d7cdda27c1365d62b5528b60b9f90e7d702 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hannes=20Rosen=C3=B6gger?= <123haynes@gmail.com> Date: Thu, 7 May 2015 18:18:23 +0200 Subject: [PATCH 069/158] Fix commit time being displayed in the wrong timezone in some cases --- CHANGELOG | 1 + app/helpers/application_helper.rb | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG b/CHANGELOG index 90e9d96e7c..0261912da4 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,6 +1,7 @@ Please view this file on the master branch, on stable branches it's out of date. v 7.11.0 (unreleased) + - Fix commit time being displayed in the wrong timezone in some cases (Hannes Rosenögger) - Make the first branch pushed to an empty repository the default HEAD (Stan Hu) - Make Reply-To config apply to change e-mail confirmation and other Devise notifications (Stan Hu) - Add application setting to restrict user signups to e-mail domains (Stan Hu) diff --git a/app/helpers/application_helper.rb b/app/helpers/application_helper.rb index 6e86400a4f..672be54e66 100644 --- a/app/helpers/application_helper.rb +++ b/app/helpers/application_helper.rb @@ -214,7 +214,7 @@ module ApplicationHelper def time_ago_with_tooltip(date, placement = 'top', html_class = 'time_ago') capture_haml do haml_tag :time, date.to_s, - class: html_class, datetime: date.getutc.iso8601, title: date.stamp('Aug 21, 2011 9:23pm'), + class: html_class, datetime: date.getutc.iso8601, title: date.in_time_zone.stamp('Aug 21, 2011 9:23pm'), data: { toggle: 'tooltip', placement: placement } haml_tag :script, "$('." + html_class + "').timeago().tooltip()" From f972a9e34c4957d01da8de3003c3a1406a0cd037 Mon Sep 17 00:00:00 2001 From: Sullivan SENECHAL Date: Tue, 5 May 2015 00:08:05 +0200 Subject: [PATCH 070/158] Fix bootlint issues on git clone holder bar --- app/assets/stylesheets/base/mixins.scss | 4 +++ app/assets/stylesheets/pages/projects.scss | 9 ++++++ app/views/shared/_clone_panel.html.haml | 34 +++++++++++----------- 3 files changed, 30 insertions(+), 17 deletions(-) diff --git a/app/assets/stylesheets/base/mixins.scss b/app/assets/stylesheets/base/mixins.scss index b8b163a42b..a0794e7825 100644 --- a/app/assets/stylesheets/base/mixins.scss +++ b/app/assets/stylesheets/base/mixins.scss @@ -21,6 +21,10 @@ @include border-radius($radius 0 0 $radius) } +@mixin border-radius-right($radius) { + @include border-radius(0 0 $radius $radius) +} + @mixin linear-gradient($from, $to) { background-image: -webkit-gradient(linear, 0 0, 0 100%, from($from), to($to)); background-image: -webkit-linear-gradient($from, $to); diff --git a/app/assets/stylesheets/pages/projects.scss b/app/assets/stylesheets/pages/projects.scss index e6a4bc62c5..5a8d466529 100644 --- a/app/assets/stylesheets/pages/projects.scss +++ b/app/assets/stylesheets/pages/projects.scss @@ -102,6 +102,15 @@ .input-group-addon { background: #FAFAFA; + + &.git-protocols { + padding: 0; + border: none; + + .input-group-btn:last-child > .btn { + @include border-radius-right(0); + } + } } } diff --git a/app/views/shared/_clone_panel.html.haml b/app/views/shared/_clone_panel.html.haml index aaab0d35f5..60bb76e898 100644 --- a/app/views/shared/_clone_panel.html.haml +++ b/app/views/shared/_clone_panel.html.haml @@ -1,22 +1,22 @@ - project = project || @project .git-clone-holder.input-group - .input-group-btn - %button{ | - :type => 'button', | - class: "btn btn-sm #{ 'active' if default_clone_protocol == 'ssh' }#{ ' has_tooltip' if current_user && current_user.require_ssh_key? }", | - :"data-clone" => project.ssh_url_to_repo, | - :"data-title" => "Add an SSH key to your profile
    to pull or push via SSH", - :"data-html" => "true", - :"data-container" => "body"} - SSH - %button{ | - :type => 'button', | - class: "btn btn-sm #{ 'active' if default_clone_protocol == 'http' }#{ ' has_tooltip' if current_user && current_user.require_password? }", | - :"data-clone" => project.http_url_to_repo, | - :"data-title" => "Set a password on your account
    to pull or push via #{gitlab_config.protocol.upcase}", - :"data-html" => "true", - :"data-container" => "body"} - = gitlab_config.protocol.upcase + .input-group-addon.git-protocols + .input-group-btn + %button{ | + class: "btn btn-sm #{ 'active' if default_clone_protocol == 'ssh' }#{ ' has_tooltip' if current_user && current_user.require_ssh_key? }", | + :"data-clone" => project.ssh_url_to_repo, | + :"data-title" => "Add an SSH key to your profile
    to pull or push via SSH", + :"data-html" => "true", + :"data-container" => "body"} + SSH + .input-group-btn + %button{ | + class: "btn btn-sm #{ 'active' if default_clone_protocol == 'http' }#{ ' has_tooltip' if current_user && current_user.require_password? }", | + :"data-clone" => project.http_url_to_repo, | + :"data-title" => "Set a password on your account
    to pull or push via #{gitlab_config.protocol.upcase}", + :"data-html" => "true", + :"data-container" => "body"} + = gitlab_config.protocol.upcase = text_field_tag :project_clone, default_url_to_repo(project), class: "js-select-on-focus form-control input-sm", readonly: true - if project.kind_of?(Project) .input-group-addon From 1c1f18b416febb8296facca37dc7504c2da880f8 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Thu, 7 May 2015 18:47:03 +0200 Subject: [PATCH 071/158] Add SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL env var It looks like SIGTERM may not be enough to shut down a Sidekiq process when its RSS has gotten too big. This change will allow us to experiment with sending SIGKILL instead of SIGTERM to Sidekiq processes on gitlab.com. --- CHANGELOG | 1 + doc/operations/sidekiq_memory_killer.md | 2 ++ lib/gitlab/sidekiq_middleware/memory_killer.rb | 11 ++++++----- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index c7fb49dc18..fadf46209b 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -44,6 +44,7 @@ v 7.11.0 (unreleased) - Allow to use non-ASCII letters and dashes in project and namespace name. (Jakub Jirutka) - Add footnotes support to Markdown (Guillaume Delbergue) - Add current_sign_in_at to UserFull REST api. + - Make Sidekiq MemoryKiller shutdown signal configurable v 7.10.2 - Fix CI links on MR page diff --git a/doc/operations/sidekiq_memory_killer.md b/doc/operations/sidekiq_memory_killer.md index 867b01b0d5..811c2192a1 100644 --- a/doc/operations/sidekiq_memory_killer.md +++ b/doc/operations/sidekiq_memory_killer.md @@ -36,3 +36,5 @@ The MemoryKiller is controlled using environment variables. Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must restart Sidekiq. +- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to 'SIGTERM'. The name of + the final signal sent to the Sidekiq process when we want it to shut down. diff --git a/lib/gitlab/sidekiq_middleware/memory_killer.rb b/lib/gitlab/sidekiq_middleware/memory_killer.rb index 0f2db50e98..f33b2dedf4 100644 --- a/lib/gitlab/sidekiq_middleware/memory_killer.rb +++ b/lib/gitlab/sidekiq_middleware/memory_killer.rb @@ -7,6 +7,7 @@ module Gitlab GRACE_TIME = (ENV['SIDEKIQ_MEMORY_KILLER_GRACE_TIME'] || 15 * 60).to_s.to_i # Wait 30 seconds for running jobs to finish during graceful shutdown SHUTDOWN_WAIT = (ENV['SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT'] || 30).to_s.to_i + SHUTDOWN_SIGNAL = (ENV['SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL'] || 'SIGTERM').to_s # Create a mutex used to ensure there will be only one thread waiting to # shut Sidekiq down @@ -24,19 +25,19 @@ module Gitlab Sidekiq.logger.warn "current RSS #{current_rss} exceeds maximum RSS "\ "#{MAX_RSS}" - Sidekiq.logger.warn "spawned thread that will shut down PID "\ - "#{Process.pid} in #{GRACE_TIME} seconds" + Sidekiq.logger.warn "this thread will shut down PID #{Process.pid} "\ + "in #{GRACE_TIME} seconds" sleep(GRACE_TIME) Sidekiq.logger.warn "sending SIGUSR1 to PID #{Process.pid}" Process.kill('SIGUSR1', Process.pid) Sidekiq.logger.warn "waiting #{SHUTDOWN_WAIT} seconds before sending "\ - "SIGTERM to PID #{Process.pid}" + "#{SHUTDOWN_SIGNAL} to PID #{Process.pid}" sleep(SHUTDOWN_WAIT) - Sidekiq.logger.warn "sending SIGTERM to PID #{Process.pid}" - Process.kill('SIGTERM', Process.pid) + Sidekiq.logger.warn "sending #{SHUTDOWN_SIGNAL} to PID #{Process.pid}" + Process.kill(SHUTDOWN_SIGNAL, Process.pid) end end From c406bce4b57fade307bf4b563b8084f97b943cff Mon Sep 17 00:00:00 2001 From: Karen Carias Date: Thu, 7 May 2015 16:48:25 +0000 Subject: [PATCH 072/158] Deleted > and commented "or" out --- doc/security/reset_root_password.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/doc/security/reset_root_password.md b/doc/security/reset_root_password.md index 58c7b0e5a7..dbbe6c3e45 100644 --- a/doc/security/reset_root_password.md +++ b/doc/security/reset_root_password.md @@ -5,7 +5,7 @@ Log into your server with root privileges. Then start a Ruby on Rails console. Start the console with this command: ```bash -> gitlab-rails console production +gitlab-rails console production ``` Wait until the console has loaded. @@ -14,15 +14,19 @@ There are multiple ways to find your user. You can search for email or username. ```bash irb(main):001:0> u = User.where(id: 1).first +``` + or + +```bash user = User.find_by(email: 'admin@local.host') ``` Now you can change your password: ```bash -> u.password = 'secret_pass' -> u.password_confirmation = 'secret_pass' +u.password = 'secret_pass' +u.password_confirmation = 'secret_pass' ``` It's important that you change both password and password_confirmation to make it work. @@ -30,7 +34,7 @@ It's important that you change both password and password_confirmation to make i Don't forget to save the changes. ```bash -> u.save! +u.save! ``` -Exit the console and try to login with your new password. +Exit the console and try to login with your new password. \ No newline at end of file From bf4b4384590c271d1dfadf874d185c2f6130ad0e Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Wed, 6 May 2015 14:33:39 -0700 Subject: [PATCH 073/158] Fix bug where avatar filenames were not actually deleted from the database during removal. This would result in a 404 error in certain views. The `save` call was being rolled back due to an error in the validation step. Relax the validation step so that this works. Closes #1570 --- CHANGELOG | 2 +- app/models/group.rb | 2 +- app/models/project.rb | 2 +- app/models/user.rb | 4 ++-- .../groups/avatars_controller_spec.rb | 17 +++++++++++++++++ .../profiles/avatars_controller_spec.rb | 17 +++++++++++++++++ .../projects/avatars_controller_spec.rb | 18 ++++++++++++++++++ 7 files changed, 57 insertions(+), 5 deletions(-) create mode 100644 spec/controllers/groups/avatars_controller_spec.rb create mode 100644 spec/controllers/profiles/avatars_controller_spec.rb create mode 100644 spec/controllers/projects/avatars_controller_spec.rb diff --git a/CHANGELOG b/CHANGELOG index 90e9d96e7c..17e5028953 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -33,7 +33,7 @@ v 7.11.0 (unreleased) - Show incompatible projects in Google Code import status (Stan Hu) - Fix bug where commit data would not appear in some subdirectories (Stan Hu) - Unescape branch names in compare commit (Stan Hu) - - + - Fix bug where avatar filenames were not actually deleted from the database during removal (Stan Hu) - Fix bug where Slack service channel was not saved in admin template settings. (Stan Hu) - Move snippets UI to fluid layout - Improve UI for sidebar. Increase separation between navigation and content diff --git a/app/models/group.rb b/app/models/group.rb index 1386a9eccc..687458adac 100644 --- a/app/models/group.rb +++ b/app/models/group.rb @@ -20,7 +20,7 @@ class Group < Namespace has_many :group_members, dependent: :destroy, as: :source, class_name: 'GroupMember' has_many :users, through: :group_members - validate :avatar_type, if: ->(user) { user.avatar_changed? } + validate :avatar_type, if: ->(user) { user.avatar.present? && user.avatar_changed? } validates :avatar, file_size: { maximum: 200.kilobytes.to_i } mount_uploader :avatar, AvatarUploader diff --git a/app/models/project.rb b/app/models/project.rb index e866681aab..6a2d789544 100644 --- a/app/models/project.rb +++ b/app/models/project.rb @@ -144,7 +144,7 @@ class Project < ActiveRecord::Base validates :star_count, numericality: { greater_than_or_equal_to: 0 } validate :check_limit, on: :create validate :avatar_type, - if: ->(project) { project.avatar && project.avatar_changed? } + if: ->(project) { project.avatar.present? && project.avatar_changed? } validates :avatar, file_size: { maximum: 200.kilobytes.to_i } mount_uploader :avatar, AvatarUploader diff --git a/app/models/user.rb b/app/models/user.rb index 1cf7cfea97..703a7d3270 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -137,7 +137,7 @@ class User < ActiveRecord::Base validates :notification_level, inclusion: { in: Notification.notification_levels }, presence: true validate :namespace_uniq, if: ->(user) { user.username_changed? } - validate :avatar_type, if: ->(user) { user.avatar_changed? } + validate :avatar_type, if: ->(user) { user.avatar.present? && user.avatar_changed? } validate :unique_email, if: ->(user) { user.email_changed? } validate :owns_notification_email, if: ->(user) { user.notification_email_changed? } validate :owns_public_email, if: ->(user) { user.public_email_changed? } @@ -298,7 +298,7 @@ class User < ActiveRecord::Base if primary_email_record primary_email_record.destroy self.emails.create(email: self.email_was) - + self.update_secondary_emails! end end diff --git a/spec/controllers/groups/avatars_controller_spec.rb b/spec/controllers/groups/avatars_controller_spec.rb new file mode 100644 index 0000000000..3dac134a73 --- /dev/null +++ b/spec/controllers/groups/avatars_controller_spec.rb @@ -0,0 +1,17 @@ +require 'spec_helper' + +describe Groups::AvatarsController do + let(:user) { create(:user) } + let(:group) { create(:group, owner: user, avatar: fixture_file_upload(Rails.root + "spec/fixtures/dk.png", "image/png")) } + + before do + sign_in(user) + end + + it 'destroy should remove avatar from DB' do + delete :destroy, group_id: group.path + @group = assigns(:group) + expect(@group.avatar.present?).to be_falsey + expect(@group).to be_valid + end +end diff --git a/spec/controllers/profiles/avatars_controller_spec.rb b/spec/controllers/profiles/avatars_controller_spec.rb new file mode 100644 index 0000000000..ad5855df0a --- /dev/null +++ b/spec/controllers/profiles/avatars_controller_spec.rb @@ -0,0 +1,17 @@ +require 'spec_helper' + +describe Profiles::AvatarsController do + let(:user) { create(:user, avatar: fixture_file_upload(Rails.root + "spec/fixtures/dk.png")) } + + before do + sign_in(user) + controller.instance_variable_set(:@user, user) + end + + it 'destroy should remove avatar from DB' do + delete :destroy + @user = assigns(:user) + expect(@user.avatar.present?).to be_falsey + expect(@user).to be_valid + end +end diff --git a/spec/controllers/projects/avatars_controller_spec.rb b/spec/controllers/projects/avatars_controller_spec.rb new file mode 100644 index 0000000000..e79b46a350 --- /dev/null +++ b/spec/controllers/projects/avatars_controller_spec.rb @@ -0,0 +1,18 @@ +require 'spec_helper' + +describe Projects::AvatarsController do + let(:project) { create(:project, avatar: fixture_file_upload(Rails.root + "spec/fixtures/dk.png", "image/png")) } + let(:user) { create(:user) } + + before do + sign_in(user) + project.team << [user, :developer] + controller.instance_variable_set(:@project, project) + end + + it 'destroy should remove avatar from DB' do + delete :destroy, namespace_id: project.namespace.id, project_id: project.id + expect(project.avatar.present?).to be_falsey + expect(project).to be_valid + end +end From 548cec6ea46f0d66a60b564b2b72281b3c800fdd Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Thu, 7 May 2015 16:26:45 -0700 Subject: [PATCH 074/158] Change the MySQL collation type to utf8_bin to ensure case-sensitive tags are supported --- ...llation_for_tag_names.acts_as_taggable_on_engine.rb | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 db/migrate/20150425164646_gitlab_change_collation_for_tag_names.acts_as_taggable_on_engine.rb diff --git a/db/migrate/20150425164646_gitlab_change_collation_for_tag_names.acts_as_taggable_on_engine.rb b/db/migrate/20150425164646_gitlab_change_collation_for_tag_names.acts_as_taggable_on_engine.rb new file mode 100644 index 0000000000..281c88d2a7 --- /dev/null +++ b/db/migrate/20150425164646_gitlab_change_collation_for_tag_names.acts_as_taggable_on_engine.rb @@ -0,0 +1,10 @@ +# This migration is a duplicate of 20150425164651_change_collation_for_tag_names.acts_as_taggable_on_engine.rb +# It shold be applied before the index additions to ensure that `name` is case sensitive. + +class GitlabChangeCollationForTagNames < ActiveRecord::Migration + def up + if ActsAsTaggableOn::Utils.using_mysql? + execute("ALTER TABLE tags MODIFY name varchar(255) CHARACTER SET utf8 COLLATE utf8_bin;") + end + end +end From f6c0ef5553fde03ae3a04ba48733455a5415a3ac Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Fri, 8 May 2015 00:27:47 -0700 Subject: [PATCH 075/158] Conditionally remove indices that may not exist in pre-GitLab v6.7 installations Closes #1593 --- ...ssing_unique_indices.acts_as_taggable_on_engine.rb | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/db/migrate/20150425164648_add_missing_unique_indices.acts_as_taggable_on_engine.rb b/db/migrate/20150425164648_add_missing_unique_indices.acts_as_taggable_on_engine.rb index 4ca676f6c7..c1b7868151 100644 --- a/db/migrate/20150425164648_add_missing_unique_indices.acts_as_taggable_on_engine.rb +++ b/db/migrate/20150425164648_add_missing_unique_indices.acts_as_taggable_on_engine.rb @@ -3,8 +3,15 @@ class AddMissingUniqueIndices < ActiveRecord::Migration def self.up add_index :tags, :name, unique: true - remove_index :taggings, :tag_id - remove_index :taggings, [:taggable_id, :taggable_type, :context] + # pre-GitLab v6.7.0 may not have these indices since there were no + # migrations for them + if index_exists?(:taggings, :tag_id) + remove_index :taggings, :tag_id + end + + if index_exists?(:taggings, [:taggable_id, :taggable_type, :context]) + remove_index :taggings, [:taggable_id, :taggable_type, :context] + end add_index :taggings, [:tag_id, :taggable_id, :taggable_type, :context, :tagger_id, :tagger_type], unique: true, name: 'taggings_idx' From b031a26376e871eb276a1d38e3c405eb695f7538 Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Fri, 8 May 2015 15:02:53 +0300 Subject: [PATCH 076/158] improved backup doc --- doc/raketasks/backup_restore.md | 72 +++++++++++++++++++++++++++++---- 1 file changed, 65 insertions(+), 7 deletions(-) diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index b83a713012..984d7b408c 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -152,11 +152,9 @@ If you have an installation from source, please consider backing up your `gitlab You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. -``` -# Omnibus package installation -sudo gitlab-rake gitlab:backup:restore +### Installation from source -# installation from source +``` bundle exec rake gitlab:backup:restore RAILS_ENV=production ``` @@ -198,11 +196,45 @@ Restoring repositories: Deleting tmp directories...[DONE] ``` +### Omnibus installations + +We will assume that you have installed GitLab from an omnibus package and run +`sudo gitlab-ctl reconfigure` at least once. + +First make sure your backup tar file is in `/var/opt/gitlab/backups`. + +```shell +sudo cp 1393513186_gitlab_backup.tar /var/opt/gitlab/backups/ +``` + +Next, restore the backup by running the restore command. You need to specify the +timestamp of the backup you are restoring. + +```shell +# Stop processes that are connected to the database +sudo gitlab-ctl stop unicorn +sudo gitlab-ctl stop sidekiq + +# This command will overwrite the contents of your GitLab database! +sudo gitlab-rake gitlab:backup:restore BACKUP=1393513186 + +# Start GitLab +sudo gitlab-ctl start + +# Create satellites +sudo gitlab-rake gitlab:satellites:create + +# Check GitLab +sudo gitlab-rake gitlab:check SANITIZE=true +``` + +If there is a GitLab version mismatch between your backup tar file and the installed +version of GitLab, the restore command will abort with an error. Install a package for +the [required version](https://www.gitlab.com/downloads/archives/) and try again. + ## Configure cron to make daily backups -For Omnibus package installations, see https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#scheduling-a-backup . - -For installation from source: +### For installation from source: ``` cd /home/git/gitlab sudo -u git -H editor config/gitlab.yml # Enable keep_time in the backup section to automatically delete old backups @@ -219,6 +251,32 @@ Add the following lines at the bottom: The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. This is recommended to reduce cron spam. +### For omnibus installations + +To schedule a cron job that backs up your repositories and GitLab metadata, use the root user: + +``` +sudo su - +crontab -e +``` + +There, add the following line to schedule the backup for everyday at 2 AM: + +``` +0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create CRON=1 +``` + +You may also want to set a limited lifetime for backups to prevent regular +backups using all your disk space. To do this add the following lines to +`/etc/gitlab/gitlab.rb` and reconfigure: + +``` +# limit backup lifetime to 7 days - 604800 seconds +gitlab_rails['backup_keep_time'] = 604800 +``` + +NOTE: This cron job does not [backup your omnibus-gitlab configuration](#backup-and-restore-omnibus-gitlab-configuration) or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). + ## Alternative backup strategies If your GitLab server contains a lot of Git repository data you may find the GitLab backup script to be too slow. From 3a18eb8d3ad927ff37b28df867388d877360201a Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Fri, 8 May 2015 15:22:19 +0300 Subject: [PATCH 077/158] Fix link to the CI documentation --- doc/raketasks/backup_restore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 984d7b408c..bca4fcfb40 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -9,7 +9,7 @@ This archive will be saved in backup_path (see `config/gitlab.yml`). The filename will be `[TIMESTAMP]_gitlab_backup.tar`. This timestamp can be used to restore an specific backup. You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. -If you are interested in GitLab CI backup please follow to the [CI backup documentation](https://gitlab.com/gitlab-org/gitlab-ci/blob/master/doc/backup.md)* +If you are interested in GitLab CI backup please follow to the [CI backup documentation](https://gitlab.com/gitlab-org/gitlab-ci/blob/master/doc/raketasks/backup_restore.md)* ``` # use this command if you've installed GitLab with the Omnibus package From acac7889023de96af61d293d855c33996bd780a1 Mon Sep 17 00:00:00 2001 From: bugagazavr Date: Sat, 24 Jan 2015 03:10:43 +0300 Subject: [PATCH 078/158] Added X-GitLab-Event header for web hooks --- CHANGELOG | 1 + app/controllers/admin/hooks_controller.rb | 2 +- app/models/hooks/service_hook.rb | 4 +++ app/models/hooks/web_hook.rb | 16 ++++++--- app/models/project.rb | 2 +- app/services/system_hooks_service.rb | 6 ++-- app/services/test_hook_service.rb | 2 +- app/workers/project_web_hook_worker.rb | 4 +-- app/workers/system_hook_worker.rb | 4 +-- doc/system_hooks/system_hooks.md | 6 ++++ doc/web_hooks/web_hooks.md | 25 ++++++++++++++ lib/api/system_hooks.rb | 2 +- spec/models/hooks/service_hook_spec.rb | 33 ++++++++++++++++++ spec/models/hooks/system_hook_spec.rb | 42 +++++++++++++++++------ spec/models/hooks/web_hook_spec.rb | 14 +++++--- 15 files changed, 132 insertions(+), 31 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 77d5f07eb5..051870269f 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -23,6 +23,7 @@ v 7.11.0 (unreleased) - Fix bug causing `@whatever` inside an inline code snippet (backtick-style) to be picked up as a user mention. - When use change branches link at MR form - save source branch selection instead of target one - Improve handling of large diffs + - Added GitLab Event header for project hooks - - Show Atom feed buttons everywhere where applicable. - Add project activity atom feed. diff --git a/app/controllers/admin/hooks_controller.rb b/app/controllers/admin/hooks_controller.rb index 0a463239d7..690096bdbc 100644 --- a/app/controllers/admin/hooks_controller.rb +++ b/app/controllers/admin/hooks_controller.rb @@ -33,7 +33,7 @@ class Admin::HooksController < Admin::ApplicationController owner_name: "Someone", owner_email: "example@gitlabhq.com" } - @hook.execute(data) + @hook.execute(data, 'system_hooks') redirect_to :back end diff --git a/app/models/hooks/service_hook.rb b/app/models/hooks/service_hook.rb index 2e11239c40..5b38ade2e6 100644 --- a/app/models/hooks/service_hook.rb +++ b/app/models/hooks/service_hook.rb @@ -17,4 +17,8 @@ class ServiceHook < WebHook belongs_to :service + + def execute(data) + super(data, 'service_hook') + end end diff --git a/app/models/hooks/web_hook.rb b/app/models/hooks/web_hook.rb index 315d96af1b..e9fd441352 100644 --- a/app/models/hooks/web_hook.rb +++ b/app/models/hooks/web_hook.rb @@ -30,12 +30,15 @@ class WebHook < ActiveRecord::Base validates :url, presence: true, format: { with: /\A#{URI.regexp(%w(http https))}\z/, message: "should be a valid url" } - def execute(data) + def execute(data, hook_name) parsed_url = URI.parse(url) if parsed_url.userinfo.blank? WebHook.post(url, body: data.to_json, - headers: { "Content-Type" => "application/json" }, + headers: { + "Content-Type" => "application/json", + "X-Gitlab-Event" => hook_name.singularize.titleize + }, verify: false) else post_url = url.gsub("#{parsed_url.userinfo}@", "") @@ -45,7 +48,10 @@ class WebHook < ActiveRecord::Base } WebHook.post(post_url, body: data.to_json, - headers: { "Content-Type" => "application/json" }, + headers: { + "Content-Type" => "application/json", + "X-Gitlab-Event" => hook_name.singularize.titleize + }, verify: false, basic_auth: auth) end @@ -54,7 +60,7 @@ class WebHook < ActiveRecord::Base false end - def async_execute(data) - Sidekiq::Client.enqueue(ProjectWebHookWorker, id, data) + def async_execute(data, hook_name) + Sidekiq::Client.enqueue(ProjectWebHookWorker, id, data, hook_name) end end diff --git a/app/models/project.rb b/app/models/project.rb index e866681aab..bc77e7217e 100644 --- a/app/models/project.rb +++ b/app/models/project.rb @@ -483,7 +483,7 @@ class Project < ActiveRecord::Base def execute_hooks(data, hooks_scope = :push_hooks) hooks.send(hooks_scope).each do |hook| - hook.async_execute(data) + hook.async_execute(data, hooks_scope.to_s) end end diff --git a/app/services/system_hooks_service.rb b/app/services/system_hooks_service.rb index c5d0b08845..60235b6be2 100644 --- a/app/services/system_hooks_service.rb +++ b/app/services/system_hooks_service.rb @@ -7,12 +7,12 @@ class SystemHooksService def execute_hooks(data) SystemHook.all.each do |sh| - async_execute_hook sh, data + async_execute_hook(sh, data, 'system_hooks') end end - def async_execute_hook(hook, data) - Sidekiq::Client.enqueue(SystemHookWorker, hook.id, data) + def async_execute_hook(hook, data, hook_name) + Sidekiq::Client.enqueue(SystemHookWorker, hook.id, data, hook_name) end def build_event_data(model, event) diff --git a/app/services/test_hook_service.rb b/app/services/test_hook_service.rb index 21ec2c01cb..e85e58751e 100644 --- a/app/services/test_hook_service.rb +++ b/app/services/test_hook_service.rb @@ -1,6 +1,6 @@ class TestHookService def execute(hook, current_user) data = Gitlab::PushDataBuilder.build_sample(hook.project, current_user) - hook.execute(data) + hook.execute(data, 'push_hooks') end end diff --git a/app/workers/project_web_hook_worker.rb b/app/workers/project_web_hook_worker.rb index 73085c046b..fb87896528 100644 --- a/app/workers/project_web_hook_worker.rb +++ b/app/workers/project_web_hook_worker.rb @@ -3,8 +3,8 @@ class ProjectWebHookWorker sidekiq_options queue: :project_web_hook - def perform(hook_id, data) + def perform(hook_id, data, hook_name) data = data.with_indifferent_access - WebHook.find(hook_id).execute(data) + WebHook.find(hook_id).execute(data, hook_name) end end diff --git a/app/workers/system_hook_worker.rb b/app/workers/system_hook_worker.rb index 3ebc62b7e7..a122c27476 100644 --- a/app/workers/system_hook_worker.rb +++ b/app/workers/system_hook_worker.rb @@ -3,7 +3,7 @@ class SystemHookWorker sidekiq_options queue: :system_hook - def perform(hook_id, data) - SystemHook.find(hook_id).execute data + def perform(hook_id, data, hook_name) + SystemHook.find(hook_id).execute(data, hook_name) end end diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index f9b6d37d84..b0e4613cde 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -6,6 +6,12 @@ System hooks can be used, e.g. for logging or changing information in a LDAP ser ## Hooks request example +**Request header**: + +``` +X-Gitlab-Event: System Hook +``` + **Project created:** ```json diff --git a/doc/web_hooks/web_hooks.md b/doc/web_hooks/web_hooks.md index 851f50f5e9..0108255519 100644 --- a/doc/web_hooks/web_hooks.md +++ b/doc/web_hooks/web_hooks.md @@ -12,6 +12,12 @@ If you send a web hook to an SSL endpoint [the certificate will not be verified] Triggered when you push to the repository except when pushing tags. +**Request header**: + +``` +X-Gitlab-Event: Push Hook +``` + **Request body:** ```json @@ -63,6 +69,13 @@ Triggered when you push to the repository except when pushing tags. Triggered when you create (or delete) tags to the repository. +**Request header**: + +``` +X-Gitlab-Event: Tag Push Hook +``` + + **Request body:** ```json @@ -92,6 +105,12 @@ Triggered when you create (or delete) tags to the repository. Triggered when a new issue is created or an existing issue was updated/closed/reopened. +**Request header**: + +``` +X-Gitlab-Event: Issue Hook +``` + **Request body:** ```json @@ -126,6 +145,12 @@ Triggered when a new issue is created or an existing issue was updated/closed/re Triggered when a new merge request is created or an existing merge request was updated/merged/closed. +**Request header**: + +``` +X-Gitlab-Event: Merge Request Hook +``` + **Request body:** ```json diff --git a/lib/api/system_hooks.rb b/lib/api/system_hooks.rb index 518964db50..22b8f90dc5 100644 --- a/lib/api/system_hooks.rb +++ b/lib/api/system_hooks.rb @@ -47,7 +47,7 @@ module API owner_name: "Someone", owner_email: "example@gitlabhq.com" } - @hook.execute(data) + @hook.execute(data, 'system_hooks') data end diff --git a/spec/models/hooks/service_hook_spec.rb b/spec/models/hooks/service_hook_spec.rb index 96bf74d45d..d9714596f5 100644 --- a/spec/models/hooks/service_hook_spec.rb +++ b/spec/models/hooks/service_hook_spec.rb @@ -21,4 +21,37 @@ describe ServiceHook do describe "Associations" do it { is_expected.to belong_to :service } end + + describe "execute" do + before(:each) do + @service_hook = create(:service_hook) + @data = { project_id: 1, data: {}} + + WebMock.stub_request(:post, @service_hook.url) + end + + it "POSTs to the web hook URL" do + @service_hook.execute(@data) + expect(WebMock).to have_requested(:post, @service_hook.url).with( + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'Service Hook'} + ).once + end + + it "POSTs the data as JSON" do + json = @data.to_json + + @service_hook.execute(@data) + expect(WebMock).to have_requested(:post, @service_hook.url).with( + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'Service Hook'} + ).once + end + + it "catches exceptions" do + expect(WebHook).to receive(:post).and_raise("Some HTTP Post error") + + expect { + @service_hook.execute(@data) + }.to raise_error + end + end end diff --git a/spec/models/hooks/system_hook_spec.rb b/spec/models/hooks/system_hook_spec.rb index 810b311a40..e4b6b88656 100644 --- a/spec/models/hooks/system_hook_spec.rb +++ b/spec/models/hooks/system_hook_spec.rb @@ -26,32 +26,47 @@ describe SystemHook do it "project_create hook" do Projects::CreateService.new(create(:user), name: 'empty').execute - expect(WebMock).to have_requested(:post, @system_hook.url).with(body: /project_create/).once + expect(WebMock).to have_requested(:post, @system_hook.url).with( + body: /project_create/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} + ).once end it "project_destroy hook" do user = create(:user) project = create(:empty_project, namespace: user.namespace) Projects::DestroyService.new(project, user, {}).execute - expect(WebMock).to have_requested(:post, @system_hook.url).with(body: /project_destroy/).once + expect(WebMock).to have_requested(:post, @system_hook.url).with( + body: /project_destroy/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} + ).once end it "user_create hook" do create(:user) - expect(WebMock).to have_requested(:post, @system_hook.url).with(body: /user_create/).once + expect(WebMock).to have_requested(:post, @system_hook.url).with( + body: /user_create/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} + ).once end it "user_destroy hook" do user = create(:user) user.destroy - expect(WebMock).to have_requested(:post, @system_hook.url).with(body: /user_destroy/).once + expect(WebMock).to have_requested(:post, @system_hook.url).with( + body: /user_destroy/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} + ).once end it "project_create hook" do user = create(:user) project = create(:project) project.team << [user, :master] - expect(WebMock).to have_requested(:post, @system_hook.url).with(body: /user_add_to_team/).once + expect(WebMock).to have_requested(:post, @system_hook.url).with( + body: /user_add_to_team/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} + ).once end it "project_destroy hook" do @@ -59,13 +74,17 @@ describe SystemHook do project = create(:project) project.team << [user, :master] project.project_members.destroy_all - expect(WebMock).to have_requested(:post, @system_hook.url).with(body: /user_remove_from_team/).once + expect(WebMock).to have_requested(:post, @system_hook.url).with( + body: /user_remove_from_team/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} + ).once end it 'group create hook' do create(:group) expect(WebMock).to have_requested(:post, @system_hook.url).with( - body: /group_create/ + body: /group_create/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} ).once end @@ -73,7 +92,8 @@ describe SystemHook do group = create(:group) group.destroy expect(WebMock).to have_requested(:post, @system_hook.url).with( - body: /group_destroy/ + body: /group_destroy/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} ).once end @@ -82,7 +102,8 @@ describe SystemHook do user = create(:user) group.add_user(user, Gitlab::Access::MASTER) expect(WebMock).to have_requested(:post, @system_hook.url).with( - body: /user_add_to_group/ + body: /user_add_to_group/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} ).once end @@ -92,7 +113,8 @@ describe SystemHook do group.add_user(user, Gitlab::Access::MASTER) group.group_members.destroy_all expect(WebMock).to have_requested(:post, @system_hook.url).with( - body: /user_remove_from_group/ + body: /user_remove_from_group/, + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'System Hook'} ).once end diff --git a/spec/models/hooks/web_hook_spec.rb b/spec/models/hooks/web_hook_spec.rb index 67ec9193ad..9f5ef3eff7 100644 --- a/spec/models/hooks/web_hook_spec.rb +++ b/spec/models/hooks/web_hook_spec.rb @@ -52,22 +52,26 @@ describe ProjectHook do end it "POSTs to the web hook URL" do - @project_hook.execute(@data) - expect(WebMock).to have_requested(:post, @project_hook.url).once + @project_hook.execute(@data, 'push_hooks') + expect(WebMock).to have_requested(:post, @project_hook.url).with( + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'Push Hook'} + ).once end it "POSTs the data as JSON" do json = @data.to_json - @project_hook.execute(@data) - expect(WebMock).to have_requested(:post, @project_hook.url).with(body: json).once + @project_hook.execute(@data, 'push_hooks') + expect(WebMock).to have_requested(:post, @project_hook.url).with( + headers: {'Content-Type'=>'application/json', 'X-Gitlab-Event'=>'Push Hook'} + ).once end it "catches exceptions" do expect(WebHook).to receive(:post).and_raise("Some HTTP Post error") expect { - @project_hook.execute(@data) + @project_hook.execute(@data, 'push_hooks') }.to raise_error end end From 3bc717f01075f5dd2ae8968002525b5237601b12 Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Fri, 8 May 2015 16:54:45 +0300 Subject: [PATCH 079/158] Adding quote_string to migration --- db/migrate/20150425164647_remove_duplicate_tags.rb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/db/migrate/20150425164647_remove_duplicate_tags.rb b/db/migrate/20150425164647_remove_duplicate_tags.rb index 1a9152cb96..13e5038db9 100644 --- a/db/migrate/20150425164647_remove_duplicate_tags.rb +++ b/db/migrate/20150425164647_remove_duplicate_tags.rb @@ -1,7 +1,8 @@ class RemoveDuplicateTags < ActiveRecord::Migration def up select_all("SELECT name, COUNT(id) as cnt FROM tags GROUP BY name HAVING COUNT(id) > 1").each do |tag| - duplicate_ids = select_all("SELECT id FROM tags WHERE name = '#{tag["name"]}'").map{|tag| tag["id"]} + tag_name = quote_string(tag["name"]) + duplicate_ids = select_all("SELECT id FROM tags WHERE name = '#{tag_name}'").map{|tag| tag["id"]} origin_tag_id = duplicate_ids.first duplicate_ids.delete origin_tag_id From 83c5aa6888f97082e1fd0850ff5cd071147501be Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Fri, 8 May 2015 14:35:05 +0000 Subject: [PATCH 080/158] remove redundant colon --- doc/security/reset_root_password.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/security/reset_root_password.md b/doc/security/reset_root_password.md index dbbe6c3e45..d791f53adf 100644 --- a/doc/security/reset_root_password.md +++ b/doc/security/reset_root_password.md @@ -1,4 +1,4 @@ -# How to reset your root password: +# How to reset your root password Log into your server with root privileges. Then start a Ruby on Rails console. From c74a0be9f84acc8e70e31b7f3ad322d84a0f2174 Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Fri, 8 May 2015 14:36:24 +0000 Subject: [PATCH 081/158] make it more obvious what happens in the code --- doc/security/reset_root_password.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/security/reset_root_password.md b/doc/security/reset_root_password.md index d791f53adf..3c13f26267 100644 --- a/doc/security/reset_root_password.md +++ b/doc/security/reset_root_password.md @@ -13,7 +13,7 @@ Wait until the console has loaded. There are multiple ways to find your user. You can search for email or username. ```bash -irb(main):001:0> u = User.where(id: 1).first +user = User.where(id: 1).first ``` or @@ -25,8 +25,8 @@ user = User.find_by(email: 'admin@local.host') Now you can change your password: ```bash -u.password = 'secret_pass' -u.password_confirmation = 'secret_pass' +user.password = 'secret_pass' +user.password_confirmation = 'secret_pass' ``` It's important that you change both password and password_confirmation to make it work. @@ -34,7 +34,7 @@ It's important that you change both password and password_confirmation to make i Don't forget to save the changes. ```bash -u.save! +user.save! ``` Exit the console and try to login with your new password. \ No newline at end of file From 15909d6411522ed11813c33e49ce695ac765b851 Mon Sep 17 00:00:00 2001 From: Marin Jankovski Date: Fri, 8 May 2015 18:12:07 +0200 Subject: [PATCH 082/158] Move project visibility from clone bar to repository details block for non empty project. --- app/views/projects/_aside.html.haml | 4 +++- app/views/shared/_clone_panel.html.haml | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/app/views/projects/_aside.html.haml b/app/views/projects/_aside.html.haml index 1865b5be8c..1241b51f9a 100644 --- a/app/views/projects/_aside.html.haml +++ b/app/views/projects/_aside.html.haml @@ -4,7 +4,9 @@ - unless @project.empty_repo? .well - %h4 Repository + %h4.visibility-level-label + = visibility_level_icon(@project.visibility_level) + = "#{visibility_level_label(@project.visibility_level).capitalize} project" %ul.nav.nav-pills %li= link_to pluralize(number_with_delimiter(@repository.commit_count), 'commit'), namespace_project_commits_path(@project.namespace, @project, @ref || @repository.root_ref) %li= link_to pluralize(number_with_delimiter(@repository.branch_names.count), 'branch'), namespace_project_branches_path(@project.namespace, @project) diff --git a/app/views/shared/_clone_panel.html.haml b/app/views/shared/_clone_panel.html.haml index 60bb76e898..3f489a04e7 100644 --- a/app/views/shared/_clone_panel.html.haml +++ b/app/views/shared/_clone_panel.html.haml @@ -18,7 +18,7 @@ :"data-container" => "body"} = gitlab_config.protocol.upcase = text_field_tag :project_clone, default_url_to_repo(project), class: "js-select-on-focus form-control input-sm", readonly: true - - if project.kind_of?(Project) + - if project.kind_of?(Project) && project.empty_repo? .input-group-addon .visibility-level-label.has_tooltip{'data-title' => "#{visibility_level_label(project.visibility_level)} project" } = visibility_level_icon(project.visibility_level) From 70bbf093aa07d416ea33da24ab015e5d22c0d501 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Fri, 8 May 2015 12:17:54 -0400 Subject: [PATCH 083/158] Remove class and id attributes from SanitizationFilter whitelist --- doc/markdown/markdown.md | 2 +- lib/gitlab/markdown/sanitization_filter.rb | 19 +++++++++++-- spec/features/markdown_spec.rb | 28 +++++++++---------- spec/fixtures/markdown.md.erb | 26 ++++++++--------- .../markdown/sanitization_filter_spec.rb | 20 +++++++++---- 5 files changed, 59 insertions(+), 36 deletions(-) diff --git a/doc/markdown/markdown.md b/doc/markdown/markdown.md index e95ddbb757..30c29084e3 100644 --- a/doc/markdown/markdown.md +++ b/doc/markdown/markdown.md @@ -423,7 +423,7 @@ Quote break. You can also use raw HTML in your Markdown, and it'll mostly work pretty well. -See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span` elements, as well as the `class`, and `id` attributes on all elements. +See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span` elements. ```no-highlight
    diff --git a/lib/gitlab/markdown/sanitization_filter.rb b/lib/gitlab/markdown/sanitization_filter.rb index 9a154e0b2f..6f33155bad 100644 --- a/lib/gitlab/markdown/sanitization_filter.rb +++ b/lib/gitlab/markdown/sanitization_filter.rb @@ -10,8 +10,9 @@ module Gitlab def whitelist whitelist = HTML::Pipeline::SanitizationFilter::WHITELIST - # Allow `class` and `id` on all elements - whitelist[:attributes][:all].push('class', 'id') + # Allow code highlighting + whitelist[:attributes]['pre'] = %w(class) + whitelist[:attributes]['span'] = %w(class) # Allow table alignment whitelist[:attributes]['th'] = %w(style) @@ -23,6 +24,9 @@ module Gitlab # Remove `rel` attribute from `a` elements whitelist[:transformers].push(remove_rel) + # Remove `class` attribute from non-highlight spans + whitelist[:transformers].push(clean_spans) + whitelist end @@ -33,6 +37,17 @@ module Gitlab end end end + + def clean_spans + lambda do |env| + return unless env[:node_name] == 'span' + return unless env[:node].has_attribute?('class') + + unless has_ancestor?(env[:node], 'pre') + env[:node].remove_attribute('class') + end + end + end end end end diff --git a/spec/features/markdown_spec.rb b/spec/features/markdown_spec.rb index 1746ce128e..8f3dfc8d5a 100644 --- a/spec/features/markdown_spec.rb +++ b/spec/features/markdown_spec.rb @@ -60,8 +60,8 @@ describe 'GitLab Markdown' do @feat.teardown end - # Given a header ID, goes to that element's parent (the header), then to its - # second sibling (the body). + # Given a header ID, goes to that element's parent (the header itself), then + # its next sibling element (the body). def get_section(id) @doc.at_css("##{id}").parent.next_element end @@ -119,18 +119,18 @@ describe 'GitLab Markdown' do describe 'HTML::Pipeline' do describe 'SanitizationFilter' do it 'uses a permissive whitelist' do - expect(@doc).to have_selector('b#manual-b') - expect(@doc).to have_selector('em#manual-em') - expect(@doc).to have_selector("code#manual-code") + expect(@doc).to have_selector('b:contains("b tag")') + expect(@doc).to have_selector('em:contains("em tag")') + expect(@doc).to have_selector('code:contains("code tag")') expect(@doc).to have_selector('kbd:contains("s")') expect(@doc).to have_selector('strike:contains(Emoji)') - expect(@doc).to have_selector('img#manual-img') - expect(@doc).to have_selector('br#manual-br') - expect(@doc).to have_selector('hr#manual-hr') + expect(@doc).to have_selector('img[src*="smile.png"]') + expect(@doc).to have_selector('br') + expect(@doc).to have_selector('hr') end it 'permits span elements' do - expect(@doc).to have_selector('span#span-class-light.light') + expect(@doc).to have_selector('span:contains("span tag")') end it 'permits table alignment' do @@ -144,13 +144,12 @@ describe 'GitLab Markdown' do end it 'removes `rel` attribute from links' do - expect(@doc).to have_selector('a#a-rel-nofollow') - expect(@doc).not_to have_selector('a#a-rel-nofollow[rel]') + body = get_section('sanitizationfilter') + expect(body).not_to have_selector('a[rel]') end it "removes `href` from `a` elements if it's fishy" do - expect(@doc).to have_selector('a#a-href-javascript') - expect(@doc).not_to have_selector('a#a-href-javascript[href]') + expect(@doc).not_to have_selector('a[href*="javascript"]') end end @@ -228,7 +227,8 @@ describe 'GitLab Markdown' do %w(code a kbd).each do |elem| it "ignores links inside '#{elem}' element" do - expect(@doc.at_css("#{elem}#autolink-#{elem}").child).to be_text + body = get_section('autolinkfilter') + expect(body).not_to have_selector("#{elem} a") end end end diff --git a/spec/fixtures/markdown.md.erb b/spec/fixtures/markdown.md.erb index bc023ecf79..64817ec670 100644 --- a/spec/fixtures/markdown.md.erb +++ b/spec/fixtures/markdown.md.erb @@ -54,36 +54,34 @@ After the Markdown has been turned into HTML, it gets passed through... ### SanitizationFilter -GitLab uses HTML::Pipeline::SanitizationFilter +GitLab uses HTML::Pipeline::SanitizationFilter to sanitize the generated HTML, stripping dangerous or unwanted tags. Its default whitelist is pretty permissive. Check it: -This text is bold and this text is emphasized. +b tag and em tag. -echo "Hello, world!" +code tag Press s to search. -Emoji Plain old images! +Emoji Plain old images! Here comes a line break: -
    +
    And a horizontal rule: -
    +
    As permissive as it is, we've allowed even more stuff: -Span elements +span tag -This is a link with a defined rel attribute, which should be removed +This is a link with a defined rel attribute, which should be removed -This is a link trying to be sneaky. It gets its link removed entirely. +This is a link trying to be sneaky. It gets its link removed entirely. ### Escaping @@ -125,9 +123,9 @@ These are all plain text that should get turned into links: But it shouldn't autolink text inside certain tags: -- http://about.gitlab.com/ -- http://about.gitlab.com/ -- http://about.gitlab.com/ +- http://about.gitlab.com/ +- http://about.gitlab.com/ +- http://about.gitlab.com/ ### Reference Filters (e.g., #<%= issue.iid %>) diff --git a/spec/lib/gitlab/markdown/sanitization_filter_spec.rb b/spec/lib/gitlab/markdown/sanitization_filter_spec.rb index ab909a6863..4a1aa76614 100644 --- a/spec/lib/gitlab/markdown/sanitization_filter_spec.rb +++ b/spec/lib/gitlab/markdown/sanitization_filter_spec.rb @@ -29,17 +29,27 @@ module Gitlab::Markdown exp = act = "
    \n
    Term
    \n
    Definition
    \n
    " expect(filter(act).to_html).to eq exp end + + it 'sanitizes `class` attribute on any element' do + act = %q{Strong} + expect(filter(act).to_html).to eq %q{Strong} + end + + it 'sanitizes `id` attribute on any element' do + act = %q{Emphasis} + expect(filter(act).to_html).to eq %q{Emphasis} + end end describe 'custom whitelist' do - it 'allows `class` attribute on any element' do - exp = act = %q{Strong} + it 'allows syntax highlighting' do + exp = act = %q{
    def
    } expect(filter(act).to_html).to eq exp end - it 'allows `id` attribute on any element' do - exp = act = %q{Emphasis} - expect(filter(act).to_html).to eq exp + it 'sanitizes `class` attribute from non-highlight spans' do + act = %q{def} + expect(filter(act).to_html).to eq %q{def} end it 'allows `style` attribute on table elements' do From a0fd2eb568cdbe3537f10a83191fde893a5d6fcd Mon Sep 17 00:00:00 2001 From: Maran Date: Fri, 8 May 2015 15:03:26 +0200 Subject: [PATCH 084/158] Don't show duplicate deploy keys. Fixes #1516 --- CHANGELOG | 1 + app/models/user.rb | 2 +- features/project/deploy_keys.feature | 7 ++++--- features/steps/project/deploy_keys.rb | 14 ++++++++++++-- 4 files changed, 18 insertions(+), 6 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 84bdf78e98..33f1344860 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,6 +1,7 @@ Please view this file on the master branch, on stable branches it's out of date. v 7.11.0 (unreleased) + - Don't show duplicate deploy keys - Make the first branch pushed to an empty repository the default HEAD (Stan Hu) - Make Reply-To config apply to change e-mail confirmation and other Devise notifications (Stan Hu) - Add application setting to restrict user signups to e-mail domains (Stan Hu) diff --git a/app/models/user.rb b/app/models/user.rb index 1cf7cfea97..a70cbaa518 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -438,7 +438,7 @@ class User < ActiveRecord::Base end def project_deploy_keys - DeployKey.in_projects(self.authorized_projects.pluck(:id)) + DeployKey.unscoped.in_projects(self.authorized_projects.pluck(:id)).distinct(:id) end def accessible_deploy_keys diff --git a/features/project/deploy_keys.feature b/features/project/deploy_keys.feature index a71f6124d9..47cf774094 100644 --- a/features/project/deploy_keys.feature +++ b/features/project/deploy_keys.feature @@ -9,9 +9,10 @@ Feature: Project Deploy Keys Then I should see project deploy key Scenario: I should see project deploy keys - Given other project has deploy key + Given other projects have deploy keys When I visit project deploy keys page - Then I should see other project deploy key + Then I should see other project deploy key + And I should only see the same deploy key once Scenario: I should see public deploy keys Given public deploy key exists @@ -26,7 +27,7 @@ Feature: Project Deploy Keys And I should see newly created deploy key Scenario: I attach other project deploy key to project - Given other project has deploy key + Given other projects have deploy keys And I visit project deploy keys page When I click attach deploy key Then I should be on deploy keys page diff --git a/features/steps/project/deploy_keys.rb b/features/steps/project/deploy_keys.rb index 50e14513a7..81d1182cd1 100644 --- a/features/steps/project/deploy_keys.rb +++ b/features/steps/project/deploy_keys.rb @@ -45,10 +45,20 @@ class Spinach::Features::ProjectDeployKeys < Spinach::FeatureSteps end end - step 'other project has deploy key' do - @second_project = create :project, namespace: create(:group) + step 'other projects have deploy keys' do + @second_project = create(:project, namespace: create(:group)) @second_project.team << [current_user, :master] create(:deploy_keys_project, project: @second_project) + + @third_project = create(:project, namespace: create(:group)) + @third_project.team << [current_user, :master] + create(:deploy_keys_project, project: @third_project, deploy_key: @second_project.deploy_keys.first) + end + + step 'I should only see the same deploy key once' do + within '.available-keys' do + page.should have_selector('ul li', count: 1) + end end step 'public deploy key exists' do From 8f75ab21d021fb5335ed9e084157cf2bc06caca1 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 22:25:46 +0200 Subject: [PATCH 085/158] Link Commits tab to current branch from Compare page. --- app/controllers/projects/compare_controller.rb | 3 ++- app/views/projects/commits/_head.html.haml | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/app/controllers/projects/compare_controller.rb b/app/controllers/projects/compare_controller.rb index 7c20b81c0b..c5f085c236 100644 --- a/app/controllers/projects/compare_controller.rb +++ b/app/controllers/projects/compare_controller.rb @@ -6,11 +6,12 @@ class Projects::CompareController < Projects::ApplicationController before_action :authorize_download_code! def index + @ref = Addressable::URI.unescape(params[:to]) end def show base_ref = Addressable::URI.unescape(params[:from]) - head_ref = Addressable::URI.unescape(params[:to]) + @ref = head_ref = Addressable::URI.unescape(params[:to]) compare_result = CompareService.new.execute( current_user, diff --git a/app/views/projects/commits/_head.html.haml b/app/views/projects/commits/_head.html.haml index a714f5f79e..66101f3f0d 100644 --- a/app/views/projects/commits/_head.html.haml +++ b/app/views/projects/commits/_head.html.haml @@ -1,6 +1,6 @@ %ul.nav.nav-tabs = nav_link(controller: [:commit, :commits]) do - = link_to namespace_project_commits_path(@project.namespace, @project, @repository.root_ref) do + = link_to namespace_project_commits_path(@project.namespace, @project, @ref || @repository.root_ref) do Commits %span.badge= number_with_precision(@repository.commit_count, precision: 0, delimiter: ',') = nav_link(controller: :compare) do From 722e3935893e0493e5783f1a61034fe1481f8cb8 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 22:26:25 +0200 Subject: [PATCH 086/158] Add Create Merge Request buttons to commits page and push event. --- CHANGELOG | 1 + app/helpers/compare_helper.rb | 24 +++++++++++----------- app/views/events/event/_push.html.haml | 14 ++++++++++++- app/views/projects/commits/show.html.haml | 16 ++++++++++----- app/views/projects/compare/_form.html.haml | 7 ++++--- 5 files changed, 41 insertions(+), 21 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 3206c625cd..2303282f0a 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -49,6 +49,7 @@ v 7.11.0 (unreleased) - Add footnotes support to Markdown (Guillaume Delbergue) - Add current_sign_in_at to UserFull REST api. - Make Sidekiq MemoryKiller shutdown signal configurable + - Add "Create Merge Request" buttons to commits page and push event. v 7.10.2 - Fix CI links on MR page diff --git a/app/helpers/compare_helper.rb b/app/helpers/compare_helper.rb index 01847c6b80..fe339d3215 100644 --- a/app/helpers/compare_helper.rb +++ b/app/helpers/compare_helper.rb @@ -1,21 +1,21 @@ module CompareHelper - def compare_to_mr_button? - @project.merge_requests_enabled && - params[:from].present? && - params[:to].present? && - @repository.branch_names.include?(params[:from]) && - @repository.branch_names.include?(params[:to]) && - params[:from] != params[:to] && + def create_mr_button?(from = params[:from], to = params[:to], project = @project) + project.merge_requests_enabled && + from.present? && + to.present? && + project.repository.branch_names.include?(from) && + project.repository.branch_names.include?(to) && + from != to && !@refs_are_same end - def compare_mr_path + def create_mr_path(from = params[:from], to = params[:to], project = @project) new_namespace_project_merge_request_path( - @project.namespace, - @project, + project.namespace, + project, merge_request: { - source_branch: params[:to], - target_branch: params[:from] + source_branch: to, + target_branch: from } ) end diff --git a/app/views/events/event/_push.html.haml b/app/views/events/event/_push.html.haml index 60d7978b13..1da702be38 100644 --- a/app/views/events/event/_push.html.haml +++ b/app/views/events/event/_push.html.haml @@ -17,15 +17,27 @@ - few_commits.each do |commit| = render "events/commit", commit: commit, project: project + - create_mr = current_user == event.author && event.new_ref? && create_mr_button?(event.project.default_branch, event.ref_name, event.project) - if event.commits_count > 1 %li.commits-stat - if event.commits_count > 2 %span ... and #{event.commits_count - 2} more commits. + - if event.md_ref? - from = event.commit_from - from_label = truncate_sha(from) - else - from = event.project.default_branch - from_label = from + = link_to namespace_project_compare_path(event.project.namespace, event.project, from: from, to: event.commit_to) do - %strong Compare → #{from_label}...#{truncate_sha(event.commit_to)} + Compare #{from_label}...#{truncate_sha(event.commit_to)} + + - if create_mr + or + = link_to create_mr_path(event.project.default_branch, event.ref_name, event.project) do + create a merge request + - elsif create_mr + %li.commits-stat + = link_to create_mr_path(event.project.default_branch, event.ref_name, event.project) do + Create Merge Request diff --git a/app/views/projects/commits/show.html.haml b/app/views/projects/commits/show.html.haml index c8531b090a..e2a8d3ffbb 100644 --- a/app/views/projects/commits/show.html.haml +++ b/app/views/projects/commits/show.html.haml @@ -8,11 +8,17 @@ .tree-ref-holder = render 'shared/ref_switcher', destination: 'commits' -- if current_user && current_user.private_token - .commits-feed-holder.hidden-xs.hidden-sm - = link_to namespace_project_commits_path(@project.namespace, @project, @ref, {format: :atom, private_token: current_user.private_token}), title: "Feed", class: 'btn' do - %i.fa.fa-rss - Commits feed +.commits-feed-holder.hidden-xs.hidden-sm + - if create_mr_button?(@repository.root_ref, @ref) + = link_to create_mr_path(@repository.root_ref, @ref), class: 'btn btn-success' do + = icon('plus') + Create Merge Request + + - if current_user && current_user.private_token + = link_to namespace_project_commits_path(@project.namespace, @project, @ref, {format: :atom, private_token: current_user.private_token}), title: "Feed", class: 'prepend-left-10 btn' do + = icon("rss") + Commits Feed + %ul.breadcrumb.repo-breadcrumb = commits_breadcrumbs diff --git a/app/views/projects/compare/_form.html.haml b/app/views/projects/compare/_form.html.haml index dfb1dded9e..a0e904cfd8 100644 --- a/app/views/projects/compare/_form.html.haml +++ b/app/views/projects/compare/_form.html.haml @@ -13,9 +13,10 @@ = text_field_tag :to, params[:to], class: "form-control"   = button_tag "Compare", class: "btn btn-create commits-compare-btn" - - if compare_to_mr_button? - = link_to compare_mr_path, class: 'prepend-left-10 btn' do - %strong Make a merge request + - if create_mr_button? + = link_to create_mr_path, class: 'prepend-left-10 btn' do + = icon("plus") + Create Merge Request :javascript From 164d47448831f64c1bb3e193172a2e9fcbb84712 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 22:55:19 +0200 Subject: [PATCH 087/158] Add Merge Request button to branches page. --- CHANGELOG | 2 +- app/views/projects/branches/_branch.html.haml | 11 +++++++---- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 2303282f0a..f069fd30d9 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -49,7 +49,7 @@ v 7.11.0 (unreleased) - Add footnotes support to Markdown (Guillaume Delbergue) - Add current_sign_in_at to UserFull REST api. - Make Sidekiq MemoryKiller shutdown signal configurable - - Add "Create Merge Request" buttons to commits page and push event. + - Add "Create Merge Request" buttons to commits and branches pages and push event. v 7.10.2 - Fix CI links on MR page diff --git a/app/views/projects/branches/_branch.html.haml b/app/views/projects/branches/_branch.html.haml index 4e7415be4a..43412624da 100644 --- a/app/views/projects/branches/_branch.html.haml +++ b/app/views/projects/branches/_branch.html.haml @@ -10,16 +10,19 @@ %i.fa.fa-lock protected .pull-right - - if can?(current_user, :download_code, @project) - = render 'projects/repositories/download_archive', ref: branch.name, btn_class: 'btn-grouped btn-group-xs' + - if create_mr_button?(@repository.root_ref, branch.name) + = link_to create_mr_path(@repository.root_ref, branch.name), class: 'btn btn-grouped btn-xs' do + = icon('plus') + Merge Request + - if branch.name != @repository.root_ref = link_to namespace_project_compare_index_path(@project.namespace, @project, from: @repository.root_ref, to: branch.name), class: 'btn btn-grouped btn-xs', method: :post, title: "Compare" do - %i.fa.fa-files-o + = icon("exchange") Compare - if can_remove_branch?(@project, branch.name) = link_to namespace_project_branch_path(@project.namespace, @project, branch.name), class: 'btn btn-grouped btn-xs btn-remove remove-row', method: :delete, data: { confirm: 'Removed branch cannot be restored. Are you sure?'}, remote: true do - %i.fa.fa-trash-o + = icon("trash-o") - if commit %ul.list-unstyled From fe75411df8d8ed41e89cfcae73d1ea34b8b339b9 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 27 Mar 2015 15:35:26 -0700 Subject: [PATCH 088/158] Add 2 factor authentication gems --- Gemfile | 4 ++++ Gemfile.lock | 16 ++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/Gemfile b/Gemfile index f950c5be15..65a641c602 100644 --- a/Gemfile +++ b/Gemfile @@ -34,6 +34,10 @@ gem 'omniauth-bitbucket' gem 'doorkeeper', '2.1.3' gem "rack-oauth2", "~> 1.0.5" +# Two-factor authentication +gem 'devise-two-factor' +gem 'rqrcode-rails3' + # Browser detection gem "browser" diff --git a/Gemfile.lock b/Gemfile.lock index 6f58c4f4fd..35faa9895b 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -46,6 +46,8 @@ GEM ast (2.0.0) astrolabe (1.3.0) parser (>= 2.2.0.pre.3, < 3.0) + attr_encrypted (1.3.3) + encryptor (>= 1.3.0) attr_required (1.0.0) autoprefixer-rails (5.1.11) execjs @@ -136,6 +138,13 @@ GEM warden (~> 1.2.3) devise-async (0.9.0) devise (~> 3.2) + devise-two-factor (1.0.1) + activemodel + activesupport + attr_encrypted (~> 1.3.2) + devise (~> 3.2.4) + rails + rotp (~> 1.6.1) diff-lcs (1.2.5) diffy (3.0.3) docile (1.1.5) @@ -147,6 +156,7 @@ GEM email_spec (1.5.0) launchy (~> 2.1) mail (~> 2.2) + encryptor (1.3.0) enumerize (0.7.0) activesupport (>= 3.2) equalizer (0.0.8) @@ -482,7 +492,11 @@ GEM rest-client (1.6.7) mime-types (>= 1.16) rinku (1.7.3) + rotp (1.6.1) rouge (1.7.7) + rqrcode (0.4.2) + rqrcode-rails3 (0.1.7) + rqrcode (>= 0.4.2) rspec (2.99.0) rspec-core (~> 2.99.0) rspec-expectations (~> 2.99.0) @@ -691,6 +705,7 @@ DEPENDENCIES default_value_for (~> 3.0.0) devise (= 3.2.4) devise-async (= 0.9.0) + devise-two-factor diffy (~> 3.0.3) doorkeeper (= 2.1.3) dropzonejs-rails @@ -762,6 +777,7 @@ DEPENDENCIES redis-rails request_store rerun (~> 0.10.0) + rqrcode-rails3 rspec-rails (= 2.99) rubocop (= 0.28.0) rugments From 7302395142dc93a45239c993b69958ca4a757c92 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 27 Mar 2015 16:23:33 -0700 Subject: [PATCH 089/158] Init 2 factor authentication for user model --- app/models/user.rb | 5 ++++- app/views/devise/sessions/_new_base.html.haml | 3 ++- config/initializers/devise.rb | 4 ++++ .../20150327223628_add_devise_two_factor_to_users.rb | 8 ++++++++ 4 files changed, 18 insertions(+), 2 deletions(-) create mode 100644 db/migrate/20150327223628_add_devise_two_factor_to_users.rb diff --git a/app/models/user.rb b/app/models/user.rb index 1cf7cfea97..b9e2890018 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -57,6 +57,9 @@ require 'carrierwave/orm/activerecord' require 'file_size_validator' class User < ActiveRecord::Base + devise :two_factor_authenticatable, + :otp_secret_encryption_key => File.read(Rails.root.join('.secret')).chomp + include Sortable include Gitlab::ConfigHelper include TokenAuthenticatable @@ -70,7 +73,7 @@ class User < ActiveRecord::Base default_value_for :hide_no_password, false default_value_for :theme_id, gitlab_config.default_theme - devise :database_authenticatable, :lockable, :async, + devise :lockable, :async, :recoverable, :rememberable, :trackable, :validatable, :omniauthable, :confirmable, :registerable attr_accessor :force_random_password diff --git a/app/views/devise/sessions/_new_base.html.haml b/app/views/devise/sessions/_new_base.html.haml index 54a3972677..4ecb74fb56 100644 --- a/app/views/devise/sessions/_new_base.html.haml +++ b/app/views/devise/sessions/_new_base.html.haml @@ -1,6 +1,7 @@ = form_for(resource, as: resource_name, url: session_path(resource_name)) do |f| = f.text_field :login, class: "form-control top", placeholder: "Username or Email", autofocus: "autofocus" - = f.password_field :password, class: "form-control bottom", placeholder: "Password" + = f.password_field :password, class: "form-control middle", placeholder: "Password" + = f.text_field :otp_attempt, class: 'form-control bottom', placeholder: 'Two-factor authentication token' - if devise_mapping.rememberable? .remember-me.checkbox %label{for: "user_remember_me"} diff --git a/config/initializers/devise.rb b/config/initializers/devise.rb index 8f8c416974..956bb048b2 100644 --- a/config/initializers/devise.rb +++ b/config/initializers/devise.rb @@ -1,6 +1,10 @@ # Use this hook to configure devise mailer, warden hooks and so forth. The first # four configuration values can also be set straight in your models. Devise.setup do |config| + config.warden do |manager| + manager.default_strategies(:scope => :user).unshift :two_factor_authenticatable + end + # ==> Mailer Configuration # Configure the class responsible to send e-mails. config.mailer = "DeviseMailer" diff --git a/db/migrate/20150327223628_add_devise_two_factor_to_users.rb b/db/migrate/20150327223628_add_devise_two_factor_to_users.rb new file mode 100644 index 0000000000..11b026ee8f --- /dev/null +++ b/db/migrate/20150327223628_add_devise_two_factor_to_users.rb @@ -0,0 +1,8 @@ +class AddDeviseTwoFactorToUsers < ActiveRecord::Migration + def change + add_column :users, :encrypted_otp_secret, :string + add_column :users, :encrypted_otp_secret_iv, :string + add_column :users, :encrypted_otp_secret_salt, :string + add_column :users, :otp_required_for_login, :boolean + end +end From ba7e2fd946ba94a9c0b3b18c3f7fc91f63fc652a Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 27 Mar 2015 17:01:24 -0700 Subject: [PATCH 090/158] Create Two-factor authentication resource for user --- .../profiles/two_factor_auths_controller.rb | 22 +++++++++++++++++++ app/views/profiles/accounts/show.html.haml | 7 ++++++ .../profiles/two_factor_auths/new.html.haml | 18 +++++++++++++++ config/routes.rb | 1 + 4 files changed, 48 insertions(+) create mode 100644 app/controllers/profiles/two_factor_auths_controller.rb create mode 100644 app/views/profiles/two_factor_auths/new.html.haml diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb new file mode 100644 index 0000000000..92ba842fac --- /dev/null +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -0,0 +1,22 @@ +class Profiles::TwoFactorAuthsController < ApplicationController + def new + issuer = "GitLab | #{current_user.email}" + uri = current_user.otp_provisioning_uri(current_user.email, issuer: issuer) + @qr_code = RQRCode::render_qrcode(uri, :svg, level: :l, unit: 2) + end + + def create + current_user.otp_required_for_login = true + current_user.otp_secret = User.generate_otp_secret + current_user.save! + + redirect_to profile_account_path + end + + def destroy + current_user.otp_required_for_login = false + current_user.save! + + redirect_to profile_account_path + end +end diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index 1c3a3d68ac..bbcd3baf61 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -26,6 +26,13 @@ %span You don`t have one yet. Click generate to fix it. = f.submit 'Generate', class: "btn success btn-build-token" + %fieldset + %legend Two-Factor Authentication + %p + Keep your account secure by enabling two-factor authentication. + Each time you log in, you’ll be required to provide your password plus a randomly generated access code. + %div + = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' - if show_profile_social_tab? %fieldset diff --git a/app/views/profiles/two_factor_auths/new.html.haml b/app/views/profiles/two_factor_auths/new.html.haml new file mode 100644 index 0000000000..77329de2e0 --- /dev/null +++ b/app/views/profiles/two_factor_auths/new.html.haml @@ -0,0 +1,18 @@ +%h2.page-title Two-Factor Authentication (TFA) +%p + Download the Google Authenticator application from App Store for iOS or + Google Play for Android and scan this code. + +%hr + += form_tag new_profile_two_factor_auth_path, method: :post, class: 'form-horizontal' do |f| + .form-group + .col-sm-2 + .col-sm-10 + = raw @qr_code + .form-group + = label_tag :pin_code, nil, class: "control-label" + .col-sm-10 + = text_field_tag :pin_code, nil, class: "form-control", required: true + .form-actions + = submit_tag 'Submit', class: 'btn btn-success' diff --git a/config/routes.rb b/config/routes.rb index 4b38dede7b..a76ababb3d 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -226,6 +226,7 @@ Gitlab::Application.routes.draw do resources :keys resources :emails, only: [:index, :create, :destroy] resource :avatar, only: [:destroy] + resource :two_factor_auth, only: [:new, :create, :destroy] end end From cde474a49f0ff44350d813aba83b6880df960f15 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 27 Mar 2015 17:53:08 -0700 Subject: [PATCH 091/158] Make 2 factor authentication work --- .../profiles/two_factor_auths_controller.rb | 31 ++++++++++++++----- app/views/profiles/accounts/show.html.haml | 10 +++++- .../profiles/two_factor_auths/new.html.haml | 5 ++- 3 files changed, 37 insertions(+), 9 deletions(-) diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb index 92ba842fac..ac14d5ca75 100644 --- a/app/controllers/profiles/two_factor_auths_controller.rb +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -1,16 +1,25 @@ class Profiles::TwoFactorAuthsController < ApplicationController def new - issuer = "GitLab | #{current_user.email}" - uri = current_user.otp_provisioning_uri(current_user.email, issuer: issuer) - @qr_code = RQRCode::render_qrcode(uri, :svg, level: :l, unit: 2) + unless current_user.otp_secret + current_user.otp_secret = User.generate_otp_secret + current_user.save! + end + + @qr_code = build_qr_code end def create - current_user.otp_required_for_login = true - current_user.otp_secret = User.generate_otp_secret - current_user.save! + if current_user.valid_otp?(params[:pin_code]) + current_user.otp_required_for_login = true + #current_user.otp_secret = User.generate_otp_secret + current_user.save! - redirect_to profile_account_path + redirect_to profile_account_path + else + @error = 'Invalid pin code' + @qr_code = build_qr_code + render 'new' + end end def destroy @@ -19,4 +28,12 @@ class Profiles::TwoFactorAuthsController < ApplicationController redirect_to profile_account_path end + + private + + def build_qr_code + issuer = "GitLab | #{current_user.email}" + uri = current_user.otp_provisioning_uri(current_user.email, issuer: issuer) + RQRCode::render_qrcode(uri, :svg, level: :m, unit: 3) + end end diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index bbcd3baf61..19b0c5bcb4 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -30,9 +30,17 @@ %legend Two-Factor Authentication %p Keep your account secure by enabling two-factor authentication. + %br Each time you log in, you’ll be required to provide your password plus a randomly generated access code. %div - = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' + - if current_user.otp_required_for_login + %strong.text-success + %i.fa.fa-check + 2-Factor Authentication enabled + .pull-right + = link_to "Disable 2-Factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm' + - else + = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' - if show_profile_social_tab? %fieldset diff --git a/app/views/profiles/two_factor_auths/new.html.haml b/app/views/profiles/two_factor_auths/new.html.haml index 77329de2e0..8332fc6b8b 100644 --- a/app/views/profiles/two_factor_auths/new.html.haml +++ b/app/views/profiles/two_factor_auths/new.html.haml @@ -5,7 +5,10 @@ %hr -= form_tag new_profile_two_factor_auth_path, method: :post, class: 'form-horizontal' do |f| += form_tag profile_two_factor_auth_path, method: :post, class: 'form-horizontal' do |f| + - if @error + .alert.alert-danger + = @error .form-group .col-sm-2 .col-sm-10 From 50a2a229e7b8b789a199bd0cf84ce76d25201198 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 27 Mar 2015 17:54:44 -0700 Subject: [PATCH 092/158] Fix rubocop complain --- app/controllers/profiles/two_factor_auths_controller.rb | 1 - app/models/user.rb | 2 +- config/initializers/devise.rb | 2 +- 3 files changed, 2 insertions(+), 3 deletions(-) diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb index ac14d5ca75..9b4070a76f 100644 --- a/app/controllers/profiles/two_factor_auths_controller.rb +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -11,7 +11,6 @@ class Profiles::TwoFactorAuthsController < ApplicationController def create if current_user.valid_otp?(params[:pin_code]) current_user.otp_required_for_login = true - #current_user.otp_secret = User.generate_otp_secret current_user.save! redirect_to profile_account_path diff --git a/app/models/user.rb b/app/models/user.rb index b9e2890018..befbcbf1a1 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -58,7 +58,7 @@ require 'file_size_validator' class User < ActiveRecord::Base devise :two_factor_authenticatable, - :otp_secret_encryption_key => File.read(Rails.root.join('.secret')).chomp + otp_secret_encryption_key: File.read(Rails.root.join('.secret')).chomp include Sortable include Gitlab::ConfigHelper diff --git a/config/initializers/devise.rb b/config/initializers/devise.rb index 956bb048b2..c003a7102a 100644 --- a/config/initializers/devise.rb +++ b/config/initializers/devise.rb @@ -2,7 +2,7 @@ # four configuration values can also be set straight in your models. Devise.setup do |config| config.warden do |manager| - manager.default_strategies(:scope => :user).unshift :two_factor_authenticatable + manager.default_strategies(scope: :user).unshift :two_factor_authenticatable end # ==> Mailer Configuration From de9e1c3bad18e4ca00cfdced75e5cc4c42905761 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Tue, 31 Mar 2015 04:19:01 +0300 Subject: [PATCH 093/158] Turn 2-factor authentication into 2 steps process. Disabled 2fa UI for ldap users since it is not supported --- app/controllers/application_controller.rb | 2 +- app/controllers/sessions_controller.rb | 24 ++++++++++++++ app/views/devise/sessions/_new_base.html.haml | 4 +-- .../devise/sessions/two_factor.html.haml | 16 ++++++++++ app/views/profiles/accounts/show.html.haml | 31 ++++++++++--------- 5 files changed, 59 insertions(+), 18 deletions(-) create mode 100644 app/views/devise/sessions/two_factor.html.haml diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index eee10d6c22..247f5cc32d 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -252,7 +252,7 @@ class ApplicationController < ActionController::Base end def configure_permitted_parameters - devise_parameter_sanitizer.sanitize(:sign_in) { |u| u.permit(:username, :email, :password, :login, :remember_me) } + devise_parameter_sanitizer.sanitize(:sign_in) { |u| u.permit(:username, :email, :password, :login, :remember_me, :otp_attempt) } end def hexdigest(string) diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index 3f11d7afe6..68cd02b2d7 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -1,4 +1,6 @@ class SessionsController < Devise::SessionsController + prepend_before_filter :two_factor_enabled?, only: :create + def new redirect_path = if request.referer.present? && (params['redirect_to_referer'] == 'yes') @@ -34,4 +36,26 @@ class SessionsController < Devise::SessionsController end end end + + private + + def two_factor_enabled? + user_params = params[:user] + @user = User.by_login(user_params[:login]) + + if user_params[:otp_attempt].present? + unless @user.valid_otp?(user_params[:otp_attempt]) + @error = 'Invalid two-factor code' + render :two_factor and return + end + else + if @user && @user.valid_password?(params[:user][:password]) + self.resource = @user + + if resource.otp_required_for_login + render :two_factor and return + end + end + end + end end diff --git a/app/views/devise/sessions/_new_base.html.haml b/app/views/devise/sessions/_new_base.html.haml index 4ecb74fb56..dcb989e8d8 100644 --- a/app/views/devise/sessions/_new_base.html.haml +++ b/app/views/devise/sessions/_new_base.html.haml @@ -1,7 +1,7 @@ = form_for(resource, as: resource_name, url: session_path(resource_name)) do |f| = f.text_field :login, class: "form-control top", placeholder: "Username or Email", autofocus: "autofocus" - = f.password_field :password, class: "form-control middle", placeholder: "Password" - = f.text_field :otp_attempt, class: 'form-control bottom', placeholder: 'Two-factor authentication token' + = f.password_field :password, class: "form-control bottom", placeholder: "Password" + = f.hidden_field :otp_attempt, value: '' - if devise_mapping.rememberable? .remember-me.checkbox %label{for: "user_remember_me"} diff --git a/app/views/devise/sessions/two_factor.html.haml b/app/views/devise/sessions/two_factor.html.haml new file mode 100644 index 0000000000..3a19e926b6 --- /dev/null +++ b/app/views/devise/sessions/two_factor.html.haml @@ -0,0 +1,16 @@ +%div + .login-box + .login-heading + %h3 Two-Factor Authentication + .login-body + = form_for(resource, as: resource_name, url: session_path(resource_name), method: :post) do |f| + - if @error + .alert.alert-danger + = @error + .hide + = f.text_field :login, class: "form-control top", placeholder: "Username or Email", autofocus: "autofocus" + = f.password_field :password, class: "form-control bottom", placeholder: "Password" + = f.text_field :otp_attempt, class: 'form-control', + placeholder: 'Two-factor authentication token', required: true, autofocus: true + .prepend-top-20 + = f.submit "Verify code", class: "btn btn-save" diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index 19b0c5bcb4..dcce29a81f 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -26,21 +26,22 @@ %span You don`t have one yet. Click generate to fix it. = f.submit 'Generate', class: "btn success btn-build-token" - %fieldset - %legend Two-Factor Authentication - %p - Keep your account secure by enabling two-factor authentication. - %br - Each time you log in, you’ll be required to provide your password plus a randomly generated access code. - %div - - if current_user.otp_required_for_login - %strong.text-success - %i.fa.fa-check - 2-Factor Authentication enabled - .pull-right - = link_to "Disable 2-Factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm' - - else - = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' + - unless current_user.ldap_user? + %fieldset + %legend Two-Factor Authentication + %p + Keep your account secure by enabling two-factor authentication. + %br + Each time you log in, you’ll be required to provide your password plus a randomly generated access code. + %div + - if current_user.otp_required_for_login + %strong.text-success + %i.fa.fa-check + 2-Factor Authentication enabled + .pull-right + = link_to "Disable 2-Factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm' + - else + = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' - if show_profile_social_tab? %fieldset From b66be0a2b3351dd10d96e3d0d0576f6d1444f342 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Tue, 31 Mar 2015 05:34:50 +0300 Subject: [PATCH 094/158] Use non-broken version of attr_encrypted --- Gemfile | 1 + Gemfile.lock | 11 +++++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Gemfile b/Gemfile index 65a641c602..ee6072b2da 100644 --- a/Gemfile +++ b/Gemfile @@ -37,6 +37,7 @@ gem "rack-oauth2", "~> 1.0.5" # Two-factor authentication gem 'devise-two-factor' gem 'rqrcode-rails3' +gem 'attr_encrypted', git: 'https://github.com/attr-encrypted/attr_encrypted.git', ref: '94d901df2ccbc579b981091d53dd641f9bed4c1d' # Browser detection gem "browser" diff --git a/Gemfile.lock b/Gemfile.lock index 35faa9895b..672bd9950e 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,3 +1,11 @@ +GIT + remote: https://github.com/attr-encrypted/attr_encrypted.git + revision: 94d901df2ccbc579b981091d53dd641f9bed4c1d + ref: 94d901df2ccbc579b981091d53dd641f9bed4c1d + specs: + attr_encrypted (1.3.3) + encryptor (>= 1.3.0) + GEM remote: https://rubygems.org/ specs: @@ -46,8 +54,6 @@ GEM ast (2.0.0) astrolabe (1.3.0) parser (>= 2.2.0.pre.3, < 3.0) - attr_encrypted (1.3.3) - encryptor (>= 1.3.0) attr_required (1.0.0) autoprefixer-rails (5.1.11) execjs @@ -684,6 +690,7 @@ DEPENDENCIES annotate (~> 2.6.0.beta2) asana (~> 0.0.6) asciidoctor (= 0.1.4) + attr_encrypted! awesome_print better_errors binding_of_caller From 802fcd051fcbc9be99230bddb13c4a7cb8067741 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Tue, 31 Mar 2015 22:42:17 +0300 Subject: [PATCH 095/158] Add support for backup codes --- .../profiles/two_factor_auths_controller.rb | 6 ++++ app/controllers/sessions_controller.rb | 3 +- app/views/profiles/accounts/show.html.haml | 35 ++++++++++++------- config/initializers/devise.rb | 1 + config/routes.rb | 6 +++- ...d_devise_two_factor_backupable_to_users.rb | 5 +++ 6 files changed, 42 insertions(+), 14 deletions(-) create mode 100644 db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb index 9b4070a76f..2841a07efb 100644 --- a/app/controllers/profiles/two_factor_auths_controller.rb +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -21,6 +21,12 @@ class Profiles::TwoFactorAuthsController < ApplicationController end end + def codes + codes = current_user.generate_otp_backup_codes! + current_user.save! + send_data codes.join("\n"), filename: 'gitlab_recovery_codes.txt' + end + def destroy current_user.otp_required_for_login = false current_user.save! diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index 68cd02b2d7..cc9d30d64d 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -44,7 +44,8 @@ class SessionsController < Devise::SessionsController @user = User.by_login(user_params[:login]) if user_params[:otp_attempt].present? - unless @user.valid_otp?(user_params[:otp_attempt]) + unless @user.valid_otp?(user_params[:otp_attempt]) || + @user.recovery_code?(user_params[:otp_attempt]) @error = 'Invalid two-factor code' render :two_factor and return end diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index dcce29a81f..1e024c45f4 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -28,20 +28,31 @@ - unless current_user.ldap_user? %fieldset - %legend Two-Factor Authentication - %p - Keep your account secure by enabling two-factor authentication. - %br - Each time you log in, you’ll be required to provide your password plus a randomly generated access code. - %div - - if current_user.otp_required_for_login - %strong.text-success - %i.fa.fa-check - 2-Factor Authentication enabled + - if current_user.otp_required_for_login + %legend.text-success + %i.fa.fa-check + Two-Factor Authentication enabled + %div .pull-right = link_to "Disable 2-Factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm' - - else - = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' + %p.slead + %i.fa.fa-warning + Please + %strong #{link_to "download recovery codes", codes_profile_two_factor_auth_path} + so you can access your account if you lose your phone. + %br + %i.fa.fa-warning + Every time you download recovery codes - we generate the new codes. Previously downloaded codes won't work anymore. + + - else + %legend Two-Factor Authentication + %div + %p + Keep your account secure by enabling two-factor authentication. + %br + Each time you log in, you’ll be required to provide your password plus a randomly generated access code. + %div + = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' - if show_profile_social_tab? %fieldset diff --git a/config/initializers/devise.rb b/config/initializers/devise.rb index c003a7102a..091548348b 100644 --- a/config/initializers/devise.rb +++ b/config/initializers/devise.rb @@ -3,6 +3,7 @@ Devise.setup do |config| config.warden do |manager| manager.default_strategies(scope: :user).unshift :two_factor_authenticatable + manager.default_strategies(scope: :user).unshift :two_factor_backupable end # ==> Mailer Configuration diff --git a/config/routes.rb b/config/routes.rb index a76ababb3d..bcd68ad6ae 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -226,7 +226,11 @@ Gitlab::Application.routes.draw do resources :keys resources :emails, only: [:index, :create, :destroy] resource :avatar, only: [:destroy] - resource :two_factor_auth, only: [:new, :create, :destroy] + resource :two_factor_auth, only: [:new, :create, :destroy] do + member do + get :codes + end + end end end diff --git a/db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb b/db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb new file mode 100644 index 0000000000..2feb49f43f --- /dev/null +++ b/db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb @@ -0,0 +1,5 @@ +class AddDeviseTwoFactorBackupableToUsers < ActiveRecord::Migration + def change + add_column :users, :otp_backup_codes, :string, array: true + end +end From 8ae712ae28eda669987c48a755fc9f94ac48b5e7 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Thu, 2 Apr 2015 11:57:36 -0700 Subject: [PATCH 096/158] Render 2fa recovery codes instead of downloading it --- .../profiles/two_factor_auths_controller.rb | 6 +++--- app/views/profiles/accounts/show.html.haml | 13 +++++-------- .../profiles/two_factor_auths/_codes.html.haml | 12 ++++++++++++ app/views/profiles/two_factor_auths/codes.html.haml | 3 +++ .../profiles/two_factor_auths/create.html.haml | 4 ++++ config/routes.rb | 2 +- 6 files changed, 28 insertions(+), 12 deletions(-) create mode 100644 app/views/profiles/two_factor_auths/_codes.html.haml create mode 100644 app/views/profiles/two_factor_auths/codes.html.haml create mode 100644 app/views/profiles/two_factor_auths/create.html.haml diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb index 2841a07efb..5c51706c3d 100644 --- a/app/controllers/profiles/two_factor_auths_controller.rb +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -11,9 +11,10 @@ class Profiles::TwoFactorAuthsController < ApplicationController def create if current_user.valid_otp?(params[:pin_code]) current_user.otp_required_for_login = true + @codes = current_user.generate_otp_backup_codes! current_user.save! - redirect_to profile_account_path + render 'create' else @error = 'Invalid pin code' @qr_code = build_qr_code @@ -22,9 +23,8 @@ class Profiles::TwoFactorAuthsController < ApplicationController end def codes - codes = current_user.generate_otp_backup_codes! + @codes = current_user.generate_otp_backup_codes! current_user.save! - send_data codes.join("\n"), filename: 'gitlab_recovery_codes.txt' end def destroy diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index 1e024c45f4..e8b4943833 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -35,14 +35,11 @@ %div .pull-right = link_to "Disable 2-Factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm' - %p.slead - %i.fa.fa-warning - Please - %strong #{link_to "download recovery codes", codes_profile_two_factor_auth_path} - so you can access your account if you lose your phone. - %br - %i.fa.fa-warning - Every time you download recovery codes - we generate the new codes. Previously downloaded codes won't work anymore. + %p + If you lost your recovery codes - you can + %strong + = link_to "generate new one", codes_profile_two_factor_auth_path, method: :post, + data: { confirm: 'After we generate new recovery codes - old codes will not be valid any more. Are you sure?' } - else %legend Two-Factor Authentication diff --git a/app/views/profiles/two_factor_auths/_codes.html.haml b/app/views/profiles/two_factor_auths/_codes.html.haml new file mode 100644 index 0000000000..dd043f3c5b --- /dev/null +++ b/app/views/profiles/two_factor_auths/_codes.html.haml @@ -0,0 +1,12 @@ +%p.slead + Please save this recovery codes so you can access your account if you lose your phone. + +.codes.well + %ul + - @codes.each do |code| + %li + %span.monospace + = code + += link_to profile_account_path, class: 'btn btn-success' do + I saved the codes diff --git a/app/views/profiles/two_factor_auths/codes.html.haml b/app/views/profiles/two_factor_auths/codes.html.haml new file mode 100644 index 0000000000..d826ba2528 --- /dev/null +++ b/app/views/profiles/two_factor_auths/codes.html.haml @@ -0,0 +1,3 @@ +%h3.page-title Two-Factor Authentication Recovery codes +%hr += render 'codes' diff --git a/app/views/profiles/two_factor_auths/create.html.haml b/app/views/profiles/two_factor_auths/create.html.haml new file mode 100644 index 0000000000..27956bd763 --- /dev/null +++ b/app/views/profiles/two_factor_auths/create.html.haml @@ -0,0 +1,4 @@ +.alert.alert-success + Congratulations! You have enabled Two-Factor Authentication! + += render 'codes' diff --git a/config/routes.rb b/config/routes.rb index bcd68ad6ae..bf2cb6421c 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -228,7 +228,7 @@ Gitlab::Application.routes.draw do resource :avatar, only: [:destroy] resource :two_factor_auth, only: [:new, :create, :destroy] do member do - get :codes + post :codes end end end From 35b62facac0737d1173f93a97b8f49876ec8519c Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 14:09:45 -0400 Subject: [PATCH 097/158] Update copy for recovery codes --- app/views/profiles/two_factor_auths/_codes.html.haml | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/app/views/profiles/two_factor_auths/_codes.html.haml b/app/views/profiles/two_factor_auths/_codes.html.haml index dd043f3c5b..1b1395eaa1 100644 --- a/app/views/profiles/two_factor_auths/_codes.html.haml +++ b/app/views/profiles/two_factor_auths/_codes.html.haml @@ -1,12 +1,11 @@ %p.slead - Please save this recovery codes so you can access your account if you lose your phone. + Should you ever lose your phone, each of these recovery codes can be used one + time each to regain access to your account. Please save them in a safe place. .codes.well %ul - @codes.each do |code| %li - %span.monospace - = code + %span.monospace= code -= link_to profile_account_path, class: 'btn btn-success' do - I saved the codes += link_to 'Proceed', profile_account_path, class: 'btn btn-success' From bd680999f9bf33ba6067a42749139c1d0b52014e Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 14:11:00 -0400 Subject: [PATCH 098/158] Be consistent with what we call the 2FA feature "Two-factor" vs. "2-Factor" --- app/views/devise/sessions/two_factor.html.haml | 2 +- app/views/profiles/accounts/show.html.haml | 18 ++++++++++-------- .../profiles/two_factor_auths/codes.html.haml | 2 +- .../profiles/two_factor_auths/create.html.haml | 2 +- 4 files changed, 13 insertions(+), 11 deletions(-) diff --git a/app/views/devise/sessions/two_factor.html.haml b/app/views/devise/sessions/two_factor.html.haml index 3a19e926b6..cfae81367b 100644 --- a/app/views/devise/sessions/two_factor.html.haml +++ b/app/views/devise/sessions/two_factor.html.haml @@ -1,7 +1,7 @@ %div .login-box .login-heading - %h3 Two-Factor Authentication + %h3 Two-factor Authentication .login-body = form_for(resource, as: resource_name, url: session_path(resource_name), method: :post) do |f| - if @error diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index e8b4943833..c8158f8f17 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -30,26 +30,28 @@ %fieldset - if current_user.otp_required_for_login %legend.text-success - %i.fa.fa-check - Two-Factor Authentication enabled + = icon('check') + Two-factor Authentication enabled %div .pull-right - = link_to "Disable 2-Factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm' + = link_to "Disable Two-factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm', + data: { confirm: 'Are you sure?' } %p - If you lost your recovery codes - you can + If you lose your recovery codes, you can %strong - = link_to "generate new one", codes_profile_two_factor_auth_path, method: :post, - data: { confirm: 'After we generate new recovery codes - old codes will not be valid any more. Are you sure?' } + = succeed '.' do + = link_to "generate new ones", codes_profile_two_factor_auth_path, method: :post, + data: { confirm: 'This will invalidate the old codes. Are you sure?' } - else - %legend Two-Factor Authentication + %legend Two-factor Authentication %div %p Keep your account secure by enabling two-factor authentication. %br Each time you log in, you’ll be required to provide your password plus a randomly generated access code. %div - = link_to "Enable 2-Factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' + = link_to "Enable Two-factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' - if show_profile_social_tab? %fieldset diff --git a/app/views/profiles/two_factor_auths/codes.html.haml b/app/views/profiles/two_factor_auths/codes.html.haml index d826ba2528..4950f7db12 100644 --- a/app/views/profiles/two_factor_auths/codes.html.haml +++ b/app/views/profiles/two_factor_auths/codes.html.haml @@ -1,3 +1,3 @@ -%h3.page-title Two-Factor Authentication Recovery codes +%h3.page-title Two-factor Authentication Recovery codes %hr = render 'codes' diff --git a/app/views/profiles/two_factor_auths/create.html.haml b/app/views/profiles/two_factor_auths/create.html.haml index 27956bd763..965466db36 100644 --- a/app/views/profiles/two_factor_auths/create.html.haml +++ b/app/views/profiles/two_factor_auths/create.html.haml @@ -1,4 +1,4 @@ .alert.alert-success - Congratulations! You have enabled Two-Factor Authentication! + Congratulations! You have enabled Two-factor Authentication! = render 'codes' From c84f1240d4dd72b53361d3cc0bccf5c7d789c8ec Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 14:18:20 -0400 Subject: [PATCH 099/158] prepend_before_filter -> prepend_before_action --- app/controllers/sessions_controller.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index cc9d30d64d..e72b003f86 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -1,5 +1,5 @@ class SessionsController < Devise::SessionsController - prepend_before_filter :two_factor_enabled?, only: :create + prepend_before_action :two_factor_enabled?, only: :create def new redirect_path = From 125ee5262a65db71fc8ba2d7a51885039b5ccc1f Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 14:18:32 -0400 Subject: [PATCH 100/158] Don't use hard-coded sign_in path --- app/controllers/sessions_controller.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index e72b003f86..8bd8fbb692 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -16,7 +16,7 @@ class SessionsController < Devise::SessionsController # Prevent a 'you are already signed in' message directly after signing: # we should never redirect to '/users/sign_in' after signing in successfully. - unless redirect_path == '/users/sign_in' + unless redirect_path == new_user_session_path store_location_for(:redirect, redirect_path) end From e512f770cd22cf82933118fb30625526dfb68c62 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 4 May 2015 16:02:33 -0400 Subject: [PATCH 101/158] Fix Devise parameter sanitizer for otp_attempt --- app/controllers/application_controller.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index 247f5cc32d..8ce881c741 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -252,7 +252,7 @@ class ApplicationController < ActionController::Base end def configure_permitted_parameters - devise_parameter_sanitizer.sanitize(:sign_in) { |u| u.permit(:username, :email, :password, :login, :remember_me, :otp_attempt) } + devise_parameter_sanitizer.for(:sign_in) { |u| u.permit(:username, :email, :password, :login, :remember_me, :otp_attempt) } end def hexdigest(string) From 6fa294292058dd5ab9bd089c446b3017aa85b868 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Tue, 5 May 2015 22:12:35 -0400 Subject: [PATCH 102/158] Update login views for two-factor auth --- app/views/devise/sessions/_new_base.html.haml | 1 - app/views/devise/sessions/two_factor.html.haml | 7 ++----- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/app/views/devise/sessions/_new_base.html.haml b/app/views/devise/sessions/_new_base.html.haml index dcb989e8d8..54a3972677 100644 --- a/app/views/devise/sessions/_new_base.html.haml +++ b/app/views/devise/sessions/_new_base.html.haml @@ -1,7 +1,6 @@ = form_for(resource, as: resource_name, url: session_path(resource_name)) do |f| = f.text_field :login, class: "form-control top", placeholder: "Username or Email", autofocus: "autofocus" = f.password_field :password, class: "form-control bottom", placeholder: "Password" - = f.hidden_field :otp_attempt, value: '' - if devise_mapping.rememberable? .remember-me.checkbox %label{for: "user_remember_me"} diff --git a/app/views/devise/sessions/two_factor.html.haml b/app/views/devise/sessions/two_factor.html.haml index cfae81367b..32ba90fd7c 100644 --- a/app/views/devise/sessions/two_factor.html.haml +++ b/app/views/devise/sessions/two_factor.html.haml @@ -7,10 +7,7 @@ - if @error .alert.alert-danger = @error - .hide - = f.text_field :login, class: "form-control top", placeholder: "Username or Email", autofocus: "autofocus" - = f.password_field :password, class: "form-control bottom", placeholder: "Password" - = f.text_field :otp_attempt, class: 'form-control', - placeholder: 'Two-factor authentication token', required: true, autofocus: true + = f.hidden_field :login + = f.text_field :otp_attempt, class: 'form-control', placeholder: 'Two-factor authentication code', required: true, autofocus: true .prepend-top-20 = f.submit "Verify code", class: "btn btn-save" From 66bc758e2b9ba736035d036b25b17bdbb83c39b3 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Tue, 5 May 2015 22:16:09 -0400 Subject: [PATCH 103/158] Update User model for two-factor auth --- app/models/user.rb | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/app/models/user.rb b/app/models/user.rb index befbcbf1a1..7b1f6fae3b 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -73,8 +73,11 @@ class User < ActiveRecord::Base default_value_for :hide_no_password, false default_value_for :theme_id, gitlab_config.default_theme - devise :lockable, :async, - :recoverable, :rememberable, :trackable, :validatable, :omniauthable, :confirmable, :registerable + devise :two_factor_authenticatable, + otp_secret_encryption_key: File.read(Rails.root.join('.secret')).chomp + devise :two_factor_backupable + devise :lockable, :async, :recoverable, :rememberable, :trackable, + :validatable, :omniauthable, :confirmable, :registerable attr_accessor :force_random_password @@ -663,4 +666,9 @@ class User < ActiveRecord::Base true end + + # Used to populate the hidden form field during Two-factor authentication + def login + username || email + end end From 5520397f048243ad7fd8a39faf9d9f7b5a97c5bc Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Tue, 5 May 2015 22:16:45 -0400 Subject: [PATCH 104/158] Make two-factor login work and add a feature spec --- app/controllers/sessions_controller.rb | 31 +++++++--- spec/features/login_spec.rb | 82 ++++++++++++++++++++++++++ 2 files changed, 106 insertions(+), 7 deletions(-) create mode 100644 spec/features/login_spec.rb diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index 8bd8fbb692..21505442e3 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -1,5 +1,5 @@ class SessionsController < Devise::SessionsController - prepend_before_action :two_factor_enabled?, only: :create + prepend_before_action :authenticate_with_two_factor, only: :create def new redirect_path = @@ -29,6 +29,9 @@ class SessionsController < Devise::SessionsController def create super do |resource| + # Remove any lingering user data from login + session.delete(:user) + # User has successfully signed in, so clear any unused reset tokens if resource.reset_password_token.present? resource.update_attributes(reset_password_token: nil, @@ -39,24 +42,38 @@ class SessionsController < Devise::SessionsController private - def two_factor_enabled? - user_params = params[:user] + def user_params + params.require(:user).permit(:login, :password, :remember_me, :otp_attempt) + end + + def authenticate_with_two_factor @user = User.by_login(user_params[:login]) - if user_params[:otp_attempt].present? - unless @user.valid_otp?(user_params[:otp_attempt]) || - @user.recovery_code?(user_params[:otp_attempt]) + if user_params[:otp_attempt].present? && session[:user] + if valid_otp_attempt? + # Insert the saved params from the session into the request parameters + # so they're available to Devise::Strategies::DatabaseAuthenticatable + request.params[:user].merge!(session[:user]) + else @error = 'Invalid two-factor code' render :two_factor and return end else - if @user && @user.valid_password?(params[:user][:password]) + if @user && @user.valid_password?(user_params[:password]) self.resource = @user if resource.otp_required_for_login + # Login is valid, save the values to the session so we can prompt the + # user for a one-time password. + session[:user] = user_params render :two_factor and return end end end end + + def valid_otp_attempt? + @user.valid_otp?(user_params[:otp_attempt]) || + @user.invalidate_otp_backup_code!(user_params[:otp_attempt]) + end end diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb new file mode 100644 index 0000000000..5ffaae5476 --- /dev/null +++ b/spec/features/login_spec.rb @@ -0,0 +1,82 @@ +require 'spec_helper' + +feature 'Login' do + let(:user) { create(:user) } + + context 'with two-factor authentication' do + before do + user.otp_required_for_login = true + user.otp_secret = User.generate_otp_secret + user.save! + end + + context 'with valid username/password' do + before do + login_with(user) + expect(page).to have_content('Two-factor Authentication') + end + + def enter_code(code) + fill_in 'Two-factor authentication code', with: code + click_button 'Verify code' + end + + context 'using one-time code' do + it 'allows login with valid code' do + enter_code(user.current_otp) + expect(current_path).to eq root_path + end + + it 'blocks login with invalid code' do + enter_code('foo') + expect(page).to have_content('Invalid two-factor code') + end + end + + context 'using backup code' do + let(:codes) { user.generate_otp_backup_codes! } + + before do + expect(codes.size).to eq 5 + + # Because `generate_otp_backup_codes!` doesn't actually do this... + user.save + end + + context 'with valid code' do + it 'allows login' do + enter_code(codes.sample) + expect(current_path).to eq root_path + end + + it 'invalidates the used code' do + # FIXME (rspeicher): Broken library is broken + expect { enter_code(codes.sample) }.to change { user.otp_backup_codes.size }.by(-1) + end + end + + context 'with invalid code' do + it 'blocks login' do + # FIXME (rspeicher): Broken library is broken + code = codes.sample + expect(user.invalidate_otp_backup_code!(code)).to eq true + expect(user.otp_backup_codes.size).to eq 4 # Passes + user.save! + user.reload + expect(user.otp_backup_codes.size).to eq 4 # Fails... WAT?! + + enter_code(code) + expect(page).to have_content('Invalid two-factor code') + end + end + end + end + end + + context 'without two-factor authentication' do + it 'allows basic login' do + login_with(user) + expect(current_path).to eq root_path + end + end +end From c891ef93371c3eeeb56b246d8d02a9e2dd5d350e Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 13:35:37 -0400 Subject: [PATCH 105/158] Add page titles for two_factor_auths views --- app/views/profiles/two_factor_auths/codes.html.haml | 2 ++ app/views/profiles/two_factor_auths/create.html.haml | 2 ++ app/views/profiles/two_factor_auths/new.html.haml | 2 ++ 3 files changed, 6 insertions(+) diff --git a/app/views/profiles/two_factor_auths/codes.html.haml b/app/views/profiles/two_factor_auths/codes.html.haml index 4950f7db12..addf356697 100644 --- a/app/views/profiles/two_factor_auths/codes.html.haml +++ b/app/views/profiles/two_factor_auths/codes.html.haml @@ -1,3 +1,5 @@ +- page_title 'Recovery Codes', 'Two-factor Authentication' + %h3.page-title Two-factor Authentication Recovery codes %hr = render 'codes' diff --git a/app/views/profiles/two_factor_auths/create.html.haml b/app/views/profiles/two_factor_auths/create.html.haml index 965466db36..e330aadac1 100644 --- a/app/views/profiles/two_factor_auths/create.html.haml +++ b/app/views/profiles/two_factor_auths/create.html.haml @@ -1,3 +1,5 @@ +- page_title 'Two-factor Authentication', 'Account' + .alert.alert-success Congratulations! You have enabled Two-factor Authentication! diff --git a/app/views/profiles/two_factor_auths/new.html.haml b/app/views/profiles/two_factor_auths/new.html.haml index 8332fc6b8b..5d62aa6339 100644 --- a/app/views/profiles/two_factor_auths/new.html.haml +++ b/app/views/profiles/two_factor_auths/new.html.haml @@ -1,3 +1,5 @@ +- page_title 'Two-factor Authentication', 'Account' + %h2.page-title Two-Factor Authentication (TFA) %p Download the Google Authenticator application from App Store for iOS or From 6369d23d581ad36e7507d355a69237b90a912697 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 6 May 2015 13:35:55 -0400 Subject: [PATCH 106/158] Fix nav and layout for TwoFactorAuthsController --- app/controllers/profiles/two_factor_auths_controller.rb | 2 +- app/views/layouts/nav/_profile.html.haml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb index 5c51706c3d..60f8ec5cf3 100644 --- a/app/controllers/profiles/two_factor_auths_controller.rb +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -1,4 +1,4 @@ -class Profiles::TwoFactorAuthsController < ApplicationController +class Profiles::TwoFactorAuthsController < Profiles::ApplicationController def new unless current_user.otp_secret current_user.otp_secret = User.generate_otp_secret diff --git a/app/views/layouts/nav/_profile.html.haml b/app/views/layouts/nav/_profile.html.haml index 31d8ed3ed8..ac37fd4c1c 100644 --- a/app/views/layouts/nav/_profile.html.haml +++ b/app/views/layouts/nav/_profile.html.haml @@ -4,7 +4,7 @@ = icon('user fw') %span Profile - = nav_link(controller: :accounts) do + = nav_link(controller: [:accounts, :two_factor_auths]) do = link_to profile_account_path, title: 'Account', data: {placement: 'right'} do = icon('gear fw') %span From 32971b0af4761fa9527c2fa0922a9b31eec5245f Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Fri, 8 May 2015 20:41:53 -0400 Subject: [PATCH 107/158] Refactor SessionsController Also adds test case for providing an invalid 2FA code and then a valid one without re-entering username and password. --- app/controllers/sessions_controller.rb | 49 ++++++++++--------- .../devise/sessions/two_factor.html.haml | 1 - spec/features/login_spec.rb | 8 +++ 3 files changed, 34 insertions(+), 24 deletions(-) diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index 21505442e3..e2a5c61257 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -29,10 +29,7 @@ class SessionsController < Devise::SessionsController def create super do |resource| - # Remove any lingering user data from login - session.delete(:user) - - # User has successfully signed in, so clear any unused reset tokens + # User has successfully signed in, so clear any unused reset token if resource.reset_password_token.present? resource.update_attributes(reset_password_token: nil, reset_password_sent_at: nil) @@ -46,34 +43,40 @@ class SessionsController < Devise::SessionsController params.require(:user).permit(:login, :password, :remember_me, :otp_attempt) end - def authenticate_with_two_factor - @user = User.by_login(user_params[:login]) + def find_user + if user_params[:login] + User.by_login(user_params[:login]) + elsif user_params[:otp_attempt] && session[:otp_user_id] + User.find(session[:otp_user_id]) + end + end - if user_params[:otp_attempt].present? && session[:user] - if valid_otp_attempt? - # Insert the saved params from the session into the request parameters - # so they're available to Devise::Strategies::DatabaseAuthenticatable - request.params[:user].merge!(session[:user]) + def authenticate_with_two_factor + user = self.resource = find_user + + return unless user && user.otp_required_for_login + + if user_params[:otp_attempt].present? && session[:otp_user_id] + if valid_otp_attempt?(user) + # Remove any lingering user data from login + session.delete(:otp_user_id) + + sign_in(user) else @error = 'Invalid two-factor code' render :two_factor and return end else - if @user && @user.valid_password?(user_params[:password]) - self.resource = @user - - if resource.otp_required_for_login - # Login is valid, save the values to the session so we can prompt the - # user for a one-time password. - session[:user] = user_params - render :two_factor and return - end + if user && user.valid_password?(user_params[:password]) + # Save the user's ID to session so we can ask for a one-time password + session[:otp_user_id] = user.id + render :two_factor and return end end end - def valid_otp_attempt? - @user.valid_otp?(user_params[:otp_attempt]) || - @user.invalidate_otp_backup_code!(user_params[:otp_attempt]) + def valid_otp_attempt?(user) + user.valid_otp?(user_params[:otp_attempt]) || + user.invalidate_otp_backup_code!(user_params[:otp_attempt]) end end diff --git a/app/views/devise/sessions/two_factor.html.haml b/app/views/devise/sessions/two_factor.html.haml index 32ba90fd7c..779928631a 100644 --- a/app/views/devise/sessions/two_factor.html.haml +++ b/app/views/devise/sessions/two_factor.html.haml @@ -7,7 +7,6 @@ - if @error .alert.alert-danger = @error - = f.hidden_field :login = f.text_field :otp_attempt, class: 'form-control', placeholder: 'Two-factor authentication code', required: true, autofocus: true .prepend-top-20 = f.submit "Verify code", class: "btn btn-save" diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb index 5ffaae5476..f1e24c5424 100644 --- a/spec/features/login_spec.rb +++ b/spec/features/login_spec.rb @@ -31,6 +31,14 @@ feature 'Login' do enter_code('foo') expect(page).to have_content('Invalid two-factor code') end + + it 'allows login with invalid code, then valid code' do + enter_code('foo') + expect(page).to have_content('Invalid two-factor code') + + enter_code(user.current_otp) + expect(current_path).to eq root_path + end end context 'using backup code' do From 4fca1fc5ab7a1fc8719483c78beed08f71144ea2 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Fri, 8 May 2015 20:45:50 -0400 Subject: [PATCH 108/158] Update copy for generating new recovery codes --- app/views/profiles/accounts/show.html.haml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index c8158f8f17..9f9a12c791 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -37,11 +37,11 @@ = link_to "Disable Two-factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm', data: { confirm: 'Are you sure?' } %p - If you lose your recovery codes, you can + If you lose your recovery codes you can %strong - = succeed '.' do - = link_to "generate new ones", codes_profile_two_factor_auth_path, method: :post, - data: { confirm: 'This will invalidate the old codes. Are you sure?' } + = succeed ',' do + = link_to "generate new ones", codes_profile_two_factor_auth_path, method: :post, data: { confirm: 'Are you sure?' } + invalidating all previous codes. - else %legend Two-factor Authentication From 8eb577ae9852d5304c28a0bc01d462b146427050 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Fri, 8 May 2015 20:55:45 -0400 Subject: [PATCH 109/158] Improve copy for enabling 2FA --- app/views/profiles/accounts/show.html.haml | 7 ++++--- app/views/profiles/two_factor_auths/new.html.haml | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index 9f9a12c791..87f827793a 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -47,9 +47,10 @@ %legend Two-factor Authentication %div %p - Keep your account secure by enabling two-factor authentication. - %br - Each time you log in, you’ll be required to provide your password plus a randomly generated access code. + Increase your account's security by enabling two-factor authentication (2FA). + %p + Each time you log in you’ll be required to provide your username and + password as usual, plus a randomly-generated code from your phone. %div = link_to "Enable Two-factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' diff --git a/app/views/profiles/two_factor_auths/new.html.haml b/app/views/profiles/two_factor_auths/new.html.haml index 5d62aa6339..f0a4a34187 100644 --- a/app/views/profiles/two_factor_auths/new.html.haml +++ b/app/views/profiles/two_factor_auths/new.html.haml @@ -1,6 +1,6 @@ - page_title 'Two-factor Authentication', 'Account' -%h2.page-title Two-Factor Authentication (TFA) +%h2.page-title Two-Factor Authentication (2FA) %p Download the Google Authenticator application from App Store for iOS or Google Play for Android and scan this code. From a4267033f2cc72e46ba5f1bfdfca296978df2117 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Fri, 8 May 2015 20:56:04 -0400 Subject: [PATCH 110/158] Add a hint on login form about using a recovery code --- app/views/devise/sessions/two_factor.html.haml | 1 + 1 file changed, 1 insertion(+) diff --git a/app/views/devise/sessions/two_factor.html.haml b/app/views/devise/sessions/two_factor.html.haml index 779928631a..30ddd0bd3c 100644 --- a/app/views/devise/sessions/two_factor.html.haml +++ b/app/views/devise/sessions/two_factor.html.haml @@ -8,5 +8,6 @@ .alert.alert-danger = @error = f.text_field :otp_attempt, class: 'form-control', placeholder: 'Two-factor authentication code', required: true, autofocus: true + %p.help-block.hint If you've lost your phone, you may enter one of your recovery codes. .prepend-top-20 = f.submit "Verify code", class: "btn btn-save" From 661d09a22b8a9a7fdead09fdb27284ff30cb0308 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Fri, 8 May 2015 20:59:09 -0400 Subject: [PATCH 111/158] Re-annotate User model --- app/models/user.rb | 5 +++++ spec/models/user_spec.rb | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/app/models/user.rb b/app/models/user.rb index 7b1f6fae3b..9dbc9e1cf2 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -50,6 +50,11 @@ # bitbucket_access_token :string(255) # bitbucket_access_token_secret :string(255) # location :string(255) +# encrypted_otp_secret :string(255) +# encrypted_otp_secret_iv :string(255) +# encrypted_otp_secret_salt :string(255) +# otp_required_for_login :boolean +# otp_backup_codes :text # public_email :string(255) default(""), not null # diff --git a/spec/models/user_spec.rb b/spec/models/user_spec.rb index 771709c127..0dddcd5bda 100644 --- a/spec/models/user_spec.rb +++ b/spec/models/user_spec.rb @@ -50,6 +50,11 @@ # bitbucket_access_token :string(255) # bitbucket_access_token_secret :string(255) # location :string(255) +# encrypted_otp_secret :string(255) +# encrypted_otp_secret_iv :string(255) +# encrypted_otp_secret_salt :string(255) +# otp_required_for_login :boolean +# otp_backup_codes :text # public_email :string(255) default(""), not null # From 2ad1334d9e0e143eb1cd1694433bfdf29314a65e Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 14:23:34 -0400 Subject: [PATCH 112/158] Quotes, icon helper in profiles/accounts/show view --- app/views/profiles/accounts/show.html.haml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/app/views/profiles/accounts/show.html.haml b/app/views/profiles/accounts/show.html.haml index 87f827793a..6ac60b01f8 100644 --- a/app/views/profiles/accounts/show.html.haml +++ b/app/views/profiles/accounts/show.html.haml @@ -34,13 +34,13 @@ Two-factor Authentication enabled %div .pull-right - = link_to "Disable Two-factor Authentication", profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm', + = link_to 'Disable Two-factor Authentication', profile_two_factor_auth_path, method: :delete, class: 'btn btn-close btn-sm', data: { confirm: 'Are you sure?' } %p If you lose your recovery codes you can %strong = succeed ',' do - = link_to "generate new ones", codes_profile_two_factor_auth_path, method: :post, data: { confirm: 'Are you sure?' } + = link_to 'generate new ones', codes_profile_two_factor_auth_path, method: :post, data: { confirm: 'Are you sure?' } invalidating all previous codes. - else @@ -52,7 +52,7 @@ Each time you log in you’ll be required to provide your username and password as usual, plus a randomly-generated code from your phone. %div - = link_to "Enable Two-factor Authentication", new_profile_two_factor_auth_path, class: 'btn btn-success' + = link_to 'Enable Two-factor Authentication', new_profile_two_factor_auth_path, class: 'btn btn-success' - if show_profile_social_tab? %fieldset @@ -65,7 +65,7 @@ class: "btn btn-lg #{'active' if oauth_active?(provider)}" - if oauth_active?(provider) = link_to unlink_profile_account_path(provider: provider), method: :delete, class: 'btn btn-lg' do - %i.fa.fa-close + = icon('close') - if show_profile_username_tab? %fieldset.update-username @@ -79,7 +79,7 @@   .loading-gif.hide %p - %i.fa.fa-spinner.fa-spin + = icon('spinner spin') Saving new username %p.light = user_url(@user) From 0c113c8dcb02e02457473823847b41df4eeedb88 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 14:52:46 -0400 Subject: [PATCH 113/158] Make otp_backup_codes a text field --- ...150331183602_add_devise_two_factor_backupable_to_users.rb | 2 +- db/schema.rb | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb b/db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb index 2feb49f43f..913958db7c 100644 --- a/db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb +++ b/db/migrate/20150331183602_add_devise_two_factor_backupable_to_users.rb @@ -1,5 +1,5 @@ class AddDeviseTwoFactorBackupableToUsers < ActiveRecord::Migration def change - add_column :users, :otp_backup_codes, :string, array: true + add_column :users, :otp_backup_codes, :text end end diff --git a/db/schema.rb b/db/schema.rb index 04abf9bb9a..3e5810d740 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -492,6 +492,11 @@ ActiveRecord::Schema.define(version: 20150502064022) do t.string "bitbucket_access_token" t.string "bitbucket_access_token_secret" t.string "location" + t.string "encrypted_otp_secret" + t.string "encrypted_otp_secret_iv" + t.string "encrypted_otp_secret_salt" + t.boolean "otp_required_for_login" + t.text "otp_backup_codes" t.string "public_email", default: "", null: false end From b050bb5bada27f38c6308ca33ac081f56cc681a5 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 15:45:22 -0400 Subject: [PATCH 114/158] Fix 2FA backup code removal --- app/models/user.rb | 3 +++ spec/features/login_spec.rb | 12 +++++------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/app/models/user.rb b/app/models/user.rb index 9dbc9e1cf2..0969fa9308 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -80,7 +80,10 @@ class User < ActiveRecord::Base devise :two_factor_authenticatable, otp_secret_encryption_key: File.read(Rails.root.join('.secret')).chomp + devise :two_factor_backupable + serialize :otp_backup_codes, JSON + devise :lockable, :async, :recoverable, :rememberable, :trackable, :validatable, :omniauthable, :confirmable, :registerable diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb index f1e24c5424..ca7fb022a2 100644 --- a/spec/features/login_spec.rb +++ b/spec/features/login_spec.rb @@ -47,7 +47,7 @@ feature 'Login' do before do expect(codes.size).to eq 5 - # Because `generate_otp_backup_codes!` doesn't actually do this... + # Ensure the generated codes get saved user.save end @@ -58,20 +58,18 @@ feature 'Login' do end it 'invalidates the used code' do - # FIXME (rspeicher): Broken library is broken - expect { enter_code(codes.sample) }.to change { user.otp_backup_codes.size }.by(-1) + expect { enter_code(codes.sample) }. + to change { user.reload.otp_backup_codes.size }.by(-1) end end context 'with invalid code' do it 'blocks login' do - # FIXME (rspeicher): Broken library is broken code = codes.sample expect(user.invalidate_otp_backup_code!(code)).to eq true - expect(user.otp_backup_codes.size).to eq 4 # Passes + user.save! - user.reload - expect(user.otp_backup_codes.size).to eq 4 # Fails... WAT?! + expect(user.reload.otp_backup_codes.size).to eq 4 enter_code(code) expect(page).to have_content('Invalid two-factor code') From 5f43cae6ca2c9ebd3f6561a7b40c64c65913e064 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 15:46:18 -0400 Subject: [PATCH 115/158] Add :two_factor trait to User factory --- spec/factories.rb | 7 +++++++ spec/features/login_spec.rb | 12 ++++-------- 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/spec/factories.rb b/spec/factories.rb index 19f2935f30..26e8a795fa 100644 --- a/spec/factories.rb +++ b/spec/factories.rb @@ -28,6 +28,13 @@ FactoryGirl.define do admin true end + trait :two_factor do + before(:create) do |user| + user.otp_required_for_login = true + user.otp_secret = User.generate_otp_secret + end + end + factory :omniauth_user do ignore do extern_uid '123456' diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb index ca7fb022a2..e44ddc1799 100644 --- a/spec/features/login_spec.rb +++ b/spec/features/login_spec.rb @@ -1,16 +1,10 @@ require 'spec_helper' feature 'Login' do - let(:user) { create(:user) } - context 'with two-factor authentication' do - before do - user.otp_required_for_login = true - user.otp_secret = User.generate_otp_secret - user.save! - end - context 'with valid username/password' do + let(:user) { create(:user, :two_factor) } + before do login_with(user) expect(page).to have_content('Two-factor Authentication') @@ -80,6 +74,8 @@ feature 'Login' do end context 'without two-factor authentication' do + let(:user) { create(:user) } + it 'allows basic login' do login_with(user) expect(current_path).to eq root_path From c845347b233b9bb40d9b304d864ac33e178429c1 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 15:46:49 -0400 Subject: [PATCH 116/158] Generate 10 2FA backup codes instead of the default of 5 --- app/models/user.rb | 2 +- spec/features/login_spec.rb | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/app/models/user.rb b/app/models/user.rb index 0969fa9308..70972eb271 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -81,7 +81,7 @@ class User < ActiveRecord::Base devise :two_factor_authenticatable, otp_secret_encryption_key: File.read(Rails.root.join('.secret')).chomp - devise :two_factor_backupable + devise :two_factor_backupable, otp_number_of_backup_codes: 10 serialize :otp_backup_codes, JSON devise :lockable, :async, :recoverable, :rememberable, :trackable, diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb index e44ddc1799..61066e7e92 100644 --- a/spec/features/login_spec.rb +++ b/spec/features/login_spec.rb @@ -39,7 +39,7 @@ feature 'Login' do let(:codes) { user.generate_otp_backup_codes! } before do - expect(codes.size).to eq 5 + expect(codes.size).to eq 10 # Ensure the generated codes get saved user.save @@ -63,7 +63,7 @@ feature 'Login' do expect(user.invalidate_otp_backup_code!(code)).to eq true user.save! - expect(user.reload.otp_backup_codes.size).to eq 4 + expect(user.reload.otp_backup_codes.size).to eq 9 enter_code(code) expect(page).to have_content('Invalid two-factor code') From c40f59d04f6c01b543f793f0884b26732fdcc0f0 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 16:52:46 -0400 Subject: [PATCH 117/158] Add otp_attempt to filtered parameters --- config/application.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/application.rb b/config/application.rb index fa399533e5..7e899cc3b5 100644 --- a/config/application.rb +++ b/config/application.rb @@ -31,7 +31,7 @@ module Gitlab config.encoding = "utf-8" # Configure sensitive parameters which will be filtered from the log file. - config.filter_parameters.push(:password, :password_confirmation, :private_token) + config.filter_parameters.push(:password, :password_confirmation, :private_token, :otp_attempt) # Enable escaping HTML in JSON. config.active_support.escape_html_entities_in_json = true From 3fb0fedd0415732b3c1bf0aaa66abf1197012d75 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 17:00:37 -0400 Subject: [PATCH 118/158] Autofocus the pin field on 2FA enable form --- app/views/profiles/two_factor_auths/new.html.haml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/views/profiles/two_factor_auths/new.html.haml b/app/views/profiles/two_factor_auths/new.html.haml index f0a4a34187..fe03a259a1 100644 --- a/app/views/profiles/two_factor_auths/new.html.haml +++ b/app/views/profiles/two_factor_auths/new.html.haml @@ -18,6 +18,6 @@ .form-group = label_tag :pin_code, nil, class: "control-label" .col-sm-10 - = text_field_tag :pin_code, nil, class: "form-control", required: true + = text_field_tag :pin_code, nil, class: "form-control", required: true, autofocus: true .form-actions = submit_tag 'Submit', class: 'btn btn-success' From 76873ce4a49be9a591082f7b96482ebdc9cace9d Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 17:02:06 -0400 Subject: [PATCH 119/158] Move "invalid 2FA code" error message to the flash This makes it consistent with the Invalid email/password error message from the previous step. --- app/controllers/sessions_controller.rb | 2 +- app/views/devise/sessions/two_factor.html.haml | 3 --- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index e2a5c61257..b975714311 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -63,7 +63,7 @@ class SessionsController < Devise::SessionsController sign_in(user) else - @error = 'Invalid two-factor code' + flash.now[:alert] = 'Invalid two-factor code.' render :two_factor and return end else diff --git a/app/views/devise/sessions/two_factor.html.haml b/app/views/devise/sessions/two_factor.html.haml index 30ddd0bd3c..22b2c1a186 100644 --- a/app/views/devise/sessions/two_factor.html.haml +++ b/app/views/devise/sessions/two_factor.html.haml @@ -4,9 +4,6 @@ %h3 Two-factor Authentication .login-body = form_for(resource, as: resource_name, url: session_path(resource_name), method: :post) do |f| - - if @error - .alert.alert-danger - = @error = f.text_field :otp_attempt, class: 'form-control', placeholder: 'Two-factor authentication code', required: true, autofocus: true %p.help-block.hint If you've lost your phone, you may enter one of your recovery codes. .prepend-top-20 From 5cd526f77fa51347ec66ab094b778ca4b83b8fce Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 17:04:02 -0400 Subject: [PATCH 120/158] Prevent "You are already signed in." error message upon 2FA login --- app/controllers/sessions_controller.rb | 10 ++++++++-- spec/features/login_spec.rb | 19 ++++++++++++++++++- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index b975714311..d4ff0d9756 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -1,5 +1,11 @@ class SessionsController < Devise::SessionsController - prepend_before_action :authenticate_with_two_factor, only: :create + prepend_before_action :authenticate_with_two_factor, only: [:create] + + # This action comes from DeviseController, but because we call `sign_in` + # manually inside `authenticate_with_two_factor`, not skipping this action + # would cause a "You are already signed in." error message to be shown upon + # successful login. + skip_before_action :require_no_authentication, only: [:create] def new redirect_path = @@ -61,7 +67,7 @@ class SessionsController < Devise::SessionsController # Remove any lingering user data from login session.delete(:otp_user_id) - sign_in(user) + sign_in(user) and return else flash.now[:alert] = 'Invalid two-factor code.' render :two_factor and return diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb index 61066e7e92..61defb8a33 100644 --- a/spec/features/login_spec.rb +++ b/spec/features/login_spec.rb @@ -15,6 +15,11 @@ feature 'Login' do click_button 'Verify code' end + it 'does not show a "You are already signed in." error message' do + enter_code(user.current_otp) + expect(page).not_to have_content('You are already signed in.') + end + context 'using one-time code' do it 'allows login with valid code' do enter_code(user.current_otp) @@ -66,7 +71,7 @@ feature 'Login' do expect(user.reload.otp_backup_codes.size).to eq 9 enter_code(code) - expect(page).to have_content('Invalid two-factor code') + expect(page).to have_content('Invalid two-factor code.') end end end @@ -80,5 +85,17 @@ feature 'Login' do login_with(user) expect(current_path).to eq root_path end + + it 'does not show a "You are already signed in." error message' do + login_with(user) + expect(page).not_to have_content('You are already signed in.') + end + + it 'blocks invalid login' do + user = create(:user, password: 'not-the-default') + + login_with(user) + expect(page).to have_content('Invalid email or password.') + end end end From 414ddc0021dfe2c8b594b240b750a700c3af2b14 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 17:04:32 -0400 Subject: [PATCH 121/158] Clear all 2FA-related fields when user disables the feature --- .../profiles/two_factor_auths_controller.rb | 9 +- .../two_factor_auths_controller_spec.rb | 126 ++++++++++++++++++ 2 files changed, 133 insertions(+), 2 deletions(-) create mode 100644 spec/controllers/profiles/two_factor_auths_controller_spec.rb diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb index 60f8ec5cf3..30ee689173 100644 --- a/app/controllers/profiles/two_factor_auths_controller.rb +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -28,8 +28,13 @@ class Profiles::TwoFactorAuthsController < Profiles::ApplicationController end def destroy - current_user.otp_required_for_login = false - current_user.save! + current_user.update_attributes({ + otp_required_for_login: false, + encrypted_otp_secret: nil, + encrypted_otp_secret_iv: nil, + encrypted_otp_secret_salt: nil, + otp_backup_codes: nil + }) redirect_to profile_account_path end diff --git a/spec/controllers/profiles/two_factor_auths_controller_spec.rb b/spec/controllers/profiles/two_factor_auths_controller_spec.rb new file mode 100644 index 0000000000..f05d1f5fbe --- /dev/null +++ b/spec/controllers/profiles/two_factor_auths_controller_spec.rb @@ -0,0 +1,126 @@ +require 'spec_helper' + +describe Profiles::TwoFactorAuthsController do + before do + # `user` should be defined within the action-specific describe blocks + sign_in(user) + + allow(subject).to receive(:current_user).and_return(user) + end + + describe 'GET new' do + let(:user) { create(:user) } + + it 'generates otp_secret' do + expect { get :new }.to change { user.otp_secret } + end + + it 'assigns qr_code' do + code = double('qr code') + expect(subject).to receive(:build_qr_code).and_return(code) + + get :new + expect(assigns[:qr_code]).to eq code + end + end + + describe 'POST create' do + let(:user) { create(:user) } + let(:pin) { 'pin-code' } + + def go + post :create, pin_code: pin + end + + context 'with valid pin' do + before do + expect(user).to receive(:valid_otp?).with(pin).and_return(true) + end + + it 'sets otp_required_for_login' do + go + + user.reload + expect(user.otp_required_for_login).to eq true + end + + it 'presents plaintext codes for the user to save' do + expect(user).to receive(:generate_otp_backup_codes!).and_return(%w(a b c)) + + go + + expect(assigns[:codes]).to match_array %w(a b c) + end + + it 'renders create' do + go + expect(response).to render_template(:create) + end + end + + context 'with invalid pin' do + before do + expect(user).to receive(:valid_otp?).with(pin).and_return(false) + end + + it 'assigns error' do + go + expect(assigns[:error]).to eq 'Invalid pin code' + end + + it 'assigns qr_code' do + code = double('qr code') + expect(subject).to receive(:build_qr_code).and_return(code) + + go + expect(assigns[:qr_code]).to eq code + end + + it 'renders new' do + go + expect(response).to render_template(:new) + end + end + end + + describe 'POST codes' do + let(:user) { create(:user, :two_factor) } + + it 'presents plaintext codes for the user to save' do + expect(user).to receive(:generate_otp_backup_codes!).and_return(%w(a b c)) + + post :codes + expect(assigns[:codes]).to match_array %w(a b c) + end + + it 'persists the generated codes' do + post :codes + + user.reload + expect(user.otp_backup_codes).not_to be_empty + end + end + + describe 'DELETE destroy' do + let(:user) { create(:user, :two_factor) } + let!(:codes) { user.generate_otp_backup_codes! } + + it 'clears all 2FA-related fields' do + expect(user.otp_required_for_login).to eq true + expect(user.otp_backup_codes).not_to be_nil + expect(user.encrypted_otp_secret).not_to be_nil + + delete :destroy + + expect(user.otp_required_for_login).to eq false + expect(user.otp_backup_codes).to be_nil + expect(user.encrypted_otp_secret).to be_nil + end + + it 'redirects to profile_account_path' do + delete :destroy + + expect(response).to redirect_to(profile_account_path) + end + end +end From 21be1b415262773e099176dbb3da9846bc36a0ca Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 17:36:19 -0400 Subject: [PATCH 122/158] Add CHANGELOG entry --- CHANGELOG | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG b/CHANGELOG index 3206c625cd..b48ef6ea50 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -24,7 +24,7 @@ v 7.11.0 (unreleased) - When use change branches link at MR form - save source branch selection instead of target one - Improve handling of large diffs - Added GitLab Event header for project hooks - - + - Add Two-factor authentication (2FA) for GitLab logins - Show Atom feed buttons everywhere where applicable. - Add project activity atom feed. - Don't crash when an MR from a fork has a cross-reference comment from the target project on one of its commits. From 11989d62d09a628015c68c161898dfabcd966825 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 17:39:16 -0400 Subject: [PATCH 123/158] Remove unnecessary User#login accessor override --- app/models/user.rb | 5 ----- 1 file changed, 5 deletions(-) diff --git a/app/models/user.rb b/app/models/user.rb index 70972eb271..b45c7f20e5 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -674,9 +674,4 @@ class User < ActiveRecord::Base true end - - # Used to populate the hidden form field during Two-factor authentication - def login - username || email - end end From e80c4447a82553d74267f327060b96807c2e5c0c Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 23:50:51 +0200 Subject: [PATCH 124/158] Fix spec. --- features/steps/project/commits/commits.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/features/steps/project/commits/commits.rb b/features/steps/project/commits/commits.rb index 30b1934b36..c888e82e20 100644 --- a/features/steps/project/commits/commits.rb +++ b/features/steps/project/commits/commits.rb @@ -12,7 +12,7 @@ class Spinach::Features::ProjectCommits < Spinach::FeatureSteps end step 'I click atom feed link' do - click_link "Feed" + click_link "Commits Feed" end step 'I see commits atom feed' do From a3edd473b936de0be841c003506a41e0ff941b9e Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 23:51:50 +0200 Subject: [PATCH 125/158] Improve create_mr_button? performance. --- app/helpers/compare_helper.rb | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/app/helpers/compare_helper.rb b/app/helpers/compare_helper.rb index fe339d3215..f1dc906cab 100644 --- a/app/helpers/compare_helper.rb +++ b/app/helpers/compare_helper.rb @@ -1,12 +1,11 @@ module CompareHelper def create_mr_button?(from = params[:from], to = params[:to], project = @project) - project.merge_requests_enabled && - from.present? && + from.present? && to.present? && - project.repository.branch_names.include?(from) && - project.repository.branch_names.include?(to) && from != to && - !@refs_are_same + project.merge_requests_enabled && + project.repository.branch_names.include?(from) && + project.repository.branch_names.include?(to) end def create_mr_path(from = params[:from], to = params[:to], project = @project) From aa4d19904418cf57369ab9c80d05730c03636baf Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sun, 10 May 2015 00:39:20 +0200 Subject: [PATCH 126/158] Fix commits feed button. --- app/views/projects/commits/show.html.haml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/app/views/projects/commits/show.html.haml b/app/views/projects/commits/show.html.haml index e2a8d3ffbb..9682100a54 100644 --- a/app/views/projects/commits/show.html.haml +++ b/app/views/projects/commits/show.html.haml @@ -14,10 +14,10 @@ = icon('plus') Create Merge Request - - if current_user && current_user.private_token - = link_to namespace_project_commits_path(@project.namespace, @project, @ref, {format: :atom, private_token: current_user.private_token}), title: "Feed", class: 'prepend-left-10 btn' do - = icon("rss") - Commits Feed + - if current_user && current_user.private_token + = link_to namespace_project_commits_path(@project.namespace, @project, @ref, {format: :atom, private_token: current_user.private_token}), title: "Feed", class: 'prepend-left-10 btn' do + = icon("rss") + Commits Feed %ul.breadcrumb.repo-breadcrumb From 19b897e998d4b376390a3e0c12ccac4d1e92597d Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sun, 10 May 2015 19:13:47 -0400 Subject: [PATCH 127/158] Remove extra `devise` call that got added by accident --- app/models/user.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/app/models/user.rb b/app/models/user.rb index b45c7f20e5..aeab503297 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -62,9 +62,6 @@ require 'carrierwave/orm/activerecord' require 'file_size_validator' class User < ActiveRecord::Base - devise :two_factor_authenticatable, - otp_secret_encryption_key: File.read(Rails.root.join('.secret')).chomp - include Sortable include Gitlab::ConfigHelper include TokenAuthenticatable From 5d08a5a56aea744a8adff833379f0b90ba9427db Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 18 Apr 2015 18:44:46 -0400 Subject: [PATCH 128/158] Note's voting specs don't need to persist to the database --- spec/models/note_spec.rb | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/spec/models/note_spec.rb b/spec/models/note_spec.rb index 4a6bfdb291..43bfd25a48 100644 --- a/spec/models/note_spec.rb +++ b/spec/models/note_spec.rb @@ -34,48 +34,46 @@ describe Note do it { is_expected.to validate_presence_of(:project) } end - describe "Voting score" do - let(:project) { create(:project) } - - it "recognizes a neutral note" do - note = create(:votable_note, note: "This is not a +1 note") + describe 'voting score' do + it 'recognizes a neutral note' do + note = build(:votable_note, note: 'This is not a +1 note') expect(note).not_to be_upvote expect(note).not_to be_downvote end - it "recognizes a neutral emoji note" do + it 'recognizes a neutral emoji note' do note = build(:votable_note, note: "I would :+1: this, but I don't want to") expect(note).not_to be_upvote expect(note).not_to be_downvote end - it "recognizes a +1 note" do - note = create(:votable_note, note: "+1 for this") + it 'recognizes a +1 note' do + note = build(:votable_note, note: '+1 for this') expect(note).to be_upvote end - it "recognizes a +1 emoji as a vote" do - note = build(:votable_note, note: ":+1: for this") + it 'recognizes a +1 emoji as a vote' do + note = build(:votable_note, note: ':+1: for this') expect(note).to be_upvote end - it "recognizes a thumbsup emoji as a vote" do - note = build(:votable_note, note: ":thumbsup: for this") + it 'recognizes a thumbsup emoji as a vote' do + note = build(:votable_note, note: ':thumbsup: for this') expect(note).to be_upvote end - it "recognizes a -1 note" do - note = create(:votable_note, note: "-1 for this") + it 'recognizes a -1 note' do + note = build(:votable_note, note: '-1 for this') expect(note).to be_downvote end - it "recognizes a -1 emoji as a vote" do - note = build(:votable_note, note: ":-1: for this") + it 'recognizes a -1 emoji as a vote' do + note = build(:votable_note, note: ':-1: for this') expect(note).to be_downvote end - it "recognizes a thumbsdown emoji as a vote" do - note = build(:votable_note, note: ":thumbsdown: for this") + it 'recognizes a thumbsdown emoji as a vote' do + note = build(:votable_note, note: ':thumbsdown: for this') expect(note).to be_downvote end end From 0e89ff0fb05973bb3ff6930906d52f10517efa62 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 18 Apr 2015 18:45:12 -0400 Subject: [PATCH 129/158] Simplify `Note#upvote?` and `Note#downvote?` --- app/models/note.rb | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/app/models/note.rb b/app/models/note.rb index cbce678668..b32c32e2a2 100644 --- a/app/models/note.rb +++ b/app/models/note.rb @@ -449,16 +449,6 @@ class Note < ActiveRecord::Base @discussion_id ||= Note.build_discussion_id(noteable_type, noteable_id || commit_id, line_code) end - # Returns true if this is a downvote note, - # otherwise false is returned - def downvote? - votable? && (note.start_with?('-1') || - note.start_with?(':-1:') || - note.start_with?(':thumbsdown:') || - note.start_with?(':thumbs_down_sign:') - ) - end - def for_commit? noteable_type == "Commit" end @@ -500,14 +490,18 @@ class Note < ActiveRecord::Base nil end - # Returns true if this is an upvote note, - # otherwise false is returned + DOWNVOTES = %w(-1 :-1: :thumbsdown: :thumbs_down_sign:) + + # Check if the note is a downvote + def downvote? + votable? && note.start_with?(*DOWNVOTES) + end + + UPVOTES = %w(+1 :+1: :thumbsup: :thumbs_up_sign:) + + # Check if the note is an upvote def upvote? - votable? && (note.start_with?('+1') || - note.start_with?(':+1:') || - note.start_with?(':thumbsup:') || - note.start_with?(':thumbs_up_sign:') - ) + votable? && note.start_with?(*UPVOTES) end def superceded?(notes) From 48e6fb532a6655b98319403c5e8699f7daf0305e Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Thu, 30 Apr 2015 23:16:19 -0400 Subject: [PATCH 130/158] Add a SystemNoteService class There's a lot of code in the Note model that only deals with creating system notes, so we're going to split that into its own class. --- app/models/concerns/mentionable.rb | 2 +- app/models/note.rb | 215 +---------- app/services/issuable_base_service.rb | 6 +- app/services/issues/close_service.rb | 2 +- app/services/issues/reopen_service.rb | 2 +- app/services/merge_requests/base_service.rb | 2 +- .../merge_requests/refresh_service.rb | 5 +- app/services/system_note_service.rb | 304 +++++++++++++++ spec/factories/notes.rb | 15 +- spec/models/note_spec.rb | 256 ------------- spec/services/system_note_service_spec.rb | 353 ++++++++++++++++++ 11 files changed, 683 insertions(+), 479 deletions(-) create mode 100644 app/services/system_note_service.rb create mode 100644 spec/services/system_note_service_spec.rb diff --git a/app/models/concerns/mentionable.rb b/app/models/concerns/mentionable.rb index 3ef3e8b67d..a5957391bb 100644 --- a/app/models/concerns/mentionable.rb +++ b/app/models/concerns/mentionable.rb @@ -39,7 +39,7 @@ module Mentionable # Determine whether or not a cross-reference Note has already been created between this Mentionable and # the specified target. def has_mentioned?(target) - Note.cross_reference_exists?(target, local_reference) + SystemNoteService.cross_reference_exists?(target, local_reference) end def mentioned_users(current_user = nil, p = project) diff --git a/app/models/note.rb b/app/models/note.rb index b32c32e2a2..f2470e5f39 100644 --- a/app/models/note.rb +++ b/app/models/note.rb @@ -63,143 +63,9 @@ class Note < ActiveRecord::Base after_update :set_references class << self - def create_status_change_note(noteable, project, author, status, source) - body = "Status changed to #{status}#{' by ' + source.gfm_reference if source}" - - create( - noteable: noteable, - project: project, - author: author, - note: body, - system: true - ) - end - - # +noteable+ was referenced from +mentioner+, by including GFM in either - # +mentioner+'s description or an associated Note. - # Create a system Note associated with +noteable+ with a GFM back-reference - # to +mentioner+. - def create_cross_reference_note(noteable, mentioner, author) - gfm_reference = mentioner_gfm_ref(noteable, mentioner) - - note_options = { - project: noteable.project, - author: author, - note: cross_reference_note_content(gfm_reference), - system: true - } - - if noteable.kind_of?(Commit) - note_options.merge!(noteable_type: 'Commit', commit_id: noteable.id) - else - note_options.merge!(noteable: noteable) - end - - create(note_options) unless cross_reference_disallowed?(noteable, mentioner) - end - - def create_milestone_change_note(noteable, project, author, milestone) - body = if milestone.nil? - 'Milestone removed' - else - "Milestone changed to #{milestone.title}" - end - - create( - noteable: noteable, - project: project, - author: author, - note: body, - system: true - ) - end - - def create_assignee_change_note(noteable, project, author, assignee) - body = assignee.nil? ? 'Assignee removed' : "Reassigned to @#{assignee.username}" - - create({ - noteable: noteable, - project: project, - author: author, - note: body, - system: true - }) - end - - def create_labels_change_note(noteable, project, author, added_labels, removed_labels) - labels_count = added_labels.count + removed_labels.count - added_labels = added_labels.map{ |label| "~#{label.id}" }.join(' ') - removed_labels = removed_labels.map{ |label| "~#{label.id}" }.join(' ') - message = '' - - if added_labels.present? - message << "added #{added_labels}" - end - - if added_labels.present? && removed_labels.present? - message << ' and ' - end - - if removed_labels.present? - message << "removed #{removed_labels}" - end - - message << ' ' << 'label'.pluralize(labels_count) - body = "#{message.capitalize}" - - create( - noteable: noteable, - project: project, - author: author, - note: body, - system: true - ) - end - - def create_new_commits_note(merge_request, project, author, new_commits, existing_commits = [], oldrev = nil) - total_count = new_commits.length + existing_commits.length - commits_text = ActionController::Base.helpers.pluralize(total_count, 'commit') - body = "Added #{commits_text}:\n\n" - - if existing_commits.length > 0 - commit_ids = - if existing_commits.length == 1 - existing_commits.first.short_id - else - if oldrev - "#{Commit.truncate_sha(oldrev)}...#{existing_commits.last.short_id}" - else - "#{existing_commits.first.short_id}..#{existing_commits.last.short_id}" - end - end - - commits_text = ActionController::Base.helpers.pluralize(existing_commits.length, 'commit') - - branch = - if merge_request.for_fork? - "#{merge_request.target_project_namespace}:#{merge_request.target_branch}" - else - merge_request.target_branch - end - - message = "* #{commit_ids} - #{commits_text} from branch `#{branch}`" - body << message - body << "\n" - end - - new_commits.each do |commit| - message = "* #{commit.short_id} - #{commit.title}" - body << message - body << "\n" - end - - create( - noteable: merge_request, - project: project, - author: author, - note: body, - system: true - ) + # TODO (rspeicher): Update usages + def create_cross_reference_note(*args) + SystemNoteService.cross_reference(*args) end def discussions_from_notes(notes) @@ -227,88 +93,19 @@ class Note < ActiveRecord::Base [:discussion, type.try(:underscore), id, line_code].join("-").to_sym end - # Determine if cross reference note should be created. - # eg. mentioning a commit in MR comments which exists inside a MR - # should not create "mentioned in" note. - def cross_reference_disallowed?(noteable, mentioner) - if mentioner.kind_of?(MergeRequest) - mentioner.commits.map(&:id).include? noteable.id - end - end - - # Determine whether or not a cross-reference note already exists. - def cross_reference_exists?(noteable, mentioner) - gfm_reference = mentioner_gfm_ref(noteable, mentioner, true) - notes = if noteable.is_a?(Commit) - where(commit_id: noteable.id, noteable_type: 'Commit') - else - where(noteable_id: noteable.id, noteable_type: noteable.class) - end - - notes.where('note like ?', cross_reference_note_pattern(gfm_reference)). - system.any? - end - def search(query) where("note like :query", query: "%#{query}%") end + end - def cross_reference_note_prefix - 'mentioned in ' - end - - private - - def cross_reference_note_content(gfm_reference) - cross_reference_note_prefix + "#{gfm_reference}" - end - - def cross_reference_note_pattern(gfm_reference) - # Older cross reference notes contained underscores for emphasis - "%" + cross_reference_note_content(gfm_reference) + "%" - end - - # Prepend the mentioner's namespaced project path to the GFM reference for - # cross-project references. For same-project references, return the - # unmodified GFM reference. - def mentioner_gfm_ref(noteable, mentioner, cross_reference = false) - if mentioner.is_a?(Commit) && cross_reference - return mentioner.gfm_reference.sub('commit ', 'commit %') - end - - full_gfm_reference(mentioner.project, noteable.project, mentioner) - end - - # Return the +mentioner+ GFM reference. If the mentioner and noteable - # projects are not the same, add the mentioning project's path to the - # returned value. - def full_gfm_reference(mentioning_project, noteable_project, mentioner) - if mentioning_project == noteable_project - mentioner.gfm_reference - else - if mentioner.is_a?(Commit) - mentioner.gfm_reference.sub( - /(commit )/, - "\\1#{mentioning_project.path_with_namespace}@" - ) - else - mentioner.gfm_reference.sub( - /(issue |merge request )/, - "\\1#{mentioning_project.path_with_namespace}" - ) - end - end - end + def cross_reference? + SystemNoteService.cross_reference?(note) end def max_attachment_size current_application_settings.max_attachment_size.megabytes.to_i end - def cross_reference? - note.start_with?(self.class.cross_reference_note_prefix) - end - def find_diff return nil unless noteable && noteable.diffs.present? diff --git a/app/services/issuable_base_service.rb b/app/services/issuable_base_service.rb index 5e1906ad2a..4d251cb30d 100644 --- a/app/services/issuable_base_service.rb +++ b/app/services/issuable_base_service.rb @@ -2,17 +2,17 @@ class IssuableBaseService < BaseService private def create_assignee_note(issuable) - Note.create_assignee_change_note( + SystemNoteService.assignee_change( issuable, issuable.project, current_user, issuable.assignee) end def create_milestone_note(issuable) - Note.create_milestone_change_note( + SystemNoteService.milestone_change( issuable, issuable.project, current_user, issuable.milestone) end def create_labels_note(issuable, added_labels, removed_labels) - Note.create_labels_change_note( + SystemNoteService.label_change( issuable, issuable.project, current_user, added_labels, removed_labels) end end diff --git a/app/services/issues/close_service.rb b/app/services/issues/close_service.rb index f670019cc6..bdc13685fd 100644 --- a/app/services/issues/close_service.rb +++ b/app/services/issues/close_service.rb @@ -14,7 +14,7 @@ module Issues private def create_note(issue, current_commit) - Note.create_status_change_note(issue, issue.project, current_user, issue.state, current_commit) + SystemNoteService.status_change(issue, issue.project, current_user, issue.state, current_commit) end end end diff --git a/app/services/issues/reopen_service.rb b/app/services/issues/reopen_service.rb index 1e5c398516..fbbf573e5a 100644 --- a/app/services/issues/reopen_service.rb +++ b/app/services/issues/reopen_service.rb @@ -14,7 +14,7 @@ module Issues private def create_note(issue) - Note.create_status_change_note(issue, issue.project, current_user, issue.state, nil) + SystemNoteService.status_change(issue, issue.project, current_user, issue.state, nil) end end end diff --git a/app/services/merge_requests/base_service.rb b/app/services/merge_requests/base_service.rb index f6e1ae6f28..9b470880ad 100644 --- a/app/services/merge_requests/base_service.rb +++ b/app/services/merge_requests/base_service.rb @@ -2,7 +2,7 @@ module MergeRequests class BaseService < ::IssuableBaseService def create_note(merge_request) - Note.create_status_change_note(merge_request, merge_request.target_project, current_user, merge_request.state, nil) + SystemNoteService.status_change(merge_request, merge_request.target_project, current_user, merge_request.state, nil) end def hook_data(merge_request, action) diff --git a/app/services/merge_requests/refresh_service.rb b/app/services/merge_requests/refresh_service.rb index e9b526d1fb..1d57fd11de 100644 --- a/app/services/merge_requests/refresh_service.rb +++ b/app/services/merge_requests/refresh_service.rb @@ -82,8 +82,9 @@ module MergeRequests mr_commit_ids.include?(commit.id) end - Note.create_new_commits_note(merge_request, merge_request.project, - @current_user, new_commits, existing_commits, @oldrev) + SystemNoteService.commit_add(merge_request, merge_request.project, + @current_user, new_commits, + existing_commits, @oldrev) end end diff --git a/app/services/system_note_service.rb b/app/services/system_note_service.rb new file mode 100644 index 0000000000..2486360b26 --- /dev/null +++ b/app/services/system_note_service.rb @@ -0,0 +1,304 @@ +# SystemNoteService +# +# Used for creating system notes (e.g., when a user references a merge request +# from an issue, an issue's assignee changes, an issue is closed, etc.) +# +# All methods creating notes should be named using a singular noun and +# present-tense verb (e.g., "assignee_change" not "assignee_changed", +# "label_change" not "labels_change"). +class SystemNoteService + # Called when the assignee of a Noteable is changed or removed + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # assignee - User being assigned, or nil + # + # Example Note text: + # + # "Assignee removed" + # + # "Reassigned to @rspeicher" + # + # Returns the created Note object + def self.assignee_change(noteable, project, author, assignee) + body = assignee.nil? ? 'Assignee removed' : "Reassigned to @#{assignee.username}" + + create_note(noteable: noteable, project: project, author: author, note: body) + end + + # Called when a Mentionable references a Noteable + # + # noteable - Noteable object being referenced + # mentioner - Mentionable object + # author - User performing the reference + # + # Example Note text: + # + # "Mentioned in #1" + # + # "Mentioned in !2" + # + # "Mentioned in 54f7727c" + # + # See cross_reference_note_content. + # + # Returns the created Note object + def self.cross_reference(noteable, mentioner, author) + return if cross_reference_disallowed?(noteable, mentioner) + + gfm_reference = mentioner_gfm_ref(noteable, mentioner) + + note_options = { + project: noteable.project, + author: author, + note: cross_reference_note_content(gfm_reference) + } + + if noteable.kind_of?(Commit) + note_options.merge!(noteable_type: 'Commit', commit_id: noteable.id) + else + note_options.merge!(noteable: noteable) + end + + create_note(note_options) + end + + # Called when one or more labels on a Noteable are added and/or removed + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # added_labels - Array of Labels added + # removed_labels - Array of Labels removed + # + # Example Note text: + # + # "Added ~1 and removed ~2 ~3 labels" + # + # "Added ~4 label" + # + # "Removed ~5 label" + # + # Returns the created Note object + def self.label_change(noteable, project, author, added_labels, removed_labels) + labels_count = added_labels.count + removed_labels.count + + references = ->(label) { "~#{label.id}" } + added_labels = added_labels.map(&references).join(' ') + removed_labels = removed_labels.map(&references).join(' ') + + body = '' + + if added_labels.present? + body << "added #{added_labels}" + body << ' and ' if removed_labels.present? + end + + if removed_labels.present? + body << "removed #{removed_labels}" + end + + body << ' ' << 'label'.pluralize(labels_count) + body = "#{body.capitalize}" + + create_note(noteable: noteable, project: project, author: author, note: body) + end + + # Called when the milestone of a Noteable is changed + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # milestone - Milestone being assigned, or nil + # + # Example Note text: + # + # "Milestone removed" + # + # "Miletone changed to 7.11" + # + # Returns the created Note object + def self.milestone_change(noteable, project, author, milestone) + body = 'Milestone ' + body += milestone.nil? ? 'removed' : "changed to #{milestone.title}" + + create_note(noteable: noteable, project: project, author: author, note: body) + end + + # Called when commits are added to a Merge Request + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # new_commits - Array of Commits added since last push + # existing_commits - Array of Commits added in a previous push + # oldrev - TODO (rspeicher): I have no idea what this actually does + # + # See new_commit_summary and existing_commit_summary. + # + # Returns the created Note object + def self.commit_add(noteable, project, author, new_commits, existing_commits = [], oldrev = nil) + total_count = new_commits.length + existing_commits.length + commits_text = "#{total_count} commit".pluralize(total_count) + + body = "Added #{commits_text}:\n\n" + body << existing_commit_summary(noteable, existing_commits, oldrev) + body << new_commit_summary(new_commits).join("\n") + + create_note(noteable: noteable, project: project, author: author, note: body) + end + + # Called when the status of a Noteable is changed + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # status - String status + # source - Mentionable performing the change, or nil + # + # Example Note text: + # + # "Status changed to merged" + # + # "Status changed to closed by bc17db76" + # + # Returns the created Note object + def self.status_change(noteable, project, author, status, source) + body = "Status changed to #{status}" + body += " by #{source.gfm_reference}" if source + + create_note(noteable: noteable, project: project, author: author, note: body) + end + + def self.cross_reference?(note_text) + note_text.start_with?(cross_reference_note_prefix) + end + + # Determine if cross reference note should be created. + # eg. mentioning a commit in MR comments which exists inside a MR + # should not create "mentioned in" note. + def self.cross_reference_disallowed?(noteable, mentioner) + if mentioner.kind_of?(MergeRequest) + mentioner.commits.map(&:id).include? noteable.id + end + end + + def self.cross_reference_exists?(noteable, mentioner) + # Initial scope should be system notes of this noteable type + notes = Note.system.where(noteable_type: noteable.class) + + if noteable.is_a?(Commit) + # Commits have non-integer IDs, so they're stored in `commit_id` + notes = notes.where(commit_id: noteable.id) + else + notes = notes.where(noteable_id: noteable.id) + end + + gfm_reference = mentioner_gfm_ref(noteable, mentioner, true) + notes = notes.where('note like ?', cross_reference_note_pattern(gfm_reference)) + + notes.count > 0 + end + + private + + def self.create_note(args = {}) + Note.create(args.merge(system: true)) + end + + def self.cross_reference_note_prefix + 'mentioned in ' + end + + # Prepend the mentioner's namespaced project path to the GFM reference for + # cross-project references. For same-project references, return the + # unmodified GFM reference. + def self.mentioner_gfm_ref(noteable, mentioner, cross_reference = false) + if mentioner.is_a?(Commit) && cross_reference + return mentioner.gfm_reference.sub('commit ', 'commit %') + end + + full_gfm_reference(mentioner.project, noteable.project, mentioner) + end + + # Return the +mentioner+ GFM reference. If the mentioner and noteable + # projects are not the same, add the mentioning project's path to the + # returned value. + def self.full_gfm_reference(mentioning_project, noteable_project, mentioner) + if mentioning_project == noteable_project + mentioner.gfm_reference + else + if mentioner.is_a?(Commit) + mentioner.gfm_reference.sub( + /(commit )/, + "\\1#{mentioning_project.path_with_namespace}@" + ) + else + mentioner.gfm_reference.sub( + /(issue |merge request )/, + "\\1#{mentioning_project.path_with_namespace}" + ) + end + end + end + + def self.cross_reference_note_content(gfm_reference) + cross_reference_note_prefix + "#{gfm_reference}" + end + + def self.cross_reference_note_pattern(gfm_reference) + # Older cross reference notes contained underscores for emphasis + "%" + cross_reference_note_content(gfm_reference) + "%" + end + + # Build an Array of lines detailing each commit added in a merge request + # + # new_commits - Array of new Commit objects + # + # Returns an Array of Strings + def self.new_commit_summary(new_commits) + new_commits.collect do |commit| + "* #{commit.short_id} - #{commit.title}" + end + end + + # Build a single line summarizing existing commits being added in a merge + # request + # + # noteable - MergeRequest object + # existing_commits - Array of existing Commit objects + # oldrev - Optional String SHA of ... TODO (rspeicher): I have no idea what this actually does. + # + # Examples: + # + # "* ea0f8418...2f4426b7 - 24 commits from branch `master`" + # + # "* ea0f8418..4188f0ea - 15 commits from branch `fork:master`" + # + # "* ea0f8418 - 1 commit from branch `feature`" + # + # Returns a newline-terminated String + def self.existing_commit_summary(noteable, existing_commits, oldrev = nil) + return '' if existing_commits.empty? + + count = existing_commits.size + + commit_ids = if count == 1 + existing_commits.first.short_id + else + if oldrev + "#{Commit.truncate_sha(oldrev)}...#{existing_commits.last.short_id}" + else + "#{existing_commits.first.short_id}..#{existing_commits.last.short_id}" + end + end + + commits_text = "#{count} commit".pluralize(count) + + branch = noteable.target_branch + branch = "#{noteable.target_project_namespace}:#{branch}" if noteable.for_fork? + + "* #{commit_ids} - #{commits_text} from branch `#{branch}`\n" + end +end diff --git a/spec/factories/notes.rb b/spec/factories/notes.rb index f1c33461b5..1e4c0b03f7 100644 --- a/spec/factories/notes.rb +++ b/spec/factories/notes.rb @@ -25,12 +25,13 @@ FactoryGirl.define do note "Note" author - factory :note_on_commit, traits: [:on_commit] - factory :note_on_commit_diff, traits: [:on_commit, :on_diff] - factory :note_on_issue, traits: [:on_issue], aliases: [:votable_note] - factory :note_on_merge_request, traits: [:on_merge_request] + factory :note_on_commit, traits: [:on_commit] + factory :note_on_commit_diff, traits: [:on_commit, :on_diff] + factory :note_on_issue, traits: [:on_issue], aliases: [:votable_note] + factory :note_on_merge_request, traits: [:on_merge_request] factory :note_on_merge_request_diff, traits: [:on_merge_request, :on_diff] - factory :note_on_project_snippet, traits: [:on_project_snippet] + factory :note_on_project_snippet, traits: [:on_project_snippet] + factory :system_note, traits: [:system] trait :on_commit do project factory: :project @@ -58,6 +59,10 @@ FactoryGirl.define do noteable_type "Snippet" end + trait :system do + system true + end + trait :with_attachment do attachment { fixture_file_upload(Rails.root + "spec/fixtures/dk.png", "`/png") } end diff --git a/spec/models/note_spec.rb b/spec/models/note_spec.rb index 43bfd25a48..4a769f3677 100644 --- a/spec/models/note_spec.rb +++ b/spec/models/note_spec.rb @@ -150,178 +150,6 @@ describe Note do end end - describe '#create_status_change_note' do - let(:project) { create(:project) } - let(:thing) { create(:issue, project: project) } - let(:author) { create(:user) } - let(:status) { 'new_status' } - - subject { Note.create_status_change_note(thing, project, author, status, nil) } - - it 'creates and saves a Note' do - is_expected.to be_a Note - expect(subject.id).not_to be_nil - end - - describe '#noteable' do - subject { super().noteable } - it { is_expected.to eq(thing) } - end - - describe '#project' do - subject { super().project } - it { is_expected.to eq(thing.project) } - end - - describe '#author' do - subject { super().author } - it { is_expected.to eq(author) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("Status changed to #{status}") } - end - - it 'appends a back-reference if a closing mentionable is supplied' do - commit = double('commit', gfm_reference: 'commit 123456') - n = Note.create_status_change_note(thing, project, author, status, commit) - - expect(n.note).to eq("Status changed to #{status} by commit 123456") - end - end - - describe '#create_assignee_change_note' do - let(:project) { create(:project) } - let(:thing) { create(:issue, project: project) } - let(:author) { create(:user) } - let(:assignee) { create(:user, username: "assigned_user") } - - subject { Note.create_assignee_change_note(thing, project, author, assignee) } - - context 'creates and saves a Note' do - it { is_expected.to be_a Note } - - describe '#id' do - subject { super().id } - it { is_expected.not_to be_nil } - end - end - - describe '#noteable' do - subject { super().noteable } - it { is_expected.to eq(thing) } - end - - describe '#project' do - subject { super().project } - it { is_expected.to eq(thing.project) } - end - - describe '#author' do - subject { super().author } - it { is_expected.to eq(author) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq('Reassigned to @assigned_user') } - end - - context 'assignee is removed' do - let(:assignee) { nil } - - describe '#note' do - subject { super().note } - it { is_expected.to eq('Assignee removed') } - end - end - end - - describe '#create_labels_change_note' do - let(:project) { create(:project) } - let(:thing) { create(:issue, project: project) } - let(:author) { create(:user) } - let(:label1) { create(:label) } - let(:label2) { create(:label) } - let(:added_labels) { [label1, label2] } - let(:removed_labels) { [] } - - subject { Note.create_labels_change_note(thing, project, author, added_labels, removed_labels) } - - context 'creates and saves a Note' do - it { is_expected.to be_a Note } - - describe '#id' do - subject { super().id } - it { is_expected.not_to be_nil } - end - end - - describe '#noteable' do - subject { super().noteable } - it { is_expected.to eq(thing) } - end - - describe '#project' do - subject { super().project } - it { is_expected.to eq(thing.project) } - end - - describe '#author' do - subject { super().author } - it { is_expected.to eq(author) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("Added ~#{label1.id} ~#{label2.id} labels") } - end - - context 'label is removed' do - let(:added_labels) { [label1] } - let(:removed_labels) { [label2] } - - describe '#note' do - subject { super().note } - it { is_expected.to eq("Added ~#{label1.id} and removed ~#{label2.id} labels") } - end - end - end - - describe '#create_milestone_change_note' do - let(:project) { create(:project) } - let(:thing) { create(:issue, project: project) } - let(:milestone) { create(:milestone, project: project, title: "first_milestone") } - let(:author) { create(:user) } - - subject { Note.create_milestone_change_note(thing, project, author, milestone) } - - context 'creates and saves a Note' do - it { is_expected.to be_a Note } - - describe '#id' do - subject { super().id } - it { is_expected.not_to be_nil } - end - end - - describe '#project' do - subject { super().project } - it { is_expected.to eq(thing.project) } - end - - describe '#author' do - subject { super().author } - it { is_expected.to eq(author) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("Milestone changed to first_milestone") } - end - end - describe '#create_cross_reference_note' do let(:project) { create(:project) } let(:author) { create(:user) } @@ -476,70 +304,6 @@ describe Note do end end - describe '#cross_reference_exists?' do - let(:project) { create :project } - let(:author) { create :user } - let(:issue) { create :issue } - let(:commit0) { project.commit } - let(:commit1) { project.commit('HEAD~2') } - - before do - Note.create_cross_reference_note(issue, commit0, author) - end - - it 'detects if a mentionable has already been mentioned' do - expect(Note.cross_reference_exists?(issue, commit0)).to be_truthy - end - - it 'detects if a mentionable has not already been mentioned' do - expect(Note.cross_reference_exists?(issue, commit1)).to be_falsey - end - - context 'commit on commit' do - before do - Note.create_cross_reference_note(commit0, commit1, author) - end - - it { expect(Note.cross_reference_exists?(commit0, commit1)).to be_truthy } - it { expect(Note.cross_reference_exists?(commit1, commit0)).to be_falsey } - end - - context 'legacy note with Markdown emphasis' do - let(:issue2) { create :issue, project: project } - let!(:note) do - create :note, system: true, noteable_id: issue2.id, - noteable_type: "Issue", note: "_mentioned in issue " \ - "#{issue.project.path_with_namespace}##{issue.iid}_" - end - - it 'detects if a mentionable with emphasis has been mentioned' do - expect(Note.cross_reference_exists?(issue2, issue)).to be_truthy - end - end - end - - describe '#cross_references_with_underscores?' do - let(:project) { create :project, path: "first_project" } - let(:second_project) { create :project, path: "second_project" } - - let(:author) { create :user } - let(:issue0) { create :issue, project: project } - let(:issue1) { create :issue, project: second_project } - let!(:note) { Note.create_cross_reference_note(issue0, issue1, author) } - - it 'detects if a mentionable has already been mentioned' do - expect(Note.cross_reference_exists?(issue0, issue1)).to be_truthy - end - - it 'detects if a mentionable has not already been mentioned' do - expect(Note.cross_reference_exists?(issue1, issue0)).to be_falsey - end - - it 'detects that text has underscores' do - expect(note.note).to eq("mentioned in issue #{second_project.path_with_namespace}##{issue1.iid}") - end - end - describe '#system?' do let(:project) { create(:project) } let(:issue) { create(:issue, project: project) } @@ -554,30 +318,10 @@ describe Note do expect(@note).not_to be_system end - it 'should identify status-change notes as system notes' do - @note = Note.create_status_change_note(issue, project, author, 'closed', nil) - expect(@note).to be_system - end - it 'should identify cross-reference notes as system notes' do @note = Note.create_cross_reference_note(issue, other, author) expect(@note).to be_system end - - it 'should identify assignee-change notes as system notes' do - @note = Note.create_assignee_change_note(issue, project, author, assignee) - expect(@note).to be_system - end - - it 'should identify label-change notes as system notes' do - @note = Note.create_labels_change_note(issue, project, author, [label], []) - expect(@note).to be_system - end - - it 'should identify milestone-change notes as system notes' do - @note = Note.create_milestone_change_note(issue, project, author, milestone) - expect(@note).to be_system - end end describe :authorization do diff --git a/spec/services/system_note_service_spec.rb b/spec/services/system_note_service_spec.rb new file mode 100644 index 0000000000..cda4e7698f --- /dev/null +++ b/spec/services/system_note_service_spec.rb @@ -0,0 +1,353 @@ +require 'spec_helper' + +describe SystemNoteService do + let(:project) { create(:project) } + let(:author) { create(:user) } + let(:noteable) { create(:issue, project: project) } + + shared_examples_for 'a system note' do + it 'sets the noteable model' do + expect(subject.noteable).to eq noteable + end + + it 'sets the project' do + expect(subject.project).to eq project + end + + it 'sets the author' do + expect(subject.author).to eq author + end + + it 'is a system note' do + expect(subject).to be_system + end + end + + describe '.assignee_change' do + let(:assignee) { create(:user) } + + subject { described_class.assignee_change(noteable, project, author, assignee) } + + it_behaves_like 'a system note' + + context 'when assignee added' do + it 'sets the note text' do + expect(subject.note).to eq "Reassigned to @#{assignee.username}" + end + end + + context 'when assignee removed' do + let(:assignee) { nil } + + it 'sets the note text' do + expect(subject.note).to eq 'Assignee removed' + end + end + end + + describe '.cross_reference' do + let(:mentioner) { create(:issue, project: project) } + + subject { described_class.cross_reference(noteable, mentioner, author) } + + it_behaves_like 'a system note' + + context 'when cross-reference disallowed' do + before do + expect(described_class).to receive(:cross_reference_disallowed?).and_return(true) + end + + it 'returns nil' do + expect(subject).to be_nil + end + end + + context 'when cross-reference allowed' do + before do + expect(described_class).to receive(:cross_reference_disallowed?).and_return(false) + end + + describe 'note_body' do + context 'cross-project' do + let(:project2) { create(:project) } + let(:mentioner) { create(:issue, project: project2) } + + context 'from Commit' do + let(:mentioner) { project2.repository.commit } + + it 'references the mentioning commit' do + expect(subject.note).to eq "mentioned in commit #{project2.path_with_namespace}@#{mentioner.id}" + end + end + + context 'from non-Commit' do + it 'references the mentioning object' do + expect(subject.note).to eq "mentioned in issue #{project2.path_with_namespace}##{mentioner.iid}" + end + end + end + + context 'same project' do + context 'from Commit' do + let(:mentioner) { project.repository.commit } + + it 'references the mentioning commit' do + expect(subject.note).to eq "mentioned in commit #{mentioner.id}" + end + end + + context 'from non-Commit' do + it 'references the mentioning object' do + expect(subject.note).to eq "mentioned in issue ##{mentioner.iid}" + end + end + end + end + end + end + + describe '.label_change' do + let(:labels) { create_list(:label, 2) } + let(:added) { [] } + let(:removed) { [] } + + subject { described_class.label_change(noteable, project, author, added, removed) } + + it_behaves_like 'a system note' + + context 'with added labels' do + let(:added) { labels } + let(:removed) { [] } + + it 'sets the note text' do + expect(subject.note).to eq "Added ~#{labels[0].id} ~#{labels[1].id} labels" + end + end + + context 'with removed labels' do + let(:added) { [] } + let(:removed) { labels } + + it 'sets the note text' do + expect(subject.note).to eq "Removed ~#{labels[0].id} ~#{labels[1].id} labels" + end + end + + context 'with added and removed labels' do + let(:added) { [labels[0]] } + let(:removed) { [labels[1]] } + + it 'sets the note text' do + expect(subject.note).to eq "Added ~#{labels[0].id} and removed ~#{labels[1].id} labels" + end + end + end + + describe '.milestone_change' do + let(:milestone) { create(:milestone, project: project) } + + subject { described_class.milestone_change(noteable, project, author, milestone) } + + it_behaves_like 'a system note' + + context 'when milestone added' do + it 'sets the note text' do + expect(subject.note).to eq "Milestone changed to #{milestone.title}" + end + end + + context 'when milestone removed' do + let(:milestone) { nil } + + it 'sets the note text' do + expect(subject.note).to eq 'Milestone removed' + end + end + end + + describe '.commit_add' do + let(:noteable) { create(:merge_request, source_project: project) } + let(:new_commits) { noteable.commits } + let(:old_commits) { [] } + let(:oldrev) { nil } + + subject { described_class.commit_add(noteable, project, author, new_commits, old_commits, oldrev) } + + it_behaves_like 'a system note' + + describe 'note body' do + let(:note_lines) { subject.note.split("\n").reject(&:blank?) } + + context 'without existing commits' do + it 'adds a message header' do + expect(note_lines[0]).to eq "Added #{new_commits.size} commits:" + end + + it 'adds a message line for each commit' do + new_commits.each_with_index do |commit, i| + # Skip the header + expect(note_lines[i + 1]).to eq "* #{commit.short_id} - #{commit.title}" + end + end + end + + describe 'summary line for existing commits' do + let(:summary_line) { note_lines[1] } + + context 'with one existing commit' do + let(:old_commits) { [noteable.commits.last] } + + it 'includes the existing commit' do + expect(summary_line).to eq "* #{old_commits.first.short_id} - 1 commit from branch `feature`" + end + end + + context 'with multiple existing commits' do + let(:old_commits) { noteable.commits[3..-1] } + + context 'with oldrev' do + let(:oldrev) { noteable.commits[2].id } + + it 'includes a commit range' do + expect(summary_line).to start_with "* #{Commit.truncate_sha(oldrev)}...#{old_commits.last.short_id}" + end + + it 'includes a commit count' do + expect(summary_line).to end_with " - 2 commits from branch `feature`" + end + end + + context 'without oldrev' do + it 'includes a commit range' do + expect(summary_line).to start_with "* #{old_commits[0].short_id}..#{old_commits[-1].short_id}" + end + + it 'includes a commit count' do + expect(summary_line).to end_with " - 2 commits from branch `feature`" + end + end + + context 'on a fork' do + before do + expect(noteable).to receive(:for_fork?).and_return(true) + end + + it 'includes the project namespace' do + expect(summary_line).to end_with "`#{noteable.target_project_namespace}:feature`" + end + end + end + end + end + end + + describe '.status_change' do + let(:status) { 'new_status' } + let(:source) { nil } + + subject { described_class.status_change(noteable, project, author, status, source) } + + it_behaves_like 'a system note' + + context 'with a source' do + let(:source) { double('commit', gfm_reference: 'commit 123456') } + + it 'sets the note text' do + expect(subject.note).to eq "Status changed to #{status} by commit 123456" + end + end + + context 'without a source' do + it 'sets the note text' do + expect(subject.note).to eq "Status changed to #{status}" + end + end + end + + describe '.cross_reference?' do + it 'is truthy when text begins with expected text' do + expect(described_class.cross_reference?('mentioned in issue #1')).to be_truthy + end + + it 'is falsey when text does not begin with expected text' do + expect(described_class.cross_reference?('this is a note')).to be_falsey + end + end + + describe '.cross_reference_disallowed?' + + describe '.cross_reference_exists?' do + let(:commit0) { project.commit } + let(:commit1) { project.commit('HEAD~2') } + + context 'issue from commit' do + before do + # Mention issue (noteable) from commit0 + described_class.cross_reference(noteable, commit0, author) + end + + it 'is truthy when already mentioned' do + expect(described_class.cross_reference_exists?(noteable, commit0)). + to be_truthy + end + + it 'is falsey when not already mentioned' do + expect(described_class.cross_reference_exists?(noteable, commit1)). + to be_falsey + end + end + + context 'commit from commit' do + before do + # Mention commit1 from commit0 + described_class.cross_reference(commit0, commit1, author) + end + + it 'is truthy when already mentioned' do + expect(described_class.cross_reference_exists?(commit0, commit1)). + to be_truthy + end + + it 'is falsey when not already mentioned' do + expect(described_class.cross_reference_exists?(commit1, commit0)). + to be_falsey + end + end + + context 'legacy note with Markdown emphasis' do + let(:mentioner) { create(:issue, project: project) } + + before do + note = "_mentioned in issue ##{mentioner.iid}_" + create(:system_note, noteable: noteable, note: note, project: project) + end + + it 'detects if a mentionable with emphasis has been mentioned' do + expect(described_class.cross_reference_exists?(noteable, mentioner)). + to be_truthy + end + + context 'when referenced project has underscores' do + let(:project) { create(:empty_project, path: 'first_project') } + let(:project2) { create(:empty_project, path: 'second_project') } + + let(:issue) { mentioner } + let(:issue2) { create(:issue, project: project2) } + + before do + described_class.cross_reference(issue, issue2, author) + end + + it 'is truthy when already mentioned' do + expect(described_class.cross_reference_exists?(issue, issue2)). + to be_truthy + end + + it 'is falsey when not already mentioned' do + expect(described_class.cross_reference_exists?(issue2, issue)). + to be_falsey + end + end + end + end +end From cdb69d728c208ba48ccaede6e0ee086ea224d852 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 14:12:43 -0400 Subject: [PATCH 131/158] Add migration to convert legacy system notes --- ...50509180749_convert_legacy_reference_notes.rb | 16 ++++++++++++++++ db/schema.rb | 2 +- 2 files changed, 17 insertions(+), 1 deletion(-) create mode 100644 db/migrate/20150509180749_convert_legacy_reference_notes.rb diff --git a/db/migrate/20150509180749_convert_legacy_reference_notes.rb b/db/migrate/20150509180749_convert_legacy_reference_notes.rb new file mode 100644 index 0000000000..b02605489b --- /dev/null +++ b/db/migrate/20150509180749_convert_legacy_reference_notes.rb @@ -0,0 +1,16 @@ +# Convert legacy Markdown-emphasized notes to the current, non-emphasized format +# +# _mentioned in 54f7727c850972f0401c1312a7c4a6a380de5666_ +# +# becomes +# +# mentioned in 54f7727c850972f0401c1312a7c4a6a380de5666 +class ConvertLegacyReferenceNotes < ActiveRecord::Migration + def up + execute %q{UPDATE notes SET note = trim(both '_' from note) WHERE system = true AND note LIKE '\_%\_'} + end + + def down + # noop + end +end diff --git a/db/schema.rb b/db/schema.rb index 04abf9bb9a..199b959a8d 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -11,7 +11,7 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema.define(version: 20150502064022) do +ActiveRecord::Schema.define(version: 20150509180749) do # These are extensions that must be enabled in order to support this database enable_extension "plpgsql" From ff3a62aad171e4211dd9c5c6e762b1b32f9921ac Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 18:09:34 -0400 Subject: [PATCH 132/158] Remove legacy special case for emphasized reference notes --- app/services/system_note_service.rb | 22 ++++++-------- spec/services/system_note_service_spec.rb | 36 ----------------------- 2 files changed, 9 insertions(+), 49 deletions(-) diff --git a/app/services/system_note_service.rb b/app/services/system_note_service.rb index 2486360b26..c4bc2339e7 100644 --- a/app/services/system_note_service.rb +++ b/app/services/system_note_service.rb @@ -196,7 +196,7 @@ class SystemNoteService end gfm_reference = mentioner_gfm_ref(noteable, mentioner, true) - notes = notes.where('note like ?', cross_reference_note_pattern(gfm_reference)) + notes = notes.where(note: cross_reference_note_content(gfm_reference)) notes.count > 0 end @@ -207,17 +207,14 @@ class SystemNoteService Note.create(args.merge(system: true)) end - def self.cross_reference_note_prefix - 'mentioned in ' - end - # Prepend the mentioner's namespaced project path to the GFM reference for # cross-project references. For same-project references, return the # unmodified GFM reference. def self.mentioner_gfm_ref(noteable, mentioner, cross_reference = false) - if mentioner.is_a?(Commit) && cross_reference - return mentioner.gfm_reference.sub('commit ', 'commit %') - end + # FIXME (rspeicher): This was breaking things. + # if mentioner.is_a?(Commit) && cross_reference + # return mentioner.gfm_reference.sub('commit ', 'commit %') + # end full_gfm_reference(mentioner.project, noteable.project, mentioner) end @@ -243,13 +240,12 @@ class SystemNoteService end end - def self.cross_reference_note_content(gfm_reference) - cross_reference_note_prefix + "#{gfm_reference}" + def self.cross_reference_note_prefix + 'mentioned in ' end - def self.cross_reference_note_pattern(gfm_reference) - # Older cross reference notes contained underscores for emphasis - "%" + cross_reference_note_content(gfm_reference) + "%" + def self.cross_reference_note_content(gfm_reference) + "#{cross_reference_note_prefix}#{gfm_reference}" end # Build an Array of lines detailing each commit added in a merge request diff --git a/spec/services/system_note_service_spec.rb b/spec/services/system_note_service_spec.rb index cda4e7698f..194687cbbd 100644 --- a/spec/services/system_note_service_spec.rb +++ b/spec/services/system_note_service_spec.rb @@ -313,41 +313,5 @@ describe SystemNoteService do to be_falsey end end - - context 'legacy note with Markdown emphasis' do - let(:mentioner) { create(:issue, project: project) } - - before do - note = "_mentioned in issue ##{mentioner.iid}_" - create(:system_note, noteable: noteable, note: note, project: project) - end - - it 'detects if a mentionable with emphasis has been mentioned' do - expect(described_class.cross_reference_exists?(noteable, mentioner)). - to be_truthy - end - - context 'when referenced project has underscores' do - let(:project) { create(:empty_project, path: 'first_project') } - let(:project2) { create(:empty_project, path: 'second_project') } - - let(:issue) { mentioner } - let(:issue2) { create(:issue, project: project2) } - - before do - described_class.cross_reference(issue, issue2, author) - end - - it 'is truthy when already mentioned' do - expect(described_class.cross_reference_exists?(issue, issue2)). - to be_truthy - end - - it 'is falsey when not already mentioned' do - expect(described_class.cross_reference_exists?(issue2, issue)). - to be_falsey - end - end - end end end From 686f6855c25ecfc09fdf199cb7e3fb7dd787ed28 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 18:18:50 -0400 Subject: [PATCH 133/158] Update SystemNoteService method naming conventions Now the verb comes first, and there is no restriction on singular/plural. --- app/services/issuable_base_service.rb | 6 +- app/services/issues/close_service.rb | 2 +- app/services/issues/reopen_service.rb | 2 +- app/services/merge_requests/base_service.rb | 2 +- .../merge_requests/refresh_service.rb | 6 +- app/services/system_note_service.rb | 220 +++++++------ spec/services/system_note_service_spec.rb | 293 +++++++++--------- 7 files changed, 264 insertions(+), 267 deletions(-) diff --git a/app/services/issuable_base_service.rb b/app/services/issuable_base_service.rb index 4d251cb30d..8960235b09 100644 --- a/app/services/issuable_base_service.rb +++ b/app/services/issuable_base_service.rb @@ -2,17 +2,17 @@ class IssuableBaseService < BaseService private def create_assignee_note(issuable) - SystemNoteService.assignee_change( + SystemNoteService.change_assignee( issuable, issuable.project, current_user, issuable.assignee) end def create_milestone_note(issuable) - SystemNoteService.milestone_change( + SystemNoteService.change_milestone( issuable, issuable.project, current_user, issuable.milestone) end def create_labels_note(issuable, added_labels, removed_labels) - SystemNoteService.label_change( + SystemNoteService.change_label( issuable, issuable.project, current_user, added_labels, removed_labels) end end diff --git a/app/services/issues/close_service.rb b/app/services/issues/close_service.rb index bdc13685fd..138465859c 100644 --- a/app/services/issues/close_service.rb +++ b/app/services/issues/close_service.rb @@ -14,7 +14,7 @@ module Issues private def create_note(issue, current_commit) - SystemNoteService.status_change(issue, issue.project, current_user, issue.state, current_commit) + SystemNoteService.change_status(issue, issue.project, current_user, issue.state, current_commit) end end end diff --git a/app/services/issues/reopen_service.rb b/app/services/issues/reopen_service.rb index fbbf573e5a..e48ca359f4 100644 --- a/app/services/issues/reopen_service.rb +++ b/app/services/issues/reopen_service.rb @@ -14,7 +14,7 @@ module Issues private def create_note(issue) - SystemNoteService.status_change(issue, issue.project, current_user, issue.state, nil) + SystemNoteService.change_status(issue, issue.project, current_user, issue.state, nil) end end end diff --git a/app/services/merge_requests/base_service.rb b/app/services/merge_requests/base_service.rb index 9b470880ad..e455fe9579 100644 --- a/app/services/merge_requests/base_service.rb +++ b/app/services/merge_requests/base_service.rb @@ -2,7 +2,7 @@ module MergeRequests class BaseService < ::IssuableBaseService def create_note(merge_request) - SystemNoteService.status_change(merge_request, merge_request.target_project, current_user, merge_request.state, nil) + SystemNoteService.change_status(merge_request, merge_request.target_project, current_user, merge_request.state, nil) end def hook_data(merge_request, action) diff --git a/app/services/merge_requests/refresh_service.rb b/app/services/merge_requests/refresh_service.rb index 1d57fd11de..66610a08a4 100644 --- a/app/services/merge_requests/refresh_service.rb +++ b/app/services/merge_requests/refresh_service.rb @@ -82,9 +82,9 @@ module MergeRequests mr_commit_ids.include?(commit.id) end - SystemNoteService.commit_add(merge_request, merge_request.project, - @current_user, new_commits, - existing_commits, @oldrev) + SystemNoteService.add_commits(merge_request, merge_request.project, + @current_user, new_commits, + existing_commits, @oldrev) end end diff --git a/app/services/system_note_service.rb b/app/services/system_note_service.rb index c4bc2339e7..a1de050ca5 100644 --- a/app/services/system_note_service.rb +++ b/app/services/system_note_service.rb @@ -2,11 +2,30 @@ # # Used for creating system notes (e.g., when a user references a merge request # from an issue, an issue's assignee changes, an issue is closed, etc.) -# -# All methods creating notes should be named using a singular noun and -# present-tense verb (e.g., "assignee_change" not "assignee_changed", -# "label_change" not "labels_change"). class SystemNoteService + # Called when commits are added to a Merge Request + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # new_commits - Array of Commits added since last push + # existing_commits - Array of Commits added in a previous push + # oldrev - TODO (rspeicher): I have no idea what this actually does + # + # See new_commit_summary and existing_commit_summary. + # + # Returns the created Note object + def self.add_commits(noteable, project, author, new_commits, existing_commits = [], oldrev = nil) + total_count = new_commits.length + existing_commits.length + commits_text = "#{total_count} commit".pluralize(total_count) + + body = "Added #{commits_text}:\n\n" + body << existing_commit_summary(noteable, existing_commits, oldrev) + body << new_commit_summary(new_commits).join("\n") + + create_note(noteable: noteable, project: project, author: author, note: body) + end + # Called when the assignee of a Noteable is changed or removed # # noteable - Noteable object @@ -21,12 +40,96 @@ class SystemNoteService # "Reassigned to @rspeicher" # # Returns the created Note object - def self.assignee_change(noteable, project, author, assignee) + def self.change_assignee(noteable, project, author, assignee) body = assignee.nil? ? 'Assignee removed' : "Reassigned to @#{assignee.username}" create_note(noteable: noteable, project: project, author: author, note: body) end + # Called when one or more labels on a Noteable are added and/or removed + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # added_labels - Array of Labels added + # removed_labels - Array of Labels removed + # + # Example Note text: + # + # "Added ~1 and removed ~2 ~3 labels" + # + # "Added ~4 label" + # + # "Removed ~5 label" + # + # Returns the created Note object + def self.change_label(noteable, project, author, added_labels, removed_labels) + labels_count = added_labels.count + removed_labels.count + + references = ->(label) { "~#{label.id}" } + added_labels = added_labels.map(&references).join(' ') + removed_labels = removed_labels.map(&references).join(' ') + + body = '' + + if added_labels.present? + body << "added #{added_labels}" + body << ' and ' if removed_labels.present? + end + + if removed_labels.present? + body << "removed #{removed_labels}" + end + + body << ' ' << 'label'.pluralize(labels_count) + body = "#{body.capitalize}" + + create_note(noteable: noteable, project: project, author: author, note: body) + end + + # Called when the milestone of a Noteable is changed + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # milestone - Milestone being assigned, or nil + # + # Example Note text: + # + # "Milestone removed" + # + # "Miletone changed to 7.11" + # + # Returns the created Note object + def self.change_milestone(noteable, project, author, milestone) + body = 'Milestone ' + body += milestone.nil? ? 'removed' : "changed to #{milestone.title}" + + create_note(noteable: noteable, project: project, author: author, note: body) + end + + # Called when the status of a Noteable is changed + # + # noteable - Noteable object + # project - Project owning noteable + # author - User performing the change + # status - String status + # source - Mentionable performing the change, or nil + # + # Example Note text: + # + # "Status changed to merged" + # + # "Status changed to closed by bc17db76" + # + # Returns the created Note object + def self.change_status(noteable, project, author, status, source) + body = "Status changed to #{status}" + body += " by #{source.gfm_reference}" if source + + create_note(noteable: noteable, project: project, author: author, note: body) + end + # Called when a Mentionable references a Noteable # # noteable - Noteable object being referenced @@ -64,113 +167,6 @@ class SystemNoteService create_note(note_options) end - # Called when one or more labels on a Noteable are added and/or removed - # - # noteable - Noteable object - # project - Project owning noteable - # author - User performing the change - # added_labels - Array of Labels added - # removed_labels - Array of Labels removed - # - # Example Note text: - # - # "Added ~1 and removed ~2 ~3 labels" - # - # "Added ~4 label" - # - # "Removed ~5 label" - # - # Returns the created Note object - def self.label_change(noteable, project, author, added_labels, removed_labels) - labels_count = added_labels.count + removed_labels.count - - references = ->(label) { "~#{label.id}" } - added_labels = added_labels.map(&references).join(' ') - removed_labels = removed_labels.map(&references).join(' ') - - body = '' - - if added_labels.present? - body << "added #{added_labels}" - body << ' and ' if removed_labels.present? - end - - if removed_labels.present? - body << "removed #{removed_labels}" - end - - body << ' ' << 'label'.pluralize(labels_count) - body = "#{body.capitalize}" - - create_note(noteable: noteable, project: project, author: author, note: body) - end - - # Called when the milestone of a Noteable is changed - # - # noteable - Noteable object - # project - Project owning noteable - # author - User performing the change - # milestone - Milestone being assigned, or nil - # - # Example Note text: - # - # "Milestone removed" - # - # "Miletone changed to 7.11" - # - # Returns the created Note object - def self.milestone_change(noteable, project, author, milestone) - body = 'Milestone ' - body += milestone.nil? ? 'removed' : "changed to #{milestone.title}" - - create_note(noteable: noteable, project: project, author: author, note: body) - end - - # Called when commits are added to a Merge Request - # - # noteable - Noteable object - # project - Project owning noteable - # author - User performing the change - # new_commits - Array of Commits added since last push - # existing_commits - Array of Commits added in a previous push - # oldrev - TODO (rspeicher): I have no idea what this actually does - # - # See new_commit_summary and existing_commit_summary. - # - # Returns the created Note object - def self.commit_add(noteable, project, author, new_commits, existing_commits = [], oldrev = nil) - total_count = new_commits.length + existing_commits.length - commits_text = "#{total_count} commit".pluralize(total_count) - - body = "Added #{commits_text}:\n\n" - body << existing_commit_summary(noteable, existing_commits, oldrev) - body << new_commit_summary(new_commits).join("\n") - - create_note(noteable: noteable, project: project, author: author, note: body) - end - - # Called when the status of a Noteable is changed - # - # noteable - Noteable object - # project - Project owning noteable - # author - User performing the change - # status - String status - # source - Mentionable performing the change, or nil - # - # Example Note text: - # - # "Status changed to merged" - # - # "Status changed to closed by bc17db76" - # - # Returns the created Note object - def self.status_change(noteable, project, author, status, source) - body = "Status changed to #{status}" - body += " by #{source.gfm_reference}" if source - - create_note(noteable: noteable, project: project, author: author, note: body) - end - def self.cross_reference?(note_text) note_text.start_with?(cross_reference_note_prefix) end diff --git a/spec/services/system_note_service_spec.rb b/spec/services/system_note_service_spec.rb index 194687cbbd..3e9528a83d 100644 --- a/spec/services/system_note_service_spec.rb +++ b/spec/services/system_note_service_spec.rb @@ -23,155 +23,13 @@ describe SystemNoteService do end end - describe '.assignee_change' do - let(:assignee) { create(:user) } - - subject { described_class.assignee_change(noteable, project, author, assignee) } - - it_behaves_like 'a system note' - - context 'when assignee added' do - it 'sets the note text' do - expect(subject.note).to eq "Reassigned to @#{assignee.username}" - end - end - - context 'when assignee removed' do - let(:assignee) { nil } - - it 'sets the note text' do - expect(subject.note).to eq 'Assignee removed' - end - end - end - - describe '.cross_reference' do - let(:mentioner) { create(:issue, project: project) } - - subject { described_class.cross_reference(noteable, mentioner, author) } - - it_behaves_like 'a system note' - - context 'when cross-reference disallowed' do - before do - expect(described_class).to receive(:cross_reference_disallowed?).and_return(true) - end - - it 'returns nil' do - expect(subject).to be_nil - end - end - - context 'when cross-reference allowed' do - before do - expect(described_class).to receive(:cross_reference_disallowed?).and_return(false) - end - - describe 'note_body' do - context 'cross-project' do - let(:project2) { create(:project) } - let(:mentioner) { create(:issue, project: project2) } - - context 'from Commit' do - let(:mentioner) { project2.repository.commit } - - it 'references the mentioning commit' do - expect(subject.note).to eq "mentioned in commit #{project2.path_with_namespace}@#{mentioner.id}" - end - end - - context 'from non-Commit' do - it 'references the mentioning object' do - expect(subject.note).to eq "mentioned in issue #{project2.path_with_namespace}##{mentioner.iid}" - end - end - end - - context 'same project' do - context 'from Commit' do - let(:mentioner) { project.repository.commit } - - it 'references the mentioning commit' do - expect(subject.note).to eq "mentioned in commit #{mentioner.id}" - end - end - - context 'from non-Commit' do - it 'references the mentioning object' do - expect(subject.note).to eq "mentioned in issue ##{mentioner.iid}" - end - end - end - end - end - end - - describe '.label_change' do - let(:labels) { create_list(:label, 2) } - let(:added) { [] } - let(:removed) { [] } - - subject { described_class.label_change(noteable, project, author, added, removed) } - - it_behaves_like 'a system note' - - context 'with added labels' do - let(:added) { labels } - let(:removed) { [] } - - it 'sets the note text' do - expect(subject.note).to eq "Added ~#{labels[0].id} ~#{labels[1].id} labels" - end - end - - context 'with removed labels' do - let(:added) { [] } - let(:removed) { labels } - - it 'sets the note text' do - expect(subject.note).to eq "Removed ~#{labels[0].id} ~#{labels[1].id} labels" - end - end - - context 'with added and removed labels' do - let(:added) { [labels[0]] } - let(:removed) { [labels[1]] } - - it 'sets the note text' do - expect(subject.note).to eq "Added ~#{labels[0].id} and removed ~#{labels[1].id} labels" - end - end - end - - describe '.milestone_change' do - let(:milestone) { create(:milestone, project: project) } - - subject { described_class.milestone_change(noteable, project, author, milestone) } - - it_behaves_like 'a system note' - - context 'when milestone added' do - it 'sets the note text' do - expect(subject.note).to eq "Milestone changed to #{milestone.title}" - end - end - - context 'when milestone removed' do - let(:milestone) { nil } - - it 'sets the note text' do - expect(subject.note).to eq 'Milestone removed' - end - end - end - - describe '.commit_add' do + describe '.add_commits' do let(:noteable) { create(:merge_request, source_project: project) } let(:new_commits) { noteable.commits } let(:old_commits) { [] } let(:oldrev) { nil } - subject { described_class.commit_add(noteable, project, author, new_commits, old_commits, oldrev) } + subject { described_class.add_commits(noteable, project, author, new_commits, old_commits, oldrev) } it_behaves_like 'a system note' @@ -241,11 +99,92 @@ describe SystemNoteService do end end - describe '.status_change' do + describe '.change_assignee' do + let(:assignee) { create(:user) } + + subject { described_class.change_assignee(noteable, project, author, assignee) } + + it_behaves_like 'a system note' + + context 'when assignee added' do + it 'sets the note text' do + expect(subject.note).to eq "Reassigned to @#{assignee.username}" + end + end + + context 'when assignee removed' do + let(:assignee) { nil } + + it 'sets the note text' do + expect(subject.note).to eq 'Assignee removed' + end + end + end + + describe '.change_label' do + let(:labels) { create_list(:label, 2) } + let(:added) { [] } + let(:removed) { [] } + + subject { described_class.change_label(noteable, project, author, added, removed) } + + it_behaves_like 'a system note' + + context 'with added labels' do + let(:added) { labels } + let(:removed) { [] } + + it 'sets the note text' do + expect(subject.note).to eq "Added ~#{labels[0].id} ~#{labels[1].id} labels" + end + end + + context 'with removed labels' do + let(:added) { [] } + let(:removed) { labels } + + it 'sets the note text' do + expect(subject.note).to eq "Removed ~#{labels[0].id} ~#{labels[1].id} labels" + end + end + + context 'with added and removed labels' do + let(:added) { [labels[0]] } + let(:removed) { [labels[1]] } + + it 'sets the note text' do + expect(subject.note).to eq "Added ~#{labels[0].id} and removed ~#{labels[1].id} labels" + end + end + end + + describe '.change_milestone' do + let(:milestone) { create(:milestone, project: project) } + + subject { described_class.change_milestone(noteable, project, author, milestone) } + + it_behaves_like 'a system note' + + context 'when milestone added' do + it 'sets the note text' do + expect(subject.note).to eq "Milestone changed to #{milestone.title}" + end + end + + context 'when milestone removed' do + let(:milestone) { nil } + + it 'sets the note text' do + expect(subject.note).to eq 'Milestone removed' + end + end + end + + describe '.change_status' do let(:status) { 'new_status' } let(:source) { nil } - subject { described_class.status_change(noteable, project, author, status, source) } + subject { described_class.change_status(noteable, project, author, status, source) } it_behaves_like 'a system note' @@ -264,6 +203,67 @@ describe SystemNoteService do end end + describe '.cross_reference' do + let(:mentioner) { create(:issue, project: project) } + + subject { described_class.cross_reference(noteable, mentioner, author) } + + it_behaves_like 'a system note' + + context 'when cross-reference disallowed' do + before do + expect(described_class).to receive(:cross_reference_disallowed?).and_return(true) + end + + it 'returns nil' do + expect(subject).to be_nil + end + end + + context 'when cross-reference allowed' do + before do + expect(described_class).to receive(:cross_reference_disallowed?).and_return(false) + end + + describe 'note_body' do + context 'cross-project' do + let(:project2) { create(:project) } + let(:mentioner) { create(:issue, project: project2) } + + context 'from Commit' do + let(:mentioner) { project2.repository.commit } + + it 'references the mentioning commit' do + expect(subject.note).to eq "mentioned in commit #{project2.path_with_namespace}@#{mentioner.id}" + end + end + + context 'from non-Commit' do + it 'references the mentioning object' do + expect(subject.note).to eq "mentioned in issue #{project2.path_with_namespace}##{mentioner.iid}" + end + end + end + + context 'same project' do + context 'from Commit' do + let(:mentioner) { project.repository.commit } + + it 'references the mentioning commit' do + expect(subject.note).to eq "mentioned in commit #{mentioner.id}" + end + end + + context 'from non-Commit' do + it 'references the mentioning object' do + expect(subject.note).to eq "mentioned in issue ##{mentioner.iid}" + end + end + end + end + end + end + describe '.cross_reference?' do it 'is truthy when text begins with expected text' do expect(described_class.cross_reference?('mentioned in issue #1')).to be_truthy @@ -274,6 +274,7 @@ describe SystemNoteService do end end + # TODO (rspeicher) describe '.cross_reference_disallowed?' describe '.cross_reference_exists?' do From b94c6d9f5b48c9ea42cb3098c9b8de8c47fa76a2 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 9 May 2015 18:19:41 -0400 Subject: [PATCH 134/158] Check if `system` is truthy in `Note.cross_reference?` --- app/models/note.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/models/note.rb b/app/models/note.rb index f2470e5f39..6939a7e73a 100644 --- a/app/models/note.rb +++ b/app/models/note.rb @@ -99,7 +99,7 @@ class Note < ActiveRecord::Base end def cross_reference? - SystemNoteService.cross_reference?(note) + system && SystemNoteService.cross_reference?(note) end def max_attachment_size From 19142f407990b7b8a8d74c68ba30bef066e01aa4 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sun, 10 May 2015 22:57:44 -0400 Subject: [PATCH 135/158] Simplify Note model specs --- spec/factories/notes.rb | 4 +- spec/models/note_spec.rb | 247 ++++++--------------------------------- 2 files changed, 36 insertions(+), 215 deletions(-) diff --git a/spec/factories/notes.rb b/spec/factories/notes.rb index 1e4c0b03f7..e1009d5916 100644 --- a/spec/factories/notes.rb +++ b/spec/factories/notes.rb @@ -34,7 +34,7 @@ FactoryGirl.define do factory :system_note, traits: [:system] trait :on_commit do - project factory: :project + project commit_id RepoHelpers.sample_commit.id noteable_type "Commit" end @@ -44,7 +44,7 @@ FactoryGirl.define do end trait :on_merge_request do - project factory: :project + project noteable_id 1 noteable_type "MergeRequest" end diff --git a/spec/models/note_spec.rb b/spec/models/note_spec.rb index 4a769f3677..ddacba5826 100644 --- a/spec/models/note_spec.rb +++ b/spec/models/note_spec.rb @@ -20,20 +20,44 @@ require 'spec_helper' describe Note do - describe "Associations" do + describe 'associations' do it { is_expected.to belong_to(:project) } it { is_expected.to belong_to(:noteable) } it { is_expected.to belong_to(:author).class_name('User') } end - describe "Mass assignment" do - end - - describe "Validation" do + describe 'validation' do it { is_expected.to validate_presence_of(:note) } it { is_expected.to validate_presence_of(:project) } end + describe '#votable?' do + it 'is true for issue notes' do + note = build(:note_on_issue) + expect(note).to be_votable + end + + it 'is true for merge request notes' do + note = build(:note_on_merge_request) + expect(note).to be_votable + end + + it 'is false for merge request diff notes' do + note = build(:note_on_merge_request_diff) + expect(note).not_to be_votable + end + + it 'is false for commit notes' do + note = build(:note_on_commit) + expect(note).not_to be_votable + end + + it 'is false for commit diff notes' do + note = build(:note_on_commit_diff) + expect(note).not_to be_votable + end + end + describe 'voting score' do it 'recognizes a neutral note' do note = build(:votable_note, note: 'This is not a +1 note') @@ -78,8 +102,6 @@ describe Note do end end - let(:project) { create(:project) } - describe "Commit notes" do let!(:note) { create(:note_on_commit, note: "+1 from me") } let!(:commit) { note.noteable } @@ -98,10 +120,6 @@ describe Note do it "should be recognized by #for_commit?" do expect(note).to be_for_commit end - - it "should not be votable" do - expect(note).not_to be_votable - end end describe "Commit diff line notes" do @@ -126,205 +144,7 @@ describe Note do end end - describe "Issue notes" do - let!(:note) { create(:note_on_issue, note: "+1 from me") } - - it "should not be votable" do - expect(note).to be_votable - end - end - - describe "Merge request notes" do - let!(:note) { create(:note_on_merge_request, note: "+1 from me") } - - it "should be votable" do - expect(note).to be_votable - end - end - - describe "Merge request diff line notes" do - let!(:note) { create(:note_on_merge_request_diff, note: "+1 from me") } - - it "should not be votable" do - expect(note).not_to be_votable - end - end - - describe '#create_cross_reference_note' do - let(:project) { create(:project) } - let(:author) { create(:user) } - let(:issue) { create(:issue, project: project) } - let(:mergereq) { create(:merge_request, :simple, target_project: project, source_project: project) } - let(:commit) { project.commit } - - # Test all of {issue, merge request, commit} in both the referenced and referencing - # roles, to ensure that the correct information can be inferred from any argument. - - context 'issue from a merge request' do - subject { Note.create_cross_reference_note(issue, mergereq, author) } - - it { is_expected.to be_valid } - - describe '#noteable' do - subject { super().noteable } - it { is_expected.to eq(issue) } - end - - describe '#project' do - subject { super().project } - it { is_expected.to eq(issue.project) } - end - - describe '#author' do - subject { super().author } - it { is_expected.to eq(author) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("mentioned in merge request !#{mergereq.iid}") } - end - end - - context 'issue from a commit' do - subject { Note.create_cross_reference_note(issue, commit, author) } - - it { is_expected.to be_valid } - - describe '#noteable' do - subject { super().noteable } - it { is_expected.to eq(issue) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("mentioned in commit #{commit.sha}") } - end - end - - context 'merge request from an issue' do - subject { Note.create_cross_reference_note(mergereq, issue, author) } - - it { is_expected.to be_valid } - - describe '#noteable' do - subject { super().noteable } - it { is_expected.to eq(mergereq) } - end - - describe '#project' do - subject { super().project } - it { is_expected.to eq(mergereq.project) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("mentioned in issue ##{issue.iid}") } - end - end - - context 'commit from a merge request' do - subject { Note.create_cross_reference_note(commit, mergereq, author) } - - it { is_expected.to be_valid } - - describe '#noteable' do - subject { super().noteable } - it { is_expected.to eq(commit) } - end - - describe '#project' do - subject { super().project } - it { is_expected.to eq(project) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("mentioned in merge request !#{mergereq.iid}") } - end - end - - context 'commit contained in a merge request' do - subject { Note.create_cross_reference_note(mergereq.commits.first, mergereq, author) } - - it { is_expected.to be_nil } - end - - context 'commit from issue' do - subject { Note.create_cross_reference_note(commit, issue, author) } - - it { is_expected.to be_valid } - - describe '#noteable_type' do - subject { super().noteable_type } - it { is_expected.to eq("Commit") } - end - - describe '#noteable_id' do - subject { super().noteable_id } - it { is_expected.to be_nil } - end - - describe '#commit_id' do - subject { super().commit_id } - it { is_expected.to eq(commit.id) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("mentioned in issue ##{issue.iid}") } - end - end - - context 'commit from commit' do - let(:parent_commit) { commit.parents.first } - subject { Note.create_cross_reference_note(commit, parent_commit, author) } - - it { is_expected.to be_valid } - - describe '#noteable_type' do - subject { super().noteable_type } - it { is_expected.to eq("Commit") } - end - - describe '#noteable_id' do - subject { super().noteable_id } - it { is_expected.to be_nil } - end - - describe '#commit_id' do - subject { super().commit_id } - it { is_expected.to eq(commit.id) } - end - - describe '#note' do - subject { super().note } - it { is_expected.to eq("mentioned in commit #{parent_commit.id}") } - end - end - end - - describe '#system?' do - let(:project) { create(:project) } - let(:issue) { create(:issue, project: project) } - let(:other) { create(:issue, project: project) } - let(:author) { create(:user) } - let(:assignee) { create(:user) } - let(:label) { create(:label) } - let(:milestone) { create(:milestone) } - - it 'should recognize user-supplied notes as non-system' do - @note = create(:note_on_issue) - expect(@note).not_to be_system - end - - it 'should identify cross-reference notes as system notes' do - @note = Note.create_cross_reference_note(issue, other, author) - expect(@note).to be_system - end - end - - describe :authorization do + describe 'authorization' do before do @p1 = create(:project) @p2 = create(:project) @@ -335,7 +155,7 @@ describe Note do @abilities << Ability end - describe :read do + describe 'read' do before do @p1.project_members.create(user: @u2, access_level: ProjectMember::GUEST) @p2.project_members.create(user: @u3, access_level: ProjectMember::GUEST) @@ -346,7 +166,7 @@ describe Note do it { expect(@abilities.allowed?(@u3, :read_note, @p1)).to be_falsey } end - describe :write do + describe 'write' do before do @p1.project_members.create(user: @u2, access_level: ProjectMember::DEVELOPER) @p2.project_members.create(user: @u3, access_level: ProjectMember::DEVELOPER) @@ -357,7 +177,7 @@ describe Note do it { expect(@abilities.allowed?(@u3, :write_note, @p1)).to be_falsey } end - describe :admin do + describe 'admin' do before do @p1.project_members.create(user: @u1, access_level: ProjectMember::REPORTER) @p1.project_members.create(user: @u2, access_level: ProjectMember::MASTER) @@ -373,6 +193,7 @@ describe Note do it_behaves_like 'an editable mentionable' do subject { create :note, noteable: issue, project: project } + let(:project) { create(:project) } let(:issue) { create :issue, project: project } let(:backref_text) { issue.gfm_reference } let(:set_mentionable_text) { ->(txt) { subject.note = txt } } From 83904275831511f6b17b33064255b669604e0e74 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sun, 10 May 2015 23:51:49 -0400 Subject: [PATCH 136/158] Spec SystemNoteService.cross_reference_disallowed? --- spec/services/system_note_service_spec.rb | 28 +++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/spec/services/system_note_service_spec.rb b/spec/services/system_note_service_spec.rb index 3e9528a83d..6ddec8e67b 100644 --- a/spec/services/system_note_service_spec.rb +++ b/spec/services/system_note_service_spec.rb @@ -274,8 +274,32 @@ describe SystemNoteService do end end - # TODO (rspeicher) - describe '.cross_reference_disallowed?' + describe '.cross_reference_disallowed?' do + context 'when mentioner is not a MergeRequest' do + it 'is falsey' do + mentioner = noteable.dup + expect(described_class.cross_reference_disallowed?(noteable, mentioner)). + to be_falsey + end + end + + context 'when mentioner is a MergeRequest' do + let(:mentioner) { create(:merge_request, :simple, source_project: project) } + let(:noteable) { project.commit } + + it 'is truthy when noteable is in commits' do + expect(mentioner).to receive(:commits).and_return([noteable]) + expect(described_class.cross_reference_disallowed?(noteable, mentioner)). + to be_truthy + end + + it 'is falsey when noteable is not in commits' do + expect(mentioner).to receive(:commits).and_return([]) + expect(described_class.cross_reference_disallowed?(noteable, mentioner)). + to be_falsey + end + end + end describe '.cross_reference_exists?' do let(:commit0) { project.commit } From 3e8efce360e3dadac69554d0bdebdcfde918f6e7 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sun, 10 May 2015 23:52:03 -0400 Subject: [PATCH 137/158] Refactor SystemNoteService.cross_reference_disallowed? --- app/services/system_note_service.rb | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/app/services/system_note_service.rb b/app/services/system_note_service.rb index a1de050ca5..0614f8689a 100644 --- a/app/services/system_note_service.rb +++ b/app/services/system_note_service.rb @@ -171,13 +171,20 @@ class SystemNoteService note_text.start_with?(cross_reference_note_prefix) end - # Determine if cross reference note should be created. - # eg. mentioning a commit in MR comments which exists inside a MR - # should not create "mentioned in" note. + # Check if a cross-reference is disallowed + # + # This method prevents adding a "mentioned in !1" note on every single commit + # in a merge request. + # + # noteable - Noteable object being referenced + # mentioner - Mentionable object + # + # Returns Boolean def self.cross_reference_disallowed?(noteable, mentioner) - if mentioner.kind_of?(MergeRequest) - mentioner.commits.map(&:id).include? noteable.id - end + return false unless MergeRequest === mentioner + return false unless Commit === noteable + + mentioner.commits.include?(noteable) end def self.cross_reference_exists?(noteable, mentioner) From 54db527b4c44b102b81f4614bee6b06d2b742f6a Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sun, 10 May 2015 23:52:46 -0400 Subject: [PATCH 138/158] Reorganize SystemNoteService spec a bit --- spec/services/system_note_service_spec.rb | 30 +++++++++++++---------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/spec/services/system_note_service_spec.rb b/spec/services/system_note_service_spec.rb index 6ddec8e67b..4e4cb6d19e 100644 --- a/spec/services/system_note_service_spec.rb +++ b/spec/services/system_note_service_spec.rb @@ -6,6 +6,10 @@ describe SystemNoteService do let(:noteable) { create(:issue, project: project) } shared_examples_for 'a system note' do + it 'is valid' do + expect(subject).to be_valid + end + it 'sets the noteable model' do expect(subject.noteable).to eq noteable end @@ -24,13 +28,13 @@ describe SystemNoteService do end describe '.add_commits' do + subject { described_class.add_commits(noteable, project, author, new_commits, old_commits, oldrev) } + let(:noteable) { create(:merge_request, source_project: project) } let(:new_commits) { noteable.commits } let(:old_commits) { [] } let(:oldrev) { nil } - subject { described_class.add_commits(noteable, project, author, new_commits, old_commits, oldrev) } - it_behaves_like 'a system note' describe 'note body' do @@ -100,10 +104,10 @@ describe SystemNoteService do end describe '.change_assignee' do - let(:assignee) { create(:user) } - subject { described_class.change_assignee(noteable, project, author, assignee) } + let(:assignee) { create(:user) } + it_behaves_like 'a system note' context 'when assignee added' do @@ -122,12 +126,12 @@ describe SystemNoteService do end describe '.change_label' do + subject { described_class.change_label(noteable, project, author, added, removed) } + let(:labels) { create_list(:label, 2) } let(:added) { [] } let(:removed) { [] } - subject { described_class.change_label(noteable, project, author, added, removed) } - it_behaves_like 'a system note' context 'with added labels' do @@ -159,10 +163,10 @@ describe SystemNoteService do end describe '.change_milestone' do - let(:milestone) { create(:milestone, project: project) } - subject { described_class.change_milestone(noteable, project, author, milestone) } + let(:milestone) { create(:milestone, project: project) } + it_behaves_like 'a system note' context 'when milestone added' do @@ -181,11 +185,11 @@ describe SystemNoteService do end describe '.change_status' do + subject { described_class.change_status(noteable, project, author, status, source) } + let(:status) { 'new_status' } let(:source) { nil } - subject { described_class.change_status(noteable, project, author, status, source) } - it_behaves_like 'a system note' context 'with a source' do @@ -204,10 +208,10 @@ describe SystemNoteService do end describe '.cross_reference' do - let(:mentioner) { create(:issue, project: project) } - subject { described_class.cross_reference(noteable, mentioner, author) } + let(:mentioner) { create(:issue, project: project) } + it_behaves_like 'a system note' context 'when cross-reference disallowed' do @@ -245,7 +249,7 @@ describe SystemNoteService do end end - context 'same project' do + context 'within the same project' do context 'from Commit' do let(:mentioner) { project.repository.commit } From 0a47d8f3f418fe2e46c3bfe0f723ca0ff63e814c Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Mon, 11 May 2015 11:11:03 +0200 Subject: [PATCH 139/158] Follow shell command guidelines in Brakeman task There is no real issue here because the shell command is completely static, but it is still nicer to just follow the guidelines in doc/development/shell_commands.md . --- lib/tasks/brakeman.rake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/tasks/brakeman.rake b/lib/tasks/brakeman.rake index 3a225801ff..52a9b017e7 100644 --- a/lib/tasks/brakeman.rake +++ b/lib/tasks/brakeman.rake @@ -1,6 +1,6 @@ desc 'Security check via brakeman' task :brakeman do - if system("brakeman --skip-files lib/backup/repository.rb -w3 -z") + if system(*%W(brakeman --skip-files lib/backup/repository.rb -w3 -z)) puts 'Security check succeed' else puts 'Security check failed' From 783b286ac0ca43033d7b64e360ca6f1aeeafc0d0 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Mon, 11 May 2015 11:55:02 +0200 Subject: [PATCH 140/158] Don't symbolize params. --- app/controllers/uploads_controller.rb | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/app/controllers/uploads_controller.rb b/app/controllers/uploads_controller.rb index 17edff68be..28536e359e 100644 --- a/app/controllers/uploads_controller.rb +++ b/app/controllers/uploads_controller.rb @@ -52,13 +52,13 @@ class UploadsController < ApplicationController def upload_model upload_models = { - user: User, - project: Project, - note: Note, - group: Group + "user" => User, + "project" => Project, + "note" => Note, + "group" => Group } - upload_models[params[:model].to_sym] + upload_models[params[:model]] end def upload_mount From 2391a43a7073b4e723e9ca1f7ff19b30f8e51452 Mon Sep 17 00:00:00 2001 From: Jakub Jirutka Date: Sat, 14 Feb 2015 20:50:08 +0100 Subject: [PATCH 141/158] Make omniauth-kerberos dependency optional (add group 'kerberos') --- Gemfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Gemfile b/Gemfile index f950c5be15..fa89112a31 100644 --- a/Gemfile +++ b/Gemfile @@ -28,7 +28,7 @@ gem 'omniauth-google-oauth2' gem 'omniauth-twitter' gem 'omniauth-github' gem 'omniauth-shibboleth' -gem 'omniauth-kerberos' +gem 'omniauth-kerberos', group: :kerberos gem 'omniauth-gitlab' gem 'omniauth-bitbucket' gem 'doorkeeper', '2.1.3' From f676bbf91802394a2624da264bfe07944fdeb660 Mon Sep 17 00:00:00 2001 From: Jakub Jirutka Date: Mon, 11 May 2015 13:37:56 +0200 Subject: [PATCH 142/158] Update installation instructions, make kerberos optional --- doc/install/installation.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/doc/install/installation.md b/doc/install/installation.md index ca25eaea79..e777f6bbb4 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -62,7 +62,13 @@ up-to-date and install it. Install the required packages (needed to compile Ruby and native extensions to Ruby gems): - sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server redis-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake libkrb5-dev nodejs + sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server redis-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake nodejs + +If you want to use Kerberos for user authentication, then install libkrb5-dev: + + sudo apt-get install libkrb5-dev + +**Note:** If you don't know what Kerberos is, then you certainly don't need it. Make sure you have the right version of Git installed @@ -276,10 +282,12 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da **Note:** As of bundler 1.5.2, you can invoke `bundle install -jN` (where `N` the number of your processor cores) and enjoy the parallel gems installation with measurable difference in completion time (~60% faster). Check the number of your cores with `nproc`. For more information check this [post](http://robots.thoughtbot.com/parallel-gem-installing-using-bundler). First make sure you have bundler >= 1.5.2 (run `bundle -v`) as it addresses some [issues](https://devcenter.heroku.com/changelog-items/411) that were [fixed](https://github.com/bundler/bundler/pull/2817) in 1.5.2. # For PostgreSQL (note, the option says "without ... mysql") - sudo -u git -H bundle install --deployment --without development test mysql aws + sudo -u git -H bundle install --deployment --without development test mysql aws kerberos # Or if you use MySQL (note, the option says "without ... postgres") - sudo -u git -H bundle install --deployment --without development test postgres aws + sudo -u git -H bundle install --deployment --without development test postgres aws kerberos + +**Note:** If you want to use Kerberos for user authentication, then omit `kerberos` in the `--without` option above. ### Install GitLab Shell From 65b9ca9c084d58fecce80329fa43dec68de278f2 Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Mon, 11 May 2015 14:34:54 +0200 Subject: [PATCH 143/158] defreeze code --- doc/release/monthly.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/doc/release/monthly.md b/doc/release/monthly.md index cfe01896d8..5dd495718b 100644 --- a/doc/release/monthly.md +++ b/doc/release/monthly.md @@ -29,7 +29,6 @@ All steps from issue template are explained below ``` Xth: (7 working days before the 22nd) -- [ ] Code freeze - [ ] Update the CE changelog (#LINK) - [ ] Update the EE changelog (#LINK) - [ ] Update the CI changelog (#LINK) @@ -79,10 +78,6 @@ Xth: (1 working day before the 22nd) - - - -## Code Freeze - -Stop merging code in master, except for important bug fixes - ## Update changelog Any changes not yet added to the changelog are added by lead developer and in that merge request the complete team is From 6a6047f80c63f55e70a3d66e70a221f319cab0d2 Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Mon, 11 May 2015 15:09:36 +0200 Subject: [PATCH 144/158] create migration style guide. Fixes #2305 --- CONTRIBUTING.md | 3 +- doc/development/README.md | 3 +- doc/development/migration_style_guide.md | 39 ++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 2 deletions(-) create mode 100644 doc/development/migration_style_guide.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3165b7379d..895202b58e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -63,7 +63,7 @@ Merge requests can be filed either at [gitlab.com](https://gitlab.com/gitlab-org If you are new to GitLab development (or web development in general), search for the label `easyfix` ([gitlab.com](https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name=easyfix), [github](https://github.com/gitlabhq/gitlabhq/labels/easyfix)). Those are issues easy to fix, marked by the GitLab core-team. If you are unsure how to proceed but want to help, mention one of the core-team members to give you a hint. -To start with GitLab download the [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit) and see [Development section](doc/development/README.md) in the help file. +To start with GitLab download the [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit) and see [Development section](doc/development/README.md) in the help file. ### Merge request guidelines @@ -161,6 +161,7 @@ If you add a dependency in GitLab (such as an operating system package) please c 1. [Shell commands](doc/development/shell_commands.md) created by GitLab contributors to enhance security 1. [Markdown](http://www.cirosantilli.com/markdown-styleguide) 1. Interface text should be written subjectively instead of objectively. It should be the gitlab core team addressing a person. It should be written in present time and never use past tense (has been/was). For example instead of "prohibited this user from being saved due to the following errors:" the text should be "sorry, we could not create your account because:". Also these [excellent writing guidelines](https://github.com/NARKOZ/guides#writing). +1. [Migrations](doc/development/migration_style_guide.md) This is also the style used by linting tools such as [RuboCop](https://github.com/bbatsov/rubocop), [PullReview](https://www.pullreview.com/) and [Hound CI](https://houndci.com). diff --git a/doc/development/README.md b/doc/development/README.md index d5d264be19..16df0b40c4 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -1,4 +1,4 @@ -# Development +# Development - [Architecture](architecture.md) of GitLab - [Shell commands](shell_commands.md) in the GitLab codebase @@ -6,3 +6,4 @@ - [CI setup](ci_setup.md) for testing GitLab - [Sidekiq debugging](sidekiq_debugging.md) - [UI guide](ui_guide.md) for building GitLab with existing css styles and elements +- [Migration Style Guide](migration_style_guide.md) for creating safe migrations diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md new file mode 100644 index 0000000000..db2d8b9972 --- /dev/null +++ b/doc/development/migration_style_guide.md @@ -0,0 +1,39 @@ +# Migration Style Guide + +When writing migrations for GitLab, you have to take into account that +these will be ran by thousands of organizations of all sizes, some with +many years of data in their database. + +In addition, having to take a server offline for a an upgrade small or big is +a big burden for most organizations. For this reason it is important that your +migrations are written carefully and adhere to the style guide below. + +When writing your migrations, also consider that databases might have stale data +or inconsistencies and guard for that. Try to make as little assumptions as possible +about the state of the database. + +## Comments in the migration + +Each migration you write needs to have the two following pieces of information +as comments. + +### Online, Offline, errors? + +First, you need to provide information on whether the migration can be applied: + +1. online without errors (works on previous version and new one) +2. online with errors on old instances after migrating +3. online with errors on new instances while migrating +4. offline (needs to happen without app servers to prevent db corruption) + +It is always preferable to have a migration run online. If you expect the migration +to take particularly long (for instance, if it loops through all notes), +this is valuable information to add. + +### Reversibility + +Your migration should be reversible. This is very important, as it should +be possible to downgrade in case of a vulnerability or bugs. + +In your migration, add a comment describing how the reversibility of the +migration was tested. From 881e11df1be593e1943fa430e5a402f672aeba45 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Mon, 11 May 2015 14:48:39 +0000 Subject: [PATCH 145/158] Small improvements to style guide. --- doc/development/migration_style_guide.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index db2d8b9972..a102d7987d 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -1,17 +1,20 @@ # Migration Style Guide When writing migrations for GitLab, you have to take into account that -these will be ran by thousands of organizations of all sizes, some with +these will be ran by hundreds of thousands of organizations of all sizes, some with many years of data in their database. In addition, having to take a server offline for a an upgrade small or big is a big burden for most organizations. For this reason it is important that your -migrations are written carefully and adhere to the style guide below. +migrations are written carefully, can be applied online and adhere to the style guide below. When writing your migrations, also consider that databases might have stale data or inconsistencies and guard for that. Try to make as little assumptions as possible about the state of the database. +Please don't depend on GitLab specific code since it can change in future versions. +If needed copy-paste GitLab code into the migration to make make it forward compatible. + ## Comments in the migration Each migration you write needs to have the two following pieces of information @@ -36,4 +39,4 @@ Your migration should be reversible. This is very important, as it should be possible to downgrade in case of a vulnerability or bugs. In your migration, add a comment describing how the reversibility of the -migration was tested. +migration was tested. \ No newline at end of file From cae978a9e814030e07f94e05dafdce38ad1f7b88 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Mon, 11 May 2015 18:11:09 +0200 Subject: [PATCH 146/158] We would like to pass 'brakeman -w2' --- lib/tasks/brakeman.rake | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/tasks/brakeman.rake b/lib/tasks/brakeman.rake index 3a225801ff..d6e40926a4 100644 --- a/lib/tasks/brakeman.rake +++ b/lib/tasks/brakeman.rake @@ -1,5 +1,7 @@ desc 'Security check via brakeman' task :brakeman do + # We get 0 warnings at level 'w3' but we would like to reach 'w2'. Merge + # requests are welcome! if system("brakeman --skip-files lib/backup/repository.rb -w3 -z") puts 'Security check succeed' else From 24bef5e67a81c5edf9dacb65ecc091cac1f4c528 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 11 May 2015 14:31:31 -0400 Subject: [PATCH 147/158] Handle password reset for users with 2FA enabled --- app/controllers/passwords_controller.rb | 21 ++++++++++ spec/features/login_spec.rb | 4 +- spec/features/password_reset_spec.rb | 53 +++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 2 deletions(-) create mode 100644 spec/features/password_reset_spec.rb diff --git a/app/controllers/passwords_controller.rb b/app/controllers/passwords_controller.rb index dcbbe5baa4..88459d4080 100644 --- a/app/controllers/passwords_controller.rb +++ b/app/controllers/passwords_controller.rb @@ -15,4 +15,25 @@ class PasswordsController < Devise::PasswordsController respond_with(resource) end end + + # After a user resets their password, prompt for 2FA code if enabled instead + # of signing in automatically + # + # See http://git.io/vURrI + def update + super do |resource| + # TODO (rspeicher): In Devise master (> 3.4.1), we can set + # `Devise.sign_in_after_reset_password = false` and avoid this mess. + if resource.errors.empty? && resource.try(:otp_required_for_login?) + resource.unlock_access! if unlockable?(resource) + + # Since we are not signing this user in, we use the :updated_not_active + # message which only contains "Your password was changed successfully." + set_flash_message(:notice, :updated_not_active) if is_flashing_format? + + # Redirect to sign in so they can enter 2FA code + respond_with(resource, location: new_session_path(resource)) and return + end + end + end end diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb index 61defb8a33..046a9f6191 100644 --- a/spec/features/login_spec.rb +++ b/spec/features/login_spec.rb @@ -1,7 +1,7 @@ require 'spec_helper' feature 'Login' do - context 'with two-factor authentication' do + describe 'with two-factor authentication' do context 'with valid username/password' do let(:user) { create(:user, :two_factor) } @@ -78,7 +78,7 @@ feature 'Login' do end end - context 'without two-factor authentication' do + describe 'without two-factor authentication' do let(:user) { create(:user) } it 'allows basic login' do diff --git a/spec/features/password_reset_spec.rb b/spec/features/password_reset_spec.rb new file mode 100644 index 0000000000..a34efce09e --- /dev/null +++ b/spec/features/password_reset_spec.rb @@ -0,0 +1,53 @@ +require 'spec_helper' + +feature 'Password reset' do + def forgot_password + click_on 'Forgot your password?' + fill_in 'Email', with: user.email + click_button 'Reset password' + user.reload + end + + def get_reset_token + mail = ActionMailer::Base.deliveries.last + body = mail.body.encoded + body.scan(/reset_password_token=(.+)\"/).flatten.first + end + + def reset_password(password = 'password') + visit edit_user_password_path(reset_password_token: get_reset_token) + + fill_in 'New password', with: password + fill_in 'Confirm new password', with: password + click_button 'Change your password' + end + + describe 'with two-factor authentication' do + let(:user) { create(:user, :two_factor) } + + it 'requires login after password reset' do + visit root_path + + forgot_password + reset_password + + expect(page).to have_content("Your password was changed successfully.") + expect(page).not_to have_content("You are now signed in.") + expect(current_path).to eq new_user_session_path + end + end + + describe 'without two-factor authentication' do + let(:user) { create(:user) } + + it 'automatically logs in after password reset' do + visit root_path + + forgot_password + reset_password + + expect(current_path).to eq root_path + expect(page).to have_content("Your password was changed successfully. You are now signed in.") + end + end +end From 22badc13136369e202dc6df06a62456110879ee4 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Tue, 12 May 2015 10:41:24 +0300 Subject: [PATCH 148/158] Bump attr_encrypted Signed-off-by: Dmitriy Zaporozhets --- Gemfile | 2 +- Gemfile.lock | 12 +++--------- 2 files changed, 4 insertions(+), 10 deletions(-) diff --git a/Gemfile b/Gemfile index ee6072b2da..a62abf2bad 100644 --- a/Gemfile +++ b/Gemfile @@ -37,7 +37,7 @@ gem "rack-oauth2", "~> 1.0.5" # Two-factor authentication gem 'devise-two-factor' gem 'rqrcode-rails3' -gem 'attr_encrypted', git: 'https://github.com/attr-encrypted/attr_encrypted.git', ref: '94d901df2ccbc579b981091d53dd641f9bed4c1d' +gem 'attr_encrypted', '1.3.4' # Browser detection gem "browser" diff --git a/Gemfile.lock b/Gemfile.lock index 672bd9950e..0ea8021815 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,11 +1,3 @@ -GIT - remote: https://github.com/attr-encrypted/attr_encrypted.git - revision: 94d901df2ccbc579b981091d53dd641f9bed4c1d - ref: 94d901df2ccbc579b981091d53dd641f9bed4c1d - specs: - attr_encrypted (1.3.3) - encryptor (>= 1.3.0) - GEM remote: https://rubygems.org/ specs: @@ -54,6 +46,8 @@ GEM ast (2.0.0) astrolabe (1.3.0) parser (>= 2.2.0.pre.3, < 3.0) + attr_encrypted (1.3.4) + encryptor (>= 1.3.0) attr_required (1.0.0) autoprefixer-rails (5.1.11) execjs @@ -690,7 +684,7 @@ DEPENDENCIES annotate (~> 2.6.0.beta2) asana (~> 0.0.6) asciidoctor (= 0.1.4) - attr_encrypted! + attr_encrypted (= 1.3.4) awesome_print better_errors binding_of_caller From 71a1689fec04093b1ee3455a11527efe93b207a6 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 23:16:33 +0200 Subject: [PATCH 149/158] Add icons to more tabs. --- app/views/projects/_section.html.haml | 2 ++ app/views/projects/commits/_head.html.haml | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/app/views/projects/_section.html.haml b/app/views/projects/_section.html.haml index 0b7f4cb780..f4f876f380 100644 --- a/app/views/projects/_section.html.haml +++ b/app/views/projects/_section.html.haml @@ -1,10 +1,12 @@ %ul.nav.nav-tabs %li.active = link_to '#tab-activity', 'data-toggle' => 'tab' do + = icon("tachometer") Activity - if @repository.readme %li = link_to '#tab-readme', 'data-toggle' => 'tab' do + = icon("file-text-o") Readme .tab-content .tab-pane.active#tab-activity diff --git a/app/views/projects/commits/_head.html.haml b/app/views/projects/commits/_head.html.haml index 66101f3f0d..66261c7336 100644 --- a/app/views/projects/commits/_head.html.haml +++ b/app/views/projects/commits/_head.html.haml @@ -1,17 +1,22 @@ %ul.nav.nav-tabs = nav_link(controller: [:commit, :commits]) do = link_to namespace_project_commits_path(@project.namespace, @project, @ref || @repository.root_ref) do + = icon("history") Commits %span.badge= number_with_precision(@repository.commit_count, precision: 0, delimiter: ',') = nav_link(controller: :compare) do - = link_to 'Compare', namespace_project_compare_index_path(@project.namespace, @project, from: @repository.root_ref, to: @ref || @repository.root_ref) + = link_to namespace_project_compare_index_path(@project.namespace, @project, from: @repository.root_ref, to: @ref || @repository.root_ref) do + = icon("exchange") + Compare = nav_link(html_options: {class: branches_tab_class}) do = link_to namespace_project_branches_path(@project.namespace, @project) do + = icon("code-fork") Branches %span.badge.js-totalbranch-count= @repository.branches.size = nav_link(controller: :tags) do = link_to namespace_project_tags_path(@project.namespace, @project) do + = icon("tags") Tags %span.badge.js-totaltags-count= @repository.tags.length From e9dff9f4103506d22a7044d2b721b74c4d0b510a Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 23:17:05 +0200 Subject: [PATCH 150/158] Fix alignment of ref picker, breadcrumb and buttons. --- app/assets/stylesheets/pages/commits.scss | 4 ---- app/assets/stylesheets/pages/projects.scss | 3 ++- app/assets/stylesheets/pages/tree.scss | 8 -------- app/views/projects/tree/show.html.haml | 2 +- 4 files changed, 3 insertions(+), 14 deletions(-) diff --git a/app/assets/stylesheets/pages/commits.scss b/app/assets/stylesheets/pages/commits.scss index 84361e1548..359f4073e8 100644 --- a/app/assets/stylesheets/pages/commits.scss +++ b/app/assets/stylesheets/pages/commits.scss @@ -29,10 +29,6 @@ .commits-feed-holder { float: right; - - .btn { - padding: 4px 12px; - } } li.commit { diff --git a/app/assets/stylesheets/pages/projects.scss b/app/assets/stylesheets/pages/projects.scss index 5a8d466529..224aea2db5 100644 --- a/app/assets/stylesheets/pages/projects.scss +++ b/app/assets/stylesheets/pages/projects.scss @@ -249,7 +249,8 @@ ul.nav.nav-projects-tabs { } .breadcrumb.repo-breadcrumb { - padding: 2px 0; + padding: 0; + line-height: 34px; background: white; border: none; font-size: 16px; diff --git a/app/assets/stylesheets/pages/tree.scss b/app/assets/stylesheets/pages/tree.scss index 57f63b52aa..34ee4d7b31 100644 --- a/app/assets/stylesheets/pages/tree.scss +++ b/app/assets/stylesheets/pages/tree.scss @@ -106,17 +106,9 @@ } } -.tree-download-holder .btn { - padding: 4px 12px; -} - .tree-ref-holder { float: left; margin-right: 15px; - - .select2-container .select2-choice, .select2-container.select2-drop-above .select2-choice { - padding: 4px 12px; - } } .readme-holder { diff --git a/app/views/projects/tree/show.html.haml b/app/views/projects/tree/show.html.haml index 72916cad18..04590f65b2 100644 --- a/app/views/projects/tree/show.html.haml +++ b/app/views/projects/tree/show.html.haml @@ -8,7 +8,7 @@ - if can? current_user, :download_code, @project .tree-download-holder - = render 'projects/repositories/download_archive', ref: @ref, btn_class: 'btn-group-sm pull-right hidden-xs hidden-sm', split_button: true + = render 'projects/repositories/download_archive', ref: @ref, btn_class: 'btn-group pull-right hidden-xs hidden-sm', split_button: true #tree-holder.tree-holder.clearfix = render "tree", tree: @tree From bfe23847977376e3b89d98f17e0b3fd0fa2ddd77 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Sat, 9 May 2015 23:43:15 +0200 Subject: [PATCH 151/158] Restructure project sidebar. --- app/helpers/icons_helper.rb | 6 +- app/views/projects/_aside.html.haml | 148 +++++++++++++++------------- 2 files changed, 85 insertions(+), 69 deletions(-) diff --git a/app/helpers/icons_helper.rb b/app/helpers/icons_helper.rb index a9030729b4..a730684f8f 100644 --- a/app/helpers/icons_helper.rb +++ b/app/helpers/icons_helper.rb @@ -26,15 +26,15 @@ module IconsHelper end def public_icon - icon('globe') + icon('globe fw') end def internal_icon - icon('shield') + icon('shield fw') end def private_icon - icon('lock') + icon('lock fw') end def file_type_icon_class(type, mode, name) diff --git a/app/views/projects/_aside.html.haml b/app/views/projects/_aside.html.haml index 1241b51f9a..8044906dbf 100644 --- a/app/views/projects/_aside.html.haml +++ b/app/views/projects/_aside.html.haml @@ -1,85 +1,101 @@ .clearfix - .append-bottom-20 - = render "shared/clone_panel" - - unless @project.empty_repo? .well %h4.visibility-level-label = visibility_level_icon(@project.visibility_level) = "#{visibility_level_label(@project.visibility_level).capitalize} project" - %ul.nav.nav-pills - %li= link_to pluralize(number_with_delimiter(@repository.commit_count), 'commit'), namespace_project_commits_path(@project.namespace, @project, @ref || @repository.root_ref) - %li= link_to pluralize(number_with_delimiter(@repository.branch_names.count), 'branch'), namespace_project_branches_path(@project.namespace, @project) - %li= link_to pluralize(number_with_delimiter(@repository.tag_names.count), 'tag'), namespace_project_tags_path(@project.namespace, @project) + + - if @repository.changelog || @repository.license || @repository.contribution_guide + %ul.nav.nav-pills + - if @repository.changelog + %li.hidden-xs + = link_to changelog_url(@project) do + = icon("list-alt fw") + Changelog + - if @repository.license + %li + = link_to license_url(@project) do + = icon("check-circle-o fw") + License + - if @repository.contribution_guide + %li + = link_to contribution_guide_url(@project) do + = icon("info-circle fw") + Contribution guide .actions - = link_to namespace_project_compare_index_path(@project.namespace, @project, from: @repository.root_ref, to: @ref || @repository.root_ref), class: 'btn btn-sm' do + - if can? current_user, :write_issue, @project + = link_to url_for_new_issue(@project, only_path: true), title: "New Issue", class: 'btn btn-sm append-right-10' do + = icon("exclamation-circle fw") + New Issue + - if can? current_user, :write_merge_request, @project + = link_to new_namespace_project_merge_request_path(@project.namespace, @project), class: "btn btn-sm", title: "New Merge Request" do + = icon("plus fw") + New Merge Request + + - if version = @repository.version + .well + %h4 + = icon("clock-o fw") + Version + .pull-right + = link_to version_url(@project) do + = @repository.blob_by_oid(version.id).data + + - if forked_from_project = @project.forked_from_project + .well + %h4 + = icon("code-fork fw") + Forked from + .pull-right + = link_to forked_from_project.namespace.try(:name), project_path(forked_from_project) + + - @project.ci_services.each do |ci_service| + - if ci_service.active? && ci_service.respond_to?(:builds_path) + .well + %h4 + = icon("check fw") + = ci_service.title + .pull-right + - if ci_service.respond_to?(:status_img_path) + = link_to ci_service.builds_path, :'data-no-turbolink' => 'data-no-turbolink' do + = image_tag ci_service.status_img_path, alt: "build status" + - else + = link_to 'view builds', ci_service.builds_path, :'data-no-turbolink' => 'data-no-turbolink' + + - unless @project.empty_repo? + .well + %h4 + = icon("archive fw") + Repository + + %ul.nav.nav-pills + %li + = link_to namespace_project_commits_path(@project.namespace, @project, @ref || @repository.root_ref) do + = icon("history fw") + = pluralize(number_with_delimiter(@repository.commit_count), 'commit') + %li + = link_to namespace_project_branches_path(@project.namespace, @project) do + = icon("code-fork fw") + = pluralize(number_with_delimiter(@repository.branch_names.count), 'branch') + %li + = link_to namespace_project_tags_path(@project.namespace, @project) do + = icon("tags fw") + = pluralize(number_with_delimiter(@repository.tag_names.count), 'tag') + + = render "shared/clone_panel" + + .actions.prepend-top-10 + = link_to namespace_project_compare_index_path(@project.namespace, @project, from: @repository.root_ref, to: @ref || @repository.root_ref), class: 'btn btn-sm append-right-10' do %i.fa.fa-exchange Compare code - if can?(current_user, :download_code, @project) -   = render 'projects/repositories/download_archive', split_button: true, btn_class: 'btn-group-sm' - - unless @project.empty_repo? - .well - %h4 Contribute - %ul.nav.nav-pills - - if @repository.changelog - %li.hidden-xs - = link_to changelog_url(@project) do - Changelog - - if @repository.contribution_guide - %li.hidden-xs - = link_to contribution_guide_url(@project) do - Contribution guide - - if @repository.license - %li - = link_to license_url(@project) do - License - .actions - = link_to url_for_new_issue(@project, only_path: true), title: "New Issue", class: 'btn btn-sm' do - %i.fa.fa-fw.fa-exclamation-circle - New issue - - if can? current_user, :write_merge_request, @project -   - = link_to new_namespace_project_merge_request_path(@project.namespace, @project), class: "btn btn-sm", title: "New Merge Request" do - %i.fa.fa-plus - New Merge Request - - - - if @project.archived? .alert.alert-warning %h4 - %i.fa.fa-exclamation-triangle + = icon("exclamation-triangle fw") Archived project! %p Repository is read-only - - - if @project.forked_from_project - .well - %h4 - Forked from - .pull-right - = link_to @project.forked_from_project.namespace.try(:name), project_path(@project.forked_from_project) - - -- if version = @repository.version - .well - %h4 - Version - .pull-right - = link_to version_url(@project) do - = @repository.blob_by_oid(version.id).data - -- @project.ci_services.each do |ci_service| - - if ci_service.active? && ci_service.respond_to?(:builds_path) - .well - %h4 - = ci_service.title - .pull-right - - if ci_service.respond_to?(:status_img_path) - = link_to ci_service.builds_path, :'data-no-turbolink' => 'data-no-turbolink' do - = image_tag ci_service.status_img_path, alt: "build status" - - else - = link_to 'view builds', ci_service.builds_path, :'data-no-turbolink' => 'data-no-turbolink' From a5ce66d83637734f79bc61c7d1bf7c71e3d26afc Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 12 May 2015 10:28:56 +0200 Subject: [PATCH 152/158] Move clone panel to bottom of project sidebar. --- app/views/projects/_aside.html.haml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/app/views/projects/_aside.html.haml b/app/views/projects/_aside.html.haml index 8044906dbf..5edec7fe65 100644 --- a/app/views/projects/_aside.html.haml +++ b/app/views/projects/_aside.html.haml @@ -83,9 +83,7 @@ = icon("tags fw") = pluralize(number_with_delimiter(@repository.tag_names.count), 'tag') - = render "shared/clone_panel" - - .actions.prepend-top-10 + .actions = link_to namespace_project_compare_index_path(@project.namespace, @project, from: @repository.root_ref, to: @ref || @repository.root_ref), class: 'btn btn-sm append-right-10' do %i.fa.fa-exchange Compare code @@ -93,6 +91,8 @@ - if can?(current_user, :download_code, @project) = render 'projects/repositories/download_archive', split_button: true, btn_class: 'btn-group-sm' + = render "shared/clone_panel" + - if @project.archived? .alert.alert-warning %h4 From 4346c39ba9578169e218ccae47501612051e6f04 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 12 May 2015 10:29:59 +0200 Subject: [PATCH 153/158] Move "Forked from" above "Version". --- app/views/projects/_aside.html.haml | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/app/views/projects/_aside.html.haml b/app/views/projects/_aside.html.haml index 5edec7fe65..c2f56996ba 100644 --- a/app/views/projects/_aside.html.haml +++ b/app/views/projects/_aside.html.haml @@ -28,11 +28,20 @@ = link_to url_for_new_issue(@project, only_path: true), title: "New Issue", class: 'btn btn-sm append-right-10' do = icon("exclamation-circle fw") New Issue + - if can? current_user, :write_merge_request, @project = link_to new_namespace_project_merge_request_path(@project.namespace, @project), class: "btn btn-sm", title: "New Merge Request" do = icon("plus fw") New Merge Request + - if forked_from_project = @project.forked_from_project + .well + %h4 + = icon("code-fork fw") + Forked from + .pull-right + = link_to forked_from_project.namespace.try(:name), project_path(forked_from_project) + - if version = @repository.version .well %h4 @@ -42,14 +51,6 @@ = link_to version_url(@project) do = @repository.blob_by_oid(version.id).data - - if forked_from_project = @project.forked_from_project - .well - %h4 - = icon("code-fork fw") - Forked from - .pull-right - = link_to forked_from_project.namespace.try(:name), project_path(forked_from_project) - - @project.ci_services.each do |ci_service| - if ci_service.active? && ci_service.respond_to?(:builds_path) .well From 125cb9b866b66a4ae21a3fec8ae5ad6e1b3ae4ec Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 12 May 2015 11:21:56 +0200 Subject: [PATCH 154/158] Don't accidentally unblock auto created users from Active Directory. --- CHANGELOG | 1 + lib/gitlab/ldap/access.rb | 2 +- spec/lib/gitlab/ldap/access_spec.rb | 27 +++++++++++++++++++++++---- 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 8a98da1a52..3e41526181 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -52,6 +52,7 @@ v 7.11.0 (unreleased) - Add current_sign_in_at to UserFull REST api. - Make Sidekiq MemoryKiller shutdown signal configurable - Add "Create Merge Request" buttons to commits and branches pages and push event. + - Fix automatic blocking of auto-created users from Active Directory. v 7.10.2 - Fix CI links on MR page diff --git a/lib/gitlab/ldap/access.rb b/lib/gitlab/ldap/access.rb index 960fb3849b..16ff03c38d 100644 --- a/lib/gitlab/ldap/access.rb +++ b/lib/gitlab/ldap/access.rb @@ -40,7 +40,7 @@ module Gitlab user.block unless user.blocked? false else - user.activate if user.blocked? + user.activate if user.blocked? && !ldap_config.block_auto_created_users true end else diff --git a/spec/lib/gitlab/ldap/access_spec.rb b/spec/lib/gitlab/ldap/access_spec.rb index 707a0521ab..2189e313d6 100644 --- a/spec/lib/gitlab/ldap/access_spec.rb +++ b/spec/lib/gitlab/ldap/access_spec.rb @@ -16,7 +16,7 @@ describe Gitlab::LDAP::Access do context 'when the user is found' do before { Gitlab::LDAP::Person.stub(find_by_dn: :ldap_user) } - context 'and the user is diabled via active directory' do + context 'and the user is disabled via active directory' do before { Gitlab::LDAP::Person.stub(disabled_via_active_directory?: true) } it { is_expected.to be_falsey } @@ -36,9 +36,28 @@ describe Gitlab::LDAP::Access do it { is_expected.to be_truthy } - it "should unblock user in GitLab" do - access.allowed? - user.should_not be_blocked + context 'when auto-created users are blocked' do + + before do + Gitlab::LDAP::Config.any_instance.stub(block_auto_created_users: true) + end + + it "does not unblock user in GitLab" do + access.allowed? + user.should be_blocked + end + end + + context "when auto-created users are not blocked" do + + before do + Gitlab::LDAP::Config.any_instance.stub(block_auto_created_users: false) + end + + it "should unblock user in GitLab" do + access.allowed? + user.should_not be_blocked + end end end From b0af379a22efff12a7561d7b89ce85105169a5cc Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 12 May 2015 12:16:34 +0200 Subject: [PATCH 155/158] Show user roles by comments. --- CHANGELOG | 1 + app/assets/stylesheets/pages/notes.scss | 11 ++++++++++ app/views/projects/notes/_note.html.haml | 26 ++++++++++++++++-------- 3 files changed, 29 insertions(+), 9 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 8a98da1a52..b21e202c54 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -52,6 +52,7 @@ v 7.11.0 (unreleased) - Add current_sign_in_at to UserFull REST api. - Make Sidekiq MemoryKiller shutdown signal configurable - Add "Create Merge Request" buttons to commits and branches pages and push event. + - Show user roles by comments. v 7.10.2 - Fix CI links on MR page diff --git a/app/assets/stylesheets/pages/notes.scss b/app/assets/stylesheets/pages/notes.scss index 589a43c426..61b907e39b 100644 --- a/app/assets/stylesheets/pages/notes.scss +++ b/app/assets/stylesheets/pages/notes.scss @@ -44,6 +44,14 @@ ul.notes { } .author-username { } + + .note-role { + float: right; + margin-top: 2px; + border: 1px solid #bbb; + background-color: transparent; + color: #999; + } } .discussion { @@ -136,6 +144,7 @@ ul.notes { .note { &.note:hover { .note-actions { display: block; } + .note-actions + .note-role { display: none; } } .discussion-header:hover { .discussion-actions { display: block; } @@ -153,6 +162,8 @@ ul.notes { } a { + margin-left: 5px; + @extend .cgray; &:hover { diff --git a/app/views/projects/notes/_note.html.haml b/app/views/projects/notes/_note.html.haml index 4d26b52df0..2f3c407d6e 100644 --- a/app/views/projects/notes/_note.html.haml +++ b/app/views/projects/notes/_note.html.haml @@ -9,22 +9,30 @@ .timeline-content .note-header .note-actions - = link_to "##{dom_id(note)}", name: dom_id(note) do - = icon('link') + = link_to "##{dom_id(note)}", name: dom_id(note), title: "Link here" do + = icon('link fw') Link here -   + - if note_editable?(note) = link_to '#', title: 'Edit comment', class: 'js-note-edit' do - = icon('pencil-square-o') + = icon('pencil-square-o fw') Edit -   - = link_to namespace_project_note_path(@project.namespace, @project, note), title: 'Remove comment', method: :delete, data: { confirm: 'Are you sure you want to remove this comment?' }, remote: true, class: 'danger js-note-delete' do - = icon('trash-o', class: 'cred') + + = link_to namespace_project_note_path(note.project.namespace, note.project, note), title: 'Remove comment', method: :delete, data: { confirm: 'Are you sure you want to remove this comment?' }, remote: true, class: 'danger js-note-delete' do + = icon('trash-o fw', class: 'cred') Remove + + - unless note.system + - member = note.project.team.find_member(note.author.id) + - if member + %span.note-role.label + = member.human_access + - if note.system = link_to user_path(note.author) do = image_tag avatar_icon(note.author_email), class: 'avatar s16', alt: '' - = link_to_member(@project, note.author, avatar: false) + + = link_to_member(note.project, note.author, avatar: false) %span.author-username = '@' + note.author.username %span.note-last-update @@ -65,7 +73,7 @@ = link_to note.attachment.url, target: '_blank' do = icon('paperclip') = note.attachment_identifier - = link_to delete_attachment_namespace_project_note_path(@project.namespace, @project, note), + = link_to delete_attachment_namespace_project_note_path(note.project.namespace, note.project, note), title: 'Delete this attachment', method: :delete, remote: true, data: { confirm: 'Are you sure you want to remove the attachment?' }, class: 'danger js-note-attachment-delete' do = icon('trash-o', class: 'cred') .clear From 474023b71a4aaf2a9072a94cf9f5018d7d9bc2ba Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Tue, 12 May 2015 14:26:46 +0300 Subject: [PATCH 156/158] Fix Gemfile.lock --- Gemfile.lock | 2 +- db/schema.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Gemfile.lock b/Gemfile.lock index 14b9a6848a..9940ab1524 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -723,7 +723,7 @@ DEPENDENCIES gitlab-grack (~> 2.0.2) gitlab-linguist (~> 3.0.1) gitlab_emoji (~> 0.1) - gitlab_git (~> 7.1.10) + gitlab_git (~> 7.1.11) gitlab_meta (= 7.0) gitlab_omniauth-ldap (= 1.2.1) gollum-lib (~> 4.0.2) diff --git a/db/schema.rb b/db/schema.rb index 6606e13d3d..c89d22d241 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -492,12 +492,12 @@ ActiveRecord::Schema.define(version: 20150509180749) do t.string "bitbucket_access_token" t.string "bitbucket_access_token_secret" t.string "location" + t.string "public_email", default: "", null: false t.string "encrypted_otp_secret" t.string "encrypted_otp_secret_iv" t.string "encrypted_otp_secret_salt" t.boolean "otp_required_for_login" t.text "otp_backup_codes" - t.string "public_email", default: "", null: false end add_index "users", ["admin"], name: "index_users_on_admin", using: :btree From 34a9aa2e8a6c3b55e726810f4a899aae23e76363 Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Tue, 12 May 2015 11:48:18 +0300 Subject: [PATCH 157/158] update migration style guide --- doc/development/migration_style_guide.md | 48 +++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index a102d7987d..4fa1961fde 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -39,4 +39,50 @@ Your migration should be reversible. This is very important, as it should be possible to downgrade in case of a vulnerability or bugs. In your migration, add a comment describing how the reversibility of the -migration was tested. \ No newline at end of file +migration was tested. + + +## Removing indices + +If you need to remove index, please add a condition like in following example: + +``` +remove_index :namespaces, column: :name if index_exists?(:namespaces, :name) +``` + +## Adding indices + +If you need to add an unique index please keep in mind there is possibility of existing duplicates. If it is possible write a separate migration for handling this situation. It can be just removing or removing with overwriting all references to these duplicates depend on situation. + +## Testing + +Make sure that your migration works with MySQL and PostgreSQL with data. An empty database does not guarantee that your migration is correct. + +Make sure your migration can be reversed. + +## Data migration + +Please prefer Arel and plain SQL over usual ActiveRecord syntax. In case of using plain SQL you need to quote all input manually with `quote_string` helper. + +Example with Arel: + +``` +users = Arel::Table.new(:users) +users.group(users[:user_id]).having(users[:id].count.gt(5)) + +#updtae other tables with this results +``` + +Example with plain SQL and `quote_string` helper: + +``` +select_all("SELECT name, COUNT(id) as cnt FROM tags GROUP BY name HAVING COUNT(id) > 1").each do |tag| + tag_name = quote_string(tag["name"]) + duplicate_ids = select_all("SELECT id FROM tags WHERE name = '#{tag_name}'").map{|tag| tag["id"]} + origin_tag_id = duplicate_ids.first + duplicate_ids.delete origin_tag_id + + execute("UPDATE taggings SET tag_id = #{origin_tag_id} WHERE tag_id IN(#{duplicate_ids.join(",")})") + execute("DELETE FROM tags WHERE id IN(#{duplicate_ids.join(",")})") +end +``` From 8acd705c8a2ac73c4c0c780e6ce0b4532dca3b59 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 12 May 2015 13:36:20 +0200 Subject: [PATCH 158/158] Update docs in line with mysql-postgresql-converter changes. --- doc/update/mysql_to_postgresql.md | 55 +++---------------------------- 1 file changed, 4 insertions(+), 51 deletions(-) diff --git a/doc/update/mysql_to_postgresql.md b/doc/update/mysql_to_postgresql.md index 50941db25f..2c43cf59c1 100644 --- a/doc/update/mysql_to_postgresql.md +++ b/doc/update/mysql_to_postgresql.md @@ -1,7 +1,7 @@ # Migrating GitLab from MySQL to Postgres *Make sure you view this [guide from the `master` branch](../../../master/doc/update/mysql_to_postgresql.md) for the most up to date instructions.* -If you are replacing MySQL with Postgres while keeping GitLab on the same server all you need to do is to export from MySQL, import into Postgres and rebuild the indexes as described below. If you are also moving GitLab to another server, or if you are switching to omnibus-gitlab, you may want to use a GitLab backup file. The second part of this documents explains the procedure to do this. +If you are replacing MySQL with Postgres while keeping GitLab on the same server all you need to do is to export from MySQL, convert the resulting SQL file, and import it into Postgres. If you are also moving GitLab to another server, or if you are switching to omnibus-gitlab, you may want to use a GitLab backup file. The second part of this documents explains the procedure to do this. ## Export from MySQL and import into Postgres @@ -14,13 +14,11 @@ sudo service gitlab stop git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab cd mysql-postgresql-converter -mysqldump --compatible=postgresql --default-character-set=utf8 -r databasename.mysql -u root gitlabhq_production -p -python db_converter.py databasename.mysql databasename.psql +mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p +python db_converter.py gitlabhq_production.mysql gitlabhq_production.psql # Import the database dump as the application database user -sudo -u git psql -f databasename.psql -d gitlabhq_production - -# Rebuild indexes (see below) +sudo -u git psql -f gitlabhq_production.psql -d gitlabhq_production # Install gems for PostgreSQL (note: the line below states '--without ... mysql') sudo -u git -H bundle install --without development test mysql --deployment @@ -28,51 +26,6 @@ sudo -u git -H bundle install --without development test mysql --deployment sudo service gitlab start ``` -## Rebuild indexes - -The lanyrd database converter script does not preserve all indexes, so we have to recreate them ourselves after migrating from MySQL. It is not necessary to shut down GitLab for this process. - -### For non-omnibus installations - -On non-omnibus installations (distributed using Git) we retrieve the index declarations from version control using `git stash`. - -``` -# Clone the database converter on your Postgres-backed GitLab server -cd /tmp -git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab - -cd /home/git/gitlab - -# Stash changes to db/schema.rb to make sure we can find the right index statements -sudo -u git -H git stash - -# Generate add_index.rb -ruby /tmp/mysql-postgresql-converter/add_index_statements.rb db/schema.rb > /tmp/mysql-postgresql-converter/add_index.rb - -# Create the indexes -sudo -u git -H bundle exec rails runner -e production 'eval $stdin.read' < /tmp/mysql-postgresql-converter/add_index.rb -``` - -### For omnibus-gitlab installations - -On omnibus-gitlab we need to get the index declarations from a file called `schema.rb.bundled`. For versions older than 6.9, we need to download the file. - -``` -# Clone the database converter on your Postgres-backed GitLab server -cd /tmp -/opt/gitlab/embedded/bin/git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab -cd /tmp/mysql-postgresql-converter - -# Download schema.rb.bundled if necessary -test -e /opt/gitlab/embedded/service/gitlab-rails/db/schema.rb.bundled || sudo /opt/gitlab/embedded/bin/curl -o /opt/gitlab/embedded/service/gitlab-rails/db/schema.rb.bundled https://gitlab.com/gitlab-org/gitlab-ce/raw/v6.9.1/db/schema.rb - -# Generate add_index.rb -/opt/gitlab/embedded/bin/ruby add_index_statements.rb /opt/gitlab/embedded/service/gitlab-rails/db/schema.rb.bundled > add_index.rb - -# Create the indexes -/opt/gitlab/bin/gitlab-rails runner 'eval $stdin.read' < add_index.rb -``` - ## Converting a GitLab backup file from MySQL to Postgres **Note:** Please make sure to have Python 2.7.x (or higher) installed.