Add whitelisted elements correctly in sanitization
Add whitelisted elements correctly in sanitization
Consider this command:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>', pipeline: :description)
puts markdown('<span>this is a span</span>')"
And the same in the opposite order:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>')
puts markdown('<span>this is a span</span>', pipeline: :description)"
Before this change, they would both output:
<p><span>this is a span</span></p>
<p>this is a span</p>
That's because `span` is added to the list of whitelisted elements in
the `SanitizationFilter`, but this method tries not to make the same
changes multiple times. Unfortunately,
`HTML::Pipeline::SanitizationFilter::LIMITED`, which is used by the
`DescriptionPipeline`, uses the same Ruby objects for all of its hash
values _except_ `:elements`.
That means that whichever of `DescriptionPipeline` and `GfmPipeline` is
called first would have `span` in its whitelisted elements, and the
second wouldn't.
Fix this by adding a special check for modifying `:elements` twice, then
checking `:transformers` as before.
See merge request !4588
Improve Gitlab::Auth method names
Auth.find was a very generic name for a very specific method.
Auth.find_in_gitlab_or_ldap was inaccurate in GitLab EE where it also
looks in Kerberos.
See merge request !4589
Consider this command:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>', pipeline: :description)
puts markdown('<span>this is a span</span>')"
And the same in the opposite order:
bundle exec rails r "include GitlabMarkdownHelper
puts markdown('<span>this is a span</span>')
puts markdown('<span>this is a span</span>', pipeline: :description)"
Before this change, they would both output:
<p><span>this is a span</span></p>
<p>this is a span</p>
That's because `span` is added to the list of whitelisted elements in
the `SanitizationFilter`, but this method tries not to make the same
changes multiple times. Unfortunately,
`HTML::Pipeline::SanitizationFilter::LIMITED`, which is used by the
`DescriptionPipeline`, uses the same Ruby objects for all of its hash
values _except_ `:elements`.
That means that whichever of `DescriptionPipeline` and `GfmPipeline` is
called first would have `span` in its whitelisted elements, and the
second wouldn't.
Fix this by creating an entirely separate hash, before either pipeline
is invoked.
Instead of updating a fixed number of rows (based on the amount of rows
available at the start of the update) the method
"update_column_in_batches" will now continue updating rows until it runs
out of rows to process.
For a table with a high rate of inserts this may result in the migration
taking quite some time. However, the alternative is not all rows being
updated or the "change_column_null" method raising an error due to there
being NULL values.
Add workhorse controller and API helpers
Adds `send_git_blob` and `send_git_archive` controller and API helpers to reduce duplication and make Workhorse easier for a developer to work with.
See merge request !4486
Wrap all rate limiting logic inside GitHub API client
## What does this MR do?
Move the actual rate limiting logic to GitHub API to clean the code inside the GitHub importer, and avoid code duplication.
## Are there points in the code the reviewer needs to double check?
No there aren't.
## Why was this MR needed?
Avoid code duplication to handle API rate limit in every call to the GitHub API.
## What are the relevant issue numbers?
There are none.
## Screenshots (if relevant)
Not relevant.
See merge request !4552
Fix bug with SQL syntax error during backup restoration
## What does this MR do?
It properly escapes the table name in the `DROP TABLE IF EXISTS ...` statement used during restore of a backup.
## Are there points in the code the reviewer needs to double check?
Please check compatibility with PostgreSQL
## Why was this MR needed?
Fix an issue with backups can not be restored if MySQL is used as database for GitLab.
## What are the relevant issue numbers?
#15259
## Screenshots (if relevant)
-
See merge request !4318
Adjust the SAML control flow to allow LDAP identities to be added to an existing SAML user.
## What does this MR do?
It correctly lets an existing SAML user to add their LDAP identity automatically at login.
## Why was this MR needed?
A customer had issues with the `auto_link_ldap_user` feature. The flow was not working if there was an account with a SAML identity, but no LDAP identity. GitLab would pick up the correct LDAP person, but due to the order of the flow, that LDAP person was never associated with the user.
## What are the relevant issue numbers?
Fixes#17346
/cc @dblessing @balameb @stanhu
See merge request !4498
Fix wiki linking behavior for markdown wiki pages
Related to #18019
- As per the documentation in !4372
## TODO
- [ ] !4432 Have wiki linking behave as per the documentation
- [x] Move `WikiLinkFilter` specs to the pipeline level
- [x] Verify current behavior on wiki `show` page
- [x] Fix current behavior on wiki `show` page
- [x] Verify current behaviour on wiki preview
- [x] Fix current behaviour on wiki preview
- [x] Rewrite all links and get preview links working
- [x] Make sure all links are on-par with the wiki `show` page
- [x] TDD `WikiLinkFilter` and get it working
- [x] Hook `WikiLinkFilter` up
- [x] Fix tests
- [x] Fix `markdown_spec`
- [x] Fix `wiki` spinach feature
- [x] Wait for [build](https://gitlab.com/gitlab-org/gitlab-ce/commit/4f50dd2/builds) to pass
- [x] Make sure all wiki-related pages are working as expected (history, all pages, etc.)
- [x] Test in different ruby versions
- [x] GitLab instances hosted on a relative URL
- [x] Non-markdown rendering formats?
- [x] RDoc
- [x] ASCIIDoc
- [x] Create issues to fix things for RDoc and ASCIIDoc
- [x] Gauge performance impact
- [x] Refactor
- [x] Re-organize commits
- [x] Make sure [build](https://gitlab.com/gitlab-org/gitlab-ce/commit/f860e9a8dcabe7d5f160c32fc549807c98baa4a1/builds) passes
- [x] Respond to @rymai's comments
- [x] `class WikiLinkFilter < HTML::Pipeline::Filter`
- [x] blank line after guard clause
- [x] keyword arguments for `wiki` and `slug`
- [x] invert the condition
- [x] inline `user` in spec
- [x] Make sure spec names are not out of date
- [x] Comment for each rewrite rule
- [x] Add CHANGELOG entry
- [x] Reorganize commits
- [x] Make sure [build](https://gitlab.com/gitlab-org/gitlab-ce/commit/19b91e749a6320d12fb299d33f1f6440777e0e26/builds) passes
- [ ] Wait for merge
See merge request !4432
Original Comments
=================
- Linking behaves as per rules documented here:
https://gitlab.com/gitlab-org/gitlab-ce/blob/16568-document-wiki-linking-behavior/doc/markdown/wiki.md
- All links (to other wiki pages) are rewritten to be at the level of
the app root. We can't use links relative to the current
page ('./foo', 'foo', '../foo'), because they won't work in the
markdown preview, where the current page is suffixed with `/edit`
- Move existing `WikiLinkFilter` specs to `WikiPipeline` spec. It makes
sense to run these tests on the combined output of the pipeline,
rather than a single filter, since we can catch issues with
conflicting filters.
- Add more tests to cover the new linking
@rymai's Review
===============
- Classes nested under `WikiLinkFilter` should declare `WikiLinkFilter`'s
inherit, so nothing changes if the nested class is loaded first.
- Add a blank line after a guard clause
- Use keyword arguments for the `Rewriter` constructor
- Invert a condition - use `if` instead of `unless`
- Inline a `let` in `WikiPipeline` spec - it was only used in a single place
- Change out of date spec names
- Add a comment for every rewrite rule in `Rewriter`
Allow custom text on 'almost there' page
Add a new application setting, after_sign_up_text. This is text to be
rendered as Markdown and shown on the 'almost there' page after a user
signs up, but before they've confirmed their account.
Closes#14098.
See merge request !4098
Add a new application setting, after_sign_up_text. This is text to be
rendered as Markdown and shown on the 'almost there' page after a user
signs up, but before they've confirmed their account.
Tweak the styles for that page so that centered lists look reasonable.
This was not a clear cut n+1 query, given that if you're directly subscribed to all issues
that the API is returning you never really need to check for the notes. However, if you're
subscribed to _all_ of them, then for each issuable you need to go once to `notes`, and
once to `users` (for the authors). By preemtively loading notes and authors, at worst you have
1 extra query, and at best you saved 2n extra queries. We also took advantage of this preloading
of notes when counting user notes.
Dismantling Grack::Auth part 1: Git HTTP clients
Part of https://gitlab.com/gitlab-org/gitlab-ce/issues/14501
This does not completely get rid of Grack::Auth yet because Git LFS
support is 'behind' it and I would like to not make this MR bigger
than needed.
- changed tests to make HTTP requests instead of calling Rack apps
- added missing test cases for Git HTTP authentication
- moved Git HTTP requests into a 'normal' Rails controller
See merge request !3361
Check if GitHub rate limite API was reached before update Webhooks
## What does this MR do?
Checks if the job needs to sleep, and wait for the rate limit to be reseted before update each Webhook.
## Are there points in the code the reviewer needs to double check?
No.
## Why was this MR needed?
The import process can fail if the API rate limit was reached during the import process.
## What are the relevant issue numbers?
https://gitlab.com/gitlab-org/gitlab-ce/issues/17498
## Screenshots (if relevant)
Not relevant.
See merge request !4509
Add new GitLab CI configuration facade class
## What does this MR do?
This MR is a first iteration of major CI configuration refactoring. See #17139 and #15060 for more details.
## What are the relevant issue numbers?
Closes#17139
See merge request !4462
Kamil Trzcinski
Douwe Maan
Sean McGivern
Yorick Peterse
Gabriel MazettoandRobert Speicher
Stan Hu
Robert Speicher
Rémy Coutable
Jacob Vosmaer
Rui AndersonandRémy Coutable
Douwe Maan
Jacob Schatz
Douglas Barbosa Alexandre
Fatih Acet
Timothy Andrew
Patricio Cano
Alfredo Sumaran
Alejandro Rodríguez