Pre-calculate Emoji digests
cc @dzaporozhets @rspeicher
@marin Do you happen to know if we allow users to configure asset hosts anywhere in Omnibus and such? I've not been able to find any reference to "asset_host" so it seems we don't allow this (which is a good thing as this simplifies the code).
See merge request !3458
By pre-calculating the digests we can manually construct the emoji URLs,
removing the need for using Rails' asset URL helpers. The reason we
don't want to use these helpers for Emojis is two-fold:
1. Rails' image_url() method is slow, really slow. For one it _might_
have to calculate digests but it also performs a lot of other
intensive operations (judging by the source code and based on
measuring timings).
2. We have a lot of Emoji which coupled with the above can result in it
taking minutes to load Emoji autocomplete data.
Using this pre-calculation setup generating the digests takes around 7
seconds (including the time it takes to start Rails/Rake), and only
around 600 milliseconds to load _all_ the autocomplete data of a project
(measured locally).
This commit _does_ change the Emoji URLs from absolute to relative URLs
as these are much easier to generate.
To update the Emoji data simply run:
rake gemojione:digests
Then commit any changes.
Fixesgitlab-org/gitlab-ce#14009
A few things to note:
- The IncomingEmail feature is now enabled even without a
correctly-formatted sub-address
- Message-ID for new thread mail are kept the same so that subsequent
notifications to this thread are grouped in the thread by the email
service that receives the notification
(i.e. In-Reply-To of the answer == Message-ID of the first thread message)
- To maximize our chance to be able to retrieve the reply key, we look
for it in the In-Reply-To header and the References header
- The pattern for the fallback reply message id is "reply-[key]@[gitlab_host]"
- Improve docs thanks to Axil
Add a `gemojione:sprite` Rake task
This task will generate a standard and Retina sprite of all of the
current Gemojione Emojis, with the accompanying SCSS map.
It will not appear in `rake -T` output, and the dependent gems are not
included in the Gemfile by default, because this task will only be
needed occasionally.
See merge request !3130
This task will generate a standard and Retina sprite of all of the
current Gemojione Emojis, with the accompanying SCSS map.
It will not appear in `rake -T` output, and the dependent gems are not
included in the Gemfile by default, because this task will only be
needed occasionally.
[ci skip]
We did a small experiment to see how a full scan of the Redis keys on
gitlab.com speeds up as we increase the batch size. The values on the
right are time in seconds for a full scan (no delete operations).
count: 10); 284.500529021
count: 100); 86.21216934
count: 1_000); 60.931676195
count: 10_000); 60.96355610
count: 100_000); 62.378172541
It looks like 1,000 is a good number.
Add user repository integrity check rake task
Corrupt repositories and stuck lock files can cause weird issues in
GitLab. Often we know which user is having these problems and then we
have to go hunt down which repository is causing it. Several times
recently that involved me running queries in the rails console to get
an array of projects and then writing a quick Ruby script to loop
through and run `git fsck`. This last time I also had to check for the
existence of `config.lock` and ref lock files.
This rake task will eliminate all of those steps and allow an admin to
simply specify a username.
I also added the lock file checks to the existing `gitlab:repo:check`
task which goes through all projects.
See merge request !2080
Ci Project migrate
- This doesn't migrate: allow_git_fetch, coverage_regex, timeout. Since this are project configuration settings I would propose to migrate them to `.gitlab-ci.yml`.
- This requires offline migrations.
- It simplifies database models making all CI objects to be attached to: Project.
- It removes Ci::Project, but makes /ci/projects working by adding method: Project.find_by_ci_id for backward compatibility (badges, triggers).
- We should add default `timeout` to Application Settings.
- It misses specs.
- It is based on ci-services-migrate for now.
- It removes CI events.
- It removes administrator CI projects overview.
- It removes CI application settings.
In 8.4 or 8.5 we can remove redundant tables and columns.
See merge request !1987
Robert Speicher
Arinde Eniola
Jacob Vosmaer
Dmitriy Zaporozhets
Yorick Peterse
Rémy Coutable
Drew Blessing
connorshea
Douwe Maan
Douwe Maan
ashleys
Robert Speicher
Kamil Trzcinski
Peter Suschlik
Marin Jankovski
Achilleas Pipinellis
Stan Hu
Zeger-Jan van de Weg