Add user repository integrity check rake task
Corrupt repositories and stuck lock files can cause weird issues in
GitLab. Often we know which user is having these problems and then we
have to go hunt down which repository is causing it. Several times
recently that involved me running queries in the rails console to get
an array of projects and then writing a quick Ruby script to loop
through and run `git fsck`. This last time I also had to check for the
existence of `config.lock` and ref lock files.
This rake task will eliminate all of those steps and allow an admin to
simply specify a username.
I also added the lock file checks to the existing `gitlab:repo:check`
task which goes through all projects.
See merge request !2080
Ci Project migrate
- This doesn't migrate: allow_git_fetch, coverage_regex, timeout. Since this are project configuration settings I would propose to migrate them to `.gitlab-ci.yml`.
- This requires offline migrations.
- It simplifies database models making all CI objects to be attached to: Project.
- It removes Ci::Project, but makes /ci/projects working by adding method: Project.find_by_ci_id for backward compatibility (badges, triggers).
- We should add default `timeout` to Application Settings.
- It misses specs.
- It is based on ci-services-migrate for now.
- It removes CI events.
- It removes administrator CI projects overview.
- It removes CI application settings.
In 8.4 or 8.5 we can remove redundant tables and columns.
See merge request !1987
- Offloads uploading to GitLab Workhorse
- Use /authorize request for fast uploading
- Added backup recipes for artifacts
- Support download acceleration using X-Sendfile
Performance is improved in two steps:
1. On PostgreSQL an expression index is used for checking lower(email)
and lower(username).
2. The check to determine if we're searching for a username or Email is
moved to Ruby. Thanks to @haynes for suggesting and writing the
initial implementation of this.
Moving the check to Ruby makes this method an additional 1.5 times
faster compared to doing the check in the SQL query.
With performance being improved I've now also tweaked the amount of
iterations required by the User.by_login benchmark. This method now runs
between 900 and 1000 iterations per second.
This benchmark suite uses benchmark-ips
(https://github.com/evanphx/benchmark-ips) behind the scenes. Specs can
be turned into benchmark specs by setting "benchmark" to "true" in the
top-level describe block like so:
describe SomeClass, benchmark: true do
end
Writing benchmarks can be done using custom RSpec matchers, for example:
describe MaruTheCat, benchmark: true do
describe '#jump_in_box' do
it 'should run 1000 iterations per second' do
maru = described_class.new
expect { maru.jump_in_box }.to iterate_per_second(1000)
end
end
end
By default the "iterate_per_second" expectation requires a standard
deviation under 30% (this is just an arbitrary default for now). You can
change this by chaining "with_maximum_stddev" on the expectation:
expect { maru.jump_in_box }.to iterate_per_second(1000)
.with_maximum_stddev(10)
This will change the expectation to require a maximum deviation of 10%.
Alternatively you can use the it block style to write specs:
describe MaruTheCat, benchmark: true do
describe '#jump_in_box' do
subject { -> { described_class.new } }
it { is_expected.to iterate_per_second(1000) }
end
end
Because "iterate_per_second" operates on a block, opposed to a static
value, the "subject" method must return a Proc. This looks a bit goofy
but I have been unable to find a nice way around this.
Improve repo cleanup task
I accidentally wrote a new script, not seeing we already had one.
But the old one did not do enough (it only handled global namespace orhpans) so I figured I should just drop in the new script.
See merge request !1298
Jacob Vosmaer
Achilleas Pipinellis
Robert Speicher
Stan Hu
Robert Speicher
Drew Blessing
Kamil Trzciński
Douwe Maan
Zeger-Jan van de Weg
Marin Jankovski
Dmitriy Zaporozhets
Mike Chmielewski
Benny Schimmer
Yorick Peterse
Zeger-Jan van de WegandZeger-Jan van de Weg