diff options
author | k0kubun <k0kubun@b2dd03c8-39d4-4d8f-98ff-823fe69b080e> | 2018-07-28 16:14:56 +0000 |
---|---|---|
committer | k0kubun <k0kubun@b2dd03c8-39d4-4d8f-98ff-823fe69b080e> | 2018-07-28 16:14:56 +0000 |
commit | 443f4d583c8fe78198bee791f2ac3da0be2dfb5e (patch) | |
tree | de6e7e3448f5a63e8307a0a1024510220d47223b /test | |
parent | 2a9cae3328c978025f267eda1681ee0af78ca7e2 (diff) | |
download | ruby-443f4d583c8fe78198bee791f2ac3da0be2dfb5e.tar.gz |
mjit.c: introduce JIT compaction [experimental]
When all compilation finishes or the number of JIT-ed code reaches
--jit-max-cache, this compacts all generated code to a single .so file
and re-loads all methods from it.
In the future, it may trigger compaction more frequently and/or limit
the maximum times of compaction to prevent unlimited memory usage.
So the current behavior is experimental, but at least the performance
improvement in this commit won't be removed.
=== Benchmark ===
In this benchmark, I'll compare following four conditions:
* trunk: r64082
* trunk JIT: r64082 w/ --jit
* single-so JIT: This commit w/ --jit
* objfcn JIT: This branch https://github.com/k0kubun/ruby/tree/objfcn w/ --jit,
which is shinh's objfcn https://github.com/shinh/ruby/tree/objfcn rebased from this commit
```
$ uname -a
Linux bionic 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
```
* Micro benchmark
Using this script https://gist.github.com/k0kubun/10e6d3387c9ab1b134622b2c9d76ef51,
calls some amount of different methods that just return `nil`. The following tables
are its average duration seconds of 3 measurements.
Smaller is better.
** 1 method (seconds)
| | trunk | trunk JIT | single-so JIT | objfcn JIT |
|:------|:------------------|:------------------|:------------------|:------------------|
| Time | 5.576067774333296 | 5.915551971666446 | 5.833641665666619 | 5.845915191666639 |
| Ratio | 1.00x | 1.06x | 1.05x | 1.05x |
** 50 methods (seconds)
| | trunk | trunk JIT | single-so JIT | objfcn JIT |
|:------|:------------------|:------------------|:------------------|:------------------|
| Time | 3.1661167996666677| 6.125825928333342 | 4.135432743666665 | 3.750358728333348 |
| Ratio | 1.00x | 1.93x | 1.31x | 1.18x |
** 1500 methods (seconds)
| | trunk | trunk JIT | single-so JIT | objfcn JIT |
|:------|:------------------|:------------------|:------------------|:------------------|
| Time | 5.971650823666664 | 19.579182102999994| 10.511108153999961| 10.854653588999932|
| Ratio | 1.00x | 3.28x | 1.76x | 1.82x |
* Discourse
Using the same benchmark strategy as https://bugs.ruby-lang.org/issues/14490 with
this branch https://github.com/k0kubun/discourse/commits/benchmark2 forked from discourse
v1.8.11 to support running trunk.
1. Run ruby script/bench.rb to warm up profiling database
2. Run RUBYOPT='--jit-verbose=1 --jit-max-cache=10000' RAILS_ENV=profile bin/puma -e production
3. WAIT 5-15 or so minutes for all jitting to stop so we have no cross talk
4. Run ab -n 100 http://localhost:9292/
5. Wait for all new jitting to finish
6. Run ab -n 100 http://localhost:9292/
** Response time (ms)
Here is the response time milliseconds for each percentile.
Skipping 99%ile because it's the same as 100%ile in 100 calls.
| | trunk| trunk|single|objfcn|
| | | JIT|so JIT| JIT|
|:----|:-----|:-----|:-----|:-----|
| 50% | 38 | 45 | 41 | 43 |
| 66% | 39 | 50 | 44 | 44 |
| 75% | 47 | 51 | 46 | 45 |
| 80% | 49 | 52 | 47 | 47 |
| 90% | 50 | 63 | 50 | 52 |
| 95% | 60 | 79 | 52 | 55 |
| 98% | 91 | 114 | 91 | 91 |
|100% | 97 | 133 | 96 | 99 |
** Ratio (smaller is better)
Here is the response time increase ratio against no-JIT trunk's one.
| | trunk| trunk|single|objfcn|
| | | JIT|so JIT| JIT|
|:----|:-----|:-----|:-----|:-----|
| 50% | 1.00x| 1.18x| 1.08x| 1.13x|
| 66% | 1.00x| 1.28x| 1.13x| 1.13x|
| 75% | 1.00x| 1.09x| 0.98x| 0.96x|
| 80% | 1.00x| 1.06x| 0.96x| 0.96x|
| 90% | 1.00x| 1.26x| 1.00x| 1.04x|
| 95% | 1.00x| 1.32x| 0.87x| 0.92x|
| 98% | 1.00x| 1.25x| 1.00x| 1.00x|
|100% | 1.00x| 1.37x| 0.99x| 1.02x|
While 50 and 60 %ile are still worse than no-JIT trunk, 75, 80, 90, 95,
98 and 100% are not slower than that.
So now it's a little harder to say "MJIT slows down Rails applications".
Probably I can close [Bug #14490] now. Let's start improving it.
Close https://github.com/ruby/ruby/pull/1921
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64094 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Diffstat (limited to 'test')
-rw-r--r-- | test/ruby/test_jit.rb | 13 |
1 files changed, 11 insertions, 2 deletions
diff --git a/test/ruby/test_jit.rb b/test/ruby/test_jit.rb index 039a2ff35f..04b0a0b74a 100644 --- a/test/ruby/test_jit.rb +++ b/test/ruby/test_jit.rb @@ -7,6 +7,11 @@ require_relative '../lib/jit_support' class TestJIT < Test::Unit::TestCase include JITSupport + IGNORABLE_PATTERNS = [ + /\ASuccessful MJIT finish\n\z/, + /\AJIT compaction \(\d+\.\dms\): Compacted \d+ methods ->/, + ] + # trace_* insns are not compiled for now... TEST_PENDING_INSNS = RubyVM::INSTRUCTION_NAMES.select { |n| n.start_with?('trace_') }.map(&:to_sym) + [ # not supported yet @@ -544,7 +549,9 @@ class TestJIT < Test::Unit::TestCase end end; assert_equal('0123456789', out) - errs = err.lines + errs = err.lines.reject do |l| + IGNORABLE_PATTERNS.any? { |pat| pat.match?(l) } + end assert_match(/\A#{JIT_SUCCESS_PREFIX}: block in <main>@-e:/, errs[0]) 9.times do |i| assert_match(/\A#{JIT_SUCCESS_PREFIX}: mjit#{i}@\(eval\):/, errs[i + 1]) @@ -776,7 +783,9 @@ class TestJIT < Test::Unit::TestCase if stdout assert_equal(stdout, out, "Expected stdout #{out.inspect} to match #{stdout.inspect} with script:\n#{code_block(script)}") end - err_lines = err.lines.reject! { |l| l.chomp.empty? || l.match?(/\A#{JIT_SUCCESS_PREFIX}/) || l == "Successful MJIT finish\n" } + err_lines = err.lines.reject! do |l| + l.chomp.empty? || l.match?(/\A#{JIT_SUCCESS_PREFIX}/) || IGNORABLE_PATTERNS.any? { |pat| pat.match?(l) } + end unless err_lines.empty? warn err_lines.join(''), uplevel: uplevel end |