| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
| |
This reverts commit 6cb6d5abc36ede9d5158c2cd90734134838e6bfb.
This reverts commit 1484b786aee8d411a9e2278ac6d6e44aedbf6662.
I think we don't need these assertions anymore. I believe the problem
is solved by abf678a4397c6c00a1bb686043e377d372e695a4
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We are seeing an error where code that is generated with MJIT contains
references to objects that have been moved. I believe this is due to a
race condition in the compaction function.
`gc_compact` has two steps:
1. Run a full GC to pin objects
2. Compact / update references
Step one is executed with `garbage_collect`. `garbage_collect` calls
`gc_enter` / `gc_exit`, these functions acquire a JIT lock and release a
JIT lock. So a lock is held for the duration of step 1.
Step two is executed by `gc_compact_after_gc`. It also holds a JIT
lock.
I believe the problem is that the JIT is free to execute between step 1
and step 2. It copies call cache values, but doesn't pin them when it
copies them. So the compactor thinks it's OK to move the call cache
even though it is not safe.
We need to hold a lock for the duration of `garbage_collect` *and*
`gc_compact_after_gc`. This patch introduces a lock level which
increments and decrements. The compaction function can increment and
decrement the lock level and prevent MJIT from executing during both
steps.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
```
compiling ../mjit.c
In file included from ../mjit.c:28:
../mjit_worker.c:1270:33: warning: incompatible pointer to integer conversion passing 'const struct rb_callcache *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(BUILTIN_TYPE(cc) != T_MOVED);
^~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../include/ruby/internal/value_type.h:153:23: note: passing argument to parameter 'obj' here
RB_BUILTIN_TYPE(VALUE obj)
^
In file included from ../mjit.c:28:
../mjit_worker.c:1271:33: warning: incompatible pointer to integer conversion passing 'const struct rb_callable_method_entry_struct *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(BUILTIN_TYPE(vm_cc_cme(cc)) != T_MOVED);
^~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../include/ruby/internal/value_type.h:153:23: note: passing argument to parameter 'obj' here
RB_BUILTIN_TYPE(VALUE obj)
^
In file included from ../mjit.c:28:
../mjit_worker.c:1272:50: warning: incompatible pointer to integer conversion passing 'const struct rb_callcache *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(!rb_objspace_garbage_object_p(cc));
^~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../gc.h:128:40: note: passing argument to parameter 'obj' here
int rb_objspace_garbage_object_p(VALUE obj);
^
In file included from ../mjit.c:28:
../mjit_worker.c:1273:50: warning: incompatible pointer to integer conversion passing 'const struct rb_callable_method_entry_struct *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(!rb_objspace_garbage_object_p(vm_cc_cme(cc)));
^~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../gc.h:128:40: note: passing argument to parameter 'obj' here
int rb_objspace_garbage_object_p(VALUE obj);
^
4 warnings generated.
```
|
|
|
|
|
|
| |
This is a temporary commit to try to find a GC issue. It seems like
mjit is pointing at a moved address in the call cache. I want to assert
that they aren't TMOVED or garbage objects at the time they get copied
|
|
|
|
|
| |
Devs do not love "3". The only exception is RUBY3_KEYWORDS in parse.y,
which seems unrelated to our interests.
|
| |
|
|
|
|
|
|
|
|
| |
For some reason 0776198486 didn't fail
https://rubyci.org/logs/rubyci.s3.amazonaws.com/solaris10-gcc/ruby-master/log/20200503T230004Z.log.html.gz
even while it was before 9aa5fe1bf8.
Anyway, there's no need to keep the change anymore.
|
|
|
|
| |
I think 9aa5fe1bf89db8cd215b24d8ddfb668714681b83 helps this issue too.
|
|
|
|
|
|
|
|
| |
MinGW test_jit fails with no error message. Perhaps linker flags should
not be passed when compilation is happening.
Anyway splitting these stages doesn't matter for performance. So let me
just split it to fix the issue. Probably this helps Solaris's issue too.
|
|
|
|
|
|
| |
using -Winvalid-pch
https://rubyci.org/logs/rubyci.s3.amazonaws.com/solaris10-gcc/ruby-master/log/20200501T170004Z.fail.html.gz
|
|
|
|
| |
This support was accidentally removed in 818d6d33368a396d9cd3d1a34a84015a9e76c5c8.
|
|
|
|
|
|
|
| |
It's to avoid memory leak for actual usage (because they don't get
unloaded properly), but also for fixing CI timed out due to JIT
compaction taking too long time on --jit-wait (which runs every time)
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2911601
|
|
|
|
|
|
|
| |
to fix
https://ci.appveyor.com/project/ruby/ruby/builds/32577700/job/yh61rom35wt2uv39
It was for JIT compaction, and we don't support it on mswin.
|
|
|
|
|
|
| |
as well. And also unit->c_file doesn't exist in mswin.
https://github.com/ruby/ruby/runs/635915704
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
to improve code locality.
Using benchmark-driver/sinatra with 100 methods JIT-ed,
[Before] 12149.97 rps
1.3M /tmp/_ruby_mjit_p31171u145.so
[After] 12818.83 rps
260K /tmp/_ruby_mjit_p32155u145.so
(VM is 13714.89 rps)
|
|
|
|
|
| |
I'm trying to make it possible to include all JIT-ed code in a single C
file. This is needed to guarantee uniqueness of all function names
|
|
|
|
|
|
|
|
|
|
| |
Running C compiler for JIT compaction inside a critical section may lock
main thread for a long time when it triggers GC. As I'm planning to
increase this duration a bit, I'd like to make sure this doesn't stop
the world.
For now, I chose to give up unloading units when it's during JIT
compaction, assuming other calls may unload them later.
|
|
|
|
| |
Namely icc defines __GNUC__, but doesn't have -Wdeprecated-declarations
|
|
|
|
|
|
|
|
|
| |
```
c:\projects\ruby\mjit_worker.c(1219) : warning C4090: 'function' : different 'const' qualifiers
```
It seems confused by passing "pointer to pointer to const object",
not "pointer to const object".
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
for perf output like:
Samples: 100K of event 'cycles:ppp', Event count (approx.): 1007750000
Children Self Command Shared Object Symbol
+ 81.58% 1.47% ruby ruby [.] rb_vm_exec
+ 81.06% 7.61% ruby ruby [.] vm_exec_core
+ 80.16% 0.00% ruby ruby [.] vm_sendish (inlined)
+ 75.03% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 74.37% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 73.42% 0.22% ruby _ruby_mjit_p11277u42.so [.] _mjit42_rack_method_override_rb_call
+ 73.25% 0.10% ruby _ruby_mjit_p11277u41.so [.] _mjit41_sinatra_show_exceptions_rb_call
+ 73.19% 0.22% ruby _ruby_mjit_p11277u44.so [.] _mjit44_rack_head_rb_call
+ 73.03% 0.15% ruby _ruby_mjit_p11277u45.so [.] _mjit45_sinatra_base_rb_call
+ 72.87% 0.26% ruby _ruby_mjit_p11277u49.so [.] _mjit49_rack_logger_rb_call
+ 70.56% 0.11% ruby _ruby_mjit_p11277u40.so [.] _mjit40_sinatra_base_rb_call
+ 68.70% 0.11% ruby _ruby_mjit_p11277u39.so [.] _mjit39_sinatra_base_rb_call
+ 68.39% 0.29% ruby _ruby_mjit_p11277u56.so [.] _mjit56_rack_protection_frame_options_rb_call
+ 67.89% 0.18% ruby _ruby_mjit_p11277u37.so [.] _mjit37_sinatra_base_rb_block_in_call
+ 67.04% 0.16% ruby _ruby_mjit_p11277u34.so [.] _mjit34_sinatra_base_rb_synchronize
Reverting deb1c7b97d, fixing `sprint_funcname`'s argument in `compact_all_jit_code`.
Also updating common.mk.
|
|
|
|
|
|
| |
This reverts commit cecebf55c476ae936f3e880477dfb62149143c46.
debugging test failure...
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
for perf output like:
Samples: 100K of event 'cycles:ppp', Event count (approx.): 1007750000
Children Self Command Shared Object Symbol
+ 81.58% 1.47% ruby ruby [.] rb_vm_exec
+ 81.06% 7.61% ruby ruby [.] vm_exec_core
+ 80.16% 0.00% ruby ruby [.] vm_sendish (inlined)
+ 75.03% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 74.37% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 73.42% 0.22% ruby _ruby_mjit_p11277u42.so [.] _mjit42_rack_method_override_rb_call
+ 73.25% 0.10% ruby _ruby_mjit_p11277u41.so [.] _mjit41_sinatra_show_exceptions_rb_call
+ 73.19% 0.22% ruby _ruby_mjit_p11277u44.so [.] _mjit44_rack_head_rb_call
+ 73.03% 0.15% ruby _ruby_mjit_p11277u45.so [.] _mjit45_sinatra_base_rb_call
+ 72.87% 0.26% ruby _ruby_mjit_p11277u49.so [.] _mjit49_rack_logger_rb_call
+ 70.56% 0.11% ruby _ruby_mjit_p11277u40.so [.] _mjit40_sinatra_base_rb_call
+ 68.70% 0.11% ruby _ruby_mjit_p11277u39.so [.] _mjit39_sinatra_base_rb_call
+ 68.39% 0.29% ruby _ruby_mjit_p11277u56.so [.] _mjit56_rack_protection_frame_options_rb_call
+ 67.89% 0.18% ruby _ruby_mjit_p11277u37.so [.] _mjit37_sinatra_base_rb_block_in_call
+ 67.04% 0.16% ruby _ruby_mjit_p11277u34.so [.] _mjit34_sinatra_base_rb_synchronize
|
|
|
|
|
|
|
|
| |
OpenBSD RubyCI has failed with SEGV since 4bcd5981e80d3e1852c8723741a0069779464128.
https://rubyci.org/logs/rubyci.s3.amazonaws.com/openbsd-current/ruby-master/log/20200312T223005Z.fail.html.gz
This was because `status->cc_entries` could be stale after `realloc` call
for inlined iseqs.
|
|
|
|
|
|
|
|
| |
jit_unit to avoid marking wrong cc entries when inlined iseq is compiled
multiple times, resolving the TODO added by daf7c48d88.
This obviates pseudo jit_unit in inlined iseq introduced by 7ec2359374
and fixes memory leak of the adhoc unit.
|
| |
|
|
|
|
|
| |
Looks like an oversight in b9007b6c548f91e88fd3f2ffa23de740431fa969 and
7ec23593746c8ccabd6c005cc34dde77d564c6c9.
|
|
|
|
| |
Fixed a TODO in b9007b6c548f91e88fd3f2ffa23de740431fa969
|
|
|
|
| |
Add a little more details in 7ec23593746c8ccabd6c005cc34dde77d564c6c9
|
|
|
|
| |
shared by ko1
|
|
|
|
|
| |
GC can invoke just after allocation of jit_unit->cc_entries so
it should be zero-cleared.
|
|
|
|
|
|
|
| |
ALLOC_N() can causes GC. Sometimes `mjit_copy_job_handler()`
can be called by mjit_worker thread which is not a Ruby thread,
so we need to prevent GC in this function. This patch has some
issues, but I introduce it to pass the tests.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch contains several ideas:
(1) Disposable inline method cache (IMC) for race-free inline method cache
* Making call-cache (CC) as a RVALUE (GC target object) and allocate new
CC on cache miss.
* This technique allows race-free access from parallel processing
elements like RCU.
(2) Introduce per-Class method cache (pCMC)
* Instead of fixed-size global method cache (GMC), pCMC allows flexible
cache size.
* Caching CCs reduces CC allocation and allow sharing CC's fast-path
between same call-info (CI) call-sites.
(3) Invalidate an inline method cache by invalidating corresponding method
entries (MEs)
* Instead of using class serials, we set "invalidated" flag for method
entry itself to represent cache invalidation.
* Compare with using class serials, the impact of method modification
(add/overwrite/delete) is small.
* Updating class serials invalidate all method caches of the class and
sub-classes.
* Proposed approach only invalidate the method cache of only one ME.
See [Feature #16614] for more details.
|
|
|
|
|
|
|
|
| |
This is a secret feature for me. It's only for testing and any behavior
with this flag override is unsupported.
I needed this because I sometimes want to add debug options but do not
want to disable optimizations, for using Linux perf.
|
|
|
|
|
|
|
| |
in active_units
Hoping to fix:
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/2311375
|
|
|
|
|
| |
`if (unit->iseq)` might have referred to a freed unit. Therefore this
commit delays its free.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now I'm not exactly sure why I needed to check `stop_worker_p` after
`mjit_copy_cache_from_main_thread` of `convert_unit_to_func`
in 4161674b2fbea6bdd01783ac5d3b39d88db22972.
If it's for avoiding deadlock under `in_gc` condition, we should keep it.
However, if it's not the case and it's just for retrying accidental
compilation failure or just to avoid `MJIT_ATOMIC_SET` and
`compact_all_jit_code`, I think this quick stop path is not mandatory.
Because this path is somewhat problematic in my upcoming fix in
mjit_worker, let me try to remove this first and see how CI goes.
|
|
|
|
|
|
| |
To avoid:
cannot locate symbol "modf" referenced by .../_ruby_mjit_XXX.so"
|
|
|
|
|
|
|
|
|
|
|
| |
for all compilations and compaction.
Prior to this commit, the last-compiled code has not been used because
MJIT worker is stopped before setting the code, and compaction has also
been skipped.
But it was not intentional and `wait: true` pause should wait until
those two things by its feature.
|
|
|
|
|
|
|
|
| |
to drop MJIT_DLDFLAGS from compile_c_to_o for OpenBSD failure
https://rubyci.org/logs/rubyci.s3.amazonaws.com/openbsd65/ruby-master/log/20190830T110008Z.fail.html.gz.
8c7f4e8f8b7f9e74195ea0eb51f39014fec342c0 did not work for i686-linux
https://travis-ci.org/ruby/ruby/jobs/578808112.
|
|
|
|
|
|
|
|
| |
I think this did not work for some environments, but this seems to fix
OpenBSD RubyCI failure:
https://rubyci.org/logs/rubyci.s3.amazonaws.com/openbsd65/ruby-master/log/20190830T110008Z.fail.html.gz
Let me check RubyCI impact by this.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To fix "gcc: -lgcc: linker input file unused because linking not done" in
https://rubyci.org/logs/rubyci.s3.amazonaws.com/openbsd65/ruby-master/log/20190826T200009Z.log.html.gz
Still I'm intentionally leaving CC_DLDFLAGS_ARGS because making compiler
options different might result in compile/link failure. (Of course
CC_DLDFLAGS_ARGS usually should not have compiler options, but it might
include by bad configure setup)
The same problem may exist in MJIT_LIBS used inside CC_LIBS as well, but
I just ignored that case because it impacts only MinGW / cygwin, hoping
that their users do not perform a wrong configure.
|
|
|
|
| |
This issue is detected by Coverity Scan.
|
|
|
|
|
| |
Having a block for mixing a declaration was confusing.
Also I moved `dev_null` and `pid` to limit their scope.
|
|
|
|
| |
This issue is detected by Coverity Scan.
|
|
|
|
| |
by C99 compound literal for array and non-constant array initializer
|
|
|
|
| |
[Bug #15986]
|
|
|
|
| |
Mostly non-constatnt array initializer, and mixed declarations and code.
|
|
|
|
|
|
|
|
| |
in GC.compact.
While `in_jit` is false, GC.compact is allowed to run and it may be
moving ISeq-related pointers. So calling rb_iseq_path() when `in_jit`
is true is illegal.
|
|
|
|
|
|
| |
by showing line number only when it's Fixnum.
When it's not Fixnum, we need to call a method to know the line number.
|
|
|
|
|
| |
because FIX2INT might crash by moving method entry pointer:
http://ci.rvm.jp/results/trunk-mjit@silicon-docker/2063412
|