aboutsummaryrefslogtreecommitdiffstats
path: root/mjit.c
Commit message (Collapse)AuthorAgeFilesLines
...
* Capture inlined iseq's cc entries in root iseq'sTakashi Kokubun2020-03-101-16/+0
| | | | | | | | jit_unit to avoid marking wrong cc entries when inlined iseq is compiled multiple times, resolving the TODO added by daf7c48d88. This obviates pseudo jit_unit in inlined iseq introduced by 7ec2359374 and fixes memory leak of the adhoc unit.
* Prevent unloading methods used in root_fiber while calling another Fiber (#2939)Takashi Kokubun2020-02-281-0/+4
| | | | | | Fixing SEGVs like: http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744905 http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744420 http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2741400
* Avoid infinite loop on --jit-waitTakashi Kokubun2020-02-281-1/+8
|
* Internalize rb_mjit_unit definition againTakashi Kokubun2020-02-261-1/+7
| | | | Fixed a TODO in b9007b6c548f91e88fd3f2ffa23de740431fa969
* Remove an unused declarationTakashi Kokubun2020-02-251-2/+0
| | | | It was unnecessary in b9007b6c548f91e88fd3f2ffa23de740431fa969
* should be initialize jit_unit->cc_entries.Koichi Sasada2020-02-251-1/+1
| | | | | GC can invoke just after allocation of jit_unit->cc_entries so it should be zero-cleared.
* prevent GC from mjit worker.Koichi Sasada2020-02-251-6/+9
| | | | | | | ALLOC_N() can causes GC. Sometimes `mjit_copy_job_handler()` can be called by mjit_worker thread which is not a Ruby thread, so we need to prevent GC in this function. This patch has some issues, but I introduce it to pass the tests.
* Introduce disposable call-cache.Koichi Sasada2020-02-221-6/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch contains several ideas: (1) Disposable inline method cache (IMC) for race-free inline method cache * Making call-cache (CC) as a RVALUE (GC target object) and allocate new CC on cache miss. * This technique allows race-free access from parallel processing elements like RCU. (2) Introduce per-Class method cache (pCMC) * Instead of fixed-size global method cache (GMC), pCMC allows flexible cache size. * Caching CCs reduces CC allocation and allow sharing CC's fast-path between same call-info (CI) call-sites. (3) Invalidate an inline method cache by invalidating corresponding method entries (MEs) * Instead of using class serials, we set "invalidated" flag for method entry itself to represent cache invalidation. * Compare with using class serials, the impact of method modification (add/overwrite/delete) is small. * Updating class serials invalidate all method caches of the class and sub-classes. * Proposed approach only invalidate the method cache of only one ME. See [Feature #16614] for more details.
* VALUE size packed callinfo (ci).Koichi Sasada2020-02-221-4/+0
| | | | | | | | | | | | | | | | | | | | Now, rb_call_info contains how to call the method with tuple of (mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and mid+argc+flags only requires 64bits. So this patch packed rb_call_info to VALUE (1 word) on such cases. If we can not represent it in VALUE, then use imemo_callinfo which contains conventional callinfo (rb_callinfo, renamed from rb_call_info). iseq->body->ci_kw_size is removed because all of callinfo is VALUE size (packed ci or a pointer to imemo_callinfo). To access ci information, we need to use these functions: vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci). struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg. rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc() is temporary removed because cd->ci should be marked.
* function pointers are not void*卜部昌平2020-02-061-1/+1
| | | | The same as 8427fca49bd85205f5a8766292dd893f003c0e48.
* decouple internal.h headers卜部昌平2019-12-261-3/+9
| | | | | | | | | | | | | | | | | | Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies).
* mjit.c: fix a mismatch of malloc'ed typeYusuke Endoh2019-12-031-2/+2
| | | | Coverity Scan found this issue.
* Simplify variable declaration by C99Takashi Kokubun2019-12-011-8/+6
|
* Constified mjit_initNobuyoshi Nakada2019-12-011-1/+1
|
* ConstifiedNobuyoshi Nakada2019-12-011-2/+2
|
* Fix a mistake excluding NULL in the endTakashi Kokubun2019-12-011-2/+2
|
* Workaround missing strndup on WindowsTakashi Kokubun2019-12-011-1/+11
| | | | https://ci.appveyor.com/project/ruby/ruby/builds/29230976/job/c910t37313edb97k
* Allow specifying arbitrary MJIT flags by --jit-debugTakashi Kokubun2019-12-011-0/+32
| | | | | | | | This is a secret feature for me. It's only for testing and any behavior with this flag override is unsupported. I needed this because I sometimes want to add debug options but do not want to disable optimizations, for using Linux perf.
* Combine call info and cache to speed up method invocationAlan Wu2019-10-241-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | To perform a regular method call, the VM needs two structs, `rb_call_info` and `rb_call_cache`. At the moment, we allocate these two structures in separate buffers. In the worst case, the CPU needs to read 4 cache lines to complete a method call. Putting the two structures together reduces the maximum number of cache line reads to 2. Combining the structures also saves 8 bytes per call site as the current layout uses separate two pointers for the call info and the call cache. This saves about 2 MiB on Discourse. This change improves the Optcarrot benchmark at least 3%. For more details, see attached bugs.ruby-lang.org ticket. Complications: - A new instruction attribute `comptime_sp_inc` is introduced to calculate SP increase at compile time without using call caches. At compile time, a `TS_CALLDATA` operand points to a call info struct, but at runtime, the same operand points to a call data struct. Instruction that explicitly define `sp_inc` also need to define `comptime_sp_inc`. - MJIT code for copying call cache becomes slightly more complicated. - This changes the bytecode format, which might break existing tools. [Misc #16258]
* Remove the quick stop path after convert_unit_to_funcTakashi Kokubun2019-10-131-2/+0
| | | | | | | | | | | | | | Now I'm not exactly sure why I needed to check `stop_worker_p` after `mjit_copy_cache_from_main_thread` of `convert_unit_to_func` in 4161674b2fbea6bdd01783ac5d3b39d88db22972. If it's for avoiding deadlock under `in_gc` condition, we should keep it. However, if it's not the case and it's just for retrying accidental compilation failure or just to avoid `MJIT_ATOMIC_SET` and `compact_all_jit_code`, I think this quick stop path is not mandatory. Because this path is somewhat problematic in my upcoming fix in mjit_worker, let me try to remove this first and see how CI goes.
* RubyVM::MJIT.pause(wait: true) should waitTakashi Kokubun2019-09-261-0/+2
| | | | | | | | | | | for all compilations and compaction. Prior to this commit, the last-compiled code has not been used because MJIT worker is stopped before setting the code, and compaction has also been skipped. But it was not intentional and `wait: true` pause should wait until those two things by its feature.
* Fix wrong allocation failure handlingTakashi Kokubun2019-09-201-2/+2
| | | | | `iseq->body->jit_unit->compile_info` should not be referenced before the null check of `iseq->body->jit_unit`.
* * expand tabs. [ci skip]git2019-08-221-1/+1
| | | | | Tabs are expanded because previously the file did not have any tab indentation. Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
* Improve the description of MJIT cancel-all [ci skip]Takashi Kokubun2019-08-221-0/+1
|
* Rename rb_gc_new_location to rb_gc_locationAaron Patterson2019-05-181-2/+2
| | | | | The function will return new or existing locations depending on whether or not the object actually moved, so give it a more appropriate name.
* Rename mjit_gc_finish_hook to mjit_gc_exit_hookTakashi Kokubun2019-05-151-3/+3
| | | | | because @ko1 said "gc_finish" is confusing like a finish of entire GC process
* glibc says memcpy cannot take NULLUrabe, Shyouhei2019-04-291-1/+1
| | | | | | | | At least since 2004, glibc's <string.h> annotates memcpy as __attribute__((__nonnull__)). This basedir is passed to it. When LOAD_RELATIVE is not defined and MJIT_SEARCH_BUILD_DIR is not set, this variable is never updated. Should initialize with meaningful default value.
* Fix wrong critical section labelk0kubun2019-04-201-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Prefix rb_ to MJIT wait call since it's exportedk0kubun2019-04-201-5/+5
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Revert "Skip ISeq reference for stale_units for debugging""k0kubun2019-04-201-1/+15
| | | | | | | | This reverts commit b79899b56a9ebadf5c493c4df2419ebf63934381. I wanted to test r67638 first. Now let me try this as well. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Invalidate JIT-ed code if ISeq is moved by GC.compactk0kubun2019-04-201-0/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Skip ISeq reference for stale_units for debugging"k0kubun2019-04-201-15/+1
| | | | | | | | This reverts commit 4bd4d4e4cfd164f767e88f844cf10b06faee14b9. Sorry, let me test another one first git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67637 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Skip ISeq reference for stale_units for debuggingk0kubun2019-04-201-1/+15
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67636 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Check ISeq references in stale_units took0kubun2019-04-201-0/+10
| | | | | | This is a possible bug from recent "JIT recompile" introduction. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Update ISeq references in stale_units as wellk0kubun2019-04-201-0/+11
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* fix strict compile errorstenderlove2019-04-201-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67625 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Update MJIT referencestenderlove2019-04-201-0/+13
| | | | | | ISeq can move, so we need to tell MJIT where the new location is. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Do not execute MJIT copy job when ISeq is GC-edk0kubun2019-04-141-1/+9
| | | | | | | | | I assumed that ISeq is never GC-ed by `in_jit` + `mjit_mark` on copy job ISeq, but unfortunately I found SEGV on `mjit_copy_job_handler` in which iseq->body was somehow Qnil. And it seems to be fixed by disabling the job when `mjit_free_iseq` is called for the ISeq. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Prefer uintptr_t over ptrdiff_t for i686k0kubun2019-04-141-1/+1
| | | | | | | | as debugged in PR [close https://github.com/ruby/ruby/pull/2130] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67549 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Try to set false explicitly"k0kubun2019-04-141-5/+1
| | | | | | | | | | | | | | | | | | This reverts commit a1b5d20068a9b1859b383b249e510cd33d196e4a. Revert "Revert "Skip recompiling tests on i686 Linux"" This reverts commit 7b88a9207b97b94bc3c3be35084c742296f4aff2. Revert "Simplify matrix for debugging" This reverts commit e85d6c5c5e62dde37c6f6ffdb7125b9008b9ebfa. Sorry, these 3 commits were under debugging in https://github.com/ruby/ruby/pull/2129 but accidentally merged by using `git svn dcommit` instead of `git push` to the topic branch :bow: git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Try to set false explicitlyk0kubun2019-04-141-1/+5
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67537 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Unify comment styles across MJIT sourcesk0kubun2019-04-141-116/+115
| | | | | | | | | | | | I'm writing `//` comments in newer MJIT code after C99 enablement (because I write 1-line comments more often than multi-line comments and `//` requires fewer chars on 1-line) and then they are mixed with `/* */` now. For consistency and to avoid the conversion in future changes, let me finish the rewrite in MJIT-related code. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67533 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Recompile JIT-ed code without optimizationk0kubun2019-04-141-8/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | based on inline cache when JIT cancel happens by that. This feature was in the original MJIT implementation by Vladimir, but on merging MJIT to Ruby it was removed for simplification. This commit adds the functionality again for the following benchmark: https://github.com/benchmark-driver/misc/blob/52f05781f65467baf895bf6ba79d172c9b0826fd/concurrent-map/bench.rb (shown float is duration seconds. shorter is better) * Before ``` $ INHERIT=0 ruby -v bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) [x86_64-linux] -- 1.6507579649914987 $ INHERIT=0 ruby -v --jit bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) +JIT [x86_64-linux] -- 1.5091587850474752 $ INHERIT=1 ruby -v bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) [x86_64-linux] -- 1.6124781150138006 $ INHERIT=1 ruby --jit -v bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) +JIT [x86_64-linux] -- 1.7495657080435194 # <-- this ``` * After ``` $ INHERIT=0 ruby -v bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) [x86_64-linux] last_commit=Recompile JIT-ed code without optimization -- 1.653559010999743 $ INHERIT=0 ruby --jit -v bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) +JIT [x86_64-linux] last_commit=Recompile JIT-ed code without optimization -- 1.4738391840364784 $ INHERIT=1 ruby -v bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) [x86_64-linux] last_commit=Recompile JIT-ed code without optimization -- 1.645227018976584 $ INHERIT=1 ruby --jit -v bench.rb ruby 2.7.0dev (2019-04-13 trunk 67523) +JIT [x86_64-linux] last_commit=Recompile JIT-ed code without optimization -- 1.523708809982054 # <-- this ``` git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67530 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Prevent mjit length counters from being 0k0kubun2019-03-291-6/+0
| | | | | | | It seems that `debug_counter_show_results_at_exit` could be called earlier than `mjit_finish` sometimes. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67380 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Add debug counter for rb_mjit_unit_listk0kubun2019-03-291-0/+6
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67378 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Drop rb_mjit_unit from mjit_copy_jobk0kubun2019-03-171-9/+18
| | | | | | | | | and guard iseq from GC by marking iseq in mjit_copy_job. This is a refactoring for implementing inlining later and should not be fixing or introducing any bugs. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Fix typos [ci skip]kazu2019-03-171-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67283 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Change defaults of --jit optionsk0kubun2019-03-171-7/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * --jit-min-calls: 5 -> 10000 --jit-min-calls=5 obviously can compile non hotspot. This was not a problem for MJIT-benchmarks and Optcarrot because the former has very few hot optimiziable methods and the latter is likely to trigger compilation of hotspot by its intensive calls to optimizable hotspot methods and has a very short window to allow limited compilations. In real-world applications, it has more time to compile more methods and it pressures computer's limited resources like icache. We should avoid compiling too many methods. Also compiling many methods exhausts time budget for compilation in one ruby process lifetime and delays the "JIT compaction" of Ruby 2.6. JVM is known to use 1,500 for C1 (client) compiler and 10,000 for C2 (server) compiler for -XX:CompileThreshold by default. https://docs.oracle.com/javase/8/embedded/develop-apps-platforms/codecache.htm When things are called X,000 times, requiring 10,000 can eliminate compilation of methods which are called only once in these X,000 iterations and obviously not hotspot. And in fact things like unicorn-worker-killer restarts unicorn process every 4096 requests. So I felt 10,000 is good for such an environment. * --jit-max-cache: 1000 -> 100 By the same reason stated above, we should not allow compiling many methods especially on MJIT which has a larger overhead due to poor code locality by dlopen and whose code is also likely to be bigger by just inlining many VM instructions with -O3 rather than directly generating low-level code. In JVM -XX:ReservedCodeCacheSize is 32M for reserved and 48M for maximum. --jit-max-cache=1,000 could be closer to it, but in this case MJIT's compilation is slow due to data synchronization between threads (to be improved in Ruby 2.7 though) and we do not want to delay the "JIT compaction" for a long time. So I chose a really conservative number for this, but by having method inlining in the future, wider range could be optimized even with this value. * Optcarrot --disable-gems, --benchmark Lan_Master.nes 12 attempts. No significant impact. | r67276 | r67276 --jit | after --jit | |:-------------------|:------------------|:------------------| | 50.44369263063978 | 72.87390680773056 | 73.47873485047297 | | 50.58788746124193 | 78.06820808947026 | 78.29723420171945 | | 50.77509250801378 | 80.29010348842613 | 78.94689404460769 | | 50.935361702064405 | 80.42796829926374 | 80.39539527351525 | | 51.27352672981195 | 81.98758158033202 | 81.6754198664817 | | 51.720715743242124 | 82.00118535811626 | 82.22960569251283 | | 51.89643169822524 | 82.2290091613556 | 82.5013636146388 | | 51.95895898113868 | 82.37318990939565 | 82.74002377794454 | | 52.10124886807968 | 82.48796686037502 | 83.23354941183932 | | 52.292280637519376 | 83.0265226541066 | 84.01552618012572 | | 52.51856237784144 | 83.8797360318052 | 84.8588319093393 | | 52.65076845986818 | 84.80037351256634 | 85.13577756273656 | * Railsbench `WARMUP=20000 BENCHMARK=1000 bin/bench` of https://github.com/k0kubun/railsbench. It gets closer to --disable=jit. | | r67276 | r67276 | after | | | | --jit | --jit | |:----------|:-------|:-------|:-------| | req/s | 891.3 | 742.2 | 841.5 | |:----------|:-------|:-------|:-------| | 50%ile ms | 1.00 | 1.21 | 1.08 | | 66%ile ms | 1.02 | 1.24 | 1.09 | | 75%ile ms | 1.03 | 1.28 | 1.10 | | 80%ile ms | 1.03 | 1.30 | 1.11 | | 90%ile ms | 1.09 | 1.42 | 1.15 | | 95%ile ms | 1.32 | 1.65 | 1.27 | | 98%ile ms | 4.79 | 2.23 | 1.81 | | 99%ile ms | 5.68 | 7.52 | 6.64 | |100%ile ms | 6.52 | 9.69 | 8.59 | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67277 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* mjit.c: use boolean type for boolean variablesk0kubun2019-01-101-26/+26
| | | | | | | | | | and functions to clarify the intention and make sure it's not used in a surprising way (like using 2, 3, ... other than 0, 1 even while it seems to be a boolean). This is a retry of r66775. It included some typos... git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66778 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "mjit.c: use boolean type for boolean variables"k0kubun2019-01-101-18/+18
| | | | | | | | This reverts commit bb1a1aeab0f2a5fe437c89b841a887ba56653453. We hit something on ci.rvm.jp, reverting until investigation is done. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66776 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-27 14:21:51 +0000