aboutsummaryrefslogtreecommitdiffstats
path: root/yjit.c
Commit message (Collapse)AuthorAgeFilesLines
* [Feature #20470] Split GC into gc_impl.cPeter Zhu2024-07-031-1/+1
| | | | | | | | | | | | | | | This commit splits gc.c into two files: - gc.c now only contains code not specific to Ruby GC. This includes code to mark objects (which the GC implementation may choose not to use) and wrappers for internal APIs that the implementation may need to use (e.g. locking the VM). - gc_impl.c now contains the implementation of Ruby's GC. This includes marking, sweeping, compaction, and statistics. Most importantly, gc_impl.c only uses public APIs in Ruby and a limited set of functions exposed in gc.c. This allows us to build gc_impl.c independently of Ruby and plug Ruby's GC into itself.
* Optimized forwarding callers and calleesAaron Patterson2024-06-181-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch optimizes forwarding callers and callees. It only optimizes methods that only take `...` as their parameter, and then pass `...` to other calls. Calls it optimizes look like this: ```ruby def bar(a) = a def foo(...) = bar(...) # optimized foo(123) ``` ```ruby def bar(a) = a def foo(...) = bar(1, 2, ...) # optimized foo(123) ``` ```ruby def bar(*a) = a def foo(...) list = [1, 2] bar(*list, ...) # optimized end foo(123) ``` All variants of the above but using `super` are also optimized, including a bare super like this: ```ruby def foo(...) super end ``` This patch eliminates intermediate allocations made when calling methods that accept `...`. We can observe allocation elimination like this: ```ruby def m x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end def bar(a) = a def foo(...) = bar(...) def test m { foo(123) } end test p test # allocates 1 object on master, but 0 objects with this patch ``` ```ruby def bar(a, b:) = a + b def foo(...) = bar(...) def test m { foo(1, b: 2) } end test p test # allocates 2 objects on master, but 0 objects with this patch ``` How does it work? ----------------- This patch works by using a dynamic stack size when passing forwarded parameters to callees. The caller's info object (known as the "CI") contains the stack size of the parameters, so we pass the CI object itself as a parameter to the callee. When forwarding parameters, the forwarding ISeq uses the caller's CI to determine how much stack to copy, then copies the caller's stack before calling the callee. The CI at the forwarded call site is adjusted using information from the caller's CI. I think this description is kind of confusing, so let's walk through an example with code. ```ruby def delegatee(a, b) = a + b def delegator(...) delegatee(...) # CI2 (FORWARDING) end def caller delegator(1, 2) # CI1 (argc: 2) end ``` Before we call the delegator method, the stack looks like this: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | 5| delegatee(...) # CI2 (FORWARDING) | 6| end | 7| | 8| def caller | -> 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The ISeq for `delegator` is tagged as "forwardable", so when `caller` calls in to `delegator`, it writes `CI1` on to the stack as a local variable for the `delegator` method. The `delegator` method has a special local called `...` that holds the caller's CI object. Here is the ISeq disasm fo `delegator`: ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` The local called `...` will contain the caller's CI: CI1. Here is the stack when we enter `delegator`: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 -> 4| # | CI1 (argc: 2) 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | 9| delegator(1, 2) # CI1 (argc: 2) | 10| end | ``` The CI at `delegatee` on line 5 is tagged as "FORWARDING", so it knows to memcopy the caller's stack before calling `delegatee`. In this case, it will memcopy self, 1, and 2 to the stack before calling `delegatee`. It knows how much memory to copy from the caller because `CI1` contains stack size information (argc: 2). Before executing the `send` instruction, we push `...` on the stack. The `send` instruction pops `...`, and because it is tagged with `FORWARDING`, it knows to memcopy (using the information in the CI it just popped): ``` == disasm: #<ISeq:delegator@-e:1 (1,0)-(1,39)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 1] "..."@0 0000 putself ( 1)[LiCa] 0001 getlocal_WC_0 "..."@0 0003 send <calldata!mid:delegatee, argc:0, FCALL|FORWARDING>, nil 0006 leave [Re] ``` Instruction 001 puts the caller's CI on the stack. `send` is tagged with FORWARDING, so it reads the CI and _copies_ the callers stack to this stack: ``` Executing Line | Code | Stack ---------------+---------------------------------------+-------- 1| def delegatee(a, b) = a + b | self 2| | 1 3| def delegator(...) | 2 4| # | CI1 (argc: 2) -> 5| delegatee(...) # CI2 (FORWARDING) | cref_or_me 6| end | specval 7| | type 8| def caller | self 9| delegator(1, 2) # CI1 (argc: 2) | 1 10| end | 2 ``` The "FORWARDING" call site combines information from CI1 with CI2 in order to support passing other values in addition to the `...` value, as well as perfectly forward splat args, kwargs, etc. Since we're able to copy the stack from `caller` in to `delegator`'s stack, we can avoid allocating objects. I want to do this to eliminate object allocations for delegate methods. My long term goal is to implement `Class#new` in Ruby and it uses `...`. I was able to implement `Class#new` in Ruby [here](https://github.com/ruby/ruby/pull/9289). If we adopt the technique in this patch, then we can optimize allocating objects that take keyword parameters for `initialize`. For example, this code will allocate 2 objects: one for `SomeObject`, and one for the kwargs: ```ruby SomeObject.new(foo: 1) ``` If we combine this technique, plus implement `Class#new` in Ruby, then we can reduce allocations for this common operation. Co-Authored-By: John Hawthorn <john@hawthorn.email> Co-Authored-By: Alan Wu <XrXr@users.noreply.github.com>
* YJIT: implement variable-length context encoding scheme (#10888)Maxime Chevalier-Boisvert2024-06-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Implement BitVector data structure for variable-length context encoding * Rename method to make intent clearer * Rename write_uint => push_uint to make intent clearer * Implement debug trait for BitVector * Fix bug in BitVector::read_uint_at(), enable more tests * Add one more test for good measure * Start sketching Context::encode() * Progress on variable length context encoding * Add tests. Fix bug. * Encode stack state * Add comments. Try to estimate context encoding size. * More compact encoding for stack size * Commit before rebase * Change Context::encode() to take a BitVector as input * Refactor BitVector::read_uint(), add helper read functions * Implement Context::decode() function. Add test. * Fix bug, add tests * Rename methods * Add Context::encode() and decode() methods using global data * Make encode and decode methods use u32 indices * Refactor YJIT to use variable-length context encoding * Tag functions as allow unused * Add a simple caching mechanism and stats for bytes per context etc * Add comments, fix formatting * Grow vector of bytes by 1.2x instead of 2x * Add debug assert to check round-trip encoding-decoding * Take some rustfmt formatting * Add decoded_from field to Context to reuse previous encodings * Remove olde context stats * Re-add stack_size assert * Disable decoded_from optimization for now
* YJIT: Stop asserting rb_objspace_markable_object_p()Alan Wu2024-04-261-1/+0
| | | | | Because of the way things are sequenced, it doesn't work properly during auto-compaction.
* YJIT: Fix reference update for `Invariants::no_ep_escape_iseqs`Alan Wu2024-04-261-8/+2
| | | | | | | | Previously, the update was done in the ISEQ callback. That effectively never updated anything because the callback itself is given an intact reference, so it could update its content, and `rb_gc_location(iseq)` never returned a new address. Update the whole table once in the YJIT root instead.
* YJIT: Optimize local variables when EP == BP (take 2) (#10607)Takashi Kokubun2024-04-251-0/+6
| | | | | | | | | | | | | * Revert "Revert "YJIT: Optimize local variables when EP == BP" (#10584)" This reverts commit c8783441952217c18e523749c821f82cd7e5d222. * YJIT: Take care of GC references in ISEQ invariants Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> --------- Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
* Revert "YJIT: Optimize local variables when EP == BP" (#10584)Alan Wu2024-04-191-6/+0
| | | | | | This reverts commit 4cc58ea0b865f2fd20f1e881ddbd4c4fab0b072c. Since the change landed call-threshold=1 CI runs have been timing out. There has also been `verify-ctx` violations. Revert for now while we debug.
* YJIT: Optimize local variables when EP == BP (#10487)Takashi Kokubun2024-04-171-0/+6
|
* Refactor VM root modulesJean Boussier2024-03-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This `st_table` is used to both mark and pin classes defined from the C API. But `vm->mark_object_ary` already does both much more efficiently. Currently a Ruby process starts with 252 rooted classes, which uses `7224B` in an `st_table` or `2016B` in an `RArray`. So a baseline of 5kB saved, but since `mark_object_ary` is preallocated with `1024` slots but only use `405` of them, it's a net `7kB` save. `vm->mark_object_ary` is also being refactored. Prior to this changes, `mark_object_ary` was a regular `RArray`, but since this allows for references to be moved, it was marked a second time from `rb_vm_mark()` to pin these objects. This has the detrimental effect of marking these references on every minors even though it's a mostly append only list. But using a custom TypedData we can save from having to mark all the references on minor GC runs. Addtionally, immediate values are now ignored and not appended to `vm->mark_object_ary` as it's just wasted space.
* YJIT: Support opt_invokebuiltin_delegate for leaf builtin (#10152)Takashi Kokubun2024-03-011-6/+8
|
* YJIT: Reject keywords hash in -1 arity cfunc splat supportAlan Wu2024-02-281-12/+10
| | | | `test_keyword.rb` caught this issue. Just need to run with `threshold=1`
* YJIT: Support splat with C methods with -1 arityAlan Wu2024-02-271-0/+45
| | | | | Usually we deal with splats by speculating that they're of a specific size. In this case, the C method takes a pointer and a length, so we can support changing sizes just fine.
* YJIT: Pass nil to anonymous kwrest when empty (#9972)Alan Wu2024-02-151-0/+6
| | | | | | | This is the same optimization as e4272fd29 ("Avoid allocation when passing no keywords to anonymous kwrest methods") but for YJIT. For anonymous kwrest parameters, nil is just as good as an empty hash. On the usage side, update `splatkw` to handle `nil` with a leaner path.
* YJIT: Add top ISEQ call counts to --yjit-stats (#9906)Takashi Kokubun2024-02-091-0/+24
|
* YJIT: Fix ruby2_keywords splat+rest and drop bogus checksAlan Wu2024-01-231-0/+13
| | | | | | | | | | | | | | | | | | YJIT didn't guard for ruby2_keywords hash in case of splat calls that land in methods with a rest parameter, creating incorrect results. The compile-time checks didn't correspond to any actual effects of ruby2_keywords, so it was masking this bug and YJIT was needlessly refusing to compile some code. About 16% of fallback reasons in `lobsters` was due to the ISeq check. We already handle the tagging part with exit_if_supplying_kw_and_has_no_kw() and should now have a dynamic guard for all splat cases. Note for backporting: You also need 7f51959ff1. [Bug #20195]
* YJIT: Fix unused warningsAlan Wu2024-01-101-6/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` warning: unused import: `condition::Condition` --> src/asm/arm64/arg/mod.rs:13:9 | 13 | pub use condition::Condition; | ^^^^^^^^^^^^^^^^^^^^ | = note: `#[warn(unused_imports)]` on by default warning: unused import: `rb_yjit_fix_mul_fix as rb_fix_mul_fix` --> src/cruby.rs:188:9 | 188 | pub use rb_yjit_fix_mul_fix as rb_fix_mul_fix; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ warning: unused import: `rb_insn_len as raw_insn_len` --> src/cruby.rs:142:9 | 142 | pub use rb_insn_len as raw_insn_len; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: `#[warn(unused_imports)]` on by default ``` Make asm public so it stops warning about unused public stuff in there.
* YJIT: Add stats option to RubyVM::YJIT.enable (#9297)Takashi Kokubun2023-12-191-1/+1
|
* YJIT: Add RubyVM::YJIT.enable (#8705)Takashi Kokubun2023-10-191-8/+3
|
* YJIT: Remove duplicate cfp->iseq accessorAlan Wu2023-10-051-7/+0
|
* YJIT: Quiet mode when running with `--yjit-stats` (#8251)ywenc2023-08-181-0/+1
| | | Quiet mode for running with --yjit-stats
* YJIT: Compile exception handlers (#8171)Takashi Kokubun2023-08-081-19/+43
| | | Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
* YJIT: Move ROBJECT_OFFSET_* to yjit.c (#8157)Takashi Kokubun2023-08-021-0/+7
|
* YJIT: Fallback send instructions to vm_sendish (#8106)Takashi Kokubun2023-07-241-0/+14
|
* YJIT: refactoring to allow for fancier call threshold logic (#8078)Maxime Chevalier-Boisvert2023-07-171-0/+6
| | | | | | | | | | | | | * YJIT: refactoring to allow for fancier call threshold logic * Avoid potentially compiling functions multiple times. * Update vm.c Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> --------- Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
* Unify length field for embedded and heap strings (#7908)Peter Zhu2023-06-061-2/+1
| | | | | | | | * Unify length field for embedded and heap strings The length field is of the same type and position in RString for both embedded and heap allocated strings, so we can unify it. * Remove RSTRING_EMBED_LEN
* `rb_bug` prints a newline after the messageNobuyoshi Nakada2023-05-201-1/+1
|
* YJIT: Fix raw sample stack lengths in exit traces (#7728)John Hawthorn2023-04-181-2/+3
| | | | | | | | | | yjit-trace-exits appends a synthetic sample for the instruction being exited, but we didn't increment the size of the stack. Fixing this count correctly lets us successfully generate a flamegraph from the exits. I also replaced the line number for instructions with 0, as I don't think the previous value had meaning. Co-authored-by: Adam Hess <HParker@github.com>
* Pull the shape tree out of the vm objectMatt Valentine-House2023-04-061-1/+1
|
* YJIT: Add codegen for Integer methods (#7665)Takashi Kokubun2023-04-051-1/+14
| | | | | | | * YJIT: Add codegen for Integer methods * YJIT: Update dependencies * YJIT: Fix Integer#[] for argc=2
* Remove an unneeded function copyTakashi Kokubun2023-04-011-2/+1
|
* YJIT: Add `--yjit-pause` and `RubyVM::YJIT.resume` (#7609)Maxime Chevalier-Boisvert2023-03-281-0/+1
| | | | | | | | | | | | | | | | | | | * YJIT: Add --yjit-pause and RubyVM::YJIT.resume This allows booting YJIT in a suspended state. We chose to add a new command line option as opposed to simply allowing YJIT.resume to work without any command line option because it allows for combining with YJIT tuning command line options. It also simpifies implementation. Paired with Kokubun and Maxime. * Update yjit.rb Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com> --------- Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
* YJIT: Constify EC to avoid an `as` pointer cast (#7591)Alan Wu2023-03-241-1/+1
|
* RJIT: Optimize String#bytesizeTakashi Kokubun2023-03-181-7/+0
|
* YJIT: Support entry for multiple PCs per ISEQ (GH-7535)Takashi Kokubun2023-03-171-0/+6
|
* Refactor jit_func_t and jit_execTakashi Kokubun2023-03-161-4/+1
| | | | | I closed https://github.com/ruby/ruby/pull/7543, but part of the diff seems useful regardless, so I extracted it.
* YJIT: Assert that we have the VM lock while markingAlan Wu2023-03-151-0/+8
| | | | | Somewhat important because having the lock is a key part of the soundness reasoning for the `unsafe` usage here.
* YJIT: Introduce no_gc attribute (#7511)Takashi Kokubun2023-03-141-12/+17
|
* YJIT: Handle rest+splat where non-splat < required (#7499)Jimmy Miller2023-03-131-0/+7
|
* Rename builtin attr :inline to :leafTakashi Kokubun2023-03-111-1/+1
|
* Support multiple attributes with Primitive.attr!Takashi Kokubun2023-03-111-7/+1
|
* YJIT: Handle splat+rest for args pass greater than required (#7468)Jimmy Miller2023-03-071-0/+3
| | | | | | | | | | | For example: ```ruby def my_func(x, y, *rest) p [x, y, rest] end my_func(1, 2, 3, *[4, 5]) ```
* Adjust `else` style to be consistent in each files [ci skip]Nobuyoshi Nakada2023-02-261-1/+2
|
* Fix incorrect line numbers in GC hookPeter Zhu2023-02-241-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the previous instruction is not a leaf instruction, then the PC was incremented before the instruction was ran (meaning the currently executing instruction is actually the previous instruction), so we should not increment the PC otherwise we will calculate the source line for the next instruction. This bug can be reproduced in the following script: ``` require "objspace" ObjectSpace.trace_object_allocations_start a = 1.0 / 0.0 p [ObjectSpace.allocation_sourceline(a), ObjectSpace.allocation_sourcefile(a)] ``` Which outputs: [4, "test.rb"] This is incorrect because the object was allocated on line 10 and not line 4. The behaviour is correct when we use a leaf instruction (e.g. if we replaced `1.0 / 0.0` with `"hello"`), then the output is: [10, "test.rb"]. [Bug #19456]
* YJIT: Show Context stats on exit (#7327)Takashi Kokubun2023-02-161-1/+1
|
* YJIT: Optimize != for Integers and Strings (#7301)Takashi Kokubun2023-02-141-0/+6
|
* Merge gc.h and internal/gc.hMatt Valentine-House2023-02-091-1/+1
| | | | [Feature #19425]
* YJIT: Handle splat with opt more fully (#7209)Jimmy Miller2023-01-311-0/+6
| | | | | | | | | * YJIT: Handle splat with opt more fully * Update yjit/src/codegen.rs --------- Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
* YJIT: implement codegen for `String#empty?` (#7148)Maxime Chevalier-Boisvert2023-01-181-0/+6
| | | YJIT: implement codegen for String#empty?
* YJIT: Add object shape count to stats (#6754)Takashi Kokubun2022-11-171-0/+8
|
* Implement optimize call (#6691)Jimmy Miller2022-11-081-0/+9
| | | This dispatches to a c func for doing the dynamic lookup. I experimented with chain on the proc but wasn't able to detect which call sites would be monomorphic vs polymorphic. There is definitely room for optimization here, but it does reduce exits.
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-05 10:56:06 +0000