aboutsummaryrefslogtreecommitdiffstats
path: root/yjit/src/asm
Commit message (Collapse)AuthorAgeFilesLines
* Add a newline at EOF [ci skip]Nobuyoshi Nakada2023-05-241-1/+1
|
* YJIT: Fix build on A64Alan Wu2023-04-111-1/+1
| | | | Typo fix for the last commit (1432b37)
* YJIT: Fix a compilation warning in x86_64Takashi Kokubun2023-04-111-0/+1
| | | | This is used only for arm64's cb.jmp_ptr_bytes().
* YJIT: Reduce paddings if --yjit-exec-mem-size <= 128 on arm64 (#7671)Takashi Kokubun2023-04-111-7/+9
| | | | | | | * YJIT: Reduce paddings if --yjit-exec-mem-size <= 128 on arm64 * YJIT: Define jmp_ptr_bytes on CodeBlock
* YJIT: Count the number of actually written bytes (#7658)Takashi Kokubun2023-04-051-13/+29
|
* YJIT: code_gc(): Assert self is inline to avoid other_cb()Alan Wu2023-03-291-3/+6
| | | | | | | The derived `&mut` from `other_cb()` overlapped with the parameter `ocb`. Use `cfg!()` instead of `#[cfg...]` to avoid unused warnings.
* YJIT: Fix overlapping &mut in Assembler::code_gc()Alan Wu2023-03-291-10/+6
| | | | | | | | | Making overlapping `&mut`s triggers Undefined Bahavior. This function previously had them through `cb` and `ocb` aliasing with `self` or live references in the caller. To fix the overlap, take `ocb` as a parameter and don't use `get_inline_cb()` in the body of the function.
* YJIT: Fix a cargo test warning on x86_64 (#7428)Takashi Kokubun2023-03-031-0/+1
|
* YJIT: Delete stale `frozen_bytes` related code (#7423)Alan Wu2023-03-021-5/+0
| | | | | | | | | | | | | | The code and comments in there have been disabled by comments for a long time. The issues that the counter used to solve are now solved more comprehensively by "runningness" [tracking][1] introduced by Code GC and [delayed deallocation][2]. Having a single counter doesn't fit our current model where code pages that could be touched or not are interleaved, anyway. Just delete the code. [1]: e7c71c6c9271b0c29f210769159090e17128e740 [2]: a0b0365e905e1ac51998ace7e6fc723406a2f157
* YJIT: Fix assertion for partially mapped last pages (#7337)Takashi Kokubun2023-02-201-1/+1
| | | Follows up [Bug #19400]
* YJIT: add counters for polymorphic send and send with known class (#7288)Maxime Chevalier-Boisvert2023-02-101-2/+2
|
* YJIT: Use the system page size when the code page size is too small (#7267)Alan Wu2023-02-091-28/+50
| | | | | | | | | | | | | | | | | | Previously on ARM64 Linux systems that use 64 KiB pages (`CONFIG_ARM64_64K_PAGES=y`), YJIT was panicking on boot due to a failed assertion. The assertion was making sure that code GC can free the last code page that YJIT manages without freeing unrelated memory. YJIT prefers picking 16 KiB as the granularity at which to free code memory, but when the system can only free at 64 KiB granularity, that is not possible. The fix is to use the system page size as the code page size when the system page size is 64 KiB. Continue to use 16 KiB as the code page size on common systems that use 16/4 KiB pages. Add asserts to code_gc() and free_page() about code GC's assumptions. Fixes [Bug #19400]
* Fix typos in YJIT [ci skip]Alan Wu2023-02-023-4/+4
|
* YJIT: other_cb is None in testsAlan Wu2023-02-021-0/+1
| | | | | Since the other cb is in CodegenGlobals, and we want Rust tests to be self-contained.
* YJIT: Move CodegenGlobals::freed_pages into an RcAlan Wu2023-02-021-13/+35
| | | | | This allows for supplying a freed_pages vec in Rust tests. We need it so we can test scenarios that occur after code GC.
* Add stats so we can keep track of x86 rel32 vs register calls (#7142)Maxime Chevalier-Boisvert2023-01-181-0/+4
| | | | | | | * Add stats so we can keep track of x86 rel32 vs register calls To know if we get that "prime real estate" as Alan put it. * Fix bug pointed by Alan
* Enable `clippy` checks for yjit in CI (#7093)Ian Ker-Seymer2023-01-121-0/+2
| | | | | | | | | | | * Add job to check clippy lints in CI * Address all remaining clippy lints * Check lints on arm64 as well * Apply latest clippy lints * Do not exit 0 on clippy warnings
* Strip trailing spaces [ci skip]Nobuyoshi Nakada2023-01-122-3/+3
|
* YJIT: Fix a compilation warning with release build (#7092)Takashi Kokubun2023-01-101-1/+4
| | | | | | | | | | | | | | | warning: unused variable: `start_addr` --> ../yjit/src/asm/mod.rs:359:39 | 359 | pub fn remove_comments(&mut self, start_addr: CodePtr, end_addr: CodePtr) { | ^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_start_addr` | = note: `#[warn(unused_variables)]` on by default warning: unused variable: `end_addr` --> ../yjit/src/asm/mod.rs:359:60 | 359 | pub fn remove_comments(&mut self, start_addr: CodePtr, end_addr: CodePtr) { |
* YJIT: Remove old comments for regenerated branches (#7083)Takashi Kokubun2023-01-091-0/+7
|
* YJIT: Remove --yjit-code-page-size (#6865)Alan Wu2022-12-051-30/+29
| | | | | Certain code page sizes don't work and can cause crashes, so having this value available as a command-line option is a bit dangerous. Remove it and turn it into a constant instead.
* YJIT: Respect destination num_bits on STUR (#6848)Takashi Kokubun2022-12-011-2/+7
|
* YJIT: fix 32 and 16 bit register store (#6840)Jemma Issroff2022-12-012-1/+22
| | | | | | | | | | | | | | | | | | | | * Fix 32 and 16 bit register store in YJIT Co-Authored-By: Takashi Kokubun <takashikkbn@gmail.com> * Remove an unnecessary diff * Reuse an rm_num_bits result * Use u16::MAX instead * Update the link Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> * Just use sturh for 16 bits Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com> Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
* YJIT: Fix IseqPayload::pages memory bloatAlan Wu2022-11-301-1/+1
| | | | | | HashSet::clear() doesn't deallocate the backing buffer and shrink the capacity. Replace with a 0-capcity set instead so we reclaim some memory each code GC.
* YJIT: Use NonNull pointer for CodePtr (#6792)Takashi Kokubun2022-11-231-1/+2
|
* Fix YJIT backend to account for unsigned int immediates (#6789)Jemma Issroff2022-11-232-4/+20
| | | | | | | | | | | | | | | | | YJIT: x86_64: Fix cmp with number where sign bit is set Before this commit, we were unconditionally treating unsigned ints as signed ints when counting the number of bits required for representing the immediate in machine code. When the size of the immediate matches the size of the other operand, no sign extension happens, so this was incorrect. `asm.cmp(opnd64, 0x8000_0000)` panicked even though it's encodable as `CMP r/m32, imm32`. Large shape ids were impacted by this issue. Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org> Co-Authored-By: Alan Wu <alanwu@ruby-lang.org> Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Co-authored-by: Alan Wu <alanwu@ruby-lang.org>
* 32 bit comparison on shape idAaron Patterson2022-11-181-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit changes the shape id comparisons to use a 32 bit comparison rather than 64 bit. That means we don't need to load the shape id to a register on x86 machines. Given the following program: ```ruby class Foo def initialize @foo = 1 @bar = 1 end def read [@foo, @bar] end end foo = Foo.new foo.read foo.read foo.read foo.read foo.read puts RubyVM::YJIT.disasm(Foo.instance_method(:read)) ``` The machine code we generated _before_ this change is like this: ``` == BLOCK 1/4, ISEQ RANGE [0,3), 65 bytes ====================== # getinstancevariable 0x559a18623023: mov rax, qword ptr [r13 + 0x18] # guard object is heap 0x559a18623027: test al, 7 0x559a1862302a: jne 0x559a1862502d 0x559a18623030: cmp rax, 4 0x559a18623034: jbe 0x559a1862502d # guard shape, embedded, and T_OBJECT 0x559a1862303a: mov rcx, qword ptr [rax] 0x559a1862303d: movabs r11, 0xffff00000000201f 0x559a18623047: and rcx, r11 0x559a1862304a: movabs r11, 0xb000000002001 0x559a18623054: cmp rcx, r11 0x559a18623057: jne 0x559a18625046 0x559a1862305d: mov rax, qword ptr [rax + 0x18] 0x559a18623061: mov qword ptr [rbx], rax == BLOCK 2/4, ISEQ RANGE [3,6), 0 bytes ======================= == BLOCK 3/4, ISEQ RANGE [3,6), 47 bytes ====================== # gen_direct_jmp: fallthrough # getinstancevariable # regenerate_branch # getinstancevariable # regenerate_branch 0x559a18623064: mov rax, qword ptr [r13 + 0x18] # guard shape, embedded, and T_OBJECT 0x559a18623068: mov rcx, qword ptr [rax] 0x559a1862306b: movabs r11, 0xffff00000000201f 0x559a18623075: and rcx, r11 0x559a18623078: movabs r11, 0xb000000002001 0x559a18623082: cmp rcx, r11 0x559a18623085: jne 0x559a18625099 0x559a1862308b: mov rax, qword ptr [rax + 0x20] 0x559a1862308f: mov qword ptr [rbx + 8], rax ``` After this change, it's like this: ``` == BLOCK 1/4, ISEQ RANGE [0,3), 41 bytes ====================== # getinstancevariable 0x5560c986d023: mov rax, qword ptr [r13 + 0x18] # guard object is heap 0x5560c986d027: test al, 7 0x5560c986d02a: jne 0x5560c986f02d 0x5560c986d030: cmp rax, 4 0x5560c986d034: jbe 0x5560c986f02d # guard shape 0x5560c986d03a: cmp word ptr [rax + 6], 0x19 0x5560c986d03f: jne 0x5560c986f046 0x5560c986d045: mov rax, qword ptr [rax + 0x10] 0x5560c986d049: mov qword ptr [rbx], rax == BLOCK 2/4, ISEQ RANGE [3,6), 0 bytes ======================= == BLOCK 3/4, ISEQ RANGE [3,6), 23 bytes ====================== # gen_direct_jmp: fallthrough # getinstancevariable # regenerate_branch # getinstancevariable # regenerate_branch 0x5560c986d04c: mov rax, qword ptr [r13 + 0x18] # guard shape 0x5560c986d050: cmp word ptr [rax + 6], 0x19 0x5560c986d055: jne 0x5560c986f099 0x5560c986d05b: mov rax, qword ptr [rax + 0x18] 0x5560c986d05f: mov qword ptr [rbx + 8], rax ``` The first ivar read is a bit more complex, but the second ivar read is much simpler. I think eventually we could teach the context about the shape, then emit only one shape guard.
* YJIT: Always encode Opnd::Value in 64 bits on x86_64 for GC offsets (#6733)Takashi Kokubun2022-11-152-0/+20
| | | | | | | | | | | | | | | | | * YJIT: Always encode Opnd::Value in 64 bits on x86_64 for GC offsets Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> * Introduce heap_object_p * Leave original mov intact * Remove unneeded branches * Add a test for movabs Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
* YJIT: Include actual memory region size in stats (#6736)Takashi Kokubun2022-11-151-2/+5
|
* Implement LDURH on Aarch64Aaron Patterson2022-11-142-0/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When RUBY_DEBUG is enabled, shape ids are 16 bits. I would like to do 16 bit comparisons, so I need to load halfwords sometimes. This commit adds LDURH so that I can load halfwords. https://developer.arm.com/documentation/ddi0596/2021-12/Base-Instructions/LDURH--Load-Register-Halfword--unscaled--?lang=en I verified the bytes using clang: ``` $ cat asmthing.s .global _start .align 2 _start: ldurh w10, [x1] ldurh w10, [x1, #123] $ as asmthing.s -o asmthing.o && objdump --disassemble asmthing.o asmthing.o: file format mach-o arm64 Disassembly of section __TEXT,__text: 0000000000000000 <ltmp0>: 0: 2a 00 40 78 ldurh w10, [x1] 4: 2a b0 47 78 ldurh w10, [x1, #123] ```
* YJIT: Reset dropped_bytes when patching codeAlan Wu2022-11-081-0/+6
| | | | | | | | | | We switch to a new page when we detect dropped_bytes flipping from false to true. Previously, when we patch code for invalidation during code gc, we start with the flag being set to true, so we failed to apply patches that straddle pages. We would write out jumps half way and then stop, which left the code corrupted. Reset the flag before patching so we patch across pages properly.
* YJIT: Free pages after ObjectSpace API usages (#6676)Takashi Kokubun2022-11-071-13/+15
|
* YJIT: Make Code GC metrics available for non-stats builds (#6665)Takashi Kokubun2022-11-031-2/+0
|
* YJIT: Stop incrementing write_pos if cb.has_dropped_bytes (#6664)Takashi Kokubun2022-11-031-6/+6
| | | | | Co-Authored-By: Alan Wu <alansi.xingwu@shopify.com> Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
* YJIT: Avoid accumulating freed pages in the payload (#6657)Takashi Kokubun2022-11-021-0/+6
| | | | | | | Co-Authored-By: Alan Wu <alansi.xingwu@shopify.com> Co-Authored-By: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
* YJIT: Add RubyVM::YJIT.code_gc (#6644)Takashi Kokubun2022-10-311-6/+18
| | | | | * YJIT: Add RubyVM::YJIT.code_gc * Rename compiled_page_count to live_page_count
* YJIT: GC and recompile all code pages (#6406)Takashi Kokubun2022-10-251-10/+169
| | | | | when it fails to allocate a new page. Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
* YJIT: Fix page rounding for icache bustingAlan Wu2022-10-211-5/+7
| | | | | | | | | | | | | | Previously, we found the current page by rounding the current pointer to the closest smaller page size. This is incorrect because pages are relative to the start of the address we reserve. For example, if the starting address is 12KiB modulo the 16KiB page size, once we have more than 4KiB of code, calculating with the address would incorrectly give us page 1 when we're actually still on page 0. Previously, I can reproduce crashes with: make btest RUN_OPTS=--yjit-code-page-size=32 on ARM64 macOS, where system page sizes are 16KiB.
* YJIT: Skip dumping code for the other cb on --yjit-dump-disasm (#6592)Takashi Kokubun2022-10-191-2/+1
| | | | | YJIT: Skip dumping code for the other cb on --yjit-dump-disasm
* YJIT: fix a #[warn(unused_parens)]Alan Wu2022-10-191-1/+1
|
* YJIT: fold the "asm_comments" feature into "disasm" (#6591)Alan Wu2022-10-192-7/+7
| | | | | Previously, enabling only "disasm" didn't actually build. Since these two features are closely related and we don't really use one without the other, let's simplify and merge the two features together.
* Code clean around unused code for some architectures or features (#6581)Jimmy Miller2022-10-181-4/+3
|
* YJIT: Interleave inline and outlined code blocks (#6460)Takashi Kokubun2022-10-171-39/+201
| | | | Co-authored-by: Alan Wu <alansi.xingwu@shopify.com> Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
* More clippy fixes (#6547)Jimmy Miller2022-10-141-6/+12
|
* fixes more clippy warnings (#6543)Jimmy Miller2022-10-131-2/+2
| | | | | * fixes more clippy warnings * Fix x86 c_callable to have doc_strings
* Make op_ext an optional for code clarity (#6542)Jimmy Miller2022-10-131-42/+50
|
* Revert "Revert "This commit implements the Object Shapes technique in CRuby.""Jemma Issroff2022-10-111-1/+1
| | | | This reverts commit 9a6803c90b817f70389cae10d60b50ad752da48f.
* YJIT: fix ARM64 bitmask encoding for 32 bit registers (#6503)Alan Wu2022-10-063-13/+114
| | | | | | | | | | | | | | For logical instructions such as AND, there is a constraint that the N part of the bitmask immediate must be 0. We weren't respecting this condition previously and were silently emitting undefined instructions. Check for this condition in the assembler and tweak the backend to correctly detect whether a number could be encoded as an immediate in a 32 bit logical instruction. Due to the nature of the immediate encoding, the same numeric value encodes differently depending on the size of the register the instruction works on. We currently don't have cases where we use 32 bit immediates but we ran into this encoding issue during development.
* Revert "This commit implements the Object Shapes technique in CRuby."Aaron Patterson2022-09-301-1/+1
| | | | This reverts commit 68bc9e2e97d12f80df0d113e284864e225f771c2.
* A bunch of clippy auto fixes for yjit (#6476)Jimmy Miller2022-09-309-19/+20
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-19 07:30:18 +0000