diff options
Diffstat (limited to 'doc/yjit/yjit.md')
| -rw-r--r-- | doc/yjit/yjit.md | 127 |
1 files changed, 102 insertions, 25 deletions
diff --git a/doc/yjit/yjit.md b/doc/yjit/yjit.md index 3c731247a3..a09ff8439d 100644 --- a/doc/yjit/yjit.md +++ b/doc/yjit/yjit.md @@ -51,39 +51,55 @@ Because there is no GC for generated code yet, your software could run out of ex ## Installation -Current YJIT versions are installed by default with CRuby. Make sure to specify the "--yjit" command line option to enable it at runtime. +### Requirements -Experimental YJIT patches that have not yet been merged with CRuby can be found in ruby-build: +You will need to install: +- A C compiler such as GCC or Clang +- GNU Make and Autoconf +- The Rust compiler `rustc` and Cargo (if you want to build in dev/debug mode) -``` -ruby-build yjit-dev ~/.rubies/ruby-yjit-dev -``` +To install the Rust build toolchain, we suggest following the [recommended installation method][rust-install]. Rust also provides first class [support][editor-tools] for many source code editors. + +[rust-install]: https://www.rust-lang.org/tools/install +[editor-tools]: https://www.rust-lang.org/tools -They can also be found in the Shopify/yjit repository, which is cloned and build like CRuby. +### Building YJIT -Start by cloning the `Shopify/yjit` repository: +Start by cloning the `ruby/ruby` repository: ``` -git clone https://github.com/Shopify/yjit +git clone https://github.com/ruby/ruby yjit cd yjit ``` -The YJIT `ruby` binary can be built with either GCC or Clang. For development, we recommend enabling debug symbols so that assertions are enabled as this makes debugging easier. Enabling debug mode will also make it possible for you to disassemble code generated by YJIT. However, this causes a performance hit. For maximum performance, compile with GCC, without the `-DRUBY_DEBUG` or `-DYJIT_STATS` build options. More detailed build instructions are provided in the [Ruby README](https://github.com/ruby/ruby#how-to-compile-and-install). -To support disassembly of the generated code, `libcapstone` is also required (`brew install capstone` on MacOS, `sudo apt-get install -y libcapstone-dev` on Ubuntu/Debian and `sudo dnf -y install capstone-devel` on Fedora). +The YJIT `ruby` binary can be built with either GCC or Clang. It can be built either in dev (debug) mode or in release mode. For maximum performance, compile YJIT in release mode with GCC. More detailed build instructions are provided in the [Ruby README](https://github.com/ruby/ruby#how-to-compile-and-install). To support disassembly of the generated code, `libcapstone` is also required (`brew install capstone` on MacOS, `sudo apt-get install -y libcapstone-dev` on Ubuntu/Debian and `sudo dnf -y install capstone-devel` on Fedora). ``` -# Configure with debugging/stats options for development, build and install +# Configure in release mode for maximum performance, build and install ./autogen.sh -./configure cppflags="-DRUBY_DEBUG -DYJIT_STATS" --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc --disable--install-rdoc -make -j16 install +./configure --enable-yjit --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc --disable--install-rdoc +make -j install +``` + +or + +``` +# Configure in dev (debug) mode for development, build and install +./autogen.sh +./configure --enable-yjit=dev --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc --disable--install-rdoc +make -j install ``` On macOS, you may need to specify where to find openssl, libyaml and gdbm: ``` -# Configure with debugging/stats options for development, build and install -./configure cppflags="-DRUBY_DEBUG -DYJIT_STATS" --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc --disable--install-rdoc --with-opt-dir=$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml):$(brew --prefix gdbm) -make -j16 install +# Install dependencies +brew install openssl readline libyaml + +# Configure in dev (debug) mode for development, build and install +./autogen.sh +./configure --enable-yjit=dev --prefix=$HOME/.rubies/ruby-yjit --disable-install-doc --disable--install-rdoc --with-opt-dir=$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml) +make -j install ``` Typically configure will choose default C compiler. To specify the C compiler, use @@ -100,7 +116,7 @@ You can test that YJIT works correctly by running: make btest # Complete set of tests -make -j16 test-all +make -j test-all ``` ## Usage @@ -128,10 +144,10 @@ The machine code generated for a given method can be printed by adding `puts Rub YJIT supports all command-line options supported by upstream CRuby, but also adds a few YJIT-specific options: -- `--disable-yjit`: turn off YJIT (enabled by default) -- `--yjit-stats`: produce statistics after the execution of a program (must compile with `cppflags=-DRUBY_DEBUG` to use this) -- `--yjit-exec-mem-size=N`: size of the executable memory block to allocate, in MiB (default 256 MiB) +- `--yjit`: enable YJIT (disabled by default) - `--yjit-call-threshold=N`: number of calls after which YJIT begins to compile a function (default 2) +- `--yjit-exec-mem-size=N`: size of the executable memory block to allocate, in MiB (default 256 MiB) +- `--yjit-stats`: produce statistics after the execution of a program (must compile with `cppflags=-DRUBY_DEBUG` to use this) - `--yjit-max-versions=N`: maximum number of versions to generate per basic block (default 4) - `--yjit-greedy-versioning`: greedy versioning mode (disabled by default, may increase code size) @@ -215,12 +231,16 @@ you can contribute things we will want to merge into YJIT. ### Source Code Organization The YJIT source code is divided between: -- `yjit_asm.c`: x86 in-memory assembler we use to generate machine code -- `yjit_codegen.c`: logic for translating Ruby bytecode to machine code -- `yjit_core.c`: basic block versioning logic, core structure of YJIT -- `yjit_iface.c`: code YJIT uses to interface with the rest of CRuby +- `yjit.c`: code YJIT uses to interface with the rest of CRuby - `yjit.h`: C definitions YJIT exposes to the rest of the CRuby - `yjit.rb`: `YJIT` Ruby module that is exposed to Ruby +- `yjit/src/asm/*`: in-memory assembler we use to generate machine code +- `yjit/src/codegen.rs`: logic for translating Ruby bytecode to machine code +- `yjit/src/core.rb`: basic block versioning logic, core structure of YJIT +- `yjit/src/stats.rs`: gathering of run-time statistics +- `yjit/src/options.rs`: handling of command-line options +- `yjit/bindgen/src/main.rs`: C bindings exposed to the Rust codebase through bindgen +- `yjit/src/cruby.rs`: C bindings manually exposed to the Rust codebase - `misc/test_yjit_asm.sh`: script to compile and run the in-memory assembler tests - `misc/yjit_asm_tests.c`: tests for the in-memory assembler @@ -229,6 +249,20 @@ The core of CRuby's interpreter logic is found in: - `vm_insnshelper.c`: logic used by Ruby's bytecode instructions - `vm_exec.c`: Ruby interpreter loop +### Generating C bindings with bindgen + +In order to expose C functions to the Rust codebase, you will need to generate C bindings: + +```sh +CC=clang ./configure --enable-yjit=dev +make -j yjit-bindgen +``` + +This uses the bindgen tools to generate/update `yjit/src/cruby_bindings.inc.rs` based on the +bindings listed in `yjit/bindgen/src/main.rs`. Avoid manually editing this file +as it could be automatically regenerated at a later time. If you need to manually add C bindings, +add them to `yjit/cruby.rs` instead. + ### Coding & Debugging Protips There are 3 test suites: @@ -240,7 +274,7 @@ There are 3 test suites: The tests can be run in parallel like this: ``` -make -j16 test-all RUN_OPTS="--yjit-call-threshold=1" +make -j test-all RUN_OPTS="--yjit-call-threshold=1" ``` Or single-threaded like this, to more easily identify which specific test is failing: @@ -273,3 +307,46 @@ You can use the Intel syntax for disassembly in LLDB, keeping it consistent with ``` echo "settings set target.x86-disassembly-flavor intel" >> ~/.lldbinit ``` + +## Running YJIT on M1 + +It is possible to run YJIT on an Apple M1 via Rosetta. You can find basic +instructions below, but there are a few caveats listed further down. + +First, install Rosetta: + +``` +$ softwareupdate --install-rosetta +``` + +Now any command can be run with Rosetta via the `arch` command line tool. + +Then you can start your shell in an x86 environment: + +``` +$ arch -x86_64 zsh +``` + +You can double check your current architecture via the `arch` command: + +``` +$ arch -x86_64 zsh +$ arch +i386 +``` + +You may need to set the default target for `rustc` to x86-64, e.g. + +``` +$ rustup default stable-x86_64-apple-darwin +``` + +While in your i386 shell, install Cargo and Homebrew, then hack away! + +### M1 Caveats + +1. You must install a version of Homebrew for each architecture +2. Cargo will install in $HOME/.cargo by default, and I don't know a good way to change architectures after install +3. `dev` won't work if you have i386 Homebrew installed on an M1 + +If you use Fish shell you can [read this link](https://tenderlovemaking.com/2022/01/07/homebrew-rosetta-and-ruby.html) for information on making the dev environment easier. |