aboutsummaryrefslogtreecommitdiffstats
path: root/enc
Commit message (Collapse)AuthorAgeFilesLines
* * enc/utf_8.c: Add new alias UTF-8-HFS for UTF8-MAC.naruse2010-05-111-0/+2
| | | | | | http://www.gnu.org/software/emacs/NEWS.23.2 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * properties.nobu2010-05-071-0/+0
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27658 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/iso2022.trans: CP50221 supports 8bit JIS.naruse2010-04-011-7/+12
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27149 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/utf_16{be,le}.c (utf16{be,le}_mbc_to_code): simplified.nobu2010-04-012-6/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27143 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * bignum.c, node.h, strftime.c, enc/trans/utf8_mac.trans: added explicit ↵muraken2010-03-251-1/+1
| | | | | | casts for supplessing warnings. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27040 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb (transcode_compile_tree): makeakr2010-03-218-17/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | valid_encoding mandatory unless from_encoding is registered in ValidEncoding. (transcode_tbl_only): ditto. (transcode_tblgen): ditto. (ValidEncoding): new function. * enc/trans/escape.trans: specify valid_encoding. * enc/trans/emoji_sjis_docomo.trans: ditto. * enc/trans/emoji.trans: ditto. * enc/trans/emoji_iso2022_kddi.trans: ditto. * enc/trans/big5.trans: ditto. * enc/trans/emoji_sjis_softbank.trans: ditto. * enc/trans/emoji_sjis_kddi.trans: ditto. * enc/trans/chinese.trans: use ValidEncoding() instead of ValidEncoding[]. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26995 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/emoji.trans: added codepoints leading 0xf4 into nomap_table.muraken2010-03-161-0/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb (transcode_tblgen): add valid_encodingakr2010-03-152-6/+7
| | | | | | | | | | | | | optional argument. * enc/trans/single_byte.trans use valid_encoding argument for transcode_tblgen. * enc/trans/chinese.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26941 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/emoji.trans: fix nomap_table.akr2010-03-141-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb: reject ambiguous mapping.akr2010-03-131-1/+1
| | | | | | | | * enc/trans/single_byte.trans: remove ambiguous maping such as \xD6 -> U+05F2 and \xD6\xC7 -> U+FB1F in Windows-1255 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/x_emoji.h: renamed from enc/x-emoji.c.muraken2010-03-101-0/+0
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26863 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/EMOJI/*.src, enc/trans/emoji*, enc/x-emoji.c, ↵muraken2010-03-0918-0/+14964
| | | | | | | | test/ruby/enc/test_emoji.rb, tool/enc-emoji-citrus-gen.rb, tool/enc-emoji4unicode.rb, tool/jisx0208.rb, tool/test/test_jisx0208.rb: new encodings to support emoji charsets, which are used by Japanese mobile phones [ruby-dev:40528]. Thanks Yoji Shidara for a lot of contribution. * tool/transcode-tblgen.rb: modified for enc-emoji4unicode.rb. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/oniguruma.h: updated to follow Oniguruma 5.9.2.matz2010-03-018-19/+10
| | | | | | | | * re.c (make_regexp): use onig_new() instead of onig_alloc_init(). * re.c (rb_reg_to_s): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26791 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf8_mac.trans (buf_shift_char): don't see uninitialisednaruse2010-01-291-3/+3
| | | | | | value. [ruby-dev:40233] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26464 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h, transcode.c, tool/transcode-tblgen.rb: Addedduerst2009-12-101-5/+105
| | | | | | | | | | | | support for new transcoding instruction FUNsio (with Tatsuya Mizuno) * enc/trans/gb18030.trans: Significantly reduced GB18030 conversion table footprint using FUNsio and differences (with Tatsuya Mizuno) * test/ruby/test_transcode.rb: Minor name fix (from Tatsuya Mizuno) git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/gb18030-tbl.rb: Fix omission of C1 region in code tableduerst2009-12-031-0/+32
| | | | | | | | | | (from Tatsuya Mizuno) * test/ruby/test_transcode.rb: Added test for converting full range of Unicode codepoints from/to GB18030 (from Tatsuya Mizuno) git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/newline.trans (fun_so_universal_newline): generate \nakr2009-11-221-11/+15
| | | | | | | | after \r\n detection instead of just after \r. [ruby-list:45988] [ruby-core:25881] [ruby-core:26788] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/big5.c, enc/trans/big5.trans, enc/trans/big5-uao-tbl.rb,duerst2009-11-173-0/+19840
| | | | | | | | test/ruby/test-transcode.rb: Added Encoding 'Big5-UAO' and transcoding for it (from Tatsuya Mizuno) (see Bug #1784) git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/enc-unicode.rb,naruse2009-10-144-15656/+8200
| | | | | | | | | enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src: use UTS#18 for POSIX character class. http://rubyspec.org/issues/show/161 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25338 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Add derived core and binary property and aliases.naruse2009-10-134-1254/+36490
| | | | | | | | | | | | * tool/enc-unicode.rb, enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src: Add DerivedCoreProperties, PropList (Binary Property), PropertyAlias and PropertyValueAlias. Now users of tool/enc-unicode.rb should specify the directory of UCD files. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25324 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/unicode/name2ctype.h: update.nobu2009-10-101-3/+5
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25275 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/enc-unicode.rb: optimized.naruse2009-10-083-9/+15
| | | | | | | | * enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src: U+100000-U+10FFFD is assigned, not Cn. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/enc-unicode.rb: parse range notation of UnicodeData.txt.naruse2009-10-084-524/+248
| | | | | | | | * enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src: follow above change. [ruby-dev:39444] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25260 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/unicode/name2ctype.h: Updated to Unicode 5.2.0.naruse2009-10-021-821/+1574
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25195 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Updated to Unicode 5.2.0.naruse2009-10-025-22660/+3752
| | | | | | | | | | | | | | | | * enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src: Updated to Unicode 5.2.0. NOTE: when you update these data, download UnicodeData.txt and Scripts.txt from http://www.unicode.org/Public/UNIDATA/ and run ruby1.9 tool/enc-unicode.rb UnicodeData.txt Scripts.txt \ > enc/unicode/name2ctype.kwd * enc/unicode/Scripts.txt: removed. * enc/unicode/UnicodeData.txt: removed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * unicode.c (onigenc_unicode_property_name_to_ctype):naruse2009-09-105-1214/+1286
| | | | | | | | | | | | ignore case of properties. * tool/enc-unicode.rb: downcase properties list. * enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src: follow above. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24836 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/st.h (st_hash_func): use st_index_t.nobu2009-09-081-4/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24792 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * unicode.c (PROPERTY_NAME_MAX_SIZE): use MAX_WORD_LENGTH.naruse2009-08-261-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24677 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/unicode.c (onigenc_unicode_mbc_case_fold): balanced braces.nobu2009-08-261-2/+3
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24658 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/unicode/name2ctype.h: updated.nobu2009-08-251-365/+9744
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24657 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Update Oniguruma's UnicodeData to 5.1.naruse2009-08-256-9062/+49628
| | | | | | | | | | | | | | | | | | | | * tool/enc-unicode.rb: added for generate name2ctype.kwd. contributed by Run Paint Run Run [ruby-core:24775] use like following: ruby19 tool/enc-unicode.rb enc/unicode/UnicodeData.txt \ enc/unicode/Scripts.txt > enc/unicode/name2ctype.kwd * enc/unicode.c (CodeRanges): move definitions to name2ctype.h. * enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src: updated to v5.1. * enc/unicode/UnicodeData.txt, enc/unicode/Scripts.txt: added v5.1. * Makefile.in: add rule to generate name2ctype.kwd from UnicodeData.txt and Scripts.txt. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/unicode/name2ctype.h: split from enc/unicode.c and made anobu2009-08-215-150/+1448
| | | | | | | perfect hash. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24613 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/utf_8.c (code_to_mbc): suppressed a warning.nobu2009-08-211-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24607 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/unicode.c (CodeRanges): initialized statically.nobu2009-08-191-163/+133
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/Makefile.in (MKDIRS): revert r24525.naruse2009-08-141-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * configure.in, Makefile.in (MAKEDIRS): used MKDIR_P instead ofnobu2009-08-131-1/+1
| | | | | | | as_mkdir_p. [ruby-dev:39063] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24525 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/encdb.c (ENC_SET_BASE): fix typo. patch by ujihisa [ruby-dev:39004]naruse2009-08-041-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* More strict for Big5 series.naruse2009-08-041-5/+74
| | | | | | | | | | | | | | * enc/big5.c (EncLen_Big5): back to original Big5 table. (EncLen_Big5_HKSCS): for Big5-HKSCS. (trans): add the lead byte table for Big5-HKSCS. (big5_mbc_enc_len): abstract function for Big5 series. (big5_mbc_enc_len): for Big5. (big5_hkscs_mbc_enc_len): for Big5-HKSCS. (BIG5_HKSCS_P): added. (BIG5_ISMB_FIRST): add routine for Big5-HKSCS. (big5_hkscs): add for Big5-HKSCS. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Add functions and macros for second encoding definitions.naruse2009-08-041-0/+2
| | | | | | | | | | | | | * encoding.c (rb_enc_set_base): Add for setting base encoding with their names. this is internal function. * template/encdb.h.tmpl: specify ENC_SET_BASE for second encodings in each encoding files. * enc/encdb.c (rb_enc_set_base): add a declaration. (ENC_SET_BASE): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/big5.c: not executable.nobu2009-07-251-0/+0
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24269 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/big5.c: Fix EncLen_BIG5 for Big5-HKSCS. see [ruby-core:24390]naruse2009-07-241-18/+34
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24267 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/big5.trans, big5-hkscs-tbl.rb:duerst2009-07-243-0/+18390
| | | | | | | | | | | | | | | | new Chinese BIG5-HKSCS transcoding (with Tatsuya Mizuno) * test/ruby/test_transcode.rb: added tests for the above (with Tatsuya Mizuno) * enc/big5.c: Added BIG5-HKSCS as a replicate encoding of BIG5 (short term solution, needs more work; with Tatsuya Mizuno) * tool/transcode-tblgen.rb: made 'pat' directly accessible in class StrSet git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * ruby.c (process_options), enc/prelude.rb: encdb and transdb arenobu2009-06-221-1/+1
| | | | | | | extension libraries. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23813 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf8_mac.trans: remove wrong optimization.naruse2009-06-131-9/+0
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23686 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Fix: DON'T move in_p because before in_p is replaced by buffered data.naruse2009-05-051-7/+7
| | | | | | | | | | | | | | | | * transcode.c: NOMAP is now multibyte direct map. * transcode.c: remove ASIS. * transcode_data.h: ditto. * tool/transcode-tb (ActionMap#generate_info): remove :asis. * tool/transcode-tb (ActionMap#generate_info): add :nomap0. * enc/trans/utf8_mac.trans: replace :asis by :nomap0. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf8_mac-tbl.rb: don't use Unicode escape.naruse2009-05-022-955/+949
| | | | | | * enc/trans/utf8_mac.trans: follow above. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23325 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf8_mac.trans: get rid of a 1.9 feature for crossnobu2009-04-301-2/+2
| | | | | | | compile. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Add new transcoder: CP51932 <-> CP50221.naruse2009-04-292-0/+201
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23307 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf8_mac.trans: Add converter for UTF8-MAC.naruse2009-04-262-0/+1202
| | | | | | | | * enc/trans/utf8_mac-tbl.rb: ditto. * test/ruby/test_econv.rb: tests for above. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23296 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend (link_so): replaces $(TARGET) with basename of thenobu2009-03-221-2/+9
| | | | | | | target. [ruby-talk:330286] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-07 09:41:08 +0000