| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
valid_encoding mandatory unless from_encoding is registered in
ValidEncoding.
(transcode_tbl_only): ditto.
(transcode_tblgen): ditto.
(ValidEncoding): new function.
* enc/trans/escape.trans: specify valid_encoding.
* enc/trans/emoji_sjis_docomo.trans: ditto.
* enc/trans/emoji.trans: ditto.
* enc/trans/emoji_iso2022_kddi.trans: ditto.
* enc/trans/big5.trans: ditto.
* enc/trans/emoji_sjis_softbank.trans: ditto.
* enc/trans/emoji_sjis_kddi.trans: ditto.
* enc/trans/chinese.trans: use ValidEncoding() instead of
ValidEncoding[].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26995 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
optional argument.
* enc/trans/single_byte.trans use valid_encoding argument for
transcode_tblgen.
* enc/trans/chinese.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26941 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* enc/trans/single_byte.trans: remove ambiguous maping such as
\xD6 -> U+05F2 and \xD6\xC7 -> U+FB1F in Windows-1255
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26863 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
test/ruby/enc/test_emoji.rb, tool/enc-emoji-citrus-gen.rb, tool/enc-emoji4unicode.rb, tool/jisx0208.rb, tool/test/test_jisx0208.rb: new encodings to support emoji charsets, which are used by Japanese mobile phones [ruby-dev:40528]. Thanks Yoji Shidara for a lot of contribution.
* tool/transcode-tblgen.rb: modified for enc-emoji4unicode.rb.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* re.c (make_regexp): use onig_new() instead of onig_alloc_init().
* re.c (rb_reg_to_s): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26791 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
value. [ruby-dev:40233]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26464 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
| |
support for new transcoding instruction FUNsio (with Tatsuya Mizuno)
* enc/trans/gb18030.trans: Significantly reduced GB18030 conversion
table footprint using FUNsio and differences (with Tatsuya Mizuno)
* test/ruby/test_transcode.rb: Minor name fix (from Tatsuya Mizuno)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
(from Tatsuya Mizuno)
* test/ruby/test_transcode.rb: Added test for converting full range of
Unicode codepoints from/to GB18030 (from Tatsuya Mizuno)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
after \r\n detection instead of just after \r.
[ruby-list:45988] [ruby-core:25881] [ruby-core:26788]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
test/ruby/test-transcode.rb: Added Encoding 'Big5-UAO' and transcoding
for it (from Tatsuya Mizuno) (see Bug #1784)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25822 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
use UTS#18 for POSIX character class.
http://rubyspec.org/issues/show/161
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25338 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
| |
* tool/enc-unicode.rb,
enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
Add DerivedCoreProperties, PropList (Binary Property),
PropertyAlias and PropertyValueAlias.
Now users of tool/enc-unicode.rb should specify
the directory of UCD files.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25324 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25275 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
U+100000-U+10FFFD is assigned, not Cn.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
follow above change. [ruby-dev:39444]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25260 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25195 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd,
enc/unicode/name2ctype.src: Updated to Unicode 5.2.0.
NOTE: when you update these data, download UnicodeData.txt
and Scripts.txt from http://www.unicode.org/Public/UNIDATA/
and run
ruby1.9 tool/enc-unicode.rb UnicodeData.txt Scripts.txt \
> enc/unicode/name2ctype.kwd
* enc/unicode/Scripts.txt: removed.
* enc/unicode/UnicodeData.txt: removed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
| |
ignore case of properties.
* tool/enc-unicode.rb: downcase properties list.
* enc/unicode/name2ctype.h, enc/unicode/name2ctype.h.blt,
enc/unicode/name2ctype.kwd, enc/unicode/name2ctype.src:
follow above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24836 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24792 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24677 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24658 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24657 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* tool/enc-unicode.rb: added for generate name2ctype.kwd.
contributed by Run Paint Run Run [ruby-core:24775]
use like following:
ruby19 tool/enc-unicode.rb enc/unicode/UnicodeData.txt \
enc/unicode/Scripts.txt > enc/unicode/name2ctype.kwd
* enc/unicode.c (CodeRanges): move definitions to name2ctype.h.
* enc/unicode/name2ctype.h.blt, enc/unicode/name2ctype.kwd,
enc/unicode/name2ctype.src: updated to v5.1.
* enc/unicode/UnicodeData.txt, enc/unicode/Scripts.txt: added v5.1.
* Makefile.in: add rule to generate name2ctype.kwd from
UnicodeData.txt and Scripts.txt.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24651 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
perfect hash.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24613 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24607 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24538 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
as_mkdir_p. [ruby-dev:39063]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24525 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* enc/big5.c (EncLen_Big5): back to original Big5 table.
(EncLen_Big5_HKSCS): for Big5-HKSCS.
(trans): add the lead byte table for Big5-HKSCS.
(big5_mbc_enc_len): abstract function for Big5 series.
(big5_mbc_enc_len): for Big5.
(big5_hkscs_mbc_enc_len): for Big5-HKSCS.
(BIG5_HKSCS_P): added.
(BIG5_ISMB_FIRST): add routine for Big5-HKSCS.
(big5_hkscs): add for Big5-HKSCS.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* encoding.c (rb_enc_set_base): Add for setting base encoding
with their names. this is internal function.
* template/encdb.h.tmpl: specify ENC_SET_BASE for second
encodings in each encoding files.
* enc/encdb.c (rb_enc_set_base): add a declaration.
(ENC_SET_BASE): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24269 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24267 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
new Chinese BIG5-HKSCS transcoding (with Tatsuya Mizuno)
* test/ruby/test_transcode.rb: added tests for the above
(with Tatsuya Mizuno)
* enc/big5.c: Added BIG5-HKSCS as a replicate encoding of BIG5
(short term solution, needs more work; with Tatsuya Mizuno)
* tool/transcode-tblgen.rb: made 'pat' directly accessible in
class StrSet
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@24264 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
extension libraries.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23813 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23686 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c: NOMAP is now multibyte direct map.
* transcode.c: remove ASIS.
* transcode_data.h: ditto.
* tool/transcode-tb (ActionMap#generate_info): remove :asis.
* tool/transcode-tb (ActionMap#generate_info): add :nomap0.
* enc/trans/utf8_mac.trans: replace :asis by :nomap0.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
* enc/trans/utf8_mac.trans: follow above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23325 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
compile.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23307 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* enc/trans/utf8_mac-tbl.rb: ditto.
* test/ruby/test_econv.rb: tests for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23296 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
target. [ruby-talk:330286]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
enc.mk twice or more.
* enc/depend (cleanobjs): added deffile.
* lib/mkmf.rb (create_makefile): removes deffile at clean instead
of distclean.
* win32/Makefile.sub (miniruby, LIBRUBY_SO): removes lib and exp
files.
* win32/Makefile.sub (clean, distclean): have moved to common.mk.
* win32/rmdirs.bat: omits `not empty' message.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
compile. [ruby-core:21345]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21512 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
new Chinese GB18030 transcoding (from Yoshihiro Kambayashi)
* test/ruby/test_transcode.rb: added tests for the above
(from Yoshihiro Kambayashi)
* transcode_data.h, transcode.c, tool/transcode_tblgen.rb:
added support for GB18030-specific 4-byte sequences
(with Yoshihiro Kambayashi)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21509 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
enc/trans/make_transdb.rb using tool/generic_erb.rb.
* common.mk (encdb.h, transdb.h): generates from avobe template.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21490 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|