aboutsummaryrefslogtreecommitdiffstats
path: root/enc
Commit message (Collapse)AuthorAgeFilesLines
* fix variable name.akr2008-08-131-4/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18558 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/newline.trans (rb_crlf_newline): new transcoder.akr2008-08-131-0/+35
| | | | | | | | | | | | | | (rb_cr_newline): new transcoder. * transcode.c (trans_open_i): one more exra room for input newline converter. (rb_trans_open): crlf newline and cr newline implemented. (Init_transcode): Encoding::Converter::CRLF_NEWLINE and Encoding::Converter::LF_NEWLINE defined. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18557 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/newline.trans: new file.akr2008-08-131-0/+56
| | | | | | | | | | | | | | | | | | | | * transcode_data.h (rb_trans_t): add last_tc field. * transcode.c (UNIVERSAL_NEWLINE): defined. (CRLF_NEWLINE): defined. (CR_NEWLINE): defined. (rb_trans_open_by_transcoder_entries): initialize last_tc. (trans_open_i): allocate one more room for newline converter. (rb_trans_open): universal newline implemented. (more_output_buffer): take max_output argument instead ts. (output_replacement_character): take tc argument instead of ts. (transcode_loop): use last_tc field. (econv_init): add flags argument for rb_trans_open. (Init_transcode): Encoding::Converter::UNIVERSAL_NEWLINE defined. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/make_transdb.rb: *.erb.c is not used anymore.akr2008-08-121-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18532 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: (transvpath_prefix): prefix has no extension, so replaceusa2008-08-121-1/+1
| | | | | | | | %s with "". git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18517 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/Makefile.in (.SUFFIXES): renamed to .trans.nobu2008-08-123-15/+44
| | | | | | | | | * enc/make_encmake.rb: added --encs and --no-encs options. * enc/depend (TRANSVPATH): fix for nmake. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18513 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoder): add resetstate_func field forakr2008-08-111-1/+2
| | | | | | | | | | | | | | | | resetting a state of stateful encoding. * enc/trans/iso2022.trans (rb_EUC_JP_to_ISO_2022_JP): specify finish_eucjp_to_iso2022jp for resetstate_func. * tool/transcode-tblgen.rb: specify NULL for resetstate_func. * transcode.c (output_replacement_character): call resetstate_func before appending the replacement character. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18503 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/iso2022.trans: renamed from iso2022.erb.c.akr2008-08-116-3/+5
| | | | | | | | | | | | | | | | | | | * enc/trans/single_byte.trans: ditto. * enc/trans/utf_16_32.trans: ditto. * enc/trans/korean.trans: ditto. * enc/trans/japanese.trans: ditto. * enc/depend: follow the renaming. * tool/build-transcode: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/Makefile.in (make-workdir): use MAKEDIRS.nobu2008-08-102-3/+6
| | | | | | | | | * enc/depend: makes target directory before compile/link. * tool/transcode-tblgen.rb: creates target directory. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18469 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoding): add fields for restartable akr2008-08-092-10/+40
| | | | | | | | | | | | | | | | | | | | | | transcoding. (rb_transcoder): add max_input field. from_unit_length field is renamed to input_unit_length. * tool/transcode-tblgen.rb: generate max_input field. * enc/trans/iso2022.erb.c: follow rb_transcoder change. * enc/trans/utf_16_32.erb.c: ditto. * transcode.c (PARTIAL_INPUT): new constant. (transcode_char_start): new function. (transcode_result_t): new type. (transcode_restartable): new function. (more_output_buffer): new function. (transcode_loop): use transcode_restartable. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/make_encdb.rb, enc/trans/make_transdb.rb: skip nonexistentnobu2008-08-082-0/+2
| | | | | | | directory. [ruby-dev:35802] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18449 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf_16_32.erb.c (fun_so_from_utf_32le): implemented.akr2008-08-081-1/+44
| | | | | | | | (fun_so_to_utf_32le): implemented. [ruby-dev:35777] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoder): from_unit_length field added.akr2008-08-082-33/+18
| | | | | | | | | | | | | | | | | | | | from_utf8 field removed. * tool/transcode-tblgen.rb: generate offsets range. follow rb_transcoder change. * transcode.c (transcode_loop): don't use from_utf8. make invalid region from_unit_length wise. * enc/trans/iso2022.erb.c: follow rb_transcoder and transcode_generate_node change. * enc/trans/utf_16_32.erb.c: follow rb_transcoder and transcode_generate_node change. explicit :invalid map removed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18445 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend (TRANSCSRCS): needs rule_subst to apply.nobu2008-08-081-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18444 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (srcs-enc): renamed from transcodes.nobu2008-08-081-1/+6
| | | | | | | * enc/Makefile.in (make-workdir): creates object directories. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18437 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (encdb.h): see both $(srcdir)/enc and enc.nobu2008-08-082-36/+42
| | | | | | | * enc/make_encdb.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18435 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/make_transdb.rb: fix for the case no transdirs are given.nobu2008-08-081-1/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/make_transdb.rb: converts only one transcoders for eachnobu2008-08-081-0/+3
| | | | | | | basename. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18433 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk: see both $(srcdir)/enc/trans and enc/trans.naruse2008-08-071-16/+18
| | | | | | * enc/trans/make_transdb.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18422 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoding): new field "stateful".akr2008-08-073-243/+158
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (rb_transcoder): preprocessor and postprocessor field removed. change arguments of func_ii, func_si, func_io and func_so. new field "finish_func". * tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio generatable. * transcode.c (transcoder_lib_table): removed. (transcoder_table): change structure. (transcoder_key): removed because the above structure change. (make_transcoder_entry): new function. (get_transcoder_entry): ditto. (rb_register_transcoder): follow the structure change. (declare_transcoder): ditto. (transcode_search_path): new function for breadth first search to find a list of converters. (transcode_search_path_i): new function. (transcode_dispatch_cb): ditto. (transcode_dispatch): use transcode_search_path. (transcode_loop): follow the argument change. (str_transcode): preprocessor and postprocessor stuff removed. * enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion re-implemented. * enc/trans/japanese.erb.c: ISO-2022-JP stuff removed. nute(23:52:53)% head -40 ChangeLog Thu Aug 7 23:43:11 2008 Tanaka Akira <akr@fsij.org> * transcode_data.h (rb_transcoding): new field "stateful". (rb_transcoder): preprocessor and postprocessor field removed. change arguments of func_ii, func_si, func_io and func_so. new field "finish_func". * tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio generatable. * transcode.c (transcoder_lib_table): removed. (transcoder_table): change structure. (transcoder_key): removed because the above structure change. (make_transcoder_entry): new function. (get_transcoder_entry): ditto. (rb_register_transcoder): follow the structure change. (declare_transcoder): ditto. (transcode_search_path): new function for breadth first search to find a list of converters. (transcode_search_path_i): new function. (transcode_dispatch_cb): ditto. (transcode_dispatch): use transcode_search_path. (transcode_loop): follow the argument change. (str_transcode): preprocessor and postprocessor stuff removed. * enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion re-implemented. * enc/trans/japanese.erb.c: ISO-2022-JP stuff removed. * enc/trans/utf_16_32.erb.c: follow argument change of FUNso. [ruby-dev:35798] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18419 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: add transdb.c.nobu2008-08-072-2/+3
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18412 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: removed needless explicit commands.nobu2008-08-071-15/+35
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18410 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: enc/*.c is source but enc/trans/*.c is generated.naruse2008-08-061-2/+3
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18400 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: for build in other than srcdir.naruse2008-08-061-4/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb (transcode_generate_node): codeakr2008-08-061-15/+5
| | | | | | | argument removed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18395 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: transcode table generation depends onakr2008-08-061-1/+1
| | | | | | | tool/transcode-tblgen.rb. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18393 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* useless comment removed.akr2008-08-061-1/+0
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (transdb.h): requires transcoders.nobu2008-08-062-4/+6
| | | | | | | * enc/depend (srcs): target for transcoders. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18387 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: replace not only $(<:...) but also $<.usa2008-08-061-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * win32/Makefile.sub (config.status): export BASERUBY.usa2008-08-061-1/+1
| | | | | | | | * enc/depend: avoid GNU make'ism. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb: show generating tables in verbose mode.akr2008-08-051-5/+5
| | | | | | | | | | (transcode_generate_node): call ActionMap#generate_node with showing table name. * enc/trans/utf_16_32.erb.c: use transcode_generate_node. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (transcodes), tool/build-transcode: generates transcodenobu2008-08-054-55102/+0
| | | | | | | | | | sources. * enc/trans/{japanese,korean,single_byte,utf_16_32}.c: to be autogenerated now. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend: added rules for .c from .erb.c.nobu2008-08-052-1/+21
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18374 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/build-transcode: new file.akr2008-08-0527-34065/+63799
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tool/transcode-tblgen.rb: new file. * enc/trans/make_transdb.rb: exclude *.erb.c. * enc/depend: exclude *.erb.c. * enc/trans/utf_16_32.erb.c: new file. * enc/trans/single_byte.erb.c: new file. * enc/trans/japanese.erb.c: new file. * enc/trans/korean.erb.c: new file. * enc/trans/iso-8859-2-tbl.rb: new file. * enc/trans/iso-8859-3-tbl.rb: new file. * enc/trans/iso-8859-4-tbl.rb: new file. * enc/trans/iso-8859-5-tbl.rb: new file. * enc/trans/iso-8859-6-tbl.rb: new file. * enc/trans/iso-8859-7-tbl.rb: new file. * enc/trans/iso-8859-8-tbl.rb: new file. * enc/trans/iso-8859-9-tbl.rb: new file. * enc/trans/iso-8859-10-tbl.rb: new file. * enc/trans/iso-8859-11-tbl.rb: new file. * enc/trans/iso-8859-13-tbl.rb: new file. * enc/trans/iso-8859-14-tbl.rb: new file. * enc/trans/iso-8859-15-tbl.rb: new file. * enc/trans/eucjp-tbl.rb: new file. * enc/trans/sjis-tbl.rb: new file. * enc/trans/euckr-tbl.rb: new file. * enc/trans/utf_16_32.c: regenerated. * enc/trans/single_byte.c: regenerated. * enc/trans/japanese.c: regenerated. * enc/trans/korean.c: regenerated. [ruby-dev:35730] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18373 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (TRANSCODE_ERROR): common transcode failurenobu2008-08-051-43/+47
| | | | | | | | | | exception, would be changed later. * enc/trans/japanese.c (UNSUPPORTED_MODE): unsupported mode transition exception. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c: add U+FF5E to EUC-JP.naruse2008-08-011-2/+2
| | | | | | [ruby-dev:35720] [ruby-dev:35722] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18315 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c: add support for CP51932,naruse2008-08-011-0/+12
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c: add U+FF0C,naruse2008-08-011-5/+5
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c (to_SHIFT_JIS_EF_BF_offsets): add U+FFF3,naruse2008-08-011-7/+19
| | | | | | | | | | | | | | | U+FFF4, U+FFF5. * enc/trans/japanese.c (to_SHIFT_JIS_EF_BF_infos): ditto. * enc/trans/japanese.c (to_EUC_JP_EF_BF_infos): added. * enc/trans/japanese.c (to_EUC_JP_EF_BF): added. * enc/trans/japanese.c (to_EUC_JP_EF_infos): change size. [ruby-dev:35714] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18305 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (transcode_loop): constified.nobu2008-07-141-8/+10
| | | | | | | | | | | | * transcode.c (str_transcode): rb_str_set_len() sets a delimiter. * transcode_data.h (rb_transcoder): constified preprocessor and postprocessor input. * enc/trans/japanese.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/shift_jis.c (code_is_ctype): HALF WIDTH KATAKANA isnaruse2008-07-011-1/+1
| | | | | | a character. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17774 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* forgot to commitshyouhei2008-07-0128-55/+55
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/make_encdb.h: always add ';' at the end of line.usa2008-07-011-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17771 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/ascii.c: ISO C does not allow extra ';' outside of ashyouhei2008-07-013-50/+50
| | | | | | | | | | | | function * enc/us_ascii.c: ditto. * enc/utf_8.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17768 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/euc_jp.c (property_name_to_ctype): core dumped when sizeof(int)mame2008-06-173-6/+7
| | | | | | | | | | | differs from sizeof(long). * enc/shift_jis.c (property_name_to_ctype): ditto. * enc/unicode.c (onigenc_unicode_property_name_to_ctype): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17381 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/depend (clean): remove build directories.nobu2008-06-091-0/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17060 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * lib/mkmf.rb (configuration): set flags.nobu2008-06-091-0/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@17058 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/iso_8859_5.c: Large omicron should lowercase to small omicron.mame2008-06-021-1/+1
| | | | | | | | | | | | | | | * test/ruby/test_big5.rb, test/ruby/test_cp949.rb, test/ruby/test_euc_jp.rb, test/ruby/test_euc_kr.rb, test/ruby/test_euc_tw.rb, test/ruby/test_gb18030.rb, test/ruby/test_gbk.rb, test/ruby/test_iso_8859.rb, test/ruby/test_koi8.rb, test/ruby/test_shift_jis.rb, test/ruby/test_windows_1251.rb: new tests for encoding. * test/ruby/test_utf16.rb, test/ruby/test_utf32.rb, test/ruby/test_regexp.rb: add tests. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16759 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/gb18030.c (gb18030_code_to_mbc): add 0x80000000naruse2008-06-011-0/+3
| | | | | | for 4bytes character. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16739 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/gb18030.c (gb18030_mbc_to_code): mask by 0x7FFFFFFFnaruse2008-06-011-1/+14
| | | | | | | | because OnigCodePoint will be used as 32bit signed int. Masking by 0x7FFFFFFF is ok on GB18030; Minumum 4bytes character is 0x81308130. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16737 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-30 10:20:15 +0000