aboutsummaryrefslogtreecommitdiffstats
path: root/enc/trans
Commit message (Collapse)AuthorAgeFilesLines
* * enc/trans/iso2022.trans: renamed from iso2022.erb.c.akr2008-08-115-0/+0
| | | | | | | | | | | | | | | | | | | * enc/trans/single_byte.trans: ditto. * enc/trans/utf_16_32.trans: ditto. * enc/trans/korean.trans: ditto. * enc/trans/japanese.trans: ditto. * enc/depend: follow the renaming. * tool/build-transcode: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoding): add fields for restartable akr2008-08-092-10/+40
| | | | | | | | | | | | | | | | | | | | | | transcoding. (rb_transcoder): add max_input field. from_unit_length field is renamed to input_unit_length. * tool/transcode-tblgen.rb: generate max_input field. * enc/trans/iso2022.erb.c: follow rb_transcoder change. * enc/trans/utf_16_32.erb.c: ditto. * transcode.c (PARTIAL_INPUT): new constant. (transcode_char_start): new function. (transcode_result_t): new type. (transcode_restartable): new function. (more_output_buffer): new function. (transcode_loop): use transcode_restartable. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/make_encdb.rb, enc/trans/make_transdb.rb: skip nonexistentnobu2008-08-081-0/+1
| | | | | | | directory. [ruby-dev:35802] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18449 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf_16_32.erb.c (fun_so_from_utf_32le): implemented.akr2008-08-081-1/+44
| | | | | | | | (fun_so_to_utf_32le): implemented. [ruby-dev:35777] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoder): from_unit_length field added.akr2008-08-082-33/+18
| | | | | | | | | | | | | | | | | | | | from_utf8 field removed. * tool/transcode-tblgen.rb: generate offsets range. follow rb_transcoder change. * transcode.c (transcode_loop): don't use from_utf8. make invalid region from_unit_length wise. * enc/trans/iso2022.erb.c: follow rb_transcoder and transcode_generate_node change. * enc/trans/utf_16_32.erb.c: follow rb_transcoder and transcode_generate_node change. explicit :invalid map removed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18445 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (encdb.h): see both $(srcdir)/enc and enc.nobu2008-08-081-2/+2
| | | | | | | * enc/make_encdb.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18435 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/make_transdb.rb: fix for the case no transdirs are given.nobu2008-08-081-1/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/make_transdb.rb: converts only one transcoders for eachnobu2008-08-081-0/+3
| | | | | | | basename. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18433 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk: see both $(srcdir)/enc/trans and enc/trans.naruse2008-08-071-16/+18
| | | | | | * enc/trans/make_transdb.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18422 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoding): new field "stateful".akr2008-08-073-243/+158
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (rb_transcoder): preprocessor and postprocessor field removed. change arguments of func_ii, func_si, func_io and func_so. new field "finish_func". * tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio generatable. * transcode.c (transcoder_lib_table): removed. (transcoder_table): change structure. (transcoder_key): removed because the above structure change. (make_transcoder_entry): new function. (get_transcoder_entry): ditto. (rb_register_transcoder): follow the structure change. (declare_transcoder): ditto. (transcode_search_path): new function for breadth first search to find a list of converters. (transcode_search_path_i): new function. (transcode_dispatch_cb): ditto. (transcode_dispatch): use transcode_search_path. (transcode_loop): follow the argument change. (str_transcode): preprocessor and postprocessor stuff removed. * enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion re-implemented. * enc/trans/japanese.erb.c: ISO-2022-JP stuff removed. nute(23:52:53)% head -40 ChangeLog Thu Aug 7 23:43:11 2008 Tanaka Akira <akr@fsij.org> * transcode_data.h (rb_transcoding): new field "stateful". (rb_transcoder): preprocessor and postprocessor field removed. change arguments of func_ii, func_si, func_io and func_so. new field "finish_func". * tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio generatable. * transcode.c (transcoder_lib_table): removed. (transcoder_table): change structure. (transcoder_key): removed because the above structure change. (make_transcoder_entry): new function. (get_transcoder_entry): ditto. (rb_register_transcoder): follow the structure change. (declare_transcoder): ditto. (transcode_search_path): new function for breadth first search to find a list of converters. (transcode_search_path_i): new function. (transcode_dispatch_cb): ditto. (transcode_dispatch): use transcode_search_path. (transcode_loop): follow the argument change. (str_transcode): preprocessor and postprocessor stuff removed. * enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion re-implemented. * enc/trans/japanese.erb.c: ISO-2022-JP stuff removed. * enc/trans/utf_16_32.erb.c: follow argument change of FUNso. [ruby-dev:35798] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18419 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb (transcode_generate_node): codeakr2008-08-061-15/+5
| | | | | | | argument removed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18395 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* useless comment removed.akr2008-08-061-1/+0
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (transdb.h): requires transcoders.nobu2008-08-061-1/+0
| | | | | | | * enc/depend (srcs): target for transcoders. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18387 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb: show generating tables in verbose mode.akr2008-08-051-5/+5
| | | | | | | | | | (transcode_generate_node): call ActionMap#generate_node with showing table name. * enc/trans/utf_16_32.erb.c: use transcode_generate_node. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (transcodes), tool/build-transcode: generates transcodenobu2008-08-054-55102/+0
| | | | | | | | | | sources. * enc/trans/{japanese,korean,single_byte,utf_16_32}.c: to be autogenerated now. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18376 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/build-transcode: new file.akr2008-08-0526-34064/+63798
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tool/transcode-tblgen.rb: new file. * enc/trans/make_transdb.rb: exclude *.erb.c. * enc/depend: exclude *.erb.c. * enc/trans/utf_16_32.erb.c: new file. * enc/trans/single_byte.erb.c: new file. * enc/trans/japanese.erb.c: new file. * enc/trans/korean.erb.c: new file. * enc/trans/iso-8859-2-tbl.rb: new file. * enc/trans/iso-8859-3-tbl.rb: new file. * enc/trans/iso-8859-4-tbl.rb: new file. * enc/trans/iso-8859-5-tbl.rb: new file. * enc/trans/iso-8859-6-tbl.rb: new file. * enc/trans/iso-8859-7-tbl.rb: new file. * enc/trans/iso-8859-8-tbl.rb: new file. * enc/trans/iso-8859-9-tbl.rb: new file. * enc/trans/iso-8859-10-tbl.rb: new file. * enc/trans/iso-8859-11-tbl.rb: new file. * enc/trans/iso-8859-13-tbl.rb: new file. * enc/trans/iso-8859-14-tbl.rb: new file. * enc/trans/iso-8859-15-tbl.rb: new file. * enc/trans/eucjp-tbl.rb: new file. * enc/trans/sjis-tbl.rb: new file. * enc/trans/euckr-tbl.rb: new file. * enc/trans/utf_16_32.c: regenerated. * enc/trans/single_byte.c: regenerated. * enc/trans/japanese.c: regenerated. * enc/trans/korean.c: regenerated. [ruby-dev:35730] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18373 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (TRANSCODE_ERROR): common transcode failurenobu2008-08-051-43/+47
| | | | | | | | | | exception, would be changed later. * enc/trans/japanese.c (UNSUPPORTED_MODE): unsupported mode transition exception. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c: add U+FF5E to EUC-JP.naruse2008-08-011-2/+2
| | | | | | [ruby-dev:35720] [ruby-dev:35722] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18315 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c: add support for CP51932,naruse2008-08-011-0/+12
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c: add U+FF0C,naruse2008-08-011-5/+5
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c (to_SHIFT_JIS_EF_BF_offsets): add U+FFF3,naruse2008-08-011-7/+19
| | | | | | | | | | | | | | | U+FFF4, U+FFF5. * enc/trans/japanese.c (to_SHIFT_JIS_EF_BF_infos): ditto. * enc/trans/japanese.c (to_EUC_JP_EF_BF_infos): added. * enc/trans/japanese.c (to_EUC_JP_EF_BF): added. * enc/trans/japanese.c (to_EUC_JP_EF_infos): change size. [ruby-dev:35714] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18305 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (transcode_loop): constified.nobu2008-07-141-8/+10
| | | | | | | | | | | | * transcode.c (str_transcode): rb_str_set_len() sets a delimiter. * transcode_data.h (rb_transcoder): constified preprocessor and postprocessor input. * enc/trans/japanese.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (prelude.c): simply depends on PREP. [ruby-dev:34877]nobu2008-05-301-8/+2
| | | | | | | * enc/make_encdb.rb, enc/trans/make_transdb.rb: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16703 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c (to_SHIFT_JIS_EF_infos): typo.usa2008-05-281-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16661 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c: add workarround for Unicode to CP932.naruse2008-05-281-15/+36
| | | | | | | U+2015->0x815C, U+2225->0x8161, U+FF0D->0x817C, U+FF3C->0x815F, U+FF5E->0x8160, U+FFE0->0x8191, U+FFE1->0x8192, U+FFE2->0x81CA git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16657 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/utf_16_32.c (fun_so_to_utf_16be, fun_so_to_utf_16le): addmame2008-04-171-3/+3
| | | | | | | | | | | | | | | | | parentheses to remove warnings of gcc. * io.c (rb_io_getc): remove unused variables. * compile.c (NODE_NEXT, NODE_REDO): remove unused labels. * ext/nkf/nkf.c (rb_nkf_convert): remove unused variables. * ext/syck/rubyext.c (syck_resolver_initialize, syck_resolver_detect_implicit, syck_emitter_emit): remove unused variables. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@16061 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (enc_init_db): moved to enc/encdb.c.nobu2008-04-071-0/+18
| | | | | | | | | | | | * transcode.c (init_transcoder_table): moved to enc/trans/transdb.c. * enc/depend (enc/encdb.o enc/trans/transdb.o): depend on corresponding headers. * common.mk (COMMONOBJS): moved transcode.o from OBJS git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Sun Mar 16 18:07:07 2008 Martin Duerst <duerst@it.aoyama.ac.jp>duerst2008-03-161-15/+15
| | | | | | | | | | | * enc/trans/utf_16_32.c: bug fix (some invalid UTF-8 sequences were legal) * test/ruby/test_transcode.rb: test for above bug git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/{depend,make_encdb.rb,trans/make_transdb.rb}: sort in alpha-numeric order.nobu2008-02-221-1/+3
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15567 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/korean.c: add support for CP949 by Park Ji-In. [ruby-dev:33626]naruse2008-02-071-6850/+34924
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15393 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/korean.c: add EUC-KR conversion support by Park Ji-In.naruse2008-02-061-0/+17938
| | | | | | [ruby-dev:33621] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15385 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * add enc/trans/make_transdb.rb, dmytranscode.cnaruse2008-01-291-0/+40
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15320 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c (rb_to_Windows_31J): to 'Windows-31J'.naruse2008-01-291-1/+1
| | | | | | | | | * common.mk: add rules for transdb.h. * transcode.c (init_transcoder_table): use transdb.h. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15317 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c (rb_from_Windows_31J, rb_to_Windows_31J):nobu2008-01-231-0/+12
| | | | | | | provisional workaround for Windows-31J. [ruby-dev:33320] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15188 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Tue Jan 22 17:52:52 2008 Martin Duerst <duerst@it.aoyama.ac.jp>duerst2008-01-221-24/+24
| | | | | | | | | * enc/trans/utf_16_32.c: Streamline parentheses, add more 'static' qualifiers. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15169 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Mon Jan 21 19:42:42 2008 Martin Duerst <duerst@it.aoyama.ac.jp>duerst2008-01-211-60/+384
| | | | | | | | | * transcode.c, enc/trans/utf_16_32.c, test/ruby/test_transcode.rb: added UTF-32BE and UTF-32LE conversions. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15156 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (transcode_loop, str_transcoding_resize): use unsignednobu2008-01-212-28/+28
| | | | | | | | | | | | | | char. [ruby-dev:33232] * transcode_data.h (rb_transcoding, rb_transcoder): removed callback parameters. * enc/trans/japanese.c: ditto. * enc/trans/utf_16_32.c: parenthesized bit-or operands. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15150 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (transcode_dispatch): constified return value.nobu2008-01-202-40/+40
| | | | | | | | | | | | | * transcode_data.h (rb_transcoding): include pointer to rb_transcoder and auxiliary data. * transcode_data.h (rb_transcoder): all callback functions shoud have their own parameters. * enc/trans/{japanese,single_byte}.c: constified. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15148 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Sun Jan 20 20:00:20 2008 Martin Duerst <duerst@it.aoyama.ac.jp>duerst2008-01-201-1/+141
| | | | | | | | | | | * transcode.c, enc/trans/utf_16_32.c, test/ruby/test_transcode.rb: added UTF-16LE conversions. * fixed changelog for last commit git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15144 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Sun Jan 20 15:08:08 2008 Martin Duerst <duerst@it.aoyama.ac.jp>duerst2008-01-201-0/+332
| | | | | | | | | | | | | | | * enc/trans/utf_16_32.c: new file, currently implementing UTF-16BE conversions only. * test/ruby/test_transcode.rb: Added tests for UTF-16BE; made check_both_ways() use force_encoding differently. * transcode_data.h, transcode.c: Support for more conversion functions. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@15142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Fri Dec 28 01:55:04 2007 Martin Duerst <duerst@it.aoyama.ac.jp>duerst2007-12-281-14/+103
| | | | | | | | | | | | | | | | | | | | | * transcode.c (transcode_dispatch): reverted some of the changes in r14746. * transcode.c, enc/trans/single_byte.c: Added conversions to/from US-ASCII and ASCII-8BIT (using data tables). * enc/trans/single_byte.c: Some spacing/ordering changes due to automatic data file generation. * transcode_data.h, transcode.c: Preliminary code for using micro-conversion functions. * test/ruby/test_transcode.rb: Added some tests for US-ASCII and ASCII-8BIT conversions. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14766 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * lib/weakref.rb, lib/irb/ruby-lex.rb, lib/irb/lc/error.rb, ↵akr2007-12-271-1/+1
| | | | | | | | | | enc/trans/japanese.c: change "illegal" to "invalid" in a context which doesn' t against a law. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c (rb_{from,to}_{SHIFT_JIS,EUC_JP}): inversednobu2007-12-251-4/+4
| | | | | | | from_encoding and to_encoding. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14684 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/japanese.c (rb_to_EUC_JP): fixed typo.nobu2007-12-251-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14682 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * enc/trans/single_byte.c (Init_single_byte): renamed.usa2007-12-251-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14668 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * common.mk (COMMONOBJS): transcode_data_*.c moved under enc/trans.nobu2007-12-252-0/+26888
* transcode_data.h (rb_transcoding, rb_transcoder): prefixed. * transcode.c (rb_register_transcoder, rb_declare_transcoder): split declaration and registration. [ruby-dev:32704] * transcode.c (transcode_dispatch): autoload pre-declared transcoder. * transcode.c (str_transcode): use rb_define_dummy_encoding(). * transcode.c (Init_transcode): initialize transcoder tables. * enc/trans/single_byte.c, enc/trans/japanese.c: moved from top. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@14666 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-15 10:48:53 +0000