.TH CONVCS 2.SH NAMEConvcs,  Btos, Stob \- character set conversion suite.SH SYNOPSIS.EXinclude "convcs.m";convcs := load Convcs Convcs->PATH; Btos: module {	init: fn(arg: string): string;	btos: fn(s: Convcs->State, b: array of byte, nchars: int)			: (Convcs->State, string, int);}; Stob: module {	init: fn (arg: string): string;	stob: fn(s: Convcs->State, str: string)			: (Convcs->State, array of byte);}; Convcs: module {	State: type string;	Startstate: con ""; 	init: fn(csfile: string): string;	getbtos: fn(cs: string): (Btos, string);	getstob: fn(cs: string): (Stob, string);	enumcs: fn(): list of (string, string, int);	aliases: fn(cs: string): (string, list of string);}; .EE.SH DESCRIPTIONThe.I Convcssuite is a collection ofmodules for converting various standardcoded character sets and character encoding schemesto and from the Limbo strings..PPThe.B Convcsmodule provides an entry point to the suite, mapping character set names and aliasesto their associated converter implementation..SS "The Convcs module".TP.BI init( csfile ).B Initshould be called once to initialise the internal state of.BR Convcs .The.I csfileargument specifies the path of the converter mapping file.If this argument is nil, the default mapping file.B /lib/convcs/charsetsis used..TP.BI getbtos( cs ).B Getbtosreturns an initialised.B Btosmodule for converting from the requested encoding..IPThe return value is a tuple, holding the module reference and an error string.If any errors were encountered in locating, loading or initialising the requestedconverter, the module reference will be nil and the string will contain an explanation..IPThe character set name,.IR cs ,is normalised by mapping all upper-case latin1 characters to lower-case beforecomparison with character set names and aliases from the.B charsetsfile..TP.BI getstob( cs ).B Getstobreturns an initialised.B Stobmodule for converting from strings to the requested encoding.Apart from the different module type, the return value and semantics arethe same as for.BR getbtos() ..TP.B enumcs()Returns a list describing the set of available converters.The list is comprised of.RI ( name ,.IR desc ,.IR mode )tuples..IP.I Nameis the standard name of the character set..IP.I Descis a more friendly description taken directly from the.B descattribute given in the.I charsetsfile.If the attribute is not given then.I descis set to.IR name ..IP.I Modeis a bitmap that details the converters available for the given character set.Valid.I modebits are.B Convcs->BTOSand.BR Convcs->STOB .For convenience,.B Convcs->BOTHis also defined..TP.BI aliases( cs )Returns the aliases for the character set name.IR cs .The return value is the tuple.BI ( desc ", " aliases )..IP.I Descis the descriptive text for the character set, as returned by.BR enumcs() ..IP.I Aliaseslists all the known aliases for.IR cs .The first name in the list is the default character set name \- the namethat all the other aliases refer to.If the.I aliaseslist is.B nilthen there was an error in looking up the character set name and.I descwill detail the error..SS "Using a Btos converter"The.B Btosmodule returned by.B getbtos()is already initialised and is ready to start the conversion.Conversions can be made on a individual basis,or in a `streamed' mode..PP.IB converter "->btos(" s ,.IB b ,.IB nchars ).RSConverts raw byte codes of the character set encoding to a Limbo string..PPThe argument.I sis a converter state as returned from the previous call to.B btoson the same input stream.The first call to.B btoson a particular input stream should give.B Convcs->Startstate(or.BR nil)as the value for.IR s .The argument.I bis the bytes to be converted.The argument.I ncharsis the maximal length of the string to be returned.If this argument is.B -1then as much of.I bwill be consumed as possible.A value of.B 0indicates to the converter that there is no more data andthat any pending state should be flushed..PPThe return value of.B btosis the tuple.RI ( state ,.IR str ,.IR nbytes )where.I stateis the new state of the converter,.I stris the converted string, and.I nbytesis the number of bytes from.I bconsumed by the conversion..PPThe same converter module can be used for multiple conversion streamsby maintaining a separate.I statevariable for each stream..RE.SS "Using an Stob converter"The.B Stobmodule returned by.B getstob()is already initialised and is ready to start the conversion..PP.IB converter "->stob(" s ,.IB str ).RSConverts the limbo string.I strto the raw byte codes of the character set encoding.The argument.I srepresents the converter state and is treated in the same way as for.IB converter ->btos()described above.To terminate a conversion.B stobshould be called with an empty string in order to flushthe converter state..PPThe return value of.B stobis the tuple.RI ( state ,.IR bytes )where.I stateis the new state of the converter and.I bytesis the result of the conversion..RE.SS "Conversion errors"When using .IB converter "->btos()"to convert data to Limbo strings,any byte sequences that are not valid for the specific character encoding schemewill be converted to the Unicode error character 16rFFFD. .PPWhen using.IB converter "->stob()"to convert Limbo strings,any Unicode characters that can not be mappedinto the character set will normally be substituted by the US-ASCII code for `?'.Note that this may be inappropriate for certain conversions, such converters will usea suitable error character for their particular character set and encoding scheme..SS "Charset file format"The file.B /lib/convcs/charsetsprovides the mapping between character set names and their implementation modules.The file format conforms to that supported by.IR cfg (2).The following description relies on terms defined in the.IR cfg (2)manual page..PPEach record name defines a character set name.If the primary value of the record is non-empty then the name is an alias,the value being the real name.An alias record must point to an actual converter record, not to another alias, as.I Convcsonly follows one level of aliasing..PPEach converter record consists of a set of tuples with the following primary attributes:.TP.B descA more descriptive name for the character set encoding.This attribute is used for the description returned by.BR enumcs() ..TP.B btosThe path to the.B Btosmodule implementation of this converter..TP.B stobThe path to the.B Stobmodule implementation of this converter..PPBoth the.B btosand.B stobtuples can have an optional.B argattribute which is passed to the.B init()function of the converter when initialised by.IR Convcs .If a converter record has neither an.B stobnor a.B btostuple, then it is ignored..PPThe following example is an extract from the standard Inferno.I charsetsfile:.PP.EXcp866=ibm866866=ibm866ibm866=    desc='Russian MS-DOS CP 866'    stob=/dis/lib/convcs/cp_stob.dis    arg=/lib/convcs/ibm866.cp    btos=/dis/lib/convcs/cp_btos.dis    arg=/lib/convcs/ibm866.cp.EE.PPThis entry defines.I Stoband.I Btosconverters for the character set called.BR ibm866 .The converters are actually the generic codepage converters.B cp_stoband.B cp_btosparamaterized with a codepage file.The entry also defines the aliases.B cp866and.B 866for the name.BR ibm866 ..SH FILES.TP.B /lib/convcs/charsetsThe default mapping between character set names and their implementation modules..TP.BI /lib/convcs/ csname .cpCodepage files for use by the generic codepage converters.B cp_stoband.BR cp_btos .Each file consists of 256 unicode runes mapping codepage byte values to unicode by their index.The runes are stored in the utf-8 encoding..SH SOURCE.TF /appl/lib/convcs/convcs.b.TP.B /appl/lib/convcs/convcs.bImplementation of the.B Convcsmodule..TP.B /appl/lib/convcs/cp_btos.bGeneric.I Btoscodepage converter..TP.B /appl/lib/convcs/cp_stob.bGeneric.I Stobcodepage converter..br.PP.SH SEE ALSO.IR tcs (1),.IR cfg (2)