Microsoft Speech API 5.1 support for CMU Flite---------------------------------------------- Copyright Cepstral, LLC, 2001 all rights reserved David Huggins-Daines <dhd@cepstral.com>December 6th, 2001 About the Flite SAPI port------------------------- Funding for this work was provided by the Instituto de Engenharia deSistemas e Computadores (Lisbon, Portugal).  The port itself was doneby David Huggins-Daines at Cepstral, LLC (Pittsburgh, USA). This work remains Copyright Cepstral, LLC but is distributed underthe same free software licence as CMU Flite. What's here----------- This directory contains a port of CMU Flite 1.1 and the included 8kHzdiphone voice to the Win32 platform under Visual C++, as well as aninterface library that allows Flite voices to be compiled into COMobjects which implement the TTS engine interfaces for Microsoft'sSpeech API 5.1 (SAPI). What isn't here--------------- There isn't a pointy-clicky tool for automatically converting voicesto SAPI objects.  You are more than welcome to write one; it should befairly straightforward to do as a Visual Studio wizard or add-in orwhatever they call those things. Instead, the procedure for making SAPI objects is documented below.Feel free to ask me questions at the address above if you don'tunderstand some parts of it. Some parts of the SAPI interface code are language specific, namelythe viseme and phoneme translation code, and to some extent the textprocessing code.  These have been implemented for US English only,although the relevant functions are abstracted with function pointersso each voice is free to choose its own.  See "Language-SpecificFunctions" below for more information. Building and testing the example voice-------------------------------------- In order to build the Flite SAPI code and example diphone voice, youwill need a copy of Visual C++ 6.0 or later, as well as the full SDKfor SAPI 5.1.  If you have the Microsoft Platform SDK, you will needto make sure to install the Internet Explorer SDK as well, as itcontains some IDL files that are required in order to build COMobjects with the Platform SDK. Before you build anything, you will need to set up Visual C++ to findthe SAPI header and IDL files.  To do this, select "Options" from the"Tools" menu.  In the resulting dialog box, switch to the"Directories" tab.  Now, add the "include" and "IDL" directories fromyour SAPI SDK installation to the list.  If you performed a defaultinstallation of the SAPI SDK on the C: drive, they will be:   C:\Program Files\Microsoft Speech SDK 5.1\Include  C:\Program Files\Microsoft Speech SDK 5.1\IDL Make sure the build type is set to "Win32 - Debug" in Visual C++.  Setthe active project to "FliteCMUKalDiphone".  Select "BuildFliteCMUKalDiphone.dll" from the "Build" menu.  This will build all ofthe other libraries before building the SAPI object. The first time you use the voice, you will need to register it withWindows.  To do this, build the "register_vox" project and executeregister_vox.exe (you can find it in theFliteCMUKalDiphone\register_vox\Debug subdirectory, or you can simplyrun it from within Visual C++). Although, normally, Visual C++ should automatically register theFliteCMUKalDiphone object as a COM server, in some cases you may haveto do it manually.  This can be done by running 'regsvr32FliteCMUKalDiphone.dll' on the command-line from the build directory(sapi\FliteCMUKalDiphone\Debug). Now you can test the voice by running the SAPI "TTSApp" exampleprogram, or any of the other examples included with the SAPI SDK. How to SAPI-enable your own Flite voices---------------------------------------- First, you'll obviously need to build your voice under Visual C++.This is relatively straightforward.  It's probably best to build it asa static library.  You'll need to make sure that it can find the Fliteheader files as well as the ones for your language and lexicon, whichprobably means adding their paths in the "Preprocessor" category ofthe "C/C++" tab of the Project Settings dialog box. One thing to be aware of is that the Microsoft linker will probablybreak if you have very large voice data files as C code.  To workaround this problem, you can change "Debug Info" from "ProgramDatabase for Edit and Continue" to "Program Database" in the "General"category of the "C/C++" tab in the Project Settings dialog box. Since there is no support for dynamically discovering and loadingvoices in Flite, each SAPI voice links to its own instance of theengine.  This also simplifies the distribution and installation ofvoices considerably. The common code used to implement the SAPI interface is in theFliteTTSEngineObj library.  Your voice should create a subclass ofCFliteTTSEngineObj.  In the minimal case, you only need to provide aconstructor that sets the 'regfunc' and 'unregfunc' members of thebase class to the registration and unregistration functions defined inyour Flite voice. To set up the voice object, create a new Visual C++ project, using theproject type "ATL COM AppWizard".  For "Server Type", choose "DynamicLink Library (DLL)". Now you must create a definition for the voice object.  To do this,switch to "ClassView" in the sidebar, right-click on the project name,and choose "New ATL Object...".  Then, from the selection box, choose"Simple Object". In the "Names" tab of the next dialog box ("ATL Object WizardProperties"), choose whatever names you like for your class.  In the"Attributes" tab, select "Both" in the "Threading Model" section, and"Custom" in the "Interface" section. Next, you need to edit the IDL file to import and use the relevantSAPI interfaces.  You should be able to find this file in the "SourceFiles" section of your new project in the "FileView" tab in thesidebar.  If your project name was "FooVoice", the IDL file will becalled "FooVoice.idl".  In order to import the SAPI interfaces, youshould add the following line to the list of 'import' statements atthe top of the file:   import "sapiddk.idl"; In order for Visual C++ to find this import, you will have to add theSAPI IDL directory it to the Tools->Options dialog box as detailedabove (do not add it to the Project Settings, because MIDL.EXE isbroken and will not accept it.) Just underneath the list of import statements, Visual C++ will havecreated a bogus interface definition for your object, which will looklike this (the UUID and interface name will be different, of course): 	[		object,		uuid(51284204-38B4-48C4-B65E-4FDAF6476D13),			helpstring("IFooVoiceObj Interface"),		pointer_default(unique)	]	interface IFooVoiceObj : IUnknown	{	}; You should delete this section.  Then, you should change the list ofinterfaces in the 'coclass' section to use the SAPI TTS Engineinterfaces.  It should look like this (replace "FooVoiceObj" with thename of your voice object): 	coclass FooVoiceObj	{		[default] interface ISpTTSEngine;		interface ISpObjectWithToken;	}; Now you will need to edit the source code for your voice object.  Asnoted above, in the minimal case, you will simply need to include"voxdefs.h" from your voice library, inherit from CFliteTTSEngineObj,provide declarations for REGISTER_VOX and UNREGISTER_VOX, andinitialize the pointers to them in a constructor. To do this, open the header file for your voice object.  It can befound in the "Header Files" section for your project in the "FileView"tab in the sidebar.  If your voice object name (as entered in the"Names" tab in the "ATL Object Wizard Properties" dialog box above)was "FooVoiceObj", then this file will be called"FooVoiceObj.h". First, add these declarations underneath the #include statements atthe top of the file:   #include "FliteTTSEngineObj.h"  #include "flite_sapi_usenglish.h"  extern "C" {  #include "voxdefs.h"  cst_voice *REGISTER_VOX(const char *voxdir);  void UNREGISTER_VOX(cst_voice *vox);  }; You will need to either put the full path to voxdefs.h in the #includestatement, or add the directory containing your voice's source code tothe list of extra include directories in the "Preprocessor" categoryof the "C/C++" tab of the Project Settings dialog box.  You may alsoneed to do the same for "FliteTTSEngineObj.h" (if you create yourvoice within the "flite_sapi" workspace included here, you can enter"..\FliteTTSEngineObj" here). Next, you must adjust the inheritance list for your voice's class.Remove the following lines marked with '-' and add the line markedwith '+': -	public CComObjectRootEx<CComMultiThreadModel>,	public CComCoClass<CFooVoiceObj, &CLSID_FooVoiceObj>,-	public IFooVoiceObj+	public CFliteTTSEngineObj You must also change the COM interface map to contain the SAPIinterfaces, by making the following changes (as above, remove thelines marked with '-' and add those marked with '+'):   BEGIN_COM_MAP(CFooVoiceObj)-  	COM_INTERFACE_ENTRY(IFooVoiceObj)+  	COM_INTERFACE_ENTRY(ISpTTSEngine)+  	COM_INTERFACE_ENTRY(ISpObjectWithToken)  END_COM_MAP() Finally, add code to the constructor to set the 'regfunc' and'unregfunc' members, and the language-specific functions, if you havethem, by adding the lines marked with '+':   public:	  CFooVoiceObject() {+		regfunc = REGISTER_VOX;+		unregfunc = UNREGISTER_VOX;+		phonemefunc = flite_sapi_usenglish_phoneme;+		visemefunc = flite_sapi_usenglish_viseme;+		featurefunc = flite_sapi_usenglish_feature;+		pronouncefunc = flite_sapi_usenglish_pronounce;	  } Before you build the SAPI object, you will need to add the voicelibrary and the Flite libraries (flite.lib, plus the libraries for thelexicon and language model, which are cmulex.lib and usenglish.lib forUS English) to the list of extra libraries (in the "Input" section ofthe "Linker" tab of the Project Settings dialog box in Visual C++).You also need to include "winmm.lib" here as it is required by theFlite library. You'll also need to make sure that Visual C++ can findthe Flite libraries - you can either set their projects up asdependencies of your SAPI object's project, or you can add a list ofrelative paths to their build directories in the Project Settings. Registering and testing your new SAPI voice object-------------------------------------------------- Now that you've built your voice as a SAPI component, you mustregister it with the system so that it can be found and used byprograms using the SAPI interface.  Predictably, this involvestwiddling bits in the Windows Registry. Source code and a Visual C++ project is provided (in register-vox.cpp)for a small command-line program which performs the necessaryoperations for the CMU diphone voice.  To use it for another voice,you will need to make the modifications noted by /* CHANGEME */comments in the source code. To test your SAPI voice, use the TTSApp program included with the SAPISDK.  You may find it helpful to build the debugging version of TTSAppfrom source code and specify it as the executable to run whendebugging for your SAPI object's project.  This will allow you to setbreakpoints in your code and get proper backtraces and so forth. Language-specific functions--------------------------- The language-specific functions for US English are contained in thefiles flite_sapi_usenglish.c and flite_sapi_usenglish.h.  The SAPIobject is set up to use them by initializing four function pointerswhich are members of class CFliteTTSEngineObj:   int (*phonemefunc)(cst_item *s); This function takes a cst_item representing a single phone (usually amember of the "Segment" relation) and returns the appropriate SAPIphone ID for it.   int (*visemefunc)(cst_item *s); This function takes a cst_item representing a single phone and returnsthe appropriate SAPI viseme ID for it.  The SAPI visemes arepotentially language independent, though they are expressed in thedocumentation in terms of US English phonemes.  A more generaldescription of them is included below. SP_VISEME_0	silenceSP_VISEME_1	low mid/front unrounded vowels (ae, ah, ax)SP_VISEME_2	low back unrounded vowels (aa)SP_VISEME_3	low/mid-low back rounded vowels (ao)SP_VISEME_4	mid front unrounded vowels (eh, ey)SP_VISEME_5	English mid rhotic vowel (er)SP_VISEME_6	high front unrounded vowels and glides (ih, iy, y)SP_VISEME_7	high back rounded vowels and glides (uw, w)SP_VISEME_8	rounded-to-rounded rising diphthongs (ow)SP_VISEME_9	unrounded-to-rounded rising diphthongs (aw)SP_VISEME_10	rounded-to-unrounded rising diphthongs (oy)SP_VISEME_11	unrounded-to-unrounded rising diphthongs (ay)SP_VISEME_12	English glottal fricative (hh)SP_VISEME_13	English retroflex approximant (r)SP_VISEME_14	English lateral approximant (l)SP_VISEME_15	grooved alveolar/dental fricatives (s, z)SP_VISEME_16	palatal fricatives and affricates (sh, zh, ch, jh)SP_VISEME_17	interdental fricatives (th, dh)SP_VISEME_18	labiodental fricatives (f, v)SP_VISEME_19	alveolar/dental occlusives (d, t, n)SP_VISEME_20	velar occlusives (k, g, ng)SP_VISEME_21	bilabial occlusives (p, b, m) When adapting them to your phoneset, remember that the position of thelips and teeth as viewed from the front is more important than theplace or manner of articulation /per se/. Also, while the US English code implements this using a table lookup,it may be more appropriate to determine them algorithmically from thefeature values used in your phoneset.   int (*featurefunc)(cst_item *s); This function returns a bitmask used by SAPI to indicate placement ofstress or emphasis (see the pages on the SPEVENTENUM and SPVFEATUREenumerations in the SAPI documentation for more information on this).If you use "stressed" and "accented" as the feature names for thesethings in your language code, you can just copyflite_sapi_usenglish_feature().   cst_val *(*pronounce_func)(SPPHONEID *spids); This function takes a zero-terminated array of SAPI phone identifiersand converts it to a list of cst_val containing the phone names asused by Flite as strings.  You will probably want to simply copyflite_sapi_usenglish_pronounce(), changing the tables it uses to lookup phone names.