VoiceXML Applications Rewritten Using SALT Tags

James A. Larson
~~April 25, 2003~~
July 1, 2004

After VoiceXML—An Introduction to Developing speech Applications was published, a new set of XML tags appeared for use in constructing both speech and multimodal applications. Speech Application Language Tags (SALT) was developed by the SALT Forum (http://www.saltforum.org) founded by Cisco, Comverse, Intel, Microsoft, Phillips and Speech Works. SALT is a small collection of XML tags for managing speech resources: speech recognition, speech synthesis, DTMF, and audio record and playback facilities.

Developers can not develop speech applications with SALT tags alone. The SALT tags contains no flow control structures. Instead, developers insert SALT tags into a host language such as HTML, SMIL, or JavaScript which provides the control structures needed to write speech-enabled applications. One popular host is HTML, which provides the visual component while SALT provides the verbal component for multimodal applications.

The basic SALT tags are:

<listen>—configures the audio record and speech recognition facilities to capture spoken words and phrases.
<DTMF>—configures the DTMF facility to process touch tone input
<prompt>—configures the audio replay and speech synthesis facilities to present verbal information to the user. The text in a <prompt> tag may contain W3C Speech Synthesis Markup Language (SSML).
<bind>—used to assign data captured either by listen or DTMF to variables in the host language.
<smex>—a simple message exchange system for sending and receiving message from other local and remote components. For example, <smex> can be used to manage telephone calls, submit data to a backend server, send and receive e-mail, and exchange messages with other components.
<grammar>—defines the words and phrases which the speech recognition system listens for. Grammars may be specified using the W3C Speech Recognition Grammar Specification Markup Language (SRGS).

SALT also supports other bindings. For example, SALT functions can be accessed from JavaScript or other scripting languages.

In addition to <bind> and <smex>, there are several features supported by SALT that are not available in VoiceXML, including:

Three speech recognition modes:
- Automatic—Used for recognition in telephony or hands-free scenarios. The speech platform rather than the application controls when to stop the recognition facility.
- Single—Used for push-to-talk applications. An explicit stop from the application returns the recognition result.
- Multiple—Used for open-microphone” or dictation. Recognition results are returned at intervals until the application makes an explicit stop.
A prompt queue and commands for managing the presentation of prompts on the queue to the user.
Flow control—rather than rely upon a default flow control algorithm such as VoiceXML’s Forms Interpretation Algorithm, SALT developers must use the host language to explicitly sequence and coordinate the various speech activities—in effect write their own version of the Forms Interpretation Algorithm.

For instructions to install SALT on Microsoft's Internet Explorer, follow these steps. We recommend installing only the SALT add-in to Internet Explorer. We do not recommend installing the entire SALT SDK unless you alread have Microsoft Visual Studio.net installed because this is an expensive piece of software.

Here are some hints and tips to follow when writing applications using HTML and SALT:

An ECMA function foobar is invoked by writing foobar(). A SALT <prompt> method foobar1 and <listen> method foorbar2 are invoked by foobar1.start() and foobar2.start().
A HTML form Form1 with input control input3 can be set by an ECMA script: Form1.input3.value = “5”; A HTML form Form1 with select control select4 can be set by an ECMA script: Form1.input4.indexedvalue = “5”;
It is necessary to explicitly start the application by supplying the initial function/method as a value of the onload attribute of <body>, e.g., <body onload="intro.start();">
In the <bind>statement, use value="/" to assign the entire semantic structure from the speech recognition engine, and value="//" to assign just the value from the semantic structure created by the speech recognition engine.
Microsoft's SALT does not support DTMF. This makes it difficult to implement applications involving passcodes and other private information.
Microsoft's SALT does not support prompt queues.
Microsoft's SALT does not support semantic interpretation. This complicates any application that involves alternative words for the same semantic operation
If you downloaded only the SALT add-in to Microsoft Internet Explorer, then use CLASSID="clsid:33cbfc53-a7de-491a-90f3-0e782a7e347a". If you downloaded the entire SALT SDK, then use CLASSID="clsid:DCF68E5B-84A1-4047-98A4-0A72276D19CC"

To illustrate how to use SALT tags, examples from my book on VoiceXML has been rewritten using SALT and HTML. You can interact with each of the simple applications by installing Microsoft's Internet Explorer add-in and then clicking the following URLs. For each example, I have kept the debug screen which shows the contents of variables; these may be instructive as you explore the SALT applications. Real SALT telephony applications can no display the screen to users without special telephones.

Figure from Book	File Name	Purpose	Comments
Figure 2.2	SALT22.html	A document containing both a form and a menu
Figure 4.9	SALT49.html	Call ring application.
Figure 4.10	SALT410.html	Call ring application with additional words in grammar
Figure 4.11	SALT411.html	Complete call ring application with event handlers
Figure 5.3	SALT53.html	Example menu
Figure 5.4	SALT54.html	Example Form with three fields
Figure 6.1	SALT61.html http://www.larson-tech.com/SALT/LIONROAR.WAV	Links identified by nospeech sounds
Figure 6.2	SALT62.html http://www.larson-tech.com/SALT/shortbeeptone.wav	Use of a turn-taking tone
Figure 6.3	SALT63.html http://www.larson-tech.com/SALT/arrowhit.wav	Use of a brand earcon
Figure 6.4	SALT64.html	Example of chunking
Figure 6.5	SALT65.html	Example of a pause between instructions and options
Figure 6.6	SALT66.html	Example of an introduction
Figure 6.7	SALT67.html	Help that is available anytime
Figure 6.8	SALT68.html	Universal independent commands
Figure 7.1	SALT71.html	Inline grammar using XML notation
Figure 7.3	SALT73.html cities.grxml	External grammar using XML notation
Figure 8.5	SALT85.html	Example two-level DTMF menu (using spoken numbers in place of DTMS)	Say digits rather than press DTMF keys were are not available in Internet Explorer
Figure 8.6	SALT86.html	Tapered prompts
Figure 8.7	SALT87.html	Progressive assistance
Figure 8.8	SALT88.html	Speech input version of Figure 8.5
Figure 8.9	SALT89.html	Voice menu that compresses menu hierarch of Figure 8.8
Figure 8.10	SALT810.html	Forward-feeding prompts
Figure 9.2	SALT92.html	Command and control
Figure 10.10	SALT1010.html	Mixed initiative dialog
Figure 10.12	SALT1012.html	Switching among activities
Figure 11.3	SALT113.html	Example VocieXML form for collecting caller preference data
Figure A.1	SALTA1.html	Solicit caller's current location
Figure A.2	SALTA2.html	Present information to the caller
Figure A.3	SALTA3.html shortBeeptone.wav	Prompts for open-ended questions	Example of recording and playback of audio. Error message appears at end of program for an unknown reason

Here are some example of SALT multimodal applications:

Purpose	url	Use
Interview the Cassini space probe	http://www.larson-tech.com/SALT/Saturn/interview.html	If the user asks a question including one of the keywords, the user sees and hears an interesting message. Keywords include Casini, launch, Titan, Galileo, Saturn, mission, JPL, instruments.
Make a meatloaf	http://www.larson-tech.com/SALT/recipie/recipie.html	Design for cooks whose hands are busy and messy, this voice activated recipie reads recipie instructions to the cook. Say next, previous, goodbye
User-directed	http://www.larson-tech.com/SALT/UserDirected.html	Click and speak
System-directed	http://www.larson-tech.com/SALT/SystemDirected.html	Question and Answer
Spell word	http://www.larson-tech.com/SALT/Catalog/change1.htm	Enter a word by spelling it. Say "change position to letter" to correct spelling. Also demonstrates semantic interpretation

Debrah Dahl's violin tuner application: http://www.larson-tech.com/SALT/Dahl/Tuner/music/tune.htm

Pranav Chadha's shopping cart application: http://www.larson-tech.com/SALT/ShoppingCart/MusicMad.htm

Selected application from Fall 2004

Topic	url	Comments
News Consolidator	http://www.larson-tech.com/Fall2004/NewsConsolidator/YourNews.htm
City navigation	http://www.larson-tech.com/Fall2004/CityNavigation/p3.html
Sound card replacement	http://www.larson-tech.com/Fall2004/Pankani/project3/salt-app-3.html
Phone card	http://www.larson-tech.com/Fall2004/PhoneCard/Project3.html
Pizza 1	http://www.larson-tech.com/Fall2004/Pizza/pizza.html
Pizza 2	http://www.larson-tech.com/Fall2004/Pizza2/p3.html
Pizza 3	http://www.larson-tech.com/Fall2004/Warner/rlwProject3.htm
Maintenance Request	http://www.larson-tech.com/Fall2004/RobertAnderson/College%20Housing%20Northwest%20-%20Apartments%20-%20Work%20Requests.htm	Application by Robert Anderson, Original Site Design and Digital Imaging by college Housing Northwest (Copyright 2004)
Movie Tickets	http://www.larson-tech.com/Fall2004/Tran/proj3.html
Synthesizer Maintenance	http://www.larson-tech.com/Fall2004/Synthesizer/	Application and pictures by Damian Ramsey
Books	http://www.larson-tech.com/Fall2004/Vu/multimodal/project3.html

For those of you implemented your projects using the SALT toolkit, you must do the following to your appliation so it will run under the SALT add-in to IE:

1. Change the CLASSID to CLASSID="clsid:33cbfc53-a7de-491a-90f3-0e782a7e347a"

2. In the prefix to all SALT tags, change "SALT" to "salt"

3. I've not been able to get the <content> element to work without posting the referenced file on my web site and using the complete web site address for the file.

Selected applications from Winter 2005

Topic	url	Comments
Verbal drawing program	http://www.larson-tech.com/SALT/Winter2005/Smith/saltpaint.html	Select the collars by clicking or by speaking. Contributed by Jessamyn Smith and Smitha Karthik.
Highlights of Portland	http://www.larson-tech.com/SALT/Winter2005/Davidson/	Select interesting places in Portland by speaking or clicking. Contributed by Thadius Davidson.
Browse using verbal commands	http://www.larson-tech.com/SALT/Winter2005/jin/Voice%20Menu/	Start by asking "help" and be sure to say "bookmark" which will display several interesting sites that can be browed. Contributed by Ho Jeong AN and Jin Rak Choi