Imagine creating a PowerBuilder application that not only speaks to the user, but also understands when the user speaks to it. Speech recognition software has been around for a while, but so far it's found only limited to success in real-world use. However, some of the latest advances in speech recognition make this form of input a viable option for business applications. In this article I'll describe how to use Microsoft Agent ActiveX technology to add speech and speech recognition to PB applications. About Microsoft Agent Version 2.0

Microsoft Agent is a royalty-free ActiveX technology that can be used to create an innovative conversational user interface that enables character animation, speech output and speech recognition for conventional 32-bit applications and Web-based applications. Microsoft Agent core components, the four Microsoft animation characters (Genie, Merlin, Peedy, Robby), speech engines, tools, full API documentation and code samples are all posted on the Microsoft Agent Web site at www.micro- soft.com/msagent. You can also design your own animated characters using the Microsoft Agent Character Editor, and then assign them to respond to voice commands that the user directs to the application. This technology is already part of Windows 2000.

What You Need to Begin
To use the Agent ActiveX control, you must first download and install it. (Before downloading, check for the minimum system requirements recommended at the Web site.) Download and install the Agent core components, at least one Agent character file (I chose Peedy, the bird), the Text-to-Speech engine, the Microsoft Speech Recognition engine and the Speech Control Panel. If you choose the default options, installation is fairly straightforward. Also, to use the ActiveX control in a PowerBuilder application you'll need the 32-bit version of PowerBuilder for development. I developed my sample application in PowerBuilder Version 6.5.

Creating the Agent Visual User Object
First, create an application using the Application painter. Then create a custom visual user object. Add an OLE control, and choose Microsoft Agent Control 2.0 by selecting it from the Insert Control tab as shown in Figure 1.

By using the events, properties and methods of this control, you'll implement animation and speech input and output features in your PowerBuilder application. (All the events, properties and methods for the Microsoft Agent are well documented at the Web site.)

Now let's add the functionality to show the ActiveX control. In the Constructor event of the user object load the Agent control (see Listing 1). To test the user object, save it (u_msagent) and close the user object painter. Create a window object of type Main and place the user object on the window. Save the window (w_msagent) object. In the Open event of the application object add the script to open the window. Save the application object and start the application. Voilˆ! The window and Peedy (or whatever character you've downloaded) should appear on the screen. This could take a little more time than usual because the Microsoft Agent animated character control is also being loaded. You can now have fun watching the various animations of Peedy. By default, pressing the Scroll Lock button activates listening. Nothing will happen now, though; I'll show you how to use voice recognition a little later.

Adding Speech Output and Animation Functionality
First, make sure a text-to-speech (TTS) speech engine is installed. (You should already have downloaded and installed the Lernout and Hauspie TTS speech engine from the download section of the Microsoft Agent Web site.) Now let's introduce speech output and animation functionality into the user object. Create a function of_Speak (see Listing 2) for the user object, which will make the Agent character speak. Then add the function of_Play (see Listing 3) to play the animated action. Save the user object, close the user object painter, then open the window. Place two command buttons on it, as illustrated in Figure 2.

Place the following code in the Clicked event for the Speak button.

uo_1.of_Play("Greet")
uo_1.of_Speak("Hi there. I am Peedy |" + &
"Hello. I am Peedy | Welcome to Speech output")

Note that the vertical line in the argument of the of_Speak function tells the Agent control to choose one of the texts separated by the vertical line to be spoken. Save the window, run the application and now click the Speak button. Wow! You've animated your application and enabled it to talk to the user. However, you still haven't made it possible for the user to talk to the Agent control. Wouldn't it be interesting to make an application respond to a voice command instead of traditional menu items on a menu bar? Let's go for it.

Adding Speech Recognition Functionality
The Agent control supports both speech input and a popup menu that appears whenever the user right-clicks on the character. To implement speech recognition functionality, create a function, of_AddCommand (see Listing 4), to add voice commands and menu items. The process of recognizing the voice command involves an Agent OLE control event called Command. In this event you trap the command that was executed and take the corresponding action. Enable these capacities by adding the appropriate code in the Command event of the OLE Agent control (see Listing 5) in the user object. Save the user object and close the user object painter. Now add the voice command to exit the application in the Open event of the window as follows:

uo_1.of_AddCommand("ue_close" , &
"Exit Application", &
"(close |exit) [the] (application|window|this)")

The code above will add the entry "Exit Application" to the popup menu of the Agent control referenced by ue_close. When the user selects the command on the popup menu or speaks a phrase that matches the third argument, the Command event of the Agent control is fired. Notice the third argument for defining the voice grammar: you can add optional words using square brackets and indicate alternative words with parentheses. The value of the command selected is in the Name property of userinput, identified by the first argument. This value will be posted as an event to the window.

To close the window or exit the application, create a user-defined event, ue_close, to call the PowerBuilder function Close(This). Save the window object and close the window object painter.

As mentioned earlier, you have to press the Scroll Lock button in order to activate the speech recognition or listening capability of the Agent control. You can programmatically press the Scroll Lock button using the Windows 32-bit SDK function keybd_event. First declare a local external function for the user object as follows:

SUBROUTINE keybd_event (int iVirtualKey, int iScan, int iFlags, int iExtraInfo) LIBRARY "user32.dll"

Refer to http://techinfo.sybase.com/
css/techinfo.nsf/DocID/ID=47760 for further details.

Next create a function of_Listen (see Listing 6) for the user object. This presses the Scroll Lock button for the Agent control and starts listening. Save the user object and close the user object painter. Add the following code in the Clicked event of the Listen button on the window:

uo_1.of_Listen()

Save the window object and run the application.

Now click the Listen button and wait until the Agent control starts to listen (see Figure 3).

Use the microphone to say "Close," "Exit" or anything that matches the voice grammar - or right-click and select the menu item "Exit Application" - and enjoy talking to the PowerBuilder application.

Conclusion
With the continuing growth in microprocessor speed, speech recognition will lead us to user interface innovations. Microsoft ActiveX Technology opens up a new conversational interface that can be profitably employed in all kinds of PowerBuilder applications. It's a new way to communicate with the computer that's incredibly easy to develop. Let your imagination run wild. There are plenty of resources on the Web to help you learn how to use this technology effectively. You can download this article and the source code from www.intac.com/~kaudata/download.

• Figure 1
• Figure 2
• Figure 3
• Source Code