Iara Commands

Commands allow that Iara voice recognizer be used as a way of interaction by the user with its application. Through the commands, a user can request, using the voice, that the application save some work content, for example.

Basic operation

Commands are compound by two main parts: one activation string and a callback function, invoked when the activation string is recognized in the current transcription. Many commands can be triggered, and they don't impact in the voice recognition performance.

Before use any command, it's important observe that the activation string needs to be compound, necessarily, by existent words in the Iara language dictionary. By this reason, if the command hello, how are you is created, it doesn't mean that it will be sent.

IMPORTANT: commands necessarily need to be compound by existent Iara language dictionary words.

The command hepatic bile ducts, for example, will be recognized, since each one of the three used words to compound the activation strings exist in the Iara dictionary.

A command can be composed by one or many words (don't have a real limit). Although, it is recommended that commands don't have more than 3 or 4 words, to ensure a good UX and a recognition of command high rate.

Basic usage

Commands are added in the recognizer through commands.add():

var recognition = new IaraSpeechRecognition();

recognition.commands.add('activation string', function callback() {
    console.log('Comando executado');
});

recognition.init({
    // ...
}).done(function(e) {
    // ...
}).fail(function(e) {
    // ...
});

TIP: As a good UX practice, it is recommended that the commands start with the word iara followed by the desired action. For example, iara text could be used to create a new text, while iara report would create a new report.

Commands can be added any time during the life cycle of your application. In this way, they can be added before or after the recognizer initialization:

var recognition = new IaraSpeechRecognition();

// Command added before the recognizer initialization
recognition.commands.add('iara texto', function callback() {
    console.log('Comando executado: iara texto');
});

recognition.init({
    // ...
}).done(function(e) {
    // Command added before the recognizer initialization
    recognition.commands.add('iara laudo', function callback() {
        console.log('Comando executado: iara laudo');
    });
}).fail(function(e) {
    // ...
});

// Command added while the application runtime.
recognition.commands.add('iara fígado', function callback() {
    console.log('Comando executado: iara fígado');
});

Commands can be removed any time through commands.remove().

Contextualization parameters

Commands can be easily contextualized through parameters. The callback function of a command will receive these parameters.

Own SDK parameters

A command callback function will receive, always, two parameters provided by the SDK: detail e command, in this order.

The example below illustrates the parameters received by the callback:

recognition.commands.add('iara texto', function(detail, command) {
    // The command "iara texto" was recognized. Parameter `detail`
    // contains the details of the recognized audio, `command` contains the
    // command iara recognized in this callback ("iara texto", in this case).
    console.log('Iara comando: ', detail, command);
});

The parameter detail of the callback contains the details of the recognized audio, which is exactly the same object of the property event.detail of a recognition, as illustrated in the example below:

var recognition = new IaraSpeechRecognition();

recognition.init({
    // (...)
}).done(function(e) {
    recognition.onresult = function(event) {
        console.log('Recognition details: ', event.detail);
        //                                          --------^^^
        // The property `detail` of `event.detail` above is the same object passed as
        // first parameter for the callback of an activated command.
    };
});

Through the parameter detail, the callback function of the command has the access to the content of the current transcription, that can be used in the own command, for example.

The parameter command, the 2º parameter received by the callback in the command, containing the activation string that caused this callback invocation. In other words, the recognized command has caused the invocation. In this section first example, the command "iara text" has been added, so the callback parameter command will contain the string "iara text".

TIP: through the parameter command of the commands callback, the application can use the just one function to treat all the commands, if it is interest. It is only necessary that the parameter command be inspected to define which command was actually fired.

Additional parameter to the callback function

When a command is added by commands.add(), you can provide some data that will be passed, by parameter, for that command callback. The data to be passed to the callback function can be of any type, object, string, integer, etc.

The value is passed as third parameter of commands.add(), and will be available to the command callback also as the third parameter:

function callback(detail, command, param) {
    // The parameter `param` will contain the string "olá mundo".
    console.log('Command ' + command + ' invoked with the parameter ' + param);
}

var recognition = new IaraSpeechRecognition();

// Add the command `iara texto` with the extra data "olá mundo"
recognition.commands.add('iara texto', callback, null, ['olá mundo']);

recognition.init({
    // ...
}).done(function(e) {
    // ..
}).fail(function(e) {
    // ...
});

See the code example iara-commands to learn more about commands.

Advanced usage

Commands can have their behavior adjusted to very specific necessities of each application. This customization is made through the command configuration object. The configuration object is passed as 4º parameter of commands.add():

function callback(detail, command, param) {
    // ...
}

// Configuration object
var cmdConfig = {
    // key: value,
    // key2: value2,
    // ...
};

var recognition = new IaraSpeechRecognition();

// Adds the command `iara texto` with the extra data `olá mundo`
// and additional configuration defined by `cmdConfig`.
recognition.commands.add('iara texto', callback, null, ['olá mundo'],  cmdConfig);

The configuration object can contain the following fields:

Field	Type	Default value	Description
`transcript`	`bool`	`true`	Keep activation string in `detail.transcript`?
`richTranscript`	`bool`	`false`	Keep activation string in `detail.richTranscript`?
`replaceValue`	`string`	`''`	To replace the command string activation if it is removed of the transcriptions.
`replaceFunction`	`function`	`null`	Function that will be used to make the activation string replacement by `replaceValue` in the transcriptions.
`resultFunction`	`function`	`null`	Function that will be called to change the recognition result.
`searchRichTranscript`	`bool`	`false`	Search for the activation string in `detail.richTranscript`?

Control over `transcript` and `richTranscript`

The Iara voice recognition will make available the transcription of any audio through the properties event.detail.transcript and event.detail.richTranscript. If your application is using commands, the activation string will also appear in the transcription.

By default, the activation string of any command is kept in transcript and removed of richTranscript. This can be customized through the configuration transcript and richTranscript. These fields show if the activation string should be kept in event.detail.transcript and/or event.detail.richTranscript, respectively. By this way, can be controlled if the audio recognition events will contain or not the transcription of the command in question.

In the example below, the command activation string will be removed both transcript and richTranscript when the command be recognized:

function callback(detail, command, param) {
    console.log('Comando ' + command + ' invocado com parâmetro ' + param);
}

// Configuration object of the command
var cmdConfig = {
    transcript: false,     // Keep in script? No, the command text, i.e. "iara alerta", must be removed from the script.
    richTranscript: false  // Keep in richTranscript? No, the command text, i.e. "iara alerta", must be removed from richTranscript.
};

var recognition = new IaraSpeechRecognition();

// Adds the command `iara alerta` with the extra data `olá mundo`
recognition.commands.add('iara alerta', callback, null, ['olá mundo'], cmdConfig);

The replacement value to `transcript` and `richTranscript`

The configuration properties transcript e richTranscript control if the command activation string should be kept or not in the transcriptions. If the activation string must be removed, you can define the string to assume its place. This is controlled by the configuration property replaceValue.

By default, replaceValue has the value '' (empty string), which means that the command activation string is replaced by an empty string. However, any string can be used. In the example below, the command activation string is replaced by @TOMO in the properties transcript and richTranscript:

function callback(detail, command, param) {
    //...
}

var cmdConfigReplace = {
    transcript: false,       // Keep in transcript? No, replace with the content of `replaceValue`.
    richTranscript: false,   // Keep in richTranscript? No, replace with the content of `replaceValue`.
    replaceValue: '@TOMO',   // String that will be used to replace the activation string.
};
recognition.commands.add('iara tomografia', callback, null, ['olá mundo'], cmdConfigReplace);

recognition.init({
    // (...)
}).done(function(e) {
    // Assuming that the user said "vias biliares iara tomografia":
    recognition.onresult = function(event) {
        console.log(event.detail.transcript); // vias biliares @TOMO
        console.log(event.detail.richTranscript); // <div>vias biliares @TOMO</div>
    };
});

Replacement custom function for `transcript` and `richTranscript`

If your application needs more control in the replacement of a command activation string beyond change it by replaceValue, you can implement your own replacement function and inform it through replaceFuncion.

The signature of replaceFunction is the following:

function(haystack, needle, replaceValue, prop) -> String

where haystack is the string containing the recognition current transcription, needle is the command activation string, replaceValue is the informed string in the command's configuration object, and prop is a string with the property name being treated, i.e. "transcript" or "richTranscript".

The function replaceFunction must, necessarily, return a string that will be assigned to the transcription (transcript or richTranscript) as result of the replacement the command activation string.

In the example below, the command iara controle is added. When it is activated, the activation string "iara controle" will be replaced by the string "_CONTROLE:transcript" in the property transcript, and by "_CONTROLE:richTranscript" in the property richTranscript:

function callback(detail, command, param) {
    //...
}

var cmdConfigFunc = {
    transcript: false,       // Keep in transcript? No, replace.
    richTranscript: false,   // Keep in richTranscript? No, replace.

    replaceFunction: function(haystack, needle, replaceValue, prop) {
        var text = (haystack || '') + '';
        var regex = new RegExp(needle, 'gi');
        replaceValue = '_CONTROLE:' + prop;

        return text.replace(regex, replaceValue);
    }
};

recognition.commands.add('iara controle', callback, null, ['olá mundo'], cmdConfigFunc);

recognition.init({
    // (...)
}).done(function(e) {
    // Assuming that the user said "vias biliares iara controle":
    recognition.onresult = function(event) {
        console.log(event.detail.transcript); // vias biliares _CONTROLE:transcript
        console.log(event.detail.richTranscript); // <div>vias biliares _CONTROLE:richTranscript</div>
    };
});

Changing a transcription result

If the application need, for some reason, change any recognition result when a command is executed, this can be done through the configuration property resultFunction.

Similarly as replaceFunction, the reported function in resultFunction will be called when the command is executed, but it will receive as parameter the object that will be propagated as a recognition result event (parameter event of recognition.onresult). The resultFunction signature is the following:

function(Object resultEvent, String needle, Object entry) -> Void;

where resultResult is the event object that will be propagated as recognition result, needle is the command activation string and entry is an object containing the command in question information, as activation string, settings, etc.

See the code example iara-commands-advanced to learn more about the subject.

Search the activation string in richTranscript

If your application needs to register an iara command whose activation string is a richTranscript format text, this can be done through the configuration property searchRichTranscript.

If this property has the value true, the activation string must be a richTranscript format text. This can make easier the use in cases where the activation contains elements as acronyms, vertebrae, numbers, and others, as seen below:

function callback(detail, command, param) {
    //...
}

var cmdConfigFunc = {
    searchRichTranscript: true
};

// If the property searchRichTranscript wasn't true, the command would be:
// recognition.commands.add('iara l cinco s um', callback, null, null, cmdConfigFunc);
recognition.commands.add('iara L5-S1', callback, null, null, cmdConfigFunc);

recognition.commands.add('iara exame 2', callback, null, null, cmdConfigFunc);

recognition.commands.add('iara Modic II', callback, null, null, cmdConfigFunc);

recognition.init({
    // ...
}).done(function(e) {
    // ...
});

Using Regex

If your application needs to register a more flexible command, you can provide a regex as activation string. For this, the activation string must follow the format specified in the documentation for RegEx objects construction with string patterns.

Some tips for use:

For the regex to have a backslash, it must be preceded by another
Try to limit the regex scope at maximum possible, because the command callback will be called in the first correspondence with regex

You can still use capture groups in the provided regex. If some group is captured, it will be provided to the callback through the parameter groups. This parameter is a string array, which the first index is the total correspondence with regex, and the following indexes are the regex captured groups. See an example usage below:

function firstCommandCallback(detail, command, param, groups) {
    if (groups) {
        console.log('iara texto:', groups[1]);
    }
    //...
}

function secondCommandCallback(detail, command, param, groups) {
    if (groups) {
        console.log('número do exame:', groups[1]);
    }
    //...
}

// Capture any recognized word after "iara texto".
// To capture special characters as ã, é, see:
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Unicode_Property_Escapes
recognition.commands.add('iara texto (\\p{Letter}+)', iaraCommandCallback);

// Specifies that the command must be compared with the parsed text, that is, the rich transcript
var searchRichTranscriptConfig = {
    searchRichTranscript: true
};

// Captures any recognized number sequence after "iara exame"
recognition.commands.add('iara exame (\\d+)', secondCommandCallback, null, null, searchRichTranscriptConfig);

recognition.init({
    // ...
}).done(function(e) {
    // ...
});

Basic operation​

Basic usage​

Contextualization parameters​

Own SDK parameters​

Additional parameter to the callback function​

Advanced usage​

Control over transcript and richTranscript​

The replacement value to transcript and richTranscript​

Replacement custom function for transcript and richTranscript​

Changing a transcription result​

Search the activation string in richTranscript​

Using Regex​