Do I have to use the mouse to start and stop dictation?

There are other ways to start and stop the dictation

There are a couple of alternatives for starting and stopping dictation that do not involve clicking the mouse.

Talkatoo has hotkeys, or a sequence of key presses that will start and stop dictation for you.  These are currently not programmable, meaning that they cannot be changed, but this option will be available in a future release.

Windows: CTRL+ALT+S 
Mac: COMMAND+OPTION+S

Press all 3 keys together and release to toggle the dictation on and off.

You can also show and hide the Talkatoo toolbar:

Windows: CTRL+ALT+H 
Mac: COMMAND+OPTION+H

Programmable microphones

An even better alternative is to use a premium handheld microphone with programmable buttons, such as the Philips SpeechMike series or the Olympus RecMic II series.  Both of these families of microphones come with software for programming hot keys.  In addition to starting and stopping dictation, they can be configured to press Tab, Enter, or any other useful key or sequence of keys, as well having a built in mouse with buttons.