What is NUIA?

  • NUIA is a software that makes your device empathic to us humans, using new sensors like 3D cameras or eye trackers. On the one hand it allows you to control your computer with your gaze, gesture and voice and on the other hand it reacts based on your posture and behavior to do things automatically for you.

Can I use NUIA with Linux, OS X or Android?

  • No, currently NUIA only supports Windows 7, Windows 8.1 and Windows 10 (all 64 Bit).

Do I need external hardware?

NUIA isn't running?

  • Make sure NUIA is running (there should be a green NUIA icon in the tray section next to the system clock).
    • If it is not running (Icon is red), right-click on the icon and press 'Start NUIA'.
    • If the icon is not visible in the tray menu check if it is automatically hidden by clicking the upwards pointing arrow in the tray menu.
    • If the NUIA icon is not in the tray menu launch NUIA from the Windows programs menu.
  • If you put Windows to sleep/standby mode sometimes the RealSense SDK crashes and the device needs to be rebooted.
  • Make sure the RealSense camera or the eye tracker is recognized. Sometimes the RealSense camera or the eye tracker aren't recognized -> replug the peripheral.
  • Make sure NUIA is fully loaded before use.

Voice commands are not recognized at all?

  • The RealSense microphone or your external microphone has to be configured as default input (before NUIA is started). The Audio Level Indicator (activated/deactivated via the NUIA settings) at the bottom center of the screen indicates the last eight seconds of the microphone input. If it does not display anything, set the RealSense microphone or your external microphone as default input: Go to 'Control Panel' -> 'Sound‘ -> 'Recording' and right click on the respective entry and select 'Set as Default Device' and 'Set as Default Communication Device'.
  • If you put Windows to sleep/standby mode sometimes the RealSense SDK crashes and the device needs to be rebooted.
  • To use the voice commands with an eye tracker you need an external microphone and select 'Speech Recognition' in the NUIA settings.

Voice commands are not/rarely recognized?

  • Make sure the application or window which you want to use has the focus (If not click on it).
  • If the Audio Level Indicator at the bottom center of the screen only displays low input signals you can try to activate the microphone boost at 'Control Panel' -> 'Sound‘ -> 'Recording'. Double click the proper microphone select the boost level in the tab 'Levels'.
  • Set the default format of the microphone at 'Control Panel' -> 'Sound‘ -> 'Recording' -> 'Microphone' -> 'Advanced' to "2 channel, 16 Bit, 48000 Hz (DVD Quality)".
  • Currently only US English is supported, so you should try to pronounce commands with an US accent.
  • Make sure to pause shortly before and after the command to increase the recognition rate.

Menus do not react to gestures as expected?

  • The NUIA Modality State Indicator (activated/deactivated via the NUIA settings) at the bottom center of the screen indicates whether you hold up your hand or not. The NUIA Hand Gesture Visualizer shows a detected hand gesture: Open hand = valid trigger gesture, closed hand = not a valid trigger gesture. Red color indicates that the hand is either too close to the camera or too far away.
  • Make sure the application or window which you want to use has the focus (If not click on it).
  • Make sure that the camera is aligned correct (your hand should be somewhere in the inner area of the camera field of view).
  • For optimum performance, position your hand at a distance of 40 cm to 100 cm (1'3" to 3'3") from the camera.
  • Try to move your hand in different speeds (maybe you are moving your hand to fast or to slow).
  • Gesture Commands are not available when using NUIA with an eye tracker.

Applications do not react to posture as expected?

  • The NUIA Modality State Indicator (activated/deactivated via the NUIA settings) at the bottom center of the screen indicates your posture. The NUIA User Position Visualizer shows the currently detected user position: Engaged, Lean Back or No Head Found.
  • Make sure that the 3D camera or the eye tracker is aligned correctly (your head should be somewhere in the inner area of the camera or eye tracker field of view).
  • Make sure that your face is looking towards the screen.
  • The Engaged Mode is active when you are in the range of 0 to 80 cm (0 to 2'4") from the camera. If no face is recognized in this range the Lean Back Mode is activated.

General

  • Actually supported applications or web sites cannot be controlled with NUIA: Before you begin to work with any supported application or web site make sure NUIA is running.
  • If you have installed an application which NUIA supports while NUIA is running, stop and restart NUIA via the NUIA tray menu.
  • Make sure the application or window which you want to use has the focus (If not click on it).

Adobe Reader

  • Currently only the "Read" view is supported.

Facebook.com

  • Currently Facebook.com works only in the Firefox and Chrome browser. Support for Edge is on the roadmap.
  • Make sure the NUIA Firefox add-on is activated (go to Firefox settings -> Add-ons -> NUIA Browser Extension).
  • Make sure the NUIA Chrome add-on is activated (go to Chrome settings -> Extensions -> NUIA Browser Extension).
  • Currently only the news feed page is supported.

Google Chrome

  • Make sure the NUIA Chrome add-on is activated (go to Chrome settings -> Extensions -> NUIA Browser Extension)

Mozilla Firefox

  • Make sure the NUIA Firefox add-on is activated (go to Firefox settings -> Add-ons -> NUIA Browser Extension).

Microsoft PowerPoint

  • Currently only the editing mode is supported.

VLC Media Player

  • The first playback needs to be started with the mouse, before NUIA control is possible.
  • NUIA commands will not work when iTunes is running at the same time.

Youtube.com

  • Currently Youtube.com works only in the Firefox and Chrome browser. Support for Edge is on the roadmap.
  • Make sure the NUIA Firefox add-on is activated (go to Firefox settings -> Add-ons -> NUIA Browser Extension).
  • Make sure the NUIA Chrome add-on is activated (go to Chrome settings -> Extensions -> NUIA Browser Extension).
  • Currently only the video detail page is supported.