Making it Personal
Public-speaking situations such as classroom lectures, seminars, and meetings, where speakers must actively engage the audience, require considerable effort from the speaker in gathering verbal and non-verbal feedback from the audience. Garnering feedback can be made easier by technologies such as augmented reality (AR) capable of displaying information in 3D space surrounding the speaker. I created an AR-enabled presentation display system to provide real-time feedback from the audience to the speaker during a presentation. The feedback includes names and affective states of the audience, icons requesting a change in volume and rate of speech, and annotated questions and comments.
The AR feedback system was implemented using a Microsoft HoloLens augmented reality head-mounted display (HMD). The HoloLens is an optical see-through display that projects images onto a transparent screen, allowing the user to see through the HMD. Using its array of cameras and sensors, it creates a 3-dimensional map of the surroundings, which allows for the placement of computer-generated elements in 3D space inside the real world. The HoloLens houses a computer within its body to allow for wireless un-tethered usage where the user can roam freely within the surroundings. In addition to the HoloLens, the presentation setup included a laptop computer placed on a podium to run the presentation, a clicker device to advance slides, and a large screen TV to display the presentation slides to the audience members. The audience provided feedback to the presenters using tablet devices. A router was used to set up a local access network to allow interfacing between the tablets and the HoloLens HMD.
The presenter interface running on the HoloLens HMD was created using the Unity game engine. AR tracking for the HoloLens was enabled using the Vuforia AR Engine within Unity, which allowed for recognition and real-time tracking of optical markers. The feedback interface on the tablets included unique full-screen optical marker images for each audience member which they held up for the HoloLens to calibrate their location at the start of the presentation. The audience feedback interface was a website built using PHP, HTML, and CSS, and was hosted using an Apache HTTP web server running on a local network.
Research Presentation at the UBICOMP 2020 Conference
In a between-subjects research study, we evaluated presentations displaying feedback information spatially overlaid above the heads of the audience members in one condition and in the periphery of the presenter’s view in another condition, as compared with a no-AR control condition. Results showed that the presenters in the overlay condition called upon the audience by name significantly more often than in the peripheral and the control conditions, and they rated the overlay condition as more reliable, helpful, and wanting to use it for future presentations compared to the peripheral condition. Overall, AR feedback was considered useful by both the presenters and the audience and did not negatively impact speaker confidence and state anxiety of the presenter.