Method and Apparatus for Correcting and Sharing Whiteboard Images using a Digital Camera

Computing & Wireless : Computing Methods

Available for licensing

Inventors

  • Alan Bovik, Ph.D. , Electrical and Computer Engineering
  • Casen Hunger , University of Texas at Austin
  • Zichong Li , University of Texas at Austin
  • Mitchell Crooks , University of Texas at Austin
  • Mark Meserve , University of Texas at Austin
  • Edward Mora, B.A. , Applied Research Laboratories
  • Michael Webb , Microelectronics Research Center

Background/unmet need

Current methods of recording classroom lectures or other such presentations involve either audio recording only or simple video recording of the lecture scene without processing. Simple audio recording is far less useful than visualization of an accompanying whiteboard image. However, simple video recording gives a visual signal that is hard to interpret because of geometric distortion, poor acquisition conditions, the obstruction of an occluding, moving speaker, and poor color reproduction. Therefore, inventors at The University of Texas at Austin have developed a processing system that both improves readability of recorded whiteboard content and removes the speaker from the frame. Moreover, this novel system provides the capability of sharing the whiteboard presentation via wireless or wired connection using an app or website access by any standard computer device such as personal computer, smartphone or tablet.

Invention Description

The invented method and apparatus teaches how to acquire a video frame or frames from a digital camera, such as a webcam, tablet, or cell phone camera, and save the frames in matrices that serves as the data type for video manipulation throughout the system. The input video frame is passed to a whiteboard detection stage, where the limiting boundaries of the whiteboard image in each frame are detected. Using a suitable quantitative ranking technique, the system determines the quadrangle that is most likely to spatially coincide with the boundaries of the image of the whiteboard. The system then crops this area from the original image, corrects perspective distortion effects, and then applies color balancing and other image correction and enhancement methods. Once the system has performed geometric correction of the image, the continuous video is processed and displayed in like manner over suitable regular or irregular time intervals so that the viewer may view the changes occurring on the whiteboard over time.

In a further aspect of the invention, the system processes and sends video frame or sequence of video frames to a "speaker removal" or "hide the professor" subsystem. The idea of this stage is to remove the image of the speaker from the video frame or frames to allow an unobscured view of the whiteboard content. This is accomplished by finding and tracking changes that occur over subsets of the multiple frames in the image. Changes arising from actual additions or deletions (e.g., writing or drawing) on the whiteboard itself are distinguished from changes arising from the speaker, including shadows and lighting changes, by measurements related to spatial area and temporal persistence.

The resulting video frame or frames, with or without application of "hide the professor," are sent via wired or wireless link to a computer server, which saves the received streams so that they may be viewed at a later time. This would be accomplished, for example, through a smartphone/tablet app or via a website the user could log into. Such interaction by the user could occur on any personal computer, Android or iOS smartphone, or tablet portable device.

Additionally, the video frame or frames may be processed to accomplish color processing of the frames to improve the readability and distinctiveness of the text and drawings in the whiteboard images. Further, the white video frame(s) can be subjected to video encoding to compress them for more efficient transmission and storage.

Benefits/Advantages

    The invention provides a vastly improved visual and educational experience over simply taking an audio and/or video recording of the whiteboard during a lecture or talk.

Features

  • Automatic whiteboard detection and correction
  • Speaker/presenter removal
  • Database storage of whiteboard videos

Market potential/applications

Companies involved in digital camera design, cellphone application designers, education-oriented companies, communications firms, etc. 

Development Stage

Proof of concept

IP Status

  • 1 U.S. patent application filed