Sketching is ubiquitous in medicine. Physicians commonly use sketches as part of their note taking in patient records and to help convey diagnoses and treatments to patients. Medical students frequently use sketches to help them think through clinical problems in individual and group problem solving. Applications ranging from automated patient records to medical education software could benefit greatly from the richer and more natural interfaces that would be enabled by the ability to understand sketches. In this paper we take the first steps toward developing a system that can understand anatomical sketches.
Methods: Understanding an anatomical sketch requires the ability to recognize what anatomical structure has been sketched and from what view (e.g. parietal view of the brain), as well as to identify the anatomical parts and their locations in the sketch (e.g. parts of the brain), even if they have not been explicitly drawn.We present novel algorithms for sketch recognition and for part identification. We evaluate the accuracy of the recognition algorithm on sketches obtained from medical students. We evaluate the part identification algorithm by comparing its results to the judgment of an experienced physician.
We have developed a virtual reality (VR) and an augmented reality (AR) dental training simulator utilizing a haptic device. The simulators utilize volumetric force feedback computation and real time modification of the volumetric data. They include a virtual mirror to facilitate indirect vision during a simulated operation. The AR environment allows students to practice surgery in correct postures by combining the 3D tooth and tool models with the real-world view and displaying the result through a video see-through head-mounted display (HMD). Preliminary results from an initial evaluation show that the system is a promising tool to supplement dental training and that there are advantages of the AR over the VR approach.
We present a dental training simulator that provides a virtual reality (VR) environment with haptic feedback for dental students to practice dental surgical skills in the context of a crown preparation procedure. The simulator addresses challenges in traditional training such as the subjective nature of surgical skill assessment and the limited availability of expert supervision.Methods and materials
We identified important features for characterizing the quality of a procedure based on interviews with experienced dentists. The features are patterns combining tool position, tool orientation, and applied force. The simulator monitors these features during the procedure, objectively assesses the quality of the performed procedure using hidden Markov models (HMMs), and provides objective feedback on the user's performance in each stage of the procedure. We recruited five dental students and five experienced dentists to evaluate the accuracy of our skill assessment method and the quality of the system's generated feedback.
The experimental results show that HMMs with selected features can correctly classify all test sequences into novice and expert categories. The evaluation also indicates a high acceptance rate from experts for the system's generated feedback.
In this work, we introduce our VR dental training simulator and describe a mechanism for providing objective skill assessment and feedback. The HMM is demonstrated as an effective tool for classifying a particular operator as novice-level or expert-level. The simulator can generate tutoring feedback with quality comparable to the feedback provided by human tutors.