22 residents were included in this crossover study, where they performed internal fixation of the ulna using both, the Sawbones simulator, and a newly developed virtual simulator to compare the validity of the two devices. Residents performance was assessed using a checklist and a global rating scale (GRS) in accordance with the Objective Structured Assessment of Technical Skill (OSATS) format along with the time to completion. For the Sawbones simulator, construct validity was established for the checklist, the GRS and the time to completion. However, the newly developed simulator only established construct validity for the GRS score. Concurrent validity of the newly developed virtual simulator could not be established. Based on these results, the virtual simulator requires a higher user-satisfaction and further development to compare to the existing Sawbones simulator.