Do Motion Metrics Lead to Improved Skill Acquisition on Simulators?

Condition(s) targeted: Performance Assessment; Motion Metrics

Intervention: skills training (Other)

Phase: N/A

Status: Completed

Sponsored by: Carolinas Healthcare System

Official(s) and/or principal investigator(s):
Dimitrios Stefanidis, MD, PhD, Principal Investigator, Affiliation: Carolinas Simulation Center

Emphasizing the growing popularity of motion metrics are the majority of available virtual reality simulators and some newer hybrid models that offer motion tracking for performance assessment. A popular hybrid model (PROMIS) allows training with regular laparoscopic instruments in a box-trainer while automatically recording task duration and movement efficiency (pathlength and smoothness) that are immediately offered as feedback to trainees. Despite the increasing availability of simulators that track motion, our knowledge of the impact those metrics have on trainee learning is severely limited. We do not know if it is more important to use speed, accuracy, motion efficiency or a combination thereof for performance assessment and how these metrics impact skill transfer to the OR. Based on sound educational principles we have developed a proficiency-based laparoscopic suturing simulator curriculum. This curriculum focuses on deliberate and distributed practice, provides trainees with augmented feedback and sets expert-derived performance goals based on time and errors. We have previously demonstrated that this curriculum leads to improved operative performance of trainees compared to controls. To measure operative performance and determine transferability, we will use a live porcine Nissen fundoplication model. Instead of placing actual patients at risk, the porcine model is preferable for this purpose as it offers objective metrics (targets are established, distances measured, knots are disrupted for slippage scoring), complete standardization, and allows multiple individuals to be tested on the same day. We hypothesize that proficiency-based simulator training in laparoscopic suturing to expert-derived levels of speed and motion will result in better operative performance compared to participants training to levels of speed or motion alone. The study is powered to detect an at least 10% performance difference between the groups. Specific Aims 1. Compare whether any performance differences between the groups persist long-term 2. Assess whether the groups demonstrate differences in safety in the operating room by comparing the inadvertent injuries in the animal OR between the groups 3. Identify the training duration required by novices to reach proficiency in laparoscopic suturing based on speed, motion efficiency, or a combination of these metrics 4. Identify any baseline participant characteristics that may predict individual metric-specific performance

Official title: Do Motion Metrics Lead to Improved Skill Acquisition on Simulators?

Study design: Allocation: Randomized, Endpoint Classification: Efficacy Study, Intervention Model: Parallel Assignment, Masking: Single Blind (Outcomes Assessor), Primary Purpose: Basic Science

Primary outcome: Laparoscopic suturing performance in the animal operating room

Secondary outcome:

inadvertent injuries in the animal OR

training duration required by novices to reach proficiency in laparoscopic suturing based on speed, motion efficiency, or a combination of these metrics

Detailed description: OBJECTIVE:: We hypothesized that training to expert-derived levels of speed and motion will lead to improved learning and will translate to better operating room (OR) performance of novices than training to goals of speed or motion alone. BACKGROUND:: Motion tracking has been suggested to be a more sensitive performance metric than time and errors for the assessment of surgical performance. METHODS:: An institutional review board-approved, single blinded, randomized controlled trial was conducted at our level-I American College of Surgeons accredited Education Institute. Forty-two novices trained to proficiency in laparoscopic suturing after being randomized into 3 groups: The speed group (n = 14) had to achieve expert levels of speed, the motion group (n = 15) expert levels of motion (path length and smoothness), and the speed and motion group (n = 13) both levels. To achieve proficiency, all groups also had to demonstrate error-free performance. The FLS suture module (task 5) was used for training inside the ProMIS simulator that tracks instrument motion. All groups participated in transfer and retention tests in the OR. OR performance was assessed by a blinded expert rater using Global Operative Assessment of Laparoscopic Skills, speed, accuracy, and inadvertent injuries.

Minimum age: N/A. Maximum age: N/A. Gender(s): Both.

Criteria:

Inclusion Criteria:

- novices with no previous laparoscopic or simulation experience

- voluntary participation

Exclusion Criteria:

- expert in or familiarity with laparoscopy or simulation

- physical condition that prevents the performance of laparoscopic suturing

Carolinas Simulation Center, Charlotte, North Carolina 28205, United States

Related publications:

A prospective analysis of 1518 laparoscopic cholecystectomies. The Southern Surgeons Club. N Engl J Med. 1991 Apr 18;324(16):1073-8. Erratum in: N Engl J Med 1991 Nov 21;325(21):1517-8.

Moore MJ, Bennett CL. The learning curve for laparoscopic cholecystectomy. The Southern Surgeons Club. Am J Surg. 1995 Jul;170(1):55-9.

Kohn LT, Corrigan JM, Donaldson MS (eds). To err is human: building a safer health system. Washington, DC: National Academy Press; 2000.

Hasan A, Pozzi M, Hamilton JR. New surgical procedures: can we minimise the learning curve? BMJ. 2000 Jan 15;320(7228):171-3. Review.

Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999 Jan;177(1):28-32.

Callery MP. Expansion beyond compression. Surg Endosc. 2003 May;17(5):677-8. Epub 2003 Apr 3.

Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005 Jul;201(1):23-9.

Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus DM, Jeyarajah DR, Thompson WM, Jones DB. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg. 2000 Sep;191(3):272-83.

Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, Tesfay ST, Jones DB. Improving operative performance using a laparoscopic hernia simulator. Am J Surg. 2001 Dec;182(6):725-8.

Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002 Oct;236(4):458-63; discussion 463-4.

Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004 Feb;91(2):146-50.

Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K; SAGES FLS Committee. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004 Jan;135(1):21-7.

Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004 Sep;240(3):518-25; discussion 525-8.

Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg. 2001 Nov;193(5):479-85.

Van Sickle KR, McClusky DA 3rd, Gallagher AG, Smith CD. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc. 2005 Sep;19(9):1227-31. Epub 2005 Jul 21.

Smith WD, Berguer R. A simple virtual instrument to monitor surgeons' workload while they perform minimally invasive surgery tasks. Stud Health Technol Inform. 2004;98:363-9.

Sierra R, Korndorffer Jr.JR, Stefanidis D, Touchard CL, Dunne JB, Scott DJ. Proficiency-based training: a new standard for laparoscopic simulation. Presented at the 2005 annual SAGES meeting in Hollywood, Fl.

Stefanidis D, Korndorffer JR Jr, Sierra R, Touchard C, Dunne JB, Scott DJ. Skill retention following proficiency-based laparoscopic simulator training. Surgery. 2005 Aug;138(2):165-70.

Stefanidis D, Korndorffer JR Jr, Markley S, Sierra R, Scott DJ. Proficiency maintenance: impact of ongoing simulator training on laparoscopic skill retention. J Am Coll Surg. 2006 Apr;202(4):599-603.

PROMIS surgical simulator.Web: http://www.haptica.com/. Accessed: 12/06/2007

Allen JW, Rivas H, Cocchione RN, Ferzli GS. Intracorporeal suturing and knot tying broadens the clinical applicability of laparoscopy. JSLS. 2003 Apr-Jun;7(2):137-40.

Ward M, MacRae H, Schlachta C, Mamazza J, Poulin E, Reznick R, Regehr G. Resident self-assessment of operative performance. Am J Surg. 2003 Jun;185(6):521-4.

Wolfe BM, Szabo Z, Moran ME, Chan P, Hunter JG. Training for minimally invasive surgery. Need for surgical skills. Surg Endosc. 1993 Mar-Apr;7(2):93-5.

Ericsson KA, Krampe RT, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993; 100:363-406.

Ericsson KA, Charness N. Expert performance: its structure and acquisition. Am Psychol 1994; 49:725-47.

Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004 Oct;79(10 Suppl):S70-81. Review.

Schmidt RA, Lee TD. Motor control and learning: a behavioral emphasis. Champaign, IL. Human Kinetics Publishers; 2005.

Magill R. Motor Learning and Control: Concepts and Applications. New York, NY: Mc Graw Hill; 2004.

Risucci D, Cohen JA, Garbus JE, Goldstein M, Cohen MG. The effects of practice and instruction on speed and accuracy during resident acquisition of simulated laparoscopic skills. Curr Surg. 2001 Mar;58(2):230-235.

Wulf G, McNevin N, Shea CH. The automaticity of complex motor skill learning as a function of attentional focus. Q J Exp Psychol A. 2001 Nov;54(4):1143-54.

Fried GM, Derossis AM, Bothwell J, Sigman HH. Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator. Surg Endosc. 1999 Nov;13(11):1077-81; discussion 1082.

Wickens CD, Hollands JG. Engineering psychology and human performance, 3rd Ed., Upper Saddle River, NJ: Prentice Hall; 2000.

Hart SG, Staveland L.E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock PA, Meshkati N (eds). Human Mental Workload. Amsterdam: Elsevier; 1987

Noether, Gottfried E., "Sample Size Determination for Some Common Nonparametric Tests". Journal of the American Statistical Association 1987; Vol. 82, No. 398, p. 647.

Starting date: November 2009
Last updated: March 22, 2013