This week I was delighted to attend the Medidata Next Basel event, and even more happy to be part of the winning team in the Hackathon! Medidata's Hackathons are a great opportunity to spend a day working on something new, surrounded by enthusiastic experts from Medidata and the industry. In Basel I worked with Panos Beretsos (Takeda) and the Medidata team; Ross Rothmeier, John Anstey and Christine Boese. We decided to focus on an area of the clinical trial process which is still largely manual, is time-consuming, costly and like all manual processes, error-prone; user acceptance testing of Rave eCRF edit checks.

After the eCRF and edit checks have been specified and programmed it's necessary to test and document that the edit checks work as expected; opening queries, adding forms and other actions. Today this typically involves a test script, maybe documented in Excel and then manually entering data into Rave EDC, recording if the results are as expected and pasting screenshots into the results document.

We estimated that for the 20 companies attending Medidata Next, if each company starts 100 studies a year and it takes 15 days effort to conduct user acceptance testing that's a total of 30,000 days effort per year, and at $300/day that equates to $9,000,000. What could we do to reduce that cost and effort?

Automated eCRF user acceptance testing

Once we'd decided to focus on this area we split our efforts; Panos worked with Christine, Medidata's Design Thinking Expert, to map out workflows:

and to create mock-ups of what an overall solution might look like:

Alongside that work, I concentrated on developing a core part of the solution, using Medidata APIs (Rave Web Services) to automatically create subjects, enter data onto eCRF forms and capture screenshots of the results. This is exactly the same process that a person follows now to verify that edit checks have been programmed correctly, but now it's automated, fast and can be repeated at the click of a button.

The application allows a free-text specification of the Edit Check to be documented and then one or more test scenarios to be specified using the Gherkin language. Gherkin is widely used in the software industry to capture requirements and to test software; Medidata use it to test Rave and other products and we use it to test our TrialGrid application. Gherkin uses a simple Given-When-Then concept to specify a starting point ('Given a subject exists'), a user action ('When I enter the following data...') and an expected outcome ('Then a query should be opened...'). We thought this would be a good starting point for expressing the Edit Check user acceptance tests in an easily-readable format that is also executable by our system:

The main work of the day was to take these scenarios and create the test runner which reads each step in the scenarios, calls Rave Web Services to take the appropriate action and documents the results including screenshots of Rave itself. As ever there were many small steps to take and unexpected issues to overcome, but steady progress during the day meant that by the time of the 'reveal' at 7pm I was ready to demonstrate a working example:

Here we're showing the documented results of running the test scenarios - each step has been successfully executed, we have screenshots embedded in the results and we can re-run again and again whenever we need to.

We'll be taking this proof-of-concept work and bringing it into our production application; please contact us us if you'd like to hear more about our plans for this!

The Medidata Hackathon was a really enjoyable, fun day. Hard work, but it paid off. If you have the opportunity I strongly recommend participating in one of the upcoming events in Berlin or London. You don't have to be a programmer to contribute; experience, ideas and enthusiasm are needed too!