Building Edit Checks is one of the most time-consuming activities during study builds.
In this post I'll show how much TrialGrid can speed up Edit Check programming, using three example Edit Checks of increasing length and complexity.

Here's a comparison of the time taken to build these Edit Checks in TrialGrid and in Medidata Rave (shorter is better!):

As you can see it is consistently faster in TrialGrid. To measure the time and effort I used a stopwatch app and an app which records how many mouse clicks and key presses you've made. As we'll see, building Edit Checks in Rave can involve a LOT of mouse clicks.

1. Simple Edit Check

Many Edit Checks are relatively straightforward, looking for inconsistencies in data entry between two Fields, and then raising a query, as in this example:

1
2
3
4

On form AE, if AEONGO is checked and AEENDAT has a value, then:
    open a query with message "End Date is provided, but Ongoing is also checked"
    on field AEONGO
    to Marking group "Site from System"

TrialGrid

Here's the TrialGrid Edit Check editor with the apps collecting timings and metrics on the right:

Note: when building these Edit Checks in both TrialGrid and Rave I copy-pasted the query text from the specification document, as would probably happen in practice, so the key presses don't include typing the query message.

We can see a readable description of the Edit Check at the top, then 2 versions of the check logic; on the left-hand side we see the TrialGrid Clinical Query Language version and on the right-hand side we see the Rave 'quickedit' format. When creating an Edit Check in TrialGrid we type in the CQL expression and the quickedit version is automatically created. We provide lots of autocomplete support to help choose Fields, Forms and other objects, and highlight any errors:

If a Field is unique to one Form then its not necessary to specify the Form - the system knows that Field 'AEONGO' is on the AE Form, and only on that Form, so we don't need to enter it. If we need to specify Forms, Folders, etc we can do so, as in the next example Edit Check.

Medidata Rave

In Medidata Rave we have to think about Edit Checks in postfix logic - this Edit Check has 6 check steps in postfix:

So we can see that creating this Edit Check in TrialGrid takes 29 seconds compared with 76 seconds in Rave - more than twice as fast. Rave needs almost 4 times as many mouse clicks. TrialGrid requires more key presses, but typing is faster than the repetitive mouse activity for Rave.

2. Medium Edit Check

Next I took Edit Checks with more complex specifications, like this one, which is checking that vitals signs have been measured within a 4 hour time window after drug administration:

1
2
3
4

In CYCLE2, check that the 2nd vitals signs measurements where taken 4 hours after drug administration (+/- 30 minutes).  If not:
    open a query with message "Vital signs were not recorded within 4 hours (+/- 30 minutes) of drug administration"
    on field VSDTC in form VS1
    to Marking group "Site from System", requires response and requires manual close

TrialGrid

Above we can see how Clinical Query Language lets you specify Folders, Forms and record positions as well as the Fields.

Rave

For this Edit Check Rave needs 21 postfix steps:

TrialGrid makes more use of the keyboard, but the Rave editor takes more than 2 minutes longer because most mouse clicks trigger a request back to the Rave web server to refresh the page.

3. Long Edit Check

Some Edit Checks are considerably longer and involve many Fields, for example checking that inclusion and exclusion criteria have been met. Sometimes this is done with a simple yes/no question asking if all criteria have been met, but sometimes each inclusion and exclusion criterion is a separate question. In the following example there are 10 inclusion questions and 28 exclusions.

The specification for this check can be quite simple:

1
2
3
4

If the patient is eligible and any inclusion or exclusion criteria is not met
    open a query with message "If at least one Inclusion/Exclusion criteria is not met, "Is the patient eligible for inclusion on this study?" should be No. Please reconcile"
    on field ELIGIBLE in form ELIGCRIT
    to Marking group "Site from System", requires response and requires manual close

TrialGrid

In TrialGrid Clinical Query Language we can use the 'any' function and list the Fields:

Rave

In Rave we have to create 159 postfix checksteps. Unsurprisingly this takes a long time and its difficult to maintain concentration during the repetitive point-and-click for each step; I'd spent 45 minutes and more than 800 mouse clicks when I lost concentration for a second and accidentally clicked on the reset button on the stopwatch app:

At this point I'd completed about 130 out of 159 check steps and I didn't have the mental strength to redo the exercise since it had already taken almost 10 times longer than in TrialGrid.

For those of you familiar with Rave Edit Checks, you might also have been thinking that using the 'quickedit' format would be quicker. In Rave 'quickedit' the checksteps are presented in a raw format, delimited by a pipe '|' character. You can edit this:

Editing using 'quickedit' is faster than the point-and-click Rave editor, but still almost 2 times slower than in TrialGrid.

Summary

Here's the comparison again of the time taken to build these example Edit Checks in Rave and TrialGrid:

Clearly its more efficient to create an Edit Check in TrialGrid, but the difference for each Edit Check ranges from just under a minute for simple checks to four minutes for longer ones. What does this add up to, how much time can be saved for a whole study? I'll answer that question in my next blog post...

Who? When? What? Why? As a Clinical Programmer assigned to maintain an existing Rave study these are all questions you will be asking yourself. Who last modified this study build? When did they change it? What did they change and perhaps most important, Why did they change it?

Here's a Form in the TrialGrid Form Editor:

This is not dissimilar to the Form Editor in Rave Architect except for the small side-bar on the right showing some additional state information. If we expand that:

Now we can see that:

1. There is a Ticket (#8) associated with this Form and the ticket is in an Open state
2. The Form has been labelled as "Blocked"
3. The user Ian Sparks has performed some activities which changed the state of this Object - this Activities view shows all changes made to the object over time. We can also see that Ian made a comment, referencing the user @anewbigging1 and explaining why it is blocked
4. The user has commenting permission on this object and can add additional comments

This is all important contextual information that would normally exist in some other system - a tracking spreadsheet, an Issue management or workflow system like Jira or in the worst case an email thread between the participants.

By including the ability to label objects with workflow state (Labels), to link with work to be done (Tickets) and to talk about the work item (Comments) we enable a team to collaborate more easily to get the study build done. The end result is faster development of your study build.

Of course, comments aren't helpful if nobody is reading them. That's why, every time you @mention another team member in a comment (or ticket, or wiki page) in TrialGrid, that user gets a notification email with a link to take them direct to the comment (via login of course):

These capabilities are just the latest addition to TrialGrid, we're continuing to work on features that bring all aspects of Rave Study Build into a single, unified environment. This is important because although we can provide tools (like Form and Edit Check editors) which are more efficient, much of the time spent on study build is in the gaps between tools where it can't be measured such as the communication between team members.

Having team communication in email is inefficient because you have to repeat all the context i.e. what URL, what Project, what Draft, what object is this question about? It also means that decisions aren't automatically documented because they are hidden in emails.

In contrast, comments in TrialGrid are attached to the object they reference so that discussion about an object is documented alongside that object. Decisions are captured for the whole team to see now and in the future which is especially important for long-running studies where the maintenance team is likely to change.

How much could your organization save by centralizing study build workflow in a tool that centralizes both the build activities and the team communication?

As veterans of several EDC systems including Medidata Rave, Andrew, Steve and I are sometimes asked if we plan to build a new EDC system?

We don't. Competition is good but whether you are a fan of Medidata Rave (as we are), Oracle Inform or any one of the many available EDC systems, a new EDC system is not likely to improve your study build process.

So No, we don't plan to write another EDC system. Instead our focus is on taking the Medidata Rave Study Builder experience from good to Awesome (and we're all British here, we don't say "Awesome" lightly).

Tools and Process

The Tufts Center for the Study of Drug Development recently reported that the time to complete study build is a major factor in the delay of study starts. A new EDC system won't improve the efficiency of the process much because it is just one tool of many that are required to complete the study build process.

To use an analogy, think of your kitchen at home. Will buying a new stove make the process of preparing meals more efficient? It might but unless the stove is really broken it is more likely that the gain will only be marginal.

Much more important to efficiency is the layout of your kitchen. Unless you live in a small space the chances are your kitchen is laid-out in a traditional triangle with the refrigerator, sink/worktop and stove all situated at the corners.

(Credit Wikipedia: https://en.wikipedia.org/wiki/Kitchen_work_triangle)

The kitchen is a familiar example of a work cell, an arrangement of tools that is optimized to make a process more efficient. The arrangement means that ingredients, stove, water etc are all in close proximity so that travel time and the time to switch between tasks is minimized.

So is your EDC system really broken or is it just that the overall process isn't designed to get the maximum efficiency from it?

Work cell software for Software Development

In recent years online tools that act as work cells have appeared for Software Developers. You may have heard of online tools such as Bitbucket, GitHub and GitLab but if you haven't, these are services which bring together source code management, code review, issue tracking, documentation, automated build and many other tools in a single environment. These tools have been hugely beneficial to us at TrialGrid, streamlining our process and allowing us focus on writing more than 45,000 lines of code, 1,800 unit tests and hundreds of requirements tests in a little over 15 months.

Our goal is to bring those same benefits to Medidata Rave study build teams.

TrialGrid : A software work cell for your Rave Study Builds

If you have a team building Medidata Rave studies, then alongside Rave itself you will have tools and processes for:

• Managing Specifications
• Managing a team roster of responsibilities for a study
• Shared file system or cloud storage for study documents such as Protocol versions
• Issues, list of tasks left to do or of findings from User Acceptace Testing
• Tracking the tested status of Edit Checks and Custom Functions
• Tools (or people) to perform best practice and quality checks on objects built for the study
• Tracking standards compliance of the build along with reasons for deviation
• Some system to gather metrics together in order to report on the whole process

You might have best of breed solutions for many of these: Jira for Issue Tracking, Microsoft Sharepoint for storing documents and collaboration, SAS Programs written specially to perform quality checks, Microsoft Project for your team management and a custom Specifications Management tool (even if this is a spreadsheet) but the constant switching between tools slows down the overall effort and improving any one of them doesn't do much to optimize the process as a whole.

TrialGrid brings together:

• Enhanced study build tools for Edit Checks, Forms, Deviations, Matrices and more
• Standards Compliance and Global Library workflow - with deviation explanations and reports
• Automated study build best practice checking tools - with instant audit-trailed fixes
• Issue management with task assignment and links to study build objects
• Project File storage
• Project Wiki for team collaboration
• Study team assignments
• Audit trail of all actions, including study build activities

Into a single system so that the team doesn't have to switch between tools. Issues can be linked directly to the objects that implement that spec (e.g. direct to an Edit Check definition) and the history of that Edit Check can be reviewed (where did it come from? who created/edited it, when) along with its compliance to a global standard (does it deviate? How? Has this been explained? Has the deviation been accepted? Rejected?) Team members can run best practice checks on the study at any time and immediately fix or explain any findings. All the tools are linked, together in once place.

Specifications management is an area we are particularly interested in. Specifications are a type of Issue and can be used to manage Specifications but we have bigger plans.

We still have work to do but if you are frustrated with the pace of your EDC study builds get in touch to see how TrialGrid can help.