CDASH, SDTM and the FDA

Hurrah!  The FDA have made an announcement on their preference towards SDTM!!  Well.   Sort of.   They met up with representatives from CDISC. The CDISC organization wrote down some notes on the discussion, and posted them to their Blog.

Ok – maybe I am being overly flippant. However, why does this message need to come out by proxy from CDISC?  Why can the FDA CDER / CBER not step off the fence and make a firm statement on what they want, and when they want it?

One point made was that applying CDASH is the key to attaining SDTM datasets.  Well.  Sort of.  It is a good start point. But, it is only a start point.

The CDASH forms are very closely modeled on the structure of SDTM domains.   Do I always want to capture one domain, on one eCRF form – not always.  Do I want to sometimes capture information that is logically grouped together according to source documents that belongs to multiple domains on the same eCRF – often I do.  We should not compromise the user friendliness and therefore compliance at the sites because of a need to capture data according to the structure of the data extracts.

CDASH was developed around the principle that the EDC or CDM system modeled eCRF’s to equal SDTM domains.   If your EDC or CDM system does not do that, then compliance with CDASH is not entirely valuable.

However – or rather HOWEVER – if you fail to apply equivalent naming conventions to CDASH/SDTM and fail to use matching Controlled Terminology, and, you expect to achieve SDTM – you will be severely disappointed. Achieving SDTM will not be hard – it will be virtually impossible.

With regards to the statement that applying CDASH can create 70-90% savings.  That is not the whole story.  Apply CDASH + standardizing all of the other elements such as rules, visits etc – and automating testing and documentation – yes, then you can achieve a 70-90% savings.

CDISC Rules 2

In my last posting, I discussed potentially using ARDEN as a syntax for expanding CDISC ODM with rules.

After a couple of months of on and off investigation, I have decided that ARDEN is dead as an option. Actually, ARDEN is largely dead as a potential syntax in general.

The value of a rules syntax lies primarily in the potential ability to put context around data once it reaches a repository or data warehouse.

In theory, the transfer of rules would be of value in transferring a study definition between systems. However, I cannot think of a really valuable situation where this might happen. If data is captured into an IVR System and then transferred across to EDC - does it really matter if they both have access to the rules? Instead, the rules could be applied by one of the systems.

That last point takes me to the other reason why rules are less critical. If the last decade was about standards development this new decade must be about standards application - and, in particular the real time exchange of data between systems. The need to validate data in System A first, before transferring it to System B is only really necessary if System A cannot check directly with System B. With the increasingly prevalent Web Services combined with standards - it will be possible to carry out these checks online.

CDISC Rules!

Ok, so a play on words. CDISC may rule in the field of Clinical Data standards, but, it does not rule in the standardisation of rules associated with data.

Let me expand here for those not familiar with the issue.

CDISC ODM provides a syntax for the definition of metadata (and data) used in the interchange of information between (and sometimes within) systems. CDISC ODM does not scope the definition of edit check rules that are applied to the data when it is captured. I feel that is a significant omission as the rules element of the data a) take a considerable time to develop and b). provide a context to the data.

Question - So, why do we not already have rules built into the standards?

Answer - rules are often technology or vendor specific. There are almost as many methods of implementing rules, as there are EDC products.

Question - Why not define a standard mechanism for creating rules that vendors could either comply with, or, support as part of interfacing?

Answer - Well, it all depends on what you want the rules to do. In their simplest form, rules are boolean expressions that result in the production of a Query or Discrepancy. However, many systems go well beyond simply raising queries. The boolean element of the rule may be consistent, but the activity performed in the situation that the boolean returns true, is often very vendor specific.

Lowest Common Denominator

So - lets assume that we are looking at implementing a lowest common denominator of rules and actions that the majority of systems support, and require

What can we do to standardize a syntax. Three options I think;

1) Choose a syntax from one of the leading vendors,

2). Develop a new syntax building on existing ODM conventions

3). Bring in another standard syntax, potentially already in the Health or LifeScience field

Lets look at them in order.   

No. 1 - Choosing a Leading Vendor Syntax is probably great for the chosen leading vendor, but, bad for most other vendors. A benefit though would be that it would already be proven as a means to represent rules and actions in a clinical study. Some syntaxes are based around standard tools such as Visual Basic for Applications, JavaScript or even SQL. This approach may create almost insurmountable boundaries for other vendor systems that do not, or cannot implement the technology - for example, it is not easy to interpret VBA on a non Microsoft platform. So - option 1 has some potential, but, depending on the chosen vendor, may result in closing the door to the standard for others.

No. 2 - Creating a new Syntax would result in something most vendors would be happy with, but, would require considerable effort from the contributors in order to develop a complete specification for the standard, as well as a reference implementation. The advantage of such approach would of course be that it would be seen as a common standard open to all, and not specifically biased to any one vendor company. In practice, the technology approach chosen would favor some more than others.

No. 3 - Leverage an existing Syntax may well bring the benefits of No. 2 without all the costs of designing something from scratch.

Ok, so lets say we go ahead with option 3 - what are the candidate standards for rules in the Health and/or LifeSciences are?

As far as I can tell, not many. In fact, I was only able to find one candidate that had any level of success - a syntax called ARDEN.

ARDEN has existed since 1989 as a syntax for describing Medical Logic. Similar to typical Rules in EDC, they are defined in Modules - Medical Logic Modules - and called based on the triggering of an event.

[For an accurate definition of ARDEN and its roots, check Google Books - search for ARDEN Syntax and examine Clinical knowledge management: opportunities and challenges By Rajeev K. Bali, Pages 209 --> 211]

As a syntax, it is mostly general purpose. Here is a snip from an Arden Syntax module,

logic:

if

last_creat is null and last_BUN is null

then

alert_text := "No recent serum creatinine available. Consider patient's kidney function before ordering contrast studies.";

conclude true;

elseif

last_creat > 1.5 or last_BUN > 30

then

alert_text := "Consider impaired kidney function when ordering contrast studies for this patient.";

conclude true;

else

conclude false;

endif;

;;

In the example, you can see that the syntax uses standard if/then/elseif/endif constructs. Assignments use the := combination etc.

HL7 have a section dedicated to ARDEN here. The activity appears to be limited with no Postings or Documentation since 2004. On walking through some of the presentations, some of the consumer companies such as Eclipsys were proposing extensions to the syntax - for example to add object definition support. It would appear that the take-up of the standard has been limited to those organizations that had a problem to solve in the EHR area, and wanted to re-use a syntax, instead of inventing their own.

The fact that HL7 has lended support for ARDEN may be sufficient in itself. However, we would need to dig considerably deeper to understand how ARDEN syntax would fit with a syntax such as ODM. The first challenge is the conversion to an XML form. There are plenty of articles on ARDEN XML for further reading.

RuleML is another standard that may address the need to create Rules, as well as meeting the perceived need of being XML based.

More about ARDEN and RuleML in a later posting I think. This one is quite long enough for today.

Source to eSource with EDC

One of the areas that I have felt for some time as being a compromise to the effectiveness of EDC, is the subject of Source data, or rather, the fact that source data is often not entered directly into an EDC system.

I appreciate that we have situations where this is impractical for logistical reasons - location of computers, circumstances of source data capture etc.

However, often, it is mandated by the sponsor that source data is not logged in the EDC system, but is instead recorded elsewhere first.   Some advocates will indicate the need to comply with the regulations that state data must remain ‘at the site’.   Personally, I don’t concur with this assessment. The data is ‘at the site’. The cable connecting the screen to the computer might be very very long (the internet) but the data is constantly available at site. I probably shouldn’t be flippant on this point, but, the conservatism in conflict with progress strikes a nerve.

Transposing information from paper to an EDC screen introduces the potential for error.   In the old world days of paper CDM, we had Double Data Entry as a method to confirm transcription errors did not occur - 2 staff enter the same data from the paper CRF’, and the differences flagged/corrected.   With onsite EDC we don’t have Double Data Entry, but, we do have Source Data Verification. Instead of 2 staff sitting next to each other double keying data, we have a monitor fly in to perform the 2^nd check. Yes, I know, they carry out other duties, but still. This seems like an enormous effort to check for transcription issues. It also has a massive effect on slowing down time to database lock.

So – where are the solutions:-

1). Data at Site

We could put on our ‘common sense hats’ and make a statement that allows the entry of source data into the online EDC system. I know of a number of EDC companies that have simply placed this in their procedures. No audit findings / concerns have been raised as a result that I am aware of. Come on Regulators – why the delay in making a statement on this subject? The confusion caused is creating a measurable delay in bringing drugs to market!

2). Physical availability

This one is harder to tackle. When you have data to log, do you always have access to a system/device to log the data? Possibly not. Do you simply try to remember it, like an Italian waiter remembering a dinner order… I don’t think so. We either need to provide portable data entry devices, or, we accept paper transcription for these elements.

3). Differentiating Source from eSource

This does open up one area of concern. If we have some data as source, and some as eSource, how do we know which is which? When a Monitor goes looking for the source data, if they don’t find it, does that make it eSource – no.   There needs to be some very simple flagging, and visual indication system built into such a mixed source/esource system that supports this. I have seen this in one system, but, it is very rare. Come on EDC vendors – you’re turn here!

4). Other systems

Health Record systems amongst others will often be the first point of entry for data that may apply to an eCRF.   The current approach for organizations such as CDISC and HL7 is to create interfaces between systems. This will be a slow burner, I predict. There are so many hurdles in the way. It requires active cooperation from both sides – EDC providers may be fully onboard, but, I am not so sure about most EHR providers.  

We may see a company emerge (maybe this is what some former PhaseForward employees are up to – who knows!) develop an online Clinical Development Electronic Health Record System that is immediately EDC ready. Of course, with such a small deployment potential, I am not sure that we will see this appear at all sites in a global study, but for domestic application – say within a single country, inside research hospitals, it could work. I digress - back to the integration issue – yes, when we have standards, the feeds will start to happen, but not for many years.

5). Patient Recorded Information

ePro, Patient Diaries etc.   These are increasingly realistic, for the right sort of study for the accurate capture of patient data. If used appropriately, they can cut down the volume of data that needs to be transposed into EDC, and therefore source data verified.

On a side note, I am sure we will see a downloadable iPhone app that will make the current hardware/software dependent or basic PDA browser based systems seem old and tired.

The advantage of diary based systems is that instead of an investigator quizzing a patient, writing down the responses, transposing the responses etc. The information is captured ‘at source’ often considerably closer to the time of event.

Expanding from my previous post, I expect to see systems like PatientsLikeMe expand onto portable devices and act as an entry point for both patient identification and data entry. Internet enabled device pervasiveness will simply make this happen.

Conclusion

A lack of eSource has direct impact on the time it takes to lock data. With adaptive clinical trials executed by forward thinking sponsor companies, the point at which an adaption can occur corresponds directly with how long it takes for the sample size end-point significant datapoints to achieve a locked status. To simplify - the less eSource you have, the longer your studies will take. Forget a few days - we are talking wasted months.

Applying Social Networking system principles to eClinical

Social Networking software is one of these technologies that has just sneaked up over the last 5 or so years.  We have sites like Twitter, Facebook, Bebo, MySpace and LinkedIn all vying for marketshare attention.  Like all new business areas, segmentation is occurring with Bebo aimed towards kids, MySpace adults and celebrities etc.  Over time, I am sure we will see accepted leaders in the same way Google leads the search engine pack.

For a number of years, we have seen small clusters of information sharing sites aimed at specific disease types.  PatientsLikeMe was one of the early success stories with an initial focus on ALS (Lou Gehrig's disease).  It is beautifully written with a user interface specifically designed to be accessible and approachable. [It is also open source, and written on the modern new platform - Ruby on Rails].  Sharing experiences, discussing issues with other sufferers, and learning of potential therapies when dealing with chronic debilitating diseases is an ideal target for the social networking concept.  For something like ALS, where patients are geographical dispersed, a site like this can act as a electronic 'drop in centre' providing a level of support not possible with any other mediums.

An example on PatientsLikeMe is one of the focus communities that supports Devic’s neuromyelitis optica.  This is an incredibly rare disease, often confused with MS that only effects a few thousand people in the world today. As of July 2009, the site has a community of 140 suffers, more than 5 times the population of the largest published clinical study.  The home page for this site is a lesson on how to provide a site that the target users will return to;

Without having to login, or register, the site show a list of treatments and symptoms that have been shared by the 140 patients that have previously registered.  What better way to encourage mutual sharing and involvement!

A replacement for Traditional Clinical Trials?

To a degree PatientsLikeMe and other health focused social network sites are providing an open source equivalent to clinical trials.   Clinical trials are highly structure organized affairs, tightly bound by regulations and rules.  In this way, they are similar to a formal software development approach where specifications are prepared, code developed and testing carried out.  Compare this to open source community approach in software development. Thousands of individuals all  contribute, self regulating, to create products that many would argue are as good, or even better than their commercial cousins. With social networking health sites you have a very large community all contributing on an ongoing basis both in terms of the raw data, but also, the analysis of the raw data to determine trends.

With a typical clinical trial, a subject with be interviewed by an investigator, measurements taken, and information recorded into a system according to pre-prepared protocol.  Together with the terms of the protocol, the investigator will be in a position to make an assessment as to whether the subject is sufficient compliant for the corresponding data to be considered part in the overall data set going forward.  With a patient community approach, the compliance with an fixed regime, or protocol will be limited.   The sample size must therefore be considerably larger in order to make any kind of assessment based on the results.  Also, there are considerable challenges in determining the degree of compliance. Statisticians would most likely struggle to offer up a statistically safe trend based on the variability of the surrounding conditions, many of which would potentially be un-recorded.

How about as an entry point?

Nothing new here of course - PatientsLikeMe are already partnering with commercial companies, but, potentially a more open approach would be to create bridge between the community and the commercial world by leverage standards to create an opt in gateway.  Sponsor companies could publish a set of entry criteria and schedules for clinical trials through a protocol standard language (CDISC PRD ?) push approach. Community health portals could utilize these criteria to determine a filtered list of individuals that comply and then give them the opportunity to opt into the published study.

Maybe it would be a shame if the 'big bad' commercial world started to spoil these not-for-profit community led efforts.  I am not sure.  I think both could benefit.  Patients that suffer these diseases are actively looking for therapies and help.  If the drug companies are better able to target and develop these study therapies, then surely it is a win/win.

eClinical Vendors - Build or Merge

Over the last year, we have seen a spat of eClinical companies either swallowing up smaller players, or, merging.   The recent procurement of eTrials brought this to mind. From a capability checklist perspective, this looks good.   As a sponsor, instead of going to many companies for many systems, they can be procured from a single source. If problems occur with a system, there is only one number to call.  If the systems aren't speaking, then no finger pointing - the single vendor is responsible.

However, with today's eClinical systems demands, I believe some challenges exist particularly related to scalability and metadata management.

In former days, where Client/Server solutions were the norm, and, the cost of setup and integration was just an understood overhead of working with eClinical systems, mergers made a lot of sense.  It often took months to fully configure a platform, carry out validation and adjust the configurable settings to make it work as required.  If the system came from 2 sources versus 1, it didn't really mater as much, as it was expected that it would take considerably time and cost to make everything work anyway.

The technology landscape today is very different.  In a Software or Platform as a Service model, sponsor companies are looking for a number of capabilities that can potentially conflict with the abilities of separate products coming from different sources.

How long should it take to setup an eClinical product so that it is ready to be configured to support a clinical study?  3 months, 1 month, 1 week, 1 day?     In a capacity managed, multi-tenanted Platform as a Service environment anything more than a day is too long. Organizations are increasingly expecting instant capacity support, and immediate responsiveness.

Let us imagine a hypothetical case study. 

A company has developed a nice multi-tiered Java EE web app, with horizontal scalability - things are looking good...  the operational configuration and management of the platform has reached a point where an instant 'on' is a reality for clients. There is a single location for user and metadata management. Security is looking solid, a development plan is in place to expand and extend the core software... and then bang...

... a merger occurs...  The companies IT Hosting group are handed an entirely new platform to co-exist with the current platform... the R&D group inherit a new technology.  This one is .Net, on a different database, using a different app and web server.   Both applications are large and complex - a complete re-design is out of the question.  So, what next?

Well,the first thing that might happen is that the systems are made to appear integrated.  A common User Interface, often called a 'portal' is created that gives the impression that the independent systems are operating closely together.

Next, an interface.   Integration would be nice, but, as the two products were designed in isolation, they don't share a common architecture or platform stack, so, an interface is the only viable solution.  Both products are not standards interface based so hard-coded bi-directional feeds as put in place.

Ok, so they share an access point, and, they share data, but, do they provide an effective solution?

CDISC & Standards

CDISC may offer potential to organizations that choose to buy rather than build.   One of the key objectives of the CDISC standards are to achieve cross system interoperability.   If the disparate systems are capable of offering CDISC ODM integration - both data and metadata - then the challenge may be less steep. 

We are also seeing the emergence of Web Services, often leveraging CDISC standards, that in theory will allow the non programmed bi-directional interchange of data and metadata between supporting systems.

The question though ultimately is, will the resulting systems deliver solutions in the way the customer wants a solution, or, will the the customers end up with having to work in a convoluted way, because that is they way the systems came together, and that is the way they need to work.

I will not expand on this commentary for now as I am interested in hearing experiences from those that have faced, or are facing these integration challenges.

Too busy for EDC?

 

When things are busy, people tend to become more re-active than pro-active.  Instead of working on a list of tasks that are set out according to a pre-prepared plan, tasks are tackled as they come through the door.   Email for example is often a curse in this regard.   The priority of work tackled is often related to the order that an email appears in your inbox, rather than any real priority for the underlying work.

With a blog, it is necessary to proactively go and monitor feedback.  If you don't go to the blog site, you don't recognize the feedback.

EDC can be impacted negatively by this.   Imagine you have very busy site personnel - not hard... Typical online EDC systems contain workflow that often demand pro-active interaction.   A Monitor might for example create a query on data that requires a response from site personnel.   The turnaround time for the query is dependent on the frequency that the site personnel access the EDC system.   For smaller sites without dedicated study personnel the likelihood of an investigator simply not finding the time to login to an online system to look for pending actions.

ECO made a suggestion a few months back regarding the potential use of RSS Feeds - a simple concept with a technical name - whereby such feeds could be used to notify a participant of a particular action.  For example, if a Monitor raised a Query, then this would trigger - at the discretion of the recipient, an RSS / email notification that the action is exists.  The recipient would then click on the link, login, and carry out the action.

So, questions -

Would this cause security concerns?  If the feed included information that might be considered patient confidential - then yes. However, the actual requirement to include any real details in the communication is limited.   The message could simply say 'You have a new Query on Subject xyz'.

How would this be implemented?   Certainly a system or study switch from the sponsor to make it available.  Next, it should be optional at the user level - when a user signs up, they could opt in for correspondence when things required action, by type.  After that - it would be relatively automatic to the users.   As far as the technology - relatively easy for web based systems provided outbound communications don't cause security issues.

Would this be called an 'RSS Feed' - Why should it?  It tends to be teenagers that recognize the term.  On the other hand, ask an investigator if they would like to be notified of actions they need to be performing - that is understandable.

So, a question for blog readers.   Anyone aware of an EDC system that is already doing this?   Maybe this is already an existing feature in Medidata Rave or PhaseForward Inform products?    Like to share some experiences?