In his blog “Human review of in silico predictions of toxicity”, Principal Scientist, Alex Cayley, outlined the reasons why expert review is essential when using in silico tools to assess toxicity, particularly in the ICH M7 use case where expert review of two complementary in silico systems is required [1].

In this post we discuss how consideration of the user’s journey led to Lhasa tackling the ICH M7 use case with a brand-new automated workflow, now available to Derek and Sarah users in Nexus 2.3. I’ll also show some worked examples of results from the new system.

Expert Review: here are your “raw materials”…

Derek Nexus (an expert tool) and Sarah Nexus (a statistical tool) are Lhasa’s solution for ICH M7 hazard assessments. Even the best in silico models will be limited by the data and methodology used to build them, so an in-depth understanding of results is extremely important for those tasked with expert review. With this in mind, Derek and Sarah predictions were designed to show the user detailed information to support the results.

This information can then be picked out by the user to support their own, human conclusion. Essentially, Derek and Sarah predictions give the raw materials for the expert to build their own conclusions and ultimately convert in silico outputs into business-critical decisions.

…so, what about the “synthesis”?

To continue our Chemistry analogy, we can say that expert review is like a synthetic route; you are combining the raw materials – Derek and Sarah outputs – to get to a final product – the overall conclusion for mutagenicity. In chemical synthesis different routes using the same starting materials can lead to different products; similarly, in expert review you can follow different arguments to get to different conclusions.

Also, much like synthetic processes, some expert review arguments are far simpler and easier than others. For example, some of the more difficult predictive scenarios arise when Derek and Sarah return conflicting results. Often the way out of this situation is to use supporting information from one in silico tool to overturn the conclusion of the other. For users who are new to ICH M7 expert review, identifying the correct information from the in silico output can be tricky.

The below graph illustrates various predictive situations that can arise in ICH M7 Expert Review, showing the effort needed to reach a final conclusion, plus the general level of occurrence of each situation.

As software vendors, Lhasa provides users with the raw materials by providing Derek and Sarah. In addition to this, Lhasa is an educational charity with a goal to promote the use of computer-aided reasoning in the chemical industry. In recent years a large part of this objective has included training sessions, events, and webinars to guide our members on how to successfully “synthesise” their ICH M7 expert review.

What if Lhasa could guide users through the Expert Review process within the Nexus platform?

This idea was sparked through close collaboration with our members. As the Scientific community became more familiar with in silico software and the expert review process, it became clear that there were further opportunities for automation in the ICH M7 in silico workflow.

So, after further conversations with members and a large development project, Nexus 2.3 now provides automated expert review arguments alongside Derek and Sarah predictions. These automated arguments guide the user by identifying the relevant pieces of supporting information within Nexus and considering the links between Derek and Sarah. Arguments are presented in the Nexus interface; the user can choose which arguments they would like to use in their review and conclude an in silico Overall Call.

Worked Examples of Expert Review Arguments

The best way to introduce the new Nexus expert review functionality and to conclude this piece is by showing some worked examples. Below shows three different compounds with a summary of results from Nexus 2.3.

Example 1 

Derek Result: Plausible, 1 toxicophore

Sarah Result: Positive, 1 toxicophore

Expert review argument(s) and argument outcome: 

Argument 36 “Toxicophore identified by both systems which cannot be adequately negated”Use of this argument would lead to a POSITIVE in silico Overall Call.

Explanation: 

Argument 36 says “Both Sarah Nexus and Derek Nexus have made a positive prediction for the query compound. The predictions are adequate and cannot be overruled.”Moreover, both systems have identified the same toxicophore, which increases our confidence in the conclusion.

Example 2

Derek Result: Inactive, no toxicophores

Sarah Result: Positive, 1 toxicophore

Expert review argument(s) and argument outcome: 

Argument 18 “Toxicophore identified by Sarah Nexus is not causative of training set examples”Use of this argument would lead to a NEGATIVE in silico Overall Call.

Explanation: 

Argument 18 says “The positive prediction produced by Sarah Nexus as a result of the identification of a positive hypothesis is negated following removal of positive training set examples which contain additional and unrelated toxicophores.”Many of the positive training set examples from Sarah are anthracenes and alkylating agents, so can be disregarded.

Example 3

Derek Result: Equivocal, 1 toxicophore

Sarah Result: Positive, 2 toxicophores

Expert review argument(s) and argument outcome: 

Argument 1 “Ames test cannot adequately assess hazard of query compound”Use of this argument would lead to an INCONCLUSIVE in silico Overall Calland…Argument 36 “Toxicophore identified by both systems which cannot be adequately negated”Use of this argument would lead to a POSITIVE in silico Overall Call

Explanation: 

Argument 1 says “The Ames test is not appropriate to measure the hazard of the compound class to which the query compound belongs (as described in the alert comments of alert 315 in Derek Nexus). As a result, an accurate in silico prediction cannot be derived.”The Derek alert commentary also provides a literature reference for this assertion [2].However, if the user wants to treat the compound as positive without consideration of the solvent effects, they could choose argument 36 instead.

 

An aside: How often do Derek and Sarah conflict?

As described earlier, difficult situations in expert review will often be the result of conflicting results in Derek and Sarah. However, these results are usually rare, at least in comparison to the occasions where Derek and Sarah agree. To show this we have generated ICH M7 predictions for the Vitic Intermediates dataset (version 2017.1.0) – a Lhasa curated dataset featuring member donated Ames data for production intermediates. The graph below shows the experimental results returned for these compounds, grouped by Derek and Sarah results.

This illustrates that Derek and Sarah agree with each other, returning Positive/Positive or Negative/Negative, a large majority of the time. In Nexus 2.3, in addition to your Derek and Sarah results you receive an automated expert review argument. We also recorded the frequency of arguments returned for compounds in the same Vitic Intermediates 2017.1.0 dataset. As expected from the results illustrated above, a very large majority of arguments returned were either arguments 36 or 37:

This supports the assertion that most predictive situations are easy to resolve just using just the in silico results. However, in those rarer, more complex situations, the other arguments available will provide guidance for users so they can come to the correct overall conclusion.

Want to find out more? Download the slides or listen to the recording of the webinar: ‘Aiding And Standardising Expert Review Under ICH M7’.

 

References

[1] ICH M7 Guideline for Genotoxic Impurity Assessment.

[2] Amberg et al. (2015) Do carboxylic/sulfonic acid halides really present a mutagenic and carcinogenic risk as impurities in final drug products? Org. Process Res. Dev. 2015, 19, 11, 1495–1506. https://doi/10.1021/acs.oprd.5b00106