Given the updates to the OWL approach (with the imminent addition of the OWL Axiom and OWL Ontology refsets), this discussion should become obsolete, as the old PERL script becomes redundant, and the new SNOMED OWL Toolkit is already open-sourced:
TRAG to consider the implications of the new scripts being compiled by the Modelling AG
Last time we all agreed to 1) Wait for the document to be completed by the MAG which defines exactly what SNOMED CT should look like in OWL 2) Ensure that both the old PERL and the new Python scripts work completely and document all caveats 3) Ensure when we open source them both, we clearly call out what kind of SNOMED packages each does/doesn't work with (eg) International + extensions, but not derivatives?
Harold Solbrig to provide an update on when the MAG the document to be available.
Most NRC's (Australia, USA, Sweden, etc) were agreed that there was no longer any need for this Member Release period, as they only ever use the Member Release for preliminary internal validation, etc. However, Guillermo Reynoso expressed concerns that the removal of this phase would present a barrier to their support of members such as Netherlands, Uruguay, etc - and also with their translation of the Spanish edition.
The UK also need to confirm whether or not this would present them with a problem, given their release cycle timelines...
TRAG to discuss whether or not there are any new pro's/con's to removing the Member Release period?
Could we, for example, use the new Alpha release stage to remove the Member Release period (in order to gain an extra month of authoring)? This has resulted in us now having 5 testing phases to each International Edition:
On-going testing during the authoring cycle, both through front end validation, plus weekly issue resolution using the advanced notice provided by the Daily Build.
Member Release period
We now have a well refined process, which has resulted in our having removed the issues from the Member Release period. This is because we are now catching most issues in the Alpha stage, instead of in the Beta stage, and therefore the Beta is now as it should be - as close to Production level as possible.
If we have a clean Member release period for say, 2 more release cycles, can we consider the Member Release period obsolete?
In order to make this plan work, we would need to all agree the following:
We would need full, consistent cooperation of members in terms of allocating sufficient resource to implement thorough Alpha and Beta testing.
We would need to ensure enough time between Alpha and Beta phases to ensure full feedback allowed.
We would need to triage any Beta issues effectively to ensure that the release remains stable for all but critical issues found in the Beta period.
No-one had any problem with removing the Member Release period - even Chris Morris confirmed that the UKTC only take the release from the Production status onwards (31st Jan / 31st July), and therefore removing the Member Release period would have no impact for them
Andrew Atkinson to discuss with Guillermo Reynoso, who is the only outstanding opponent to this proposal, and see if we can work out how to ensure that they can still meet their translation commitments...
Depending on outcome of strategy discussions with Rory we may decide it's better to just leave this one to expire naturally as part of the introduction of Continuous Delivery (as that will inherently remove the Member release period)?
These strategic discussions have now confirmed the need to move to Continuous Delivery, and therefore this issue should be able to be closed down in lieu of the necessary removal of the Member release period inherent with this new approach.
Is everyone happy to close this down?
Structure of the TRAG repository
We've refined the TRAG repo in Confluence to break it up into "Open Discussions" and "Resolved Discussions", both due to the unexpected way in which Confluence managed the auto-closure of discussions, and also to allow quick and easy access to the relevant open discussions as we go forward and add more and more discussions to the repo.
The MAG just today accepted the choice that's been made to make end users' lives easier by maintaining associations to the most relevant active concept(s) for any given inactive concept. This decision was taken in order to avoid a snowballing snapshot file by allowing the reuse of refset rows. That is, keeping the same UUID and modifying the target.
The following file has been included in the International Release package for some time now, but rarely changes from release to release:
It would therefore make far more sense to publish it on Confluence instead (along with the Technical Guide doc) so that people can access it whenever they need to.
Hopefully you have already discussed with your various stakeholders to see if this move would cause anyone any issues? If no issues raised we'll plan to move it from the July 2018 release onwards...
Everyone is on board with this change, and can foresee no problems whatsoever.
Andrew Atkinson to therefore implement in the relevant International Edition
As planned, this was removed from the July 2018 International Edition - The Technical Guide Exemplars document has now been moved from the International Edition release package to a Confluence page. This page can be found as part of the ICD-10 Mapping Technical Guide (see Appendix B), which is hosted here: http://snomed.org/icd10map
Was everyone happy with the implementation, or are there any improvements that people would like to see?
Everyone was very happy with the implementation, and therefore this discussion topic can be closed down.
In January 2017 we removed the OWL conversion script from the International Edition package, and have since been publishing it alongside the International Edition in each Release.
However, as this script is now stable and self-maintaining (wrt release dates, etc) there should be no need to actually include the physical file anymore - instead we can streamline the release further by simply posting a link to the script in either the Release Notes or the Release wording in MLDS.
We should be able to now remove this OWL script from the International Edition release package entirely now, due to it no longer being relevant in the new world of OWL Axiom files.
Andrew Atkinson TO COMMUNICATE THIS OUT AND REMOVE IT AS OF THE RELEASE IN WHICH WE'RE GOING TO FIRST INCLUDE THE NEW OWL Toolkit (July 2018??)
The TRAG would like to see a link to the new SNOMED OWL Toolkit included in the International Release package:
This was all incorporated into the July 2018 International Edition - was everyone happy with the implementation, or are there any improvements that people would like to see? NO, EVERYONE HAPPY TO CLOSE THIS DOWN
Thanks to the UKTC for identifying and raising this issue - it has helped us to further refine our Validation service.
The question for the TRAG is: are there any other similar scenarios that we should attempt to pre-empt by adding validation to cover them, before they occur?
All agreed this is a very difficult issue to identify - can't use length, and no hard and fast rules. So perhaps the only rule we could apply is that all records in the TextDefinition file must be a Definition?
October 2018 - Any suggestions for improvement?
If not then we'll close it down, as the UKTC has been informed of plans to resolve it, the content fix was implemented in the July release, and the RVF improvement ticket has been raised (RVF-273).
TRAG to discuss and agree on categorisation of the Alpha/Beta/Member release feedback issues.
TRAG to suggest these categories, and provide many examples of each level so that we can then document and communicate out the proposed categories, and plan when to implement depending on feedback received.
Segregate into technical and clinical issues
Prioritise only Legal issues, critical patient safety issues, and major technical issues.
Continue to communicate through the JIRA tickets, so there's a good audit trail
Provide access to the JIRA tickets not just through Release notes at the end of the Release cycle, but also mid-cycle so people can see what we're discussing/agreeing...
Everyone agreed this can be closed down due to the fact that a) everyone is happy with the way in which the Alpha/Beta feedback is prioritised (no releases have gone out without any feedback being addressed), and b) This will naturally become obsolete when we move to more frequent releases
I think substantially this is the discussion we had today about modularisation, specifically the thing we can action in the short term about ensuring rules are declared for derivative module concepts to be defined on the new module and any metadata (refset concepts for example) go on that module.
Then the other part to it is analysing the violations of these rules that currently exist and if/how/when to fix them.
Can we agree that we can progress this item soon? I think it represents the low hanging parts of the module discussion we had today.
Dion then commented on the TRAG discussion page:
Having re-read this I now disagree with myself I think lateralisable body structure is core. Anyway I is probably not a controversial issue whether to include it or not anyway - it should be included.
Yong's response was:
Sorry, I am at the FHIR meeting as well. It is an important topic and would be very helpful if we can progress.
My view is that derivatives should be on different modules. Currently, the module dependency refset only has three active entries. One entry is about the dependency between the core module and module component module. The other two are related to ICD10 mapping module. We discussed to use this refset for generating OWL ontology imports statement. So, we need to avoid ontology imports for derivatives, e.g. most refsets.
If we are not going to distinguish derivatives from core content in dependency refset, we might need to look at specifying imports in the OWL ontology refset. I will cover this in the MAG meeting this afternoon.
To be discussed in October 2018
Dion McMurtrie agreed with the consensus that everyone should move toward distributing all files included in the International Edition, with the caveat that we then drive forward the discussions of modularising the international content and removing the less nationally-applicable modules (such as the GB + US language refsets etc) from the international Edition packages. This needs to be taken off line and discussed with the various relevant stakeholders as part of the ongoing Modularisation discussions.
Dion McMurtrie therefore agreed that he was happy with the conclusion and that we can close this discussion down...
This was a query raised in the Member Forum in April 2018 - Suzy introduced the topic on behalf of the requesting member.
Andrew to talk to Linda about what she needs...
Suzy to provide more feedback to the MF on regular updates to the Release AG
Andrew Atkinson to create a new page for Linda, with "TRAG Decisions" listed. This will be kept in line with all future TRAG meetings, and will therefore provide one location for people to search when looking for final decisions made by the TRAG.
This is part of the wider Drugs and Substances improvements that are currently taking place. Other than the obvious content updates, these technical changes are those which will be likely to have the highest impact on those within our AG.
We need to discuss the plan and ensure that we have answered all of the possible questions in advance, in order that we have a workable plan with no unwanted surprises over the next few release cycles.
As a starting point, we should discuss the following:
July 2018 - initial OWL refsets introduced Jan 2019 - included in the Release package: a) Stated Relationship file b) the partial OWL axiom refset including all description logic features that cannot be represented in the stated relationship file. The Extended OWL refset file will be available on demand. July 2019 - the stated relationship file will be replaced by the complete OWL Axiom refset file. The stated relationship file will NOT be included in the international release; however, it may still be available on request to support migration to the OWL Axiom refset.
2. The communications required to ensure that ALL impacted parties are completely informed of the Schedule, and the changes that they may need to make in order to transition cleanly to the new format.
3. The technical changes that we need to make to the Release package itself, in order to support the planned schedule.
For example, when we "replace" the Stated Relationship file in July 2019, do we remove the file from the release package immediately (in Jan 2020 once everyone has had a chance to run the inactivation file through their systems), or do we take the more measured approach of inactivating all records and leaving the inactivated file in the package for, say, 2 years, and then planning to deprecate the Stated Relationship file by July 2021?
Further, should we be deprecating the file itself at all, or can we see any other (valid) use for the Stated Relationship file (obviously not just repurposing it for a completely different use!)?
Harold Solbrig to talk to Yong and others in the MAG about his proposals for future proofing against the possibility of having multiple ontologies referenced, prefixed axioms, etc.
Some opposition to reverting back to having the OWL file on-demand for Jan 2019 - need to discuss through with Kai in tomorrow's session - preference is to release both Stated Rel's + the "addtiional" info only in the OWL files - as with July 2018 release. Is this the current intention?
Also, has the decision already been made to NOT create a full history back to 2002 (or 2011 at least)? Sounds like most extensions will do it anyway, so maybe we should?
Discussion on whether or not to go back and re-represent the content all the way back to 2002 in the new complete OWL file:
Prevents the need of new tooling providers to create support for the ols Stated Rel way of doing things
If the International Edition doesn't go all the way back then the Extensions are restricted to not doinh it either, if the international Edition does then the Extensions have a choice.
Ability to go back through history and analyse prevent modelling decisions (if errors come up in future), even for those authors who haven't heard of Stated Rel's because they've now been deprecated for several years.
Cost involved in creating the pure historical view
If the extensions have a choice as to whether or not to go back, then interoperability could be impacted - better to enforce going back if the international edition does.
Need to address the issue of some implmentations having both Stated Rel + OWL Axioms in the same full files going forward.
Uncertain use cases for most implementers
This discussion needs further input in order to enable us to reach an informed conclusion. The relevant internal and external stakeholders (NRC's such as Australia) will take this away and come back with the results of feasibility studies and estimates as to how long the necessary work would take to complete..... a decision must then be made well in advance of the January 2019 International Edition, in order to ensure that we agree on the correct approach before creating the initial Alpha release in November...
Due to it coming to the end of its useful life, SNOMED International would like to propose planning for the deprecation of the CTV3 Identifier simple map (that currently resides in the RF2 International Edition package) as of the January 2020 International Edition.
Some Member countries have already identified the reduction of the effectiveness of the product, and have already put plans in place to withdraw support for the CTV3 Identifiers from 2020 onwards.
The TRAG therefore need to discuss whether or not there are any apparent problems with the proposed deprecation, and if so how they can be mitigated.
We must also discuss the most effective method to pro-actively communicate out announcements to the community to warn them of the upcoming changes, in order to ensure that everyone who may still be using the Identifiers has plenty of notice in order to be able to make the necessary arrangements well in advance.
Finally, we will need to decide on the best method for extricating it from the package, in order to ensure the smoothest transition for all parties, whilst remaining in line with the RF2 standards and best practices.
AAT CHECKED THE PREVIOUS IMPLEMENTATIONS OF DEPRECATION OF BOTH ICD-9-CM and RT Identifiers, AND AS THOUGHT BOTH WERE IN THE CORE MODULE, AND REMAINED IN THE CORE MODULE IN THE STATIC PACKAGES - SO ANY ISSUES WITH DOING THIS AGAIN???
So the plan would be to follow the same deprecation process as we did with ICD-9-CM (ie)
move all of the content to a Static Package in Jan 2020, and inactivate all of the content
publish reason for inactivation in historical associations
CREATE A STATIC PACKAGE FOR ICD-9CM BASED ON THE JANUARY 2016 MAP FILES AND PUBLISH ON MLDS (and link through from Confluence link as well). ALSO LIFT THE ICD-9CM SPECIFIC DOCS FORM THE JAN 2016 RELEASE NOTES TO INCLUDE IN THE PACKAGE.
Date of the files should be before the July 2016 edition (so say 1st May), in order to prevent inference of dependency on theJuly 2016 Internationaledition
Inside the files should be all Internatioanle edition file structures, all empty except for:
Delta ComplexMap file needs to be cleared down (headers only), as no change in the content since the Jan 2016 files, so no Delta
Full and Snapshot ComplexMap files exactly as they were in Jan 2016 release (including the effectiveTimes)
ModuleDependency file needs to be blank, as ICD-9-CM was in the core module (not in its own like ICD-10 is), and therefore the dependency of the core (and therefore the ICD-9-CM content) module on the January 2016 edition is already called out in the Jan 2016 ModuelDependency file, and therefore persists for the static package too.
Date of all of the files inside the package should be the new date (1st May)
But all effectiveTimes remain as they were in January
Leave refsetDescriptor records as they are in the International edition
RELEASE Notes Should describe all of the thinking we went through when creating this package, why the moduleDependency file remains blank, and why we’ve wiped the Delta, etc (see above)
AND ALSO WITH THE RT IDENTIFIER REFSET DEPRECATION:
RT Identifier Refset deprecation:
Just wanted to confirm what we just agreed in the call, in terms of the legal risk profile, and the operational plan for the July 2017 International Edition:
1. Due to the fact that by its very inclusion in the International Edition package we are continuing to licence the SNOMED Identifier refset, we would be significantly increasing the risk of legal liability due to unauthorised use of the unlicensed content.
We are all agreed, therefore, that the best practice approach is to remove the SNOMED Identifier refset in its entirety, rather than merely inactivating the content.
AAT will communicate the decision to the members who raised the issue of our contravention of the RF2 standard, in order to explain to them why this is preferable to increasing our legal liability. - DONE
AAT will then proceed to remove the SNOMED Identifier refset from the July 2017 International Edition release package, and create the distinct static package as per the previous comms.
2. We need additional comms around the July 2017 release, in addition to the usual Release Notes wording, in order to confirm what is happening and the rationale behind it.
AAT will send draft notifications to KKU and request that they are posted in all other forums (website, etc) - everyone here will be copied in order to allow refinement of the message where required.
3. To re-iterate what was discussed on the previous call, Legal counsel confirmed that from a legal perspective, he doesn’t consider that it’s either necessary (or even advisable) for us to send CAP any further communications on this matter. Legal counsel is confident that the informal discussions that we’ve already had with them (in order to remind them about what they need to do), are sufficient to cover our legal obligations, given that the licence is theirs and not SNOMED International's. Therefore we no longer need to send a formal letter to CAP.
Andrew Atkinson to begin formal deprecation process as everyone was in favour (only discussions are still on the exact process to use to deprecate).
To be reviewed in detail by everyone, and all feedback to be discussed in the meetings. AS OF OCTOBER 2017MOST PEOPLE STILL NEEDED TIME TO REVIEW THE DOC - Andrew Atkinson INFORMED EVERYONE THAT THIS DOCUMENT WILL BE ENFORCED AS OF THE JAN 2018 RELEASE AND THEREFORE WE NEED REVIEWS COMPLETED ASAP... so now need to check if reviews still outstanding, or if all complete and signed off??
AAT to add in to the Release Versioning spec that the time stamp is UTC
AAT to add the trailing "Z" into the Release packaging conventions to bring us in line with ISO8601
AAT to add new discussion point in order to completely review the actual file naming conventions. An example, would be to add into the Delta/Full/Snapshot element the dependent release that the Delta is from (eg) "_Delta-20170131_" etc. AAT to discuss with Linda/David. Or we hold a zero byte file in the Delta folder containing this info as this is less intrusive to existing users. Then publish the proposal, and everyone would then take this to their relevant stakeholders for feedback before the next meeting in October. If this is ratified, we would then update the TIG accordingly.
AAT to add in a statement to the section 4 (Release package configuration) to state that multiple Delta's are not advised within the same package.
AAT to add in appendix with human readable version of the folder structure. Done - see section 7
ALSO we need to discuss Dion's point from the joint AG where he talked about nailing down the rules for derivative modules... -
IN ADDITION, we should discuss both the File Naming convention recommendations in the Requirements section (at the top of the page), PLUS Dion's suggestions further below (with the diagram included).
Dion McMurtrie to discuss syndication options for MLDS in October 2018 - see hwat they've done (using Atom) and discuss with Rory as to what we can do. Suzy would be interested is this as well from an MS persepctive. UK also interested. This shouldn't hold up the publishing of the document. Discussions to continue in parallel with the creation of this document...
URI Specs to be updated and aligned accordingly - Reuben Daniels to assist
EVERYONE TO REVIEW TONIGHT AND SIGN OFF TOMRORROW
Everyone is now happy with the current version, therefore Andrew Atkinson to publish - we can then start refining it as we use it.
Andrew Atkinson to therefore agree all of the relevant changes that will be required as a result of this document internally in SNOMED International, and publish the document accordingly. FIRST POINT WAS THEREFORE TO SEND IT TO LINDA/DAVID ON 16/10/2018 for first review.....
Andrew Atkinson to then ensure all relevant sections incorporated in the TIG (or at least linked through)
Once complete, the document should be published and opened up to anyone to view.
All agreed that whilst this is a large topic, we should start somewhere, and get at least some of the quick wins in (then request the change to content via the CMAG):
Check where the progress with the namespace metadata has got to - can we progress this?
Code systems (and versions) of the map baselines
Common strings such as boiler plate licence text etc
Description of use cases for the various refsets (using the text definition of the Refset concetps themselves) - either json or markdown representation of multiple pieces of info within the same field.
TRAG agreed that this should be incorporated into the discussions with the continuous delivery, in order that we can plan the changes here in line with the transition to more frequent releases.
Discussions around the future strategy for SNOMED CT have included the potential for adding new statuses for content.
In particular, many people have suggested that problems are created for those either mapping or translating from content that's still "in development". If (as is often the case) they use Daily Builds etc as input data, they can often get tripped up by content which is created but then withdrawn before it's versioned and officially released. It would be extremely useful to those users to have access to traceability data describing the reasons behind why they were removed, in order to support accurate mapping/translation.
In another use case, there's the possibility that content needs to be formally withdrawn from the International Edition AFTER it's been officially released. This would be the case if, for example, content has unintentionally been published that breaks the RF2 paradigm, or contravenes licensing laws, etc. In this case mere inactivation is not sufficient, the content instead needs to be completely withdrawn from the releases and sometimes even from history.
The TRAG needs to discuss all of this and be ready with recommendations if these proposals are taken forward.
undo a historical issue (that break RF2 paradigm, etc) but don't want to pretend it never happened - in this case we should use the Negative Delta approach - but only used in EXTREME circumstances
Legal contraventions - in this case we should use the Negative Delta approach - but only used in EXTREME circumstances
Dead on arrival components - it should be okay to have these, and have them openly dead on arrival and therefore inactive to not map to them etc. However it's useful to be able to see these (even though they'd been activated + inactivated within the same release cycle) - so for those people who need to map/translate etc DURING the release cycle, they have to rely on the Daily Build and use live data still in development. Therefore if those concepts disappear by the time of the International Edition it causes problems for those maps/translations already including those concepts.
Therefore the best answer is for us to move to having 2x Daily Builds - the existing one + a separate true Daily Builds - where each Daily Build is built relative to the previous Day, and NOT to the previous Published release. This new Daily Build could then be properly relied upon by mapping and translation projects.
Can we align this with the transition to the more Frequent Releases?
This is a subject that would be helpful to include Jim in the discussions, as he has some definite opinions on how to improve the metadata in this area.
Some suggestions would be to make more detailed information available for authors to describe their reasons for inactivation (especially in those areas where currently they are forced to use inactivation reason codes that aren't completely representative of the reasons in that instance).
Once we've collated the feedback from the revised comms processes that we've implemented over the past year (in the items above), we'll incorporate that into the final version and discuss with the SNOMED International Executive Lead for Communications (Kelly Kuru), to ensure that it is aligned with the new overall Communication strategy. Once complete, the Release Management comms plan will be transferred to Confluence and opened up for everyone to view.
We have publicised the Release Management confluence portal to both NRC's and the end users to get people to sign up as and when they require the information. Do we know of anyone still not getting the information they need?
I've left the previous items up (from the July 2017 International Edition) because there are no examples yet from the Jan 2018 editing cycle - so please take a look and provide feedback on whether or not this is useful, and how it can be improved.
Suzy Roy will discuss the end use cases of her users with them and come back to use with feedback on the practical uses of SNOMED CT and any improvements that we can make, etc
We may now also need to add a new section in here wrt the comms for the TRAG, so that this is standardised and agreed with the community? Or is it outside of the scope for the Release Communication Plan? This was felt to be out of scope, and should this be restricted only to communication related to actual releases of products.
Everyone is now happy with the current version, therefore Andrew Atkinson to publish - we can then start refining it as we use it.
Andrew Atkinson to therefore agree all of the relevant changes that will be required as a result of this document internally in SNOMED International, and publish the document accordingly.
FIRST POINT IS TO MIGRATE THE DOCUMENT FROM WORD TO CONFLUENCE, AND THEN SEND IT TO LINDA/DAVID for first review.....
Andrew Atkinson to then ensure all relevant sections incorporated in the TIG (or at least linked through)
Once complete, the document should be published and opened up to anyone to view.
Harold would like to introduce this topic for discussion...
Language refset conflicts are not yet resolved - Linda has been discussing this in terms of how to merge Language refsets or dictate whether or not one should override the other in cases of multiple language refsets - in the UK they combine them all into one but this is not ideal either. In translation situations they use the EN-US preferred term as the default where there is no translated term in the local language. Perhaps we need to do a survey on the members and who's using what how.
Suzy Roy (or Harold Solbrig) to get Olivier's initial analysis and come back to us on what worked and what didn't, and we can take it from there.
Suzy would like to ask Matt Cordell if he can share his ppt from his CMAG extensions comparison project.
Matt Cordellwill distribute this to everyone for review before the April 2019 meeting.....
TRAG to continue the ongoing discussions. The SEP refsets have now been deployed, so are we happy that this has resolved the majority of the issues, or do we still need to push for a full re-factoring of the anatomy content asap?
No change expected for the Jan 2018 release. Part-Of relationships will once again become defining once we have the OWL Refset changes in place. So they will become stated relationships rather than additional.
The MAG are proposing that they are perhaps going to put these non-defining relationships into a new Relationships file completely, in order to allow them to be viewed, but to also ensure that it's clear that they are distinct from normal, defining relationships (the MAG discussions are still ongoing on this)........
Question is whether or not this will happen before they become inherently redundant as a result of the Anatomy remodelling.
ORSI CONFIRMED THAT A LOT OF END USERS ARE USING THESE ADDITIONAL RELATIONSHIPS IN THEIR IMPLEMENTATIONS, AND THEREFORE IT WOULD BE PAINFUL FOR THEM TO CHANGE THEIR EXISTING SYSTEMS IN ORDER TO USE A NEW, SEPARATE RELATIONSHIP FILE.
Harold confirmed no furhter progress made on this as yet by the MAG, SO WE JUST NEED TO KEEP AN EYE ON THIS MAG TOPIC AND RESPOND ACCORDINGLY - Harold Solbrig to update on progress in April 2019 TRAG meeting....
Dion McMurtrie completed the Alpha release - did anyone have chance to review it? (I haven't had any requests for access to the remainder of the package)
The subject of Modularisation needs to be discussed between the various AG's who are considering the topic, before we can proceed with the Release-specific sections.
We need to discuss any red flags expected for the major areas of the strategy:
Members who want to abstain from monthly releases, and therefore need to use delta's with mulitple effective times contained within.
Also need to consider if we continue to hold the date against the root concept - works perhaps still for 12 monthly releases, but not necessarily for continuous delivery daily!
THIS NOW BECOMES CRITICAL TO THE STRATEGIC DIRECTION WE DISCUSSED IN TERMS OF MODULARISING OUR CONTENT, AND IMPROVING THE WAY THAT THE MDRS WORKS, IN ORDER TO ALLOW RANGES OF DEPENDENCIES. THIS WILL ALLOW THE "UNIT" OF RELEASE TO BE REFINED ACCORDING TO THE RELEVANT USE CASES.
Understand the Use cases thoroughly, and refine the proposal doc to provide people with more real information - Dion McMurtrie TO PROVIDE THESE USE CASES FOR Andrew Atkinson TO DOCUMENT
Does the POC allow for concepts to be contained within multiple modules? NO - BUT DION CAN'T THINK OF ANY CONCRETE EXAMPLES WHERE THIS WOULD BE NECESSARY
What about cross module dependencies? Michael Lawley's idea on having a separate Module purely for managing module dependencies
IN THE FINAL PROPOSAL, WE NEED TO CREATE A NESTED MDRS TO MANAGE THE INTER-MODULE DEPENDENCIES (as per Michael's comments)
NEED TO PROVIDE GOOD EXAMPLES AND WHITE PAPERS OF THE USE CASES FOR MODULARISATION IN ORDER TO ENGAGE OTHERS...
AFTER SIGNIFICANT DISCUSSION AND CONSIDERATION, THERE ARE NO VALID USE CASES LEFT FOR MODULARISATION. IT CAUSES A LOT OF WORK AND POTENTIAL CONFUSION, WITHOUT ANY TANGIBLE BENEFIT.
THE PERCEIVED BENEFIT OF HAVING A WAY TO REDUCE THE SIZE/SCOPE FO RELEASE PACKAGES IS BOTH a) invalid (due to everyone's experience of being unable to successfully do anything useful with any small part of SNOMED!), and b) easily answered by tooling that using the ECL to identify sub-sections of SNOMED to pull out for research purposes, etc.
THEREFORE AS OF APRIL 2018 THE FEEDBACK FOR RORY AND THE STRATEGY TEAM WAS THAT MODULARISATION SHOULD NOT BE IMPLEMENTED UNLESS A VALID USE CASE CAN BE IDENTIFIED.
HOWEVER, KNOWING THE HISTORY OF THIS ISSUE, THIS WASN'T NECESSARILY GOING TO BE THE FINAL WORD ON THE MATTER, SO IS EVERYONE STILL SURE THAT THERE ARE NO KNOWN USE CASES FOR MODULARISATION?? (eg) linking modules to use cases, as Keith was talking about with Suicide risk assessment in Saturday's meeting,etc??
This topic came up several times again during other discussions in the October 2019 meetings, and it was clear that people had not yet given up on the idea of Modularisation - we therefore need to discuss further in April 2019....
See linked discussions above as well on MDRS, etc.....
We need to continue discussions on this on-going item, in light of the strategic meeting before the conference. In addition we now have new members with additional experience, and we have also now lived with the more stable International Edition release process for the past couple of years.
Last time we discussed this everyone thought it a good idea in principle, but were concerned that we are not yet in a position to deliver the same level of quality on a daily basis than as on a monthly basis (due to the current gap in our manual/automated testing). Therefore we were going to discuss this once we had further progressed our automated testing - however as the new working group for the RVF service will testify this is a slow process, and therefore it may not be possible to wait for this to be completed in its entirety.
We have identified several additional potential issues with moving to Continuous Delivery, which we should consider before proposing a solution:
Perceived quality issues:
There would be no time for Alpha or Beta Releases - so all Members would have to be comfortable with issues in Production for until the next interim release
All issues that normally get tidied up as part of the normal Content Authoring cycle will become public - they will get fixed quickly but in the meantime there may be an impact to the reputation of the quality of SNOMED CT.
Roll up Releases:
The 6 monthly delta releases would need to be relative to the prior 6 month release, and therefore named as such somehow (ie) we would need to somehow make it explicit as to which previous release the delta is a differential to.
Other possibility is that each month is the same interim release, and then every 6 months we also release the Delta's relative to the priori 6 monthly release, in addition to the usual monthly release. In this case we would need to reserve the 31st Jan + 31st July effectiveTime's /package naming for the 6 monthly roll up releases, so that the users who want to remain on 6 monthly schedule would remain unaffected.
The other option is to have no roll up releases at all, thus releasing a stand-alone package every day/week/month, depending on the agreed frequency. The issue with this approach though is that anyone using the Delta files (rather than Snapshot) for uploads would need to keep up with the continuous schedule.
UPDATE FROM THE EVOLUTION WORKSHOP:
Allows people to choose whether or not the users take one release every 6 months, or frequent monthly releases...
Derivative maps wouldn't be a huge issue as just release them whenever we had a chance, dependent on whichever edition
One of the plus points are that when we're still at 6 monthly releases, if the vendors miss a release its a big deal, whereas if they miss monthly releases then they have a smaller impact
One drawback is for the non english speaking members, who need to keep up with translations - shouldn't really have an impact if they keep up with each smaller release.
Could be painful for translations when a monthly release happens to contain a drop of a huge project like Drugs or something...
What about interoperability issues, with some people taking each monthly release, and others still waiting for every 6 months? ADHA believe this hasn't caused a huge problem for them, just an addition to the existing problem even with 6 monthly releases...
Also need to implement the metadata for identifying which dependent release each Delta is relative to...
Refsets aren't too much work to keep up to date - however Mapping is a different ball game - this can take some time
Maps that are still inherent in the int Edition (ICD-0 ICD-11 etc) are potentially problematic, and the workflow would need to be carefully worked out...
If your projects happen to drop in-between the normal 6 monthly releases, then someone who might have taken Jan and July still, might miss out on the important big releases that happen in April and November!
Also quality might be an issue - need to have the automated testing completely airtight before we move to continuous delivery! Thereafter you would run all major validation at input stage and ensure authors only ever promote to MAIN when everything perfectly clean. Then we run Daily builds with automated release validation every night, and provides a RAG status on release issues every morning. Then by the end of the month, we publish the last Green Daily build!
Andrew Atkinsonto continue to feed all of this into the continued internal discussions on whether or not moving to Continuous Delivery is feasible, and if so plan what the timelines would look like:
Move to Monthly Releases before we go to full continuous delivery - yes, everyone agreed
How do we best automate all of the validation? - Best thing is to make the RVF the central source of truth for all International validation. -Therefore NRC's like Australia will promote all International related content to the core RVF, and only retain and run validation that is local to themselves. -This would mean that whenever they identify a new issue, they can simply promote the new test up to us and we can run it and replicate the issue for ourselves, and therefore fix it quickly. -It will also share the burden of maintaining the validation rules.
Question: can we do any automation for Modelling issues? ECL? New validation using the editorial rules in the new templates as a basis for automating modelling QA? ECL the best bet - plus MRCM doing well so far - can we extend this? (Australia so far only implemented modelling validation by writing manual rules for known issues)
What's the impact of multiple effectiveTimes in Delta files? Should be negligible, Australia and US already implemented with no effect to users (despite initial complaints!)
Creation of a bespoke Delta using a new tool - Delta at the International level is very simple, but at the Extension level is much more complex due to all of the dependencies, etc. This could also become more involved when we modularise... -Australia intended to build this as well, but it never happended because no one ever requested it in the end! -The other issue was the traditional issue of never knowing (in a machine readbale way within the Delta file itself) what the Delta file is a Delta from (ie) is it a delta from the Jan 2014 release, or the July 2016 release, etc. -So there as a lot of discussion over whether or not they should create roll up Delta's, or provide the service - but in the end they found that only a few people were actually using Delta's, and those were the people who knoew what they were doing already, and so nothing was ever required! -So we need to decide whether or not this is useful... -We also need to be wary of the fact that there are two different things to be relative to - so you can have a Delta to a release, or a Delta to a date in time, and they can be very different things. -Suzy has always released a delta with multiple effectiveTimes in it (due to the Edition) and no-one has any issues of this ever. -If we remove the Delta files completely everyone would definitely need to provide a Service to download bespoke Delta's (both International and local Extension level) - AT THE SAME TIME WE SHOULD FIX THE ISSUE OF LACK OF METADATA PROVIDED FOR WHAT THE BASELINE OF THE DELTA IS -For local extensions this service does get a lot more complex than for International, as they need a range of Delta dates PER MODULE, as they have a lot more going on than just the International Edition - so the service would need to be a) clever enough to correctly get the relevant depednencies from all sources, plus b) Validate that the resulting Delta is correct and valid - provide a checksum of some kind (needs to be identified). -SNOMED INTERNATIONAL TO CREATE A SMALL, TARGETED SURVEY TO QUESTION WHETHER OR NOT THERE WOULD BE ANY IMPACT TO ANYONE TO PROVIDING A DELTA SERVICE INSTEAD OF DELTA FILES... Everyone will happily disseminate this to their users and get responses asap... -SUZY ISN'T ALLOWED TO ASK HERSELF, SO WE SHOULD CREATE AN ONLINE VERSION THEN SHE CAN ISSUE A NOTE TO ALL HER AFFILIATES SAYING THAT SNOMED INTERNATIONAL HAVE REQUESTED THEM TO FILL OUT OUR SURVEY (as otherwise if it comes from the NLM thy have a lot of loopholes to jump through!)
Release Notes automation - simple, just attach notes metadata to each change in MAIN then export on Release
Question: Is it worth starting off with a trial using just Content Requests monthly, and then bring everything else in line once happy? NO! Everyone feels strongly like there would be no benefit to this whatsoever, as the majority of urgent cases in CRS are to do with getting an ID to use in refsets etc before the next 6 monthly release, and as this has already been mitigated due to the new tooling providing those ID's early, there's no benefit in moving to CRS early. Small risk in moving to monthly at all, so better off just moving everything at once to prevent a) confusion for users b) confusion in message about continuous delivery, and c) overhead for SNOMED managing 2 different delivery schedules during the pilot
Question: What are the next steps that we need to consider to help move this forward? Central RVF service, communication with community (survey etc)
Question: Is everyone happy with the new plan to remove the Delta files from the RF2 packages completely, and just provide the Delta service to create Delta's on the fly? YES
Question: How can we get a survey out to as many implementers as possible in order to ask a lot of these questions and get the
Question: How do we manage translations? (including the Spanish release) - How do we cope with the likelihood that one month could have only 50 changes, and the next month 50,000 (Drugs project, etc)? - no impact, as should allow for incremental translations - just need to not set expectations with your users that you stay one month behind the International Edition! Just need to decouple the translation release schedule from the International Edition schedule. ARTURO woudl prefer the Spanish edition to also move the Monthly (or even more frequent) releases, but he fully understands the natural latency required for translated Editions, and so understands even if we went to monthly we can't keep up with the monthly content changes
Question: How do we manage extensions? Again need to decouple them - MDRS will naturally get a lot bigger - also the versioning process internally currently takes a long tiem and a lot of effort for each upgrade to the new International Edition...
Question: How do we manage derivatives? Just keep them decoupled from the International Edition release schedule, and do not set false expectations by promising to keep them closely up to date with monthly International Releases!
Question: How do we manage maps? So again there is a natural latency here where we can't keep up to date with monthly releases. WE ALSO NEED TO DEFINE WHAT AN ACCEPTABLE UNIT OF RELEASE IS FOR EACH TYPE F CONTENT CHANGE (so what our concept of "DONE" is for each type of change) - FOR EXAMPLE SOME CONCEPTS SHOULD NOT BE RELEASED UNTILT HE RELEVANT ICD-10 MAP COULD BE CREATED AND PUBLISHED AT THE SMAE TIEM. OTHERS COULD BE RELEASED NO PROBLEM AND WAIT FOR 6 MONTHS FOR THE RELATED MAPS...
WE ALSO NEED TO CAREFULLY DEFINE AND COMMUNICATE OUT WHAT THE SCOPE AND GOALS OF MOVING TO CONTINUOUS DELIVERY ARE - TO ENSURE THA WE MANAGE EVERYONE'S EXPECTATIONS. FOR EXAMPLE, WHAT IT DOESN'T MEAN IS THAT EVERYONE WILL GET THEIR CHANGE INTO SNOMED WITHIN 4 WEEKS, JUST BECAUSE WE'RE RELEASING MONTHLY!!!
Question: What questions would we like to ask the vendors and affiliates to a) Ensure we cover off all problems/potential issues, but b) do NOT put us in a position where they think that we might not go ahead with the plans despite their answers.... just wording the survey to ensure that they know we're going ahead, but just want to ensure there's no negative impact to them that might tweak our plans, and c) How much time do they need to adapt to the change for multiple effectiveTimes in the same Delta, and d) How do we promote the benefits? (responsiveness to changes with more frequent releases, improvement to quality with more frequent fixes, etc)
Andrew Atkinsonto create a survey to provide to everyone so that they can send out to all users and get feedback on the proposed changes (especially multiple effective Times in Delta files, and removal of Delta files - just a service now):
Andrew Atkinson to refine survey to ensure that it's accessible to those with more limited SNOMED knowledge/experience, as these are the preferable target market for the survey, given that the more advanced users will (or have already) speak up for themselves:
GDPR questions - verify with Terance whether or not we just need to provide a link to our data policy (https://www.iubenda.com/privacy-policy/46600952), or if we specifically need to ask the questions (of whether or not they're happy for us to store their data, etc) as questions in the survey? (check box) - If the latter, ask if we have standard legal wording I can use?
Small intro - description + pros/cons
Couple of fairly wide ranging questions as to whether or not they think they'll be impacted
If so, then either fill in the details here (conditional question in google forms) OR please just get in contact with your NRC to discuss
Avoid technical language for non native English speakers
Andrew Atkinson to verify final survey with SMT (Terance and Kelly in particular, from GDPR and comms perspective) to ensure in line with company strategy and verify whether or not they'd prefer this to be an SI survey or NRC surveys?
RVF has now been open sourced to allow people to contribute towards it more easily, so that Implementation issues can be reverse engineered into the assertions. All of the NRC validation systems should remain separate, in order to ensure as great a coverage across the board as possible.
However, it makes sense to ensure the critical tests are included in all systems, in order to ensure that if, say, one NRC doesn't have the capacity to run Alpha/Beta testing for a certain release, we don't miss critical checks out. We are working on this in the Working Group, and also in the RVF Improvement program, where we are including the DROOLS rules, etc. These are also being incorporated into the front end input validation for the SCA.
TRAG to therefore discuss taking the Implementation Load test forward, including the potential to incorporate key rules from NRC validation systems into the RVF. So we should discuss the tests that are specific to the Implementation of vendor and affiliate systems, in order that we can facilitate the best baseline for the RVF when agreeing the generic testing functionality in the Working Group.
Matt Cordell will promote some useful new ADHA specific rules to the RVF so we can improve the scope... report back in April 2019
Chris Morris to do the same - get the RVF up and running and then promote any missing rules that they run locally.... report back in April 2019
The short term proposal of precoordinating the numbers and measures as concepts (and therefore not changing the RF2 format) was generally well accepted, though there were concerns raised regarding the longevity of this approach, and whether or not this addresses the original target of the project (which was to allow a standardised approach across all extensions, instead of perpetuating distinct coding for different users). The other concern raised was that any solution needs to be implemented rapidly, as otherwise the various members will be forced to start/continue implementing their own solutions.
Peter G. Williams, therefore, will take this forward in the Modelling AG and further implementation. The functionality has been rolled in to the wider discussion of enhancing SNOMED’s DL capabilities. The Modelling AG is planning a targeted discussion on this in June 2017, and will then produce a document which would then be reviewed by the MAG at the October conference.This Proposal document will be shared when complete.
Last update from Peter was that the OWL Refset solution allows us to classify with concrete domains. The thing we’re still discussing, is how to represent that in the release. The currently most popular approach suggested is to create a 2nd inferred file which contains concrete values in the destination column, rather than SCTIDs. This allows them to be added without impact to the current approach i.e. ignore it if you don’t want to use them. The new file would only contain concrete values.
Harold to give an update on the MAG's plans?
October 2018 - Harold Solbrig to give an update on the MAG's plans? No further updates yet, check back in April 2019....
If extensions promote content via RF2 delta, we just need to retain all ID's, and only change the ModuleID and effectiveTime, and therefore it is all managed by effectiveTime.
If IHTSDO reject content this is also managed
The only issue then comes if IHTSDO want to change the FSN, then we ned a way to manage the change of the meaning of the concept without creating 2 FSN's - as then we need a feedback loop to ensure that it's also corrected at source in the extension as well as in the International edition.
TRAG to continue the discussion and come to a conclusion that will work for all.
Has this been answered in its entirety by Jim's new agreed approach? (link here to his final position)
Most people consider that Jim's approach covers this under most circumstances. We also need to ensure that we follow the approach listed to the left - so we should confirm all of this has been working in practice since April 2018, and if so close down.
Last meeting the TRAG proposed use cases for creating an actual service (with a user-friendly UI, etc) to enable people to load up their release packages and run them through the standard validation assertions.
Standardisation is the primary use case here - everyone agrees that there is a significant benefit to interoperability by ensuring that all RF2 packages are standard and conformant to the basic standards at least - and so this is a strong business case for the service.
We agreed that whilst we have the appetite to have one, this will be a long term goal - to get us started we should use the open sourced RVF as a basis to refine the rules.
We therefore setup a working group to decide a) What the scope/targets should be b) What technology platform would be most appropriate c) What the high level rules should be (packaging format, content etc) - Working Group: Generic Validation service
The good news is that we've now used the initial discussions we had as part of the working group to refine the requirements for the ongoing RVF improvement program. This is due to complete within the next few months, at which point the working group will meet again in order to begin the full gap analysis between the various streams of validation that we all have.
Liara also discussed validation with ADHA during the London conference - Dion do you have a quick update on where those discussions got up to?
Plan is to ensure that the generic service is flexible enough to fail gracefully if certain extensions don't contain some of the expected files, etc.
We should also provide a standard Manifest to show what files they should include wherever possible (even if blank)
We'll now take this forward with the working group, using the comprehensive list of current SI assertions as the baseline:
The Canadian NRC has asked Suzy to raise this proposal on their behalf, and so Suzy will present this at the next TRAG meeting:
1. Before sending the Refsets Release announcement if the refsets could be loaded in the SNOMED Browser, that would be great. This way, stakeholders would be able to browse the content from the tool they are used to. 2. Add to the announcement that the refsets can be browsed through the Browser as well as they can be accessed through MLDS. 3. Send the announcement.
This would require us to include the addition of all refsets to browser into our formal release processes, as so far they have been added as and when available, and on an informal basis (without comms, etc). So the question here is not whether or not we can do the process above, as this is very simple - it's more a question of whether or not we think having the refsets visible in the browser is a useful feature, and not misleading in any way?
Andrew Atkinson to check whether or not the refset function in the browser allows searching?
ANSWER - Yes, but in reverse, so you don't go into the Refset and search within the refset, instead you run a normal search for the concept, and then hit the Refset tab on the right, and this shows you all the refsets that this concept is part of. You can also then click on "Open maps for this concept" and get linked through to all maps that contain this concept in the Mapping Tool.
Andrew Atkinson to check also if the ECL expression functionality works in the refset section of the browser, as it does in the normal content?
ANSWER - Not at the moment - please submit a feature request or build it locally and submit the code for us to incorporate into the browser.
Ask Rory if there is a way around the fact that the resets like gp/fp are removed in the rf2 to json representation when the new international editions are loaded?
The Release AG agreed that, now that we have all of the refsets uploaded as part of the standard release process, it would be overkill to add the announcement of the browser update to each Production Refset release announcement, to confirm it will be updated in the browser as well (as this can now be taken as read that it will be done as part of the release process)
This approach was successfully implemented in order to resolve the issues found in the September 2017 US Edition - is everyone comfortable with using this approach for all future similar situations? If so we can document it as the accepted practice in these circumstances...
NO! Everyone is decidedly uncomfortable with this solution! In particular Keith Campbell, Michael Lawley and Guillermo are all vehemently opposed to changing history.
The consensus is that in the particular example of the US problem, we should have instead granted permission for the US to publish an update record in the International module, thus fixing the problem (though leaving the incorrect history in place). This would have been far preferable to changing history.
ACTION POINT FOR EVERYONE FOR OCTOBER 2018: (Dion McMurtrie, Matt Cordell, Orsolya Bali, Suzy Roy, Corey Smith, Harold Solbrig, Mikael Nyström, Chris Morris We therefore all need to come up with potential scenarios where going forward we may need to implement a similar solution to the Negative Delta, and send them to AAT. Once I've documented them all, we can then discuss again and agree on the correct approach in each place, then AAT will document all of these as standard, proportionate responses to each situation, and we will use these as guidelines in future. If we have issues come up that fall outside of these situations, we'll then come back to the group to discuss each one subjectively, and then add them back into the list of agreed solutions.
Preference now is to retain EVERYTHING in the Full file, regardless of errors - this is because the Full File should show the state at that point in time, even if it was an error! This is because there is not an error in the Full file, the Full file is accurately representing the error in the content/data at that time.
The problem here is that the tools are unable to cope with historical errors - so we perhaps need to update the tools to allow for these errors.
So we need the tools to be able to whitelist the errors, and honestly document the KNOWN ISSUES (preferably in a machine readable manner), so that everyone knows what the historical errors were.
The manner of this documentation is up for debate - perhaps we add it to a new refset? Then we could use something very similar in format to the Negative delta, but instead of it actually changing history retrospectively, we simply document them as known issues, and allowing people to deal with the information in their own extensions and systems in whatever way they feel is appropriate.
Only situation we can think of where we couldn't apply the above gentle response, would be copyright infringement - whereby if we discovered (several releases after the fact) that we had released content that was in direct infringement of copyright, then we would potentially have to revoke all releases since the issues occurred. However, this would raise a very interesting situation where patient safety might be compromised - as if we remove all historical content that contravened the copyright, then we run the risk of patient data being impacted, thus potentially adversely affecting decision support. This is simple to resolve when the problem is in the latest release (simply recall the release), but if found in a 5 year old release for example, it could be very problematic to recall 5 years' worth of content and change it!
October 2018 - Guillermo proposed a separate possibility, which is to introduce a new Status (eg) -1 whereby if you find this status in the latest snpashot you would just ignore it - this doesn't however address the use case where there is a legal contravention and you need to physically remove the content from the package - the use case where you would have something that contravenes RF2 paradigm, you can't use the RF2 format to correct something that is RF2 invalid! So this is unlikely to work...
If we're still all in agreement with this, then steps 1-5 above should all be documented and disseminated to get confirmation of approval from everyone??
Did everyone read through everything? Has anyone got any further scenarios that we can include in the documentation?
As most of you know, new OWL refset files are being planned for introduction in future International Edition releases - for those who haven't heard about this here is some background reading: SNOMED CT Logic Profile Enhancement
We need to agree on the best naming convention for the new OWL refset file, and also on the best location within the release package.
The current naming convention proposal is as follows:
As for the location, there are two choices - as it's technically a refset the obvious place is to put it in the Refset/Content folder.
However, there is also an argument for keeping the Owl Refset in the Terminology directory. This is because it's an axiom refset, and as such is distinctly different from any other reference set. In addition, the content of the Terminology directory is actually incomplete without it, due to the fact that it's the new format of stated relationships. Therefore there's a school of thought that suggests that we should organize the files by their meaning rather than their format.
In the Alpha release of the Drugs package, the following naming conventions were applied:
Please let me know if you have any questions before our meeting, once you've read through the info, as it would be great for everyone to start on the same page in order to have the most effective discussion on the day.
MICHAEL LAWLEY CAME TO THE OCTOBER 2018 TRAG MEETINGS AND WALKED THROUGH THE PROPOSAL FOR THE BENEFIT OF ANYONE WHO HASN'T BEEN ABLE TO GO THROUGH IT YET, OR WHO MIGHT HAVE QUESTIONS AFTER HAVING READ THROUGH IT....
We need to discuss and agree an approach that allows us to both express the correct moduledependencies + the new module composition (to express which modules comprise the Edition package, for URI purposes).
Michael Lawley has written up a proposal and shared this - there has been a lot of feedback on this proposal in the Confluence discussion - so in this meeting we should:
a) Ask Michael to walk through the proposal in person to ensure that everyone's on the same page
b) Answer the feedback (plus any new feedback)
c) Agree what the final proposal should be, and what are the next steps we need to take in order to get it signed off (design authority, etc?)
Michael, Dion and Reuben to create the Australian version as an example, in order to include that in Michael's updated version of the proposal document.
Linda and David came to the second session in October 2018 and asked for clarification on various points. Michael Lawley will update the proposal document and re-circulate for everyone's consideration, in order that we can then decide whether or not to take this forward...