Overview
Documents editing, tracking of changes, and workflows.
Information about Editing and Tracking Change
The mapping tool supports a variety of actions to edit map records, add notes, assign advice, etc. All of the "map objects" (see Domain Model) are audited, meaning all changes to these objects are recorded with each commit. Thus assignment of a new code, movement through the worfklow, addition of advice, and any other changes are recorded. All prior states of map records can be searched and retrieved as needed.
Tracking Changes
There is no log of changes, per se, but each prior state of a map object can be viewed and read. In the "mapped concept view" in the application, the prior history of a map record is shown, with the last state of each map record for a particular concept (within the focus project) being displayed. Currently, there is no interface for seeing all of the incremental changes to a map record, though something like that would be fairly straightforward (but probably too fine-grained to be useful).
If editing causes any undesirable changes, admin tools can be developed to recover any prior state of a map record.
Note, prior states of objects not audited cannot be recovered. This is most important when considering changes in terminology content due to loading new versions, processing deltas (e.g. "drip feed") and tree position recomputation. This is intentional to improve performance and limit the number of archived states. Terminology objects include version information so auditing a change in version would cause an entire copy of the terminology to be recorded in the audit trails - this is the main reason they are not audited.
Origin ids
Map records undergo a complex workflow in which individual specialists edit their own copies of the record and submit them for comparison. When conflicts are detected, new records owned by the corresponding lead are created and at that time, all current states for all the records are available for review by the lead. A mechanism is used to connect the specialists' records to the lead record, called "origin ids". When a record is created from another record (or collection of records), the origin ids of that record and that record's id itself are added as origin ids of the later record. What this means is that for any workflow, records retain information about the prior records that were used ultimately to generate them. Thus the chain of "provenance" is recorded and can be searched.
At the moment, there are no services to query or use origin ids in sophisticated ways, however the tracking of what records lead to what other records is faithfully preserved and can be reported on. Consider the following chain of events
- A new concept C1 enters the workflow.
- Two specialists each edit this concept leading to two map records M1 and M2.
- Upon comparison, suppose there is a conflict. This results in a lead-assigned map record M3 that has M1 and M2 as origins.
- Later, the lead resolves the conflict and the record gets submitted for publication.
- In this event, the specialists' records are removed (M1 and M2) and the lead's conflict resolution record (M3) gets a workflow status change to PUBLISHED.
- Now, say in a future editing cycle an error is discovered in the lead-resolved record and another lead edits this record on the "fix error" path.
- In this event a 4th record M4 is created and assigned to the editor and gets origin ids M1, M2, and M3.
- Upon being completed, the record is submitted for review where another record M5 is created and assigned to the lead reviewing the error correction. this record will have M1, M2, M3, and M4 as origin ids.
- Any subsequent changes will repeat the prior few steps, adding additional origins. In this way, the complete history of the editing of this case can be tracked back and all information about workflow and changes can be recovered.
Workflow Concepts
Following are the standard "workflow paths" that a record can follow while it is being edited.
- Non-legacy path
- For unmapped, in scope concepts where we have no information about starting records
- Legacy/Semi-automated path - UNIMPLEMENTED
- For unmapped, in scope concepts where we have some information about starting records
- Fix error path
- For previously mapped and marked as PUBLISHED or READY_FOR_PUBLICATION where an error in target code, advice, or other information has been identified
- QA path
- For previously mapped and marked as PUBLISHED or READY_FOR_PUBLICATION that meet some QA criteria.
- Consensus management path - UNIMPLEMENTED
- For non-legacy or legacy path cases that need consensus review because they are particularly complicated.
Workflow Tracking Record
This domain object represents the complete known state of the workflow, including information about:
- Assigned user count
- Assigned users and workflow statuses
- Concept terminologyId, terminology, version, and default preferred name
- Map project id
- Map records
Workflow path
Summary workflow status - an aggregation of statuses of the underlying map records.
Workflow Actions
The workflow model is built around actions that affect the various workflows in a variety of ways.
ASSIGN_FROM_SCRATCH
- Create and assign a record when no records for it exist yet.
- ASSIGN_FROM_INITIAL_RECORD
- Assign a record based on an existing record (e.g. for when correcting an error)
- UNASSIGN
- Return assigned work to the pool
- START_EDITING
- Indicate that editing has started on a record
- SAVE_FOR_LATER
- Indicate that changes are in progress on a record.
- FINISH_EDITING
- Finalize work on a record.
- PUBLISH
- Send a finished record to publication
- CANCEL
- TBD
- CREATE_RECORD - ??
- Used by workflows that need to create records that are not yet assigned (e.g. QA records)
Workflow Paths
The mapping tool supports a number of different workflows that have different actions and state transition diagrams. The currently supported (and planned) workflows include:
- NON_LEGACY_PATH
- Used when dual independent review is desired and there is no candidate starting point for the map.
- LEGACY_PATH
- Used when dual independent review is desired and there is a candidate starting point for the map.
- FIX_ERROR_PATH
- Used for fixing errors among published or ready for publication records.
- REVIEW
- Used when single review is desired and there may or may not be a starting point for the map.
- QA_PATH
- Used for addressing reported QA errors
- CONSENSUS_PATH
- Used for "consensus management" process - similar to non legacy path.
- Used for "consensus management" process - similar to non legacy path.
Workflow Status
Some workflow status values are reused by different paths and some are specific to certain workflow paths.
CONFLICT_DETECTED - when a conflict between two EDITING_DONE records is detected, the EDITING_DONE records are set to this.
CONFLICT_IN_PROGRESS - when a lead has started editing a conflict but not yet finished
CONFLICT_NEW - created when a conflict is detected
CONFLICT_RESOLVED - when a conflict is resolved but has not yet been sent to publication
CONSENSUS_IN_PROGRESS - unimplemented
CONSENSUS_NEEDED - unimplemented
CONSENSUS_NEW - unimplemented
EDITING_DONE - when a specialist clicks "Finished" on the editing screen
EDITING_IN_PROGRESS - when a specialist clicks "edit"
NEW - in case we want to create placeholder record
PUBLISHED - published record
QA_IN_PROGRESS -
QA_NEEDED
QA_NEW - created when QA is put into the workflow
QA_RESOLVED
READY_FOR_PUBLICATION - when an edited/resolved record is sent back for publiction, the terminal status for editing
REVIEW_IN_PROGRESS
REVIEW_NEEDED
REVIEW_NEW
REVIEW_RESOLVED
REVISION - when a PUBLISHED record is being changed, it gets changed to this status- READY_FOR_PUBLICATION - when comparison finds two specialists' records are equivalent, or when a lead resolves a conflict.
Workflow Path Documentation
The supported workflow paths are described in detail here.
Non-legacy Workflow Path
TBD: state transition diagram
Non-legacy workflow statuses, actions, and results:
| Workflow Status | Action | Result | Notes |
|---|---|---|---|
| EDITING_IN_PROGRESS **Initial state | SAVE_FOR_LATER | Record retains the EDITING_IN_PROGRESS status. | |
| EDITING_IN_PROGRESS |
Valid combinations of map record statuses for a concept in this workflow
| Status 1 | Status 2 | Status 3 | Explanation |
|---|---|---|---|
| EDITING_IN_PROGRESS | EDITING_IN_PROGRESS | n/a | Two specialists are both still editing records. |
| EDITING_IN_PROGRESS | EDITING_DONE | n/a | One specialist has finished editing the record and the other is still in progress. |
| EDITING_DONE | EDITING_DONE | n/a | Exists only temporarily until conflict analysis occurs, at which point either a conflict is detected and the status changes, or a conflict is not detected and these records are removed. |
| CONFLICT_DETECTED | CONFLICT_DETECTED |
TODO: consider abberant worfklows (like specialist or lead quitting project)
Workflow action | Map records that exist at that time |
Map record is assigned to specialist 1 (via “claim” or “assign” mechanisms)
| Record created with status=NEW |
Map record is assigned to specialist 2 (via “claim” or “assign” mechanisms) | Second record with status=NEW |
Specialist 1 clicks to “edit” map record | Record for specialist 1 has status changed to EDITING_IN_PROGRESS |
Specialist 1 clicks “Finsihed” after editing map record | Record for specialist 1 has status changed to EDITING_DONE |
#3 and #4 repeated for Specialist 2 | Record for specialist 2 first has status changed to EDITING_IN_PROGRESS then EDITING_DONE |
Conflict detection algorithm runs on a schedule | IF conflict detected, set workflow status of all EDITING_DONE records to CONFLICT_DETECTED.
|
Conflict exists and lead clicks to “claim” the conflict. | Create new empty map record with status=CONFLICT_IN_PROGRESS and owner=lead and set origin ids to the originIds + ids of all of the CONFLICT_DETECTED records. |
Lead clicks to “edit” the conflict | No change |
Lead clicks “finished” after editing the map record | Record with CONFLICT_IN_PROGRESS gets changed to READY_FOR_PUBLICATION and the two starting records are removed. |
Publication is performed | Records with READY_FOR_PUBLICATION get changed to PUBLISHED |
UNASSIGN behavior for Non-Legacy Workflow Path
Workflow Action | Existing Records | Action |
|---|---|---|
| Specialist clicks "Return" on the Assigned Work tab of the assignedList widget | Map record owned by the specialist with workflow status in (NEW, EDITING_IN_PROGRESS, EDITING_DONE) | Delete the map record. No further action. |
| A specialist is removed from a Map Record by a Lead or Admin | CASE 1: The specialist's map record has workflow status in (NEW, EDITING_IN_PROGRESS, or EDITING_DONE) CASE 2: Two map records exist and both records are marked CONFLICT_DETECTED CASE 3: Three map records exist, two of which are marked CONFLICT_DETECTED and one is marked CONFLICT_IN_PROGRESS | CASE 1: Delete the map record owned by the specialist. No further action. CASE 2: Delete the map record owned by the specialist. Change the workflow status of the remaining record to EDITING_DONE CASE 3: Delete the map record owned by the specialist. Change the workflow status of the CONFLICT_DETECTED record to EDITING_DONE. Delete the CONFLICT_IN_PROGRESS record, as no conflict remains. |
| Lead clicks "Return" on the Assigned Conflicts tab of the assignedList widget OR a lead is removed from a conflict record by another Lead or Admin | Two map records with status 'CONFLICT_DETECTED', one map record (owned by Lead) with status 'CONFLICT_IN_PROGRESS' | Delete the CONFLICT_IN_PROGRESS record. No further action. |
Legacy Workflow Path
TBD
Consensus Workflow Path
TBD
Fix Error Workflow Path
Workflow step | Map records that exist at that time |
| An error in a published record is identified and a specialist or lead finds it using the Map Record View. | A single map record exists already with status in (PUBLISHED, READY_FOR_PUBLICATION) |
A specialist or lead clicks the "Edit" link/button for the map record and a new record is created and assigned to that specialist or lead | Record created with status=EDITING_IN_PROGRESS based on the previous record, owner=specialist or lead, initial map record is updated with status=REVIEW |
Specialist or lead clicks “Finished”, "Save for later", or "Cancel" after editing map record | If "Finished", record for specialist 1 has status changed to READY_FOR_PUBLICATION and initial map record is removed. If "Cancel" -> no action |
Publication is performed | Records with READY_FOR_PUBLICATION get changed to PUBLISHED |
Drip Feed Workflow Path -
Go over this with Brian to make sure we're handling things correctly. First thoughts are displayed below.
Proposal: do not clear TrackingRecords on computeWorkflow.
Add field to trackingRecord (w/enum) to track InScope, OutofScope. etc, typos blah blah.
Out of Scope | Retired | No longer exists | |
|---|---|---|---|
No map records | 1 | 1 | 1 |
| Specialist Editing | Alert | Alert | Alert, 3 |
| Resolving Conflict | Alert, 4 | Alert, 4 | Alert, 4 |
| Publication Ready | 2 | 2 | 2 |
(1) Remove tracking record for concept.
(2) Handle at release time
(ALERT) Display alert of current state of concept when a user tries to edit
(3) Disable link to SNOMEDCT browser (and show msg)
(4) Provide lead a means to "jettison/flush" all map records in this case.
QA C
| Publication Ready | Delete | Delete |
riteria Workflow Path
Workflow step | Map records that exist at that time |
| A QA condition is identified in a published record and a specialist or lead finds it using the QA Widget. | A single map record exists already with status in (PUBLISHED, READY_FOR_PUBLICATION) |
A specialist or lead clicks the "Edit" link/button for the map record and a new record is created and assigned to that specialist or lead | Record created with status=EDITING_IN_PROGRESS based on the previous record, owner=specialist or lead, initial map record is updated with status=REVIEW |
Specialist or lead clicks “Finished”, "Save for later", or "Cancel" after editing map record | If "Finished", record for specialist 1 has status changed to READY_FOR_PUBLICATION and initial map record is removed. If "Cancel" -> no action |
Publication is performed | Records with READY_FOR_PUBLICATION get changed to PUBLISHED |
References/Links