Lab Importers

Lab importers are complex scripts that need to take into account a variety of use cases for lab test ordering and results processing.

Overview of laboratory investigation

Some clinical measurements can be performed directly by the clinician, examples being height and weight, and blood pressure.

Blood and urine measurements are usually referred-out to a medical diagnostic laboratory. Lab specimens (blood, urine etc.) will, in some cases, be collected by the provider and in other cases, a technician at the lab will collect the specimens. In Germany, collection at the point of care allows the praxis to attach pre-coded labels onto the probe specimen bottles or tubes.

Lab test results share, with other medical testing, the concepts of "ordering" doctor (provider), copy-to doctors, patient identification, date of testing, and some kind of result. While lab test results may deliver paragraphs of text and indeed entire reports, they more typically deliver numeric observations along with a "normal", "reference" or "therapeutic" range. Lab tests can be performed by sometimes-different analytical methods which may yield method-specific, non-identical results.

Tracking order status

GNUmed intends to support the tracking of "outgoing" requests. As at 0.9, a table has been developed to hold them (clin.lab_requests).

Commonly however, results will arrive which did not originate in GNUmed:

  1. Lab tests are, in some situations, ordered via paper forms.
  2. Lab tests are, in other cases (even in the same locale) ordered by electronic request.
  3. Results may concern patients unknown to – and tests that are unexpected by – any one praxis software:
    • out-of-praxis doctor may be referring a patient to a praxis doctor designated as "copy-to"
    • in-praxis doctor may, from in hospital, order tests without recording them in the praxis software
    • ordering or copy-to doctors may have been entered in error
  4. Results issuers (the labs) plus minus an intermediary (data broker) may use a variety of formats and encodings:
    • British Columbia, Canada
      • e.g. Excelleris XML-wrapped HL7 files (either through https scripting, or browser-based login, and saving, of an XML file)
    • Germany
      • encoding in the Labordatenträger (LDT) specification among the *DT protocols endorsed by the German physicians statutory association and briefly outlined here
    • Australia
      • unencoded PIT files supplying individual patient results as text blobs (thread here) and delivered by encrypted email

Further complicating matters, the same result may be issued more than once, when it formed part of a partial result or when one or more already-received results are followed by one or more corrections.

To help manage this, it is proposed that GNUmed auto-generate requests from not-previously-known-about incoming results. These will inform the praxis about live requests that are in process and which – if they originated elsewhere – may be redundant to ourselves order for the patient. These will also provide the means to distinguish first-time results from subsequent updates and/or corrected results. This is covered in further detail below, in resolving order status.

Importer basics

EMRs should:
  • be able to receive and process results without a prior record of the testing having been requested
  • create, from incoming results, receipts and
  • reconcile the receipts against any existing record of the requests /orders

Importers should:
  1. need to be signalled to process newly-received results, whether daisy-chained to the "retriever" process, or through a cron tab watching a directory, or by manual pointing
  2. need to evaluate each line (or message) in the source file, and ensure that
    • only valid lines and messages are accepted (see next section, Checking validity), and
    • those valid lines and messages that cannot be matched to a patient need to be flagged and dealt with
  3. should be able to be re-run ("reprocess") manually-resolved problem records

Matching should:
  • result in one of two situations. Either
    • rule-based criteria for "adequate matching" are met, in which case process the received information, or
    • rule-based criteria were not met, in which case leave these messages in clin.incoming_data_unmatched where they can be worked on by praxis admin who, on confirming that these results should be imported, can manually attach them to patients (or approve new patient creation).

Checking syntactical validity, and logging the importer's success or failure

It should be possible to detect
  1. file-level syntactical badness and
  2. message-level syntactical badness

File-level syntactical badness

Using Health Level 7 (HL7) as an example, files can be split into individual messages, each having as its message delimiter start block <Hex 0b> and end block <Hex 1c><Hex 0d> and content "foo" under the following construct:

<Hex 0b> foo <Hex 1c><Hex 0d>

At the first mismatch of start and end blocks the file can, by definition, be construed as "broken" requiring a rollback. An import error log message should identify the line at which the badness was noted.

Message-level syntactical badness

Message-level badness in HL7 could be defined based on expected (valid) message segment IDs and sequences and lengths. Generally if there is some invalidity in HL7 messages, it's only one or some, and not the entire batch, so we need to be able to import everything but the 'bad' message(s). Brokers tell us they have no issue with resetting one or more individual accessions once any 'badness' is resolved.

Message-level syntactical badness should result in
  • an import error log message in which is identified
    • the total number of syntactically bad messages and
    • the total number of syntactically non-bad messages
  • an inbox item bringing to attention the existence of bad messages
  • bad messages being left behind in the staging table
  • non-bad messages being normally imported

Logging badness

A log of the attempted import should be kept, it is suggested, in its own table and a virtual copy of this record should appear in an admin inbox. It is suggested that this might warrant signing, at least if any errors had been encountered.

An importer log schema is suggested here.

Overview of required importer logic

The basic idea is:

  • write incoming data into a table that makes no assumptions about whether the records are even going to be matchable
    • maybe this data has already been processed
    • maybe the lab goofed, and sent tons of information about patients who do not even correctly get their care 'here'
    • maybe the file got corrupted during its download or maybe it represents some haphazardly-clicked word-processed document
  • any obvious problems identified while trying to write the data into the initial table should be logged
  • incoming data should be sequentially "split" at the level of "patient entity", one incoming "patient message" per row
  • the data needs to be assessed as to whether it is recognizable
    • recognizability is achieved through efforts to "match" against existing patients and providers
      • it is helped by first populating "comparator columns" using information extracted from the raw data
      • an 'adequate' match on the patient and/or provider will be understood to "green-light" (approve) this data to be processed
  • even if the patient or provider "make sense" to be coming into this praxis, there is the possibility that
    • it may be inappropriately double-processed, or
    • it may contain updates which need to deprecate / overwrite what may have been previously received

British Columbia, Canada (data broker Excelleris, with sample message)

The most-structured results currently available within BC CA consist of file downloads containing XML-wrapped, HL7 "ORU" (Observation Result) messages having the form


MSH (Message Header, indicating the type of message),
PID (Patient Identification),
ORC (Common Order, indicating the lab's accession number and TestCode) and
OBR (Observation Request, containing lab and specimen and doctor admin information) segments, followed by zero to many
OBX (Observation/Result) segments and, optionally after OBR and OBX,
NTE (Notes and Comments) segments, whose position can follow any of OBR, OBX. It is not known at this time whether, in other implementations, there might be an ORC NTE.

Note that other implementations may not always supply (i.e. they could omit) the ORC… see this example.

The number of OBX segments will depend on whether the "test" that was requested was a single test or a "panel" of tests, such as "Electrolytes".

The OBX segment is used to transmit a single observation or observation fragment. It represents the smallest indivisible unit of a report. The principal mission of the segment is to carry information about observations in report messages. Whereas OBR gives general information about the order of the test, the OBX segment gives the specific, individual tests performed (OBX 003) and the specific results for each test (OBX 005). An OBX segment will appear for each specific observation (or "panel" component), so you can end up with multiple OBX for any one ORC OBR pair.

See below, section "resolve the original request" for a discussion of duplicate and updated message processing.

Fuller documentation of the source specifications must be obtained directly from Excelleris. This wiki topic limits itself to defining the field mappings, and the processing, that are essential to GNUmed.

MSH|^~\&|PATHL7|BCB|HTTPCLIENT|vendor1|20071101120533||ORU^R01|MDC20071101120533673|P|2.3|||ER|AL
PID||9012345678|||EXCELLERIS^BPATIENT||19430102|F|||||(604)658-2107
ORC|RE||07-9999999-PT-0|||||||||90909^MDCARE^BOB
OBR|1||07-9999999-PT-0|PT^INR||20071009092600|20071009092600|||||||20071009092600||90909^MDCARE^BOB||079999999||07|079999999|20071010002500||HAEM3|F|||90909^MDCARE^BOB
OBX|1|NM|6301-6^INR||2.5||2.0 - 3.0||||F|||20071009134500
ORC|RE||07-9999999-CLOZ-0|||||||||90909^MDCARE^BOB
OBR|2||07-9999999-CLOZ-0|CLOZ^Clozapine||20071009092600|20071009092600|||||||20071009092600||90909^MDCARE^BOB||079999999||07|079999999|20071010002500||REFER1|F|||90909^MDCARE^BOB
OBX|1|FT|X500^Referred Test||Sent to Provincial Toxicology Centre.\.br\Telephone: 604-707-2710||||||F|||20071009092700

Import messages and setup for both auto- and manual matching

Incoming data (messages) should be written into clin.incoming_data_unmatched – see v16 table specification pending v17

Each message should be XML-unwrapped (?) and written, one per row, into clin.incoming_data_unmatched with other columns populated as follows:

_unmatched column Incoming message segment field number (info) Comments
.external_data_id   hook for source-specific hl7.py script or HAPI routine or Mirth channel that was "run/called" and which can potentially be "re-run/re-called" after manual matching
.type   value 'HL7'
.data the raw data bytea column… keep vs strip XML?
.error_info   new column
.firstnames PID 005 (Patient Name:Last^First^Middle [as supplied to broker]) extract First^Middle and space-separate the component words
.lastnames PID 005 (Patient Name:Last^First^Middle [as supplied to broker]) extract 'Last'
.dob PID 007 (Date of Birth YYYYMMDD [null if unknown])  
.gender PID 008 (Sex / Gender of patient F/M/U-unknown) omit if 'U'
.postcode    
.other_info PID 002 (External patient ID / BC Personal Health Number [PHN] if known) +
PID 003 ( Addn Patient ID / Additional Patient Identifier – MRN nonunique) +
PID 004 (Alternate External Patient ID / e.g. Chart number) +
PID 013 (Home Phone / May be partial and irregularly formatted)
 
.request_id ORC 003 (Filler Order Number / Order Number ID of lab performing tests / Accession number-test code-tiebreaker)  
.requestor ORC 12 (ordering provider), OBR 028 (Result Copies To) see #AutoMatchProvider
.fk_identity_disambiguated   dem.identity.pk (auto-matched or manually matched)
v16 .fk_provider_disambiguated   fk to dem.staff.pk (auto-matched or manually matched)
.comment    

Auto-match…

The plan in GNUmed is

  1. if the patient can be identified, import the results
  2. if the patient cannot be identified but the ordering or copy-to providers are identifiable praxis staff, import the results and create the patient in the process

auto-match to an existing patient

Names are notoriously challenging to match…
  • mis-spellings
  • multiple last names (unmarried and married)
  • inconsistent use, by the patient, of first names / middle names / "nicknames"

GNUmed builds in support, for any one dem.identity (individual), multiple dem.names which can be tested-against. For some of the background on which the algorithm below was built, refer to this thread. For information on the cPatientSearcher_SQL(), see this one.

Auto-match criteria:

  • incoming Personal Health Number (which is unique in BC), combined with
    • a match on at least one among:
      • dob against against GNUmed dem.identity.dob
      • Patient Name: Last against GNUmed lastnames
  • incoming dob against GNUmed dem.identity.dob along with
    • Patient Name: Last
    • along with…
      • any first-2-characters of Patient Name:First^Middle against GNUmed firstnames
        • along with gender (where incoming gender is 'M' or 'F' but not 'U')
  • future: clin.lab_requestwill need to be nutted-out

resolution of the above:

  • if a unique match is achieved via any of the above, then update .fk_identity_disambiguated using dem.identity.pk
  • if multiple matches are achieved, then update .fk_patient_candidates integer[] with ?? Karsten
  • if no match is achieved… (leave alone or input value 0)?

Fields of interest from imported Excelleris-brokered HL7 messages:
HL7 field source notes GNUmed field
PID 002 External Patient ID / PHN number [null if unknown] dem.lnk_identity2ext_id where dem.enum_ext_id_types.name = 'PHN Personal Health Number' and dem.enum_ext_id_types.name = 'BC.CA_MSP'
PID 007 Date of Birth YYYYMMDD [null if unknown] dem.identity.dob
PID 005 Patient Name Last^First^Middle [as supplied] component 1(Last): dem.names.lastnames; components 2 (First) and 3 (Middle): first 2 characters of any among GNUmed firstnames
PID 008 Sex of patient F/M/U - unknown dem.identity.gender but ignore where incoming value is "U"

Targets to match against (from among existing GNUmed records):
table columns Notes on GNUmed
dem.lnk_identity2ext_id external_id keyed from id_identity <10-digit value representing patient's PHN>
dem.identity field: pk integer
dem.identity dob, gender use to_char(foo, 'YYYYMMDD') , upper(substring(foo for 1))
dem.names lastnames, firstnames use upper(foo)
dem.enum_ext_id_types issuer 'BC.CA_MSP'
dem.enum_ext_id_types name 'Personal Health Number'

auto-match to an in-praxis provider or – if patient is already known valid (matched within praxis) – resolve a provider

In GNUmed, all providers exist as persons in dem.identity having one or more names in dem.names and zero to many external IDs such as a regulatory "doctor number" in dem.lnk_identity2ext_id and finally a record in dem.staff. While the dem.staff_role is typically that of "doctor", other practitioner designations may be valid ordering providers and so matches should not require any one particular staff_role.

A match can be found by parsing provider codes and names from a concatenation of HL7 ORC 12 (Ordering Provider) || HL7 OBR 028 (Result Copies To) where each is formatted as BCMSP billing number (if known, else '00000')^LastName^FirstName^Middle Name against
  • dem.lnk_identity2ext_id of suitable external id type joined with dem.identity and where dem.identity.pk exists in dem.staff.fk_identity
  • or lab_request.fk_requestor ( future GNUmed functionality )

from which match to extract dem.staff.pk.

Where no match on provider was established but where the patient had been auto-matched or manually matched, the responsibility of review for the result needs to be assigned to
  • the patient's value for dem.identity.fk_primary_provider (the in-praxis primary doctor for this patient), or if there is none…
  • the result of SELECT value FROM cfg.v_cfg_opts_numeric WHERE option = 'patient.fallback_primary_provider' (if configured)

The latter will have been settable through the UI, via GNUmed menu > Preferences > EMR > Primary doctor ( which is the value of the pk in Dem.staff).

Where – on the other hand – the result had been matched to a provider within the praxis, but no patient, then the patient needs to be created and assigned that provider as their in-praxis primary. For this to be achievable, there must exist in the incoming record sufficient information to populate:

New patient creation will need:
table columns Notes on GNUmed
dem.identity field: pk integer --> gets auto-assigned
dem.identity dob, gender use to_char(foo, 'YYYYMMDD') , upper(substring(foo for 1))
dem.names lastnames, firstnames, id_identity, comment, id_identity is dem.identity.pk, comment like 'imported from org name'
dem.enum_ext_id_types issuer 'BC.CA_MSP'
dem.enum_ext_id_types name 'Personal Health Number'
dem.lnk_identity2ext_id external_id, id_identity, fk_origin <10-digit value representing patient's PHN>, dem.identity.pk, dem.enum_ext_id_types.pk where issuer, name…

On match… begin transaction

resolve the status of the orders in this message, and their associated results

Each MSH-initiated message, despite that it only ever concerns a single patient, may in any one download contain one or multiple ORC OBR test code order pairs, as well as zero or more OBX. The next MSH message on this same patient may contain all, some, or none of the previous ORC OBR order pairs, as well as some, none, or all of the earlier OBX segments. Therefore, it is not the MSH as a whole that is to be tracked.

Within each MSH message, it is the {ORC OBR} order pair that will be recognizably and uniquely identifiable each time it comes through:
  • Uniqueness is definable by the combination of Sending facility (MSH 004) and Filler order number (ORC 003)
  • Uniqueness is similarly definable by the combination of Sending facility (MSH 004) and Filler order number in OBR (OBR 003) as when both an ORC and an OBR are supplied, ORC 003 and OBR 003 must be the same.
  • Any increment of OBR 022 in (MSH 004, ORC 003, OBR 022) will require that any and all associated OBX which had been earlier written into clin.test_results must be overwritten, and new OBX results inserted and re-signed by a provider.

Each new (MSH 004, ORC 003) must therefore be checked against existing values among clin.lab_request and
  1. where it does not already exist, a new request must be appended and
  2. where it does already exist, check whether the incoming OBR 022 is newer
    1. if incoming OBR 022 is newer, delete all existing rows linked, from test_results, to the lab_request.pk for this (MSH 004, ORC 003)
  3. and in either case above, import each of the associated OBX segments

Each ORC will give rise to exactly one child OBR. Each OBR inherits and carries information which was either directly supplied as part of the original order (for example, instructions as to the copy-to providers), or information arising as a consequence of the order (for example, the nature of what was ordered will determine which Diagnostic Service Section within the lab is to perform the measurements).

Excelleris appends each "resulted" OBX – as soon as it is received from the lab – into the "built" message. Until the last component of a request becomes finalized, each download of this request's results will include all previously-built OBX results for this request. It is not possible to reliably link already-received OBX segments from newly-received segments, for the following reasons:
  1. already-sent OBX results get re-included with new, first-time OBX results however
  2. the value in OBX 001 (Numeric counter) has no guaranteed constancy in later, updated transmissions under the enclosing ORC OBX segments. Whichever OBX was, for example, previously first among OBX segments may not be first in later messages… any characterization of OBX 001 as a "Unique sequence number" is true within any instance of a message, is just isn't constant
  3. despite that the OBX 014 date-time stamp for the lab's measurement may not have changed, it is not clear that it is safe to make assumptions about whether any one OBX can or cannot be left as they are when other elements within that ORC OBR have changed.

Fortunately, despite that any one patient's "order" might get broken up into multiple ORCs (let us call them ORC1 and ORC2 despite it is not proper nomenclature), most labs will separate their partial or full reporting granularly by {ORC OBR}. Many do not even bother to report any OBX within {ORC OBR} until all constituent OBX become available. Thus where, for example, the test panel required for ORC2 would need 3 measurements {a,b,c} in order to be complete

Lab A will not issue ORC2 until all of {a,b,c} are available:

MSH PID ORC1 OBR1 no OBX (status of pending)
ORC2 OBR2 OBX2a OBX2b OBX2c (final)

Lab B does issue preliminaries within an ORC, for example …

MSH PID ORC1 OBR1 OBX (F = complete)
ORC2 OBR2 OBX2a OBX2b (P = preliminary)

and follows it later with a more-complete final message …

MSH PID ORC1 OBR1 OBX (F = complete)
ORC2 OBR2 OBX2a OBX2b OBX2c (F = complete)

without touching the datetime stamp (in the first OBR's OBR 022). Lab B will update OBR 022 only in the affected OBR.

Lab A and Lab B both make it possible for a praxis to need to sign the result of OCR1 OBR1 only once.

Lab A will not supply the results of the second panel until they are complete, so these too would need signing only once. Lab B, on the other hand, will provide early partial results for that second panel (measurements OBX2a and OBX2b) however the praxis should prefer to delete these and to replace them with the OBX2a and the OBX2b that will be re-transmitted with the OBX2c measurement as part of the finalized report, and which would needs only to be signed once. The alternative would be to tolerate multiple copies of OBX2a and OBX2b but it may be a problem to keep it in the situation where the lab later issues corrected results that disagree with what was earlier supplied.

In BC CA
  • most labs adopt the method of Lab A, except for
  • microbiology results, which are usually handled in the manner of Lab B with a preliminary and then finals once the speciation, sensitivity and resistance are known
  • there does exist a single instance of "Lab C" which, any time an OBR 022 is updated, propagates that update to the other OBRs within that same accession, despite that none of the OBX in those other ORC OBR pairs had changed!

Proposed handling of GNUmed table clin.lab_requests
lab_requests column fill using comments
.clin_when OBR 007 (Observation Date-Time) Specimen collection timestamp YYYYMMDDHHSS if time known, YYYYMMDD if time unknown
.fk_encounter   auto-generated encounter of type "lab encounter"
.fk_episode   clin.episode.pk to existing unattributed "lab episode", else auto-generate
.narrative    
.soap_cat   DEFAULT 'p'::text
.pk   primary key (auto-assigned)
.request_id   (future) GNUmed outgoing token, supportible in ORC 002 (Placer Order Number) – unused by Excelleris
.fk_test_org MSH 004 (Sending Facility) match or auto-create
.fk_requestor   (if computable) pk of ordering provider
.lab_request_id ORC 003 (Filler Order Number / Order Number ID of lab performing tests / Accession number-test code-tiebreaker)  
rename .lab_rxd_when OBR 014 (Specimen received Date timestamp) rename column to lab_rcvd_when
.results_reported_when OBR 022 (Report Status Change timestamp) this value makes it possible to determine if a result is being reprocessed, and whether it has changed
.request_status OBR 025 (Result Status) I = pending
P = preliminary,
F = complete,
C = corrected,
X = deleted (available on request; not always preceded by non-X OBRs in an earlier transmission)
.is_pending   if request_status is in [F,C], true, otherwise false
.diagnostic_service_section OBR 024 (Diagnostic Service Section)  
.ordered_service OBR 004 (Universal Service ID) Test code^Test name
v16 .note_test_org NTE data holds NTE content concerning order modifications

UNIQUE {.fk_test_org .lab_request_id}

Additional notes

  • future : do not auto-create if lab_requests already has something pending for this patient under this requestor (evaluate fk_requestor data against ORC 12 "Ordering Provider" data). In the event of a match, this pending request could be updated in place of auto-creating a new record.

import ± correct / replace test results

clin.test_result source of value notes
fk_request clin.lab_request.pk needed as fk by which to replace interim results with complete / corrected results, where applicable
fk_type clin.test_type.pk if exists, still check for update of comment from OBR NTE 003
otherwise auto-generate  
fk_encounter clin.lab_requests.fk_encounter  
otherwise auto-generate  
fk_episode clin.lab_requests.fk_episode  
otherwise auto-generate  
clin_when OBR 007 (Observation Date-Time) Specimen collection timestamp YYYYMMDDHHSS if time known, YYYYMMDD if time unknown
val_num OBX 005 (Result) if OBX 002 (Value Type) == NM
val_alpha OBX 005 (Result) if OBX 002 (Value Type) == FT
val_unit OBX 006 "Units"  
val_normal_range OBX 007 (Reference Range)  
abnormality_indicator OBX 008 (Abnormal Flags)  
note_test_org OBX NTE 003 (Comment)  
fk_intended_reviewer ORC 12 (ordering provider), OBR 028 (Result Copies To) see above, Auto-match to a provider / intended reviewer

As noted in the section resolve the status of the orders in this message, and their associated results, it may be necessary to delete some of these test results if they get superseded by one or more newer values.

create any missing test types

clin.test_type source of value notes
fk_test_org clin.test_org.pk  
otherwise auto-generate  
code OBX 003 (Observation identifier) substring 1 LOINC code
name OBX 003 (Observation identifier) substring2 Result name
coding_system "LOINC" fixed string; version info ?
conversion_unit OBX 006 (Units) in Canada SI, later verify against ref.units table

create any missing laboratories (test orgs)

  • check whether the test-providing facility (lab org) already exists in GNUmed
    • MSH 004 ("Sending Facility" e.g. BCB MDS VCH)
    • the check is against whether MSH 004 in clin.test_org where dem.org_category.description = 'Laboratory'
    • if it does not exist, auto-create it and assign dem.org_category.pk where dem.org_category.description = 'Laboratory'

  • set its org key value clin.test_org.pk into fk_test_org in lab_requests (needed to manage ORC 003)

create – if does not yet exist in session – an encounter of type 'data import' and attach to results

When a given test result has no corresponding lab request (which in CA will be true, except if created already on prior import of a result that is being re-sent):

  • append to clin.encounter a record having clin.encounter_type.description = 'data import'
  • re-use the pk from this clin.encounter.pk for each result imported for this patient during this import instance / session

create – if does not yet exist for patient – an episode of name 'received information' and attach to results

When a given test result has no corresponding lab request or receipt:

  • check for existence, for patient, of unattributed episode where description is 'received information'
  • if necessary create it in clin.episode
    • fk_health_issue = NULL (unattributed)
    • description = 'unassociated lab data'
    • is_open = FALSE

… end transaction

Blueprint for code for this importer

Please refer to the wiki topic LabImporterBC

Notify clinicians' inboxes (automatic)

GNUmed code will auto-populate (virtual) inbox messages for the doctors based on values in clin.test_result.fk_intended_reviewer to .

When patients are inactive in the praxis…

Ideally, results on non-patients (inactive patients) should not be received. Given however the receipt of results, it can be sensible to import them. Options for efficient handling could include:

  • when a patient's status is "inactive" (functionality for which is lacking GNUmed 0.9x):
    • import them only when a praxis provider was the ordering provider
    • auto-sign such results by a virtual clinician
    • keep the results on such patients in the "unmatched" table, or
    • otherwise dispose of them.

Previous notes & comments (some may no longer be applicable)

Dealing with redundant (duplicate / update) messages

here is how correctly identifying the message components (PID, OBRs+ORCs, OBXs associated with OBR-ORC pairs) has been implemented:

  • patient is identified as described in sections above.
  • lab is looked up or auto-created
  • clin.encounter and associated lab request is looked up by ORC 003 filler_order_code (or auto-created)
    • actually the above needs to be MSH 004 + ORC 003 and this modification factored in the bullets below
  • clin.test_result is looked up by:
    • patient dem.identity id
    • clin.encounter id
    • lab test_org id
    • ORC 003 filler_order_code
    • OBX 003.1 LOINC code and OBX 003.2 descriptive name
    • OBX 004 sub-id if it exists

earlier thread here Redundant (duplicate) messages.

notes:

  • mirthcorp's XML spec files for "observation reports" were cursorily examined and found to always contain OBR-OBX hierarchy (for observational reports)
    • --> correct
  • it was therefore logically concluded that no OBX will ever be received which does not have an associated OBR, with a filler_order_code.
    • --> correct * clin.encounter is presently the method by which the ORC is associated with its OBXs. one HL7 message can contain multiple ORCs; one ORC can have multiple OBXs. there is no other method by which ORCs can be associated with OBXs; thus, one new and unique clin.encounter is required per ORC. this should be solved by adding a fk_lab_request (default NULL) to clin.test_result.
    • --> correct fk_lab_request (but not default NULL, instead serial)
  • the "current_encounter" cannot be used if it already has an existing clin.lab_request associated with it, unless the HL7 message can be identified as being associated with the encounter by virtue of the ORC 003 filler_order_code anyway
    • --> not sure about this * OBX 004 sub-id was added to the name of the measurement type, to make it distinguishable and unique from other "measurements". for example "X10011^Pathology Comments" plus sub-id of 1 is treated as a separate measurement type from the exact same LOINC code plus name and a sub-id of 2, by appending "space (1)" to the first and "space (2)" to the second. in this way, the OBX is guaranteed unique. it is less than ideal: ideally a sub-id field named "test_type_sub_id" as text (not numeric, just in case) - needs to be added to clin.test_result.

TODO - which also involves reviewing

  • stop using clin.encounter as means to link clin.lab_request with clin.test_result, use clin.test_result.fk_lab_result
    • this can now (July 2011) be done, as clin.test_result.fk_lab_result now exists
  • place OBR 002 (placer not filler) into comment field in clin.lab_request (and ORC 002 probably as well)
  • use OBX 014 on clin.test_result and have an "updated timestamp" on clin.lab_request, update it when latest HL7 message overwrites anything. timestamp should ideally/conveniently be filled in with an appropriate timestamp of the HL7 message (to be decided. anyone any suggestions? MSH 007 - date/timestamp of message sound good enough?)
  • on "patient merge", use entire key-set (see above) to merge clin.lab_requests and clin.test_results based on previously-mentioned "last updated" field
  • add
    • clin.imported_data_matched table, to which HL7 raw data is moved (from clin.imported_data_umatched).
    • many-to-many (pk fk_imported_data_matched fk_lab_request) as well as many-to-many (pk fk_imported_data_matched fk_test_result) tables
    • --> no, I don't think this is necessary -- Jim
Topic revision: 30 Sep 2011, JamesBusser
 
Download.png
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback
Powered by Olark