Subscription Architecture

From OpenIndivo Documentation Wiki

Original Author: Jonathan Abbett, Children's Hospital Informatics Program

Contents 1 Overview 2 Strategy 3 Use Case Narratives 3.1 Agent Creation 3.2 Patient Subscription 3.3 Synchronization 4 Functional Components 4.1 Agent Registry 4.2 The Subscription Document 4.3 The Subscription Agent 5 Discussion 5.1 Read-Only Documents

Overview

Pre-populating an Indivo record with hospital data was first described in the foundational 2001 IJMI paper on PING for purposes of collecting newborn medical records, reporting lab test results, and such. The specific situations put forth lent themselves to simple data transfers from external sources. Even on a more generalized scale, as in the ping-registration and related projects, it is a relatively trivial task to load data on a one-time basis from an external source into an Indivo record.

Maintaining that data over a period of time – during which data may be added, removed, or altered within the Indivo Server, within an external source, or both – is quite non-trivial. Immediately recognizable are issues of concurrency and consistency due to simultaneous, distributed data operations. A more elusive challenge lies in the very nature of the Indivo record as a highly flexible, independent, and XML-based animal: how can we persistently "relate" an Indivo document to its proprietary source object without jeopardizing portability or patient control? In addition, we must not sacrifice the institutional neutrality of the Indivo Server by loading it with excessive domain-specific functionality.

This document describes a data subscription framework which mostly eliminates the challenges in maintaining concurrency and consistency, tightly manages relations to external documents, and still provides both a great degree of utility to the end-user and sufficient flexibility and independence to the other parties involved.

Strategy

First, we will voluntarily limit the scope of our data management capabilities by providing one-way information transfer. Data from external sources may be loaded into subscribed Indivo records, but the external data will not be modifiable within Indivo. Thus, no major consistency or concurrency conflicts can arise as changes will only originate from the external source.

Second, we will define a new system role, that of "subscription agent," which will act as a bridge between the Indivo Server and a proprietary data source – trusted by, but external to, both.

Third, we will define an "agent registry" which will keep track of available subscription agents, and the user-specific data they require from each subscribed Indivo user to operate.

Fourth, we will provide the user a way to indicate his subscription preferences by means of "subscription documents" and appropriate policies defined in the user's Indivo record.

Fifth, we will leverage the subscription documents to accurately maintain relationships between the user's health documents and external sources' data objects.

Use Case Narratives

Agent Creation

1. HOSPITAL wishes to make its data available to its patients who are users of Indivo. HOSPITAL implements a Subscription AGENT which has access to HOSPITAL's internal database.
2. HOSPITAL contacts ADMINISTRATOR of Indivo Server and requests that an Indivo record be created for AGENT's use.
3. ADMINISTRATOR audits AGENT software to confirm that the code is properly implemented, and reviews HOSPITAL's intended operating procedure for security and regulatory compliance.
4. ADMINISTRATOR creates an Indivo record for AGENT and provides the credentials to HOSPITAL. AGENT's role will be that of "agent." ADMINISTRATOR adds AGENT to the public Agent Registry.
5. HOSPITAL configures AGENT with credentials and technical information about ADMINISTRATOR's Indivo Server and schedules it to run on a regular basis.

Patient Subscription

1. PATIENT wishes to synchronize his Indivo record with HOSPITAL database. PATIENT logs into his Indivo client which connects him to an Indivo Server where, incidentally, HOSPITAL's Subscription AGENT has a Indivo record.
2. PATIENT accesses the subscription interface in his Indivo client. Client retrieves Agent Registry which lists all available agents on the Indivo Server, and displays them to Patient. PATIENT selects AGENT.
3. The client acts on this request by creating a new Subscription Document in PATIENT's record and by updating PATIENTS's policies to give AGENT necessary access to PATIENT's record.

Synchronization

1. AGENT process initiates at some regular interval and logs itself into an Indivo Server to which it has access.
2. AGENT queries for all records it can access, and reads the Subscription Document within each record.
   1. AGENT uses data in subscription document to get PATIENT's metadata and issues queries on the external datasource for new or updated items.
   2. AGENT writes new and updated items to PATIENT's Indivo record.
3. AGENT updates the subscription document with relationship information and other metadata.

Functional Components

Agent Registry

The Agent Registry is a simple Internet-accessible XML file which provides information to Indivo Clients about available Subscription Agents on a particular Indivo Server. Indivo Clients will be configured with the address of the appropriate Agent Registry to retrieve.

Each agent description in the XML response will conform to this schema:

<element name="SubscriptionAgent" type="sag:SubscriptionAgentType"/>
<complexType name="SubscriptionAgentType">
  <sequence>
    <element name="AgentIndivoId" type="string" minOccurs="1"/>
    <element name="AgentName" type="string" minOccurs="1"/>
    <element name="Description" type="string" minOccurs="1"/>
    <element name="UpdateInterval" type="string" minOccurs="1"/>
    <element name="RequiredValues" type="RequiredValueType" minOccurs="0" 
        maxOccurs="unbounded"/>
  </sequence>
</complexType>

<element name="RequiredValue" type="sag:RequiredValueType"/>
<complexType name="RequiredValueType">
  <sequence>
    <element name="FieldName" type="string" minOccurs="1"/>
    <element name="Description" type="string" minOccurs="1"/>
  </sequence>
</complexType>

The Subscription Document

The Subscription Document (SD), as mentioned above, allows the user to maintain his data subscription preferences and a catalog of relationships between Indivo documents and external data. This document shall be defined by the following XML schema:

<element name="Subscription" type="sub:SubscriptionType"/>
<complexType name="SubscriptionType">
  <sequence>
    <element name="SubscriptionDate" type="date" minOccurs="1"/>
    <element name="AgentIndivoId" type="string" minOccurs="1"/>
    <element name="ExternalId" type="string" minOccurs="1"/>
    <element name="DocumentRelationships" type="DocumentRelationshipType" 
        minOccurs="0" maxOccurs="unbounded"/>
    <element name="LastSynchronizedDate" type="date" minOccurs="0"/>
    <element name="KnownHandlers" type="string" minOccurs="0" maxOccurs="unbounded"/>
    <any minOccurs="0" processContents="skip">
      <annotation><appinfo>
        <xjc:dom />
      </appinfo></annotation>
    </any>
  </sequence>
</complexType>
    
<element name="DocumentRelationship" type="sub:DocumentRelationshipType"/>
<complexType name="DocumentRelationshipType">
  <sequence>
    <element name="DocumentIndex" type="string" minOccurs="0"/>
    <element name="ExternalIndex" type="string" minOccurs="0"/>
  </sequence>
</complexType>

The definitions of each field are as follows:

SubscriptionDate 

Date of subscription

AgentIndivoId 

Indivo ID of the responsible subscription agent

ExternalId 

The user's unique identifier within the external datasource (i.e. hospital MRN)

DocumentRelationships 

A list of relationships between Indivo documents and external resources, each represented by a DocumentRelationship object (see following fields)

DocumentIndex 

Indivo document index

ExternalIndex 

A unique identifier which the agent can understand/parse to relate the Indivo document to an external resource. This could be as simple as a table's primary key, but it will likely be something that conveys database name, schema name, table name, and unique identifier on an external database.

LastUpdatedDate 

The date the external resource was loaded into Indivo, so the agent can later determine if the resource has been updated within the external database.

KnownHandlers 

A list of Document Handler class names that were used by the agent to perform synchronization. With this list, an agent can decide to run a new Document Handler over all dates (rather than just from the last synchronization) if it's never been run before on this user.

Any 

Any additional information the agent may need to perform synchronization. Used at agent developer's discretion.

When the user creates a subscription document, he will leave LastUpdatedDate empty until the Agent sets it on first data transfer.

The Subscription Agent

The Subscription Agent is an Indivo client process that transmits on a regular basis data from an external source into subscribed Indivo records via a Indivo Server. The high-level synchronization procedure is described in the "Synchronizaton" use case above. A more technical walkthrough is as follows:

1. AGENT authenticates with Indivo Server, and queries the server for all Indivo records to which it has access.
2. AGENT reads Subscription Document in each accessible Indivo record one at a time and performs a data transfer.
   1. AGENT queries the external datasource for tuples created or modified after the subscription's LastUpdatedDate. (This will likely happen several times, once per data type, e.g. labs, immunizations, notes.)
   2. AGENT iterates through result set. AGENT checks if each tuple is already identified in DocumentRelationships collection. If so, it updates the Indivo document specified by the relationship's DocumentIndex in the PATIENT's record. If not, it creates the Indivo document in the PATIENT's record and adds a new DocumentRelationship to the subcription document.
3. AGENT updates the Subscription Document with new LastUpdatedDate.
4. Optionally, the AGENT may "send" a Indivo Message to the PATIENT with details of the synchronization event.

To allow this, some groundwork is required. Each Subscription Agent that a Indivo Server "trusts" must be a valid user on that Indivo Server, as described in the first Use Case. In addition, each user who wishes to synchronize with an Agent must modify his policies in regard to the Agent, as described in the second Use Case. The necessary policies are listed in the table below.

Action	Resource
Add	Any Indivo Document
Update	Any Indivo Document it has already added
Send	Message
Query	Record root
Read	Subscription Document
Update	Subscription Document

Indivo clients should include a simple interface to establish this policy set for a particular agent.

Discussion

Read-Only Documents

The stability of this design is predicated upon a Subscription Agent's ability to create read-only documents in a subscriber's Indivo record. Due to Indivo's core principle of giving the patient full control over his record, the suggestion of limiting that access requires serious discussions from both technological and philosophical perspectives.

First, we must devise a technical way to prevent a user from modifying documents created by an Agent. This could probably be done by implementing a policy denying the patient update or delete access to records created by a Indivo user of a certain name (in our case, the Agent's Indivo ID). But how do we prevent the user from changing his policies? Should we? Must we instead implement a solution at a lower level, adding perhaps some sort of read-only flag to Indivo documents that allows only the author of a document the ability to update or delete it?

And even if we add all the restrictions we can think of, still nothing prevents a user from exporting his record to XML and changing the data by hand.

It may be that the only true solution will involve digitally signed documents and the data consumer's obligation to confirm that a document has not been improperly modified.