Page Comparison

...

SUBJECT	DA - Extraction Processing
STATUS	Open
DECISION	TBD
DISCUSSION	In openIDL when a data call (or a stat reporting) is "consented to" by the carrier, the data must first be accessed from somewhere and then transformed into the result format and lastly converted into a report that the target party (usually a regulator) can access or be sent. The transformation of the data from its "harmonized" state to the result state is called the "extraction", "extraction pattern" or "extraction process". Since accessing the data can take multiple forms (see other architecture decision "Harmonized Data", there is some variability in this decision while that decision is undecided. We can assume that the data being accessed for the extraction is "harmonized", meaning for every execution of the extraction on a single node or multiple nodes for a single carrier or multiple carrier, the schema and semantics of the data are known and consistent. Creation and Management of extractions can be organization. For stat reporting, this is the stat-agent (such as AAIS), for data calls, it could be the regulator or a representative like the stat-agent.

SUBJECT	DA - Harmonized Data Format
STATUS	Proposed
DECISION	Proposal: data format / schema will be standardized across nodes, for a given use case
DISCUSSION	The data available for extraction must be normalized for multiple extractions across multiple use cases across multiple members. Is this one single model? Is the data at rest in the same model as the data in motion?

SUBJECT	DA - Harmonized Data Loading / Normalization
STATUS	Proposed
DECISION	Proposal: If the HDS is at rest, the loading of that data is the responsiblity of the member owner of the node. If the HDS is an API, the maintenance of that API and the mapping to other data sources is the responsiblity of the member owner of the node.
DISCUSSION	Loading data will be via an API, IDM or ...? Will a direct SQL load be allowed? This will use the ingestion model.

SUBJECT	DA - Harmonized Data Format Governance
STATUS	Proposed
DECISION	Proposal: data schema, enumeration and the data dictionary will be standardized, and endorsed by the RRSC (and other groups per use case)
DISCUSSION

SUBJECT	DA - Harmonized Data Store
STATUS	Proposed
DECISION	Proposal: HDS can be persistent or transient. It's a member's decision. See next HDS can be persistent and can be used by member's as a "warehouse on the edge" for sharing data via openIDL
DISCUSSION	The data available for extraction must be normalized for multiple extractions across multiple use cases across multiple members. Is this one single model? Is this one database? Is this data at rest and/or available through an API

SUBJECT	DA - Harmonized Datastore DBMS Implementation
STATUS	Open
DECISION	Proposal: HDS will be a relational database. It cannot be a noSQL, graph, document DB etc. technical implementation of a HDS is non-prescriptive i.e. it can be MySQL, MS SQL, Oracle etc.
DISCUSSION	If data is at rest in the harmonized datastore, what is the technology? Does it need to be a single dbms? Should it be noSQL? Can it just be an interface?

SUBJECT	DA - Harmonized Data Model
STATUS	Open
DECISION	??
DISCUSSION	The data available for extraction must be normalized for multiple extractions across multiple use cases across multiple carriers. The data must be produced by the carrier from their original sources. Is this an ETL process? What is the format of the load data? Is it the schema of a standing HDS database or is it a messaging format? Is there just one "in transit" model or are there different ones for different contexts? Contexts might be the data call, the line of business, etc.

...

Versions Compared