...
DATA ARCHITECTURE
A. Using the HDS DB
.png?version=1&modificationDate=1633970116524&cacheVersion=1&api=v2)
B. Using only the API
...
| SUBJECT | DA - Extraction Processing |
|---|---|
| STATUS | Open |
DECISION | TBD |
DISCUSSION |
In openIDL when a data call (or a stat reporting) is "consented to" by the carrier, the data must first be accessed from somewhere and then transformed into the result format and lastly converted into a report that the target party (usually a regulator) can access or be sent. The transformation of the data from its "harmonized" state to the result state is called the "extraction", "extraction pattern" or "extraction process". Since accessing the data can take multiple forms (see other architecture decision "Harmonized Data", there is some variability in this decision while that decision is undecided. We can assume that the data being accessed for the extraction is "harmonized", meaning for every execution of the extraction on a single node or multiple nodes for a single carrier or multiple carrier, the schema and semantics of the data are known and consistent. Creation and Management of extractions can be organization specific. For stat reporting, this is the stat-agent (such as AAIS), for data calls, it could be the regulator or a representative like the stat-agent. In either option from the diagrams above, the extraction processor will access the data through an api instead of accessing a database directly. It is proposed that graphQL be considered as the language used to access for extraction and summarization. The extraction processor could apply some correlation to the data, like using an address to look up census data or similar. This means the extraction is more than just a data access. See the diagram above for how this component may be architected. |
| SUBJECT | DA - Harmonized Data Scope |
|---|---|
| STATUS | Proposed |
DECISION | Proposal: data format / schema will be standardized across nodes, for a given use case |
DISCUSSION | The data available for extraction must be normalized for multiple extractions across multiple use cases across multiple members. Is this one single model? Is the data at rest in the same model as the data in motion? |
...
| SUBJECT | DA - Harmonized Data Format Governance |
|---|---|
| STATUS | Proposed |
DECISION | Proposal: data schema, enumeration enumerations and the data dictionary will be standardized, and endorsed by the RRSC (and other groups per use case) |
DISCUSSION | The Technical Steering Committee, Regulatory Reporting Steering Committee and the Data Model Steering Committee are all possible owners of this. |
...
| SUBJECT | DA - Harmonized Data Store |
|---|---|
| STATUS | Proposed |
DECISION | Proposal:
|
DISCUSSION | The data available for extraction must be normalized for multiple extractions across multiple use cases across multiple members. Is this one single model? Is this one database? Is this data at rest and/or available through an API |
...
| SUBJECT | DA - Harmonized Datastore DBMS Implementation |
|---|---|
| STATUS | Open |
DECISION | Proposal: If the HDS will be a relational database. It cannot be a noSQL, graph, is a physical db inside the node, then the HDS DBMS must support our chosen access language (graphQL?) HDS will be a relational database. It cannot be a noSQL, graph, document DB etc. technical implementation of a HDS is non-prescriptive i.e. it can be MySQL, MS SQL, Oracle etc. |
DISCUSSION | If data is at rest in the harmonized datastore, what is the technology? Does it need to be a single dbms? Should it be noSQL? Can it just be an interface? |
...
| SUBJECT | DA - Harmonized Data Model |
|---|---|
| STATUS | Open |
DECISION | ?? |
DISCUSSION | The data available for extraction must be normalized for multiple extractions across multiple use cases across multiple carriers. The data must be produced by the carrier from their original sources. Is this an ETL process? What is the format of the load data? Is it the schema of a standing HDS database or is it a messaging format? Is there just one "in transit" model or are there different ones for different contexts? Contexts might be the data call, the line of business, etc. |
| SUBJECT | DA - Export Data Model |
|---|---|
| STATUS | Open |
DECISION | Proposal: The export data model is specific to each use case and must be specified in that data call / regulatory report |
DISCUSSION | The extraction process results in an export of data in an agreed format. The format of this must be defined as part of the specific data call or regulatory report. |
APPLICATION DEVELOPMENT
| SUBJECT | Common UI Code Management |
|---|---|
| STATUS | Open |
DECISION | Single Application Angular UI Variations will utilize angular libraries |
DISCUSSION | The library will be a different kind of angular app located in the same super library as all apps that use it. This is the approach for the data-call-ui. (openidl-ui and openidl-carrier-ui) For common / shared libraries we will use an npm registry. |
...