Skip to content

Code Introduction

Jump to bottom
GCHQDeveloper42 edited this page Jan 19, 2024 · 3 revisions

Code Introduction

Having introduced the roots of the Top-Level Data Model we can now explain how it has been implemented in the Magma Core release. One of the key achievements of the HQDM is that it is fully, universally normalised; the author refers to this as 7th Normal Form. This means that we can abstract the implementation pattern for the data model and base it on the root entity and build the structure based on the type hierarchy. We can now start to talk about data itself. Every coherent record that is implemented using Magma Core is a collection of attributes that conform to the pattern specified by the TLDM. We shall call these records "data objects". Each data object is an instance of one or more of the entity types in the TLDM (based on HQDM). All of the TLDM can itself be represented as data objects too. In this way we can directly map between the sets (universals) and elements (particulars), drawn from set theory, on which HQDM is based, to entities and relationships within HQDM and to any (and potentially all) extensions of the TLDM using the high-level architecture indicated here.

In order to achieve the implementation goal of having consistency within the Data Realm in the overview to this section, we chose an approach that allowed direct, and complete, mapping between data objects held in the database (based on Linked Data specifications) and the code used to implement data processing functions (such as the application to create and query objects). We will proceed by jumping into the use of some of the Magma Core code to create a data object that represents the notionally real "Arthur Dent". We know that if "Arthur Dent" is a particular individual then they have a lifetime that we can loosely call their whole-life existence; a spatio-temporal extent that exists for a period of time and represents the state of an individual, in this case Arthur Dent, for the whole of their life. As illustrated in the previous introduction to the TLDM any state can be composed of other states so the state of Arthur Dent while they are in a particular meeting will be a state that is a temporal_part of the whole-life state of Arthur Dent. If we know their date of birth then we know when that whole-life existence started. In HQDM there is already a Person entity and it is a good example of multiple inheritance as person is a sub-type of state_of_person (it is the whole-life state of a person), biological_system (a system that is biological and includes living things and ecosystems, composed of biological parts) and party (a party can be a person or organisation and can participate in a capacity that no other types of thing can). This simple pattern of states and the states that they are composed of provides a robust way to address the existence of things through time and is something that is hard to incorporate robustly into non-4D data models.

The code below shows the instantiation of two HQDM Java objects that are involved in the representation of "Arthur Dent", subsequently written to a database, that we'll examine:

// Arthur Dent whole life object
final Event e1 = dataObjectSvc.event("1991-02-18T00:00:00", possibleWorld, dataObjectSvc.USER_BASE);
final Person arthurDent = new PersonImpl.Builder(new IRI(dataObjectSvc.USER_BASE, dataObjectSvc.uid())
        .member_Of_Kind(kindOfPerson)
        .natural_Role_M(personRole)
        .part_Of_Possible_World_M(possibleWorld)
        .beginning(e1)
        .build();
arthurDent.addStringValue(HQDM.ENTITY_NAME, "ARTHUR_DENT");
objectDB.create(e1);
objectDB.create(arthurDent);

Firstly, the two Java objects that are in the example above have names that match the equivalent entity from the HQDM data model; Event and Person. There is a Java class for every entity in the HQDM data model**. Secondly, the Event object is created using a data object service class that hides the builder pattern for the creation of the Event object e1 but the builder pattern, based on a fluent interface to the class, for the creation of object arthurDent is shown on lines 3-8. Some characteristics of the HQDM object builder pattern are worth noting:

  • The Builder method is passed the IRI for the object's identity; comprising an IRI base (commonly set to that of a dataset) and a value part that is a uuid. The IRIs are a powerful feature of the RDF Linked Data Specifications.
  • Some Builder methods are then chained to allow the object's membership and relationship with other required objects to be committed (those objects have to exist already and are passed as required with each method that requires it - in an application these can be queried directly from a database but in this example they are already objects created locally). This chain of methods must include mandatory relationships that are specified in HQDM - a runtime exception will be called if they are not created. This enforces cardinalities in the TLDM. The Event object e1 that represents the start of the whole-life existence of "Arthur Dent" is passed to the Builder using the .beginning() method. These methods all have names that closely match those specified in HQDM as published*. We shall ignore the role of these dependencies until the Worked Example section but some of the relationship names can be seen in the top level diagrams on this page.
  • The build() method is then called at the end of the chain.

After initial creation of other objects on which this instance of the hqdm:Person object depends, the arthurDent object can be added. In this case the entity name ARTHUR_DENT name string is added. This is a terrible short-cut that is best avoided in general applications (it is an exercise for the reader to consider what happens if ARTHUR_DENT has one or more other name(s), or changes their name). Of course, there is a solution to the challenge of things having multiple names, or name changes, and that is the sign & representation_by_sign association pattern in HQDM. An example of this may be introduced in a future version of this documentation. Contact the authors if you want an introduction to it before it is added to this wiki.

A summary of the Java object created using the Java class Person.java from the snippet above is provided below:

HQDMObject{getIri=https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#e5cdb431-86b1-4b66-a8a4-df409e5b2b2c
    values=
        key: https://hqdmtop.github.io/hqdm#part_of_possible_world,
            values: [https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#3f2eee08-b2b2-4228-82e1-321eaf36c0f4 ]
        key: https://hqdmtop.github.io/hqdm#beginning,
            values: [https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#4e4fa379-62fd-482a-9efd-366d0dc59f95 ]
        key: https://hqdmtop.github.io/hqdm#natural_role,
            values: [https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide-rdl#7f6f8696-3868-4132-8795-f01904ea5fb1 ]
        key: http://www.w3.org/1999/02/22-rdf-syntax-ns#type,
            values: [https://hqdmtop.github.io/hqdm#person ]
        key: https://hqdmtop.github.io/hqdm#data_EntityName,
            values: [ARTHUR_DENT ]
        key: https://hqdmtop.github.io/hqdm#member_of_kind,
            values: [https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide-rdl#74c9e8bd-89df-4f7f-ae5d-0104f79ae6e9 ]
        key: https://magma-core.gchq.gov.uk/ontologies/universe/class_name,
            values: [uk.gov.gchq.hqdm.model.impl.PersonImpl]
}

The root object (from which all the HQDM objects inherit) is HqdmObject. All the HqdmObject properties are based around IRIs that are mappable directly to RDF Triples. The IRI for each object is the primary property for that object (accessible by calling .getIri()). The IRIs are composed of a Base part and a locally unique (relative) value part, see here for details. The remaining object properties are stored in a key-value hashMap that holds the RDF-compatible object properties, with the keys being the RDF predicate IRI. Each predicate can have multiple entries if the cardinality for that predicate allows. A number of methods allow search, reading and writing of these predicates (including case-insensitive search).

From the key-value list for the HqdmObject representing arthurDent there are predicate entries for all of the builder methods called when creating the object. For predicates that refer to other instances of HqdmObject the IRI reference to that object is used, obtained by passing that object to the builder method. An extra predicate is also held for the HqdmObject for arthurDent that contains a reference to the Java class that specifies the object characteristics based on the HQDM model blueprint. This creates a binding between the set of predicates and the HQDM entity that specifies its required properties.

The representation of the RDF triples (in turtle format) for the HqdmObject arthurDent is shown below:

<https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#e5cdb431-86b1-4b66-a8a4-df409e5b2b2c>
        a       <https://hqdmtop.github.io/hqdm#person> ;
        <https://hqdmtop.github.io/hqdm#beginning>
                <https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#4e4fa379-62fd-482a-9efd-366d0dc59f95> ;
        <https://hqdmtop.github.io/hqdm#data_EntityName>
                "ARTHUR_DENT" ;
        <https://hqdmtop.github.io/hqdm#member_of_kind>
                <https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide-rdl#74c9e8bd-89df-4f7f-ae5d-0104f79ae6e9> ;
        <https://hqdmtop.github.io/hqdm#natural_role>
                <https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide-rdl#7f6f8696-3868-4132-8795-f01904ea5fb1> ;
        <https://hqdmtop.github.io/hqdm#part_of_possible_world>
                <https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#3f2eee08-b2b2-4228-82e1-321eaf36c0f4> ;
        <https://magma-core.gchq.gov.uk/ontologies/universe/class_name>
                "uk.gov.gchq.hqdm.model.impl.PersonImpl"

The IRI for the identity of that object is the RDF subject for all the predicates that were listed in the HqdmObject's predicate hashMap. The first predicate a is a turtle abbreviation for rdf:type, meaning that this object is an instance of a RDFS Class that is hqdm:person. This is exactly what we created when we built the HqdmObject in the first place. All the predicates match those stored for the HqdmObject for arthurDent.

This illustrates the 'complete' match between the HqdmObject in the example Java code and the RDF triples that are stored in a suitable triplestore. The Java object can be re-created by reading the predicates from the triplestore. If implemented strictly (as intended) there should be complete consistency between the representation (and handling) of objects in application code and the storage of those objects in a triplestore. The objectDB.create() method writes the object to a compatible database.

For completeness the HqdmObject that represents the Event (an instant in time) that has a record of that instant as an XML date time string as the object's EntityName is shown in turtle below:

<https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#4e4fa379-62fd-482a-9efd-366d0dc59f95>
        a       <https://hqdmtop.github.io/hqdm#point_in_time> ;
        <https://hqdmtop.github.io/hqdm#data_EntityName>
                "1991-02-18T00:00:00"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
        <https://hqdmtop.github.io/hqdm#part_of_possible_world>
                <https://magma-core.gchq.gov.uk/ontologies/hitchhikers-guide#3f2eee08-b2b2-4228-82e1-321eaf36c0f4> ;
        <https://magma-core.gchq.gov.uk/ontologies/universe/class_name>
                "uk.gov.gchq.hqdm.model.impl.PointInTimeImpl" .

The total number of HQDM data objects that are constructed to represent the scenario in the space-time diagrams on the previous page is 34. This initially sounds like a lot but the summary below shows that much of it is reusable:

  • Reference Data Objects: 15

    The reference data objects are the classes created to allow the instances of the spatio-temporal states represented in the diagram to be created and have the correct set membership. These are all reusable - not just for other applications of this scenario but for any other scenario that involves these entity types.

    • 6 classes of whole-life individuals were created, introducing a basic class hierarchy.
    • 2 classes of states
    • 4 roles
    • 3 Class objects were created to enable application-specific object handling. This is an illustration of how the management of data objects that may be directly significant to users (for example) can be constructed into the model itself.
    • Extension of the state entity types were not included. This may be suitable for many applications but a 'complete' extension of the model would require these to be created too, particularly if a subset of the model was to be adopted by a system that interfaces directly to a distributed information environment based on the consistency offered by HQDM. Alternatively these can be generated when required.

  • User Data Objects: 19

    • 1 Possible world object: This Example only makes use of a single 'possible world' in which all the data objects are aggregated into. The use of the 'possible world' model feature is a large topic but for now it will be left as a feature for users to explore where there is interest in allowing different projections of object collections to explore the possibility of different sets of objects representing what was actual (or closest to 'actual').
    • 2 Whole-life individual objects (the house and the person): There need only be one of each for the lifetime of the use of this scenario even if it is applied for every circumstance in which that particular person is in that particular house. All the states representing these circumstances are temporal parts of their respective whole-life object.
    • 4 States: These are the states of the material objects that are involved with each other (i.e. the states of the house and the person when 'at' home)
    • 6 Events: These are the state bounds of all the objects involved. They can be applied to any state that shares that event boundary (in a large system a mapped store, or stores, of events may be more efficient and just as compliant)
    • 6 Associations and Participants: These are the model patterns that allow the rigorous construction of the 'at home' records of the person, and the house, involved. There is a more efficient implementation approach that is in the HQDM issue list.

How did we find out about the need to create these objects? There are three aspects worth taking stock of:

  1. The space-time diagram contained a succinct, easy to interpret, view of our information modelling requirement. The requirement is key and the use of a clear way to analyse it and break it down to identifiable component parts is critical.
  2. Some knowledge of the underlying data model and its model 'commitments' is necessary but the process becomes easier the more it is used. There is a learning curve but the aim of the Magma Core and HQDM release is to ease the adoption of the modelling techniques.
  3. The HQDM Java classes have object builders methods that enforce the relationship requirements (e.g. cardinalities) of the underlying model. This provides a useful back-stop to ensure that anything that is needed but is not made explicit in the space-time diagram (e.g to avoid clutter) can be identified.

As more modelling scenarios are implemented it becomes apparent that there are repeating patterns of objects that can be built into application code (or could be added to Magma Core itself).

Next: Implementation Architecture Considerations

**This also leads to the unusual situation that there is a HQDM Java class called "Class". Care should be taken to scope it properly, for example by specifying the full package path for that object as Java Class ≠ HQDM Class Entity.

Clone this wiki locally