https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/pnd01/niem-6.0-arch-changes-v1.0-pnd01.md
(Authoritative)
https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/pnd01/niem-6.0-arch-changes-v1.0-pnd01.html
https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/pnd01/niem-6.0-arch-changes-v1.0-pnd01.pdf
N/A
https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/niem-6.0-arch-changes-v1.0.md
(Authoritative)
https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/niem-6.0-arch-changes-v1.0.html
https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/niem-6.0-arch-changes-v1.0.pdf
NIEM Technical Architecture Committee (NTAC) of the OASIS NIEMOpen OP
Katherine Escobar (katherine.b.escobar.civ@mail.mil), Joint Staff J6
Jim Cabral (jim.cabral@infotrack.com), InfoTrack
Scott Renner (sar@mitre.org), MITRE
Thomas Carlson (thomas.carlson@gtri.gatech.edu),
GTRI
Christina Medlin (christina.medlin@gtri.gatech.edu),
GTRI
Scott Renner (sar@mitre.org), MITRE
This document describes the major architectural changes planned for the NIEM Naming and Design Rules specification for Version 6.0.
This is a Non-Standards Track Work Product. The patent provisions of the OASIS IPR Policy do not apply.
This document was last revised or approved by the Project Governing Board of the OASIS NIEMOpen OP on the above date. The level of approval is also listed above. Check the "Latest stage" location noted above for possible later revisions of this document. Any other numbered Versions and other technical work produced by the Open Project (OP) are listed at http://www.niemopen.org/.
Comments on this work can be provided by opening issues in the project repository or by sending email to the project's public comment list: niemopen-comment@lists.oasis-open-projects.org. List information is available at https://lists.oasis-open-projects.org/g/niemopen-comment.
When referencing this document the following citation format should be used:
[NIEM-6-Arch-Changes-v1.0]
NIEM 6.0 Architectural Changes Version 1.0. Edited by Thomas Carlson, Christina Medlin, and Scott Renner. 27 June 2023. OASIS Project Note Draft 01. https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/pnd01/niem-6.0-arch-changes-v1.0-pnd01.html. Latest stage: https://docs.oasis-open.org/niemopen/niem-6.0-arch-changes/v1.0/niem-6.0-arch-changes-v1.0.html.
Copyright © OASIS Open 2023. All Rights Reserved.
Distributed under the terms of the OASIS IPR Policy.
For complete copyright information please see the full Notices section in Appendix D below.
sequenceID
attribute in structures namespaceRoleOf
properties in modelsThis document describes the major architectural changes planned for the NIEM Naming and Design Rules specification for version 6.0. It is a working document, and may change several times before the NIEM 6 publication is complete. The document contains phrases such as "NIEM 6 will… " and "NIEM 6 does not…", but in this draft these describe current intentions, not final decisions. Comments, criticism, and questions are welcome.
Term | Description |
---|---|
Class | A type that contains properties |
Object property | A property that has a class type (contains properties) |
Datatype | A type that contains attribute properties and carries a value (e.g., a string, number, boolean, etc.) |
Data property | A property that has a datatype (contains a value, plus optional attribute properties). |
Term | Description |
---|---|
CMF | Common Model Format |
EXT | NDR conformance target for Extension Schema Document |
IC-ISM | Intelligence Community Information Security Markings |
IC-NTK | Intelligence Community Need-to-Know |
NBAC | NIEM Business Architecture Committee Technical Steering Committee |
NTAC | NIEM Technical Architecture Committee Technical Steering Committee |
NDR | Naming and Design Rules |
REF | NDR conformance target for Reference Schema Document (stricter rule set for reusable reference schemas) |
The following architecture changes are briefly described here; details appear in later sections of this project note as needed.
sequenceID
attribute in structures namespaceroleOf
properties in modelsThere are two kinds of data model in NIEM: reference models and message models. These models are represented in XSD as reference schemas and message schemas.
The NIEM core and domain models are reference models. These models express semantics, providing names and definitions for concepts, and relationships among concepts. They are characterized by "optionality and over-inclusiveness". That is, they define more concepts than needed for any particular data exchange specification, without cardinality constraints, so it is easy to select the concepts that are needed and omit the rest.
The data model in a message specification (or IEPD) is a message model. These models include cardinality constraints and datatype restrictions to define the mandatory and optional content of a particular message format in a data exchange. They are formed by assembling subsets of the reference schema documents that define the NIEM model, plus one or more extension schema documents that contain any exchange-specific definitions.
NIEM 6 introduces a new conformance target for message schema
documents that omits many of the conformance rules for
reference schema documents and extension schema
documents. For example, a message schema may use
xs:choice
instead of element substitution. In NIEM 6, a
message schema is focused on validation and code binding, and need not
follow all the rules that promote reuse and impose semantics on XSD
constructs. As a result, software which produces and consumes NIEM data
will be easier to implement. (Semantics and reuse are still available
and supported through the reference model.)
The subset schema rule continues to apply in NIEM 6. Any XML that is valid when assessed against a message schema must also be valid against the reference schema.
NIEM 6 introduces the Common Model Format, a technology-neutral data modeling formalism equivalent to NIEM XSD, CMF and NIEM XSD are both fully supported in NIEM 6.
CMF is a NIEM-conforming message specification (or IEPD) for the information in a NIEM data model. A CMF document is a message (or IEP) representing one particular data model, and is equivalent to the XSD schema document set for that model.
CMF was designed in the same way as any other NIEM message specification. The first step was to describe the information requirements. What are the facts that make up a NIEM data model? The result is the NIEM metamodel, an abstract model for NIEM data models, depicted below as a UML diagram.
The CMF specification instantiates that abstract model for models as a message schema in NIEM XSD. It may be examined at its NIEMOpen github repo. A more detailed description of CMF and the metamodel is available at the niem.gov site.
A CMF model may be transformed into developer artifacts for many technologies. For example, NIEMOpen provides free and open-source software to transform CMF into JSON Schema, suitable for validating NIEM JSON data. (Transforming CMF is much easier than transforming the equivalent NIEM XSD.) Many other transformations are possible; some are imagined in the diagram below.
The NIEMOpen project includes CMFTool, which is free and open-source software capable of translating a CMF model to the equivalent NIEM XSD schema, and vice versa. Developers can therefore work with either formalism and still take advantage of the tooling for both.
NIEM XML elements with simple content and attributes are not easily represented in NIEM JSON. NIEM 6 resolves this difficulty by creating new data properties in CMF models. The change is invisible in NIEM XSD and XML, but apparent in CMF and in any serialization that does not have XML's distinction between elements and attributes – which is probably all of them.
In NIEM 6, an XML element of complex content is an object property
and has a class. Here is an example of the PersonName
object property in NIEM XML and NIEM JSON:
<nc:PersonName> | "nc:PersonName": {
<nc:PersonGivenName>Tommy</nc:PersonGivenName> | "nc:PersonGivenName": "Tommy",
<nc:PersonSurName>Atkins</nc:PersonSurName> | "nc:PersonSurName": "Atkins"
</nc:PersonName> | }
A simple content element with no attributes is a data property and has a datatype.
<nc:PersonGivenName>Tommy</nc:PersonGivenName> | "nc:PersonGivenName": "Tommy"
But what to do for a simple content element with attributes? There is no good JSON key for the simple content value.
<nc:PersonMiddleName @nc:partialIndicator="true"> | "nc:PersonMiddleName": {
Bartholomew | "nc:partialIndicator": true,
</nc:PersonMiddleName> | ????? : "Bartholmew"
| }
In NIEM 6, a simple content element with attributes is an object
property. When translating from XSD to CMF, a new data property is
created for the CMF model – nc:PersonMiddleNameLiteral
–
which appears in the JSON serialization, like this:
<nc:PersonMiddleName @nc:partialIndicator="true"> | "nc:PersonMiddleName": {
Bartholomew | "nc:partialIndicator": true,
</nc:PersonMiddleName> | "nc:PersonMiddleNameLiteral" : "Bartholmew"
| }
Most simple content elements do not have attributes from the NIEM model or extension schemas. Other sections of this project note explain why none of the attributes in the structures namespace routinely appear on simple content elements. Almost all simple content elements will therefore be data properties. If a simple content element does have attributes, that will be a choice made by a data modeler, and it will be an object property.
NIEM has always defined semantics in terms of RDF equivalents. This
means an element that can have a referencing attribute (id
,
ref
, or uri
in the structures namespace) must
be an object property, because literal values do not have identifiers in
RDF.
Referencing simple content is rare in NIEM XML It is unusual to see something like
nc:PersonName s:id="link">Tommy</nc:PersonName>
<nc:PersonName s:ref="link"/> <
NIEM 6 therefore assumes that an element declaration of simple content without attributes is a data property and cannot have any of the referencing attributes. This keeps the common case simple. NIEM 6 provides new appinfo for the unusual case. For an XML message to include something like
my:SimpleContent s:id="link">FOO</my:SimpleContent> <
the corresponding model in XSD must include that appinfo, like this:
xs:element name="SimpleContent" type="xs:token" appinfo:isObjectProperty="true" ... <
An element of simple content with attributes is always an object property and does not require this appinfo in the schema.
Sometimes a message designer wants to avoid the complexity of references to an element that will always appear inline. At present, NIEM 5 allows the designer to remove the referencing attributes from every element declaration. NIEM 6 introduces a way to remove the reference properties from particular element declarations, while leaving them on others.
Augmentation allows a data modeler to add elements to a type with
complex content in a namespace that belongs to another. For example, the
NIEM Justice domain uses augmentation to add
j:PersonHasChildrenIndicator
(and 97 other elements) to
nc:PersonType
.
In NIEM 6, augmentation is enhanced in two ways: The component that is added can be an attribute as well as an element, and the component being augmented can be a type with simple content as well as complex content.
NIEM 3 added support for the US Intelligence Community's
Information Security Marking and Need To Know
standards, by adding xs:anyAttribute
elements to several
types in the structures namespace. That was a hack, but it satisfied a
large user community and, since a message specification could always
remove the attribute wildcards, didn't offend anyone very much. We can
achieve the same result in NIEM 6 with attribute augmentations, so we
have removed the hack.
In order to apply an augmentation to every element in a model (for
instance, to support ISM and NTK), the model must have a base type from
which all types are derived. In NIEM 5 those base types are defined in
the structures namespace. In NIEM 6, the structures namespace applies
only to NIEM XML – it has nothing to do with models in CMF, or messages
in JSON – and so it is not properly part of any NIEM model. The base
types for a NIEM model belong in the NIEM Core, and so NIEM 6 adds
nc:ObjectType
to the core, and derives all other types from
those base types. NIEMOpen has FOSS tools to assist migration by
modifying existing NIEM XSD to use these new base types.
sequenceID
attribute in structures namespaceBy default there is no meaning ascribed to the order of a repeated
element in NIEM XSD. In NIEM 5 a message could use
structures:sequenceID
to indicate a meaningful order. For
example, a Track comprised of a sequence of Positions might look like
this:
my:Track>
<my:Position structures:sequenceID="01"> ...
<my:Position structures:sequenceID="02"> ...
<my:Track> </
In NIEM 6, meaningful order is asserted in the model, not the
message. NIEM 6 removes sequenceID
from the structures
namespace, and adds a new appinfo:orderedPropertyIndicator
for use in NIEM XSD. The message model for the example above would look
like
xs:complexType name="TrackType">
<xs:complexContent>
<xs:extension base="nc:ObjectType">
<xs:sequence>
<xs:element ref="my:Position" maxOccurs="unbounded" appinfo:orderedPropertyIndicator="true"/> <
RoleOf
properties in modelsExisting versions of NIEM include a number of role properties and role types.
Role types were introduced into NIEM after XML Schema extension proved to be insufficient in certain situations. An object may have multiple functions in the same instance document, each with associated data. For example, a person might be both a j:CrashDriver and a j:ArrestSubject. Without roles, information about the person would be duplicated in extensions, or would be left ambiguously blank in some places. NDR 10.2.2
For example, a NIEM 5 message might look like this:
my:Message>
1 <j:Crash>
2 <j:CrashVehicle>
3 <j:CrashDriver>
4 <nc:RoleOfPerson structures:id="P01">
5 <nc:PersonFullName>Peter Wimsey</nc:PersonFullName>
6 <nc:RoleOfPerson>
7 </j:DriverLicense>
8 <j:DriverLicenseCardIdentification>
9 <nc:IdentificationID>A1234567</nc:IdentificationID>
10 <j:DriverLicenseCardIdentification>
11 </j:DriverLicense>
12 </j:CrashDriver>
13 </j:CrashVehicle>
14 </j:CrashPerson>
15 <nc:RoleOfPerson structures:ref="P01" xsi:nil="true"/>
16 <j:CrashPersonInjury>
17 <nc:InjuryDescriptionText>Broken Arm</nc:InjuryDescriptionText>
18 <j:CrashPersonInjury>
19 </j:CrashPerson>
20 </j:Crash>
21 </my:Message> 22 </
In this example the CrashDriver
element on line 4 and
the CrashPerson
element on line 15 are describing the same
real-world entity: one person, named Peter Wimsey, with license A1234567
and a broken arm. Role properties allow different elements to describe
the same entity without duplicating data in the message. We can simplify
NIEM 6 by removing role properties and using the
structures:uri
attribute instead, like this:
my:Message>
1 <j:Crash>
2 <j:CrashVehicle>
3 <j:CrashDriver structures:uri="P01">
4 <nc:PersonFullName>Peter Wimsey</nc:PersonFullName>
5 <j:DriverLicense>
6 <j:DriverLicenseCardIdentification>
7 <nc:IdentificationID>A1234567</nc:IdentificationID>
8 <j:DriverLicenseCardIdentification>
9 </j:DriverLicense>
10 </j:CrashDriver>
11 </j:CrashVehicle>
12 </j:CrashPerson structures:uri="P01">
13 <j:CrashPersonInjury>
14 <nc:InjuryDescriptionText>Broken Arm</nc:InjuryDescriptionText>
15 <j:CrashPersonInjury>
16 </j:CrashPerson>
17 </j:Crash>
18 </my:Message> 19 </
NIEM has always defined the semantics of NIEM in terms of the RDF conceptual model. In NIEM 4 and NIEM 5 that definition was enhanced by defining the RDF triples that are entailed by NIEM XSD and XML. NIEM 6 improves its formal semantics with additional RDF entailments, making use of OWL, SHACL, and other ontology vocabularies. It may then be possible to represent all aspects of a NIEM model in RDF. It will be possible to convert NIEM data to RDF, for use as a a knowledge graph.
Sometimes a NIEM XML element needs to modify the relationship expressed by its parent instead of the parent object itself. For example:
nc:Person>
<nc:PersonName>
<nc:PersonFullName>Clark Kent</nc:PersonFullName>
<nc:PersonName>
</nc:PersonName>
<nc:PersonFullName>Superman</nc:PersonFullName>
<my:PersonNameAugmentation>
<my:Classification>Secret</my:Classification>
<my:PersonNameAugmentation>
</nc:PersonName>
</nc:Person> </
The author of this message does not mean to say that the name
Superman
is itself a secret. Everybody knows that name. The
author is saying that the relationship between the
nc:Person
object and that name is a secret. The
my:Classification
element modifies that relationship; it is
a relationship property.
NIEM 6 uses RDF-star, an extension of RDF, to represent relationship properties. For example, the NIEM XML above is equivalent to this RDF:
_:n0 a nc:PersonType .
_:n0 nc:PersonName _:n1 .
_:n0 nc:PersonName _:n2 {| my:Classification "Secret" |} .
_:n1 a nc:PersonNameType .
_:n1 nc:PersonFullName "Clark Kent" .
_:n2 a nc:PersonNameType .
_:n2 nc:PersonFullName "Superman" .
NIEM 6 has new appinfo to mark relationship properties in NIEM XSD. The message schema for the above message would include
xs:element name="Classification" appinfo:relationshipPropertyIndicator="true" ... <
NIEM is in the process of introducing a platform- and technology-independent representation for NIEM data models via the new Common Model Format (CMF). This is a format that will represent NIEM concepts and data generically so that they can then be transformed into specific languages, such as XML Schema and JSON Schema. Transformations will typically be done via automated tooling.
The NDR will be reorganized so concepts and rules that apply to all languages are expressed in terms of CMF. Language-specific features will be moved to supplemental representations of the NDR, such as the "NIEM Naming and Design Rules: XML Representation" (NDR-XML). These changes will make it easier for developers to focus on only the parts of the NDR that apply to their needs. They also enable NIEM to better support multiple representations without having to first go through the NIEM XML representation for tool support.
In addition to being used by the NDR for NIEM 6.0 to describe concepts and rules, CMF will also be used alongside XML Schema as the official canonical representations of the NIEM data model.
More information about CMF is available at https://github.com/niemopen/common-model-format.
niemopen/niem-naming-design-rules#10
The NTAC proposess to use a new NDR conformance target for subset schemas instead of the original ReferenceSchemaDocument (REF) target.
Subset schemas from the NIEM data model can contain conformance errors because subsets are not required to follow all of the rules required for reference schema documents.
The following issues are known to be incorrectly raised from NDR conformance tests on subset schemas:
The NTAC proposes to generate NIEM subset schemas from NIEM-supported tooling with a new conformance target (TBD). The NDR rules for this new conformance target will resemble the rules for the REF conformance target with exceptions for the above rules known to be incorrectly raised during conformance testing.
niemopen/niem-naming-design-rules#21
Do not allow elements and attributes in the same namespace to share the same name with the only difference being the capitalization.
NDR rules require element names to be upper camel case (e.g.,
UpperCamelCase) and attribute names to be lower camel case (e.g.,
lowerCamelCase). In some prior versions of NIEM, elements and attributes
shared the same name in the same namespace with the only difference
being the capitalization of the first letter. For example, NIEM Core 4.0
contained both element nc:ConfidencePercent
and attribute
nc:confidencePercent
. This can present issues in
case-insensitive environments.
The impact of this change should be low. The model does not currently have any properties with overlapping names. Properties in IEPDs / message specifications may be affected. This would however require that semantically-equivalent elements and attributes be given unique names.
niemopen/niem-naming-design-rules#29
Prohibit properties in content namespaces (e.g., Core, domains, code
sets, etc.) from having types from the structures
namespace.
Types in the structures
namespace provide NIEM
infrastructure support but do not define actual content. These
types:
The NTAC proposes a new rule to prohibit NIEM-conformant properties
from having a type from the structures
namespace. This is
meant to ensure that NIEM properties are capable of carrying data
content.
The impact of this change should be low. Within the data model,
properties cbrn:CaseMetadata
and
cbrn:DataFileMetadata
are currently of type
structures:MetadataType
. These two properties would need to
be removed or updated, perhaps to be of type
nc:MetadataType
.
niemopen/niem-naming-design-rules#9
Support the declaration of facets on EXT complex value types to prevent the need for message designers to define two corresponding types (one complex, one simple) for each code set.
An example schema demonstrating the proposal is available under the examples/complex-code-types subdirectory.
NIEM requires that enumerations must be defined on simple types in
REF schemas. As all NIEM property elements are required to have complex
types, this means that code sets and types with other kinds of facets
are typically defined in pairs - a simple code type with enumerations,
and a corresponding complex code type that extends the simple code type
and adds the NIEM-required
structures:SimpleObjectAttributeGroup
.
The example below shows how code sets are typically defined in NIEM.
Notes:
structures:SimpleObjectAttributeGroup
.xs:simpleType name="AddressCategoryCodeSimpleType">
<xs:annotation>
<xs:documentation>A data type for a kind of address.</xs:documentation>
<xs:annotation>
</xs:restriction base="xs:token">
<xs:enumeration value="business">
<xs:annotation>
<xs:documentation>business</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="registered office">
<xs:annotation>
<xs:documentation>registered office</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="residential">
<xs:annotation>
<xs:documentation>residential</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="residential or business">
<xs:annotation>
<xs:documentation>residential or business</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="unspecified">
<xs:annotation>
<xs:documentation>unspecified</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:restriction>
</xs:simpleType>
</
xs:complexType name="AddressCategoryCodeType">
<xs:annotation>
<xs:documentation>A data type for a kind of address.</xs:documentation>
<xs:annotation>
</xs:simpleContent>
<xs:extension base="nc:AddressCategoryCodeSimpleType">
<xs:attributeGroup ref="structures:SimpleObjectAttributeGroup"/>
<xs:extension>
</xs:simpleContent>
</xs:complexType> </
These requirements provide three benefits:
For message designers who do not need to create attributes or unions off of their code sets, the NTAC proposes to explicitly support the declaration of facets directly on complex value types in message schemas with an EXT conformance target.
This still supports reusability in reference schemas (REF), while making message extension schemas (EXT) easier to build and maintain.
Notes:
niem-xs:token
, which already contains
structure:SimpleObjectAttributeGroup
. Other types from the
niem-xs
namespace could be used if needed to support
numeric or other types.xs:complexType name='AddressCategoryCodeType'>
<xs:annotation>
<xs:documentation>A data type for a kind of address.</xs:documentation>
<xs:annotation>
</xs:simpleContent>
<xs:restriction base='niem-xs:token'>
<xs:enumeration value="business">
<xs:annotation>
<xs:documentation>business</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="registered office">
<xs:annotation>
<xs:documentation>registered office</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="residential">
<xs:annotation>
<xs:documentation>residential</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="residential or business">
<xs:annotation>
<xs:documentation>residential or business</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:enumeration value="unspecified">
<xs:annotation>
<xs:documentation>unspecified</xs:documentation>
<xs:annotation>
</xs:enumeration>
</xs:restriction>
</xs:simpleContent>
</xs:complexType> </
This is currently supported by the NDR and requires no changes, but has never been explicitly covered as an available option in NIEM IEPD developer resources or training. This scenario should be covered in NDR tests to ensure it remains valid for EXT schemas.
niemopen/niem-naming-design-rules#30
There are some code types in NIEM that repeat the same enumeration with different definitions. A new rule is being proposed to require that enumerations are unique within a type. For code sources outside of NIEM where duplication cannot be resolved, definitions of overlapping codes will be concatenated.
Type | FacetValue | Definition |
---|---|---|
usps:StreetSuffixCodeSimpleType | OVAL | OVAL |
usps:StreetSuffixCodeSimpleType | PARK | PARK |
usps:StreetSuffixCodeSimpleType | PARK | PARKS |
usps:StreetSuffixCodeSimpleType | PASS | PASS |
Type | FacetValue | Definition |
---|---|---|
usps:StreetSuffixCodeSimpleType | OVAL | OVAL |
usps:StreetSuffixCodeSimpleType | PARK | PARK; PARKS |
usps:StreetSuffixCodeSimpleType | PASS | PASS |
There are 16 types in NIEM 5.2 that do not have unique enumerations:
Canada Post and United States Postal Service street codes, DEA drug codes, FIPS county codes, and hazmat codes cannot be modified at the source and will either need to have concatenated definitions or other workarounds. The other codes may be able to be adjusted to remove the overlapping code definitions directly.
See comments on the GitHub issue (link above) for more details on the affected types and codes.
Canada Post street direction codes provide two definitions per enumerations: one definition in English and one definition in French.
niemopen/niem-naming-design-rules#12
The NTAC proposes to make definitions required for all pattern facets.
NIEM currently requires definitions for enumeration facets only. Definitions for pattern facets should provide extra human-readable documentation to make the regular expressions easier to interpret.
The impact to the data model would be low. There are 438 patterns in NIEM 5.2:
Type | Pattern |
---|---|
ag_codes:TaxIdentificationIDSimpleType | [0-9]{9} |
ag_codes:CropYearSimpleType | ^([1][9]\d\d|[2]\d\d\d)$ |
biom:LipPatternSimpleType | <> |
mo:MILSTD2525-C-SIDC-SimpleType | [A-Z0-9\-]{15} |
mo:MILSTD2525-B-SIDC-SimpleType | [A-Z0-9\-]{15} |
https://github.com/niemopen/ntac-admin/discussions/58
CMF either needs to be aware of some parts of the
structures
namespace, or we need a
nc:ObjectType
in the model because domains and messages
should be allowed to augment the root-level ObjectType
.
Notes:
u:SuperType
.nc:CodeType
contains three attributes from the
code-lists-instance utility namespacestructures
namespace
structures
, then we could say that
nc:PersonType
or j:PersonEyeColorCode
contains
structures:metadata
.structures
-aware, we could say:
nc:Person
contains structures:id
and
hs:Caregiver
contains structures:ref
.nc:PersonType
contains structures:uri
as
required and not have structures:uri
appear elsewhere.niemopen/niem-naming-design-rules#5
The NTAC proposes to require that the representation term "Adapter" appear at the end of all adapter property and type names.
NIEM already has representation terms for associations, augmentations, and metadata. Adding representation term "Adapter" for properties and types would make it clear which components contain non-conformant content from external standards. A few properties and types currently in NIEM already follow this convention.
The following is a list of adapter properties and types from NIEM 5.2. These names, unless otherwise noted, would be updated. Property names would end in "Adapter"; type names would end in "AdapterType".
Property | Type | Notes |
---|---|---|
ag:LocationLineStringCoordinates | geo:LineStringType | |
ag:LocationMultiSurfaceCoordinates | geo:MultiSurfaceType | |
ag:LocationPointCoordinates | geo:PointType | |
ag:LocationPolygonCoordinates | geo:PolygonType | |
cbrn:SpecialEventSecurityArea | geo:PolygonType | |
edxl-cap:AlertAdapter | edxl-cap:AlertAdapterType | No change |
edxl-de:DistributionElementAdapter | edxl-de:DistributionElementAdapterType | No change |
edxl-have:HaveAdapter | edxl-have:HaveAdapterType | No change |
geo:AreaCurve | geo:CurveType | |
geo:AreaEnvelope | geo:EnvelopeType | |
geo:AreaPoint | geo:PointType | |
geo:AreaPolygon | geo:PolygonType | |
geo:AreaRegionGeometry | geo:GeometryType | |
geo:Ellipse | geo:EllipseType | |
geo:Feature | geo:FeatureType | |
geo:Geometry | geo:GeometryType | |
geo:LocationFeature | geo:FeatureType | |
geo:LocationGeometry | geo:GeometryType | |
geo:LocationGeospatialPoint | geo:PointType | |
mo:WaypointPoint | geo:PointType | |
mo:WGS84LocationEllipse | mo:WGS84EllipseType | |
mo:WGS84LocationExternalPolygon | mo:WGS84ExternalPolygonType | |
mo:WGS84LocationLineString | mo:WGS84LineStringType | |
mo:WGS84LocationPoint | mo:WGS84LocationPointType |
structures:AdapterType
niemopen/niem-naming-design-rules#4
The NTAC proposes to create new type AdapterType
in the
structures
namespace to serve as the parent type for all
adapter types in NIEM.
Types in the structures namespace can typically bew used to identify what basic kind of type something is (an object type, an association type, a metadata type, etc.). Adapter types do not follow the same pattern. They require the use of a special attribute to identify themselves as adapters.
The basic category of NIEM classes can currently be identified as described in the table below:
Type | Rule |
---|---|
association types | Extend structures:AssociationType or a derivative |
augmentation types | Extend structures:AugmentationType |
metadata types | Extend structures:MetadataType |
object types | Extend structures:ObjectType or a derivative |
adapter types | Extend structures:ObjectType and use appinfo:externalAdapterTypeIndicator="true" |
structures:AdapterType
to serve as the parent
type for all adapter types defined in NIEM.appinfo:externalAdapterTypeIndicator
from the appinfo utility namespace.- <xs:complexType name="GeometryType" appinfo:externalAdapterTypeIndicator="true">
+ <xs:complexType name="GeometryType">
<xs:complexContent>- <xs:extension base="structures:ObjectType">
+ <xs:extension base="structures:AdapterType">
<xs:sequence>
<xs:element ref="gml:AbstractGeometry" minOccurs="1" maxOccurs="1"/>
</xs:sequence>
</xs:extension>
</xs:complexContent> </xs:complexType>
Low. The declaration of 20 types currently in NIEM would need to be adjusted, which would be handled by model maintainers.
niemopen/niem-naming-design-rules#35
The NTAC is proposing to prohibit multiple augmentations from the same namespace on the same object in message instances.
An augmentation is a container that lets us plug additional properties into existing types from other namespaces - types that we cannot modify ourselves. These containers on their own have no meaning, and they do not appear in the NIEM RDF or NIEM JSON-LD representations.
Because augmentation containers have no meaning on their own and because it would not be possible to reconstruct multiple containers correctly when transforming from NIEM RDF or NIEM JSON to NIEM XML, the NTAC proposes to prohibit multiple augmentations from the same namespace on the same object.
Notes:
j:DriverLicense
(valid)hs:HealthInsurance
(valid)ext:Message>
<nc:Person>
<nc:PersonFullName>Alice Smith</nc:PersonFullName>
<nc:PersonName>
</j:PersonAugmentation>
<j:DriverLicense>
<<!-- ... -->
j:DriverLicense>
</j:DriverLicense>
<<!-- ... -->
j:DriverLicense>
</j:PersonAugmentation>
</hs:PersonAugmentation>
<hc:HealthInsurance>
<<!-- ... -->
hc:HealthInsurance>
</hc:HealthInsurance>
<<!-- ... -->
hc:HealthInsurance>
</hs:PersonAugmentation>
</ext:Message> </
Notes:
ext:Message>
<nc:Person>
<nc:PersonFullName>Alice Smith</nc:PersonFullName>
<nc:PersonName>
</j:PersonAugmentation>
<j:DriverLicense>
<<!-- ... -->
j:DriverLicense>
</j:PersonAugmentation>
</j:PersonAugmentation> <!-- Invalid: j:PersonAugmentation should not appear twice -->
<j:DriverLicense>
<<!-- ... -->
j:DriverLicense>
</j:PersonAugmentation>
</hs:PersonAugmentation>
<hc:HealthInsurance>
<<!-- ... -->
hc:HealthInsurance>
</hs:PersonAugmentation>
</ext:Message> </
To correct this example, the second j:DriverLicense
object should be moved under the first j:PersonAugmentation
container and the second j:PersonAugmentation
container
should be removed.
This change is expected to have low impact. It does not affect the data model, and it is not expected to be a meaningful or frequently-used feature in instance messages. Augmentations can contain multiple instances of properties, so they can be consolidated if needed.
niem-open/niem-naming-design-rules#8
NIEM currently requires the use of ids and special metadata reference attributes to link metadata and the objects to which they apply. While this will continue to be necessary to link metadata consisting of elements to NIEM data properties (elements that carry attributes and a value), it is not necessary for object properties, which can contain the metadata directly or via augmentation.
The NTAC proposes to allow metadata contained in an object to be applicable to that object without the need for references. The NTAC also proposes to support metadata specialization and augmentation, treating metadata types more like regular object types.
NIEM currently requires that object properties and data properties
use the structures:metadata
or
structures:relationshipMetadata
attributes in order to
create a relationship between those properties and their applicable
metadata.
According to the NDR, adding metadata directly to a type creates no
special meaning or relationship to the metadata. For example, adding
nc:Metadata
to nc:PersonType
does not mean
that the metadata actually applies to the person that contains it; only
the use of one of the structures
metadata attributes would
make that metadata applicable to that person.
Metadata types are also required to extend
structures:MetadataType
. This prevent specialization and
does not allow one metadata type to build off of another via derivation.
Metadata types are also prohibited from creating augmentation points.
Brand new metadata types must be created even if there are no special
semantics required.
Notes:
nc:MetadataType
.nc:Person
property in the message, nc:Person
is required to use the structures:metadata
attribute (or
the structures:relationshipMetadata
attribute) and
reference the ids of any applicable metadata objects in the
message.ext:Message>
<nc:Person structures:metadata="m1 m2">
<nc:PersonBirthDate>
<nc:Date>1945-12-01</nc:Date>
<nc:PersonBirthDate>
</nc:PersonName>
<nc:PersonFullName>John Doe</nc:PersonFullName>
<nc:PersonName>
</nc:Person>
</
nc:Metadata structures:id="m1">
<nc:SourceText>Adam Barber</nc:SourceText>
<nc:ReportedDate>
<nc:Date>2005-04-26</nc:Date>
<nc:ReportedDate>
</nc:Metadata>
</
j:Metadata structures:id="m2">
<j:CriminalInformationIndicator>true</j:CriminalInformationIndicator>
<j:Metadata>
</ext:Message> </
The NTAC proposes the following changes to simplify the use of metadata in messages:
structures:metadata
attribute.appinfo:appliesToProperties
and
appinfo:appliesToTypes
attributes on metadata, as these are
rarely use and applicability can be provided by adding metadata to the
types.structures:relationshipMetadata
attribute on object properties.structures:metadata
and
structures:relationshipMetadata
attributes on data
properties.structures:id
. The
root of a message may need to carry metadata objects that are linked to
via ids / references but are not applicable to the message itself.Under the new proposal, metadata can be added directly to types just like other kinds of properties. To add metadata to a type in another namespace, extension or augmentation may be used.
The following example shows metadata attached to a person via augmentation.
Notes:
nc:MetadataType
rather than
creating its own independent j:MetadataType
.nc:PersonType
to add nc:Metadata
. This
metadata now applies to the person object in which it is contained
without the need for ids or references.ext:Message>
<nc:Person>
<nc:PersonBirthDate>
<nc:Date>1945-12-01</nc:Date>
<nc:PersonBirthDate>
</nc:PersonName>
<nc:PersonFullName>John Doe</nc:PersonFullName>
<nc:PersonName>
</ext:PersonAugmentation>
<nc:Metadata>
<nc:SourceText>Adam Barber</nc:SourceText>
<nc:ReportedDate>
<nc:Date>2005-04-26</nc:Date>
<nc:ReportedDate>
</j:MetadataAugmentation>
<j:CriminalInformationIndicator>true</j:CriminalInformationIndicator>
<j:MetadataAugmentation>
</nc:Metadata>
</ext:PersonAugmentation>
</nc:Person>
</ext:Message> </
The proposal updates metadata in NIEM so that they are treated much
more similarly to regular objects. Referencing via
structures:id
and structures:ref
is supported
for objects and would now also be supported for metadata.
The following example adds a second person to the message. In order to have the same metadata apply to both people without duplicating information, a reference to the metadata object can be added to each person:
Notes:
nc:Metadata
is placed under the message root and
contains attribute structures:id
structures:ref
attribute instead of the
structures:metadata
attribute to link to the applicable
metadata.nc:Metadata
contains structures:id
(indicates that it is being used via references), it is not applicable
to the root of the message.ext:Message>
<nc:Person>
<nc:PersonBirthDate>
<nc:Date>1945-12-01</nc:Date>
<nc:PersonBirthDate>
</nc:PersonName>
<nc:PersonFullName>John Doe</nc:PersonFullName>
<nc:PersonName>
</ext:PersonAugmentation>
<j:Metadata structures:ref="m3">
<ext:PersonAugmentation>
</nc:Person>
</nc:Person>
<nc:PersonName>
<nc:PersonFullName>Alice Smith</nc:PersonFullName>
<nc:PersonName>
</ext:PersonAugmentation>
<j:Metadata structures:ref="m3">
<ext:PersonAugmentation>
</nc:Person>
</nc:Metadata structures:id="m3">
<nc:SourceText>Adam Barber</nc:SourceText>
<nc:ReportedDate>
<nc:Date>2005-04-26</nc:Date>
<nc:ReportedDate>
</j:MetadataAugmentation>
<j:CriminalInformationIndicator>true</j:CriminalInformationIndicator>
<j:MetadataAugmentation>
</nc:Metadata>
</ext:Message> </
While metadata objects can be added to class types (types that
contain properties), the same is not true for datatypes (types that
contain a value and attributes). While the Attribute Wildcards proposal in this
document enables adding new attributes to existing object and data
properties, attaching metadata consisting of object properties
(elements) still requires special handling. NIEM datatypes will continue
to carry the structures:metadata
and
structures:relationshipMetadata
attributes so that data
properties can link to applicable metadata.
The example below shows how to use references to apply metadata to a mix of object and data properties.
Notes:
structures:ref
)nc:PersonFullName
continues to use
attribute structures:metadata
to link to the metadata.nc:Metadata
does not apply to the root
of the message since it contains attribute structures:id
(indicates it is being included there for the purposes of
referencing)ext:Message>
<nc:Person>
<nc:PersonBirthDate>
<nc:Date>1945-12-01</nc:Date>
<nc:PersonBirthDate>
</nc:PersonName>
<nc:PersonFullName structures:metadata="m3">John Doe</nc:PersonFullName>
<nc:PersonName>
</ext:PersonAugmentation>
<j:Metadata structures:ref="m3">
<ext:PersonAugmentation>
</nc:Person>
</nc:Person>
<nc:PersonName>
<nc:PersonFullName>Alice Smith</nc:PersonFullName>
<nc:PersonName>
</ext:PersonAugmentation>
<j:Metadata structures:ref="m3">
<ext:PersonAugmentation>
</nc:Person>
</nc:Metadata structures:id="m3">
<nc:SourceText>Adam Barber</nc:SourceText>
<nc:ReportedDate>
<nc:Date>2005-04-26</nc:Date>
<nc:ReportedDate>
</j:MetadataAugmentation>
<j:CriminalInformationIndicator>true</j:CriminalInformationIndicator>
<j:MetadataAugmentation>
</nc:Metadata>
</ext:Message> </
NIEM currently contains 11 metadata types which would need to reviewed and potentially refactored.
The NBAC may want to add nc:Metadata
to certain common
objects like nc:PersonType
and
nc:ActivityType
.
Message designers would need to treat metadata differently in messages starting with 6.0 These changes would make metadata handling much more similar to other kinds of data, but that would be a change from previous requirements.
niem-open/niem-naming-design-rules#21
The NTAC proposes to leverage the RDF-star and JSON-LD-star specifications in order to simplify the representation of relationship metadata in NIEM RDF and NIEM JSON.
RDF triples consist of statements with a subject, predicate, and object.
Example:
@prefix nc: <http://release.niem.gov/niem/niem-core/5.0> .
@prefix xs: <http://www.w3.org/2001/XMLSchema> .
# Person node 0 has a birth date, with value Date node 0.
_:P0 nc:PersonBirthDate _:D0 .
# Person node 0 also has a name, with value Name node 0.
_:P0 nc:PersonName _:N0 .
# Date node 0 has a date with literal value "1950-01-01".
_:D0 nc:Date "1950-01-01"^^xs:date .
# Name node 0 has a given name, with literal value "Alice".
_:N0 nc:PersonGivenName "Alice" .
# Name node 0 also has a surname, with literal value "Smith".
_:N0 nc:PersonSurName "Smith" .
The example above may be shortened as follows, with semicolons used to concatenate statements with the same subject:
@prefix nc: <http://release.niem.gov/niem/niem-core/5.0> ;
xs: <http://www.w3.org/2001/XMLSchema> .
# Person node 0 has a birth date (with value Date node 0) and a name (with value Name node 0).
_:P0 nc:PersonBirthDate _:D0 ;
nc:PersonName _:N0 .
# Date node 0 has a date with literal value "1950-01-01".
_:D0 nc:Date "1950-01-01"^^xs:date .
# Name node 0 has a given name (with literal value "Alice") and a surname (with literal value "Smith").
_:N0 nc:PersonGivenName "Alice" ;
nc:PersonSurName "Smith" .
Triples work to represent most NIEM data but cannot by themselves represent relationship metadata. The NDR currently uses the N-Quads notation (N-Triples plus graphs) to represent relationship metadata. N-Quads allow one or more triples to be named together as a graph. The graph (consisting of multiple tripes) may then serve as a subject of another triple.
@prefix nc: <http://release.niem.gov/niem/niem-core/5.0> .
@prefix xs: <http://www.w3.org/2001/XMLSchema> .
# These two statements about Person Node 0 have collectively been given the graph name "G0".
_:P0 nc:PersonBirthDate _:D0 ;
nc:PersonName _:N0 G0.
_:D0 nc:Date "1950-01-01"^^xs:date .
_:N0 nc:PersonGivenName "Alice" ;
nc:PersonSurName "Smith" .
# Reported date (with value "2023-06-01") is metadata that applies to the triples defined by graph G0.
_:G0 nc:ReportedDate "2023-06-01"^^xs:date .
The NTAC proposes to leverage RDF-star (and its JSON-LD-star counterpart) to support applying metadata to statements directly, without the need to create named graphs to identify one or more sets of statements. This can be used on statements with node objects and with literal objects, so it works in NIEM for both object properties and data properties.
RDF-star introduces the following concepts:
A quoted triple wraps a triple in double angle brackets:
<<_:P0 nc:PersonBirthDate _:D0 >> nc:ReportedDate "2023-06-01"^^xs:date .
This statement contains the following triples:
_:P0 nc:PersonBirthDate _:D0
<<_:P0 nc:PersonBirthDate _:D0 >> nc:ReportedDate "2023-06-01"^^xs:date .
Note that the RDF does not assert that the birth date of the person represented by node P0 is the value represented by node D0. The RDF only asserts that the information was reported on 2023-06-01. In order to assert that this (1) is the birth date of the person and (2) that the information ws reported on 2023-06-01, it is necessary to explicitly represent #1 as its own triple:
_:P0 nc:PersonBirthDate _:D0 .
<<_:P0 nc:PersonBirthDate _:D0 >> nc:ReportedDate "2023-06-01"^^xs:date .
This updated statement contains the following triples:
_:P0 nc:PersonBirthDate _:D0
(from
line 1)_:P0 nc:PersonBirthDate _:D0
(from line
2)<<_:P0 nc:PersonBirthDate _:D0 >> nc:ReportedDate "2023-06-01"^^xs:date .
(from line 2)RDF-star provides annotation syntax as a shorthand to eliminate the need to duplicate statement information so that the original statement and the metadata about the statement may be asserted.
The following asserts both the birth date of the person and the reported date of the birth date of the person:
_:P0 nc:PersonBirthDate _:D0 {| nc:ReportedDate "2023-06-01"^^xs:date |} .
In NIEM JSON-LD, annotation metadata would be represented via a new
@annotation
property:
{
"nc:PersonBirthDate": {
"nc:Date": "1950-01-01"
},
"nc:PersonName": {
"nc:PersonGivenName": "Alice",
"nc:PersonSurName": "Smith"
},
"@annotation": {
"nc:ReportedDate": "2023-06-01"
}
}
The impact is expected to be low. This does not affect content within the data model, only how relationship metadata is to be represented in NIEM RDF and NIEM JSON. Tool support for RDF-star includes Apache Jena and Eclipse RDF4J.
niemopen/niem-naming-design-rules#6
The NTAC is considering a proposal to simplify roles within NIEM. The
proposal will remove the special syntax and rules related to roles.
Rather than requiring special RoleOf
properties to be
contained within role types, roles would be implemented via
specialization and the use of the existing structures:uri
attribute to link related objects to the same entity.
Roles are used in NIEM to represent a non-exclusive function or part played by an object. An object may have one or more roles. For example, victim, witness, and officer can all be roles of a person. One person may end up playing each of these roles within the same message.
Roles differ from specialization in NIEM, which is reserved for typically-exclusive special functions of an object. For example, a vehicle (e.g., a car) and an aircraft are specializations of a conveyance, which is a specialization of an item.
Exclusivity is the key difference between roles and specialization. One person might end up participating in multiple roles within the same message (e.g., victim, witness, and officer), while a vehicle is a specialization because it is typically not also a plane, train, firearm, or cellphone at the same time.
There is some special syntax in NIEM used for roles that allow us to link related occurrences together in messages so that it is clear they each represent the same entity.
The following are the main techniques in NIEM for creating new classes based on existing classes.
Technique | Use case | Schema mechanism |
---|---|---|
Roles | Non-exclusive function of an object. Example: Victim, witness, officer, teacher, parent, child are all roles of a person object. |
Extend structures:ObjectType and add one or more
RoleOf properties |
Augmentation | Mechanism used to "drop in" additional content into an existing type
from another namespace without actually making changes to the original.
Example: Justice, Emergency Management, Biometrics, and other NIEM domains each define their own augmentation for nc:PersonType with common person-related properties within their subject area. |
Substitution groups |
Specialization | Exclusive special function of an object. Example: A vehicle (car) is a specialization of conveyance, which is a specialization of item. |
Type extension |
The rules and syntax for the current role mechanism fulfills two requirements:
structures:uri
attribute to link
related roles back to the same object.The syntax for roles was developed as part of NIEM 1.0. The
structures:uri
attribute was not added until NIEM 4.0. This
attribute can be leveraged to meet the same requirements without the
special role rules or syntax.
This example is based on the Crash Driver IEPD using in NIEM
training. Role types are modified to extend nc:PersonType
and to remove the nc:RoleOfPerson
property.
Notes:
CrashDriverType
changes from
structures:ObjectType
to nc:PersonType
.nc:RoleOfProperty
is no longer needed.structures:uri
attribute is inherited by
CrashDriverType
and by any other type that extends from
structures:ObjectType
(which is the parent of
nc:PersonType
).
<xs:complexType name="CrashDriverType">
<xs:annotation>
<xs:documentation>A data type for a motor vehicle driver involved ...</xs:documentation>
</xs:annotation>
<xs:complexContent>- <xs:extension base="structures:ObjectType">
+ <xs:extension base="nc:PersonType">
<xs:sequence>- <xs:element ref="nc:RoleOfPerson" minOccurs="0" maxOccurs="unbounded"/>
<xs:element ref="j:DriverLicense" minOccurs="0" maxOccurs="unbounded"/>
<!-- ... -->
</xs:sequence>
</xs:extension>
</xs:complexContent> </xs:complexType>
Notes:
structures:id
and
structures:ref
is replaced by the use of
structures:uri
.nc:RoleOfPerson
properties are removed.<exch:CrashDriverInfo>
- <nc:Person structures:id="P01">
+ <nc:Person structures:uri="P01">
<nc:PersonBirthDate>
<nc:Date>1890-05-04</nc:Date>
</nc:PersonBirthDate>
<nc:PersonName nc:personNameCommentText="copied">
<nc:PersonGivenName>Peter</nc:PersonGivenName>
<nc:PersonMiddleName>Death</nc:PersonMiddleName>
<nc:PersonMiddleName>Bredon</nc:PersonMiddleName>
<nc:PersonSurName>Wimsey</nc:PersonSurName>
</nc:PersonName>
</nc:Person>
<j:Crash>
<j:CrashVehicle>- <j:CrashDriver>
- <nc:RoleOfPerson structures:ref="P01" xsi:nil="true"/>
+ <j:CrashDriver structures:uri="P01">
<j:DriverLicense>
<j:DriverLicenseCardIdentification>
<nc:IdentificationID>A1234567</nc:IdentificationID>
</j:DriverLicenseCardIdentification>
</j:DriverLicense>
</j:CrashDriver>
</j:CrashVehicle>- <j:CrashPerson>
- <nc:RoleOfPerson structures:ref="P01" xsi:nil="true"/>
+ <j:CrashPerson structures:uri="P01">
<j:CrashPersonInjury>
<nc:InjuryDescriptionText>Broken Arm</nc:InjuryDescriptionText>
</j:CrashPersonInjury>
</j:CrashPerson>
</j:Crash>
<j:Charge structures:id="CH01">
<j:ChargeDescriptionText>Furious Driving</j:ChargeDescriptionText>
<j:ChargeFelonyIndicator>false</j:ChargeFelonyIndicator>
</j:Charge>
<j:PersonChargeAssociation>- <nc:Person structures:ref="P01" xsi:nil="true"/>
+ <nc:Person structures:uri="P01" xsi:nil="true"/>
<j:Charge structures:ref="CH01" xsi:nil="true"/>
</j:PersonChargeAssociation></exch:CrashDriverInfo>
Notes:
structures:uri
has been available since NIEM 4.0 and
can already be used to link multiple properties to the same entity.nc:RoleOfPerson
properties from the
message.<exch:CrashDriverInfo>
<j:Crash>
<j:CrashVehicle>
<j:CrashDriver structures:uri="P01">- <nc:RoleOfPerson>
<nc:PersonBirthDate>
<nc:Date>1890-05-04</nc:Date>
</nc:PersonBirthDate>
<nc:PersonName nc:personNameCommentText="copied">
<nc:PersonGivenName>Peter</nc:PersonGivenName>
<nc:PersonMiddleName>Death</nc:PersonMiddleName>
<nc:PersonMiddleName>Bredon</nc:PersonMiddleName>
<nc:PersonSurName>Wimsey</nc:PersonSurName>
</nc:PersonName>- </nc:RoleOfPerson>
<j:DriverLicense>
<j:DriverLicenseCardIdentification>
<nc:IdentificationID>A1234567</nc:IdentificationID>
</j:DriverLicenseCardIdentification>
</j:DriverLicense>
</j:CrashDriver>
</j:CrashVehicle>
<j:CrashPerson structures:uri="P01">- <nc:RoleOfPerson>
<nc:PersonBirthDate>
<nc:Date>1890-05-04</nc:Date>
</nc:PersonBirthDate>
<nc:PersonName nc:personNameCommentText="copied">
<nc:PersonGivenName>Peter</nc:PersonGivenName>
<nc:PersonMiddleName>Death</nc:PersonMiddleName>
<nc:PersonMiddleName>Bredon</nc:PersonMiddleName>
<nc:PersonSurName>Wimsey</nc:PersonSurName>
</nc:PersonName>- </nc:RoleOfPerson>
<j:CrashPersonInjury>
<nc:InjuryDescriptionText>Broken Arm</nc:InjuryDescriptionText>
</j:CrashPersonInjury>
</j:CrashPerson>
</j:Crash>
<j:Charge structures:id="CH01">
<j:ChargeDescriptionText>Furious Driving</j:ChargeDescriptionText>
<j:ChargeFelonyIndicator>false</j:ChargeFelonyIndicator>
</j:Charge>
<j:PersonChargeAssociation>
<nc:Person structures:uri="P01">
<nc:PersonBirthDate>
<nc:Date>1890-05-04</nc:Date>
</nc:PersonBirthDate>
<nc:PersonName nc:personNameCommentText="copied">
<nc:PersonGivenName>Peter</nc:PersonGivenName>
<nc:PersonMiddleName>Death</nc:PersonMiddleName>
<nc:PersonMiddleName>Bredon</nc:PersonMiddleName>
<nc:PersonSurName>Wimsey</nc:PersonSurName>
</nc:PersonName>
</nc:Person>
<j:Charge structures:ref="CH01" xsi:nil="true"/>
</j:PersonChargeAssociation></exch:CrashDriverInfo>
Roles are part of the learning curve for new users. They require special instruction not only when new users are building their own schemas, but also when they are just browsing the content of NIEM. Users can look at roles like witness or student and not know that person properties are already available.
Roles are also easy to forget to model correctly for more experienced users. QA checks catch a lot of cases where specialization is used in place of roles.
The more-complicated special role syntax is always required, no matter whether or not messages will leverage the referencing capabilities to link related properties together. There is no way to opt out of the syntax if it is not needed.
The current NIEM role technique results in inconsistencies..
RoleOf
properties are only available when custom role types
must be created.
For example, the Human Services domain had special student-related
properties, so they created hs:StudentType
as a role of
nc:PersonType
. Student properties will contain
nc:RoleOfPerson
. However, there were no special properties
required for caregiver, so property hs:Caregiver
is
directly of type nc:PersonType
.
This makes the representation of people within the model inconsistent. You have to look at the declaration of the type to determine if an entity is-a person, or has-a role-of-person. This can make it more difficult to process and query data:
hs:Student
-> nc:RoleOfPerson
->
nc:PersonName
hs:Caregiver
-> nc:PersonName
Without looking at the schema or memorizing the model, it's almost
impossible to know if j:Counselor
is defined as a person or
as a role of a person.
The custom role rules and syntax may no longer provide enough
benefit to justify the complexities now that the
structures:uri
attribute is available. This attribute can
be leveraged already to link related properties together, whether or not
the role syntax remains in place.
Almost 100 types in NIEM 5.2 contain a RoleOf property and would need to be adjusted. This would likely affect many migrated message schemas and their instances.
RoleOf properties allow the validation of additional cardinality
constraints through XML Schema. Because structure:id
and
structures:ref
can be used with RoleOf
properties, properties on the object of a role type can be marked as
required, but do not have to appear when structures:ref
is
used.
From the first instance example, nc:PersonName
and
nc:PersonBirthDate
could be set as required
(minOccurs="1"
) under nc:PersonType
, but
wouldn't have to appear under j:CrashDriver
or
j:CrashPerson
because these two properties use
structures:ref
to point to where their data is
contained.
Removing these special RoleOf
properties would mean that
nc:PersonName
and nc:PersonBirthDate
would
need to be set as optional (minOccurs="0"
) if they are
supposed to appear under nc:Person
but not appear under
j:CrashDriver
or j:CrashPerson
. This would be
similar to other business rules that cannot be implemented via XML
Schema and rely on validation through Schematron or other application
validation.
It may also be possible in the future to generated customized message schemas that hardcode exactly which properties can appear by replacing the object-oriented nature of the reusable NIEM reference data model with flattened types for improved validation.
niemopen/niem-naming-design-rules#20
The NTAC proposes to expand the use of attribute wildcards, currently used in NIEM to support security markup (IC-ISM and IC-NTK), to all NIEM-conformant and external attributes. This would allow any schema-resolvable attribute to be attached to almost any NIEM element property in messages. Message designers would have the ability to subset the wildcards down to only attributes within specific namespaces, or to remove the wildcard entirely if they wish to prevent the use of attributes in messages that are not declared in the message schemas.
While attributes can be added to existing types via augmentation or extension, attribute wildcards provide an easy means to extend existing data properties and datatypes with new attributes, which is otherwise not easy to do.
See the examples/attribute-wildcards
subfolder for the wildcard example extension schema, the modified
structures schema (structures-modified.xsd
), the NIEM
subset schemas updated to refer to the modified structures schema, and a
valid and an invalid sample message.
NIEM supports extensibility for objects via type extension, augmentation, and adapters. None of these solutions, however, work very well to support attaching additional NIEM-conformant or external attributes to existing NIEM data properties (properties that carry a value, like a string or a number or a boolean).
The NIEM metadata mechanism can be used to link NIEM data properties to metadata objects that contain new attributes, but this can be a very cumbersome approach in message instances as it could require a metadata property counterpart for each NIEM data property that needs to carry custom attribute information.
ext:Message
< xmlns:ext="http://www.example.com/wildcard-attributes/extension"
xmlns:nc='http://release.niem.gov/niem/niem-core/5.0/'
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.example.com/wildcard-attributes/extension ./example.xsd">
nc:Person>
<nc:PersonName>
<nc:PersonGivenName structures:metadata="M1">...</nc:PersonGivenName>
<nc:PersonSurName structures:metadata="M2">...</nc:PersonSurName>
<nc:PersonName>
</nc:Person>
</nc:Address>
<nc:AddressFullText structures:metadata="M3">...</nc:AddressFullText>
<nc:AddressCityName structures:metadata="M4">...</nc:AddressCityName>
<nc:Address>
</
nc:Metadata structures:id="M1" ext:transliterationText="..."/>
<nc:Metadata structures:id="M2" ext:transliterationText="..."/>
<nc:Metadata structures:id="M3" ext:transliterationText="..."/>
<nc:Metadata structures:id="M4" ext:transliterationText="..."/>
<ext:Message> </
NIEM currently leverages attribute wildcards from XML Schema to
support the use of Intelligence Community Information Security Marking
(ISM) and Need-to-Know (NTK) metadata attributes on any NIEM-conformant
element. These wildcards take the form of the
xs:anyAttribute
element on the following types and
attribute group in structures.xsd
:
structures:ObjectType
structures:AssociationType
structures:MetadataType
structures:AugmentationType
structures:SimpleObjectAttributeGroup
The following is an extract from structures.xsd
, showing
the declaration of attribute wildcards on
structures:SimpleObjectAttributeGroup
and
structures:ObjectType
:
xs:attributeGroup name="SimpleObjectAttributeGroup">
<xs:annotation>
<xs:documentation>A group of attributes that are applicable to objects, to be used when defining a complex type that is an extension of a simple type.</xs:documentation>
<xs:annotation>
</xs:attribute ref="structures:id"/>
<xs:attribute ref="structures:ref"/>
<xs:attribute ref="structures:uri"/>
<xs:attribute ref="structures:metadata"/>
<xs:attribute ref="structures:relationshipMetadata"/>
<xs:attribute ref="structures:sequenceID"/>
<xs:anyAttribute namespace="urn:us:gov:ic:ism urn:us:gov:ic:ntk" processContents="lax"/>
<xs:attributeGroup>
</
xs:complexType name="ObjectType" abstract="true">
<xs:annotation>
<xs:documentation>A data type for a thing with its own lifespan that has some existence.</xs:documentation>
<xs:annotation>
</xs:sequence>
<xs:element ref="structures:ObjectAugmentationPoint" minOccurs="0" maxOccurs="unbounded"/>
<xs:sequence>
</xs:attribute ref="structures:id"/>
<xs:attribute ref="structures:ref"/>
<xs:attribute ref="structures:uri"/>
<xs:attribute ref="structures:metadata"/>
<xs:attribute ref="structures:relationshipMetadata"/>
<xs:attribute ref="structures:sequenceID"/>
<xs:anyAttribute namespace="urn:us:gov:ic:ism urn:us:gov:ic:ntk" processContents="lax"/>
<xs:complexType> </
The NTAC proposes to expand the use of attribute wildcards to any
defined schema source. The value of the processContents
attribute will be required to be "strict", meaning that a schema
definition for the attributes must be resolvable and the attributes in
messages must validate against the schema.
xs:complexType name="ObjectType" abstract="true">
<xs:sequence>
<xs:element ref="structures:ObjectAugmentationPoint" minOccurs="0" maxOccurs="unbounded"/>
<xs:sequence>
</xs:attribute ref="structures:id"/>
<xs:attribute ref="structures:ref"/>
<xs:attribute ref="structures:uri"/>
<xs:attribute ref="structures:metadata"/>
<xs:attribute ref="structures:relationshipMetadata"/>
<xs:attribute ref="structures:sequenceID"/>
<xs:anyAttribute namespace="##other" processContents="strict"/>
<xs:complexType> </
Notes:
namespace="##other"
attribute on
xs:anyAttribute
means that any attribute is allowed from
any other namespace than the current one, in this case
structures.xsd
.processContents="strict"
attribute on
xs:anyAttribute
means that the XML processor must be able
to resolve the schema identified by the namespace URI and the attributes
must be valid against that schema.ext:Message
< xmlns:ext="http://www.example.com/wildcard-attributes/extension"
xmlns:attr='http://www.example.com/wildcard-attributes/attributes'
xmlns:nc='http://release.niem.gov/niem/niem-core/5.0/'
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.example.com/wildcard-attributes/extension ./example.xsd
http://www.example.com/wildcard-attributes/attributes ./attributes.xsd">
nc:Person>
<nc:PersonName>
<nc:PersonName>
</nc:Person>
</nc:Address>
<nc:AddressFullText attr:transliterationText="...">...</nc:AddressFullText>
<nc:AddressCityName attr:transliterationText="...">...</nc:AddressCityName>
<nc:Address>
</ext:Message> </
Notes:
Attribute attr:transliterationText
is allowed to appear
in the message because:
xmlns:attr
)xsi:schemaLocation
, which accepts one or more pairs of
schema target namespace URIs and relative or absolute schema paths, each
value and pair separated by whitespace.transliterationText
attribute actual exists at the
provided schema location.This updated example is shown again below, with the differences highlighted from the current approach of using NIEM metadata:
<ext:Message
xmlns:ext="http://www.example.com/wildcard-attributes/extension"
xmlns:nc='http://release.niem.gov/niem/niem-core/5.0/'
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.example.com/wildcard-attributes/extension ./example.xsd">
<nc:Person>
<nc:PersonName>- <nc:PersonGivenName structures:metadata="M1">...</nc:PersonGivenName>
- <nc:PersonSurName structures:metadata="M2">...</nc:PersonSurName>
+ <nc:PersonGivenName attr:transliterationText="...">...</nc:PersonGivenName>
+ <nc:PersonSurName attr:transliterationText="...">...</nc:PersonSurName>
</nc:PersonName>
</nc:Person>
<nc:Address>- <nc:AddressFullText structures:metadata="M3">...</nc:AddressFullText>
- <nc:AddressCityName structures:metadata="M4">...</nc:AddressCityName>
+ <nc:AddressFullText attr:transliterationText="...">...</nc:AddressFullText>
+ <nc:AddressCityName attr:transliterationText="...">...</nc:AddressCityName>
</nc:Address>
- <nc:Metadata structures:id="M1" ext:transliterationText="..."/>
- <nc:Metadata structures:id="M2" ext:transliterationText="..."/>
- <nc:Metadata structures:id="M3" ext:transliterationText="..."/>
- <nc:Metadata structures:id="M4" ext:transliterationText="..."/>
</ext:Message>
The proposal with the attribute wildcard permits any attribute from
any namespace other than structures to appear on any NIEM-conformant
element. Message designers may want to restrict the set of attributes
that appear, or prevent them entirely. This can be done by modifying the
value of the namespace
attribute on the
xs:anyAttribute
element in structures.xsd
from
"##other" to one or more space-delimited namespace URIs, or removing the
xs:anyAttribute
element entirely.
niemopen/niem-naming-design-rules#33
The NTAC has proposed to drop the
structures:relationshipMetadata
from
structure:SimpleObjectAttributeGroup
.
niemopen/niem-naming-design-rules#34
The NTAC has proposed to drop the following attributes from
structures:AugmentationType
, as these attributes are meant
to apply to objects, not to containers of additional properties to apply
to an object:
structures:id
structures:ref
structures:uri
Augmentations in NIEM are used to attach additional properties to types defined in another namespace that cannot be modified. Augmentations are used when the semantics of the object are sufficient, but the additional data requirements are needed. For example, augmentations are used by multiple NIEM domains and by IEPD developers to attach additional domain-specific properties to the generic nc:PersonType.
As augmentations are fairly meaningless containers on their own and must be used to attach extra properties to existing types defined in other namespaces, NIEM does not consider them to be independent objects. IDs and references, linked data, and security markup do not apply to these containers; they should be used directly on the base objects themselves.
The impact of this change should be low. It affects the declaration
of AugmentationType
in the structures
namespace. It is not expected to affect many, if any, IEPDs in
practice.
niemopen/niem-naming-design-rules#19
appinfo:orderedPropertyIndicator
(or perhaps
appinfo:isOrdered
?)Integrate the following notes into the document:
xs:any
is
OKxs:choice
OKappinfo.xsd
appinfo:orderedPropertyIndicator
(or perhaps
appinfo:isOrdered
?)appinfo:relationshipPropertyIndicator
(or perhaps
appinfo:isRelationshipProperty
?)appinfo:referenceableIndicator
(or perhaps
isReferenceable
?)appinfo:referenceAttributeIndicator
(or perhaps
isReference
?)structures.xsd
@structures:onlyRef
(True for an XML
element that does not apply to its parent.)@structures:qname
(like URI)Relative URIs are blank nodes in messages with no base URI.
@xml:lang
only applies to elements with a
string-valued CSC that includes @xml:lang
This appendix contains the informative references that are used in this document.
While any hyperlinks included in this appendix were valid at the time of publication, OASIS cannot guarantee their long-term validity.
JSON-LD-star, 12 April 2023. Edited by Gregg Kellogg and Pierre-Antoine Champin. Latest editor's draft: https://json-ld.github.io/json-ld-star/. Latest published version: https://json-ld.github.io/json-ld-star/publications/2021-02-18.html
RDF-star and SPARQL-star. Dörthe Arndt, Jeen Broekstra, Bob DuCharme, Ora Lassila, Peter F. Patel-Schneider, Eric Prud'hommeaux, Ted Thibodeau, Jr., and Bryan Thompson (Amazon), Authors. Olaf Hartig, Pierre-Antoine Champin, Gregg Kellogg, and Andy Seaborne, Editors. Draft Community GroupReport, 08 December 2022, Latest editor's draft available at https://w3c.github.io/rdf-star/cg-spec/editors_draft.html.
The following individuals have participated in the creation of this document and are gratefully acknowledged:
First Name | Last Name | Company |
---|---|---|
Aubrey | Beach | Joint Staff J6 |
Jim | Cabral | InfoTrack US |
Tom | Carlson | GTRI |
Mike | Douklias | Joint Staff J6 |
Katherine | Escobar | Joint Staff J6 |
Mike | Hulme | Unisys |
Eric | Jahn | Alexandria Consulting |
Dave | Kemp | NSA |
Vamsikrishna | Kondannagari | DHS |
Peter | Madruga | GTRI |
Christina | Medlin | GTRI |
Joe | Mierwa | Mission Critical Partners |
Scott | Renner | MITRE |
Duncan | Sparrell | sFractal Consulting |
Jennifer | Stathakis | FBI |
Stephen | Sullivan | BAH |
Revision | Date | Editor | Changes Made |
---|---|---|---|
niem-6.0-arch-changes-v1.0-pn01.md | 2023-06-12 | Carlson, Medlin, Dr. Renner | Initial working draft |
Copyright © OASIS Open 2023. All Rights Reserved.
All capitalized terms in the following text have the meanings assigned to them in the OASIS Intellectual Property Rights Policy (the "OASIS IPR Policy"). The full Policy may be found at the OASIS website.
This document is published under Attribution 4.0 International (CC BY 4.0).
All contributions made to this project have been made under the OASIS Contributor License Agreement (CLA).
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published, and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this section are included on all such copies and derivative works. However, this document itself may not be modified in any way, including by removing the copyright notice or references to OASIS, except as needed for the purpose of developing any document or deliverable produced by an OASIS Technical Committee (in which case the rules applicable to copyrights, as set forth in the OASIS IPR Policy, must be followed) or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by OASIS or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and OASIS DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY OWNERSHIP RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
The name "OASIS" is a trademark of OASIS, the owner and developer of this specification, and should be used only to refer to the organization and its official outputs. OASIS welcomes reference to, and implementation and use of, specifications, while reserving the right to enforce its marks against misleading uses. Please see https://www.oasis-open.org/policies-guidelines/trademark/ for above guidance.