Fix checksum encoding issue. Proposal: do not use explicit hexBinary-type declaration in submitted records #351

New issue

Closed

opened 2026-06-03 08:29:41 +00:00 by cmo · 2 comments

cmo commented

2026-06-03 08:29:41 +00:00

Owner

Currently the RDFlib-LinkML code in dumpthings renders the notation in dlidentifiers:Checksum with explicit type declaration as python encoded string representation of the represented binary content, i.e., "41"^^xsd:hexBinary becomes "b'A'". This happens only if the explicit type declaration, ^^xsd:hexBinary is provided.

This underlying issue seems to be that range of notation is HexBinary, which defines its uri to be xsd:hexBinary. AFAICS the rdf-linkml code does get confused with an xsd:hexBinary URI, but an instance of a generic HexBinary-class. It uses the URI-registered reader to read the content, yielding a byte-string, but the generic HexBinary-instance methods to emit the content of the byte-string ---which is str--- resulting in something like b'A'.

This can be fixed by:

Not using HexBinary in the notation-range, but using xsd:hexBinary instead.
Implementing support for HexBinary-instances in the python code generators
Not providing the explicit ^^xsd:hexBinary-type delcaration in the TTL record.

Proposed solution: 3: To quickly fix the checksum encoding issue, the explicit hexBinary-type declaration should be removed from the checksum in the submitted TTL-record (this would drop the format check because checksum would be treated as string, but it can be done without re-releasing schemas or adding schema specific code to linkml libraries).

Currently the RDFlib-LinkML code in dumpthings renders the `notation` in `dlidentifiers:Checksum` with explicit type declaration as python encoded string representation of the represented binary content, i.e., `"41"^^xsd:hexBinary` becomes `"b'A'"`. This happens only if the explicit type declaration, `^^xsd:hexBinary` is provided. This underlying issue seems to be that range of `notation` is `HexBinary`, which defines its uri to be `xsd:hexBinary`. AFAICS the rdf-linkml code does get confused with an `xsd:hexBinary` URI, but an instance of a generic `HexBinary`-class. It uses the URI-registered reader to read the content, yielding a byte-string, but the generic `HexBinary`-instance methods to emit the content of the byte-string ---which is `str`--- resulting in something like `b'A'`. This can be fixed by: 1. Not using `HexBinary` in the `notation`-range, but using `xsd:hexBinary` instead. 2. Implementing support for `HexBinary`-instances in the python code generators 3. Not providing the explicit `^^xsd:hexBinary`-type delcaration in the TTL record. **Proposed solution: 3**: To quickly fix the checksum encoding issue, the explicit hexBinary-type declaration should be removed from the checksum in the submitted TTL-record (this would drop the format check because checksum would be treated as string, but it can be done without re-releasing schemas or adding schema specific code to linkml libraries).

cmo referenced this issue from www/pool.psychoinformatics.de-ui

2026-06-03 08:30:02 +00:00

Fix checksum encoding issue: do not use explicit hexBinary-type declaration in submitted records #27

jsheunis commented

2026-06-22 10:34:03 +00:00

Owner

One of the suggested fixes involves changing behaviour in shacl-vue, I will comment on that here.

Even if the HexBinary type is specified in LinkML, shacl-vue does not see it at all. The incoming SHACL says:

    [ sh:datatype xsd:hexBinary ;
        sh:description "Lower case hexadecimal encoded checksum digest value." ;
        sh:maxCount 1 ;
        sh:minCount 1 ;
        sh:name "Notation" ;
        sh:nodeKind sh:Literal ;
        sh:path skos:notation ;
        sh:pattern "^[a-fA-F0-9]+$" ],

Introducing alternative behaviour in shacl-vue just for the xsd:HexBinary literal seems like a patchy solution. Also, this would have to be applied to shacl-tulip, which has the code that handles saving form inputs to named/literal/blank nodes in the graph, which means unnecessarily changing dependency versions just for a weird edge case

That being said, one possible other way to address this edge case could be via shacl-vue config using the update_shapes option, by changing the incoming shacl to remove the sh:datatype xsd:hexBinary; from that property shape, or changing it to xsd:string. I am not sure, however, what knock-on effects this might have; it will have to be tested. The shacl shape still has the sh:pattern, i.e. UI-side field validation should still be functional.

After looking into this: it doesn't seem to help. I used update_shapes to change the notation's datatype from xsd:hexBinary to xsd:string (at first, setting it to null caused the field not to be displayed in the UI). This worked fine in the frontend, but the submission still had the xsd:hexBinary literal type included. I think this actually highlights an inconsistency in shacl-vue->shacl-tulip; shacl-tulip is possibly not receiving the changes from update_shapes and hence not taking it into account during submission. I am unsure though whether this was intentional or not: will need more investigation...

Either way, it does not help with a workaround solution to the original problem.

One of the suggested fixes involves changing behaviour in `shacl-vue`, I will comment on that here. Even if the `HexBinary` type is specified in LinkML, `shacl-vue` does not see it at all. The incoming SHACL says: ```turtle [ sh:datatype xsd:hexBinary ; sh:description "Lower case hexadecimal encoded checksum digest value." ; sh:maxCount 1 ; sh:minCount 1 ; sh:name "Notation" ; sh:nodeKind sh:Literal ; sh:path skos:notation ; sh:pattern "^[a-fA-F0-9]+$" ], ``` Introducing alternative behaviour in `shacl-vue` just for the `xsd:HexBinary` literal seems like a patchy solution. Also, this would have to be applied to `shacl-tulip`, which has the code that handles saving form inputs to named/literal/blank nodes in the graph, which means unnecessarily changing dependency versions just for a weird edge case That being said, one possible other way to address this edge case could be via `shacl-vue` config using the `update_shapes` option, by changing the incoming shacl to remove the `sh:datatype xsd:hexBinary;` from that property shape, or changing it to `xsd:string`. I am not sure, however, what knock-on effects this might have; it will have to be tested. The shacl shape still has the `sh:pattern`, i.e. UI-side field validation should still be functional. After looking into this: it doesn't seem to help. I used `update_shapes` to change the notation's datatype from `xsd:hexBinary` to `xsd:string` (at first, setting it to `null` caused the field not to be displayed in the UI). This worked fine in the frontend, but the submission still had the `xsd:hexBinary` literal type included. I think this actually highlights an inconsistency in `shacl-vue`->`shacl-tulip`; `shacl-tulip` is possibly not receiving the changes from `update_shapes` and hence not taking it into account during submission. I am unsure though whether this was intentional or not: will need more investigation... Either way, it does not help with a workaround solution to the original problem.

jsheunis referenced this issue

2026-06-22 10:39:19 +00:00

update_shapes should flow through to shacl-tulip #355

mih referenced this issue from orinoco/dump-things-service

2026-06-24 05:32:48 +00:00

Checksum notation validation error ignored #219

mih referenced this issue from orinoco/dump-things-service

2026-06-24 12:26:03 +00:00

Resolve broken value encoding for HexBinary #226

mih commented

2026-06-25 05:56:27 +00:00

Owner

A final analysis was posted in orinoco/dump-things-server#226 and it was addressed with https://github.com/psychoinformatics-de/datalad-concepts/pull/552

A final analysis was posted in https://hub.psychoinformatics.de/orinoco/dump-things-server/issues/226 and it was addressed with https://github.com/psychoinformatics-de/datalad-concepts/pull/552

mih closed this issue

2026-06-25 05:56:27 +00:00