orinoco/dump-things-pyclient
5
0
Fork 1
Code Issues 1 Pull requests Packages Activity Actions

Add option to skip posting of records to curated area, if a "similar" record already exists #37

Merged
cmo merged 7 commits from post-only-changed into master 2026-03-31 15:08:13 +00:00
Conversation 2 Commits 7 Files changed 9 +406 -2
cmo commented 2026-03-27 13:24:13 +00:00
Owner
Copy link

This PR adds the --only-if-modifying option to dtc post-records.

If --curate is provided, the flag instructs post-records to check whether a similar
record already exists in the curated area of the collection, and ---if the record already exists---
not to overwrite it.

"Similar" means identical, or identical if the elements that are defined by --ignore-spec
are not compared. This can be used to exclude, for example, elements like annotations from
the comparison. In this case, two records with differing annotations, but otherwise
identical content, would be considered to be identical.

This PR adds the `--only-if-modifying` option to `dtc post-records`. If `--curate` is provided, the flag instructs `post-records` to check whether a similar record already exists in the curated area of the collection, and ---if the record already exists--- not to overwrite it. "Similar" means identical, or identical if the elements that are defined by `--ignore-spec` are not compared. This can be used to exclude, for example, elements like `annotations` from the comparison. In this case, two records with differing `annotations`, but otherwise identical content, would be considered to be identical.
add RecordComparer-class
All checks were successful
Test execution / Test-all (push) Successful in 33s
Details
f0e3ad6eda 
This commit adds the class RecordComparer. Instances
of this class can compare dictionaries, while
ignoring certain elements.

This can be useful, for example, to determine if
two records are equal, except for the content of
the key 'annotations'.
add --only-if-modifying option to post-records
All checks were successful
Test execution / Test-all (push) Successful in 33s
Details
bd231d21de 
This commit adds the flag --only-if-modifying to
the dtc subcommand `post-records`. If `--curate`
is provided, it will check whether an identical
record already exists in the curated area of the
collection, and ---if the record already exists---
refuse to overwrite it.

Record matching can be "relaxed" by exluding
elements from the matching process. This can be
done using the option `--ignore-spec`.
adina commented 2026-03-27 14:12:21 +00:00
Owner
Copy link

I am still figuring out how to run this, but I think eventually it would be very powerful if this could also be exposed to auto-curate, so that I for example could define for the curation of an entire inbox that sole changes in annotations are counting as not modified

I am still figuring out how to run this, but I think eventually it would be very powerful if this could also be exposed to auto-curate, so that I for example could define for the curation of an entire inbox that sole changes in annotations are counting as not modified
add --only-if-modifying to auto-curate
All checks were successful
Test execution / Test-all (push) Successful in 34s
Details
4512c19d30
replace --ignore-spec with --jsonpath-spec
All checks were successful
Test execution / Test-all (push) Successful in 38s
Details
ccf810aec7 
Use `jsonpath_ng` to implement comparsion with
ignored record elements.
add tests for --only-if-modifying in post_records
All checks were successful
Test execution / Test-all (push) Successful in 32s
Details
91cfad91b5
cmo force-pushed post-only-changed from 91cfad91b5
All checks were successful
Test execution / Test-all (push) Successful in 32s
Details
to 1ebc58f38c
All checks were successful
Test execution / Test-all (push) Successful in 37s
Details
2026-03-31 08:24:54 +00:00 Compare
add tests for --only-if-modifying in auto-curate
All checks were successful
Test execution / Test-all (push) Successful in 37s
Details
23714d4f8f
cmo commented 2026-03-31 15:08:00 +00:00
Author
Owner
Copy link

Added to auto-curate as well.

@adina, please note that we now use jsonpath_ng for path specifications, and that the path specification option was renamed from --ignore-spec to --jsonpath-spec.

Added to auto-curate as well. @adina, please note that we now use jsonpath_ng for path specifications, and that the path specification option was renamed from `--ignore-spec` to `--jsonpath-spec`.
👍 1
cmo merged commit f25bad3ee8 into master 2026-03-31 15:08:13 +00:00
cmo deleted branch post-only-changed 2026-03-31 15:08:48 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
orinoco/dump-things-pyclient!37
No description provided.