Add audit trail functionality for collections #198

Merged

cmo merged 17 commits from audit-trail into master 2026-03-17 09:03:05 +00:00

Conversation 0 Commits 17 Files changed 13 +531 -3

cmo commented 2026-03-12 13:43:25 +00:00

Owner

Copy link

This PR introduces a git-based audit trail backend. An audit trail can be added to any collection. On curated write, it is automatically records timestamp, user-id, diff, and the resulting record. Reports can be generated based on PIDs.

The backend tries to minimize git commit calls. It will only commit the changes in the current worktree, if a new audit-entry would overwrite them.

This PR introduces a git-based audit trail backend. An audit trail can be added to any collection. On curated write, it is automatically records timestamp, user-id, diff, and the resulting record. Reports can be generated based on PIDs. The backend tries to minimize git commit calls. It will only commit the changes in the current worktree, if a new audit-entry would overwrite them.

cmo added 3 commits 2026-03-12 13:43:26 +00:00

add a git-based audit trail backend f3c20c913e

use three-level directory hierarchies for files c1287e65b9

add change reports for IDs ... acc0981336

This commit adds a method that reports all
changes that were applied to a record with
a specific ID.

The report is a dictionaries. The keys are
ISO timestamps, the values are tuples containing
the following entries:

0: the user id that did the change
2: the git diff of the change
3: the content of the record after the change

cmo added 3 commits 2026-03-12 13:46:25 +00:00

add gitaudit-description to changelog e6306f65c2

add dump-things-report-gitaudit command 038d18399b

create gitaudit-dir if it does not exist f87256fd86

cmo added 1 commit 2026-03-13 13:04:30 +00:00

implement worktree-free audit trail ... 87f8e91011

Add changes directly to git-index, and
commit from there. This keeps the worktree
empty.

cmo force-pushed audit-trail from 87f8e91011 to e49ca98c1b 2026-03-13 15:17:46 +00:00 Compare

cmo added 2 commits 2026-03-16 08:25:49 +00:00

add an audit test with non-trivial change numbers c5be11dc38

use apply_change_set ... 016576a6eb

Use the code from the minilad branch of
datalad-core.

cmo added 1 commit 2026-03-16 12:42:03 +00:00

add audit-backdend handling ... f0451da314

This commit adds `audit_backends` to configuration
specifications. It also adds audit trails for every
record that is stored via the curator-interface.

For each record that is stored via the curator interface,
an audit entry is added to every audit-backend. The
audit entry contains:

1. timestamp when the record was added to the audit trail
2. the id of the token that was used to store the record
3. the changes that were made to the record (git diff format)
4. the content of the new record

The audit trail for a given PID can be printed by
using the command:

	dump-things-report-gitaudit <git-audit-dir> <PID>

cmo added 1 commit 2026-03-16 13:31:43 +00:00

add reference to branch minilad of datalad-core 581948edbf

cmo added 1 commit 2026-03-16 13:33:56 +00:00

remove unused import from test_gitaudit.py 89c7252d63

cmo added 1 commit 2026-03-16 13:53:40 +00:00

set git-identity before runing tests e5f648a86f

cmo changed title from WIP: add audit trail functionality for collections to Add audit trail functionality for collections 2026-03-16 14:13:36 +00:00

cmo added 1 commit 2026-03-16 14:33:34 +00:00

add audit-backend documentation ... ee6c3a2f15

This commit adds a description of the
audit-backend of type `git-audit`. That
means, it describes its configuration and
features in the changelog and in
`README.md`.

It also adds a short description of the
associated reporting tool, i.e, of
`dump-things-report-gitaudit`.

cmo added 2 commits 2026-03-17 08:54:15 +00:00

improve gitaudit tests 2fcf475235

refactor GitAuditBackend ... 300a32665e

Introduce an abstract `AuditBackend` base
class and derive `GitAuditBackend` from the
base class.

cmo merged commit dc089724f5 into master 2026-03-17 09:03:05 +00:00

cmo referenced this pull request from a commit 2026-03-17 09:03:07 +00:00

Merge pull request 'Add audit trail functionality for collections' (#198) from audit-trail into master

cmo deleted branch audit-trail 2026-03-17 09:03:14 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

bug

Something isn't working

  

documentation

Improvements or additions to documentation

  

duplicate

This issue or pull request already exists

  

enhancement

New feature or request

  

good first issue

Good for newcomers

  

help wanted

Extra attention is needed

  

invalid

This doesn't seem right

  

question

Further information is requested

  

wontfix

This will not be worked on

No labels

bug

documentation

duplicate

enhancement

good first issue

help wanted

invalid

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

orinoco/dump-things-server!198

No description provided.