orinoco/dump-things-server
orinoco/dump-things-server
5
0
Fork 1
Code Issues 28 Pull requests Packages Activity Actions

Add dynamic configuration #206

Merged
cmo merged 64 commits from dynamic-configuration into master 2026-06-12 10:47:49 +00:00
Conversation 0 Commits 64 Files changed 64 +4630 -2810
cmo commented 2026-06-12 10:47:22 +00:00
Owner
Copy link

This PR adds an interface for dynamic configuration of collections and tokens in running instances.

This PR adds an interface for dynamic configuration of collections and tokens in running instances.
cmo added 64 commits 2026-06-12 10:47:22 +00:00 
Handle admin tokens seperately from the
traditional token handling
Ensure that generated endpoints are tagged
and sorted correctly in the openapi docs.
When a collection is deleted, the collection
specific openapi doc tags are deleted as well.
By default the admin-token representation is hashed.
That means the request should contain the plain token,
unless the request-attribute `hashed` is set. In this
case, the `POST /admin-tokens` endpoint assumes that
the representation is already hashed. It will perform
a simple syntax verification to assure that the
representation is a 40-digit hex number.
Test that endpoints of deleted collections are
removed from the openapi documentation and
generate 404-errors when accessed.

Adapt token listing test to the modified token
result structure, which does not contain the
token representation anymore.
When adding the plain admin token `t` via
`/admin_tokens`, `AdminTokenRequest.representation`
must contain `hash_token_representation(t)`.

The value of `AdminTokenRequest.representation`
is checked for the correct pattern (a 40-digit hex
number) and stored verbatim in the configuration.

To authenticate an administrator, the token
provided in the `x-dumpthings-token`-header is
hashed and compared to the stored values of
`AdminTokenRequest.representation`.
Format backend code, add __all__ variables.
Update format, remove unused code and debug code
Add a command that interprets configuration
files and creates the objects defined in the
configuration file in a service.

This command can be used to migrate from
old dump-things server to new dump-things
server, when the option --old-format is used.

Note that there are a few differences
between the old and the new format:

1. Keys in `tokens` are now token names,
   that map to token configurations.

2. Token configurations now contain the
   additional attributes `representation`
   and `hashed`.

3. `schema` is now an attribute of a
   collection. It is removed from
   sqlite backend configurations.
Remove schema-definitions from sqlite-backend configuration.
Use empty dicitionary as default for Configuration.admin_tokens.
Check the following two conditions when
new collections are created:

1. the name of the collection is not a
   reserved name.

2. collections do not share incoming or
   curated directories with other collections.
   This is done to prevent data leaks, which
   could happen when different collections
   share directories.
This commit removes the option `--admin-token` and
adds the option `--admin-token-hash`, which accepts
a hashed token (sha256 in 64-digits hex representation).
Use the correct token names, when converting version 1
configuration files to version 2 configuration files.
Add scanner error detection to configuration loading. Raise
an error if the configuration YAML-file cannot be loaded.

Detect and reject token definitions with write permissions
and without incoming-labels or write access for
collections that do not define an incoming directory.

Rename `get_permissions` to `get_token_permissions`
Test that errors in the format of the configuration
file are caught and converted to `ConfigError`.
Do not allow unknown keys in configuration files.
Add a test for structure error detection.
Verify that incoming paths exist when a collection
is created a token has write-access to the
collection. Checks that incoming-label is set
for the token.

Add tests for incoming path validation.
The test-fixture `fast_api_simple` now
returns a tuple containing:

- test_client instance
- store path
- admin token
Replaces the `schema`-attribute with the
attribute `schema_location`. That prevents
shadowing of pydantic's internal `schema`-attribute
and gets rid of pydantic-issues warnings.
Add code to check whether the directory specified
int the git-audit configuration is either non-existing,
empty, or compatible with git-audit.
If a record-dir backend is created on a directory,
check if the config file exists, if not write it.
Adapt the code of `dump-things-pid-check` to the dynamic
configuration changes.
add test for unicode IRI handling
All checks were successful
Test execution / Test-all (push) Successful in 2m12s
Details
15e2f67631
cmo merged commit bb783adfa8 into master 2026-06-12 10:47:49 +00:00
cmo deleted branch dynamic-configuration 2026-06-12 10:48:10 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
orinoco/dump-things-server!206
No description provided.