From 653d351449b10af9f881a449718e2f05261053f8 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Tue, 20 Jan 2026 22:33:33 -0800 Subject: [PATCH 01/31] doc(typo): correct typo in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f1b18fa..d8725ca 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ The service supports schemas that are based on Datalad's *Thing* schema, i.e. on It assumes that the classes of stored records are subclasses of `Thing`, and inherit the properties `pid` and `schema_type` from the `Thing`-baseclass. The general workflow in the service is as follows. -We distinguish between two areas of a collection, an **incoming** are and a **curated** area. +We distinguish between two areas of a collection, an **incoming** area and a **curated** area. Data written to a collection is stored in a collection-specific **incoming** area. A curation process, which is outside the scope of the service, moves data from the incoming area of a collection to the **curated** area of the collection. -- 2.52.0 From e23ea12197a6caa5c4382619e8c0ef58042925e5 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Tue, 20 Jan 2026 22:58:47 -0800 Subject: [PATCH 02/31] doc(typo): correct typo in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d8725ca..5d3bc49 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ So any read- and write-operations on an incoming area are actually restricted to Multiple tokens can share the same zone. That allows multiple submitters to work together when storing records in the service. -The service provides a HTTP-based API to store and retrieve data objects, and to verify token capabilities. +The service provides an HTTP-based API to store and retrieve data objects, and to verify token capabilities. ### Installing the service -- 2.52.0 From a0bce1bbf3c4dc62c7b05cab9289b4717a1ddd9e Mon Sep 17 00:00:00 2001 From: Isaac To Date: Wed, 21 Jan 2026 18:28:11 -0800 Subject: [PATCH 03/31] doc: update link to dump-things-storage in DL concepts --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5d3bc49..e1bfcc9 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ collections: # The path to the curated data of the collection. This path should contain the # ".dumpthings.yaml"-configuration for collections that is described - # here: . + # here: . # A relative path is interpreted relative to the storage root, which is provided on # service start. An absolute path is interpreted as an absolute path. curated: curated/personal_records -- 2.52.0 From f38ed8c3b315c0addd6ae1ef41733ccc1dd964a2 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Wed, 21 Jan 2026 22:51:29 -0800 Subject: [PATCH 04/31] doc(typo): correct typo in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e1bfcc9..c872dec 100644 --- a/README.md +++ b/README.md @@ -108,7 +108,7 @@ collections: # Optionally a list of classes that will be ignored when store- or validate-endpoints # are created. If `use_classes` is present, the entries of this list will further reduce # the classes that will receive endpoints. If `use_classes` is not present, the entries - # of this list will reduce the classes from the schema, the will receive endpoints. + # of this list will reduce the classes from the schema that will receive endpoints. # The classes listed here must be listed in `use_classes` if that is defined. If # `use_classes` is not defined, they must be listed in the schema. ignore_classes: -- 2.52.0 From 83160438d1afa3b810e95ba1c8e7c6cc5d225646 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Wed, 21 Jan 2026 23:14:28 -0800 Subject: [PATCH 05/31] doc(typo): correct typo in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c872dec..57d7200 100644 --- a/README.md +++ b/README.md @@ -136,7 +136,7 @@ tokens: # access to the two collections: "rooms_and_buildings" and "fixed_data". basic_access: - # The value of "user-id" will be added as an annotation to each record that is + # The value of "user_id" will be added as an annotation to each record that is # uploaded with this token. user_id: anonymous -- 2.52.0 From ef9402eb6135b8baa0f9044f419089c2b66b560a Mon Sep 17 00:00:00 2001 From: Isaac To Date: Thu, 22 Jan 2026 22:38:24 -0800 Subject: [PATCH 06/31] doc(typo): correct typo in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 57d7200..d41a78c 100644 --- a/README.md +++ b/README.md @@ -238,7 +238,7 @@ The backend will be used for the curated area and for the incoming areas of the If no backend is defined for a collection, the `record_dir+stl`-backend is used by default. The `+stl`-backends can be useful if an endpoint returns records of multiple classes, because it allows clients to determine the class of each result record. -The service guarantees that backends of all types can co-exist independently in the same directory, i.e., there are no name collisions in files that are used for different backends (as long as no class name starts with `.` or `_`)). +The service guarantees that backends of all types can co-exist independently in the same directory, i.e., there are no name collisions in files that are used for different backends (as long as no class name starts with `.` or `_`). The following configuration snippet shows how to define a backend for a collection: -- 2.52.0 From d0e02d343315e0c29d0510a85cae9c7c10622fce Mon Sep 17 00:00:00 2001 From: Isaac To Date: Thu, 22 Jan 2026 22:51:05 -0800 Subject: [PATCH 07/31] doc: correct key in Forgejo auth config `repo` is not a valid key in Forgejo auth config but `repository` is --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index d41a78c..b09a0a2 100644 --- a/README.md +++ b/README.md @@ -291,10 +291,10 @@ collections: # `forgejo-user-` label_type: team # An optional repository. The token will only be authorized - # if the team has access to the repository. Note: if `repo` + # if the team has access to the repository. Note: if `repository` # is set, the token must have at least repository read # permissions. - repo: reference-repository + repository: reference-repository # Fallback to the config file. - type: config # check tokens from the configuration file @@ -385,10 +385,10 @@ collections: # `forgejo-user-` label_type: team # An optional repository. The token will only be authorized - # if the team has access to the repository. Note: if `repo` + # if the team has access to the repository. Note: if `repository` # is set, the token must have at least repository read # permissions. - repo: reference-repository + repository: reference-repository # Fallback to the config file. - type: config # check tokens from the configuration file @@ -566,10 +566,10 @@ collections: # `forgejo-user-` label_type: team # An optional repository. The token will only be authorized - # if the team has access to the repository. Note: if `repo` + # if the team has access to the repository. Note: if `repository` # is set, the token must have at least repository read # permissions. - repo: reference-repository + repository: reference-repository # Fallback to the config file. - type: config # check tokens from the configuration file -- 2.52.0 From bdb0d83069c189211449dbe3764f8c1086bd9617 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Fri, 23 Jan 2026 17:14:08 -0800 Subject: [PATCH 08/31] doc: correct comment about record_dir+stl backend setup --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b09a0a2..2f524fc 100644 --- a/README.md +++ b/README.md @@ -310,8 +310,8 @@ collections: default_token: anon_read curated: collection_3/curated backend: - # The record_dir-backend is identified by the - # type: "record_dir". No more attributes are + # The record_dir+stl backend is identified by the + # type: "record_dir+stl". No more attributes are # defined for this backend. type: record_dir+stl -- 2.52.0 From 41e779af7dc2a782ddf3bb787648e23b1b1f700a Mon Sep 17 00:00:00 2001 From: Isaac To Date: Sun, 25 Jan 2026 19:39:52 -0800 Subject: [PATCH 09/31] doc: complete comments about `config` authentication source --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 2f524fc..cd8a6d6 100644 --- a/README.md +++ b/README.md @@ -304,7 +304,7 @@ collections: # permissions for a token, those permissions will be used and no other # authorization sources will be queried. # The default authorization source is `config`, which reads the token - # permissions, user-id, and incoming + # permissions, user-id, and incoming from the config file. collection_with_explicit_record_dir+stl_backend: default_token: anon_read @@ -351,7 +351,7 @@ If an identical authentication source is listed multiple time in the configurati These authentication sources are available: -- config: use the configuration file to +- config: use the configuration file to authenticate tokens - forgejo: use a Forgejo-instance to authenticate tokens All authentication source configurations contain the key `type`. @@ -398,7 +398,7 @@ collections: # permissions for a token, those permissions will be used and no other # authorization sources will be queried. # The default authorization source is `config`, which reads the token - # permissions, user-id, and incoming + # permissions, user-id, and incoming from the config file. ... @@ -579,7 +579,7 @@ collections: # permissions for a token, those permissions will be used and no other # authorization sources will be queried. # The default authorization source is `config`, which reads the token - # permissions, user-id, and incoming + # permissions, user-id, and incoming from the config file. collection_with_explicit_record_dir+stl_backend: default_token: anon_read -- 2.52.0 From 3c76032d2f7a2148e202009de26e4ec1a3bc6ca2 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Sun, 25 Jan 2026 21:52:51 -0800 Subject: [PATCH 10/31] doc: remove duplicate section about backends --- README.md | 106 ------------------------------------------------------ 1 file changed, 106 deletions(-) diff --git a/README.md b/README.md index cd8a6d6..c4c866e 100644 --- a/README.md +++ b/README.md @@ -496,112 +496,6 @@ collections: ``` -The service currently supports the following backends for storing records: -- `record_dir`: this backend stores records as YAML-files in a directory structure that is defined [here](https://concepts.datalad.org/dump-things-storage-v0/). It reads the backend configuration from a "record collection configuration file" as described [here](https://concepts.datalad.org/dump-things-storage-v0/). - -- `sqlite`: this backend stores records in a SQLite database. There is an individual database file, named `__sqlite-records.db`, for each curated area and incoming area. - -- `record_dir+stl`: here `stl` stands for "schema-type-layer". - This backend stores records in the same format as `record_dir`, but adds special treatment for the `schema_type` attribute in records. - It removes `schema_type`-attributes from the top-level mapping of a record before storing it as YAML-file. When records are read from this backend, a `schema_type` attribute is added back into the record, using a schema to determine the correct class-URI. - In other words, all records stored with this backend will have no `schema_type`-attribute in the top-level, and all records read with this backend will have a `schema_type` attribute in the top-level. - -- `sqlite+stl`: This backend stores records in the same format as `sqlite`, but adds the same special treatment for the `schema_type` attribute as `record_dir+stl`. - -Backends can be defined per collection in the configuration file. -The backend will be used for the curated area and for the incoming areas of the collection. -If no backend is defined for a collection, the `record_dir+stl`-backend is used by default. -The `+stl`-backends can be useful if an endpoint returns records of multiple classes, because it allows clients to determine the class of each result record. - -The service guarantees that backends of all types can co-exist independently in the same directory, i.e., there are no name collisions in files that are used for different backends (as long as no class name starts with `.` or `_`)). - -The following configuration snippet shows how to define a backend for a collection: - -```yaml -... -collections: - collection_with_default_record_dir+stl_backend: - # This is a collection with the default backend, i.e. `record_dir+stl` and - # the default authentication, i.e. config-based authentication. - default_token: anon_read - curated: collection_1/curated - - collection_with_forgejo_authentication_source: - # This is a collection with the default backend, i.e. `record_dir+stl` and - # a forgejo-based authentication source. That means it will use a forgejo - # instance to determine the permissions of a token for this collection. - # The instance is also used to determine the user-id and the incoming label. - # In the case of forgejo, the user-id and the incoming label are the - # forgejo login associated with the token. - - # We still need the name of a default token. If the token is defined in this - # config file, its properties will be determined by the - # config file. If the token is not defined in the config file, its - # properties will be determined by the authentication sources. In this - # example by the forgejo-instance at `https://forgejo.example.com`. - # If there is more than one authentication source, they will be tried - # in the order they are defined in the config file. - default_token: anon_read # We still need a default token - curated: collection_2/curated - - # Token permissions, user-ids (for record annotations), and incoming - # label can be determined by multiple authentication sources. - # If no source is defined, `config` will be used, which reads token - # information from the config file. - # This example explicitly defines `config` and a second authentication - # source, a `forgejo` authentication source. - auth_sources: - - type: forgejo # requires `user`-read and `organization`-read permissions on token - # The API-URL of the forgejo instance that should be used - url: https://forgejo.example.com/api/v1 - # An organization - organization: data_handling - # A team in the organization. The authorization of the team - # determines the permissions of the token - team: data_entry_personal - # `label_type` determines how an incoming label is created for - # a Forgejo token. If `label_type` is `team`, the incoming label - # will be `forgejo-team--`. If `label_type` - # is `user`, the incoming label will be - # `forgejo-user-` - label_type: team - # An optional repository. The token will only be authorized - # if the team has access to the repository. Note: if `repository` - # is set, the token must have at least repository read - # permissions. - repository: reference-repository - - # Fallback to the config file. - - type: config # check tokens from the configuration file - - # Multiple authorization sources are allowed. They will be tried in the - # order defined in the config file. If an authorization source returns - # permissions for a token, those permissions will be used and no other - # authorization sources will be queried. - # The default authorization source is `config`, which reads the token - # permissions, user-id, and incoming from the config file. - - collection_with_explicit_record_dir+stl_backend: - default_token: anon_read - curated: collection_3/curated - backend: - # The record_dir-backend is identified by the - # type: "record_dir". No more attributes are - # defined for this backend. - type: record_dir+stl - - collection_with_sqlite_backend: - default_token: anon_read - curated: collection_4/curated - backend: - # The sqlite-backend is identified by the - # type: "sqlite". It requires a schema attribute - # that holds the URL of the schema that should - # be used in this backend. - type: sqlite - schema: https://concepts.inm7.de/s/flat-data/unreleased.yaml -``` - ### Command line parameters: -- 2.52.0 From 61cc871e41a5c2f403acebfb1ce4ce8d8ab190ce Mon Sep 17 00:00:00 2001 From: Isaac To Date: Sun, 25 Jan 2026 22:00:36 -0800 Subject: [PATCH 11/31] doc: provide better explanation of handling of CURIE submission tag --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c4c866e..45e60a9 100644 --- a/README.md +++ b/README.md @@ -478,7 +478,7 @@ The default annotation tag classes can be overridden in the configuration on a p To override the defaults tags, add a `submission_tags`-attribute to a collection definition. The `submission_tags`-attribute should contain a mapping that maps either `submitter_id_tag`, or `submitter_time_tag` or both to an IRI or a CURIE. If the schema defines a matching prefix, IRIs are automatically converted to CURIEs before storing the record. -The service validates that the prefix of a CURIE is defined in the schema of the collection. +If a tag is given as a CURIE, the service validates that the prefix of the CURIE is defined in the schema of the collection. ```yaml type: collections -- 2.52.0 From 0514d40b20a6b51b57459736e4ff6d8bbfa4110f Mon Sep 17 00:00:00 2001 From: Isaac To Date: Sun, 25 Jan 2026 22:57:10 -0800 Subject: [PATCH 12/31] doc(update): remove docs about `--sort-by` option This option is no longer available --- README.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/README.md b/README.md index 45e60a9..7e08ab5 100644 --- a/README.md +++ b/README.md @@ -53,11 +53,6 @@ The following command line parameters are supported: - `--root-path `: Set the ASGI 'root_path' for applications submounted below a given URL path. -- `--sort-by `: By default result records are sorted by the field `pid`. - This parameter allows overriding the sort field. - The parameter can be repeated to define secondary, tertiary, etc. sorting fields. - If a given field is not present in the record, the record will be sorted behind all records that possess the field. - ### Configuration file -- 2.52.0 From 8500d0505c8f142059bace0d4e781872184fafe0 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Sun, 25 Jan 2026 23:03:43 -0800 Subject: [PATCH 13/31] doc: merge two sections about command line parameters --- README.md | 42 +++++++----------------------------------- 1 file changed, 7 insertions(+), 35 deletions(-) diff --git a/README.md b/README.md index 7e08ab5..7ef343d 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,13 @@ The following command line parameters are supported: - `--root-path `: Set the ASGI 'root_path' for applications submounted below a given URL path. +- `--log-level`: set the log level for the service, allowed values are `ERROR`, `WARNING`, `INFO`, `DEBUG`. The default-level is `WARNING`. + +```bash +dump-things-service /data-storage/store --host 127.0.0.1 --port 8000 +``` + +The above command runs the service on the network location `127.0.0.1:8000` and provides access to the store under `/data-storage/store`. ### Configuration file @@ -491,41 +498,6 @@ collections: ``` - -### Command line parameters: - -The service supports the following command line parameters: - -- ``: this is a mandatory parameter that defines the directory that serves as root for relative `curated`- and `incoming`-paths. Unless the `-c/--config` option is given, the configuration is loaded from `/.dumpthings.yaml`. - -- `--host`: (optional): the IP address of the host the service should run on - - -- `--port`: the port number the service should listen on - - -- `-c/--config`: if set, the service will read the configuration from the given path. Otherwise it will try to read the configuration from `/.dumpthings.yaml`. - - -- `--log-level`: set the log level for the service, allowed values are `ERROR`, `WARNING`, `INFO`, `DEBUG`. The default-level is `WARNING`. - - -- `--root-path`: set the ASGI `root_path` for applications sub-mounted below a given URL path. - - -The service can be started with the following command: - -```bash -dump-things-service -``` -In this example the service will run on the network location `0.0.0.0:8000` and provide access to the stores under `/data-storage/store`. - -To run the service on a specific host and port, use the command line options `--host` and `--port`, for example: - -```bash -dump-things-service /data-storage/store --host 127.0.0.1 --port 8000 -``` - ### Endpoints Most endpoints require a *collection*. These correspond to the names of the "data record collection"-directories (for example `myschema-v3-fmta` in [Dump Things Service](https://concepts.datalad.org/dump-things/)) in the stores. -- 2.52.0 From 90971d2749a247b4c433174e3e2ad75a661243a3 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 09:41:19 -0800 Subject: [PATCH 14/31] doc: update link to Dump Things Service in DataLad Concept --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 7ef343d..0011866 100644 --- a/README.md +++ b/README.md @@ -500,7 +500,7 @@ collections: ### Endpoints -Most endpoints require a *collection*. These correspond to the names of the "data record collection"-directories (for example `myschema-v3-fmta` in [Dump Things Service](https://concepts.datalad.org/dump-things/)) in the stores. +Most endpoints require a *collection*. These correspond to the names of the "data record collection"-directories (for example `myschema-v3-fmta` in [Dump Things Service](https://concepts.datalad.org/dump-things-storage-v0/)) in the stores. The service provides the following user endpoints (in addition to user-endpoints there exist endpoints for curators, to view them check the `/docs`-path in an installed service): @@ -511,7 +511,7 @@ The service provides the following user endpoints (in addition to user-endpoints It can be set to `json` (the default) or to `ttl` (Terse RDF Triple Language, a.k.a. Turtle). If the `json`-format is selected, the content-type should be `application/json`. If the `ttl`-format is selected, the content-type should be `text/turtle`. - The service supports extraction of inlined records as described in [Dump Things Service](https://concepts.datalad.org/dump-things/). + The service supports extraction of inlined records as described in [Dump Things Service](https://concepts.datalad.org/dump-things-storage-v0/). On success, the endpoint will return a list of all stored records. This might be more than one record if the posted object contains inlined records. @@ -522,7 +522,7 @@ The service provides the following user endpoints (in addition to user-endpoints It can be set to `json` (the default) or to `ttl` (Terse RDF Triple Language, a.k.a. Turtle). If the `json`-format is selected, the content-type should be `application/json`. If the `ttl`-format is selected, the content-type should be `text/turtle`. - The service supports extraction of inlined records as described in [Dump Things Service](https://concepts.datalad.org/dump-things/). + The service supports extraction of inlined records as described in [Dump Things Service](https://concepts.datalad.org/dump-things-storage-v0/). On success, the endpoint will return a list of all stored records. This might be more than one record if the posted object contains inlined records. -- 2.52.0 From c1fc20e6c0fdcc472ca8756f3901880fadfc0612 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 09:47:22 -0800 Subject: [PATCH 15/31] doc: add punctuation to two sentences. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0011866..2c7540a 100644 --- a/README.md +++ b/README.md @@ -502,7 +502,7 @@ collections: Most endpoints require a *collection*. These correspond to the names of the "data record collection"-directories (for example `myschema-v3-fmta` in [Dump Things Service](https://concepts.datalad.org/dump-things-storage-v0/)) in the stores. -The service provides the following user endpoints (in addition to user-endpoints there exist endpoints for curators, to view them check the `/docs`-path in an installed service): +The service provides the following user endpoints (In addition to user-endpoints, there exist endpoints for curators. To view them, check the `/docs`-path in an installed service): - `POST //record/`: an object of type `` (defined by the schema associated with ``) can be posted to this endpoint. It will be stored in the incoming area for this collection and the user defined by the provided token. -- 2.52.0 From af9dacbe602ed12c428232ab6f8dab84afdd127e Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 10:38:21 -0800 Subject: [PATCH 16/31] doc: use "user endpoints" instead of "user-endpoints" For consistency --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2c7540a..a9d3b8a 100644 --- a/README.md +++ b/README.md @@ -502,7 +502,7 @@ collections: Most endpoints require a *collection*. These correspond to the names of the "data record collection"-directories (for example `myschema-v3-fmta` in [Dump Things Service](https://concepts.datalad.org/dump-things-storage-v0/)) in the stores. -The service provides the following user endpoints (In addition to user-endpoints, there exist endpoints for curators. To view them, check the `/docs`-path in an installed service): +The service provides the following user endpoints (In addition to user endpoints, there exist endpoints for curators. To view them, check the `/docs`-path in an installed service): - `POST //record/`: an object of type `` (defined by the schema associated with ``) can be posted to this endpoint. It will be stored in the incoming area for this collection and the user defined by the provided token. -- 2.52.0 From cf59e4e50233e653ba5d2a8fcad86aa2b67cc335 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 11:37:15 -0800 Subject: [PATCH 17/31] doc: improve wording --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a9d3b8a..3937d46 100644 --- a/README.md +++ b/README.md @@ -513,7 +513,7 @@ The service provides the following user endpoints (In addition to user endpoints If the `ttl`-format is selected, the content-type should be `text/turtle`. The service supports extraction of inlined records as described in [Dump Things Service](https://concepts.datalad.org/dump-things-storage-v0/). On success, the endpoint will return a list of all stored records. - This might be more than one record if the posted object contains inlined records. + The list may contain more than one record if the posted object contains inlined records. - `POST //validate/record/`: an object of type `` (defined by the schema associated with ``) can be posted to this endpoint. It will validate the posted data. @@ -524,7 +524,7 @@ The service provides the following user endpoints (In addition to user endpoints If the `ttl`-format is selected, the content-type should be `text/turtle`. The service supports extraction of inlined records as described in [Dump Things Service](https://concepts.datalad.org/dump-things-storage-v0/). On success, the endpoint will return a list of all stored records. - This might be more than one record if the posted object contains inlined records. + The list may contain more than one record if the posted object contains inlined records. - `GET //records/`: retrieve all readable objects from collection `` that are of type `` or any of its subclasses. Objects are readable if the default token for the collection allows reading of objects or if a token is provided that allows reading of objects in the collection. -- 2.52.0 From 21da42e7b7af328ee6fd817820192f1f21e9af19 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 11:44:55 -0800 Subject: [PATCH 18/31] doc: improve wording --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3937d46..a4bd9bd 100644 --- a/README.md +++ b/README.md @@ -527,7 +527,7 @@ The service provides the following user endpoints (In addition to user endpoints The list may contain more than one record if the posted object contains inlined records. - `GET //records/`: retrieve all readable objects from collection `` that are of type `` or any of its subclasses. - Objects are readable if the default token for the collection allows reading of objects or if a token is provided that allows reading of objects in the collection. + Objects are readable if the default token or the token provided has the permission to read the objects in the collection. Objects from incoming spaces will take precedence over objects from curated spaces, i.e. if there are two objects with identical `pid` in the curated space and in the incoming space, the object from the incoming space will be returned. The endpoint supports the query parameter `format`, which determines the format of the query result. It can be set to `json` (the default) or to `ttl`, @@ -578,7 +578,7 @@ The service provides the following user endpoints (In addition to user endpoints - `GET //records/`: retrieve all readable objects from collection ``. - Objects are readable if the default token for the collection allows reading of objects or if a token is provided that allows reading of objects in the collection. + Objects are readable if the default token or the token provided has the permission to read the objects in the collection. Objects from incoming spaces will take precedence over objects from curated spaces, i.e. if there are two objects with identical `pid` in the curated space and in the incoming space, the object from the incoming space will be returned. The endpoint supports the query parameter `format`, which determines the format of the query result. It can be set to `json` (the default) or to `ttl`, -- 2.52.0 From f3f78a346b3ca5d78ef7b7175f2d90dfcde73513 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 12:08:55 -0800 Subject: [PATCH 19/31] doc: remove unneeded commas before if clause --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a4bd9bd..de3489b 100644 --- a/README.md +++ b/README.md @@ -553,7 +553,7 @@ The service provides the following user endpoints (In addition to user endpoints } ``` -- `GET //record?pid=`: retrieve an object with the pid `` from the collection ``, if the provided token allows reading. If the provided token allows reading of incoming and curated spaces, objects from incoming spaces will take precedence. +- `GET //record?pid=`: retrieve an object with the pid `` from the collection `` if the provided token allows reading. If the provided token allows reading of incoming and curated spaces, objects from incoming spaces will take precedence. The endpoint supports the query parameter `format`, which determines the format of the query result. It can be set to `json` (the default) or to `ttl`, @@ -604,7 +604,7 @@ The service provides the following user endpoints (In addition to user endpoints ``` -- `DELETE //record?pid=`: delete an object with the pid `` from the incoming area of the collection ``, if the provided token allows writing to the incoming area. +- `DELETE //record?pid=`: delete an object with the pid `` from the incoming area of the collection `` if the provided token allows writing to the incoming area. The result is either `True` if the object was deleted or `False` if the object did not exists or was not deleted. -- 2.52.0 From 6337141052f50c17b2401f7101a45523886e9a70 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 12:13:45 -0800 Subject: [PATCH 20/31] doc: improve wording --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index de3489b..2c968fc 100644 --- a/README.md +++ b/README.md @@ -608,7 +608,7 @@ The service provides the following user endpoints (In addition to user endpoints The result is either `True` if the object was deleted or `False` if the object did not exists or was not deleted. -- `GET /docs`: provides information about the API of the service, i.e. about all endpoints. +- `GET /docs`: provides information about the service's API, i.e. about all endpoints. #### Curation endpoints -- 2.52.0 From 2c5525380585167b80d20e45feb26e8e41bafe81 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 12:20:07 -0800 Subject: [PATCH 21/31] doc: replace "curation-endpoints" with "curation endpoints" for consistency and clarity --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 2c968fc..6a2df22 100644 --- a/README.md +++ b/README.md @@ -612,9 +612,9 @@ The service provides the following user endpoints (In addition to user endpoints #### Curation endpoints -The service support a set of curation-endpoints that give direct access to the curated area as well as to existing incoming areas. +The service support a set of curation endpoints that give direct access to the curated area as well as to existing incoming areas. This access requires a `CURATOR`-token. -Details about the curation-endpoints can be found in [this issue](https://github.com/christian-monch/dump-things-server/issues/118). +Details about the curation endpoints can be found in [this issue](https://github.com/christian-monch/dump-things-server/issues/118). ### Tips & Tricks -- 2.52.0 From d0dc48d27d225843d1cc84081f57e300cb9fab93 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 12:26:26 -0800 Subject: [PATCH 22/31] doc: improve wording about curation endpoints --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6a2df22..3fb8b02 100644 --- a/README.md +++ b/README.md @@ -612,8 +612,8 @@ The service provides the following user endpoints (In addition to user endpoints #### Curation endpoints -The service support a set of curation endpoints that give direct access to the curated area as well as to existing incoming areas. -This access requires a `CURATOR`-token. +The service supports a set of curation endpoints that allows direct access to the curated area as well as the incoming areas. +A `CURATOR`-token required to access these endpoints. Details about the curation endpoints can be found in [this issue](https://github.com/christian-monch/dump-things-server/issues/118). -- 2.52.0 From 4e69352be1d3b2be4aec34447a4358b7de3902a4 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 13:46:24 -0800 Subject: [PATCH 23/31] doc: improve wording --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3fb8b02..2a956d0 100644 --- a/README.md +++ b/README.md @@ -673,7 +673,7 @@ For example, to migrate from a `record_dir+stl` backend, the command is similar, ``` (Note: a `record_dir:` can be used to copy without the schema type layer from a `record_dir+stl` backend. But in this case the copied records will not have a `schema_type` attribute, because the `record_dir` backend does not "put it back in", unlike a `record_dir+stl` backend.) -If the source backend is a `record_dir` or `record_dir+stl` backend and the store was manually modified outside the service (for example, by adding or removing files), it is recommended to run the command `dump-things-rebuild-index` on the source store before copying. This ensures that the index is up to date and all records are copied. +If the source backend is a `record_dir` or `record_dir+stl` backend and the store was manually modified outside the service (for example, by adding or removing files), it is recommended to run the command `dump-things-rebuild-index` on the source store before copying. This ensures that the index is up to date and all records will be copied. If any backend is a `record_dir+stl` backend, a schema has to be supplied via the `-s/--schema` command line parameter. The schema is used to determine the `schema_type` attribute of the records that are copied. -- 2.52.0 From 6c0e87fd552c1ddb18cf32956f11a1b6f7b0331e Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 13:47:52 -0800 Subject: [PATCH 24/31] doc: add comma for clarity --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2a956d0..55e2081 100644 --- a/README.md +++ b/README.md @@ -673,7 +673,7 @@ For example, to migrate from a `record_dir+stl` backend, the command is similar, ``` (Note: a `record_dir:` can be used to copy without the schema type layer from a `record_dir+stl` backend. But in this case the copied records will not have a `schema_type` attribute, because the `record_dir` backend does not "put it back in", unlike a `record_dir+stl` backend.) -If the source backend is a `record_dir` or `record_dir+stl` backend and the store was manually modified outside the service (for example, by adding or removing files), it is recommended to run the command `dump-things-rebuild-index` on the source store before copying. This ensures that the index is up to date and all records will be copied. +If the source backend is a `record_dir` or `record_dir+stl` backend, and the store was manually modified outside the service (for example, by adding or removing files), it is recommended to run the command `dump-things-rebuild-index` on the source store before copying. This ensures that the index is up to date and all records will be copied. If any backend is a `record_dir+stl` backend, a schema has to be supplied via the `-s/--schema` command line parameter. The schema is used to determine the `schema_type` attribute of the records that are copied. -- 2.52.0 From 08956d2fe7adce228d0a24d3132dd76b128d8ea4 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 14:00:06 -0800 Subject: [PATCH 25/31] doc: correct typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 55e2081..7b57de6 100644 --- a/README.md +++ b/README.md @@ -688,7 +688,7 @@ If any backend is a `record_dir+stl` backend, a schema has to be supplied via th record_dir:/penguis/curated \ sqlite:/penguis/curated ``` - The copy command will add the copied records to any existing record in the destination store. + The copy command will add the copied records to any existing records in the destination store. Note: when records are copied from a `record-dir` store, the index is used to locate the records in the source store. If the index is not up-to-date, the copied records might not be complete. In this case, it is recommended to run `dump-things-rebuild-index` on the source store before copying. - `dump-things-pid-check`: this command checks the pids in all collections of a store to verify that they can be resolved (if they are in CURIE form). -- 2.52.0 From a1ac574946c04040a913cff14a4630d959d70d8c Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 14:02:59 -0800 Subject: [PATCH 26/31] doc: improve wording for clarity --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7b57de6..f2295de 100644 --- a/README.md +++ b/README.md @@ -689,7 +689,7 @@ If any backend is a `record_dir+stl` backend, a schema has to be supplied via th sqlite:/penguis/curated ``` The copy command will add the copied records to any existing records in the destination store. - Note: when records are copied from a `record-dir` store, the index is used to locate the records in the source store. If the index is not up-to-date, the copied records might not be complete. In this case, it is recommended to run `dump-things-rebuild-index` on the source store before copying. + Note: when records are copied from a `record-dir` store, the index is used to locate the records in the source store. If the index is not up-to-date, some records may not be copied. To ensure all records are copied, it is recommended to run `dump-things-rebuild-index` on the source store before copying. - `dump-things-pid-check`: this command checks the pids in all collections of a store to verify that they can be resolved (if they are in CURIE form). This is useful to validate the proper definition of prefixes after schema-changes. -- 2.52.0 From 2bd0fbda96952b254b1df93f50b5bb1694d7728c Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 14:09:15 -0800 Subject: [PATCH 27/31] doc: improve wording for clarity --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f2295de..5373d98 100644 --- a/README.md +++ b/README.md @@ -694,8 +694,8 @@ If any backend is a `record_dir+stl` backend, a schema has to be supplied via th - `dump-things-pid-check`: this command checks the pids in all collections of a store to verify that they can be resolved (if they are in CURIE form). This is useful to validate the proper definition of prefixes after schema-changes. -- `dump-things-create-merged-schema`: this command creates a new schema that statically contains all schemas that the original schema imported. - The new schema is fully self contained and does not reference any other schemas anymore. +- `dump-things-create-merged-schema`: this command creates a new schema that statically contains all schemas that the original schema imports. + The new schema is fully self-contained and does not reference any other schemas. ### If things go wrong -- 2.52.0 From c3a467a39b25e6eaf5fed1a81dee4777300fc52e Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 14:17:45 -0800 Subject: [PATCH 28/31] doc: improve wording for clarity --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5373d98..6488629 100644 --- a/README.md +++ b/README.md @@ -701,7 +701,7 @@ If any backend is a `record_dir+stl` backend, a schema has to be supplied via th #### Delete a record manually -If a schema was changed, for example a prefix-definition changed, the service might not be able anymore to delete a record. +If a schema is changed, for example a prefix-definition changed, the service may not be able to delete a record anymore. In this case the record can be deleted manually if you have access to the storage root. To delete the record, open a shell and navigate (`cd`) to the directory where the store is located. -- 2.52.0 From 9c4fdea18615c241523884048179fdeb0d756a22 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 14:18:03 -0800 Subject: [PATCH 29/31] doc: add comma for clarity --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6488629..15ba8a9 100644 --- a/README.md +++ b/README.md @@ -702,7 +702,7 @@ If any backend is a `record_dir+stl` backend, a schema has to be supplied via th #### Delete a record manually If a schema is changed, for example a prefix-definition changed, the service may not be able to delete a record anymore. -In this case the record can be deleted manually if you have access to the storage root. +In this case, the record can be deleted manually if you have access to the storage root. To delete the record, open a shell and navigate (`cd`) to the directory where the store is located. The location can be determined from the configuration file. -- 2.52.0 From 5cca9f29ff5f9e1b866372b3ffc2965d92cc6ae5 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 14:20:38 -0800 Subject: [PATCH 30/31] doc: improve wording --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 15ba8a9..04ab3f6 100644 --- a/README.md +++ b/README.md @@ -706,7 +706,7 @@ In this case, the record can be deleted manually if you have access to the stora To delete the record, open a shell and navigate (`cd`) to the directory where the store is located. The location can be determined from the configuration file. -Depending on the storage backend, the next steps are different. +Depending on the storage backend, the subsequent steps are different. ##### `record-dir` backend -- 2.52.0 From f34f1fde56a179ac2db9fa68ce2fd71d93671ff1 Mon Sep 17 00:00:00 2001 From: Isaac To Date: Mon, 26 Jan 2026 22:59:12 -0800 Subject: [PATCH 31/31] doc: correct typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 04ab3f6..d36a09a 100644 --- a/README.md +++ b/README.md @@ -208,7 +208,7 @@ tokens: # to `True`. A hashed token has the structure # `-`. It will match an incoming token if the incoming token has # the structure `-` and if sha256(``) equals ``. - # In this example, if the client present sthe token `bob-hello`, he will be + # In this example, if the client presents the token `bob-hello`, he will be # granted access because `sha256('hello')` equals # `2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824` bob-2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824: -- 2.52.0