- Python 99.6%
- Shell 0.4%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| .forgejo/workflows | ||
| artwork | ||
| docs | ||
| dump_things_service | ||
| tools | ||
| .gitignore | ||
| .gitmodules | ||
| .readthedocs.yaml | ||
| CHANGELOG.md | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| WHAT_IS_IT.md | ||
Dump Things Service
This is an implementation of a Dump Things Service (DTS). The Dump Things Service is a central component of a self-hostable research information infrastructure, which contains tools to collect, organize, validate, visualize, and serve structured research (meta)data.
It provides a REST API for storing and retrieving records that are structured according to a schema (defined in LinkML). The service supports multiple storage backends, flexible authentication sources, and a curation workflow for distributed data acquisition.
It is complemented by a client library and command line tool, which provide convenient access to the service from Python code and the shell dump-things-pyclient.
Dump Things Service serves as backend for shacl-vue, a generic graphical metadata acquisition tool.
For more information on the service, see the documentation.
Overview of the service architecture
Data is stored in collections. Each collection has a name and an associated schema. All data records in the collection have to adhere to the given schema.
The canonical format for schemas is LinkML.
The service supports schemas that are based on Datalad's Thing schema, i.e. on https://concepts.datalad.org/s/things/v1/.
It assumes that the classes of stored records are subclasses of Thing, and inherit the properties pid and schema_type from the Thing-baseclass.
The general workflow in the service is as follows. We distinguish between two areas of a collection, an incoming area (aka inbox) and a curated area. Data written to a collection is stored in a collection-specific incoming area. A curation process, which is outside the scope of the service, moves data from the incoming area of a collection to the curated area of the collection.
Acknowledgements
This work was funded, in part, by
-
Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under grant TRR 379 (546006540, Q02 project)
-
MKW-NRW: Ministerium für Kultur und Wissenschaft des Landes Nordrhein-Westfalen under the Kooperationsplattformen 2022 program, grant number: KP22-106A