From 3501e823804ed1ec77a70142cb1191b44a3fbe7b Mon Sep 17 00:00:00 2001 From: Stephan Heunis Date: Wed, 13 May 2026 00:45:05 +0200 Subject: [PATCH 1/2] New approach to generating/maintaining list of records The premise of this change is that the 'getInstanceItems' approach to generating the list of displayed records (from graph data) results in bad performance. This is mainly because 'getInstanceItems' is called many times (mainly triggered by a watcher reacting to any/all graph changes), each time regenerating the full list of records. The proposed new approach is based on maintaining a global list of items that is incrementally updated when new nodes are added to or updated in the graph. 'parseTTLandDedup' is the 'ReactiveRdfDataset' class method that receives incoming TTL strings parses it into quads, deduplicates blank nodes, and adds it to the 'N3' store. It already provided an array with all added quads as the return value. A new utility function, 'getUniqueRootNodes', is now used to determine the unique named nodes that are included in the added quads. This allows saying: 'these are the records that were just added or updated' in the graph store. 'parseTTLandDedup' now returns both the added quads and the array of unique added/updated record IRIs, and downstream functionality using it has been updated to access its outputs correctly. A new 'ReactiveRdfDataset' class method 'emitAddedRecords(records)' is also added, which can be called with an array of named node (i.e. record) IRIs and which uses 'dispatchEvent' to emit the new 'recordsChanged' event, for which listeners can be added. 'emitAddedRecords' is now used in multiple places in order to emit records that have just been added/updated (because of this, the previous calls to 'rdfDS.triggerReactivity' become redundant): - In the 'useData' composable, inside 'fetchRdfData' right after 'parseTTLandDedup' returns the list of unique added/updated records. This happens downstream for every call to 'fetchFromService', which is the standard way in which 'shacl-vue' gets backend data in response to user interaction. - In the 'useWizard' composable: here we only call 'emitAddedRecords' if the context is not '_record' because it means there is no open form (which would need to process added quads in its own way); it means the wizard immediately adds all quads to the graph. - In the 'FormEditor' component when the user saved the record, i.e. in the 'saveForm' function. Here it is called specifically when the record is a PID record (i.e. not a blank node). Now, onto the algorithmic change for 'getInstanceItems', which is done in the 'useRecords' composable. As mentioned, we now maintain a global list of items ('recordItemsAll' and 'recordItemsByClass') that is incrementally updated when new nodes are added to or updated in the graph. This composable handles the new process as follows: - adds an event listener to 'rdfDS' for the new 'recordsChanged' event, which calls the new 'enqueueChanges' function - 'enqueueChanges' adds all added/updated records to a queue, and runs the code to process the queue - 'processQueue' will keep on running as long as there are records in the queue, and will process records in batches of 10 by calling the 'updateRecordItem' function per record - 'updateRecordItem' runs all the code that used to be run per item by 'getInstanceItems', and then adds the processed record to the 'recordItemsAll' and 'recordItemsByClass' reactive variables - any further computations on the global lists, such as filtering based on user-typed text, are done via computed refs that depend on the above reactive refs. Important new computed refs include 'filteredRecordItemsAll' and 'filteredRecordItemsByClass' which filters all records and record per class based on search text, as well as 'filteredRecordItemsForClassWithSubclassItems' which filters records per class while also including all records of a class's subclasses, which is required for the existing 'priority_classes'/'include_subclasses' config feature. The main 'ShaclVue' component has been updated to use the adjusted 'useRecords' composable correctly, specifically passing the new filtered computed refs to the 'ShaclVueRecords' component, which has updated props 'filteredRecords' and 'fetchedItemCount' to handle the new composable structure. Also, as part of cleanup several previous computed refs were moved as functions to the 'useShapes' composable: - 'getIdFilteredNodeShapeNames' - 'getNoEditClassList' - 'getFilteredNodeShapeNames' - 'getPriorityFilteredNodeShapeNames' - 'getOrderedNodeShapeNames' - 'getAllClassItems' These are now called on app startup during config processing. Lastly, the 'useWizard' composable has been updated to include a new feature needed by the upcoming kickstarter component. This feature allows a wizard to be shown not specifically for a record, nor specifically for a class, but for all classes. It's goal is to allow the configuration of very generic wizards that would be applicable for all classes. The config spec for it's selection is as follows (with 'AddRecordWizard' being an example wizard name): wizard_editor_selection: _classes: - AddRecordWizard To allow distinction of the applicable class IRI when the associated wizard template is processed, the class IRI is now made part of the wizard data by default in 'handleWizardSave' (accessible as 'class_uri'). Also 'showWizardGroup' and 'setupWizards' functions are updated to handle the new 'wizard_editor_selection' key. --- src/classes/ReactiveRdfDataset.js | 23 +- src/components/FormEditor.vue | 2 + src/components/InstancesUploadEditor.vue | 10 +- src/components/ShaclVue.vue | 154 ++----- src/components/ShaclVueRecords.vue | 15 +- src/components/WizardEditorInputPrimitive.vue | 1 + src/composables/configuration.js | 2 +- src/composables/useData.js | 17 +- src/composables/useRecords.js | 375 ++++++++++-------- src/composables/useShapes.js | 156 +++++++- src/composables/useWizard.js | 28 +- src/modules/utils.js | 35 ++ 12 files changed, 498 insertions(+), 320 deletions(-) diff --git a/src/classes/ReactiveRdfDataset.js b/src/classes/ReactiveRdfDataset.js index 6a0e718..291d709 100644 --- a/src/classes/ReactiveRdfDataset.js +++ b/src/classes/ReactiveRdfDataset.js @@ -2,7 +2,7 @@ import { reactive, ref } from 'vue'; import { RdfDataset } from 'shacl-tulip'; import { DataFactory, Store, Parser } from 'n3'; import { RDF } from '@/modules/namespaces'; -import { hashSubgraph, getNodeContextKey, collectBlankNodeHierarchy} from '@/modules/utils'; +import { hashSubgraph, getNodeContextKey, collectBlankNodeHierarchy, getUniqueRootNodes} from '@/modules/utils'; const { blankNode} = DataFactory; export class ReactiveRdfDataset extends RdfDataset { @@ -92,7 +92,16 @@ export class ReactiveRdfDataset extends RdfDataset { }); } + emitAddedRecords(records) { + this.dispatchEvent( + new CustomEvent('recordsChanged', { + detail: {records: records} + }) + ); + } + async parseTTLandDedup(ttlString) { + let uniqueRecords const parser = new Parser(); const tempStore = new Store(); let addedQuads = []; @@ -128,7 +137,9 @@ export class ReactiveRdfDataset extends RdfDataset { console.warn('No root named node detected in TTL, skipping deduplication steo and adding all quads to graph.'); let bnQuads = tempStore.getQuads(null, null, null, null) this.data.graph.addQuads(bnQuads); - return addedQuads.concat(bnQuads); + let allAddedQuads = addedQuads.concat(bnQuads); + uniqueRecords = getUniqueRootNodes(allAddedQuads, this.data.graph) + return {quads: allAddedQuads, records: uniqueRecords} } // Now get all unique blank-node subject values const allTempQuads = tempStore.getQuads(null, null, null, null); @@ -136,7 +147,10 @@ export class ReactiveRdfDataset extends RdfDataset { allTempQuads.filter(q => q.subject.termType === 'BlankNode').map(q => q.subject.value) ); // If there are no blank-node subject values, no need to deduplicate - if (blankSubjects.size == 0) return addedQuads; + if (blankSubjects.size == 0) { + uniqueRecords = getUniqueRootNodes(addedQuads, this.data.graph) + return {quads: addedQuads, records: uniqueRecords} + } // Initialize set of fingerprints in the root-node context if (!this.data.subgraphFingerprintsByRoot.has(root_node)) { this.data.subgraphFingerprintsByRoot.set(root_node, new Set()); @@ -169,6 +183,7 @@ export class ReactiveRdfDataset extends RdfDataset { // console.log(`Skipping duplicate subgraph for root ${root_node}, blank node ${bnodeId}`); } } - return addedQuads; + uniqueRecords = getUniqueRootNodes(addedQuads, this.data.graph) + return {quads: addedQuads, records: uniqueRecords} } } diff --git a/src/components/FormEditor.vue b/src/components/FormEditor.vue index aa14138..93dd49b 100644 --- a/src/components/FormEditor.vue +++ b/src/components/FormEditor.vue @@ -368,6 +368,8 @@ async function saveForm() { ) ) { nodesToSubmit.value.push(saved_node); + // we also need to add/update the record to the global list + rdfDS.emitAddedRecords([saved_node.node_iri]) } removeForm(saved_node); if (nodesToSubmit.value.length) { diff --git a/src/components/InstancesUploadEditor.vue b/src/components/InstancesUploadEditor.vue index 84631b8..5222f4e 100644 --- a/src/components/InstancesUploadEditor.vue +++ b/src/components/InstancesUploadEditor.vue @@ -90,6 +90,7 @@ const templates = reactive({ }) const hasCreatedQuads = ref(false) const createdQuads = ref([]) +const createdRecords = ref([]) const createdDistributions = new Set() const savedNodes = inject('savedNodes'); const nodesToSubmit = inject('nodesToSubmit'); @@ -163,9 +164,12 @@ async function onUploadComplete(result) { TTLdata.pid = toIRI(TTLdata.pid, allPrefixes) let newTTL = fillStringTemplate(templates.ttl, TTLdata) let newQuads = await rdfDS.parseTTLandDedup(newTTL); - rdfDS.triggerReactivity(); // Keep track of quads that were added, so that we can delete them if form is cancelled - createdQuads.value = createdQuads.value.concat(newQuads) + // For the same reason, we also don't yet call rdfDS.emitAddedRecords() here, as we do in useData + // after a call to rdfDS.parseTTLandDedup; if the form is cancelled we don't want these items in + // the global list. We let the form save function handle this. + createdQuads.value = createdQuads.value.concat(newQuads.quads) + createdRecords.value = createdRecords.value.concat(newQuads.records) hasCreatedQuads.value = true; // Keep track of distributions that were added createdDistributions.add(hash) @@ -204,6 +208,8 @@ onBeforeUnmount( () => { } } } + // we also emit the added records + rdfDS.emitAddedRecords(createdRecords.value) } }) diff --git a/src/components/ShaclVue.vue b/src/components/ShaclVue.vue index 66f2e5a..151bfd0 100644 --- a/src/components/ShaclVue.vue +++ b/src/components/ShaclVue.vue @@ -135,11 +135,10 @@ { // -------------- // // Computed props // // -------------- // -// These are all arays of classes that are eventually represented in the -// main class-selection pane in this component -const idFilteredNodeShapeNames = computed(() => { - if (configVarsMain.showShapesWoId === true) { - return shapesDS.data.nodeShapeNamesArray; - } - var shapeNames = []; - for (var n of shapesDS.data.nodeShapeNamesArray) { - if ( - findObjectByKey( - shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[n]] - .properties, - SHACL.path.value, - ID_IRI.value - ) - ) { - shapeNames.push(n); - } - } - return shapeNames; -}); - -const filteredNodeShapeNames = computed(() => { - var names = idFilteredNodeShapeNames.value; - // If all relevant config arrays are empty, show all classes - if ( - configVarsMain.showClasses?.length == 0 && - configVarsMain.showClassesWithPrefix?.length == 0 && - configVarsMain.hideClasses?.length == 0 && - configVarsMain.hideClassesWithPrefix?.length == 0 && - configVarsMain.noEditClasses?.length == 0 - ) { - return names; - } - var shapeNames = []; - for (var n of names) { - // First get IRI and prefix - var n_iri = shapesDS.data.nodeShapeNames[n] - if (includeClass(n_iri, configVarsMain, allPrefixes) && configVarsMain.noEditClasses.indexOf(toCURIE(n_iri, allPrefixes)) < 0) { - shapeNames.push(n); - } - } - return shapeNames; -}); - -const priorityFilteredNodeShapeNames = computed(() => { - var names = filteredNodeShapeNames.value; - var shapeNames = []; - for (var n of names) { - var n_iri = shapesDS.data.nodeShapeNames[n] - if (!priorityClassList.value.includes(n_iri)) { - shapeNames.push(n); - } - } - return shapeNames; -}) - -const orderedNodeShapeNames = computed(() => { - return priorityFilteredNodeShapeNames.value.sort((a, b) => - getDisplayName( - shapesDS.data.nodeShapeNames[a], - configVarsMain, - allPrefixes, - shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[a]] - ).toLowerCase() - .localeCompare( - getDisplayName( - shapesDS.data.nodeShapeNames[b], - configVarsMain, - allPrefixes, - shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[b]] - ).toLowerCase() - ) - ); -}) - -const noEditClassList = computed(() => { - if (configVarsMain.noEditClasses?.length == 0) return [] - var names = idFilteredNodeShapeNames.value; - var shapeNames = []; - for (var n of names) { - // First get IRI and prefix - var n_iri = shapesDS.data.nodeShapeNames[n] - if (includeClass(n_iri, configVarsMain, allPrefixes) && - configVarsMain.noEditClasses?.indexOf(toCURIE(n_iri, allPrefixes)) >= 0) { - shapeNames.push(n); - } - } - return shapeNames.sort((a, b) => - getDisplayName( - shapesDS.data.nodeShapeNames[a], - configVarsMain, - allPrefixes, - shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[a]] - ).toLowerCase() - .localeCompare( - getDisplayName( - shapesDS.data.nodeShapeNames[b], - configVarsMain, - allPrefixes, - shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[b]] - ).toLowerCase() - ) - ); -}) // --------- // // Functions // @@ -688,7 +605,6 @@ async function selectType(IRI, fromUser, fromBackButton, includeSubs=false) { totalItemCount.value = 0 isFetchingPage.value = false; showScrollTopBtn.value = false; - newTypeSelected.value = true; var tempSearchText = searchText.value; var tempIRI = selectedIRI.value; searchText.value = ''; @@ -708,10 +624,8 @@ async function selectType(IRI, fromUser, fromBackButton, includeSubs=false) { console.error(result.error); classRecordsLoading.value = false; } - // If any of the results were successful, don't set classRecordsLoading to false - // because it will be set during the watch event for instanceItemsComp + // If any of the results were successful, do nothing if (result.status.length && result.status.indexOf('success') >= 0) { - // do nothing } else { classRecordsLoading.value = false; } @@ -723,7 +637,7 @@ async function selectType(IRI, fromUser, fromBackButton, includeSubs=false) { totalItemCount.value = totalItems } } - getInstanceItems(); + classRecordsLoading.value = false scrollToTop(); if (fromUser) { updateURL(IRI, false, null, allPrefixes); diff --git a/src/components/ShaclVueRecords.vue b/src/components/ShaclVueRecords.vue index 2937db6..2494016 100644 --- a/src/components/ShaclVueRecords.vue +++ b/src/components/ShaclVueRecords.vue @@ -3,8 +3,8 @@ -
- +
+ use reactive. + const recordItemsAll = reactive({}); + const recordItemsByClass = reactive({}); // --------------------- // // Lifecycle/Vue methods // // --------------------- // + onMounted(() => { + rdfDS.addEventListener('recordsChanged', (e) => { + enqueueChanges(e.detail.records); + }); + }); + watch(isFetchingPage, (newVal) => { if (newVal) { // If fetching starts → show immediately @@ -73,20 +92,6 @@ export function useRecords( } }) - watch( - instanceItemsComp, - (newVal, oldVal) => { - if (newTypeSelected.value) { - newTypeSelected.value = false; - return; - } - if (classRecordsLoading.value) { - classRecordsLoading.value = false; - } - }, - { deep: true } - ); - watchEffect(async () => { // If we are using a backend service AND // there are a minimum amount of characters in the search field AND @@ -98,7 +103,7 @@ export function useRecords( hasUnfetchedPages(selectedIRI.value, searchText.value)) { // Only trigger fetch if not already fetching if (!isFetchingPage.value) { - await fetchNextPage(searchText.value); + await fetchNextPage(selectedIRI.value, searchText.value); } } }); @@ -112,13 +117,6 @@ export function useRecords( onTypingPause(newVal); }, configVarsMain.serviceConstrainedSearch.typing_debounce); }); - // regenerate list if the graph data is updated - const debouncedUpdate = debounce(() => { - if (openForms.length == 0) { - getInstanceItems(); - } - }, 500); - watch(() => rdfDS.data.graphChanged, debouncedUpdate, { deep: true }); // -------------- // // Computed props // @@ -142,63 +140,102 @@ export function useRecords( return 100; } }) - - const filteredInstanceItemsComp = computed(() => { + + const showScrollTopBtn = computed(() => { + if (filteredRecordItemsByClass.value[selectedIRI.value]?.length > 7) return true; + return false; + }); + + const fetchedItemCount = computed(() => { + if (includeSubClasses.value && Array.isArray(allSubClasses[selectedIRI.value]) && allSubClasses[selectedIRI.value].length > 0 ) { + let allclass_array = [selectedIRI.value].concat(allSubClasses[selectedIRI.value]) + let itemCount = 0; + for (const cl of allclass_array) { + if (recordItemsByClass[cl]) { + itemCount += Object.values(recordItemsByClass[cl]).length; + } + } + return itemCount; + } + if (recordItemsByClass[selectedIRI.value]) { + return Object.values(recordItemsByClass[selectedIRI.value]).length; + } + return null; + }); + + const filteredRecordItemsAll = computed(() => { let txt = searchText.value.toLowerCase().trim(); + if (txt.length == 0) return sortItems(Object.values(recordItemsAll)) return sortItems( - [...instanceItemsComp.value].filter((item) => { + Object.values(recordItemsAll).filter((item) => { if (txt.length == 0) return true; - return searchableFields.some((field) => { - if (!(field in item.props)) return false; - const value = item.props[field]?.toString().toLowerCase().trim(); - if (Array.isArray(value)) { - return value.some((val) => { - return val.includes(txt); - }) - } else { - return value.includes(txt); - } - }); + if (!('_searchBlob' in item.props)) return false; + return item.props._searchBlob.includes(txt); }) ) }); - const matchedInstanceItemsComp = computed(() => { + const filteredRecordItemsByClass = computed(() => { let txt = searchText.value.toLowerCase().trim(); - return sortItems( - [...instanceItemsComp.value].filter((item) => { - if (txt.length == 0) return true; - return searchableFields.some((field) => { - if (!(field in item.props)) return false; - const value = item.props[field]?.toString().toLowerCase().trim(); - return value === txt; - }); - }) - ) + const map = {}; + for (const cl of Object.keys(recordItemsByClass)) { + map[cl] = sortItems( + Object.values(recordItemsByClass[cl]).filter((item) => { + if (txt.length == 0) return true; + if (!('_searchBlob' in item.props)) return false; + if (textMatchType.value == 'exact') { + return searchableFields.some((field) => { + if (!(field in item.props)) return false; + const value = item.props[field]?.toString().toLowerCase().trim(); + return value === txt; + }); + } else { + return item.props._searchBlob.includes(txt); + } + }) + ) + } + return map }); - + + const filteredRecordItemsForClassWithSubclassItems = computed(() => { + let items = []; + if (!includeSubClasses.value) { + return items; + } + if (Array.isArray(allSubClasses[selectedIRI.value]) && allSubClasses[selectedIRI.value].length > 0 ) { + let allclass_array = [selectedIRI.value].concat(allSubClasses[selectedIRI.value]) + for (const cl of allclass_array) { + if (filteredRecordItemsByClass.value[cl]) { + items = items.concat(filteredRecordItemsByClass.value[cl]) + } + + } + } + return sortItems(items) + }); + // --------- // // Functions // // --------- // // fetch new items at bottom of scroller function onScrollEnd() { - debouncedScrollEnd(); + debouncedScrollEnd(selectedIRI.value); } - const debouncedScrollEnd = debounce(async () => { + const debouncedScrollEnd = debounce(async (classIRI) => { // Only fetch new items at bottom of scroller if there is not any search text // Continued fetching of more items while there is search text will be handled // by the watcheffect function. if (searchText.value) { return } - if (config.value.use_service) { - if (hasUnfetchedPages(selectedIRI.value) && !isFetchingPage.value) { - await fetchNextPage(); + if (hasUnfetchedPages(classIRI) && !isFetchingPage.value) { + await fetchNextPage(classIRI); } else { - console.log("Last page already fetched") + console.log(`Last page already fetched: ${classIRI}`) } } }, 1000); @@ -211,7 +248,7 @@ export function useRecords( async function onTypingPause(textVal) { if (!searchText.value || searchText.value.length < configVarsMain.serviceConstrainedSearch.min_characters ) return; - await fetchNextPage(searchText.value); + await fetchNextPage(selectedIRI.value, searchText.value); } // User types, debounce effect monitors pauses and waits for configured time // before making the first constrained request. @@ -225,20 +262,19 @@ export function useRecords( // Continued fetching of more items while there is search text is handled by // the watcheffect function. - async function fetchNextPage(matchText='') { - if (isFetchingPage.value || !hasUnfetchedPages(selectedIRI.value, matchText)) return; + async function fetchNextPage(classIRI, matchText='') { + if (isFetchingPage.value || !hasUnfetchedPages(classIRI, matchText)) return; isFetchingPage.value = true; try { const result = await fetchFromService( 'get-paginated-records-constrained', - selectedIRI.value, + classIRI, allPrefixes, matchText ); if (result.status === null) { console.error(result.error); } - getInstanceItems(); // rebuild local list of items } catch (err) { console.error(err); } finally { @@ -246,100 +282,6 @@ export function useRecords( } } - function getInstanceItems() { - // --- - // The goal of this method is to populate the list of data objects of the selected type - // --- - var x = itemsTrigger.value; - if (!selectedIRI.value) { - return []; - } - // find nodes with triple predicate == rdf:type, and triple object == the selected class - // if the class is a configured priority class with include_subclasses = true, find nodes - // for the selected class and all of its subclasses - var quads; - if (includeSubClasses.value) { - let allclass_array = [selectedIRI.value] - if (Array.isArray(allSubClasses[selectedIRI.value]) && allSubClasses[selectedIRI.value].length > 0 ) { - allclass_array = allclass_array.concat(allSubClasses[selectedIRI.value]) - } - quads = []; - for (const cl of allclass_array) { - const mySubArray = rdfDS.getLiteralAndNamedNodes( - namedNode(RDF.type.value), - cl, - allPrefixes - ) - quads = quads.concat(mySubArray); - } - } else { - quads = rdfDS.getLiteralAndNamedNodes( - namedNode(RDF.type.value), - selectedIRI.value, - allPrefixes - ); - } - // Create list items from quads - var instanceItemsArr = []; - quads.forEach((quad) => { - var extra = ''; - if (quad.subject.termType === 'BlankNode') { - extra = ' (BlankNode)'; - } - var relatedTrips = rdfDS.getSubjectTriples(quad.subject); - var item = { - title: quad.subject.value + extra, - value: quad.subject.value, - props: { - subtitle: quad.object.value, - quad: quad, - itemValue: quad.subject.value, - }, - }; - let labelTemplate = hasConfigDisplayLabel(quad.object.value, allPrefixes, configVarsMain) - let labelParts = {} - relatedTrips.forEach((quad) => { - if (!Object.hasOwn(item.props, quad.predicate.value)) { - item.props[quad.predicate.value] = []; - } - if (quad.object.termType === 'BlankNode') { - var bnItem = {}; - var blankNodeTrips = rdfDS.getSubjectTriples(quad.object); - blankNodeTrips.forEach((bnquad) => { - bnItem[bnquad.predicate.value] = bnquad.object.value; - }); - item.props[quad.predicate.value].push(bnItem); - } else { - item.props[quad.predicate.value].push(quad.object.value); - } - let predCuri = toCURIE(quad.predicate.value, allPrefixes) - // If current predicate is used for display label generation, store it - if ( labelTemplate && labelTemplate.includes(predCuri)) { - if (!labelParts[predCuri]) { - labelParts[predCuri] = [] - } - labelParts[predCuri].push(quad.object.value) - } - }); - item.props._prefLabel = ''; - if (item.props.hasOwnProperty(SKOS.prefLabel.value)) { - item.props._prefLabel = item.props[SKOS.prefLabel.value][0]; - } - // Generate display label if possible - item.props._displayLabel = ''; - if (labelTemplate) { - let displayLabel = getConfigDisplayLabel(labelTemplate, labelParts, configVarsMain, rdfDS, allPrefixes) - if (displayLabel) { - item.props._displayLabel = displayLabel; - } - } - instanceItemsArr.push(item); - }); - instanceItemsComp.value = [...instanceItemsArr]; - if (instanceItemsComp.value.length > 7) showScrollTopBtn.value = true; - fetchedItemCount.value = instanceItemsComp.value.length; - } - function getSortValue(item) { for (const field of searchableFields) { const value = item.props[field]; @@ -364,6 +306,110 @@ export function useRecords( }) } + function enqueueChanges(records) { + for (const r of records) { + itemQueue.add(r); + } + processQueue(); + } + + async function processQueue() { + if (isProcessingItemQueue) return; + isProcessingItemQueue = true; + while (itemQueue.size > 0) { + // take a batch + const batch = Array.from(itemQueue).slice(0, 10); + batch.forEach((record) =>{ + itemQueue.delete(record) + updateRecordItem(record) + }); + // yield to UI thread + await nextTick(); + } + isProcessingItemQueue = false; + } + + function updateRecordItem(record) { + // This function does not care if a record item already exists in the + // object; it builds the item and adds it nevertheless. + // First we get the record's quad + let mainQuad = getPidQuad(record, rdfDS.data.graph); + if (!mainQuad) { + console.log(`No PID quad found in graph for record: ${record}; skipping update of this item`) + return + } + let recordClass = mainQuad.object.value; + // Now get related quads + var relatedTrips = rdfDS.getSubjectTriples(mainQuad.subject); + // Initialize item + var item = { + title: record, + value: record, + props: { + subtitle: recordClass, + quad: mainQuad, + itemValue: record, + }, + }; + let labelTemplate = hasConfigDisplayLabel(recordClass, allPrefixes, configVarsMain) + let labelParts = {} + relatedTrips.forEach((quad) => { + if (!Object.hasOwn(item.props, quad.predicate.value)) { + item.props[quad.predicate.value] = []; + } + if (quad.object.termType === 'BlankNode') { + var bnItem = {}; + var blankNodeTrips = rdfDS.getSubjectTriples(quad.object); + blankNodeTrips.forEach((bnquad) => { + bnItem[bnquad.predicate.value] = bnquad.object.value; + }); + item.props[quad.predicate.value].push(bnItem); + } else { + item.props[quad.predicate.value].push(quad.object.value); + } + let predCuri = toCURIE(quad.predicate.value, allPrefixes) + // If current predicate is used for display label generation, store it + if ( labelTemplate && labelTemplate.includes(predCuri)) { + if (!labelParts[predCuri]) { + labelParts[predCuri] = [] + } + labelParts[predCuri].push(quad.object.value) + } + }); + item.props._prefLabel = ''; + if (item.props.hasOwnProperty(SKOS.prefLabel.value)) { + item.props._prefLabel = item.props[SKOS.prefLabel.value][0]; + } + // Generate display label if possible + item.props._displayLabel = ''; + if (labelTemplate) { + let displayLabel = getConfigDisplayLabel(labelTemplate, labelParts, configVarsMain, rdfDS, allPrefixes) + if (displayLabel) { + item.props._displayLabel = displayLabel; + } + } + // Now put together single searchable blob + item.props._searchBlob = '' + for (const field of searchableFields) { + if (!(field in item.props)) continue; + let value = item.props[field]?.toString().toLowerCase().trim(); + if (!Array.isArray(value)) { + value = [value] + } + for (const v of value) { + item.props._searchBlob = item.props._searchBlob + v + } + } + // Now that we have the complete item, we can add it to the tracking objects + // Class records tracker + if (!recordItemsByClass.hasOwnProperty(recordClass)) { + recordItemsByClass[recordClass] = {}; + } + recordItemsByClass[recordClass][record] = item; + // All records tracker + recordItemsAll[record] = item; + } + // ------- // // Returns // // ------- // @@ -371,14 +417,11 @@ export function useRecords( classRecordsLoading, currentProgress, fetchedItemCount, - filteredInstanceItemsComp, - getInstanceItems, + fetchNextPage, headingHover, + includedClasses, includeSubClasses, - instanceItemsComp, isFetchingPage, - matchedInstanceItemsComp, - newTypeSelected, onScrollEnd, onUserTyping, orderTopDown, @@ -388,5 +431,9 @@ export function useRecords( showScrollTopBtn, textMatchType, totalItemCount, + recordItemsByClass, + filteredRecordItemsAll, + filteredRecordItemsByClass, + filteredRecordItemsForClassWithSubclassItems, }; } \ No newline at end of file diff --git a/src/composables/useShapes.js b/src/composables/useShapes.js index e80e064..e7f11b3 100644 --- a/src/composables/useShapes.js +++ b/src/composables/useShapes.js @@ -3,9 +3,9 @@ * @description This composable reads a ttl file with shacl shapes and returns * a set of reactive variables used by the root application component */ -import { reactive, toRaw} from 'vue'; +import { reactive, toRaw, ref} from 'vue'; import { ShapesDataset } from 'shacl-tulip'; -import { findObjectByKey, toIRI } from '@/modules/utils' +import { findObjectByKey, toIRI, toCURIE, getDisplayName, includeClass} from '@/modules/utils' import { SHACL, RDFS} from '@/modules/namespaces'; const basePath = import.meta.env.BASE_URL || '/'; @@ -16,6 +16,14 @@ export function useShapes(config) { // ---- // const defaultURL = `${basePath}dlschemas_shacl.ttl`; const shapesDS = new ShapesDataset(reactive({})); + // These are all arays of classes that are eventually represented in the + // main class-selection pane in the ShaclVue* components + const idFilteredNodeShapeNames = ref([]); + const noEditClassList = ref([]); + const filteredNodeShapeNames = ref([]); + const priorityFilteredNodeShapeNames = ref([]); + const orderedNodeShapeNames = ref([]); + const allClassItems = ref([]); // ----------------- // // Lifecycle methods // @@ -134,6 +142,138 @@ export function useShapes(config) { shapesDS.data.propertyGroups['_default'][RDFS.label.value] = "Additional properties"; shapesDS.data.propertyGroups['_default'][SHACL.order.value] = high_order + 100; } + + const getIdFilteredNodeShapeNames = ((configVarsMain, ID_IRI) => { + if (configVarsMain.showShapesWoId === true) { + return shapesDS.data.nodeShapeNamesArray; + } + var shapeNames = []; + for (var n of shapesDS.data.nodeShapeNamesArray) { + if ( + findObjectByKey( + shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[n]].properties, + SHACL.path.value, + ID_IRI.value + ) + ) { + shapeNames.push(n); + } + } + return shapeNames; + }); + + const getNoEditClassList = ((configVarsMain, allPrefixes) => { + if (configVarsMain.noEditClasses?.length == 0) return [] + var names = idFilteredNodeShapeNames.value; + var shapeNames = []; + for (var n of names) { + // First get IRI and prefix + var n_iri = shapesDS.data.nodeShapeNames[n] + if (includeClass(n_iri, configVarsMain, allPrefixes) && + configVarsMain.noEditClasses?.indexOf(toCURIE(n_iri, allPrefixes)) >= 0) { + shapeNames.push(n); + } + } + return shapeNames.sort((a, b) => + getDisplayName( + shapesDS.data.nodeShapeNames[a], + configVarsMain, + allPrefixes, + shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[a]] + ).toLowerCase() + .localeCompare( + getDisplayName( + shapesDS.data.nodeShapeNames[b], + configVarsMain, + allPrefixes, + shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[b]] + ).toLowerCase() + ) + ); + }); + + const getFilteredNodeShapeNames = ((configVarsMain, allPrefixes) => { + var names = idFilteredNodeShapeNames.value; + // If all relevant config arrays are empty, show all classes + if ( + configVarsMain.showClasses?.length == 0 && + configVarsMain.showClassesWithPrefix?.length == 0 && + configVarsMain.hideClasses?.length == 0 && + configVarsMain.hideClassesWithPrefix?.length == 0 && + configVarsMain.noEditClasses?.length == 0 + ) { + return names; + } + var shapeNames = []; + for (var n of names) { + // First get IRI and prefix + var n_iri = shapesDS.data.nodeShapeNames[n] + if (includeClass(n_iri, configVarsMain, allPrefixes) && configVarsMain.noEditClasses.indexOf(toCURIE(n_iri, allPrefixes)) < 0) { + shapeNames.push(n); + } + } + return shapeNames; + }); + + const getPriorityFilteredNodeShapeNames = ((priorityClassList) => { + var names = filteredNodeShapeNames.value; + var shapeNames = []; + for (var n of names) { + var n_iri = shapesDS.data.nodeShapeNames[n] + if (!priorityClassList.value.includes(n_iri)) { + shapeNames.push(n); + } + } + return shapeNames; + }) + + const getOrderedNodeShapeNames = ((configVarsMain, allPrefixes) => { + return priorityFilteredNodeShapeNames.value.sort((a, b) => + getDisplayName( + shapesDS.data.nodeShapeNames[a], + configVarsMain, + allPrefixes, + shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[a]] + ).toLowerCase() + .localeCompare( + getDisplayName( + shapesDS.data.nodeShapeNames[b], + configVarsMain, + allPrefixes, + shapesDS.data.nodeShapes[shapesDS.data.nodeShapeNames[b]] + ).toLowerCase() + ) + ); + }) + + const getAllClassItems = ((configVarsMain, allPrefixes, getClassIcon) => { + let items = []; + for (const node of orderedNodeShapeNames.value) { + const classIRI = shapesDS.data.nodeShapeNames[node]; + const displayName = getDisplayName( + classIRI, + configVarsMain, + allPrefixes, + shapesDS.data.nodeShapes[classIRI] + ); + const description = shapesDS.data.nodeShapes[classIRI][RDFS.comment.value]; + items.push( + { + title: displayName, + value: classIRI, + props: { + title: displayName, + iri: classIRI, + subtitle: toCURIE(classIRI, allPrefixes), + icon: getClassIcon(classIRI), + description: description, + totalItemCount: null, + }, + } + ) + } + return items; + }) // ------- // // Returns // @@ -144,5 +284,17 @@ export function useShapes(config) { updateShapesFromDefault, updateShapes, updatePropertyGroups, + idFilteredNodeShapeNames, + noEditClassList, + filteredNodeShapeNames, + priorityFilteredNodeShapeNames, + orderedNodeShapeNames, + allClassItems, + getIdFilteredNodeShapeNames, + getNoEditClassList, + getFilteredNodeShapeNames, + getPriorityFilteredNodeShapeNames, + getOrderedNodeShapeNames, + getAllClassItems, }; } diff --git a/src/composables/useWizard.js b/src/composables/useWizard.js index 0442f47..65ce6ab 100644 --- a/src/composables/useWizard.js +++ b/src/composables/useWizard.js @@ -1,5 +1,5 @@ -import { ref, reactive, toRaw} from "vue"; -import { fillStringTemplate, findObjectByKey, findObjectIndexByKey, nodeShapeHasProperty} from "@/modules/utils"; +import { ref, reactive, toRaw } from "vue"; +import { fillStringTemplate, findObjectByKey, findObjectIndexByKey, nodeShapeHasProperty } from "@/modules/utils"; import { toCURIE, toIRI } from "shacl-tulip"; import { RDF } from "@/modules/namespaces"; import { DataFactory } from 'n3'; @@ -8,10 +8,11 @@ import { useNunjucks } from "@/composables/useNunjucks"; const { fillNunjucksTemplate } = useNunjucks(); export function showWizardGroup(configVarsMain, context, classUri, allPrefixes, shapesDS) { - console.log("Checking if wizard group should be shown") const classCurie = toCURIE(classUri, allPrefixes); + // all classes wizards + const all_class_selection = context == '_class' && configVarsMain.wizardEditorSelection?._classes; // class-based wizards ? - const selection = configVarsMain.wizardEditorSelection?.[classCurie]?.[context] + const selection = configVarsMain.wizardEditorSelection?.[classCurie]?.[context]; // slot-based wizards ? let slot_selection = false; if (configVarsMain.wizardEditorSelection?._slots) { @@ -27,7 +28,7 @@ export function showWizardGroup(configVarsMain, context, classUri, allPrefixes, } } } - const rval = slot_selection || selection && Array.isArray(selection) && selection.length > 0; + const rval = all_class_selection || slot_selection || selection && Array.isArray(selection) && selection.length > 0; return rval } @@ -52,7 +53,13 @@ export function useWizard() { let classCurie = toCURIE(class_IRI, allPrefixes) // Load wizard editors if any let wizardsToAdd = new Set(); - // First, class-specific wizards + // First, any wizards that should show for all classes + if (context == '_class' && configVarsMain.wizardEditorSelection?._classes){ + for (const wizard of configVarsMain.wizardEditorSelection?._classes) { + wizardsToAdd.add(wizard) + } + } + // then class-specific wizards if (configVarsMain.wizardEditorSelection?.[classCurie]?.[context]){ for (const wizard of configVarsMain.wizardEditorSelection?.[classCurie]?.[context] || []) { wizardsToAdd.add(wizard) @@ -90,6 +97,8 @@ export function useWizard() { async function handleWizardSave(context, class_uri, wizardData, rdfDS, savedNodes, nodesToSubmit, subject_uri=null, formData) { wizardDialog.value = false; selectedWizard.value = null; + // Add class uri to wizard data + wizardData.class_uri = class_uri; // if the context is '_record', add the current formData node ID as "pid" if (context == '_record') { wizardData.pid = subject_uri; @@ -103,12 +112,11 @@ export function useWizard() { } // And then parse TTL, adding quads to graph data let newQuads = await rdfDS.parseTTLandDedup(newTTL); - rdfDS.triggerReactivity(); // Now we process each added quad differently based on context: // if context is _record, we need to work with formData of current record being edited // if context is _class or higher level, we can ignore formData because everything happens via template if (context == '_record') { - for (const q of newQuads) { + for (const q of newQuads.quads) { // If the quad has the current node ID as subject, we need to add it to formdata, and also remove the quad from graph store // If the quad has a different named node as subject, we need to keep track of it for submission purposes if (q.subject.value == subject_uri) { @@ -140,8 +148,8 @@ export function useWizard() { } } } else { - console.log("The context was not _record...") - for (const q of newQuads) { + rdfDS.emitAddedRecords(newQuads.records) + for (const q of newQuads.quads) { // Here we do not have to keep track of quads added to the graph, // because there's no parent form that can still be cancelled. // We need to keep track of the named nodes saved to the graph, for submission diff --git a/src/modules/utils.js b/src/modules/utils.js index 40687b3..619f569 100644 --- a/src/modules/utils.js +++ b/src/modules/utils.js @@ -108,6 +108,14 @@ export function addCodeTagsToText(text, prepend, append) { return result; } +export function truncateText(str, n, useWordBoundary){ + if (str.length <= n) { return str; } + const subString = str.slice(0, n-1); + return (useWordBoundary + ? subString.slice(0, subString.lastIndexOf(" ")) + : subString) + "…"; +}; + export function findObjectByKey(array, key, value) { return array.find((obj) => obj[key] === value); } @@ -621,6 +629,33 @@ export function collectBlankNodeHierarchy(store, rootBNode) { return collected; } +export function getRootNodes(store, node, visited = new Set()) { + if (visited.has(node.value)) return []; + visited.add(node.value); + const parents = store.getQuads(null, null, node, null); + let roots = []; + for (const p of parents) { + if (p.subject.termType === 'NamedNode') { + roots.push(p.subject.value); + } else { + roots = roots.concat(getRootNodes(store, p.subject, visited)); + } + } + return roots; +} + +export function getUniqueRootNodes(quads, store) { + let uniqueRecords = new Set(); + for (const q of quads) { + if (q.subject.termType === 'NamedNode') { + uniqueRecords.add(q.subject.value); + } else { + let rootNodeValues = getRootNodes(store, q.subject) + for (const r of rootNodeValues) uniqueRecords.add(r) + } + } + return Array.from(uniqueRecords) +} export function getRecordQuads(pid, graph, recursive=false) { // Return an array of quads related to a specific named node -- 2.52.0 From 7cdde788783de6ede13dca4a7e9f372746febc59 Mon Sep 17 00:00:00 2001 From: Stephan Heunis Date: Wed, 13 May 2026 11:55:03 +0200 Subject: [PATCH 2/2] Introduce new 'ShaclVueStarter' app variant This work is in response to the identified need for kick-starting the user-based entry of domain-specific knowledge into a 'shacl-vue' metadata system, so that a critical mass for continued use of the system is available soon after deploying it for the first time. The new 'ShaclVueStarter' component is designed to encourage users to enter minimal records in a quick, intuitive, and interactive way. It has the following features: 1. It serves as a new variant of the main entrypoint of a 'shacl-vue' deployment. This requires the specification of a Vite-recognizable environment variable, 'VITE_SHACLVUE_VARIANT' that influences whether the application is built with the (default) 'ShaclVue' component or the new 'ShaclVueStarter' component. An example command for building the starter app: VITE_SHACLVUE_VARIANT=starter npm run build:app If the environment variable is not specified, the default application will be built. 2. Instead of making only a single class selectable at a time, 'ShaclVueStarter' shows all record types (aka classes), and many records per class. The user can scroll down the main view through all classes, and relevant records can be scrolled within each class 'card'. The left-hand-side pane now shows the list of classes with added checkboxes, which allows including/exluding classes from being displayed in the main view. 3. A 'ShaclVueStarter' deployment uses the new '_classes' option of the 'wizard_selection' configuration to specify a wizard that will be shown for all classes. This allows the use of a generic 'add new record' wizard that only asks for a display name and description, with example config: AddRecordWizard: name: Add New Record Wizard tooltip: Add a new record icon: mdi-plus-circle-outline description: Add the display name and description and then hit *Save* inputs: - prop: name name: Display name description: The display name of the new record type: text placeholder: 'My record title' required: true - prop: description name: Description description: The description of the new record type: text-paragraph placeholder: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod...' template: content:add-record 4. Text search functionality provides the same UX as before, however the search now happens across all records, i.e. not only on a single class-level. 5. Unnecessary detail and functionality from the default 'ShaclVue' application are not available in the starter application. This mainly means the record viewer is minimalistic, containing just the display name and a mini menu button with options to edit the record (which will open the established form) and view the record RDF. New/updated functionality to support all of the above: - New 'ShaclVueStarter', 'ShaclVueRecordsMini', and 'NodeShapeViewerMini' components - 'App.vue' now dynamically loads the main entrypoint components and uses 'component :is=' to render the component that is selected based on the 'VITE_SHACLVUE_VARIANT' value - 'ShaclVueStarter' uses a new 'getFirstPages()' function during app startup in order to fetch the first page from the dumpthings backend for all classes. This allows getting upfront info about number of records, and some actual records, to display on the main view that shows all classes and their records. - In 'ShaclVueStarter', the 'selectType' is only called once on app startup, using the 'Thing' class as the selected type. - 'ShaclVueStarter' has a new computed variable, 'itemsByClass', which takes all filtered records as input and groups them by class. This reactive ref is provided as the input of records to 'ShaclVueRecordsMini' per class. - As with 'ShaclVueRecords', 'ShaclVueRecordsMini' emits an event when the user scrolls to the bottom of the list of records. However, 'ShaclVueStarter' has a new function that listens for this event and will then fetch the next page of records for that specific class. - 'NodeShapeViewerMini' has the simplistic rendering of a record display name and menu options. --- src/App.vue | 12 +- src/components/NodeShapeViewerMini.vue | 392 ++++++++++++ src/components/ShaclVueRecordsMini.vue | 125 ++++ src/components/ShaclVueStarter.vue | 841 +++++++++++++++++++++++++ 4 files changed, 1368 insertions(+), 2 deletions(-) create mode 100644 src/components/NodeShapeViewerMini.vue create mode 100644 src/components/ShaclVueRecordsMini.vue create mode 100644 src/components/ShaclVueStarter.vue diff --git a/src/App.vue b/src/App.vue index 47627e5..c831c39 100644 --- a/src/App.vue +++ b/src/App.vue @@ -1,15 +1,23 @@ diff --git a/src/components/NodeShapeViewerMini.vue b/src/components/NodeShapeViewerMini.vue new file mode 100644 index 0000000..14b9cc9 --- /dev/null +++ b/src/components/NodeShapeViewerMini.vue @@ -0,0 +1,392 @@ + + + + + \ No newline at end of file diff --git a/src/components/ShaclVueRecordsMini.vue b/src/components/ShaclVueRecordsMini.vue new file mode 100644 index 0000000..f8005f7 --- /dev/null +++ b/src/components/ShaclVueRecordsMini.vue @@ -0,0 +1,125 @@ + + + + + + + \ No newline at end of file diff --git a/src/components/ShaclVueStarter.vue b/src/components/ShaclVueStarter.vue new file mode 100644 index 0000000..94f9e8d --- /dev/null +++ b/src/components/ShaclVueStarter.vue @@ -0,0 +1,841 @@ + + + + + + + + + + + -- 2.52.0