Learning Object Metadata Crowd Tagging (LOMCT) BB – Design Document – Prometheus-X Components & Services

Learning Object Metadata Crowd Tagging (LOMCT) BB – Design Document

Learning Object Metadata Crowd Tagging is a method for tagging and reviewing digital learning resources such as videos, courses, and documents using a crowd of individuals. The goal is to make these resources more discoverable and searchable by adding relevant keywords, descriptions, reviews and other useful metadata. This process is carried out via the combination of a browser (Chrome) extension and a Learning Record Store (LRS) that allows multiple users to submit review and metadata edit proposal of learning resources. All the reviews and metadata edit proposals are sent in xAPI format and stored in a LRS.

The LOMCT is a data service (browser extension) which facilitates the collection of feedback from teachers, learners, course designers, ... about LO (metadata, reviews). The LO index is a data source (LRS) for organizations which want to store metadata and reviews of LO submitted by their learners, teachers or course designers.

Here is a visual of the LOMCT extension home

The LOMCT extension plays a crucial role within the dataspace ecosystem, serving as a vital component that, despite its integral presence, does not engage in direct interaction with the dataspace itself. This design choice highlights the modular nature of the system, where each component has specialized functions.

For the LOMCT to effectively support its intended use cases, it must collaborate with a Learning Record Store (LRS). The LRS is the custodian of the LOMCT traces that will be shared in the dataspace. This connection from the LRS to the dataspace is established via a Prometheus-X Dataspace Connector (PDC), which ensures secure and efficient data transfer between components.

The primary function of the LOMCT is to generate xAPI traces. These traces are essential for LO discovery and provide valuable information on learning processes. In addition, LOMCT facilitates data exchange, ensuring that relevant information is easily accessible and usable.

However, it is important to note that while the LOMCT is deeply involved in data generation, the actual exchange of data with the data space itself is mediated by the LRS. This underlines the essential role of the LRS in the architecture, as a gateway for the flow of data in and out of the data space, thus maintaining a streamlined and organized data management system.

Technical usage scenarios & Features

Key functionalities:

Value-added:

Features/main functionalities

Features:

The LOMCT interacts with one or more Learning Records Stores. This LRS is deployed by an organization. Each organization will have its own metadata ecosystem, which can be shared at business model level in the dataspace. Data sent from the LOMCT to the LRS belongs to the organization holding the LRS. This implies :

The Inokufu organization will offer an additional metadata moderation service.

Technical usage scenarios

Here is an example of how LOMCT is used by an university professor:

Here is an example of how LOMCT is used by an employee of company:

Requirements

Requirement ID Short description BB input format BB output format Any other constraints Verified by scenario Requirement type
BB-REQ_ID__1 LOMCT must request the consent of collect data
BB-REQ_ID__1.1 Individuals must consent to the use of their data in LOMCT : mail, username and biography chek box chek box If the answer is no, the user has no access to the extension BB-SC-LOMCT-01 DEP
BB-REQ_ID__2 LOMCT must be connected to an LRS credential of LRS
BB-REQ_ID__2.1 The organization's LRS must be connected to the dataspace to access the orchestrator's LRS PDC PDC BB-SC-LOMCT-02 DEP
BB-REQ_ID__2.2 LOMCT must send traces to organization's LRS in less than 30 seconds (after triggering the individual) xAPI xAPI BB-SC-LOMCT-03 PERF

Integrations

Direct Integrations with Other BBs

Interact with LRS

How?

Why?

Relevant Standards

Data Format Standards

Data format:

Mapping to Data Space Reference Architecture Models


block-beta

columns 3

block:group1
columns 4
LearningObject1 LOMCT_orga1 LRS_orga1 LRS_PDC1 space space space space LearningObject2 LOMCT_orga2 LRS_orga2 LRS_PDC2 space space space space LearningObject3 LOMCT_orga3 LRS_orga3 LRS_PDC3
end

LRS_orchestrator_PDC:1 LRS_orchestrator:1

classDef colorA fill:#D22FF7,color:#fff
classDef colorEx fill:#01D1D1,color:#fff
classDef colorED fill:#6E7176,color:#fff
class LearningObject1 colorEx
class LearningObject2 colorEx
class LearningObject3 colorEx
class LRS_orga1 colorEx
class LRS_orga2 colorEx
class LRS_orga3 colorEx
class LRS_orchestrator colorEx
class LOMCT_orga1 colorED
class LOMCT_orga2 colorED
class LOMCT_orga3 colorED
class LRS_PDC1 colorA
class LRS_PDC2 colorA
class LRS_PDC3 colorA
class LRS_orchestrator_PDC colorA

PDC : Prometheus-X Dataspace Connector

Each organization will generate metadata in its LRS, then send it to the LRS orchestrator for exchange in dataspace.

Input / Output Data

Input and output data are in the same format: xAPI. For reviews, data will be in xAPI format with the verb “http://id.tincanapi.com/verb/reviewed”. For metadata edit proposal, data will be in xAPI format with the verb “https://w3id.org/xapi/dod-isd/verbs/proposed”.

As LOMCT is not an API, no BB other than LRS will need to send or receive data.

Example 1: Review

An individual, Martin Dupont, teacher (lauriane.marxer+LOMCT@inokufu.com) want to write a review about this video:

https://www.youtube.com/watch?v=RFXGpG7agA0

His review is:

Here is the corresponding xAPI statement:

{
  "stored": "2024-12-04T08:25:06.571Z",
  "priority": "MEDIUM",
  "active": true,
  "client": "XXX",
  "lrs_id": "XXX",
  "activities": [
    "https://www.youtube.com/watch?v=RFXGpG7agA0"
  ],
  "agents": [
    "mailto:lauriane.marxer+lomct@inokufu.com"
  ],
  "statement": {
    "authority": {
      "objectType": "Agent",
      "name": "Mockup LOMCT all",
      "mbox": "mailto:contact@inokufu.com"
    },
    "stored": "2024-12-04T08:25:06.571Z",
    "context": {
      "extensions": {
        "http://schema.prometheus-x.org/extension/username": "Martin Dupont",
        "http://schema.prometheus-x.org/extension/biography": "Teacher "
      },
      "language": "en"
    },
    "actor": {
      "objectType": "Agent",
      "name": "Martin Dupont",
      "mbox": "mailto:lauriane.marxer+lomct@inokufu.com"
    },
    "timestamp": "2024-12-04T08:25:06.571Z",
    "version": "1.0.0",
    "id": "XXX",
    "result": {
      "score": {
        "raw": 4.5                  // Representing a 4.5/5 score
      },
      "response": "This Moodle plugin looks very interesting."
    },
    "verb": {
      "id": "http://id.tincanapi.com/verb/reviewed",
      "display": {
        "en-US": "reviewed"
      }
    },
    "object": {
      "id": "https://www.youtube.com/watch?v=RFXGpG7agA0",
      "objectType": "Activity"
    }
  },
  "verbs": [
    "http://id.tincanapi.com/verb/reviewed"
  ],
  "timestamp": "2024-12-04T08:25:06.571Z",
  "relatedActivities": [
    "https://www.youtube.com/watch?v=RFXGpG7agA0"
  ],
  "relatedAgents": [
    "mailto:lauriane.marxer+lomct@inokufu.com",
    "mailto:contact@inokufu.com"
  ],
}

Example 2: Metadata Edit Proposal

An individual, Martin Dupont, teacher (lauriane.marxer+LOMCT@inokufu.com) want to propose metadata edit about this video:

https://www.youtube.com/watch?v=hLE-5ElGlPM

His proposals are:

Here is the corresponding xAPI statement:


{
  "stored": "2024-12-04T08:25:06.571Z",
  "priority": "MEDIUM",
  "active": true,
  "client": "XXX",
  "lrs_id": "XXX",
  "activities": [
    "https://www.youtube.com/watch?v=hLE-5ElGlPM"
  ],
  "agents": [
    "mailto:lauriane.marxer+lomct@inokufu.com"
  ],
  "statement": {
    "authority": {
      "objectType": "Agent",
      "name": "Mockup LOMCT all",
      "mbox": "mailto:contact@inokufu.com"
    },
    "stored": "2024-12-04T08:25:06.571Z",
    "context": {
      "extensions": {
        "http://schema.prometheus-x.org/extension/username": "Martin Dupont",
        "http://schema.prometheus-x.org/extension/biography": "Teacher ",
	"http://id.tincanapi.com/extension/bloom": "discover",
	"http://id.tincanapi.com/extension/provider": "YouTube"
      },
      "language": "en"
    },
    "actor": {
      "objectType": "Agent",
      "name": "Martin Dupont",
      "mbox": "mailto:lauriane.marxer+lomct@inokufu.com"
    },
    "timestamp": "2024-12-04T08:25:06.571Z",
    "version": "1.0.0",
    "id": "XXX",
    "verb": {
      "id": "https://w3id.org/xapi/dod-isd/verbs/proposed",
      "display": {
        "en-US": "proposed"
      }
    },
    "object": {
      "id": "https://www.youtube.com/watch?v=hLE-5ElGlPM",
      "objectType": "Activity"
    }
  },
  "verbs": [
    "https://w3id.org/xapi/dod-isd/verbs/proposed"
  ],
  "timestamp": "2024-12-04T08:25:06.571Z",
  "relatedActivities": [
    "https://www.youtube.com/watch?v=hLE-5ElGlPM"
  ],
  "relatedAgents": [
    "mailto:lauriane.marxer+lomct@inokufu.com",
    "mailto:contact@inokufu.com"
  ],
}

Architecture

For the architecture diagrams, we assume that the organization has :

We also assume that individuals have completed the download and account creation process to use LOMCT:

Dynamic Behaviour

Behavior for displaying LO metadata to the individual in the LOMCT. Feature : Visualize all metadata edit proposals and reviews associated to a LO :

sequenceDiagram
   actor Individual as Individual
   Individual->>LOMCT: Open LOMCT extension
   LOMCT->>LOMCT: Detects the url where the user is located
   LOMCT->>LRS_organization: Request LO metadata
   LOMCT->>LRS_orchestrator: Request LO metadata
   LRS_organization->>LOMCT: Send LO metadata
   LRS_orchestrator->>LOMCT: Send LO metadata
   LOMCT->>LOMCT: Data organization of the 2 LRS
   LOMCT->>Individual: Display the extension with metadata

PDC : Prometheus-X Dataspace Connector

Behavior to edit the metadata of a resource by an individual in the LOMCT. Features : Submit metadata edit proposal for a Learning object (LO) by an individual and Write a review for a LO by an individual :

sequenceDiagram
   actor Individual as Individual
   Individual->>LOMCT: Open the LOMCT extension and click on the edit button OR on the add review button
   LOMCT->>Individual: Display edit form OR add review form
   Individual->>LOMCT: Fill in and return editing form OR review form
   LOMCT->>LOMCT: Convert edition form OR review form into xAPI
   LOMCT->>LRS_organization: Send trace
   LRS_organization->>PDC_organization: Send trace
   PDC_organization->>PDC_orchestrator: Send trace
   PDC_orchestrator->>LRS_orchestrator: Send trace

Configuration and deployment settings

Deployment and installation:

The organization must :

The individual (learner, teacher, ...) must :

Configuration

The organization must :

The user must :

Error scenarios defined

The idea of the risk table is to define the probable causes of failure in order to estimate the probability of encountering this failure, to evaluate its secondary effects and therefore to plan preventive or corrective actions.

We will assign 3 scores on a scale of 1 to 10 to potential failures:

Criticality is calculated as follows: criticality = detection x occurrence x severity

If criticality is greater than 10, then preventive action must be taken. If not, no.

ID Function involved Description of risk Effect of failure Cause of failure Evaluation - Detection Evaluation - Occurrence Evaluation - Severity Evaluation - Criticality Preventive actions
Error-Scenario_1 Submit metadata edit proposal and write a review of a Learning object (LO) by an individual Data may be lost during migration The organization doesn't receive the complete statements about the reviews or the metadata edit proposals Incorrect connection between LOMCT and LRS 1 2 4 8
Error-Scenario_2 Data could be transmitted to other non-targeted LRSs Exported data may be accessible to unauthorized persons They are not properly secured 6 1 9 54 Tracing output
Error-Scenario_3 Personal information is sent to LRS without consent Non-compliance with RGPD Implementation 4 2 9 72 Add a checkbox to consent to the use of personal data in the organization's LRS and require consent in the PDI to share it in the orchestrator's LRS
Error-Scenario_4 Visualize all metadata edit proposals and reviews associated to a LO The user proposes a false edition False metadata Space for expression 1 3 7 21 A unique user id is used to distinguish individuals contributions from the admin statements. The statements of the administrator (owner of the LRS), seen as the truth, are displayed in the “information” area. The statements of the individual (non-LRS owner), unverified, are visible in the “Suggested edits” area
Error-Scenario_5 Metadatas don't update Poor visualization of the metadata Slow update due to servers 1 2 4 8
Error-Scenario_6 Inadequate user interface No use of the platform UI design is misleading 4 7 5 140 Optimize extension, perform quantitative and qualitative tests
Error-Scenario_7 Wrong design choices: colors, shapes, ... No use of the platform UX visual choices such as colors and graphics can subliminally influence the perception of data. Graphs are non-inclusive 4 7 5 140 Perform quantitative and qualitative tests
Error-Scenario_8 Identical statements in the same or several LRS Duplicated information Several sources of LRS 1 7 5 45 Organization of LRS data: if they are identical, only 1 should be displayed
Error-Scenario_9 The user is linked to several organizations Too much information on the front page, information conflict Implementation 1 7 5 35 Only the most recent statement from the primary LRS is visible
Error-Scenario_10 Several reviews from the same individual are detected Obsolete reviews visible Implementation 2 5 3 30 Display only the most recent review
Error-Scenario_11 Several metadata edit proposal from the same individual are detected Obsolete metadata visible Implementation 2 5 3 30 All statements are displayed chronologically
Error-Scenario_12 Submit metadata edit proposal and write a review of a Learning object (LO) by the organization The organization may decide to change its LRS Complication in determining the source of truth Change of LRS 1 2 1 2
Error-Scenario_13 The organization's statements are not differentiated from individual's statements Impossible to determine right from wrong (assuming the organization is right) Implementation 5 5 5 125 Detect when actor = authority it is organization
Error-Scenario_14 Other Content Fragmentation No valorization of statements The same LO is available on multiple sites and platforms with different URLs 1 7 5 35 Assign a Global Unique Identifier (GUID) to each LO

Third Party Components & Licenses

External components and licenses:

OpenAPI Specification

The LOMCT code has no API. In fact, data exchange is generated by the LRS, when an individual submits an edit or review. This data is stored in a third-party LRS, which itself has an API. In our case, we use the LRS learning locker. API calls are generic to all LRSs.

The OpenAPI specification for an LRS is available here.

Codebase : Mockup version

To get a functional understanding of this mockup and see some sample traces, go here.

To have an access to the statements or simply test the extension without deploying LRS on your side, make a request on this document where the source and basc auth of LRS Inokufu are located : https://docs.google.com/document/d/15sV1DR43r0GAP8EV8HmT0b-k0m4glPzQUwgn95XYrSQ/edit

As explained above, the LOMCT code does not include any APIs, so there is no such thing as a dummy code. The only codebase used is that of the LRS with the /PUT /POST and /GET endpoints.

PUT

description: Store a single statement as a single member of a set.

POST

description: Store a set of statements (or a single statement as a single member of a set).

GET

description: Read a single xAPI Statement or multiple xAPI Statements.

Answer API of Learning Locker

First version

A first version of LOMCT has been deployed. Its purpose is to validate the functionalities mentioned in this design document. The “dataspace integration” part will be fine-tuned later”. Nevertheless, the principle is the same for this extension.

Installation

How to install LOMCT. Ask for an access to the Inokufu's LRS.

UX/UI

First Time Onboarding

First-onboarding

Home extension

home

Informations

More information

metadata-more

Metadata edit proposal

metadata-edit

Reviews

More reviews

Review-more

Add review

Review-add

Options

Options

Test specification

The Learning Object Metadata Crowd Tagging tests ensure that:

Test

The LOMCT testing strategy will focus on ensuring the accuracy, reliability, and performance of its functionality. We will use a combination of unit testing, integration testing, and user interface testing. The test environment will reproduce conditions similar to those in production in order to accurately validate BB behavior. Acceptance criteria will be defined based on user stories, functional requirements, and performance criteria.

Validate requirements and potential risks

Tests to validate requirements and potential risks.

Verified by scenario Description Test Status
BB-SC-LOMCT-01 Individuals must consent to the use of their data in LOMCT : mail, username and biography When creating an account, try to validate the account without validating the check box Validated : It is not possible to access the LOMCT without consenting to the sharing of its data
BB-SC-LOMCT-02 The organization's LRS must be connected to the data space to access the orchestrator's LRS Try to access the orchestrator's LRS without the organization's LRS being linked to the dataspace Not yet validated
BB-SC-LOMCT-03 LOMCT must send traces to organization's LRS in less than 30 seconds (after triggering the individual) Average send time < 30 seconds Not yet validated
Error-Scenario_2 Data could be transmitted to other non-targeted LRSs See where statements are sent Validated : Statements are sent only to the target LRS
Error-Scenario_3 Personal information is sent to LRS (other than email) Check the LRS for visible statements Validated : No personal information other than the e-mail is sent to LRS
Error-Scenario_4 The user proposes a false edition Post a proposal edit with fals information and see if it is diplayed in the "information" or in "Suggested edits" section Validated : It is displayed in the "Suggested edits"
Error-Scenario_6 Inadequate user interface Conduct quantitative and qualitative tests Not yet validated
Error-Scenario_7 Wrong design choices: colors, shapes, ... Conduct quantitative and qualitative tests Not yet validated
Error-Scenario_8 Identical statements in the same or several LRS Post the same metadata 2 times and see whether it is displayed 1 or 2 times Validated
Error-Scenario_9 The user is linked to several organizations Have 2 LRS and see if information from the first source is highlighted Validated : Primary source information is highlighted
Error-Scenario_10 Several reviews from the same individual are detected Post 2 reviews and see whether it is displayed 1 or 2 times Validated : display one time (most recent)
Error-Scenario_11 Several metadata edit proposal from the same individual are detected Post 2 metadata editing proposals and see if they are both displayed (chronologically) Validated : The 2 are displayed chronologically
Error-Scenario_13 The organization's statements are not differentiated from individual's statements Manually post an organization statement in the LRS with the verb “proposed” associated with an LO, then check in the LOMCT that this statement is visible in the “information” section Validated : The statement is display in the "information" section
Error-Scenario_19 Content Fragmentation Check if the statements are the same if the endpoints of the same url are different Partially validated : Ok for canonical urls

Manual Scenario (small features)

Small manual scenarios to prove features (button, ...)

Test Case 1: Verify Logo Icon

Objective: Ensure the extension logo matches the specified icon and color.

Precondition: The extension is installed and enabled.

Steps:

Expected Result: The extension logo icon and color match the specified design.

Result: Validated.

Test Case 2: Verify Extension Display on Click

Objective: Ensure the extension is displayed when the logo is clicked.

Precondition: The extension is installed and enabled.

Steps:

Expected Result: The extension popup is displayed as a 300 px square on top of the page.

Result: Validated.

Test Case 3: Verify Extension Removal on Click

Objective: Ensure the extension is hidden when the logo is clicked again.

Precondition: The extension popup is displayed.

Steps:

Expected Result: The extension popup is hidden.

Result: Validated.

...

In conclusion, all the small functional tests are validated.

Here you will find detailed tests.

Manual Usage Scenario

Using the personas, user stories, user flow, and data flow from the Wiki LOM use case, we established several test scenarios.

Persona 1: mmegauss (1 LRS)

First time install with 1 LRS

Validation:

Persona 2: mcgonagall (2 LRS)

First time install with 2 LRSs

Validation:

Persona 3: the authority of LRS: "https://XXX.com/data/xAPIx"

Test scenario 1

mmegauss (persona 1) writes a review:

Validation:

Test scenario 2

mcgonagall (persona 2) writes a review:

Validation:

Test scenario 3

mmegauss (persona 1) submits a metadata edit proposal:

Validation:

Test scenario 4

mcgonagall (persona 2) submits a metadata proposal:

Validation:

Test scenario 5

The authority (Persona 3) posts metadata edit directly in the LRS:

Validation:

Test scenario 6

mmegauss (persona 1) writes 2 reviews on the same learning object:

Validation:

Test scenario 7

mcgonagall (persona 2) writes 2 reviews on the same learning object:

Validation:

Test scenario 8

mmegauss (persona 1) submits several metadata edit proposals:

Validation:

UX/UI test

Conduct quantitative and qualitative tests to validate the extension's visuals. We will conduct interviews and “Attrakdiff” questionnaires.

Partners & roles

Inokufu (BB leader):

edtake

Usage in the dataspace

The LOMCT will be used as a potential source of data for a LRS and thus be part of the service chains:

Diagram of service chain Sharing LMS/Moodle Data for Visualization PDC : Prometheus-X Dataspace Connector