Web Analytics Learning Records Universal Connector (WALRUC) BB – Design Document – Prometheus-X Components & Services

Web Analytics Learning Records Universal Connector (WALRUC) BB – Design Document

Web Analytics Learning Records Universal Connector allows for the integration of web analytics data with a Learning Record Store (LRS) using the xAPI (Experience API) DASES standard. It enables the conversion of analytics data, such as the data collected by Matomo, into an xAPI format that can be stored and tracked in a LRS.

The WALRUC plugin plays a crucial role within the dataspace ecosystem, serving as a vital component that, despite its integral presence, does not engage in direct interaction with the dataspace itself. This design choice highlights the modular nature of the system, where each component has specialized functions.

For the WALRUC to effectively support its intended use cases, it must collaborate with a Learning Record Store (LRS). The LRS is the custodian of the WALRUC traces that will be shared in the dataspace. This connection from the LRS to the dataspace is established via a Prometheus-X Dataspace Connector (PDC), which ensures secure and efficient data transfer between components.

WALRUC's primary function is to generate xAPI traces from Matomo. These anonymized/pseudonymized traces are essential for training AI. In addition, WALRUC triggers data exchange, ensuring that relevant information is easily accessible and usable by converting the format.

However, it is important to note that while the WALRUC is deeply involved in data generation, the actual exchange of data with the data space itself is mediated by the LRS. This underlines the essential role of the LRS in the architecture, as a gateway for the flow of data in and out of the data space, thus maintaining a streamlined and organized data management system.

Technical Choice

LRS

The Walruc Plugin will connect an account of Matomo Analytics with Learning Record Stores of the same organization in order to export web analytics data from Matomo.

To perform our tests, we have deployed 1 LRS on Learning Locker. Github link to deploy an LRS with Learning Locker : https://github.com/LearningLocker/deploy

Please note that we are not making any changes to the LRS. The LRS just needs to be deployed for testing, but only the WALRUC extension is deliverable. The LRS will be deployed by each organization and will link the WALRUC extension to their matomo.

Matomo

Matomo is an open-source web analytics platform that offers detailed insights into website traffic and user behavior, serving as a powerful alternative to Google Analytics. Chosen for its strong focus on data privacy and compliance, Matomo ensures our deployment adheres to regulations like GDPR by providing tools for managing user consent and data.

Matomo can be deployed in two ways, and for the purpose of strengthening the test instance of the plugin, Matomo Cloud and Matomo On-premise must be functional:

Please note that we are not making any changes to the Matomo. The Matomo just needs to be deployed for testing, but only the WALRUC extension is deliverable. The matomo will be deployed by each organization, and they will install the WALRUC plugin on it.

Website

As previously explained, Matomo will serve as the tool to collect web analytics data, which will then be transmitted to the organization's LRS through the Walruc plugin. For this purpose, three websites should be used as sources of navigation traces: https://becomino.com/home https://constellation.inokufu.com/ https://inokufu.com/ http://localhost:3000

For our tests, we will be using these sites because Inokufu owns them and has already deployed a matomo on them. But any site with a matomo can test WALRUC.

Technical usage scenarios & Features

Key functionalities:

Value-added:

Features/main functionalities

Features:


flowchart LR

A[Collect Matomo web browsing analytics]--> B[Convert Matomo web browsing analytics into xAPI DASES format]--> C[Transfer data to an LRS]

Technical usage scenarios

A training organization can use WALRUC service to track and analyze the engagement and performance of their learners on their website. Here's an example of how this could work:

  1. The training organization integrates their website with the web analytics tool Matomo, which tracks user interactions and engagement with the website.

  2. The organization also sets up a Learning Record Store (LRS) like Learning Locker.

  3. The WALRUC is implemented to connect the web analytics tool with the LRS. This allows the organization to convert the web browsing analytics data into an xAPI DASES format that can be stored and tracked in the LRS.

  4. As learners interact with the website, data is collected and tracked in the LRS. This includes information like pages viewed, time spent on the site, ...

  5. The organization can use this data to track the performance of anonymized learners (or individual learners if they have their consent) and identify areas where they may need specific or additional support or resources.

  6. The organization can also use the data to track the overall engagement and effectiveness of their website, and make improvements as needed.

Requirements

Requirement ID Short description BB input format BB output format Any other constraints Verified by scenario Requirement type
BB-REQ_ID__1 The Matomo connected to WALRUC must ask users for consent to use their personal data
BB-REQ_ID__1.1 WALRUC converts the data collected by matomo. If the data is anonymized then WALRUC processes the anonymized data. The same applies if the data is pseudonomized or clear Matomo policies Matomo settings BB-SC-WALRUC-01 DEP
BB-REQ_ID__2 WALRUC must be connected to an LRS credential of LRS
BB-REQ_ID__2.1 The organization's LRS must be connected to the dataspace PDC PDC BB-SC-WALRUC-02 DEP
BB-REQ_ID__2.2 WALRUC must send traces to LRS in less than 30 seconds (after receiving the converted trace) xAPI xAPI BB-SC-WALRUC-03 PERF
BB-REQ_ID__3 WALRUC must be connected to BB LRC call API call API
BB-REQ_ID__3.1 BB LRC (ARIANE project) must convert traces from WALRUC matomo statement xAPI DASES statement BB-SC-WALRUC-04 DEP
BB-REQ_ID__3.2 WALRUC must send traces to LRC in less than 30 seconds (after receiving a matomo trace in WALRUC) matomo statement xAPI DASES statement BB-SC-WALRUC-05 PERF
BB-REQ_ID__3.3 LRC must send traces to WALRUC in less than 30 seconds (after conversion) matomo statement xAPI DASES statement BB-SC-WALRUC-06 PERF
BB-REQ_ID__4 WALRUC must be connected to Matomo
BB-REQ_ID__4.1 Matomo must send traces to WALRUC in less than 30 seconds matomo statement matomo statement BB-SC-WALRUC-07 DEP

Integrations

Direct Integrations with Other BBs

Interact with Learning Records Store

How?

Why?

Interact with Learning Record Converter

How?

Why?

Interact with Matomo

How?

Why?

Relevant Standards

Data Format Standards

Data format:

Mapping to Data Space Reference Architecture Models


block-beta

columns 4

block:group1
columns 4
Matomo1 WALRUC1 LRS_orga1 LRS_PDC1 space space space space Matomo2 WALRUC2 LRS_orga2 LRS_PDC2 space space space space Matomo3 WALRUC3 LRS_orga3 LRS_PDC3
end

orchestrator_PDC LRS_orchestrator MarketPlace_trainAI


classDef colorA fill:#D22FF7,color:#fff
classDef colorEx fill:#01D1D1,color:#fff
classDef colorED fill:#6E7176,color:#fff
classDef colorMP fill:#e69138,color:#fff
class Matomo1 colorEx
class Matomo2 colorEx
class Matomo3 colorEx
class LRS_orga1 colorEx
class LRS_orga2 colorEx
class LRS_orga3 colorEx
class LRS_orchestrator colorEx
class WALRUC1 colorED
class WALRUC2 colorED
class WALRUC3 colorED
class LRS_PDC1 colorA
class LRS_PDC2 colorA
class LRS_PDC3 colorA
class orchestrator_PDC colorA
class MarketPlace_trainAI colorMP

PDC : Prometheus-X Dataspace Connector Each organization will generate metadata in its LRS, then send it to the LRS orchestrator for exchange in dataspace.

Input / Output Data

Input in matomo format

Input and output data are in the same format: xAPI. For accessed page, data will be in xAPI format with the verb “https://w3id.org/xapi/netc/verbs/accessed”.

As WALRUC is not an API, no BB other than LRS will need to send or receive data.

WALRUC uses matomo data format as input data format.

Here is an example of Matomo logs for a user who visited a page on http://localhost:3000/test.html

{
    "siteName": "http://localhost:3000",
    "visitIp": "172.19.0.1",
    "user_id": null,
    "actionDetails": {
        "type": "action",
        "timestamp": 1733927799,
        "url": "http://localhost:3000/test.html",
        "title": "Test Matomo - Page test",
        "timeSpent": 4
    },
    "idVisit": 1,
    "visitorId": "7&k=",
    "languageCode": "fr-fr",
    "browserName": "CH",
    "countryCode": "fr",
    "regionCode": null,
    "city": null,
    "latitude": null,
    "longitude": null,
    "referrerKeyword": null,
    "referrerUrl": "",
    "interactions": 0,
    "location_ip": "\u0013\u0000\u0000",
    "visit_last_action_time": 1733927799,
    "visit_first_action_time": 1733925893,
    "visit_exit_idaction_url": 4,
    "visit_exit_idaction_name": 3,
    "visitor_returning": 0,
    "visitor_seconds_since_first": 0,
    "visitor_seconds_since_order": null,
    "visitor_count_visits": 1,
    "visit_goal_buyer": 0,
    "referer_name": null,
    "referer_type": 1,
    "idsite": "1",
    "profilable": 1,
    "visit_entry_idaction_url": 2,
    "visit_total_actions": "visit_total_actions + 1",
    "visit_total_interactions": "visit_total_interactions + 1",
    "visit_total_searches": 0,
    "config_client_type": 1,
    "config_device_brand": "",
    "config_device_model": "generic desktop",
    "config_device_type": 0,
    "visit_total_events": 0,
    "config_resolution": "1920x1080",
    "visit_total_time": 1907,
    "last_idlink_va": null,
    "custom_dimension_1": null,
    "custom_dimension_2": null,
    "custom_dimension_3": null,
    "custom_dimension_4": null,
    "custom_dimension_5": null,
    "time_spent_ref_action": 4,
    "action_name": "Test Matomo - Page test",
    "rec": "1",
    "r": "179197",
    "h": "15",
    "m": "36",
    "s": "39",
    "url": "http://localhost:3000/test.html",
    "_id": "89d6da37ae266b3d",
    "_idn": "0",
    "send_image": "0",
    "_refts": "0",
    "pv_id": "neds9i",
    "pf_net": "1",
    "pf_srv": "1",
    "pf_tfr": "2",
    "pf_dm1": "4",
    "uadata": {
        "formFactors": [
            "Desktop"
        ],
        "fullVersionList": [
            {
                "brand": "Chromium",
                "version": "131.0.6778.108"
            },
            {
                "brand": "Not_A Brand",
                "version": "24.0.0.0"
            }
        ],
        "mobile": false,
        "model": "",
        "platform": "Linux",
        "platformVersion": "6.8.0"
    },
    "pdf": "1",
    "qt": "0",
    "realp": "0",
    "wma": "0",
    "fla": "0",
    "java": "0",
    "ag": "0",
    "cookie": "1",
    "res": "1920x1080"
}

Output in xAPI format

To convert the given Matomo log example into an xAPI DASES statement, we will map the most relevant information from the log to the appropriate xAPI fields. This involves identifying the actor (the user), the verb (the action taken), and the object (the website accessed), as well as including relevant context where applicable. For example, the matomo statement above breaks down into an xAPI DASES statement:

"output_trace": {
		"timestamp": "2024-12-11T14:36:39+00:00",
		"actor": {
			"account": {
				"name": "7&k=",
				"homePage": "http://localhost:3000"
			},
			"objectType": "Agent"
		},
		"object": {
			"id": "http://localhost:3000/test.html",
			"definition": {
				"name": {
					"en-US": "Test Matomo - Page test"
				},
				"type": "https://w3id.org/xapi/acrossx/activities/webpage"
			}
		},
		"result": {
			"duration": "PT4S",
		},
		"context": {
			"language": "fr-fr",
			"extensions": {
				"http://id.tincanapi.com/extension/browser-info": {
					"name": "CH"
				},
				"http://id.tincanapi.com/extension/geojson": {
					"type": "Feature",
					"geometry": {
						"type": "Point"
					},
					"properties": {
						"countryCode": "fr"
					}
				},
				"http://id.tincanapi.com/extension/ip-address": "172.19.0.1",
				"http://id.tincanapi.com/extension/irl": "http://localhost:3000/test.html"
			},
			"contextActivities": {
				"category": [
					{
						"id": "https://w3id.org/xapi/lms",
						"definition": {
							"type": "http://adlnet.gov/expapi/activities/profile"
						}
					}
				]
			}
		},
		"verb": {
			"id": "https://w3id.org/xapi/netc/verbs/accessed",
			"display": {
				"en-US": "Accessed a page"
			}
		},
		"version": "1.0.0"
	},
	"recommendations": [
		{
			"rule": "presence",
			"path": "$.object.definition.extensions['https://w3id.org/xapi/acrossx/extensions/type']",
			"expected": "included",
			"actual": "missing"
		},
		{
			"rule": "any",
			"path": "$.object.definition.extensions['https://w3id.org/xapi/acrossx/extensions/type']",
			"expected": [
				"course",
				"course_list",
				"user_space"
			],
			"actual": []
		}
	],
	"meta": {
		"input_format": "matomo",
		"output_format": "xapi",
		"profile": "lms.accessed-page"
	}

Architecture

classDiagram
   WALRUC <|-- Matomo
   WALRUC <|-- LRC
   LRC <|-- WALRUC
   LRS <|-- WALRUC
   LRS <|-- LRS_PDC
   LRS_PDC <|-- LRS

   LRC: convert()
   WALRUC: convert()
   WALRUC: send_lrs()
   Matomo: track_statement()
   Matomo: send_walruc()
   LRS: send_database()
   
   class LRS_PDC{
     identity()
     catalog()
     contract()
     consent()
   }

PDC : Prometheus-X Dataspace Connector

Dynamic Behaviour WALRUC building blocks communicate with other building blocks, in a precise order.

sequenceDiagram
   actor User as User
   User->>Matomo: Visit a web page (where matomo and WALRUC are)
   Matomo->>WALRUC: Send the matomo trace
   WALRUC->>LRC: Send the matomo trace
   LRC->>LRC: Convert the matomo trace into xAPI DASES format (accessed page)
   LRC->>WALRUC: Send the xAPI DASES trace
   WALRUC->>LRS: Provide the xAPI DASES trace

PDC : Prometheus-X Dataspace Connector

Configuration and deployment settings

Installation

The organization must :

Settings

The organization must :

Error Scenarios Defined

The idea of the risk table is to define the probable causes of failure in order to estimate the probability of encountering this failure, to evaluate its secondary effects and therefore to plan preventive or corrective actions.

We will assign 3 scores on a scale of 1 to 10 to potential failures:

Criticality is calculated as follows:

criticality = detection x occurrence x severity

If criticality is greater than 10, then preventive action must be taken. If not, no action is required.

ID Function involved Description of risk Effect of failure Cause of failure Evaluation - Detection Evaluation - Occurrence Evaluation - Severity Evaluation - Criticality Preventive actions
Error-Scenario_1 export web analytics data from Matomo to LRS Data may be lost when sending to LRS The organization doesn't receive the complete statements in its LRS Incorrect connection between Walruc and LRS 2 2 9 36 Exponential dispatch call
Error-Scenario_2 export web analytics data from Matomo to LRS Data may be lost when sending to LRC Impossible to convert API temporarily unavailable 2 2 9 36 Exponential dispatch call
Error-Scenario_3 export web analytics data from Matomo to LRS The same data can be exported several times Export several times Duplicate dat 1 2 4 8
Error-Scenario_4 export web analytics data from Matomo to LRS The organization may decide to change its LRS Reconnecting Matomo-plugin and the new LRS Change of LRS/LMS 1 2 1 2
Error-Scenario_5 Convert Matomo trace into xAPI DASES format The Matomo trace settings do not match the requested settings. Impossible to convert Not same settings 2 3 7 42 Set up a parameter sheet

Third Party Components & Licenses

External components and licenses :

OpenAPI Specification

The WALRUC code has no API. In fact, data exchange is generated by the LRS, when an individual submits an edit or review. This data is stored in a third-party LRS, which itself has an API. In our case, we use the LRS learning locker. API calls are generic to all LRSs.

The OpenAPI specification for an LRS is available here.

Codebase : Mockup version

To get a functional understanding of this mockup and see some sample traces, go here.

To have a write access to the traces make a request on this mockup document. To have a read access to the traces make a request on this mockup document.

As explained above, the WALRUC code does not include any APIs, so there is no such thing as a dummy code. The only codebase used is that of the LRS with the /PUT /POST and /GET endpoints.

PUT

description: Store a single statement as a single member of a set.

POST

description: "Store a set of statements (or a single statement as a single member of a set).

GET

description: Read a single xAPI Statement or multiple xAPI Statements.

Answer API of Learning Locker

First version

A first version of WALRUC has been deployed. Its purpose is to validate the functionalities mentioned in this design document. The “dataspace integration” part will be fine-tuned later”. Nevertheless, the principle is the same for this extension.

The Walruc plugin is triggered when a visit is made to the site and works as follows:

Insofar as WALRUC makes calls to external APIs (the LRC and LRS), services may be temporarily unavailable for various reasons. For this reason, an exponential backoff system has been implemented, i.e. if an HTTP call fails, it is retried, if necessary several times, with an increasingly long delay between each attempt and a maximum number of attempts.

WALRUC Plugin Installation Guide

Prerequisites

Before installing the WALRUC plugin, ensure you have:

Installation

  1. Download the latest release
  2. Extract the archive in a Walruc folder inside Matomo's plugins folder
  3. Activate the plugin in the plug-in manager of your administration interface

Configuration

  1. Log in to Matomo administration interface
  2. Navigate to Administration > Plugin Settings
  3. Find "WALRUC" in the list
  4. Configure the following settings:
    • LRS Endpoint URL
    • LRS basic auth
    • LRC link if hosted elsewhere than Inokufu

Test specification

The Web Analytics Learning Record Universal Connector tests ensure that :

Test

The Walruc testing strategy will focus on ensuring the accuracy, reliability and performance of its functionality. We will use a combination of unit testing, integration testing and user interface testing. The test environment will reproduce conditions similar to those in production in order to accurately validate BB behavior. Acceptance criteria will be defined on the basis of user stories, functional requirements and performance criteria.

Methodology

We will run manual and units tests.

Using the personas, user stories, userflow and dataflow from the Wiki LOM use case, we established several test scenarios. Before putting V0 into the hands of users, we need to make sure that WALRUC works in a number of applications. We will need to test it at least on the "becomino" application linked to the LRS Learning Locker. The tests will be conclusive if we see the traces appear on Learning Locker.

Validate requirements and potential risks

Tests to validate requirements and potential risks.

Verified by scenario Description Test Status
BB-SC-WALRUC-01 WALRUC converts the data collected by matomo. If the data is anonymized then WALRUC processes the anonymized data. The same applies if the data is pseudonomized or clear Check whether the state of the input trace is the same as that of the output trace (anonymized, pseudonymized, clear) Validated
BB-SC-WALRUC-02 The organization's LRS must be connected to the dataspace Try data exchange in dataspace without a PDC Not yet tested
BB-SC-WALRUC-03 WALRUC must send traces to LRS in less than 30 seconds (after receiving the converted trace) Average send time < 30 seconds Validated
BB-SC-WALRUC-04 BB LRC (ARIANE project) must convert traces from WALRUC Send a dataset for conversion Validated : no error message
BB-SC-WALRUC-05 WALRUC must send traces to LRC in less than 30 seconds (after receiving a matomo trace in WALRUC Average send time < 30 seconds Validated
BB-SC-WALRUC-06 LRC must send traces to WALRUC in less than 30 seconds (after conversion) Average send time < 30 seconds Validated
BB-SC-WALRUC-07 Matomo must send traces to WALRUC in less than 30 seconds Average send time < 30 seconds Validated
Error-Scenario_1 Data may be lost when sending to LRS Check the declarations visible in the associated LRS Validated
Error-Scenario_2 Data may be lost when sending to LRC Check the declarations visible in the associated LRS Validated
Error-Scenario_3 The LRS doesn't have enough storage space for all statements Check the storage of Matomo and associated LRS Not yet tested
Error-Scenario_5 The Matomo trace settings do not match the requested settings Check the declarations visible in the associated LRS, for a non-matching trace Validated : error message

Manual test

Several manual tests of the same type (visiting several pages of a website) are carried out. Example:

Persona 1 : kylian (Learner) Persona 2 : mmedupont (LRS admin)

Scenario : On October 17, 2024, Kylian visits page https://becomino.com/home at 2:30pm for 4 seconds, then page https://becomino.com/board/valorisation-batiment-1701336809956x653082361463832600 for 8 seconds, then page https://becomino.com/category/economie-circulaire for 36 seconds. The WALRUC extension is connected to mmedupont's LRS.

Validation : the scenario is validated if it appears in mmedupont's LRS under 3 traces :

Statements

{
 "actor": {
    "account": {
       "name": "kylian",
       "homePage": "https://www.becomino.com/"
    }
 },
 "verb": {
    "id": "https://w3id.org/xapi/netc/verbs/accessed"
 },
 "object": {
    "objectType": "Activity",
    "id": "https://becomino.com/home",
    "definition": {
       "type": "https://w3id.org/xapi/acrossx/activities/webpage",
       "name": {
          "en": "Home"
       },
       "extensions": {
          "https://w3id.org/xapi/acrossx/extensions/type": "course"
       }
    }
 },
 "context": {
    "contextActivities": {
       "category": [
          {
             "id": "https://w3id.org/xapi/lms",
             "definition": {
                "type": "http://adlnet.gov/expapi/activities/course_list"
             }
          }
       ]
    }
       "extensions": {
          "http://id.tincanapi.com/extension/duration": "4s",
          "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
       }
 }
 "timestamp": "2024-10-17T14:30:0.887Z"
}

{
   "actor": {
      "account": {
         "name": "kylian",
         "homePage": "https://www.becomino.com/"
      }
   },
   "verb": {
      "id": "https://w3id.org/xapi/netc/verbs/accessed"
   },
   "object": {
      "objectType": "Activity",
      "id": "https://becomino.com/board/valorisation-batiment-1701336809956x653082361463832600",
      "definition": {
         "type": "https://w3id.org/xapi/acrossx/activities/webpage",
         "name": {
            "en": "Valorisation batiment"
         },
         "extensions": {
            "https://w3id.org/xapi/acrossx/extensions/type": "course"
         }
      }
   },
   "context": {
      "contextActivities": {
         "category": [
            {
               "id": "https://w3id.org/xapi/lms",
               "definition": {
                  "type": "http://adlnet.gov/expapi/activities/course"
               }
            }
         ]
      }
         "extensions": {
            "http://id.tincanapi.com/extension/duration": "8s",
            "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
         }
   }
   "timestamp": "2024-10-17T14:30:04.887Z"
}

{
   "actor": {
      "account": {
         "name": "kylian",
         "homePage": "https://www.becomino.com/"
      }
   },
   "verb": {
      "id": "https://w3id.org/xapi/netc/verbs/accessed"
   },
   "object": {
      "objectType": "Activity",
      "id": "https://becomino.com/category/economie-circulaire",
      "definition": {
         "type": "https://w3id.org/xapi/acrossx/activities/webpage",
         "name": {
            "en": "Economie Circulaire"
         },
         "extensions": {
            "https://w3id.org/xapi/acrossx/extensions/type": "course"
         }
      }
   },
   "context": {
      "contextActivities": {
         "category": [
            {
               "id": "https://w3id.org/xapi/lms",
               "definition": {
                  "type": "http://adlnet.gov/expapi/activities/course"
               }
            }
         ]
      }
         "extensions": {
            "http://id.tincanapi.com/extension/duration": "36s",
            "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
         }
   }
   "timestamp": "2024-10-17T14:30:12.887Z"
}

Unit tests

Thanks to our unit tests, we were able to prove these next elements.

For the “Visit information formatting recovery” section:

For the parts concerning HTTP calls to the LRC or LRS:

Tests are carried out on the entire process:

Partners & roles

Inokufu (BB leader):

Usage in the dataspace

The WALRUC will be used as a potential source of data for a LRS and thus be part of the service chains:

Enter image alt description PDC: Prometheus-X Dataspace Connector