Prometheus-X Components & Services

Web Analytics Learning Records Universal Connector (WALRUC) BB – Design Document

Web Analytics Learning Records Universal Connector allows for the integration of web analytics data with a Learning Record Store (LRS) using the xAPI (Experience API) DASES standard. It enables the conversion of analytics data, such as the data collected by Matomo, into an xAPI format that can be stored and tracked in a LRS.

Technical usage scenarios & Features

Key functionalities:

Value-added:

Features/main functionalities

Features:


flowchart LR

A[Collect Matomo web browsing analytics]--> B[Convert Matomo web browsing analytics into xAPI DASES format]--> C[Transfer data to an LRS]

Technical usage scenarios

A training organization can use WALRUC service to track and analyze the engagement and performance of their learners on their website. Here's an example of how this could work:

  1. The training organization integrates their website with the web analytics tool Matomo, which tracks user interactions and engagement with the website.

  2. The organization also sets up a Learning Record Store (LRS) like Learning Locker.

  3. The WALRUC is implemented to connect the web analytics tool with the LRS. This allows the organization to convert the web browsing analytics data into an xAPI DASES format that can be stored and tracked in the LRS.

  4. As learners interact with the website, data is collected and tracked in the LRS. This includes information like pages viewed, time spent on the site, ...

  5. The organization can use this data to track the performance of anonymized learners (or individual learners if they have their consent) and identify areas where they may need specific or additional support or resources.

  6. The organization can also use the data to track the overall engagement and effectiveness of their website, and make improvements as needed.

Requirements

Requirement ID Short description BB input format BB output format Any other constraints Verified by scenario Requirement type
BB-REQ_ID__1 WALRUC must request building block consent via the Prometheus-X Dataspace Connector API call API response
BB-REQ_ID__1.1 Individuals must consent to the use and export of their web analytics data as learning records in the WALRUC plugin of the organization API call API response If the answer is no, the data cannot be used, nor transferred into or from the WALRUC. If the answer is yer, the data can be used, and transferred into or from the WALRUC. BB-SC-WALRUC-01 DEP
BB-REQ_ID__1.2 Consent must be asked and verified in less than 30s API call API response BB-SC-WALRUC-02 PERF
BB-REQ_ID__2 WALRUC must request contracts from the building block consent via the Prometheus-X Dataspace Connector API call API response
BB-REQ_ID__2.1 The WALRUC must check with the contract manager through the Dataspace connector if a contract for the corresponding organization exists API call API response If the answer is no, the data cannot be accessed, nor transferred into or from the PLRS. If the answer is yer, the data can be accessed, and transferred into or from the PLRS. BB-SC-WALRUC-03 DEP
BB-REQ_ID__2.2 Contract must be asked and verified in less than 30s API call API response BB-SC-WALRUC-04 PERF
BB-REQ_ID__3 WALRUC must connect with BB Consent/contracts negotiating agent (EDGE-Skill)
BB-REQ_ID__3.1 BB must send the individual's consent profile when the WALRUC asks to adjust what and when they are tracked: all-time connection, only on weekends, certain keywords, etc. API call consent profile Request consent 1 time, then update if the profile is modified in the corresponding building bloc. Could be asynchronous BB-SC-WALRUC-05 DEP
BB-REQ_ID__3.2 BB must update the individual's consent profile to WALRUC when there are changes consent profile / update if the profile is modified in the corresponding building bloc. Could be asynchronous BB-SC-WALRUC-06 DEP

Integrations

Direct Integrations with Other BBs

Interact with consent/contract

How?

Why?

Integrations via Connector

Connection with connector:

Why?

Connection with contract:

Why?

Connection with consent:

Why?

What?

Connection with identity:

Why?

What?

Relevant Standards

Data Format Standards

Data format:

Mapping to Data Space Reference Architecture Models


block-beta

columns 4

Matomo:1 WALRUC:1 LRS:1 Organization_PDC:1

classDef colorA fill:#D22FF7,color:#fff
classDef colorEx fill:#01D1D1,color:#fff
classDef colorED fill:#6E7176,color:#fff
class Matomo colorEx
class LRS colorEx
class WALRUC colorED
class Organization_PDC colorA

PDC : Prometheus-X Dataspace Connector

Input / Output Data

Input in matomo format

WALRUC uses matomo data format as input data format.

Here is an example of Matomo logs for a user who visited 3 pages on www.inokufu.com

{
    "idSite": "1",
    "idVisit": "68536",
    "visitIp": "12.123.0.0",
    "visitorId": "abcdefg",
    "fingerprint": "hijklmnop",
    "actionDetails": [
{
        "type": "action",
        "url": "https://www.inokufu.com/",
        "pageTitle": "Inokufu | Better data for better learning",
        "pageIdAction": "2",
        "idpageview": "CWrLQ5",
        "serverTimePretty": "4 juil. 2024 18:43:05",
        "pageId": "182005",
        "timeSpent": 8,
        "timeSpentPretty": "8s",
        "pageviewPosition": "1",
        "title": "Inokufu | Better data for better learning",
        "subtitle": "https://www.inokufu.com/",
        "icon": "",
        "iconSVG": "plugins/Morpheus/images/action.svg",
        "timestamp": 1720118585
      },
      {
        "type": "action",
        "url": "https://www.inokufu.com/learning-platform/",
        "pageTitle": "Learning platform | Inokufu",
        "pageIdAction": "4715",
        "idpageview": "iTnAa1",
        "serverTimePretty": "4 juil. 2024 18:43:13",
        "pageId": "182006",
        "timeSpent": 24,
        "timeSpentPretty": "24s",
        "pageviewPosition": "2",
        "title": "Learning platform | Inokufu",
        "subtitle": "https://www.inokufu.com/learning-platform/",
        "icon": "",
        "iconSVG": "plugins/Morpheus/images/action.svg",
        "timestamp": 1720118593
      },
      {
        "type": "action",
        "url": "https://www.inokufu.com/search-engine/",
        "pageTitle": "Search Engine | Inokufu",
        "pageIdAction": "4710",
        "idpageview": "2vf6wq",
        "serverTimePretty": "4 juil. 2024 18:43:37",
        "pageId": "182007",
        "timeSpent": 0,
        "timeSpentPretty": "0s",
        "pageviewPosition": "3",
        "title": "Search Engine | Inokufu",
        "subtitle": "https://www.inokufu.com/search-engine/",
        "icon": "",
        "iconSVG": "plugins/Morpheus/images/action.svg",
        "timestamp": 1720118617
      }
    ],
    "goalConversions": 0,
    "siteCurrency": "EUR",
    "siteCurrencySymbol": "€",
    "serverDate": "2024-07-04",
    "visitServerHour": "16",
    "lastActionTimestamp": 1720111417,
    "lastActionDateTime": "2024-07-04 16:43:37",
    "siteName": "www.inokufu.com",
    "serverTimestamp": 1720111417,
    "firstActionTimestamp": 1720111385,
    "serverTimePretty": "18:43:37",
    "serverDatePretty": "Jeudi 4 juillet 2024",
    "serverDatePrettyFirstAction": "Jeudi 4 juillet 2024",
    "serverTimePrettyFirstAction": "18:43:05",
    "userId": null,
    "visitorType": "new",
    "visitorTypeIcon": null,
    "visitConverted": "0",
    "visitConvertedIcon": null,
    "visitCount": "1",
    "visitEcommerceStatus": "none",
    "visitEcommerceStatusIcon": null,
    "daysSinceFirstVisit": 0,
    "secondsSinceFirstVisit": "0",
    "daysSinceLastEcommerceOrder": 0,
    "secondsSinceLastEcommerceOrder": null,
    "visitDuration": "33",
    "visitDurationPretty": "33s",
    "searches": "0",
    "actions": "3",
    "interactions": "3",
    "referrerType": "search",
    "referrerTypeName": "Moteurs de recherche",
    "referrerName": "Google",
    "referrerKeyword": "Mot clef indéfini",
    "referrerKeywordPosition": null,
    "referrerUrl": "https://www.google.com/",
    "referrerSearchEngineUrl": "http://google.com",
    "referrerSearchEngineIcon": "plugins/Morpheus/icons/dist/searchEngines/google.com.png",
    "referrerSocialNetworkUrl": null,
    "referrerSocialNetworkIcon": null,
    "languageCode": "fr-fr",
    "language": "Code langue fr-fr",
    "deviceType": "Bureau",
    "deviceTypeIcon": "plugins/Morpheus/icons/dist/devices/desktop.png",
    "deviceBrand": "Apple",
    "deviceModel": "Générique Bureau",
    "operatingSystem": "Mac 10.15",
    "operatingSystemName": "Mac",
    "operatingSystemIcon": "plugins/Morpheus/icons/dist/os/MAC.png",
    "operatingSystemCode": "MAC",
    "operatingSystemVersion": "10.15",
    "browserFamily": "Blink",
    "browserFamilyDescription": "Blink (Chrome, Opera)",
    "browser": "Chrome 126.0",
    "browserName": "Chrome",
    "browserIcon": "plugins/Morpheus/icons/dist/browsers/CH.png",
    "browserCode": "CH",
    "browserVersion": "126.0",
    "events": "0",
    "continent": "Europe",
    "continentCode": "eur",
    "country": "France",
    "countryCode": "fr",
    "countryFlag": "plugins/Morpheus/icons/dist/flags/fr.png",
    "region": "Île-de-France",
    "regionCode": "IDF",
    "city": "Bezons",
    "location": "Bezons, Île-de-France, France",
    "latitude": "48.924000",
    "longitude": "2.213000",
    "visitLocalTime": "18:43:05",
    "visitLocalHour": "18",
    "daysSinceLastVisit": 0,
    "secondsSinceLastVisit": "0",
    "resolution": "1440x900",
    "plugins": "cookie, pdf",
    "pluginsIcons": [
      {
        "pluginIcon": "plugins/Morpheus/icons/dist/plugins/cookie.png",
        "pluginName": "cookie"
      },
      {
        "pluginIcon": "plugins/Morpheus/icons/dist/plugins/pdf.png",
        "pluginName": "pdf"
      }
    ]
  }

Output in xAPI format

To convert the given Matomo log example into an xAPI DASES statement, we will map the most relevant information from the log to the appropriate xAPI fields. This involves identifying the actor (the user), the verb (the action taken), and the object (the website ccessed), as well as including relevant context where applicable. For example, the matomo statement above breaks down into 3 xAPI DASES statements:


{
   "actor": {
      "account": {
         "name": "abcdefg",
         "homePage": "https://www.inokufu.com/"
      }
   },
   "verb": {
      "id": "https://w3id.org/xapi/netc/verbs/accessed"
   },
   "object": {
      "objectType": "Activity",
      "id": "https://www.inokufu.com/",
      "definition": {
         "type": "https://w3id.org/xapi/acrossx/activities/webpage",
         "name": {
            "en": "Inokufu | Better data for better learning"
         },
         "extensions": {
            "https://w3id.org/xapi/acrossx/extensions/type": "website"
         }
      }
   },
   "context": {
      "contextActivities": {
         "category": [
            {
               "id": "homepage",
               "definition": {
                  "type": "http://adlnet.gov/expapi/activities/profile"
               }
            }
         ]
      }
         "extensions": {
            "http://id.tincanapi.com/extension/duration": "8s",
            "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
         }
   }
   "timestamp": "2024-07-04T18:43:05.887Z"
}


{
   "actor": {
      "account": {
         "name": "abcdefg",
         "homePage": "https://www.inokufu.com/"
      }
   },
   "verb": {
      "id": "https://w3id.org/xapi/netc/verbs/accessed"
   },
   "object": {
      "objectType": "Activity",
      "id": "https://www.inokufu.com/learning-platform/",
      "definition": {
         "type": "https://w3id.org/xapi/acrossx/activities/webpage",
         "name": {
            "en": "Learning platform | Inokufu"
         },
         "extensions": {
            "https://w3id.org/xapi/acrossx/extensions/type": "website"
         }
      }
   },
   "context": {
      "contextActivities": {
         "category": [
            {
               "id": "https://w3id.org/xapi/lms",
               "definition": {
                  "type": "http://adlnet.gov/expapi/activities/profile"
               }
            }
         ]
      }
         "extensions": {
            "http://id.tincanapi.com/extension/duration": "24s",
            "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
         }
   }
   "timestamp": "2024-07-04T18:43:13.887Z"
}


{
   "actor": {
      "account": {
         "name": "abcdefg",
         "homePage": "https://www.inokufu.com/"
      }
   },
   "verb": {
      "id": "https://w3id.org/xapi/netc/verbs/accessed"
   },
   "object": {
      "objectType": "Activity",
      "id": "https://www.inokufu.com/search-engine/",
      "definition": {
         "type": "https://w3id.org/xapi/acrossx/activities/webpage",
         "name": {
            "en": "Search Engin | Inokufu"
         },
         "extensions": {
            "https://w3id.org/xapi/acrossx/extensions/type": "website"
         }
      }
   },
   "context": {
      "contextActivities": {
         "category": [
            {
               "id": "https://w3id.org/xapi/lms",
               "definition": {
                  "type": "http://adlnet.gov/expapi/activities/profile"
               }
            }
         ]
      }
         "extensions": {
            "http://id.tincanapi.com/extension/duration": "0s",
            "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
         }
   }
   "timestamp": "2024-07-04T18:43:37.887Z"
}

Architecture

classDiagram
   WALRUC <|-- Matomo
   LRS <|-- WALRUC
   LRS <|-- LRS_PDC
   LRS_PDC <|-- LRS
   LRS_PDC <|-- CC_PDC
   CC_PDC <|-- LRS_PDC
   CC_PDC <|-- Consent_Contracts
   Consent_Contracts <|-- CC_PDC
   WALRUC: convert()
   WALRUC: send_lrs()
   Matomo: track_statement()
   Matomo: send_walruc()
   LRS: send_database()
   class LRS_PDC{
     identity()
     catalog()
     contract()
     consent()
   }
   class CC_PDC{
     identity()
     catalog()
     contract()
     consent()
   }
   class Consent_Contracts{
     bool week[7]
     int begin[7]
     int end[7]
     string keywords[]
     add_keyword(string)
     change_track()
   }

PDC : Prometheus-X Dataspace Connector

Dynamic Behaviour WALRUC building blocks communicate with other building blocks, in a precise order.

sequenceDiagram
   actor User as User
   User->>Matomo: Made a web browsing trace
   Matomo->>WALRUC: Send the trace
   WALRUC->>WALRUC: Convert the trace into xAPI DASES format
   WALRUC->>LRS: Provide trace

PDC : Prometheus-X Dataspace Connector

Configuration and deployment settings

Installation and deployment :

Error Scenarios Defined

The idea of the risk table is to define the probable causes of failure in order to estimate the probability of encountering this failure, to evaluate its secondary effects and therefore to plan preventive or corrective actions.

We will assign 3 scores on a scale of 1 to 10 to potential failures:

Criticality is calculated as follows:

criticality = detection x occurrence x severity

If criticality is greater than 10, then preventive action must be taken. If not, no action is required.

ID Function involved Description of risk Effect of failure Cause of failure Evaluation - Detection Evaluation - Occurrence Evaluation - Severity Evaluation - Criticality Preventive actions
1 export web analytics data from Matomo to LRS Data may be lost during migration The organization doesn't receive the complete statements in its LRS Incorrect connection between Matomo-Walruc- LRS 2 2 9 36 Set up recurring connection tests
2 export web analytics data from Matomo to LRS The same data can be exported several times Export several times Duplicate dat 1 2 4 36 Program ignore if already in LRS
3 export web analytics data from Matomo to LRS The LRS doesn't have enough storage space for all statements No more statement export Too little storage 1 3 9 24
4 export web analytics data from Matomo to LRS The system may require downtime for large exports Disrupting normal operations Low-performance servers 1 3 4 12 Test the service's scalability
5 export web analytics data from Matomo to LRS The organization may decide to change its LRS Reconnecting Matomo-plugin and the new LRS Change of LRS/LMS 1 2 1 2
6 Convert Matomo trace into xAPI DASES format The Matomo trace settings do not match the requested settings. Impossible to convert Not same settings 2 3 7 42 Set up a parameter sheet

Third Party Components & Licenses

External components and licenses :

OpenAPI Specification

In the future: link your OpenAPI spec here.

openapi: 3.0.0 \
info: \
     version: 0.0.1 \
     title: Web Analytics Learning Records Universal Connector \
   description: Web Analytics Learning Records Universal Connector allows for the integration of web analytics data with a Learning Record Store (LRS) using the xAPI (Experience API) standard. It enables the conversion of web analytics data, such as the data collected by Matomo, into an xAPI format that can be stored in a LRS. \
paths: \
     /list: \
          get: \
               description: Returns a list of stuff \
                    responses: \
                         '200': \
                              description: Successful response

Codebase : Mockup version

To get a functional understanding of this mockup and see some sample traces, go here : https://github.com/Prometheus-X-association/walruc/blob/main/docs/Walruc%20-%20Mock%20up%20.pdf

To have a write access to the traces make a request on this mockup document : https://docs.google.com/document/d/1DwXxVtfRkWuW_jwCQBVjJ2eYNImKd3HI8f0f4iQYyi8/edit To have a read access to the traces make a request on this mockup document: https://docs.google.com/document/d/12ZXHjXoZ5jNAhmsdc0IPs6e-oJAG6zCzdDe5awPe1_I/edit

The WALRUC code does not have an API. In fact, data exchange is generated by matomo, when an individual visits the site. This data is stored in a third-party LRS, which itself has an API. In our case, we are using the LRS learning locker. API calls are generic to all LRS.

PUT

description: Store a single statement as a single member of a set.

POST

description: "Store a set of statements (or a single statement as a single member of a set).

GET

description: Read a single xAPI Statement or multiple xAPI Statements.

Answer API of Learning Locker

Test specification

The Web Analytics Learning Record Universal Connector tests ensure that :

Test plan

The Walruc testing strategy will focus on ensuring the accuracy, reliability and performance of its functionality. We will use a combination of unit testing, integration testing and user interface testing. The test environment will reproduce conditions similar to those in production in order to accurately validate BB behavior. Acceptance criteria will be defined on the basis of user stories, functional requirements and performance criteria.

Methodology We will run manual and automatic tests.

Using the personas, user stories, userflow and dataflow from the Wiki LOM use case, we established several test scenarios. Before putting V0 into the hands of users, we need to make sure that WALRUC works in a number of applications. We will need to test it at least on the "becomino" application linked to the LRS Learning Locker. The tests will be conclusive if we see the traces appear on Learning Locker.

Manual test

For your information, the tests will be extended in the future.

Persona 1 : kylian (Learner)

Persona 2 : mmedupont (LRS admin)

Scenario 1 : On October 17, 2024, Kylian visits page https://becomino.com/home at 2:30pm for 4 seconds, then page https://becomino.com/board/valorisation-batiment-1701336809956x653082361463832600 for 8 seconds, then page https://becomino.com/category/economie-circulaire for 36 seconds. The WALRUC extension is connected to mmedupont's LRS.

Validation : the scenario is validated if it appears in mmedupont's LRS under 3 traces :

Statements

{
 "actor": {
    "account": {
       "name": "kylian",
       "homePage": "https://www.becomino.com/"
    }
 },
 "verb": {
    "id": "https://w3id.org/xapi/netc/verbs/accessed"
 },
 "object": {
    "objectType": "Activity",
    "id": "https://becomino.com/home",
    "definition": {
       "type": "https://w3id.org/xapi/acrossx/activities/webpage",
       "name": {
          "en": "Home"
       },
       "extensions": {
          "https://w3id.org/xapi/acrossx/extensions/type": "course"
       }
    }
 },
 "context": {
    "contextActivities": {
       "category": [
          {
             "id": "https://w3id.org/xapi/lms",
             "definition": {
                "type": "http://adlnet.gov/expapi/activities/course_list"
             }
          }
       ]
    }
       "extensions": {
          "http://id.tincanapi.com/extension/duration": "4s",
          "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
       }
 }
 "timestamp": "2024-10-17T14:30:0.887Z"
}


{
   "actor": {
      "account": {
         "name": "kylian",
         "homePage": "https://www.becomino.com/"
      }
   },
   "verb": {
      "id": "https://w3id.org/xapi/netc/verbs/accessed"
   },
   "object": {
      "objectType": "Activity",
      "id": "https://becomino.com/board/valorisation-batiment-1701336809956x653082361463832600",
      "definition": {
         "type": "https://w3id.org/xapi/acrossx/activities/webpage",
         "name": {
            "en": "Valorisation batiment"
         },
         "extensions": {
            "https://w3id.org/xapi/acrossx/extensions/type": "course"
         }
      }
   },
   "context": {
      "contextActivities": {
         "category": [
            {
               "id": "https://w3id.org/xapi/lms",
               "definition": {
                  "type": "http://adlnet.gov/expapi/activities/course"
               }
            }
         ]
      }
         "extensions": {
            "http://id.tincanapi.com/extension/duration": "8s",
            "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
         }
   }
   "timestamp": "2024-10-17T14:30:04.887Z"
}


{
   "actor": {
      "account": {
         "name": "kylian",
         "homePage": "https://www.becomino.com/"
      }
   },
   "verb": {
      "id": "https://w3id.org/xapi/netc/verbs/accessed"
   },
   "object": {
      "objectType": "Activity",
      "id": "https://becomino.com/category/economie-circulaire",
      "definition": {
         "type": "https://w3id.org/xapi/acrossx/activities/webpage",
         "name": {
            "en": "Economie Circulaire"
         },
         "extensions": {
            "https://w3id.org/xapi/acrossx/extensions/type": "course"
         }
      }
   },
   "context": {
      "contextActivities": {
         "category": [
            {
               "id": "https://w3id.org/xapi/lms",
               "definition": {
                  "type": "http://adlnet.gov/expapi/activities/course"
               }
            }
         ]
      }
         "extensions": {
            "http://id.tincanapi.com/extension/duration": "36s",
            "http://id.tincanapi.com/extension/browser-info": "Chrome 126.0"
         }
   }
   "timestamp": "2024-10-17T14:30:12.887Z"
}

UI test (where relevant)

The only UI/UX that will be visible is the parameter side of the Matomo plugin. matomo parameters (2)

Partners & roles

Inokufu (BB leader):

Usage in the dataspace

The WALRUC will be used as a potential source of data for a LRS and thus be part of the service chains:

Enter image alt description PDC: Prometheus-X Dataspace Connector