How to Build a Powerful E-Learning Platform Using Scala and Redis

Never before has online learning been so accessible. Whether you want to discover more about cryptocurrency, sharpen your programming skills or even just learn a new language, the digital age has gifted everyone access to a phenomenal amount of content.

However, over time e-learning has been viewed as just another digital commodity, where users expect all online content to be instantaneous. Speed remains crucial to performance, where any lags or delays in page loading time kills the user’s experience.

And so these high expectations require any competent e-learning platform to be powered by a database that’s capable of handling, processing and transmitting data with hyper-efficiency…which is exactly why this Launchpad App used Redis.

From start to finish, this application was built to connect, educate and empower learners by connecting them with the most relevant courses based on their interests.

Let’s investigate how this was achieved. But before we dive in, you may also want to check out all of the other amazing applications that we have on the Launchpad.

What will you build?
What will you need?
Architecture
Getting started

1. What will you build?

You’ll build a powerful e-learning platform that will connect students and teachers with one another along with a diverse library of online courses. With speed being the linchpin to performance, you’ll deploy a number of different Redis components to achieve this objective.

Below we’ll reveal what components are required to make this application come to fruition along with the functionality of each item.

2. What will you need?

Scala/Play Framework/Akka Streams: used as an open-sourced web application that makes building apps with Java and Scala easy.
React: used as a Javascript library to build user interfaces easily.
RedisGraph: used as a powerful graph database that translates Cypher queries to matrix operations executed over a GraphBLAS engine.
Redis Streams: manages data consumption
RedisBloom: provides Redis with support for additional probabilistic data structures.
RedisGears: used as the engine for data processing in Redis
RediSearch: provides a powerful search engine for Redis.
RedisJSON: implements ECMA-404 The JSON Data Interchange Standard as a native data type
RedisTimeSeries: provides time series data
Keycloak: used as an open-sourced identity and access management solution for modern applications

3. Architecture

The data model is expressed through nodes and relations using RedisGraph. The model is very simple since it involves the Student, Course and Topic entities expressing the different kinds of relations between each other.

X-Mentor follows an Event Driven Architecture approach in which the following Domain Events are considered:

student-enrolled
student-interested
student-interest-lost
course-created
course-rated
course-recommended
Student-progress-registered

4. Getting started

Prerequisites

Docker Engine and Docker Compose

Step 1: Clone the repository

https://github.com/redis-developer/x-mentor

Step 2: Start the docker

$ docker-compose ps
Name                         Command               State            Ports              
---------------------------------------------------------------------
x-mentor_x-gears_1           python3 init.py --url redi ...   Up                                   
x-mentor_x-keycloak_1        /opt/jboss/tools/docker-en ...   Up       0.0.0.0:8880->8080/tcp, 8443/tcp
x-mentor_x-mentor-client_1   /docker-entrypoint.sh ngin ...   Up       0.0.0.0:3000->80/tcp            
x-mentor_x-mentor-core_1     /opt/docker/conf/wait-for- ...   Up       0.0.0.0:9000->9000/tcp          
x-mentor_x-redis_1           redis-server --loadmodule  ...   Up       0.0.0.0:6379->6379/tcp          
[node1] (local) root@192.168.0.8 ~/x-mentor
$

Step 3: Accessing the application

Wait until Keycloak and x-mentor-core are ready, then go to http://localhost:3000.

You can access Keycloak via 8880 port as shown below:

Use admin/admin to login into keycloak.

Step 4: Logging in

This step starts the authentication process against Keycloak by:

Verifying if the user’s username already exists in users bloom filter
Providing an auth token

Use the following code to check and see whether the username already exists in users bloom filter:

BF.EXISTS users '${student.username}'

Step 5: Signing up

Signing up involves 4 steps:

Registering a user against Keycloak
Adding a user’s username to users bloom filter
Creating a user in RedisGraph
Adding student’s time series key (needed for registering student progress)

To add a username to users bloom filter, insert the following code

BF.ADD users '${student.username}'

To integrate students into the graph, use the below code

GRAPH.QUERY xmentor "CREATE (:Student {username: '${student.username}', email: '${student.email}'})"

To create student progress timeseries key, use the below code

TS.CREATE studentprogress:${username} RETENTION 0 LABELS student ${username}

Step 6: Creating Courses

In this step we’re going to show you how to create courses to go on the e-learning platform. Each course is going to be stored as a JSON in RedisJSON.

Follow the commands below:

Obtain the last course id from the Redis key: course-last-index

GET course-last-index

Increase course id key in 1

INCR course-last-index

Store course as JSON in redisJSON

JSON.SET course:${course.id} . '${course.asJson}'

Add course id to courses bloom filter

BF.ADD courses '${course.id}'

Create course in the graph

GRAPH.QUERY xmentor "CREATE (:Course {name: '${course.title}', id: '${course.id.get}', preview: '${course.preview}'})"

Publish course-created event which sends notifications by Server Sent Event to the frontend

XADD course-created $timestamp title ${course.title} topic ${course.topic}

Step 7: Enrolling courses

Here we’ll uncover how you can enroll a student in a specific course.

Below are the steps for you to follow:

To verify if a student exists in users bloom filter:

BF.EXISTS users ${student.username}

To get course as JSON from redisJSON

JSON.GET course:${course.id}

To highlight a relationship between the student and the course in redisGraph

GRAPH.QUERY xmentor "MATCH (s:Student), (c:Course) WHERE s.username = '${studying.student}' AND c.name = '${studying.course}' CREATE (s)-[:studying]->(c)"

Step 8: Course review

As part of any online resource, users generally are able to provide a review. To make this functionality happen, you need to carry out the following steps:

First, verify if a studying relationship exists between the student and the course.
Next, verify whether a rates relation exists between the student and the course.
Thirdly, create the rate relation in the graph (see diagram below).
Publish event course-rated stream

The following diagram illustrates the interaction between Redis Graph and Redis Streams.

To bring make this functionality happen, follow the below commands:

Filter courses by student

GRAPH.QUERY xmentor "MATCH (student)-[:studying]->(course) where student.username = '$student' RETURN course"

Get courses rated by user

GRAPH.QUERY xmentor "MATCH (student)-[:rates]->(course) where student.username ='$student' RETURN course"

Create rates relation in the graph

GRAPH.QUERY xmentor "MATCH (s:Student), (c:Course) WHERE s.username = '${rating.student}' AND c.name = '${rating.course}' CREATE (s)-[:rates {rating:${rating.stars}}]->(c)"

Publish event to course-rated stream

XADD course-rated $timestamp student $student_username course $course 
	starts $stars

Step 9: Course search

All

The following commands retrieves courses by query from redisJSON with rediSearch

FT.SEARCH courses-idx ${query}*

By ID

BF.EXISTS courses ${course.id}

JSON.GET course:${course.id}

By Student

GRAPH.QUERY xmentor “MATCH (student)-[:studying]->(course) where student.username = ‘$student’ RETURN course”

FT.SEARCH courses-idx ${course.title}

Step 10: Student’s interests

Now we’ll show you how to allow students to filter preferred courses based on their interests. Here’s how to do it:

Get all of the interested relations from RedisGraph.
Distinguish the difference between already existing relations and new ones. This’ll allow you to separate new interests from existing ones.
Create new interested relations into RedisGraph
Remove interested relations that don’t apply anymore
Publish to student-interest-lost and student-interested stream

The following diagram shows the interaction between RedisGraph and Redis Streams.

Below are the commands for you to follow:

Capture all student interests

GRAPH.QUERY xmentor "MATCH (student)-[:interested]->(topic) WHERE 
student.username ='$student' RETURN topic"

Create interest relation

GRAPH.QUERY xmentor "MATCH (s:Student), (t:Topic) WHERE s.username = 
	'${interest.student}' AND t.name = '${interest.topic}' CREATE 
(s)-[:interested]->(t)"

Delete interest relation

GRAPH.QUERY xmentor "MATCH (student)-[interest:interested]->(topic) WHERE student.username='${interest.student}' and topic.name='${interest.topic}' DELETE interest"

Publishing to student-interested stream

XADD student-interest-lost $timestamp student ${student.username} topic $topic

Publishing to student-interest-lost stream

XADD student-interest-lost $timestamp student ${student.username} topic $topic

Step 11: Course recommendation system

Here we’re going to show you how to create a course recommendation system that connects users with courses that are most relevant to their interests. A lot of this comes down to the advanced capabilities of RedisGraph.

Searching for relations between nodes in the graph database is the easiest way to implement the most effective recommendation strategies. Let’s have a look at how to do this.

In order for you to create a special recommendation system that matches users’ personal interests with the most relevant courses

How to carry out the enrolled recommendation strategy

Randomly select a course the student is enrolled in
Get the topic of the course
Look for students enrolled on the same course
Search for courses on the same topic based on the students who are currently enrolled
Recommend those courses

How to carry out the interest recommendation strategy

Randomly select a student’s interest
Look for students who are enrolled to the course of that topic
Search for other courses of the same topic based on the students who are currently enrolled
Provide the recommended courses

How to carry out the discover recommendation strategy

Get all topics
Identify all of the topics that the students are interested in
Gather all of the topics the user is enrolled in
Choose a topic where the user is neither interested in nor enrolled for
Get the courses of that chosen topic and recommend them

How the graph data is accessed

All student’s courses

GRAPH.QUERY xmentor "MATCH (student)-[:studying]->(course) where student.username = '$student' RETURN course"

Get all topics

GRAPH.QUERY xmentor "MATCH (topic:Topic) RETURN topic"

Obtain topic by course

GRAPH.QUERY xmentor "MATCH (topic:Topic)-[:has]->(course:Course)

Identify students who are enrolled in (studying relation) a course

GRAPH.QUERY xmentor "MATCH (student)-[:studying]->(course) WHERE course.name = '$course' RETURN student"

Generate courses by topic

GRAPH.QUERY xmentor "MATCH (topic)-[:has]->(course) WHERE 
	topic.name = '${topic.name}' RETURN course"

Uncover student’s interests

GRAPH.QUERY xmentor "MATCH (student)-[:interested]->(topic) WHERE student.username ='$student' RETURN topic"

Filter courses the student is enrolled in based on topic

GRAPH.QUERY xmentor "MATCH (student)-[:studying]->(course), 
	(topic)-[:has]->(course) where student.username = '${student.username}'  
	and topic.name = '${topic.name}' RETURN course"

Identify topics the user is enrolled in

GRAPH.QUERY xmentor "MATCH (student)-[:studying]->(course), (topic)-[:has]->(course) WHERE student.username = '${student.username}' RETURN topic"

Step 12: Student progress registration

This functionality will allow you to track the amount of time users spend watching courses on the platform. That information will then be used to implement the Leaderboard.

Once x-mentor-core receives the request, it will then publish the Student Progress Registration Domain Event. This will end up as an element inside student-progress-registered stream (which is a Redis Stream) via the following command:

XADD student-progress-registered $timestamp student $student_username duration $duration

All of the data sent to RedisGears will be pushed to the stream and will then sink this data into Redis TimeSeries using the following command:

TS.ADD studentprogress:$student_username $timestamp $duration RETENTION 0 LABELS student $student_username

Step 13: Leaderboard

The Leaderboard functionality enables you to have a board that displays the rankings of the top students that use X-Mentor. Students are ranked based on the amount of time they spend watching content on the platform – the more you watch, the higher you rank.

To accomplish this, you need to separate two functionalities:

Register the student progress
Getting the board data

When the user request for the leaderboard data, first look at Redis for the time series keys

LRANGE student-progress-list 0 -1 // to retrieve all the list elements

For each key, you need to use Redis TimeSeries to get the range of samples in a time window of three months performing sum aggregation. You can use the code below to make this happen:

TS.RANGE $student_key $thee_months_back_timestamp $timestamp AGGREGATION sum 1000

Below are additional requisites that need to be implemented for this to happen.

student_key is the student’s time series key. For example: studentprogress:codi.sipes is the time series key for student codi.sipes.
three_months_back_timestamp is a Unix Timestamp that represents a point in time three months back than timestamp(in order to have a time window of three months).
timestamp the current timestamp (in Unix Timestamp format).
Next perform a sum aggregation of the sample values in those time windows using a Time Bucket of 1000 milliseconds.

Carrying out these commands will provide you with the accumulated screen time of each student. Once you receive these rankings, you can create a ranking system based on those who have the most screen time.

Conclusion: Empowering Learners With Redis

Being this far into the digital age, a simple prerequisite of any application is for it to operate at maximum speed. This is especially true for e-learning platforms where users are meant to be engaged with its course content for long periods of time.

A mere lag will create friction between users and the application, inhibiting its ability to connect teachers with students as well as providing value through its courses. Having Redis as the application’s main database removed this threat and helped to create a fully optimal application that catered to the user’s demands with ease.

To get a more visual insight into how this application was created, then you can watch this YouTube video here. We also have a diverse range of applications for you to check out on the Redis Launchpad that are having an impact on everyday life around the world.

So make sure to check them out!

Who built this application?

Sergio Cano

Sergio is a full-stack engineer and in his own words ‘loves solving problems and learning new stuff.’

Being an enthusiastic learner, it’s not difficult to see where he got the inspiration to build this application.

Make sure to check out his profile here and see what other projects he’s been involved in.