Real-time chat app is an online communication channel that allows you to conduct real-time conversations. More and more developers are tapping into the power of Redis as it is extremely fast & due to its support for variety of rich data structure such as Lists, Sets, Sorted Sets, Hashes etc. Redis comes along with a Pub/Sub messaging feature functionality that allows developers to scale the backend by spawning multiple server instances.
While the traditional chat application tutorial on Redis remains insightful, we encourage you to dive into our AI chatbot tutorial to unlock the potential of modern conversational experiences.
Please note that this code is open source. You can find the link at the end of this tutorial.
In this tutorial, we will see how to develop real time messaging apps with Flask, Socket.IO and Redis. This example uses Redis Pub/sub feature combined with websockets for implementing the real time chat app communication between client and server.
Please note that this code is open source and implements the basic features of a live chat app. You can find the link at the end of this tutorial.
In order to perform this instant messaging app development, you will need the following software:
First of all, we will clone the project that implements basic chat functionality.
git clone https://github.com/redis-developer/basic-redis-chat-app-demo-pythoncd client
yarn installTo run the frontend of the chat app, run the following command:
yarn startYou can now access a chat window in the browser.
Local: http://localhost:3000
On Your Network: http://192.168.1.9:3000cd ..
pip3 install -r requirements.txtTo start the fully chat app, run the following commands:
python3 -m venv venv/
source venv/bin/activate
python3 app.pypython3 app.py
* Restarting with stat
* Debugger is active!
* Debugger PIN: 220-696-610
(8122) wsgi starting up on http://127.0.0.1:5000This instant messaging app server works as a basic REST API which involves keeping the session and handling the user state in the chat room (besides the WebSocket/real-time part). When the server starts, the initialization step occurs. At first, a new Redis connection is established and it's checked whether it's needed to load the demo data.
For simplicity, a key with total_users value is checked: if it does not exist, we fill the Redis database with initial data. EXISTS total_users (checks if the key exists) The demo data initialization is handled in multiple steps:
We create a new user id: INCR total_users. Then we set a user ID lookup key by user name: e.g.
SET username:nick user:1And finally, the rest of the data is written to the hash set:
Example:
HSET user:1 username "nick" password "bcrypt_hashed_password".Additionally, each user is added to the default "General" room. For handling chat rooms for each user, we have a set that holds the chat room ids. Here's an example command of how to add the room:
SADD user:1:rooms "0"Populate private messages between users. At first, private rooms are created: if a private room needs to be established, for each user a room id: room:1:2 is generated, where numbers correspond to the user ids in ascending order.
E.g. Create a private room between 2 users:
SADD user:1:rooms 1:2 and SADD user:2:rooms 1:2Then we add messages to this room by writing to a sorted set:
ZADD room:1:2 1615480369 "{'from': 1, 'date': 1615480369, 'message': 'Hello', 'roomId': '1:2'}"We use a stringified JSON for keeping the message structure and simplify the implementation details for this demo-app.
Populate the "General" room with messages. Messages are added to the sorted set with id of the "General" room: room:0
After initialization, a pub/sub subscription is created: SUBSCRIBE MESSAGES. At the same time, each server instance will run a listener on a message on this channel to receive real-time updates.
Again, for simplicity, each message is serialized to JSON, which we parse and then handle in the same manner, as WebSocket messages.
Pub/sub allows connecting multiple servers written in different platforms without taking into consideration the implementation detail of each server.
When a WebSocket/real-time server is instantiated, which listens for the next events:
A global set with online_users key is used for keeping the online state for each user. So on a new connection, a user ID is written to that set:
SADD online_users 1Here we have added user with id 1 to the set online_users
After that, a message is broadcasted to the clients to notify them that a new user is joined the chat.
PUBLISH message "{'serverId': 4132, 'type':'message', 'data': {'from': 1, 'date': 1615480369, 'message': 'Hello', 'roomId': '1:2'}}"Note we send additional data related to the type of the message and the server id. Server id is used to discard the messages by the server instance which sends them since it is connected to the same MESSAGES channel.
The type field of the serialized JSON corresponds to the real-time method we use for real-time communication (connect/disconnect/message).
The data is method-specific information. In the example above it's related to the new message.
Redis is used mainly as a database to keep the user/messages data and for sending messages between connected servers.
The real-time functionality is handled by Socket.IO for server-client messaging. Additionally each server instance subscribes to the MESSAGES channel of pub/sub and dispatches messages once they arrive. Note that, the server transports pub/sub messages with a separate event stream (handled by Server Sent Events), this is due to the need of running pub/sub message loop apart from socket.io signals.
The chat data is stored in various keys and various data types. User data is stored in a hash set where each user entry contains the next values:
username: unique user name;
password: hashed password
Additionally a set of chat rooms is associated with user
Rooms are sorted sets which contains messages where score is the timestamp for each message
Each chat room has a name associated with it
The "online" set is global for all users is used for keeping track on which user is online.
Each user hash's set is accessed by key user:{userId}. The data for it stored with HSET key field data. User ID is calculated by incrementing the total_users key (INCR total_users)
Usernames are stored as separate keys (username:{username}) which returns the userId for quicker access and stored with SET username:{username} {userId}.
Rooms which a user belongs to are stored at user:{userId}:rooms as a set of chat room ids. A room is added by SADD user:{userId}:rooms {roomId} command.
Messages are stored at room:{roomId} key in a sorted set (as mentioned above). They are added with the ZADD room:{roomId} {timestamp} {message} command. Messages are serialized to an app-specific JSON string.
Get User HGETALL user:{id}.
HGETALL user:2where we get data for the user with id: 2.
SMEMBERS user:2:roomsThis will return IDs of chat rooms for user with ID: 2
ZREVRANGE room:{roomId} {offset_start} {offset_end}. Example: ZREVRANGE room:1:2 0 50It will return 50 messages with 0 offsets for the private room between users with IDs 1 and 2.
