Create the Book-Category-Book-Ratings domain, load and transform JSON data, and implement the Books API.
In this lesson, students will learn:
This lesson will start by fleshing out the Book, Category, and BookRating models with their respective Spring Repositories.
We’ll start with the Category
. A Book
belongs to one or more categories. The Category
has a name that we will derive from the JSON data
files in src/main/resources/data/books
. As we’ve done previously, we will map the class to a Redis Hash.
Add the file src/main/java/com/redislabs/edu/redi2read/models/Category.java
with the following contents:
package com.redislabs.edu.redi2read.models;
import org.springframework.data.annotation.Id;
import org.springframework.data.redis.core.RedisHash;
import lombok.Builder;
import lombok.Data;
@Data
@Builder
@RedisHash
public class Category {
@Id
private String id;
private String name;
}
The corresponding repository extends Spring’s CrudRepository. Add the file src/main/java/com/redislabs/edu/redi2read/repositories/CategoryRepository.java
with the following contents:
package com.redislabs.edu.redi2read.repositories;
import com.redislabs.edu.redi2read.models.Category;
import org.springframework.data.repository.CrudRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface CategoryRepository extends CrudRepository<Category, String> {
}
The Book model maps directly to the JSON payload in the *.json files in src/main/resources/data/books
.
For example, the JSON object shown below came from the the file redis_0.json:
{
"pageCount": 228,
"thumbnail": "http://books.google.com/books/content?id=NsseEAAAQBAJ&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api",
"price": 9.95,
"subtitle": "Explore Redis - Its Architecture, Data Structures and Modules like Search, JSON, AI, Graph, Timeseries (English Edition)",
"description": "Complete reference guide to Redis KEY FEATURES ● Complete coverage of Redis Modules.",
"language": "en",
"currency": "USD",
"id": "8194837766",
"title": "Redis(r) Deep Dive",
"infoLink": "https://play.google.com/store/books/details?id=NsseEAAAQBAJ&source=gbs_api",
"authors": ["Suyog Dilip Kale", "Chinmay Kulkarni"]
},
...
}
The category name is extracted from the file name as "redis". The same applies to any book from the files: redis_0.json
, redis_1.json
, redis_2.json
, and redis_3.json
.
The Book class contains a Set<Category>
which will for now contain the single category extracted from the filename. The Set<String>
for authors gets mapped from the "authors" JSON array in the payload.
Add the file src/main/java/com/redislabs/edu/redi2read/repositories/Book.java
with the following contents:
package com.redislabs.edu.redi2read.models;
import java.util.HashSet;
import java.util.Set;
import org.springframework.data.annotation.Id;
import org.springframework.data.annotation.Reference;
import org.springframework.data.redis.core.RedisHash;
import lombok.Data;
import lombok.EqualsAndHashCode;
@Data
@EqualsAndHashCode(onlyExplicitlyIncluded = true)
@RedisHash
public class Book {
@Id
@EqualsAndHashCode.Include
private String id;
private String title;
private String subtitle;
private String description;
private String language;
private Long pageCount;
private String thumbnail;
private Double price;
private String currency;
private String infoLink;
private Set<String> authors;
@Reference
private Set<Category> categories = new HashSet<Category>();
public void addCategory(Category category) {
categories.add(category);
}
}
In the BookRepository
we introduce the usage of the PaginationAndSortingRepository
. The PaginationAndSortingRepository
extends the CrudRepository
interface
and adds additional methods to ease paginated access to entities.
We will learn more about the usage of the PagingAndSortingRepository
when we implement the BookController
.
Add the file src/main/java/com/redislabs/edu/redi2read/repositories/BookRepository.java
with the following contents:
package com.redislabs.edu.redi2read.repositories;
import com.redislabs.edu.redi2read.models.Book;
import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface BookRepository extends PagingAndSortingRepository<Book, String> {
}
The BookRating
model represents a rating of a book by a user. We implement a traditional 5 star rating system as a many-to-many relationship.
The BookRating
model plays a role equivalent to that of a joining table or bridging table in a relational model.
BookRating
sits between the two other entities of a many-to-many relationship. Its purpose is to store a record for each of the combinations
of these other two entities (Book and User).
We keep the links to the Book and User models using the @Reference
annotation (the equivalent of having foreign keys in a relational database)
Add the file src/main/java/com/redislabs/edu/redi2read/models/BookRating.java
with the following contents:
package com.redislabs.edu.redi2read.models;
import javax.validation.constraints.NotNull;
import org.springframework.data.annotation.Id;
import org.springframework.data.annotation.Reference;
import org.springframework.data.redis.core.RedisHash;
import lombok.Builder;
import lombok.Data;
@Data
@Builder
@RedisHash
public class BookRating {
@Id
private String id;
@NotNull
@Reference
private User user;
@NotNull
@Reference
private Book book;
@NotNull
private Integer rating;
}
The corresponding Repository simply extends Spring’s CrudRepository. Add the file src/main/java/com/redislabs/edu/redi2read/repositories/BookRatingRepository.java
with the following contents:
package com.redislabs.edu.redi2read.repositories;
import com.redislabs.edu.redi2read.models.BookRating;
import org.springframework.data.repository.CrudRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface BookRatingRepository extends CrudRepository<BookRating, String> {
}
Now that we have our models and repositories defined, let’s load the books from the provided JSON data in the src/main/resources/data/books
directory.
We’ll create a CommandLineRunner
to iterate over every .json file in the data/books
directory. We will map the content of each file to a Book object using Jackson.
We'll create a category using the characters in the filename up to the last underscore. If there is no category with that name already, we will create one.
The category is then added to the set of categories for the book.
Add the file src/main/java/com/redislabs/edu/redi2read/boot/CreateBooks.java
with the following contents:
package com.redislabs.edu.redi2read.boot;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.redislabs.edu.redi2read.models.Book;
import com.redislabs.edu.redi2read.models.Category;
import com.redislabs.edu.redi2read.repositories.BookRepository;
import com.redislabs.edu.redi2read.repositories.CategoryRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import lombok.extern.slf4j.Slf4j;
@Component
@Order(3)
@Slf4j
public class CreateBooks implements CommandLineRunner {
@Autowired
private BookRepository bookRepository;
@Autowired
private CategoryRepository categoryRepository;
@Override
public void run(String... args) throws Exception {
if (bookRepository.count() == 0) {
ObjectMapper mapper = new ObjectMapper();
TypeReference<List<Book>> typeReference = new TypeReference<List<Book>>() {
};
List<File> files = //
Files.list(Paths.get(getClass().getResource("/data/books").toURI())) //
.filter(Files::isRegularFile) //
.filter(path -> path.toString().endsWith(".json")) //
.map(java.nio.file.Path::toFile) //
.collect(Collectors.toList());
Map<String, Category> categories = new HashMap<String, Category>();
files.forEach(file -> {
try {
log.info(">>>> Processing Book File: " + file.getPath());
String categoryName = file.getName().substring(0, file.getName().lastIndexOf("_"));
log.info(">>>> Category: " + categoryName);
Category category;
if (!categories.containsKey(categoryName)) {
category = Category.builder().name(categoryName).build();
categoryRepository.save(category);
categories.put(categoryName, category);
} else {
category = categories.get(categoryName);
}
InputStream inputStream = new FileInputStream(file);
List<Book> books = mapper.readValue(inputStream, typeReference);
books.stream().forEach((book) -> {
book.addCategory(category);
bookRepository.save(book);
});
log.info(">>>> " + books.size() + " Books Saved!");
} catch (IOException e) {
log.info("Unable to import books: " + e.getMessage());
}
});
log.info(">>>> Loaded Book Data and Created books...");
}
}
}
There's a lot to unpack here, so let’s take it from the top:
ObjectMapper
and a TypeReference
to perform the mapping.Map
of String
s to Category
objects to collect the categories as we process
the files and quickly determine whether we have already created a category.Now we can implement the initial version of the BookController
: our Bookstore Catalog API. This first version of the BookController
will have three endpoints:
src/main/java/com/redislabs/edu/redi2read/controllers/BookController.java
with the following contents:package com.redislabs.edu.redi2read.controllers;
import com.redislabs.edu.redi2read.models.Book;
import com.redislabs.edu.redi2read.models.Category;
import com.redislabs.edu.redi2read.repositories.BookRepository;
import com.redislabs.edu.redi2read.repositories.CategoryRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/api/books")
public class BookController {
@Autowired
private BookRepository bookRepository;
@Autowired
private CategoryRepository categoryRepository;
@GetMapping
public Iterable<Book> all() {
return bookRepository.findAll();
}
@GetMapping("/categories")
public Iterable<Category> getCategories() {
return categoryRepository.findAll();
}
@GetMapping("/{isbn}")
public Book get(@PathVariable("isbn") String isbn) {
return bookRepository.findById(isbn).get();
}
}
To get all books, we issue a GET request to http://localhost:8080/api/books/
. This endpoint is implemented in the all method, which calls the BookRepository findAll method. Using curl:
curl --location --request GET 'http://localhost:8080/api/books/'
The result is an array of JSON objects containing the books:
[
{
"id": "1783980117",
"title": "RESTful Java Web Services Security",
"subtitle": null,
"description": "A sequential and easy-to-follow guide which allows you to understand the concepts related to securing web apps/services quickly and efficiently, since each topic is explained and described with the help of an example and in a step-by-step manner, helping you to easily implement the examples in your own projects. This book is intended for web application developers who use RESTful web services to power their websites. Prior knowledge of RESTful is not mandatory, but would be advisable.",
"language": "en",
"pageCount": 144,
"thumbnail": "http://books.google.com/books/content?id=Dh8ZBAAAQBAJ&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api",
"price": 11.99,
"currency": "USD",
"infoLink": "https://play.google.com/store/books/details?id=Dh8ZBAAAQBAJ&source=gbs_api",
"authors": [
"Andrés Salazar C.",
"René Enríquez"
],
"categories": [
{
"id": "f2ada1e2-7c18-4d90-bfe7-e321b650c0a3",
"name": "redis"
}
]
},
...
]
To get a specific book, we issue a GET request to http://localhost:8080/api/books/{isbn}
. This endpoint is implemented in the get method, which calls the BookRepository findById method. Using curl:
curl --location --request GET 'http://localhost:8080/api/books/1680503545'
The result is a JSON object containing the book:
{
"id": "1680503545",
"title": "Functional Programming in Java",
"subtitle": "Harnessing the Power Of Java 8 Lambda Expressions",
"description": "Intermediate level, for programmers fairly familiar with Java, but new to the functional style of programming and lambda expressions. Get ready to program in a whole new way. ...",
"language": "en",
"pageCount": 196,
"thumbnail": "http://books.google.com/books/content?id=_g5QDwAAQBAJ&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api",
"price": 28.99,
"currency": "USD",
"infoLink": "https://play.google.com/store/books/details?id=_g5QDwAAQBAJ&source=gbs_api",
"authors": ["Venkat Subramaniam"],
"categories": [
{
"id": "9d5c025e-bf38-4b50-a971-17e0b7408385",
"name": "java"
}
]
}
To get all categories, we issue a GET request to http://localhost:8080/api/books/categories
. It’s implemented in the getCategories
method, which calls the CategoriesRepository
findAll
method. Using curl:
curl --location --request GET 'http://localhost:8080/api/books/categories'
The result is an array of JSON objects containing the categories:
[
{
"id": "2fd916fe-7ff8-44c7-9f86-ca388565256c",
"name": "mongodb"
},
{
"id": "9615a135-7472-48fc-b8ac-a5516a2c8b22",
"name": "dynamodb"
},
{
"id": "f2ada1e2-7c18-4d90-bfe7-e321b650c0a3",
"name": "redis"
},
{
"id": "08fc8148-d924-4d2e-af7e-f5fe6f2861f0",
"name": "elixir"
},
{
"id": "b6a0b57b-ebb8-4d98-9352-8236256dbc27",
"name": "microservices"
},
{
"id": "7821fd6a-ec94-4ac6-8089-a480a7c7f2ee",
"name": "elastic_search"
},
{
"id": "f2be1bc3-1700-45f5-a300-2c4cf2f90583",
"name": "hbase"
},
{
"id": "31c8ea64-cad2-40d9-b0f6-30b8ea6fcbfb",
"name": "reactjs"
},
{
"id": "5e527af7-93a1-4c00-8f20-f89e89a213e8",
"name": "apache_spark"
},
{
"id": "9d5c025e-bf38-4b50-a971-17e0b7408385",
"name": "java"
},
{
"id": "bcb2a01c-9b0a-4846-b1be-670168b5d768",
"name": "clojure"
},
{
"id": "aba53bb9-7cfa-4b65-8900-8c7e857311c6",
"name": "couchbase"
},
{
"id": "bd1b2877-1564-4def-b3f7-18871165ff10",
"name": "riak"
},
{
"id": "47d9a769-bbc2-4068-b27f-2b800bec1565",
"name": "kotlin"
},
{
"id": "400c8f5a-953b-4b8b-b21d-045535d8084d",
"name": "nosql_big_data"
},
{
"id": "06bc25ff-f2ab-481b-a4d9-819552dea0e0",
"name": "javascript"
}
]
Next, we will create a random set of book ratings. Later in the course, we’ll use these for an example. Following the same recipe we used to seed Redis with a CommandLineRunner
, add the file src/main/java/com/redislabs/edu/redi2read/boot/CreateBookRatings.java
with the following contents:
package com.redislabs.edu.redi2read.boot;
import java.util.Random;
import java.util.stream.IntStream;
import com.redislabs.edu.redi2read.models.Book;
import com.redislabs.edu.redi2read.models.BookRating;
import com.redislabs.edu.redi2read.models.User;
import com.redislabs.edu.redi2read.repositories.BookRatingRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.CommandLineRunner;
import org.springframework.core.annotation.Order;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Component;
import lombok.extern.slf4j.Slf4j;
@Component
@Order(4)
@Slf4j
public class CreateBookRatings implements CommandLineRunner {
@Value("${app.numberOfRatings}")
private Integer numberOfRatings;
@Value("${app.ratingStars}")
private Integer ratingStars;
@Autowired
private RedisTemplate<String, String> redisTemplate;
@Autowired
private BookRatingRepository bookRatingRepo;
@Override
public void run(String... args) throws Exception {
if (bookRatingRepo.count() == 0) {
Random random = new Random();
IntStream.range(0, numberOfRatings).forEach(n -> {
String bookId = redisTemplate.opsForSet().randomMember(Book.class.getName());
String userId = redisTemplate.opsForSet().randomMember(User.class.getName());
int stars = random.nextInt(ratingStars) + 1;
User user = new User();
user.setId(userId);
Book book = new Book();
book.setId(bookId);
BookRating rating = BookRating.builder() //
.user(user) //
.book(book) //
.rating(stars).build();
bookRatingRepo.save(rating);
});
log.info(">>>> BookRating created...");
}
}
}
This CommandLineRunner
creates a configurable number of random ratings for a random set of books and users. We use RedisTemplate.opsForSet().randomMember()
to request a random ID from the set of users and books.
Then we choose a random integer between 1 and the total number of stars in our rating system to create the rating.
This class introduces the use of the @Value
annotation, which will grab the property inside the String param ${foo}
from the application’s property file.
In the file src/main/resources/application.properties
add the following values:
app.numberOfRatings=5000
app.ratingStars=5
Pagination is helpful when we have a large dataset and want to present it to the user in smaller chunks. As we learned earlier in the lesson, the BookRepository
extends the PagingAndSortingRepository
, which is built on top of the CrudRepository
.
In this section, we will refactor the BookController
all method to work with the pagination features of the PagingAndSortingRepository
. Replace the previously created all method with the following contents:
@GetMapping
public ResponseEntity<Map<String, Object>> all( //
@RequestParam(defaultValue = "0") Integer page, //
@RequestParam(defaultValue = "10") Integer size //
) {
Pageable paging = PageRequest.of(page, size);
Page<Book> pagedResult = bookRepository.findAll(paging);
List<Book> books = pagedResult.hasContent() ? pagedResult.getContent() : Collections.emptyList();
Map<String, Object> response = new HashMap<>();
response.put("books", books);
response.put("page", pagedResult.getNumber());
response.put("pages", pagedResult.getTotalPages());
response.put("total", pagedResult.getTotalElements());
return new ResponseEntity<>(response, new HttpHeaders(), HttpStatus.OK);
}
Let’s break down the refactoring:
ResponseEntity
, which is an extension of HttpEntity
and gives us control over the HTTP status code, headers, and body.Map<String,Object>
to return the collection of books as well as pagination data.Pageable
and PageRequest
abstractions to construct the paging request.Page<Book>
result by invoking the findAll
method, passing the Pageable
paging request.ResponseEntity
.Let’s fire up a pagination request with curl as shown next:
curl --location --request GET 'http://localhost:8080/api/books/?size=25&page=2'
Passing a page size of 25 and requesting page number 2, we get the following:
{
"total": 2403,
"books": [
{
"id": "1786469960",
"title": "Data Visualization with D3 4.x Cookbook",
"subtitle": null,
"description": "Discover over 65 recipes to help you create breathtaking data visualizations using the latest features of D3...",
"language": "en",
"pageCount": 370,
"thumbnail": "http://books.google.com/books/content?id=DVQoDwAAQBAJ&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api",
"price": 22.39,
"currency": "USD",
"infoLink": "https://play.google.com/store/books/details?id=DVQoDwAAQBAJ&source=gbs_api",
"authors": [
"Nick Zhu"
],
"categories": [
{
"id": "f2ada1e2-7c18-4d90-bfe7-e321b650c0a3",
"name": "redis"
}
]
},
{
"id": "111871735X",
"title": "Android Programming",
"subtitle": "Pushing the Limits",
"description": "Unleash the power of the Android OS and build the kinds ofbrilliant, innovative apps users love to use ...",
"language": "en",
"pageCount": 432,
"thumbnail": "http://books.google.com/books/content?id=SUWPAQAAQBAJ&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api",
"price": 30.0,
"currency": "USD",
"infoLink": "https://play.google.com/store/books/details?id=SUWPAQAAQBAJ&source=gbs_api",
"authors": [
"Erik Hellman"
],
"categories": [
{
"id": "47d9a769-bbc2-4068-b27f-2b800bec1565",
"name": "kotlin"
}
]
},