Get started with MongoDB using Spring Data.
Get started with MongoDB using Spring Data.
Spring and Spring Boot are vast topics that you can spend weeks together to get a grasp of. You will be learning one portion of it here - How to access and manipulate data in Mongodb through the Spring Data project. If you don’t know anything about Spring / Spring Boot, please go through the Spring Boot byte before attempting this one.
Spring
In layman’s terms, it’s an application framework that helps you build Java applications with all bells and whistles really fast.
Spring Data
Spring Data is an umbrella project that contains many sub-projects, each of which intends to provide a Spring based programming model for data access through a variety of data sources. One of these sub-projects is for MongoDB. Few other sub-project examples are those that deal with Redis, Couchbase, and so on.
The context for this byte is a typical forum which has users, and these users all make posts on the forum. For the sake of this exercise, the content of the posts is randomized.
You are given the forum data and must use Spring Data and its modules to achieve the milestones given.
Understand how Spring Data for Mongo DB works
Perform queries using MongoRepository
Understand the difference between MongoRepository and MongoTemplate
Get started with MongoDB using Spring Data.
Spring and Spring Boot are vast topics that you can spend weeks together to get a grasp of. You will be learning one portion of it here - How to access and manipulate data in Mongodb through the Spring Data project. If you don’t know anything about Spring / Spring Boot, please go through the Spring Boot byte before attempting this one.
Spring
In layman’s terms, it’s an application framework that helps you build Java applications with all bells and whistles really fast.
Spring Data
Spring Data is an umbrella project that contains many sub-projects, each of which intends to provide a Spring based programming model for data access through a variety of data sources. One of these sub-projects is for MongoDB. Few other sub-project examples are those that deal with Redis, Couchbase, and so on.
The context for this byte is a typical forum which has users, and these users all make posts on the forum. For the sake of this exercise, the content of the posts is randomized.
You are given the forum data and must use Spring Data and its modules to achieve the milestones given.
Understand how Spring Data for Mongo DB works
Perform queries using MongoRepository
Understand the difference between MongoRepository and MongoTemplate
Create ~/workspace/bytes/
directory and cd
to it
mkdir -p ~/workspace/bytes/
cd ~/workspace/bytes/
Download the source code to the ~/workspace/bytes/
directory using one of the following commands:
git clone https://gitlab.crio.do/crio_bytes/me_byte_springdata.git
git clone git@gitlab.crio.do:crio_bytes/me_byte_springdata.git
The repository has a data.json
file which is a data dump you need to load into MongoDB. To do this, use these commands:
cd ~/workspace/bytes/me_byte_springdata
mongoimport --db=forumdb --type=json --file=data.json --collection=users --jsonArray
You can poke around the tables and documents using the mongo shell if you like. If you want to know more about using MongoDB, you can also take up the MongoDB Byte.
Can you move to your repo directory and start the server now using the following command? It may take 1-2 minutes when you run it the first time as a lot of dependencies get downloaded.
cd ~/workspace/bytes/me_byte_springdata
./gradlew bootRun
It will print the forum stats if you now visit http://localhost:8081/stats on your browser preview. It can be found on the left side at the bottom, the icon is shown in the screenshot. Clicking it opens an embedded browser window as shown in the screenshot as well.
or if you prefer the terminal, open the embedded terminal and use curl. You should get a HTML response.
# type in terminal
curl http://localhost:8081/stats
You are now running a REST API server which will respond back to your API requests.
It is recommended that you become familiar with the file and folder structure before you begin. Spend some time opening files and simply reading the code. It is not necessary to understand what is happening. It is to be able to remember the files, classes and perhaps method names that are used. Let’s take a look at some of the important files:
MainApplication.java: The main file, serves as the entry point to the application. It also has the code which starts the spring backend server.
ForumController.java: Controller file that intercepts the incoming requests, processes them and sends it onwards to the service layer.
ForumService.java and ForumServiceImpl.java: Files in the service layer. These two files simply forward the request to the forum’s repository layer.
ForumRepositoryService.java and ForumRepositoryServiceImpl.java: Repository layer files that issue queries to MongoDB. It also takes care of converting objects from entity objects to DTO objects.
UserRepository.java: It is the class through which interaction with the database occurs as you will soon see.
The forum controller prints the "Hello World" output but also a statistic on the number of users in the forum. Dig into the controller class and see where this number came from.
Open the file - src/main/java/com/crio/springdatabyte/controller/ForumController.java
and look for this snippet of code.
@GetMapping("/stats")
public ResponseEntity<String> stats() {
Stats stats = service.getForumStats();
String message = "<p>Hello!</p>"
+ "<div><b>Forum stats</b></div>"
+ "<div>---------------</div>"
+ "<div>Number of users: " + stats.getNumUsers() + "</div>";
return ResponseEntity.ok().body(message);
}
What is happening is that when we request the /stats url, it is intercepted by the method above and the output is sent to the browser. So the natural question to ask is where are the stats coming from?
The line responsible for that is service.getForumStats()
. A quick peek into ForumServiceImpl.java
will lead you to ForumRepositoryServiceImpl.java
. This is where you can see the code that talks to the database.
@Override
public Stats getForumStats() {
List<UserEntity> users = userRepository.findAll();
return new Stats(users.size());
}
This time, the line of interest is userRepository.findAll()
.
And once more, you can take a quick peek into the UserRepository.java
file to see the implementation, except for one problem… There’s no implementation!
Unlike the other times where there was an interface and a corresponding implementation, for the repository class, there is only an interface. Where does the implementation come from?
Spring Data provides it. The interface acts as a marker to help you capture the types you need to work with. And the method name is parsed by Spring to generate the query internally. If you observe the interface, it extends MongoRepository<UserEntity, Integer>
.
The "types" here are UserEntity
and Integer
. UserEntity
is the entity to manage (The Entity is a presentation of the row or document in the table or collection in your database). Integer
is the id of the managed entity object, in this case it is an integer. Also note, the entity class must be marked with @Document
for MongoDB to signify to Spring that it must manage it.
The other big advantage is that this repository pattern is an abstraction for different databases. This means, you can use a technology specific abstraction for different databases. The one you are using is MongoRepository
, but you can use JpaRepository
for relational databases such as MySQL.
The method name you have invoked is the findAll()
method Spring Data knows you have invoked the findAll() method, from the mongo repository class, then it generates the appropriate query automatically and sends it to the database that the server connects to using the details in application.properties
file.
Phew! That’s a lot of things that are happening under the hood.
That is precisely the advantage of using this pattern. Spring manages the hard work of query generation, connecting and disconnecting to databases and so on so that you can simply invoke one method and continue to focus on the business logic.
So much for the introduction. Next, you will see how to handle querying using this powerful abstraction.
Spring Data offers multiple projects under the same umbrella to connect to various data sources. You can find the entire list here: https://spring.io/projects/spring-data
Spring Data Mongo Repository reference: https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#repositories
Is the method name findAll()
fixed? What if you use findEverything()
, can you try it out and see if it works? Why or why not? Can you search the documentation for CrudRepository
and see how it relates to MongoRepository
?
Entities managed for Mongo DB must use @Document
. A "document" is a term specific for MongoDB, so does this mean a data source backed by a different technology uses a different annotation? Can you try to figure out what this would be for MySQL?
What if the entity property is longer in words? For example, let’s assume there the entity has properties: birthPlace
which signifies city of birth, birth
which is for date of birth and place
which is for current city of residence. How will you put this in a method that uses all three as criteria? Is the method name findByBirthPlaceBirthPlace
correct?
Hint: All the method parsing algorithm needs is a way to differentiate between the criteria. The default is to use camel case as a delimiter, which means you have to use a different delimiter in the above method name. How to do this?
The repository method returns List<UserEntity>
, but the username is generally unique in a forum. How would you change the method to return a single user document? Can you try it out?
The objective is to get users whose posts contain the string "cricket" in their post content. To do this, you need to search for the string inside an array of posts for every user document. This is not straightforward to do using a query method. But we can turn to the expressive power of @Query
annotation to solve this problem.
The code in the repository service is as follows:
@Override
public List<User> getUsersWithPostText(String text) {
List<UserEntity> userEntityList = userRepository.findUsersWithPostText(text);
List<User> userList = new ArrayList<>();
for(UserEntity userEntity : userEntityList) {
userList.add(modelMapper.map(userEntity, User.class));
}
return userList;
}
And the repository method with the annotation is:
@Query("{'posts.content': {$regex: ?0}}")
List<UserEntity> findUsersWithPostText(String content);
Of course, you are not limited to regex operators. You are only limited by the query capabilities offered by mongo. Thus the method annotation form offers pretty much the full expressive power of querying directly on MongoDB.
How many posts contain the word "ipl"?
Can you try searching for posts that have a substring and order it by the time of post creation?
The third form of querying the database that Spring Data offers is MongoTemplate. With each increasing step you have taken, you have traded convenience for expressiveness and flexibility. MongoTemplate is the most powerful form, allowing you to construct dynamic, ad-hoc queries which offer granular control over the other forms.
Let’s go through a previous use case - get users by username.
// ... other code
@Autowired
MongoTemplate mongoTemplate
// ... other code
Query query = new Query();
query.addCriteria(Criteria.where("username").is(username));
// ...
mongoTemplate.findOne(query, User.class);
Let’s break down this usage:
There are three objects here: Query
object, Criteria
object, MongoTemplate
object. The Query
and Criteria
objects have a fluent style API which makes it easy to chain together criteria and queries programmatically. This offers one big advantage over the other forms: You can construct queries dynamically. Previously, you were effectively limited to "hard-coded" queries which means if you query had to be changed based on some system condition, it was not possible unless there was already another query method that could be selected. With this chainable API, you can construct queries in an ad-hoc fashion.
Once the query is created, mongoTemplate can be used to execute the query. Notice the findOne
method. If you remember your mongo queries, it should be familiar. There are a whole host of methods MongoTemplate
has to help you beside this one.
The answer as always is: It depends. If your queries do not change much, or are relatively straightforward you may prefer query methods. If you want full flexibility to write your own ad-hoc queries, MongoTemplate
may be more preferable. You could also go for a mixed approach through proper layering and decoupling, where you could start off with the relatively simple repository query methods and then start adopting MongoTemplate as and when more expressive queries are required.
getPostsByUser
in the repository service layer to use MongoTemplate
.