In this Hibernate tutorial, you will learn how to write code that efficiently executes insert/update a large number of rows in the database, using Hibernate batch insert/update support. The code examples in this tutorial were tested with Hibernate framework 5.3 and Oracle Express 18c database.
1. Why do we need Hibernate Batch Update/Insert?
Consider the following code snippet that is supposed to insert 1 million rows into a database table:
Transaction transaction = session.beginTransaction();
long startTime = System.currentTimeMillis();
for (int i = 1 ; i <= 1_000_000; i++) {
Customer customer = new Customer(...);
session.save(customer);
}
transaction.commit();
This may cause OutOfMemoryError because Hibernate manages each instance of the Customerobject in its session’s cache. A solution is to flush and clear the session repeatedly after a number of objects persisted, like this:
for (int i = 1; i <= 1_000_000; i++) {
Customer customer = new Customer(...);
session.save(customer);
if (i % 100 == 0) {
session.flush();
session.clear();
}
}
This will avoid the OutOfMemoryError but it is still inefficient because there will be many roundtrips to the database. So we need to use the batch update feature supported by Hibernate in order to have best performance. If configured properly, Hibernate will send a group of SQL statements each time, e.g. 20 statements in a batch, to the database – thus greatly reduces the number of roundtrips to the database, which means performance will be improved.Note that Hibernate batch update relies on JDBC batch update, so it will work only with databases that support batching. And Hibernate batch update work only with databases that supports sequence like Oracle. It won’t work with databases that support auto-increment for primary key column like MySQL.To make sure that Hibernate actually executes batch update, you should enable the log category org.hibernate at debug level to see all the logging information. Follow this article to know to configure Hibernate logging with log4j2.
2. Enable Hibernate Batch Update
By default, batch update is disabled in Hibernate. To enable, you must set the hibernate.jdbc.batch_size property to value greater than zero. For example, in the hibernate.cfg.xml file:
This sets the batch size to 20, which means Hibernate will group 20 SQL statements to be sent to the database.
3. Hibernate Batch Insert for Single Table
Suppose that we have the Customer class maps to the CUSTOMERS table in the database as follows:
package net.codejava;
import javax.persistence.*;
@Entity
@Table(name = "CUSTOMERS")
public class Customer {
private Integer id;
private String name;
private String email;
public Customer() {
}
public Customer(String name, String email) {
super();
this.name = name;
this.email = email;
}
@Id
@Column(name = "CUSTOMER_ID")
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "c_generator")
@SequenceGenerator(name = "c_generator", sequenceName = "CUSTOMER_SEQUENCE")
public Integer getId() {
return id;
}
// other getters and setters...
}
Note that we use @SequenceGenerator for the ID field so you must create sequence named CUSTOMER_SEQUENCE in the database.The following program illustrates how to persists 1,000 Customer objects using Hibernate batch update:
package net.codejava;
import org.hibernate.*;
import org.hibernate.cfg.Configuration;
public class SingleTableBatchInsertTest {
public static void main(String[] args) {
Configuration config = new Configuration();
config.setProperty("hibernate.connection.driver_class", "oracle.jdbc.OracleDriver");
config.setProperty("hibernate.connection.url", "jdbc:oracle:thin:@localhost:1521:xe");
config.setProperty("hibernate.connection.username", "user");
config.setProperty("hibernate.connection.password", "pass");
config.setProperty("hibernate.dialect", "org.hibernate.dialect.Oracle8iDialect");
config.setProperty("hibernate.jdbc.batch_size", "50");
config.addAnnotatedClass(Customer.class);
SessionFactory sessionFactory = config.buildSessionFactory();
Session session = sessionFactory.openSession();
Transaction transaction = session.beginTransaction();
for (int i = 1 ; i <= 1000; i++) {
Customer customer = new Customer("Name-" + i, "Email-" + i);
session.save(customer);
if (i % 50 == 0) {
session.flush();
session.clear();
}
}
transaction.commit();
session.close();
sessionFactory.close();
}
}
As you can see, we use programmatic configuration for Hibernate and set the batch size value is 50. Run this program and look at the log lines printed in the console, you can see:
That means Hibernate actually sent SQL Insert statements in batches to the database.
4. Hibernate Batch Insert for One to Many Relationship
Let’s see how to configure Hibernate batch insert for one-to-many relationship. Suppose that in the database we have two tables Category and Product. A category can have one to many products. Code of the Category class:
package net.codejava;
import java.util.*;
import javax.persistence.*;
@Entity
public class Category {
private Integer id;
private String name;
private Set<Product> products = new HashSet<>();
public Category() {
}
public Category(String name) {
super();
this.name = name;
}
@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE)
@Column(name = "CATEGORY_ID")
public Integer getId() {
return id;
}
@OneToMany(mappedBy = "category", cascade = CascadeType.ALL)
public Set<Product> getProducts() {
return products;
}
public void setProducts(Set<Product> products) {
this.products = products;
}
// other getters and setters are hidden...
}
Code for the Product class:
package net.codejava;
import javax.persistence.*;
@Entity
public class Product {
private Integer id;
private String name;
private Float price;
private Category category;
public Product(String name, Float price, Category category) {
this.name = name;
this.price = price;
this.category = category;
}
public Product() {
}
@Id
@Column(name = "PRODUCT_ID")
@GeneratedValue(strategy = GenerationType.SEQUENCE)
public Integer getId() {
return id;
}
@ManyToOne
@JoinColumn(name = "CATEGORY_ID")
public Category getCategory() {
return category;
}
// other getters and setters are hidden...
}
And here is code of the test program:
And here is code of the test program:
package net.codejava;
import org.hibernate.*;
import org.hibernate.cfg.Configuration;
public class One2ManyBatchInsertTest {
public static void main(String[] args) {
Configuration config = new Configuration();
config.setProperty("hibernate.connection.driver_class", "oracle.jdbc.OracleDriver");
config.setProperty("hibernate.connection.url", "jdbc:oracle:thin:@localhost:1521:xe");
config.setProperty("hibernate.connection.username", "user");
config.setProperty("hibernate.connection.password", "pass");
config.setProperty("hibernate.dialect", "org.hibernate.dialect.Oracle8iDialect");
config.setProperty("hibernate.jdbc.batch_size", "10");
config.addAnnotatedClass(Category.class);
config.addAnnotatedClass(Product.class);
SessionFactory sessionFactory = config.buildSessionFactory();
Session session = sessionFactory.openSession();
Transaction transaction = session.beginTransaction();
for (int i = 1; i <= 5; i++) {
Category category = new Category("Category-" + i);
Product product1 = new Product("Product-" + i, 100f, category);
Product product2 = new Product("Product-" + i, 200f, category);
Product product3 = new Product("Product-" + i, 300f, category);
Product product4 = new Product("Product-" + i, 400f, category);
Product product5 = new Product("Product-" + i, 500f, category);
category.getProducts().add(product1);
category.getProducts().add(product2);
category.getProducts().add(product3);
category.getProducts().add(product4);
category.getProducts().add(product5);
session.save(category);
}
transaction.commit();
session.close();
sessionFactory.close();
}
}
As you can see, we set the batch size to 10. In the for loop, we persist 5 Categoryobjects, with each we persist 5 Product objects. Run this program and you will see the following log output:
As you can see, Hibernate issues a batch insert for each entity type. The log shows that Hibernate sent 5 batches of size 1 for each Category persist, and 5 batches of size 5 for Product persist – too many small batches is very inefficient.So to improve the performance, you can set the hibernate.order_inserts property to true:
You see, Hibernate now orders the SQL Insert statements by entity type: 1 batch for category (5 objects) and 3 batches for product (25 objects).So remember to set the hibernate.order_inserts property to true so Hibernate will order similar SQL statements to improve the performance.
5. Hibernate Batch Update
To get best performance for batch update using Hibernate, you should set true for the properties hibernate.order_updates and hibernate.batch_versioned_data. For example, with programmatic configuration:
This program retrieves all Categoryobjects from the database, then for each category it updates category name and names of associated products. Run this program and you will see the log output something like this:
[DEBUG] [main] SQL - update Category set name=? where CATEGORY_ID=?
[DEBUG] [main] BatchingBatch - Executing batch size: 16
[DEBUG] [main] SQL - update Product set CATEGORY_ID=?, name=?, price=? where PRODUCT_ID=?
[DEBUG] [main] BatchingBatch - Executing batch size: 50
[DEBUG] [main] SQL - update Product set CATEGORY_ID=?, name=?, price=? where PRODUCT_ID=?
[DEBUG] [main] BatchingBatch - Executing batch size: 24
As you can see, we use batch size 50 in the above program. The bigger the size, the less number of batches. You should experiment different batch size values to get the optimal performance.
6. Summary
So far you have learned how to execute batch insert/update with Hibernate. It’s not difficult, all you have to do is putting some extra properties, to summary:
Remember that Hibernate batch update/insert doesn’t work with auto-increment (identity) primary key column (e.g. MySQL). So you should use sequence for your entity’s primary key field and the database has to support it (e.g. Oracle).You can also watch the video version here:
Nam Ha Minh is certified Java programmer (SCJP and SCWCD). He began programming with Java back in the days of Java 1.4 and has been passionate about it ever since. You can connect with him on Facebook and watch his Java videos on YouTube.
In the case of one to many or many to many relationships and batch updates what happens when one of the record fails. Is the parent also rolled back in that case ? Also when a record fails to save using batch how can we log that somewhere
Nam Ha Minh is certified Java programmer (SCJP and SCWCD). He began programming with Java back in the days of Java 1.4 and has been passionate about it ever since. You can connect with him on Facebook and watch his Java videos on YouTube.
Comments
In the case of one to many or many to many relationships and batch updates what happens when one of the record fails. Is the parent also rolled back in that case ? Also when a record fails to save using batch how can we log that somewhere