In this post we will learn about how to use Spring Batch to read from MySQL database using
JdbcCursorItemReader and write to a Flat file using FlatFileItemWriter . We will also witness the usage of JobExecutionListener and itemProcessor . Letâs get going.
Oct 13, 2017 - 0 â Spring Batch CSV Processing Example Application. States FlatFileIteamReader will âread lines of data from a flat file that typically describe. If we wanted to apply business logic before writing to the database, you could. Jul 11, 2018 - Learn to use Spring batch to read records from CSV file and insert into database using JdbcBatchItemWriter. I am using embedded database.
Other interesting posts you may like
Following technologies being used:
Letâs begin.
Step 1: Create project directory structure
Following will be the final project structure:
We will be reading MySQL database and write to a flat file (
project/csv/examResult.txt ).
Step 2: Create Database Table and populate it with sample data
Create a fairly simple table in MySQL database which maps to our domain model(and sufficient for this example).
Please visit MySQL installation on Local PC in case you are finding difficulties in setting up MySQL locally.
Now letâs add all contents mentioned in project structure in step 1.
Step 3: Update pom.xml to include required dependencies
Following is the updated minimalistic
pom.xml
As we need to interact with db this time, we will use spring-jdbc support. We will also need mysql connector to communicate with MySQL, and since we are also using joda-time for any date-time processing we might need, we will include that dependency as well.
Step 4: Create domain object & Mapper (RowMapper implementaion)
We will be mapping the data from database table to properties of our domain object.
com.websystique.springbatch.model.ExamResult
Below class will eventually map the data from database into domain object based on actual properties datatypes.
com.websystique.springbatch.ExamResultRowMapper
Step 5: Create an ItemProcessorItemProcessor is Optional, and called after item read but before item write. It gives us the opportunity to perform a business logic on each item. In our case, for example, we will filter out all the items whose percentage is less than 80. So final result will only have records with percentage >= 80.
com.websystique.springbatch.ExamResultItemProcessor
Step 6: Add a Job listener(JobExecutionListener)Job listener is Optional and provide the opportunity to execute some business logic before job start and after job completed.For example setting up environment can be done before job and cleanup can be done after job completed.
com.websystique.springbatch.ExamResultJobListener
Step 7: Create Spring Context with job configuration
Create dataSource bean needed for database communication
src/main/resources/context-datasource.xml
Create the Spring context with batch job configuration.
src/main/resources/spring-batch-context.xml
As you can see, we have setup a job with only one step. Step uses
JdbcCursorItemReader to read the records from MySQL database, itemProcessor to process the record and FlatFileItemWriter to write the records to a flat file. commit-interval specifies the number of items that can be processed before the transaction is committed/ before the write will happen.Grouping several record in single transaction and write them as chunk provides performance improvement. We have also shown the use of jobListener which can contain any arbitrary logic you might need to run before and after the job.
Step 8: Create Main application to finally run the job
Create a Java application to run the job.
com.websystique.springbatch.Main
Running above program as java application, you will see following output
You can see that we have processed all input records from Database. Below is the generated flat file (txt) found in project/csv folder
Only the records which are meeting specific condition ( percentage >=80 ) are included here, thanks to itemProcessor filtering logic.
Thatâs it.
Download Source CodeReferences
If you like tutorials on this site, why not take a step further and connect me on Facebook , Google Plus & Twitter as well? I would love to hear your thoughts on these articles, it will help improve further our learning process.
Related posts:Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign up
Find file Copy path
Cannot retrieve contributors at this time
Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |