Spring Batch – Configuring Retry and Skip Logic

Retry and skip logic are essential features in Spring Batch that help manage errors and exceptions during batch processing. They ensure that batch jobs can recover from failures and continue processing data without causing job failures. Here’s an explanation of retry and skip logic in Spring Batch:

Retry Logic

Spring Batch includes retry logic which lets configure and manage the number of attempts the process will attempt when encountering a non-fatal error. Non-fatal errors usually pass with a retry and tend to be temporary problems. Using retry logic in your batch processing tasks makes the jobs more robust, in that before labeling a job as a failure, an unsuccessful one is tried again. Below are steps that can be followed to implement retry logic in Spring Batch:

1. Identify the Batch Step:

The first step is to identify the specific batch step where you want to implement retry logic. You typically apply retry logic to a chunk-based step, where data is read, processed, and written in chunks.

2. Configure the Step:

Use StepBuilderFactory to create and configure the batch step in your Spring Batch configuration. Below is an example

Java

@Bean
public Step myStep() {
    return stepBuilderFactory.get("myStep")
        .<Input, Output>chunk(10) // Chunk size
        .reader(reader())
        .processor(processor())
        .writer(writer())
        .faultTolerant() // Enable fault tolerance
        .retryLimit(3) // Number of retry attempts
        .retry(Exception.class) // Retry on specific exception(s)
        .backOffPolicy(new ExponentialBackOffPolicy()) // Optional backoff policy
        .listener(new MyRetryListener()) // Custom retry listener (optional)
        .build();
}

Where,

1. chunk(10): This defines the chunk size. It specifies how many items should be read, processed, and written in each transaction. You can adjust this value based on your requirements.
2. faultTolerant(): This method enables fault tolerance for the step, allowing you to configure retry and skip logic.
3. retryLimit(3): This sets the maximum number of retry attempts for each item in the chunk. In this example, it’s set to 3, meaning each item will be retried up to 3 times if an exception occurs.
4. retry(Exception.class): This specifies the type of exception(s) that should trigger a retry. In this case, it’s set to retry on any exception of type Exception. You can customize this to retry on specific exceptions relevant to your use case.
5. backOffPolicy(): Optionally, you can configure a backoff policy to introduce delays between retry attempts. The ExponentialBackOffPolicy is one of the available policies.
6. listener(): You can attach a custom retry listener to the step if you want to perform actions or logging during retries. This is optional but can be useful for monitoring and custom handling.

3. Implement Custom Retry Listeners (Optional)

If you attached a custom retry listener using listener(), you need to implement the listener as a Spring bean. A custom retry listener can be used to perform actions before and after each retry attempt. For example, you can log retry attempts or customize the behaviour:

Java

public class MyRetryListener implements RetryListener {
    @Override
    public <T, S> boolean open(RetryContext context, RetryCallback<T, S> callback) {
        // Code to run before a retry attempt
        return true;
    }

    @Override
    public <T, S> void close(RetryContext context, RetryCallback<T, S> callback, Throwable throwable) {
        // Code to run after a retry attempt, regardless of success or failure
    }

    @Override
    public <T, S> void onError(RetryContext context, RetryCallback<T, S> callback, Throwable throwable) {
        // Code to run when a retryable exception occurs
    }
}

4. Handle Exceptions in ItemProcessor (Optional)

If you want to apply retry logic specifically to exceptions thrown by your ItemProcessor, make sure your ItemProcessor is designed to throw these exceptions when necessary. Spring Batch will then catch these exceptions and apply the retry logic as configured.

5. Run Your Batch Job

With retry logic configured, you can now run your Spring Batch job. When exceptions occur during item processing, Spring Batch will automatically retry items based on your configuration. If the maximum number of retries is reached or if the retry logic determines that no more retries should be attempted, Spring Batch will proceed to the next item or take appropriate action based on your configuration.

Skip Logic

When you want to skip problematic items after having reached some predefined number of retrial failures, this is done with the help of skip logic. These products keep failing after retry attempts, hence it is right for these services. Spring Batch does not terminate the whole process but overlooks the defected elements and continues with the rest. Below steps can be followed To implement skip logic in Spring Batch,

• Configuration: Configure the skip policy to specify which exceptions should trigger a skip. You can use the skip method on the step builder.
• SkipListeners: Similar to retry listeners, you can add skip listeners to customize skip behaviour. For example, you can log skipped items or take specific actions when an item is skipped.

Below source code explains how to configure skip logic in spring batch:

Java

@Bean
public Step myStep() {
    return stepBuilderFactory.get("myStep")
        .<Input, Output>chunk(10)
        .reader(reader())
        .processor(processor())
        .writer(writer())
        .faultTolerant()
        .skipLimit(10) // Maximum number of items to skip
        .skip(Exception.class) // Skip on specific exceptions (customize as needed)
        .listener(new MySkipListener()) // Custom skip listener
        .build();
}

/*the skipLimit specifies the maximum number of items to skip.
After reaching this limit, the job will fail if further items encounter exceptions.
*/

Comparing Retry and Skip Logic in Spring Batch

Aspect	Retry logic	Skip Logic
Objective	Retry the same operation multiple times to recover from transient errors.	Skip problematic items and continue processing with the next item.
Configuration	It is Configured at the step level using .retry() and .retryLimit() methods.	It is Configured at the step level using .skip() and .skipLimit() methods.
Exception Trigger	Retries are triggered when an exception of specified type(s) occurs during item processing.	Skips are triggered when the same item fails to process multiple times or when a specific exception(s) occurs.
Number of Attempts	Retry logic allows you to specify the maximum number of retry attempts per item (e.g., .retryLimit(3)).	Skip logic allows you to specify the maximum number of items to skip (e.g., .skipLimit(10)).
Back-off Policy	You can configure a backoff policy to introduce delays between retry attempts. (e.g., .backOffPolicy(new ExponentialBackOffPolicy())) .	Back off policies are not used in skip logic, as skipping items does not involve retrying them.
Handling Logic	Each retry attempts results in a retry logic processing of the same item . The item fails if the maximum no. of retry is achieved.	After a certain number of retries, skip logic skips problematic items. An item that gets past the skip limit is not retried and is deemed a failure.
Typical Use Cases	For transient errors anticipated to resolve upon retry like network timeouts.	Applicable in handling consistently problematic things that are not retry-able. This stops job failure because of a handful defects.