Sunday, March 13, 2016

How SpringBoot AutoConfiguration magic works?

In my previous post Why SpringBoot? we have looked at how to create a SpringBoot application. But you may or may not understand what is going on behind the scenes. You may want to understand the magic behind the SpringBoot’s AutoConfiguration.

But before that you should know about Spring’s @Conditional feature based on which all the SpringBoot’s AutoConfiguration magic depends.

Exploring the power of @Conditional 
While developing Spring based applications we may come across of a need to register beans conditionally.

For example, you may want to register a DataSource bean pointing to DEV Database while running application locally and point to a different PRODUCTION Database while running in production. 

You can externalize the database connection parameters into properties files and use the file appropriate for the environment. But you need to change the configuration whenever you need to point to a different environment and build the application.

To address this problem Spring 3.1 introduced the concept of Profiles. You can register multiple beans of the same type and associate them to one or more profiles. When you run the application you can activate the desired profiles and beans associated with the activated profiles only will be registered.

@Configuration
public class AppConfig
{
    @Bean
    @Profile("DEV")
    public DataSource devDataSource() {
        ...
    }

    @Bean
    @Profile("PROD")
    public DataSource prodDataSource() {
        ...
    }
}

Then you can specify the active profile using System Property -Dspring.profiles.active=DEV

This approach works for simple cases like enable or disable bean registrations based on activated profiles. But if you want to register beans based on some conditional logic then the Profiles approach itself is not sufficient.

To provide much more flexibility for registering Spring beans conditionally, Spring 4 introduced the concept of @Conditional. By using @Conditional approach you can register a bean conditionally based on any arbitrary condition.

For example, you may want to register a bean when:

  • A specific class is present in classpath 
  • A Spring bean of certain type doesn’t already registered in ApplicationContext 
  • A specific file exists on a location 
  • A specific property value is configured in a configuration file 
  • A specific system property is present/absent 


These are just a few examples only and you can have any condition you want.

Let us take a look at how Spring’s @Conditional works.

Suppose we have a UserDAO interface with methods to get data from a data store. We have two implements of UserDAO interface namely JdbcUserDAO which talks to MySQL database and MongoUserDAO which talks to MongoDB.

We may want to enable only one of JdbcUserDAO and MongoUserDAO based on a System Property say dbType.

If the application is started using java -jar myapp.jar -DdbType=MySQL then we want to enable JdbcUserDAO, otherwise if the application is started using java -jar myapp.jar -DdbType=MONGO we want to enable MongoUserDAO.

Suppose we have UserDAO interface and JdbcUserDAO, MongoUserDAO implementations as follows:

public interface UserDAO
{
    List<String> getAllUserNames();
}

public class JdbcUserDAO implements UserDAO
{
    @Override
    public List<String> getAllUserNames()
    {
        System.out.println("**** Getting usernames from RDBMS *****");
        return Arrays.asList("Siva","Prasad","Reddy");
    }
}

public class MongoUserDAO implements UserDAO
{
    @Override
    public List<String> getAllUserNames()
    {
        System.out.println("**** Getting usernames from MongoDB *****");
        return Arrays.asList("Bond","James","Bond");
    }
}

We can implement the Condition MySQLDatabaseTypeCondition to check whether the System Property dbType is "MYSQL" as follows:

public class MySQLDatabaseTypeCondition implements Condition
{
    @Override
    public boolean matches(ConditionContext conditionContext, AnnotatedTypeMetadata metadata)
    {
        String enabledDBType = System.getProperty("dbType");
        return (enabledDBType != null && enabledDBType.equalsIgnoreCase("MYSQL"));
    }
}

We can implement the Condition MongoDBDatabaseTypeCondition to check whether the System Property dbType is "MONGODB" as follows:

public class MongoDBDatabaseTypeCondition implements Condition
{
    @Override
    public boolean matches(ConditionContext conditionContext, AnnotatedTypeMetadata metadata)
    {
        String enabledDBType = System.getProperty("dbType");
        return (enabledDBType != null && enabledDBType.equalsIgnoreCase("MONGODB"));
    }
}

Now we can configure both JdbcUserDAO and MongoUserDAO beans conditionally using @Conditional as follows:

@Configuration
public class AppConfig
{
    @Bean
    @Conditional(MySQLDatabaseTypeCondition.class)
    public UserDAO jdbcUserDAO(){
        return new JdbcUserDAO();
    }

    @Bean
    @Conditional(MongoDBDatabaseTypeCondition.class)
    public UserDAO mongoUserDAO(){
        return new MongoUserDAO();
    }
}

If we run the application like java -jar myapp.jar -DdbType=MYSQL then only JdbcUserDAO bean will be registered. But if you set the System property like -DdbType=MONGODB then only MongoUserDAO bean will be registered.

Now that we have seen how to conditionally register a bean based on System Property.

Suppose we want to register MongoUserDAO bean only when MongoDB java driver class "com.mongodb.Server" is available on classpath, if not we want to register JdbcUserDAO bean. 

To accomplish that we can create Conditions to check the presence or absence of MongoDB driver class "com.mongodb.Server" as follows:

public class MongoDriverPresentsCondition implements Condition
{
    @Override
    public boolean matches(ConditionContext conditionContext,AnnotatedTypeMetadata metadata)
    {
        try {
            Class.forName("com.mongodb.Server");
            return true;
        } catch (ClassNotFoundException e) {
            return false;
        }
    }
}

public class MongoDriverNotPresentsCondition implements Condition
{
    @Override
    public boolean matches(ConditionContext conditionContext, AnnotatedTypeMetadata metadata)
    {
        try {
            Class.forName("com.mongodb.Server");
            return false;
        } catch (ClassNotFoundException e) {
            return true;
        }
    }
}

We just have seen how to register beans conditionally based on presence/absence of a class in classpath.

What if we want to register MongoUserDAO bean only if no other Spring bean of type UserDAO is already registered.

We can create a Condition to check if there is any existing bean of a certain type as follows:

public class UserDAOBeanNotPresentsCondition implements Condition
{
    @Override
    public boolean matches(ConditionContext conditionContext, AnnotatedTypeMetadata metadata)
    {
        UserDAO userDAO = conditionContext.getBeanFactory().getBean(UserDAO.class);
        return (userDAO == null);
    }
}

What if we want to register MongoUserDAO bean only if property app.dbType=MONGO is set in properties placeholder configuration file?

We can implement that Condition as follows:

public class MongoDbTypePropertyCondition implements Condition
{
    @Override
    public boolean matches(ConditionContext conditionContext,
    AnnotatedTypeMetadata metadata)
    {
        String dbType = conditionContext.getEnvironment()
                            .getProperty("app.dbType");
        return "MONGO".equalsIgnoreCase(dbType);
    }
}

We have just seen how to implement various types of Conditions. But there is even more elegant way to implement Conditions using Annotations. Instead of creating a Condition implementation for both MYSQL and MongoDB, we can create aDatabaseType annotation as follows:

@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Conditional(DatabaseTypeCondition.class)
public @interface DatabaseType
{
    String value();
}

Then we can implement DatabaseTypeCondition to use the DatabaseType value to determine whether to enable or disable bean registration as follows:

public class DatabaseTypeCondition implements Condition
{
    @Override
    public boolean matches(ConditionContext conditionContext,
    AnnotatedTypeMetadata metadata)
    {
        Map<String, Object> attributes = metadata.getAnnotationAttributes(DatabaseType.class.getName());
        String type = (String) attributes.get("value");
        String enabledDBType = System.getProperty("dbType","MYSQL");
        return (enabledDBType != null && type != null && enabledDBType.equalsIgnoreCase(type));
    }
}

Now we can use the @DatabaseType annotation on our bean definitions as follows:

@Configuration
@ComponentScan
public class AppConfig
{
    @DatabaseType("MYSQL")
    public UserDAO jdbcUserDAO(){
        return new JdbcUserDAO();
    }

    @Bean
    @DatabaseType("MONGO")
    public UserDAO mongoUserDAO(){
        return new MongoUserDAO();
    }
}

Here we are getting the metadata from DatabaseType annotation and checking against the System Property dbType value to determine whether to enable or disable the bean registration.

We have seen good number of examples to understand how we can register beans conditionally using @Conditional annotation.

SpringBoot extensively uses @Conditional feature to register beans conditionally based on various criteria.

You can find various Condition implementations that SpringBoot uses in org.springframework.boot.autoconfigure package of spring-boot-autoconfigure-{version}.jar. 

Now that we come to know about how SpringBoot uses @Conditional feature to conditionally check whether to register a bean or not. But what exactly triggers the auto-configuration mechanism?

This is what we are going to look at in next section.

SpringBoot AutoConfiguration 
The key to the SpringBoot’s auto-configuration magic is @EnableAutoConfiguration annotation. Typically we annotate our Application entry point class with either @SpringBootApplication or if we want to customize the defaults we can use the following annotations:

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class Application
{

}

The @EnableAutoConfiguration annotation enables the auto-configuration of Spring ApplicationContext by scanning the classpath components and registers the beans that are matching various Conditions. 

SpringBoot provides various AutoConfiguration classes in spring-boot-autoconfigure-{version}.jar which are responsible for registering various components.

Typically AutoConfiguration classes are annotated with @Configuration to mark it as a Spring configuration class and annotated with @EnableConfigurationProperties to bind the customization properties and one or more Conditional bean registration methods.

For example consider org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration class.

@Configuration
@ConditionalOnClass({ DataSource.class, EmbeddedDatabaseType.class })
@EnableConfigurationProperties(DataSourceProperties.class)
@Import({ Registrar.class, DataSourcePoolMetadataProvidersConfiguration.class })
public class DataSourceAutoConfiguration 
{
    ...
    ...
    @Conditional(DataSourceAutoConfiguration.EmbeddedDataSourceCondition.class)
    @ConditionalOnMissingBean({ DataSource.class, XADataSource.class })
    @Import(EmbeddedDataSourceConfiguration.class)
    protected static class EmbeddedConfiguration {

    }

    @Configuration
    @ConditionalOnMissingBean(DataSourceInitializer.class)
    protected static class DataSourceInitializerConfiguration {
        @Bean
        public DataSourceInitializer dataSourceInitializer() {
        return new DataSourceInitializer();
        }
    }

    @Conditional(DataSourceAutoConfiguration.NonEmbeddedDataSourceCondition.class)
    @ConditionalOnMissingBean({ DataSource.class, XADataSource.class })
    protected static class NonEmbeddedConfiguration {
        @Autowired
        private DataSourceProperties properties;

        @Bean
        @ConfigurationProperties(prefix = DataSourceProperties.PREFIX)
        public DataSource dataSource() {
            DataSourceBuilder factory = DataSourceBuilder
                    .create(this.properties.getClassLoader())
                    .driverClassName(this.properties.getDriverClassName())
                    .url(this.properties.getUrl()).username(this.properties.getUsername())
                    .password(this.properties.getPassword());
            if (this.properties.getType() != null) {
                factory.type(this.properties.getType());
            }
            return factory.build();
        }
    }
    ...
    ...
    @Configuration
    @ConditionalOnProperty(prefix = "spring.datasource", name = "jmx-enabled")
    @ConditionalOnClass(name = "org.apache.tomcat.jdbc.pool.DataSourceProxy")
    @Conditional(DataSourceAutoConfiguration.DataSourceAvailableCondition.class)
    @ConditionalOnMissingBean(name = "dataSourceMBean")
    protected static class TomcatDataSourceJmxConfiguration {
        @Bean
        public Object dataSourceMBean(DataSource dataSource) {
        ....
        ....
        }
    }
    ...
    ...
}

Here DataSourceAutoConfiguration is annotated with @ConditionalOnClass({ DataSource.class,EmbeddedDatabaseType.class }) which means that Auto Configuration of beans within DataSourceAutoConfiguration will be considered only if DataSource.class and EmbeddedDatabaseType.class classes are available on classpath.

The class is also annotated with @EnableConfigurationProperties(DataSourceProperties.class) which enables binding the properties in application.properties to the properties of DataSourceProperties class automatically.

@ConfigurationProperties(prefix = DataSourceProperties.PREFIX)
public class DataSourceProperties implements BeanClassLoaderAware, EnvironmentAware, InitializingBean {

    public static final String PREFIX = "spring.datasource";
    ...
    ...
    private String driverClassName;
    private String url;
    private String username;
    private String password;
    ...
    //setters and getters
}

With this configuration all the properties that starts with spring.datasource.* will be automatically binds to DataSourceProperties object.

spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.username=root
spring.datasource.password=secret
spring.datasource.driver-class-name=com.mysql.jdbc.Driver

You can also see some inner classes and bean definition methods that are annotated with SpringBoot’s Conditional annotations such as @ConditionalOnMissingBean, @ConditionalOnClass and @ConditionalOnProperty etc.

These bean definitions will be registered in ApplicationContext only if those conditions are matched.

You can also explore many other AutoConfiguration classes in spring-boot-autoconfigure-{version}.jar such as

  • org.springframework.boot.autoconfigure.web.DispatcherServletAutoConfiguration 
  • org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration 
  • org.springframework.boot.autoconfigure.data.jpa.JpaRepositoriesAutoConfiguration 
  • org.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration etc etc. 


I hope now you have an understanding of how SpringBoot auto-configuration works by using various AutoConfiration classes along with @Conditional features.

If you want to learn more about SpringBoot take a look at my 

SpringBoot : Learn By Example book


Why SpringBoot?

Spring is a very popular Java based framework for building web and enterprise applications. Unlike many other frameworks which focuses on only one area, Spring framework provides a wide verity of features addressing the modern business needs via its portfolio projects.

Spring framework provides flexibility to configure the beans in multiple ways such as XML, Annotations and JavaConfig. With the number of features increased the complexity also gets increased and configuring Spring applications becomes tedious and error-prone.

Spring team created SpringBoot to address the complexity of configuration. But before diving into SpringBoot, we will take a quick look at Spring framework and see what kind of problems SpringBoot is trying to address.

In this article we will cover: 
  • Overview of Spring framework 
  • A web application using Spring MVC and JPA(Hibernate) 
  • A quick taste of SpringBoot 
Overview of Spring framework 
If you are a Java developer then there is a high chance that you might have heard about Spring framework and probably have used it in your projects. Spring framework was created primarily as a Dependency Injection container but it is much more than that.

Spring is very popular because of several reasons: 

  • Spring’s dependency injection approach encourages writing testable code 
  • Easy to use but powerful database transaction management capabilities 
  • Spring simplifies integration with other Java frameworks like JPA/Hibernate ORM, Struts/JSF etc web frameworks 
  • State of the art Web MVC framework for building web applications 


Along with Spring framework there are many other Spring sister projects which helps to build applications addressing modern business needs:

  • Spring Data: Simplifies data access from relational and NoSQL data stores. 
  • Spring Batch: Provides powerful batch processing framework. 
  • Spring Security: Robust security framework to secure applications. 
  • Spring Social: Supports integration with social networking sites like Facebook, Twitter, LinkedIn, GitHub etc. 
  • Spring Integration: An implementation of Enterprise Integration Patterns to facilitate integration with other enterprise applications using lightweight messaging and declarative adapters. 


There are many other interesting projects addressing various other modern application development needs. For more information take a look at http://spring.io/projects.

In the initial days, Spring framework provides XML based approach for configuring beans. Later Spring introduced XML based DSLs, Annotations and JavaConfig based approaches for configuring beans.

Let us take a quick look at how each of those configuration styles looks like.

XML based configuration

<bean id="userService" class="com.sivalabs.myapp.service.UserService">
    <property name="userDao" ref="userDao"/>
</bean>

<bean id="userDao" class="com.sivalabs.myapp.dao.JdbcUserDao">
    <property name="dataSource" ref="dataSource"/>
</bean>

<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url" value="jdbc:mysql://localhost:3306/test"/>
    <property name="username" value="root"/>
    <property name="password" value="secret"/>
</bean>

Annotation based configuration

@Service
public class UserService
{
    private UserDao userDao;

    @Autowired
    public UserService(UserDao dao){
        this.userDao = dao;
    }
    ...
    ...
}
@Repository
public class JdbcUserDao
{
    private DataSource dataSource;

    @Autowired
    public JdbcUserDao(DataSource dataSource){
        this.dataSource = dataSource;
    }
    ...
    ...
}

JavaConfig based configuration

@Configuration
public class AppConfig
{
    @Bean
    public UserService userService(UserDao dao){
        return new UserService(dao);
    }

    @Bean
    public UserDao userDao(DataSource dataSource){
        return new JdbcUserDao(dataSource);
    }

    @Bean
    public DataSource dataSource(){
        BasicDataSource dataSource = new BasicDataSource();
        dataSource.setDriverClassName("com.mysql.jdbc.Driver");
        dataSource.setUrl("jdbc:mysql://localhost:3306/test");
        dataSource.setUsername("root");
        dataSource.setPassword("secret");
        return dataSource;
    }
}

Wow… Spring provides many approaches for doing the same thing and we can even mix the approaches as well like you can use both JavaConfig and Annotation based configuration styles in the same application.

That is a lot of flexibility and it is one way good and one way bad. People new to Spring framework may gets confused about which approach to follow. As of now the Spring team is suggesting to follow JavaConfig based approach as it gives more flexibility. 

But there is no one-size fits all kind of solution. One has to choose the approach based on their own application needs.

OK, now that you had a glimpse of how various styles of Spring bean configurations looks like.

Let us take a quick look at the configuration of a typical SpringMVC + JPA/Hibernate web application configuration looks like.

A web application using Spring MVC and JPA(Hibernate) 

Before getting to know what is SpringBoot and what kind of features it provides, let us take a look at how a typical Spring web application configuration looks like, what are the pain points and then we will discuss about how SpringBoot addresses those problems.

Step 1: Configure Maven Dependencies 

First thing we need to do is configure all the dependencies required in our pom.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
                        http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sivalabs</groupId>
    <artifactId>springmvc-jpa-demo</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>springmvc-jpa-demo</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>      
        <failOnMissingWebXml>false</failOnMissingWebXml>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>4.2.4.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-jpa</artifactId>
            <version>1.9.2.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jcl-over-slf4j</artifactId>
            <version>1.7.13</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.13</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.13</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
        </dependency>
        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.190</version>
        </dependency>
        <dependency>
            <groupId>commons-dbcp</groupId>
            <artifactId>commons-dbcp</artifactId>
            <version>1.4</version>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.38</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-entitymanager</artifactId>
            <version>4.3.11.Final</version>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>3.1.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.thymeleaf</groupId>
            <artifactId>thymeleaf-spring4</artifactId>
            <version>2.1.4.RELEASE</version>
        </dependency>
    </dependencies>
</project>

We have configured all our Maven jar dependencies to include Spring MVC, Spring Data JPA, JPA/Hibernate, Thymeleaf and Log4j.

Step 2: Configure Service/DAO layer beans using JavaConfig.

@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(basePackages="com.sivalabs.demo")
@PropertySource(value = { "classpath:application.properties" })
public class AppConfig 
{
    @Autowired
    private Environment env;

    @Bean
    public static PropertySourcesPlaceholderConfigurer placeHolderConfigurer()
    {
        return new PropertySourcesPlaceholderConfigurer();
    }
    
    @Value("${init-db:false}")
    private String initDatabase;
    
    @Bean
    public PlatformTransactionManager transactionManager()
    {
        EntityManagerFactory factory = entityManagerFactory().getObject();
        return new JpaTransactionManager(factory);
    }

    @Bean
    public LocalContainerEntityManagerFactoryBean entityManagerFactory()
    {
        LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();

        HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
        vendorAdapter.setGenerateDdl(Boolean.TRUE);
        vendorAdapter.setShowSql(Boolean.TRUE);

        factory.setDataSource(dataSource());
        factory.setJpaVendorAdapter(vendorAdapter);
        factory.setPackagesToScan("com.sivalabs.demo");

        Properties jpaProperties = new Properties();
        jpaProperties.put("hibernate.hbm2ddl.auto", env.getProperty("hibernate.hbm2ddl.auto"));
        factory.setJpaProperties(jpaProperties);

        factory.afterPropertiesSet();
        factory.setLoadTimeWeaver(new InstrumentationLoadTimeWeaver());
        return factory;
    }

    @Bean
    public HibernateExceptionTranslator hibernateExceptionTranslator()
    {
        return new HibernateExceptionTranslator();
    }
    
    @Bean
    public DataSource dataSource()
    {
        BasicDataSource dataSource = new BasicDataSource();
        dataSource.setDriverClassName(env.getProperty("jdbc.driverClassName"));
        dataSource.setUrl(env.getProperty("jdbc.url"));
        dataSource.setUsername(env.getProperty("jdbc.username"));
        dataSource.setPassword(env.getProperty("jdbc.password"));
        return dataSource;
    }
    
    @Bean
    public DataSourceInitializer dataSourceInitializer(DataSource dataSource) 
    {
        DataSourceInitializer dataSourceInitializer = new DataSourceInitializer();
        dataSourceInitializer.setDataSource(dataSource);
        ResourceDatabasePopulator databasePopulator = new ResourceDatabasePopulator();
        databasePopulator.addScript(new ClassPathResource("data.sql"));
        dataSourceInitializer.setDatabasePopulator(databasePopulator);
        dataSourceInitializer.setEnabled(Boolean.parseBoolean(initDatabase));
        return dataSourceInitializer;
    }   
}

In our AppConfig.java configuration class we have done the following:

  • Marked it as as a Spring Configuration class using @Configuration annotation.
  • Enabled Annotation based transaction management using @EnableTransactionManagement 
  • Configured @EnableJpaRepositories to indicate where to look for Spring Data JPA repositories 
  • Configured PropertyPlaceHolder bean using @PropertySource annotation and PropertySourcesPlaceholderConfigurer bean definition which loads properties from application.properties file. 
  • Defined beans for DataSource, JPA EntityManagerFactory, JpaTransactionManager. 
  • Configured DataSourceInitializer bean to initialize the database by executing data.sql script on application start up. 

we need to configure property placeholder values in application.properties as follows:

jdbc.driverClassName=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://localhost:3306/test
jdbc.username=root
jdbc.password=admin
init-db=true
hibernate.dialect=org.hibernate.dialect.MySQLDialect
hibernate.show_sql=true
hibernate.hbm2ddl.auto=update

we can create a simple sql script data.sql to populate sample data into USER table.

delete from user;
insert into user(id, name) values(1,'Siva');
insert into user(id, name) values(2,'Prasad');
insert into user(id, name) values(3,'Reddy');

We can create log4j.properties file with basic configuration as follows:

log4j.rootCategory=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p %t %c{2}:%L - %m%n

log4j.category.org.springframework=INFO
log4j.category.com.sivalabs=DEBUG

Step 3: Configure Spring MVC web layer beans 

We will have to configure Thymeleaf ViewResolver, static ResourceHandlers, MessageSource for i18n etc.

@Configuration
@ComponentScan(basePackages = { "com.sivalabs.demo"}) 
@EnableWebMvc
public class WebMvcConfig extends WebMvcConfigurerAdapter
{
    @Bean
    public TemplateResolver templateResolver() {
        TemplateResolver templateResolver = new ServletContextTemplateResolver();
        templateResolver.setPrefix("/WEB-INF/views/");
        templateResolver.setSuffix(".html");
        templateResolver.setTemplateMode("HTML5");
        templateResolver.setCacheable(false);
        return templateResolver;
    }

    @Bean
    public SpringTemplateEngine templateEngine() {
        SpringTemplateEngine templateEngine = new SpringTemplateEngine();
        templateEngine.setTemplateResolver(templateResolver());
        return templateEngine;
    }

    @Bean
    public ThymeleafViewResolver viewResolver() {
        ThymeleafViewResolver thymeleafViewResolver = new ThymeleafViewResolver();
        thymeleafViewResolver.setTemplateEngine(templateEngine());
        thymeleafViewResolver.setCharacterEncoding("UTF-8");
        return thymeleafViewResolver;
    }

    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry)
    {
        registry.addResourceHandler("/resources/**").addResourceLocations("/resources/");
    }

    @Override
    public void configureDefaultServletHandling(DefaultServletHandlerConfigurer configurer)
    {
        configurer.enable();
    }

    @Bean(name = "messageSource")
    public MessageSource configureMessageSource()
    {
        ReloadableResourceBundleMessageSource messageSource = new ReloadableResourceBundleMessageSource();
        messageSource.setBasename("classpath:messages");
        messageSource.setCacheSeconds(5);
        messageSource.setDefaultEncoding("UTF-8");
        return messageSource;
    }
}

In our WebMvcConfig.java configuration class we have done the following:
  • Marked it as as a Spring Configuration class using @Configuration annotation. 
  • Enabled Annotation based Spring MVC configuration using @EnableWebMvc 
  • Configured Thymeleaf ViewResolver by registering TemplateResolver, SpringTemplateEngine, ThymeleafViewResolver beans. 
  • Registered ResourceHandlers bean to indicate requests for static resources with URI /resources/** will be served from the location /resources/ directory. 
  • Configured MessageSource bean to load i18n messages from ResourceBundle messages-{country-code}.properties from classpath. 
For now we do not have any messages to be configured, so create an empty messages.properties file in src/main/resources folder.

Step 4: Register Spring MVC FrontController servlet DispatcherServlet. 

Prior to Servlet 3.x specification we have to register Servlets/Filters in web.xml. Since Servlet 3.x specification we can register Servlets/Filters programatically using ServletContainerInitializer.

Spring MVC provides a convenient class AbstractAnnotationConfigDispatcherServletInitializer to register DispatcherServlet.

public class SpringWebAppInitializer extends AbstractAnnotationConfigDispatcherServletInitializer
{

    @Override
    protected Class<?>[] getRootConfigClasses()
    {
        return new Class<?>[] { AppConfig.class};
    }

    @Override
    protected Class<?>[] getServletConfigClasses()
    {
        return new Class<?>[] { WebMvcConfig.class };
    }

    @Override
    protected String[] getServletMappings()
    {
        return new String[] { "/" };
    }

    @Override
    protected Filter[] getServletFilters() {
       return new Filter[]{ new OpenEntityManagerInViewFilter() };
    }
}

In our SpringWebAppInitializer.java configuration class we have done the following:

  • We have configured AppConfig.class as RootConfirationClasses which will become the parent ApplicationContext that contains bean definitions shared by all child (DispatcherServlet) contexts. 
  • We have configured WebMvcConfig.class as ServletConfigClasses which is child ApplicationContext that contains WebMvc bean definitions. 
  • We have configured ”/” as ServletMapping means all the requests will be handled by DispatcherServlet. 
  • We have registered OpenEntityManagerInViewFilter as a Servlet Filter so that we can lazy load the JPA Entity lazy collections while rendering the view. 

Step 5: Create a JPA Entity and Spring Data JPA Repository 

Create a JPA entity User.java and a Spring Data JPA repository for User entity.

@Entity
public class User
{
    @Id @GeneratedValue(strategy=GenerationType.AUTO)
    private Integer id;
    private String name;

    //setters and getters
}
public interface UserRepository extends JpaRepository<User, Integer>
{
}

Step 6: Create a SpringMVC Controller 

Create a SpringMVC controller to handle URL “/” and render list of users.

@Controller
public class HomeController
{
    @Autowired UserRepository userRepo;

    @RequestMapping("/")
    public String home(Model model)
    {
        model.addAttribute("users", userRepo.findAll());
        return "index";
    }
}

Step 7: Create a thymeleaf view /WEB-INF/views/index.html to render list of Users.

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" 
      xmlns:th="http://www.thymeleaf.org">
<head>
<meta charset="utf-8"/>
<title>Home</title>
</head>
<body>
    <table>
        <thead>
            <tr>
                <th>Id</th>
                <th>Name</th>
            </tr>
        </thead>
        <tbody>
            <tr th:each="user : ${users}">
                <td th:text="${user.id}">Id</td>
                <td th:text="${user.name}">Name</td>
            </tr>
        </tbody>
    </table>
</body>
</html>

We are all set now to run the application. But before that we need to download and configure the server like Tomcat or Jetty or Wildfly etc in your IDE.

You can download Tomcat 8 and configure in your favourite IDE,run the application and point your browser to http://localhost:8080/springmvcjpa-demo. You should see the list of users details in a table. Yay…We did it.

But wait..Isn’t it too much work to just show a list of user details pulled from a database table? Let us be honest and fair. All this configuration is not just for this one use-case. This configuration is basis for rest of the application also. 

But again, this is too much of work to do if you want to quickly get up and running. Another problem with it is, assume you want to develop another SpringMVC application with similar technical stack? 

Well, you copy-paste the configuration and tweak it. Right? But remember one thing: if you have to do the same thing again and again, you should find an automated way to do it.

Apart from writing the same configuration again and again, do you see any other problems here? 

Well, let me list our what are the problems I am seeing here. 
  • You need to hunt for all the compatible libraries for the specific Spring version and configure them. 
  • 95% of the times we configure the DataSource, EntitymanagerFactory, TransactionManager etc beans in the same way. Wouldn’t it be great if Spring can do it for me automatically. 
  • Similarly we configure SpringMVC beans like ViewResolver, MessageSource etc in the same way most of the times. 
If Spring can automatically do it for me that would be awesome!!!. 

Imagine, what if Spring is capable of configuring beans automatically? What if you can customize the automatic configuration using simple customizable properties?

For example, instead of mapping DispatcherServlet url-pattern to “/” you want to map it to “/app/”. Instead of putting thymeleaf views in “/WEB-INF/views” folder you may want to place them in “/WEB-INF/templates/” folder.

So basically you want Spring to do things automatically but provide the flexibility to override the default configuration in a simpler way?

Well, you are about to enter into the world of SpringBoot where your dreams come true!!!

A quick taste of SpringBoot 
Welcome to SpringBoot!. SpringBoot do what exactly you are looking for. It will do things automatically for you but allows you to override the defaults if you want to.

Instead of explaining in theory I prefer to explain by example.

So let us implement the same application that we built earlier but this time using SpringBoot.

Step 1: Create a Maven based SpringBoot Project 

Create a Maven project and configure the dependencies as follows:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
                        http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sivalabs</groupId>
    <artifactId>hello-springboot</artifactId>
    <packaging>jar</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>hello-springboot</name>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.3.2.RELEASE</version>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>1.8</java.version>
    </properties>

    
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-thymeleaf</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>
    </dependencies>
</project>

Wow our pom.xml suddenly become so small!!.

Step 2: Configure datasource/JPA properties in application.properties as follows.

spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.username=root
spring.datasource.password=admin
spring.datasource.initialize=true
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true

you can copy the same data.sql file into src/main/resources folder.

Step 3: Create a JPA Entity and Spring Data JPA Repository Interface for the entity. 

Create User.java, UserRepository.java and HomeController.java same as in springmvc-jpa-demo application.

Step 4: Create Thymeleaf view to show list of users 

Copy /WEB-INF/views/index.html that we created in springmvc-jpa-demo application into src/-main/resources/templates folder in our new project.

Step 5: Create SpringBoot EntryPoint Class. 

Create a Java class Application.java with main method as follows:

@SpringBootApplication
public class Application
{
    public static void main(String[] args)
    {
        SpringApplication.run(Application.class, args);
    }
}

Now run Application.java as a Java Application and point your browser to http://localhost:8080/. You should see the list of users in table format. Coool!!!

Ok ok, I hear you are shouting “What is going on???”.

Let me explain what just happened.

1. Easy dependency Management 
  • First thing to observe is we are using some dependencies named like spring-boot-starter-*.
    Remember I said “95% of the times I use same configuration”. So when you add springboot-starter-web dependency by default it will pull all the commonly used libraries while developing Spring MVC applications such as spring-webmvc, jackson-json, validation-api and tomcat.
  • We have added spring-boot-starter-data-jpa dependency. This pulls all the spring-data-jpa dependencies and also adds Hibernate libraries because majority of the applications use Hibernate as JPA implementation. 
2. Auto Configuration 
  • Not only the spring-boot-starter-web adds all these libraries but also configures the commonly registered beans like DispatcherServlet, ResourceHandlers, MessageSource etc beans with sensible defaults.
  • We also added spring-boot-starter-thymeleaf which not only adds the thymeleaf library dependencies but also configures ThymeleafViewResolver beans as well automatically.
  • We haven’t defined any of the DataSource, EntityManagerFactory, TransactionManager etc beans but they are automatically gets created. How?

    If we have any in-memory database drivers like H2 or HSQL in our classpath then SpringBoot will automatically creates an in-memory DataSource and then registers EntityManagerFactory, TransactionManager beans automatically with sensible defaults.

    But we are using MySQL, so we need to explicitly provide MySQL connection details. We have configured those MySQL connection details in application.properties file and SpringBoot creates a DataSource using these properties. 
3. Embedded Servlet Container Support

The most important and surprising thing is we have created a simple Java class annotated with some magical annotation @SpringApplication having a main method and by running that main we are able to run the application and access it at http://localhost:8080/.

Where is the servlet container comes from? 
We have added spring-boot-starter-web which pull the spring-boot-starter-tomcat automatically and when we run the main() method it started tomcat as an embedded container so that we don’t have to deploy our application on any externally installed tomcat server.

By the way have you observe that our packaging type in pom.xml is ‘jar’ not ‘war’. Wonderful!

Ok, but what if I want to use Jetty server instead of tomcat? 
Simple, exclude spring-bootstarter-tomcat from spring-boot-starter-web and include spring-boot-starter-jetty.

That’s it. But, this looks all magical!!!

I can imagine what you are thinking. You are thinking like SpringBoot looks cool and it is doing lot of things automatically for me. But still I am not fully understand how it is all really working behind the scenes. Right?

I can understand. Watching magic show is fun normally, but not in Software Development. Don’t worry, we will be looking at each of those things and explain in-detail how things are happening behind the scenes in future articles. But I don’t want to overwhelm you by dumping everything on to your hear right now in this article.

Summary 

In this article we had a quick overview of various Spring configuration styles and understand the complexity of configuring Spring applications. Also, we had a quick look at SpringBoot by creating a simple web application. In next article we will deep dive into SpringBoot and understand how it works.

If you want to learn more about SpringBoot take a look at my 

SpringBoot : Learn By Example book


Tuesday, February 16, 2016

Creating Custom SpringBoot Starter for Twitter4j

SpringBoot provides lot of starter modules to get up and running quickly. SpringBoot’s auto-configure mechanism takes care of configuring SpringBeans on our behalf based on various criteria.

In addition to the springboot starters that comes out-of-the-box provided by Core Spring Team, we can also create our own starter modules.

In this post we will look into how to create a custom SpringBoot starter. To demonstrate it we are going to create twitter4j-spring-boot-starter which will auto-configure Twitter4J beans.

To accomplish this, we are going to create:
  1.  twitter4j-spring-boot-autoconfigure module which contains Twitter4J AutoConfiguration bean definitions 
  2. twitter4j-spring-boot-starter module which pulls in twitter4j-spring-boot-autoconfigure and twitter4j-core dependencies 
  3. Sample application which uses twitter4j-spring-boot-starter 

Create Parent Module spring-boot-starter-twitter4j

First we are going to create a parent pom type module to define dependency versions and sub-modules.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                        http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.sivalabs</groupId>
    <artifactId>spring-boot-starter-twitter4j</artifactId>
    <packaging>pom</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>spring-boot-starter-twitter4j</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <twitter4j.version>4.0.3</twitter4j.version>
        <spring-boot.version>1.3.2.RELEASE</spring-boot.version>
    </properties>

    <modules>
        <module>twitter4j-spring-boot-autoconfigure</module>
        <module>twitter4j-spring-boot-starter</module>
        <module>twitter4j-spring-boot-sample</module>
    </modules>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-dependencies</artifactId>
                <version>${spring-boot.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>

            <dependency>
                <groupId>org.twitter4j</groupId>
                <artifactId>twitter4j-core</artifactId>
                <version>${twitter4j.version}</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

</project>

In this pom.xml we are defining the SpringBoot and Twitter4j versions in section so that we don’t need to specify versions all over the places. 


Create twitter4j-spring-boot-autoconfigure module 

Create a child module with name twitter4j-spring-boot-autoconfigure in our parent maven module spring-boot-starter-twitter4j. 

Add the maven dependencies such as spring-boot, spring-boot-autoconfigure, twitter4j-core and junit as follows:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
               http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sivalabs</groupId>
    <artifactId>twitter4j-spring-boot-autoconfigure</artifactId>
    <packaging>jar</packaging>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>com.sivalabs</groupId>
        <artifactId>spring-boot-starter-twitter4j</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.twitter4j</groupId>
            <artifactId>twitter4j-core</artifactId>
            <optional>true</optional>
        </dependency>
    </dependencies>
</project>

Note that we have specified twitter4j-core as optional dependency because twitter4j-core should be added to the project only when twitter4j-spring-boot-starter is added to the project.

Create Twitter4jProperties to hold the Twitter4J config parameters 

Create Twitter4jProperties.java to hold the Twitter4J OAuth config parameters.

package com.sivalabs.spring.boot.autoconfigure;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.NestedConfigurationProperty;

@ConfigurationProperties(prefix= Twitter4jProperties.TWITTER4J_PREFIX)
public class Twitter4jProperties {

    public static final String TWITTER4J_PREFIX = "twitter4j";

    private Boolean debug = false;

    @NestedConfigurationProperty
    private OAuth oauth = new OAuth();

    public Boolean getDebug() {
        return debug;
    }

    public void setDebug(Boolean debug) {
        this.debug = debug;
    }

    public OAuth getOauth() {
        return oauth;
    }

    public void setOauth(OAuth oauth) {
        this.oauth = oauth;
    }

    public static class OAuth {

        private String consumerKey;
        private String consumerSecret;
        private String accessToken;
        private String accessTokenSecret;

        public String getConsumerKey() {
            return consumerKey;
        }
        public void setConsumerKey(String consumerKey) {
            this.consumerKey = consumerKey;
        }
        public String getConsumerSecret() {
            return consumerSecret;
        }
        public void setConsumerSecret(String consumerSecret) {
            this.consumerSecret = consumerSecret;
        }
        public String getAccessToken() {
            return accessToken;
        }
        public void setAccessToken(String accessToken) {
            this.accessToken = accessToken;
        }
        public String getAccessTokenSecret() {
            return accessTokenSecret;
        }
        public void setAccessTokenSecret(String accessTokenSecret) {
            this.accessTokenSecret = accessTokenSecret;
        }
    }
}

With this configuration object we can configure the twitter4j properties in application.properties as follows:
 
twitter4j.debug=true
twitter4j.oauth.consumer-key=your-consumer-key-here
twitter4j.oauth.consumer-secret=your-consumer-secret-here
twitter4j.oauth.access-token=your-access-token-here
twitter4j.oauth.access-token-secret=your-access-token-secret-here

Create Twitter4jAutoConfiguration to auto-configure Twitter4J 

Here comes the key part of our starter. Twitter4jAutoConfiguration configuration class contains the bean definitions that will be automatically configured based on some criteria. 

What is that criteria? 
  • If twitter4j.TwitterFactory.class is on classpath 
  • If TwitterFactory bean is not already defined explicitly 

So, the Twitter4jAutoConfiguration goes like this.

package com.sivalabs.spring.boot.autoconfigure;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import twitter4j.Twitter;
import twitter4j.TwitterFactory;
import twitter4j.conf.ConfigurationBuilder;

@Configuration
@ConditionalOnClass({ TwitterFactory.class, Twitter.class })
@EnableConfigurationProperties(Twitter4jProperties.class)
public class Twitter4jAutoConfiguration {

    private static Log log = LogFactory.getLog(Twitter4jAutoConfiguration.class);

    @Autowired
    private Twitter4jProperties properties;

    @Bean
    @ConditionalOnMissingBean
    public TwitterFactory twitterFactory(){

        if (this.properties.getOauth().getConsumerKey() == null
            || this.properties.getOauth().getConsumerSecret() == null
            || this.properties.getOauth().getAccessToken() == null
            || this.properties.getOauth().getAccessTokenSecret() == null)
        {
            String msg = "Twitter4j properties not configured properly." + 
                         " Please check twitter4j.* properties settings in configuration file.";
            log.error(msg);
            throw new RuntimeException(msg);
        }

        ConfigurationBuilder cb = new ConfigurationBuilder();
        cb.setDebugEnabled(properties.getDebug())
          .setOAuthConsumerKey(properties.getOauth().getConsumerKey())
          .setOAuthConsumerSecret(properties.getOauth().getConsumerSecret())
          .setOAuthAccessToken(properties.getOauth().getAccessToken())
          .setOAuthAccessTokenSecret(properties.getOauth().getAccessTokenSecret());
        TwitterFactory tf = new TwitterFactory(cb.build());
        return tf;
    }

    @Bean
    @ConditionalOnMissingBean
    public Twitter twitter(TwitterFactory twitterFactory){
        return twitterFactory.getInstance();
    }

}

We have used @ConditionalOnClass({ TwitterFactory.class, Twitter.class }) to specify that this auto configuration should take place only when TwitterFactory.class, Twitter.class classes are present. 

We have also used @ConditionalOnMissingBean on bean definition methods to specify consider this bean definition only if TwitterFactory/Twitter beans are not already defined explicitly. 

Also note that we have annotated with @EnableConfigurationProperties(Twitter4jProperties.class) to enable support for ConfigurationProperties and injected Twitter4jProperties bean. 

Now we need to configure our custom Twitter4jAutoConfiguration in src/main/resources/META-INF/spring.factories file as follows: org.springframework.boot.autoconfigure.EnableAutoConfiguration=com.sivalabs.spring.boot.autoconfigure.Twitter4jAutoConfiguration 

Create twitter4j-spring-boot-starter module 

Create a child module with name twitter4j-spring-boot-starter in our parent maven module spring-boot-starter-twitter4j.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                        http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sivalabs</groupId>
    <artifactId>twitter4j-spring-boot-starter</artifactId>
    <packaging>jar</packaging>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>com.sivalabs</groupId>
        <artifactId>spring-boot-starter-twitter4j</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>


    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>com.sivalabs</groupId>
            <artifactId>twitter4j-spring-boot-autoconfigure</artifactId>
            <version>${project.version}</version>
        </dependency>

        <dependency>
            <groupId>org.twitter4j</groupId>
            <artifactId>twitter4j-core</artifactId>
        </dependency>

    </dependencies>

</project>

Note that in this maven module we are actually pulling in twitter4j-core dependency. 

We don’t need to add any code in this module, but optionally we can specify what are the dependencies we are going to provide through this starter in src/main/resources/META-INF/spring.provides file as follows: 
provides: twitter4j-core 

That’s all for our starter. 

Let us create a sample using our brand new starter twitter4j-spring-boot-starter. 

Create twitter4j-spring-boot-sample sample application 

Let us create a simple SpringBoot application and add our twitter4j-spring-boot-starter dependency.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                        http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sivalabs</groupId>
    <artifactId>twitter4j-spring-boot-sample</artifactId>
    <packaging>jar</packaging>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.3.2.RELEASE</version>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>1.8</java.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <dependencies>

        <dependency>
            <groupId>com.sivalabs</groupId>
            <artifactId>twitter4j-spring-boot-starter</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

Create the entry-point class SpringbootTwitter4jDemoApplication as follows:
 
package com.sivalabs.demo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SpringbootTwitter4jDemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringbootTwitter4jDemoApplication.class, args);
    }
}

Create TweetService as follows:
 
package com.sivalabs.demo;

import java.util.ArrayList;
import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import twitter4j.ResponseList;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;

@Service
public class TweetService {

    @Autowired
    private Twitter twitter;

    public List<String> getLatestTweets(){
        List<String> tweets = new ArrayList<>();
        try {
            ResponseList<Status> homeTimeline = twitter.getHomeTimeline();
            for (Status status : homeTimeline) {
                tweets.add(status.getText());
            }
        } catch (TwitterException e) {
            throw new RuntimeException(e);
        }
        return tweets;
    }
}

Now create a Test to verify our Twitter4j AutoConfigutation. 

Before that make sure you have set your twitter4j oauth configuration parameter to your actual values. You can get them from https://apps.twitter.com/

 
package com.sivalabs.demo;

import java.util.List;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.SpringApplicationConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import twitter4j.TwitterException;

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(SpringbootTwitter4jDemoApplication.class)
public class SpringbootTwitter4jDemoApplicationTest  {


    @Autowired
    private TweetService tweetService;

    @Test
    public void testGetTweets() throws TwitterException {
        List<String> tweets = tweetService.getLatestTweets();
        for (String tweet : tweets) {
            System.err.println(tweet);
        }
    }

}

Now you should be able to see the latest tweets on your console output. 



If you want to learn more about SpringBoot take a look at my 

SpringBoot : Learn By Example book


Tuesday, February 2, 2016

Retrying Method Execution using Spring AOP

One of my blog follower sends an email asking me to show an example of "RealWorld Usage of Spring AOP". He mentioned that in most of the examples the usage of Spring AOP is demonstrated for logging method entry/exit or Transaction management or Security checks.

He wanted to know how Spring AOP is being used in "Real Project for Real Problems". So I would like to show how I have used Spring AOP for one of my project to handle a real problem.

We won't face some kind of problems in development phases and only come to know during Load Testing or in production environments only.

For example:
  • Remote WebService invocation failures due to network latency issues
  • Database query failures because of Lock exceptions etc
In most of the cases just retrying the same operation is sufficient to solve these kind of failures.

Let us see how we can use Spring AOP to automatically retry the method execution if any exception occurs. We can use Spring AOP @Around advice to create a proxy for those objects whose methods needs to be retried and implement the retry logic in Aspect.

Before jumping on to implementing these Spring Advice and Aspect, first let us write a simple utility to execute a "Task" which automatically retry for N times ignoring the given set of Exceptions.

public interface Task<T> {
 T execute();
}


import java.util.HashSet;
import java.util.Set;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class TaskExecutionUtil 
{
 
 private static Logger logger = LoggerFactory.getLogger(TaskExecutionUtil.class);

 @SafeVarargs
 public static <T> T execute(Task<T> task, 
        int noOfRetryAttempts, 
        long sleepInterval, 
        Class<? extends Throwable>... ignoreExceptions) 
 {
  
  if (noOfRetryAttempts < 1) {
   noOfRetryAttempts = 1;
  }
  Set<Class<? extends Throwable>> ignoreExceptionsSet = new HashSet<Class<? extends Throwable>>();
  if (ignoreExceptions != null && ignoreExceptions.length > 0) {
   for (Class<? extends Throwable> ignoreException : ignoreExceptions) {
    ignoreExceptionsSet.add(ignoreException);
   }
  }
  
  logger.debug("noOfRetryAttempts = "+noOfRetryAttempts);
  logger.debug("ignoreExceptionsSet = "+ignoreExceptionsSet);
  
  T result = null;
  for (int retryCount = 1; retryCount <= noOfRetryAttempts; retryCount++) {
   logger.debug("Executing the task. Attemp#"+retryCount);
   try {
    result = task.execute();
    break;
   } catch (RuntimeException t) {
    Throwable e = t.getCause();
    logger.error(" Caught Exception class"+e.getClass());
    for (Class<? extends Throwable> ignoreExceptionClazz : ignoreExceptionsSet) {
     logger.error(" Comparing with Ignorable Exception : "+ignoreExceptionClazz.getName());
     
     if (!ignoreExceptionClazz.isAssignableFrom(e.getClass())) {
      logger.error("Encountered exception which is not ignorable: "+e.getClass());
      logger.error("Throwing exception to the caller");
      
      throw t;
     }
    }
    logger.error("Failed at Retry attempt :" + retryCount + " of : " + noOfRetryAttempts);
    if (retryCount >= noOfRetryAttempts) {
     logger.error("Maximum retrial attempts exceeded.");
     logger.error("Throwing exception to the caller");
     throw t;
    }
    try {
     Thread.sleep(sleepInterval);
    } catch (InterruptedException e1) {
     //Intentionally left blank
    }
   }
  }
  return result;
 }

}

I hope this method is self explanatory. It is taking a Task and retries noOfRetryAttempts times in case method task.execute() throws any Exception and ignoreExceptions indicates what type of exceptions to be ignored while retrying.

Now let us create a Retry annotation as follows:

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public  @interface Retry {
 
 public int retryAttempts() default 3;
 
 public long sleepInterval() default 1000L; //milliseconds
 
 Class<? extends Throwable>[] ignoreExceptions() default { RuntimeException.class };
 
}

We will use this @Retry annotation to demarcate which methods needs to be retried.

Now let us implement the Aspect which applies to the method with @Retry annotation.

import java.lang.reflect.Method;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;

@Component
@Aspect
public class MethodRetryHandlerAspect {
 
 private static Logger logger = LoggerFactory.getLogger(MethodRetryHandlerAspect.class);
 
 @Around("@annotation(com.sivalabs.springretrydemo.Retry)")
 public Object audit(ProceedingJoinPoint pjp) 
 {
  Object result = null;
  result = retryableExecute(pjp);
     return result;
 }
 
 protected Object retryableExecute(final ProceedingJoinPoint pjp)
 {
  MethodSignature signature = (MethodSignature) pjp.getSignature();
  Method method = signature.getMethod();
  logger.debug("-----Retry Aspect---------");
  logger.debug("Method: "+signature.toString());

  Retry retry = method.getDeclaredAnnotation(Retry.class);
  int retryAttempts = retry.retryAttempts();
  long sleepInterval = retry.sleepInterval();
  Class<? extends Throwable>[] ignoreExceptions = retry.ignoreExceptions();
  
  Task<Object> task = new Task<Object>() {
   @Override
   public Object execute() {
    try {
     return pjp.proceed();
    } catch (Throwable e) {
     throw new RuntimeException(e);
    }
   }
  };
  return TaskExecutionUtil.execute(task, retryAttempts, sleepInterval, ignoreExceptions);
 }
}

That's it. We just need some test cases to actually test it.

First create AppConfig.java configuration class as follows:

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;

@Configuration
@ComponentScan
@EnableAspectJAutoProxy
public class AppConfig {

}

And couple of dummy Service beans.

import org.springframework.stereotype.Service;

@Service
public class ServiceA {
 
 private int counter = 1;
 
 public void method1() {
  System.err.println("----method1----");
 }
 
 @Retry(retryAttempts=5, ignoreExceptions={NullPointerException.class})
 public void method2() {
  System.err.println("----method2 begin----");
  if(counter != 3){
   counter++;
   throw new NullPointerException();
  }
  System.err.println("----method2 end----");  
 }
}

import java.io.IOException;
import org.springframework.stereotype.Service;

@Service
public class ServiceB {
 
 @Retry(retryAttempts = 2, ignoreExceptions={IOException.class})
 public void method3() {
  System.err.println("----method3----");
  if(1 == 1){
   throw new ArrayIndexOutOfBoundsException();
  }
 }
 
 @Retry
 public void method4() {
  System.err.println("----method4----");
 }
}

Finally write a simple Junit test to invoke these methods.

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes=AppConfig.class)
public class RetryTest 
{
 @Autowired ServiceA svcA;
 @Autowired ServiceB svcB;
 
 @Test
 public void testA()
 {
  svcA.method1();
 }
 
 @Test
 public void testB()
 {
  svcA.method2();
 }
 
 @Test(expected=RuntimeException.class)
 public void testC()
 {
  svcB.method3();
 }
 
 @Test
 public void testD()
 {
  svcB.method4();
 }
}

Yeah, I know I could have written these test methods a bit better, but I hope you got the idea.

Run the JUnit tests and observe the log statement to verify whether the method retry is happening in case of Exception or not.

Case#1: When invoking ServiceA.method1() is invoked MethodRetryHandlerAspect won't be applied at all.

Case#2: When invoking ServiceA.method2() is invoked, we are maintaining a counter and throwing NullPointerException for 2 times. But we have marked that method to ignore NullPointerExceptions. So it will continue to retry for 5 times. But 3rd time method will be executed normally and exits the method normally.

Case#3: When invoking ServiceB.method3() is invoked, we are throwing ArrayIndexOutOfBoundsException but that method is marked to ignore only IOException only. So this method execution won't be retried and throws the Exception immediately.

Case#4: When invoking ServiceB.method4() is invoked, everything is fine so it should exit in the first attempt itself normally.

I hope this example demonstrate a good enough real world usage of Spring AOP :-)

JCart : Iteration-8 - Customer MyAccount Page

In this Iteration#8 we will implement showing the Customer Account and Order History functionality in our ShoppingCart application.
  • Customer MyAccount Page
    • Profile
    • Order History
Once the customer is logged in our system he can click on MyAccount link at the top of the header and view his profile details and order history.

First let us write the Controller handler method in our CustomerController to show myAccount details.

@Controller
public class CustomerController extends JCartSiteBaseController
{ 
 @Autowired private CustomerService customerService;
 ...
 ... 
 
 @RequestMapping(value="/myAccount", method=RequestMethod.GET)
 protected String myAccount(Model model)
 {
  String email = getCurrentUser().getCustomer().getEmail();
  Customer customer = customerService.getCustomerByEmail(email);
  model.addAttribute("customer", customer);
  List<Order> orders = customerService.getCustomerOrders(email);
  model.addAttribute("orders", orders);
  return "myAccount";
 }
}

Now create the myAccount.html view to render customer details and customer order history.

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"
 xmlns:th="http://www.thymeleaf.org"
 xmlns:sec="http://www.thymeleaf.org/thymeleaf-extras-springsecurity3"
 layout:decorator="layout/mainLayout">
<head>
<title>My Account</title>
</head>
<body>
 <div layout:fragment="content">
  <div class="single-product-area">
   <div class="zigzag-bottom"></div>
   <div class="container">
    <div role="tabpanel">
     <ul class="customer-tab" role="tablist">
      <li role="presentation" class="active"><a href="#profile"
       aria-controls="profile" role="tab" data-toggle="tab">Customer
        Info</a></li>
      <li role="presentation"><a href="#orders"
       aria-controls="orders" role="tab" data-toggle="tab">Orders</a></li>
     </ul>
     <div class="tab-content">
      <div role="tabpanel" class="tab-pane fade in active" id="profile">
       <h2>Customer Info</h2>
       <form role="form" action="#" th:object="${customer}"
        method="post">        
        <div class="form-group">
         <label>FirstName</label> <input type="text"
          class="form-control" th:field="*{firstName}"
          readonly="readonly" />
        </div>
        <div class="form-group">
         <label>LastName</label> <input type="text" class="form-control"
          th:field="*{lastName}" readonly="readonly" />
        </div>
        <div class="form-group">
         <label>Email</label> <input type="email" class="form-control"
          th:field="*{email}" readonly="readonly" />
        </div>
        <div class="form-group">
         <label>Phone</label> <input type="text" class="form-control"
          th:field="*{phone}" readonly="readonly" />
        </div>
       </form>
      </div>
      <div role="tabpanel" class="tab-pane fade" id="orders">
       <h2>Orders</h2>
       <table cellspacing="0" class="shop_table cart">
        <thead>
         <tr>
          <th>#</th>
          <th>Order Number</th>
          <th>Status</th>
         </tr>
        </thead>
        <tbody>
         <tr th:each="order,iterStat : ${orders}">
          <td><span th:text="${iterStat.count}">1</span></td>
          <td><a href="#" th:text="${order.orderNumber}"
           th:href="@{/orders/{orderNumber}(orderNumber=${order.orderNumber})}">OrderNumber</a>
          </td>
          <td><span th:text="${order.status}">Status</span></td>

         </tr>
        </tbody>
       </table>
      </div>
     </div>
    </div>
   </div>
  </div>
 </div>
</body>
</html>

Now you can login as customer and click on MyAccount and see the profile. When you click on Orders tab you can see the list of orders that customer is placed. Also you can click on Order Number to see more details of the Order.