What are Spring Boot, Liquibase and Flyway?

The Spring Ecosystem

When you need to create a web application or an API in Java, be that RESTful, SOAP or GraphQL; whether you’re looking at synchronous HTTP, Asynchronous or Reactive, Messages on Queues or Event Sourced with Kafka, it’s hard to go past the Spring Ecosystem. If nobody ever got fired for buying IBM, you could probably say that nobody ever got fired for creating a Spring-based application.

Spring is an extremely popular, well maintained and well known Java Web Application Framework. Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can “just run”.

Spring Boot is to Spring and Java as Ruby on Rails is to Ruby. So if you’re working in a Java environment, there’s a pretty good chance you’re using Spring or Spring Boot. Even if you’re using another framework, it’s worth learning about these - they’re so ubiquitous they’ll be around for a long time!

Database Migrations

If you’re using Spring, there’s a chance you’re using persistence tech like Hibernate, Jooq or Ebean to get data out of your database.

Features come and go; data models change; what happens when you need to change your data model? You run a database migration to add or remove columns or make other changes.

It used to be that this would happen manually; someone would log on to a database, run a bunch of SQL and rely on the fact that you’ve made the same changes in your application. There is a bunch of room for human error here, though. So we improve on this by trying to make our database changes safer. Version control, repeatability, and testability become applicable to more parts of the stack. Databases included.

Both Liquibase and Flyway help us with these migrations. Spring helps us with Liquibase and Flyway in turn. The result is a database changeset that does what we have specified every single time (of course, we might still specify the wrong thing - computers remain computers). All we need to do is write a migration and dump it into a folder in our Spring project to achieve that result. Spring handles the rest.

Liquibase

Liquibase is offered free and paid. They don’t share pricing on their website, though, which makes me suspicious. They don’t do an excellent job of describing why I would want to pay either. Liquibase offers migrations both through XML and SQL. The basic concept is that you have a master file that describes your database configuration and the changesets you want to include in your run. Spring comes in at that point, parses your config and your included changesets, and manages them appropriately.

Flyway

Flyway is offered free and paid, the same as liquibase. The pricing and feature breakdown is available here - and is a lot more detailed than anything I could find on liquibase.

Flyway focuses on your migrations as a first-class concept. You write SQL scripts, place them in a folder in your Spring project, add some config to your application.yml files and then Spring runs the migrations as per your config.

Using Liquibase

Note: I’m using Spring Boot 2.2.6 with Gradle here, but you can do the equivalent with a dependency block in Maven.

Step One is adding your dependencies to build.gradle:

    // Persistence
    implementation('com.h2database:h2')
    implementation "org.liquibase:liquibase-core"
    liquibaseRuntime "org.liquibase:liquibase-core"
    liquibaseRuntime sourceSets.main.compileClasspath
    liquibaseRuntime "org.postgresql:postgresql"
    liquibaseRuntime "com.h2database:h2"
    implementation('org.postgresql:postgresql')
    implementation('org.springframework.boot:spring-boot-starter-jdbc')

Next we need to make new folders in our project:

  • In src/main/resources add a liquibase folder
  • Add two subfolders, changelog and fake-data

Add some minimal configuration to your application.yml file and you’re good to go.

  liquibase:
    contexts: dev, faker
    change-log: classpath:liquibase/master.xml

At this point, you should be able to start writing some XML.

You’ll need a master.xml that describes your environment setup and the migrations that you want to run. Here’s an example:

<?xml version="1.0" encoding="utf-8"?>
<databaseChangeLog
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.6.xsd">

    <property name="now" value="now()" dbms="h2"/>
    <property name="now" value="current_timestamp" dbms="postgresql"/>

    <property name="floatType" value="float4" dbms="postgresql, h2"/>
    <property name="clobType" value="longvarchar" dbms="h2"/>
    <property name="clobType" value="clob" dbms="postgresql"/>
    <property name="uuidType" value="uuid" dbms="h2, postgresql"/>

    <changeSet id="00000000000000" author="alex">
        <createSequence sequenceName="sequence_generator" startValue="1050" incrementBy="1"/>
    </changeSet>

    <include file="liquibase/changelog/20191024203226_added_entity_Company.xml" relativeToChangelogFile="false"/>
    <include file="liquibase/changelog/20191024203227_added_entity_Team.xml" relativeToChangelogFile="false"/>
    <include file="liquibase/changelog/20191024203234_added_entity_Project.xml" relativeToChangelogFile="false"/>
</databaseChangeLog>

It looks a bit busy but let’s break it down. First, we need to tell Liquibase about our database setup, and what values it should use for various data types:

    <property name="now" value="now()" dbms="h2"/>
    <property name="now" value="current_timestamp" dbms="postgresql"/>

    <property name="floatType" value="float4" dbms="postgresql, h2"/>
    <property name="clobType" value="longvarchar" dbms="h2"/>
    <property name="clobType" value="clob" dbms="postgresql"/>
    <property name="uuidType" value="uuid" dbms="h2, postgresql"/>

Next, we need to describe the changes we want Liquibase to make; in our case, we want Liquibase to create a sequence and run three migrations. Note that if you write migrations but don’t include them in master.xml, they won’t run.

    <changeSet id="00000000000000" author="alex">
        <createSequence sequenceName="sequence_generator" startValue="1050" incrementBy="1"/>
    </changeSet>

    <include file="liquibase/changelog/20191024203226_added_entity_Company.xml" relativeToChangelogFile="false"/>
    <include file="liquibase/changelog/20191024203227_added_entity_Team.xml" relativeToChangelogFile="false"/>
    <include file="liquibase/changelog/20191024203234_added_entity_Project.xml" relativeToChangelogFile="false"/>

Here’s the company migration:

<?xml version="1.0" encoding="utf-8"?>
<databaseChangeLog
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.6.xsd">

    <changeSet id="20191024203226-1" author="alex">
        <createTable tableName="company">
            <column name="id" type="bigint" autoIncrement="${autoIncrement}">
                <constraints primaryKey="true" nullable="false"/>
            </column>
            <column name="name" type="varchar(255)">
                <constraints nullable="false"/>
            </column>
            <column name="web_page_address" type="varchar(255)">
                <constraints nullable="true"/>
            </column>
            <column name="billing_contact_email_address" type="varchar(255)">
                <constraints nullable="true"/>
            </column>
            <column name="primary_contact_email_address" type="varchar(255)">
                <constraints nullable="true"/>
            </column>

        </createTable>
    </changeSet>

    <changeSet id="20191024203226-1-data" author="alex" context="faker">
        <loadData
                file="liquibase/fake-data/company.csv"
                separator=";"
                tableName="company">
            <column name="id" type="numeric"/>
            <column name="name" type="string"/>
            <column name="web_page_address" type="string"/>
            <column name="billing_contact_email_address" type="string"/>
            <column name="primary_contact_email_address" type="string"/>
        </loadData>
    </changeSet>

</databaseChangeLog>

The first thing this migration creates is a table called Company. Since Liquibase uses XML as an abstraction on top of SQL, we don’t need to worry too much about compatibility, Liquibase will look at the XML file and translate our request to the appropriate SQL dialect.

    <changeSet id="20191024203226-1" author="alex">
        <createTable tableName="company">
            <column name="id" type="bigint" autoIncrement="${autoIncrement}">
                <constraints primaryKey="true" nullable="false"/>
            </column>
            <column name="name" type="varchar(255)">
                <constraints nullable="false"/>
            </column>
            <column name="web_page_address" type="varchar(255)">
                <constraints nullable="true"/>
            </column>
            <column name="billing_contact_email_address" type="varchar(255)">
                <constraints nullable="true"/>
            </column>
            <column name="primary_contact_email_address" type="varchar(255)">
                <constraints nullable="true"/>
            </column>

        </createTable>
    </changeSet>

Having defined our table, we can take advantage of one of the neat functions Liquibase can give us - Liquibase will load data into our database from CSV:

    <changeSet id="20191024203226-1-data" author="alex" context="faker">
        <loadData
                file="liquibase/fake-data/company.csv"
                separator=";"
                tableName="company">
            <column name="id" type="numeric"/>
            <column name="name" type="string"/>
            <column name="web_page_address" type="string"/>
            <column name="billing_contact_email_address" type="string"/>
            <column name="primary_contact_email_address" type="string"/>
        </loadData>
    </changeSet>

The loading of data is a changeset in its own right; this means you can flexibly load data when you create the table, or at a later stage. Loading data in this way can be useful if you’ve got complex relationships to model. Another trick here is the context="faker" statement above. If you want data to be loaded, make sure your application.yml’s liquibase entry contains that tag. If you remove that tag, the data won’t be loaded.

Here’s the CSV referenced above with some sample data in it:

id,name,web_page_address,billing_contact_email_address,primary_contact_email_address
1,mobileFish,HomeLoanAccountTableComputer,Trace,SavingsAccountalarm
2,TastyMetalBacon,redChipsSoap,HomeLoanAccount,web-readiness

That’s it. When you run your spring boot application, during startup Liquibase will apply this migration, and when your app connects to the database, the tables and data will be ready immediately.

How does liquibase know the database credentials? I’m not sure! It’s magic. I assume it hooks into your spring database config, an example of which might be:

datasource:
    type: com.zaxxer.hikari.HikariDataSource
    url: jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
    username: sa
    password:
    hikari:
      poolName: Hikari
      auto-commit: false

Using Flyway

Flyway is a little bit different. The focus on SQL tidies things up, in my opinion.

First, you add the relevant dependencies:

// Persistence
    implementation('com.h2database:h2')
    implementation('org.postgresql:postgresql')
    implementation('org.springframework.boot:spring-boot-starter-jdbc')
    implementation "org.flywaydb:flyway-core"

Tell Flyway how to connect to your database (in my case, via application.yml):

  flyway:
    locations: classpath:db/migration/dev
    url: jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1
    user: sa
    password:

Make some folders, again in src/main/resources:

  • Create db/migration
  • In my case, I have H2 in dev and PostgreSQL in test and prod, so I have two sibling folders, dev and prod (test and prod scripts both live in prod as I want those to be the same, and they are the same SQL dialect).

The next step is to write some migrations in the appropriate SQL dialect. Where before our Company migration was in XML, now we’re in SQL, note that there is a naming convention here that allows us to cut down on boiler-plate. You have to name files like so V1.0__create_company.sql where the VX.x will define the order in which migrations are run. Anyway, here’s the file:

create sequence hibernate_sequence start with 1050;

create table company
(
    id                            uuid         not null,
    name                          varchar(255) not null,
    web_page_address              varchar(255),
    billing_contact_email_address varchar(255),
    primary_contact_email_address varchar(255),
    constraint pk_company
        primary key (id)
);

Dropping the XML leaves things a lot cleaner and readable, in my opinion. We load data as well, but this is just a SQL script rather than CSV:

INSERT INTO PUBLIC.COMPANY (ID, NAME, WEB_PAGE_ADDRESS, BILLING_CONTACT_EMAIL_ADDRESS, PRIMARY_CONTACT_EMAIL_ADDRESS)
VALUES ('1a689e52-f35b-4bda-934c-ea4f076bdc2c', 'Blue Fish Software Inc', 'bluefish.io', 'bills@bluefish.io',
        'hi@bluefish.io');

That’s it. When the application runs, it’ll load those SQL scripts and manage migrations and state for us. The disadvantage here is that we need two sets of scripts, one for H2 and one for PostgreSQL - this could be because my SQL is super weak and I can combine them but don’t realise it.

Comparing Liquibase and Flyway

I found Flyway to be a lot easier to use, despite having used Liquibase for a lot longer.

I like that Flyway is cleaner and less repetitive, and assuming I fix my dialect issue, it’ll be a case of 1000+ fewer lines of code with Flyway than with the same setup in Liquibase. I also found the docs Flyway provides to be a lot better, and I encountered far fewer undocumented tricks needed with Flyway. If crucial information needed to get going is easier to find on Stack Overflow than on your docs pages, you’ve done something wrong. Liquibase is bad for this.

Moving from bigint to uuid primary keys on all of my entities was impossible with Liquibase due to a bug around mis-detecting specific UUID values as strings.

Flyway had no such issues, and where I did make a mistake with Flyway, I got printed SQL exceptions rather than SQL exceptions that have abstracted by Liquibase. I encountered far less friction with Flyway.

u/hooba_stank via Reddit, makes the point that liquibase’s profiles grant it excellent flexibility. Using contexts in different environments, composing profiles for different testing requirements and conditional change sets and rollbacks make the free Liquibase offering compelling. It’s a great point.

Flyway’s requirement for sequential ordering of SQL files can lead to sprawl if you prefer small changes per file. Encountering failures becuase you’ve fat-fingered a SQL script version number is annoying too. u/koreth adds that this problem can be avoided by configuring a pre-commit hook and CI check to detect conflicting sequence numbers so this issue can be automated to a degree, though ideally it’s not something you have to deal with at all.

Wrap Up

Both Liquibase and Flyway are better than nothing by far. My personal preference is Flyway. Flyway is cleaner to implement and easier to use on an ongoing basis and more readable. That being said, Liquibase is a powerful tool. Chosing either of these tools to use in your development will give you good gains in productivity.