/cdn.vox-cdn.com/uploads/chorus_asset/file/25829979/STK051_TIKTOKBAN_B_CVirginia_D.jpg)
Flyway Migrations in Multi-Module Gradle Projects (Clean Architecture)
Automating Database Migrations in Java with Flyway Database migrations are a crucial aspect of software development, particularly in environments where continuous integration and delivery (CI/CD) are standard practice. As your application grows and evolves, so too must the database schema it depends on. Manually managing these schema changes can lead to errors and consume significant time. Enter Flyway, an invaluable open-source tool tailored for simplifying database migrations. Flyway introduces version control to your database, allowing you to migrate your schema safely and with reliability. In this article, we'll explore how to automate database migrations in multi-module gragle java project using Flyway, ensuring that managing database changes becomes a streamlined, error-resistant process. More details on flyway Understanding Multi-Project Builds in Gradle While some smaller projects or monolithic applications might manage with just one build file and a unified source structure, larger projects are frequently organized into multiple, interdependent modules. The term "interdependent" is key here, highlighting the need to connect these modules via a singular build process. Gradle caters to this setup with its multi-project build capability, often termed as a multi-module project. In Gradle's terminology, these modules are called subprojects. A multi-project build is structured around one root project and can include several subprojects beneath it. The directory structure should look as follows: ├── .gradle │ └── ⋮ ├── gradle │ ├── libs.versions.toml │ └── wrapper ├── gradlew ├── gradlew.bat ├── settings.gradle.kts (1) ├── sub-project-1 │ └── build.gradle.kts (2) ├── sub-project-2 │ └── build.gradle.kts (2) └── sub-project-3 └── build.gradle.kts (2) (1) The settings.gradle.kts file should include all subprojects. (2) Each subproject should have its own build.gradle.kts file. Leveraging Gradle Sub-Modules for Clean Architecture Clean Architecture is a design pattern that emphasizes separation of concerns, making software easier to maintain and test. One of the practical ways to implement this architecture in a project involves using Gradle's sub-module structure to organize your codebase. Here's how you can align Clean Architecture with Gradle sub-modules: Clean Architecture Layers: Core: Contains business logic, domain models, and application rules. Has no dependency on External or Web. Should be independent of framework-specific implementations where possible. External: Handles external actions or integrations, such as database migrations or third-party service interactions. May depend on Core for business logic but should not depend on Web. Web: The entry point, exposing a REST API and handling HTTP requests. Depends on Core for business logic and may depend on External for integrations. SchoolStaff/ ├── Core/ │ ├── src/ │ │ └── main/ │ │ ├── java/ # Business logic and domain objects │ │ └── resources/ # Core-specific resources (if any) │ └── build.gradle.kts ├── External/ │ ├── src/ │ │ └── main/ │ │ ├── java/ # External integration code │ │ └── resources/ # db/migration and other external resources │ └── build.gradle.kts ├── Web/ │ ├── src/ │ │ └── main/ │ │ ├── java/ # REST controllers and entry-point logic │ │ └── resources/ # Application-specific configuration │ └── build.gradle.kts ├── build.gradle.kts # Root Gradle build └── settings.gradle.kts # Project module settings Step 1: Create a Java-based Gradle project and name it "SchoolStaff". Step 2: Go to Spring Initializr and generate a REST API project named Web. Step 3: Create a Java-based Gradle project and name it External. Step 4: Create a Java-based Gradle project and name it Core. Root build.gradle.kts plugins { id("java") } allprojects { group = "school.staff" version = "1.0.0" repositories { mavenLocal() mavenCentral() } } subprojects { apply(plugin = "java") dependencies { testImplementation(platform("org.junit:junit-bom:5.10.0")) testImplementation("org.junit.jupiter:junit-jupiter") } tasks.test { useJUnitPlatform() } } settings.gradle.kts rootProject.name = "SchoolStaff" include("Core", "External", "Web") Required dependencies for the "Web" project. dependencies { implementation(project(":Core")) implementation(project(":External")) } Required dependencies for the "Core" project. dependencies { runtimeOnly(project(":External")) } Required dependencies for the "External" project. import java.sql.DriverManager import java.util.Properties // Function to load properties based on the environment fun loadProperties(env: String): Properties { val properties = Properties() val propsFile = file("..
Automating Database Migrations in Java with Flyway
Database migrations are a crucial aspect of software development, particularly in environments where continuous integration and delivery (CI/CD) are standard practice. As your application grows and evolves, so too must the database schema it depends on. Manually managing these schema changes can lead to errors and consume significant time.
Enter Flyway, an invaluable open-source tool tailored for simplifying database migrations. Flyway introduces version control to your database, allowing you to migrate your schema safely and with reliability. In this article, we'll explore how to automate database migrations in multi-module gragle java project using Flyway, ensuring that managing database changes becomes a streamlined, error-resistant process.
Understanding Multi-Project Builds in Gradle
While some smaller projects or monolithic applications might manage with just one build file and a unified source structure, larger projects are frequently organized into multiple, interdependent modules. The term "interdependent" is key here, highlighting the need to connect these modules via a singular build process.
Gradle caters to this setup with its multi-project build capability, often termed as a multi-module project. In Gradle's terminology, these modules are called subprojects.
A multi-project build is structured around one root project and can include several subprojects beneath it.
The directory structure should look as follows:
├── .gradle
│ └── ⋮
├── gradle
│ ├── libs.versions.toml
│ └── wrapper
├── gradlew
├── gradlew.bat
├── settings.gradle.kts (1)
├── sub-project-1
│ └── build.gradle.kts (2)
├── sub-project-2
│ └── build.gradle.kts (2)
└── sub-project-3
└── build.gradle.kts (2)
(1) The settings.gradle.kts file should include all subprojects.
(2) Each subproject should have its own build.gradle.kts file.
Leveraging Gradle Sub-Modules for Clean Architecture
Clean Architecture is a design pattern that emphasizes separation of concerns, making software easier to maintain and test. One of the practical ways to implement this architecture in a project involves using Gradle's sub-module structure to organize your codebase. Here's how you can align Clean Architecture with Gradle sub-modules:
Clean Architecture Layers:
Core:
- Contains business logic, domain models, and application rules. Has no dependency on External or Web.
- Should be independent of framework-specific implementations where possible.
External:
- Handles external actions or integrations, such as database migrations or third-party service interactions.
- May depend on Core for business logic but should not depend on Web.
Web:
- The entry point, exposing a REST API and handling HTTP requests.
- Depends on Core for business logic and may depend on External for integrations.
SchoolStaff/
├── Core/
│ ├── src/
│ │ └── main/
│ │ ├── java/ # Business logic and domain objects
│ │ └── resources/ # Core-specific resources (if any)
│ └── build.gradle.kts
├── External/
│ ├── src/
│ │ └── main/
│ │ ├── java/ # External integration code
│ │ └── resources/ # db/migration and other external resources
│ └── build.gradle.kts
├── Web/
│ ├── src/
│ │ └── main/
│ │ ├── java/ # REST controllers and entry-point logic
│ │ └── resources/ # Application-specific configuration
│ └── build.gradle.kts
├── build.gradle.kts # Root Gradle build
└── settings.gradle.kts # Project module settings
Step 1: Create a Java-based Gradle project and name it "SchoolStaff".
Step 2: Go to Spring Initializr and generate a REST API project named Web.
Step 3: Create a Java-based Gradle project and name it External.
Step 4: Create a Java-based Gradle project and name it Core.
Root build.gradle.kts
plugins {
id("java")
}
allprojects {
group = "school.staff"
version = "1.0.0"
repositories {
mavenLocal()
mavenCentral()
}
}
subprojects {
apply(plugin = "java")
dependencies {
testImplementation(platform("org.junit:junit-bom:5.10.0"))
testImplementation("org.junit.jupiter:junit-jupiter")
}
tasks.test {
useJUnitPlatform()
}
}
settings.gradle.kts
rootProject.name = "SchoolStaff"
include("Core", "External", "Web")
Required dependencies for the "Web" project.
dependencies {
implementation(project(":Core"))
implementation(project(":External"))
}
Required dependencies for the "Core" project.
dependencies {
runtimeOnly(project(":External"))
}
Required dependencies for the "External" project.
import java.sql.DriverManager
import java.util.Properties
// Function to load properties based on the environment
fun loadProperties(env: String): Properties {
val properties = Properties()
val propsFile = file("../web/src/main/resources/application-$env.properties")
if (propsFile.exists()) {
propsFile.inputStream().use { properties.load(it) }
} else {
throw GradleException("Properties file for environment '$env' not found: ${propsFile.absolutePath}")
}
return properties
}
// Set the environment (default is 'dev' if no argument is passed)
val env = project.findProperty("env")?.toString() ?: "dev"
// Load properties for the chosen environment
val dbProps = loadProperties(env)
buildscript {
dependencies {
classpath("org.flywaydb:flyway-database-postgresql:11.1.0") // This is required for the flyway plugin to work on the migration, otherwise it will throw an error as No Database found
classpath("org.postgresql:postgresql:42.7.4")
}
}
plugins {
id("java-library")
id("org.flywaydb.flyway") version "11.0.1"
}
group = "school.staff"
version = "unspecified"
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
implementation("org.springframework.boot:spring-boot-starter-data-jpa:3.4.0")
implementation("org.postgresql:postgresql:42.7.4")
implementation("org.flywaydb:flyway-core:11.0.1")
implementation("org.flywaydb:flyway-database-postgresql:11.0.1")
implementation("org.flywaydb:flyway-gradle-plugin:11.0.1")
implementation (project(":Core"))
testImplementation(platform("org.junit:junit-bom:5.10.0"))
testImplementation("org.junit.jupiter:junit-jupiter")
}
tasks.test {
useJUnitPlatform()
}
// Task to create the database if it doesn't exist
tasks.register("createDatabase") {
doLast {
val dbUrl = dbProps["spring.datasource.url"] as String
val dbUsername = dbProps["spring.datasource.username"] as String
val dbPassword = dbProps["spring.datasource.password"] as String
// Extract the base URL and database name
val baseDbUrl = dbUrl.substringBeforeLast("/")+ "/"
val dbName = dbUrl.substringAfterLast("/")
// Connect to the PostgreSQL server (without the specific database)
DriverManager.getConnection(baseDbUrl, dbUsername, dbPassword).use { connection ->
val stmt = connection.createStatement()
val resultSet = stmt.executeQuery("SELECT 1 FROM pg_database WHERE datname = '$dbName'")
if (!resultSet.next()) {
println("Database '$dbName' does not exist. Creating it...")
stmt.executeUpdate("CREATE DATABASE \"$dbName\"")
println("Database '$dbName' created successfully.")
} else {
println("Database '$dbName' already exists.")
}
}
}
}
flyway {
url = dbProps["spring.datasource.url"] as String
user = dbProps["spring.datasource.username"] as String
password = dbProps["spring.datasource.password"] as String
locations = arrayOf("classpath:db/migration")
baselineOnMigrate = true
}
//Ensure classes are built before migration
tasks.named("flywayMigrate").configure {
dependsOn(tasks.named("createDatabase"))
dependsOn(tasks.named("classes"))
}
We use the following plugin for Flyway migration:
plugins {
id("org.flywaydb.flyway") version "11.0.1"
}
This approach is well-suited for production environments, as it ensures controlled and reliable migrations. Instead of running migrations automatically on each application startup, we execute them only when necessary, providing greater flexibility and control.
We are also utilizing the application.properties file in the Spring application to manage database connections and credentials. The baselineOnMigrate = true setting ensures that the initial migration is used as the baseline for future migrations.
flyway {
url = dbProps["spring.datasource.url"] as String
user = dbProps["spring.datasource.username"] as String
password = dbProps["spring.datasource.password"] as String
locations = arrayOf("classpath:db/migration")
baselineOnMigrate = true
}
We can use JPA Buddy to generate all the migration files within the External project's resources/db/migration directory.
V1__Initial_Migration
CREATE TABLE _user
(
id UUID NOT NULL,
created_by UUID,
created_date TIMESTAMP WITH TIME ZONE,
last_modified_by UUID,
last_modified_date TIMESTAMP WITH TIME ZONE,
first_name VARCHAR(255),
last_name VARCHAR(255),
email VARCHAR(255),
password VARCHAR(255),
tenant_id UUID,
CONSTRAINT pk__user PRIMARY KEY (id)
);
From the root project, we can execute the Flyway migration using the following command:
./gradlew clean build flywayMigrate
This will apply all the migration files to the database.
Conclusion
We've explored how to automate database migrations using Flyway within a Gradle multi-module project, which is crucial for maintaining schema consistency in CI/CD environments.
We also covered how Gradle supports multi-project builds, organizing complex projects into manageable subprojects, each with its own build configuration, unified under a root build script.
Lastly, we aligned Clean Architecture with Gradle modules, structuring the project into Core, External, and Web layers, promoting a clean separation of concerns and dependency management.
These practices enhance modularity, automation, and maintainability, setting the stage for scalable, error-free software development.
