Chapter 23. Dependency Management

23.1. Introduction

Dependency management is a critical feature of every build, and Gradle has placed an emphasis on offering first-class dependency management that is both easy to understand and compatible with a wide variety of approaches. If you are familiar with the approach used by either Maven or Ivy you will be delighted to learn that Gradle is fully compatible with both approaches in addition to being flexible enough to support fully-customized approaches.

Here are the major highlights of Gradle's support for dependency management:

  • Transitive dependency management: Gradle gives you full control of your project's dependency tree.

  • Support for non-managed dependencies: If your dependencies are simply files in version control or a shared drive, Gradle provides powerful functionality to support this.

  • Support for custom dependency definitions.: Gradle's Module Dependencies give you the ability to describe the dependency hierarchy in the build script.

  • A fully customizable approach to Dependency Resolution: Gradle provides you with the ability to customize resolution rules making dependency substitution easy.

  • Full Compatibility with Maven and Ivy: If you have defined dependencies in a Maven POM or an Ivy file, Gradle provides seamless integration with a range of popular build tools.

  • Integration with existing dependency management infrastructure: Gradle is compatible with both Maven and Ivy repositories. If you use Archiva, Nexus, or Artifactory, Gradle is 100% compatible with all repository formats.

With hundreds of thousands of interdependent open source components each with a range of versions and incompatibilities, dependency management has a habit of causing problems as builds grow in complexity. When a build's dependency tree becomes unwieldy, your build tool shouldn't force you to adopt a single, inflexible approach to dependency management. A proper build system has to be designed to be flexible, and Gradle can handle any situation.

23.1.1. Flexible dependency management for migrations

Dependency management can be particularly challenging during a migration from one build system to another. If you are migrating from a tool like Ant or Maven to Gradle, you may be faced with some difficult situations. For example, one common pattern is an Ant project with version-less jar files stored in the filesystem. Other build systems require a wholesale replacement of this approach before migrating. With Gradle, you can adapt your new build to any existing source of dependencies or dependency metadata. This makes incremental migration to Gradle much easier than the alternative. On most large projects, build migrations and any change to development process is incremental because most organizations can't afford to stop everything and migrate to a build tool's idea of dependency management.

Even if your project is using a custom dependency management system or something like an Eclipse .classpath file as master data for dependency management, it is very easy to write a Gradle plugin to use this data in Gradle. For migration purposes this is a common technique with Gradle. (But, once you've migrated, it might be a good idea to move away from a .classpath file and use Gradle's dependency management features directly.)

23.1.2. Dependency management and Java

It is ironic that in a language known for its rich library of open source components that Java has no concept of libraries or versions. In Java, there is no standard way to tell the JVM that you are using version 3.0.5 of Hibernate, and there is no standard way to say that foo-1.0.jar depends on bar-2.0.jar. This has led to external solutions often based on build tools. The most popular ones at the moment are Maven and Ivy. While Maven provides a complete build system, Ivy focuses solely on dependency management.

Both tools rely on descriptor XML files, which contain information about the dependencies of a particular jar. Both also use repositories where the actual jars are placed together with their descriptor files, and both offer resolution for conflicting jar versions in one form or the other. Both have emerged as standards for solving dependency conflicts, and while Gradle originally used Ivy under the hood for its dependency management. Gradle has replaced this direct dependency on Ivy with a native Gradle dependency resolution engine which supports a range of approaches to dependency resolution including both POM and Ivy descriptor files.

23.2. Dependency Management Best Practices

While Gradle has strong opinions on dependency management, the tool gives you a choice between two options: follow recommended best practices or support any kind of pattern you can think of. This section outlines the Gradle project's recommended best practices for managing dependencies.

No matter what the language, proper dependency management is important for every project. From a complex enterprise application written in Java depending on hundreds of open source libraries to the simplest Clojure application depending on a handful of libraries, approaches to dependency management vary widely and can depend on the target technology, the method of application deployment, and the nature of the project. Projects bundled as reusable libraries may have different requirements than enterprise applications integrated into much larger systems of software and infrastructure. Despite this wide variation of requirements, the Gradle project recommends that all projects follow this set of core rules:

23.2.1. Put the Version in the Filename (Version the jar)

The version of a library must be part of the filename. While the version of a jar is usually in the Manifest file, it isn't readily apparent when you are inspecting a project. If someone asks you to look at a collection of 20 jar files, which would you prefer? A collection of files with names like commons-beanutils-1.3.jar or a collection of files with names like spring.jar? If dependencies have file names with version numbers you can quickly identify the versions of your dependencies.

If versions are unclear you can introduce subtle bugs which are very hard to find. For example there might be a project which uses Hibernate 2.5. Think about a developer who decides to install version 3.0.5 of Hibernate on her machine to fix a critical security bug but forgets to notify others in the team of this change. She may address the security bug successfully, but she also may have introduced subtle bugs into a codebase that was using a now-deprecated feature from Hibernate. Weeks later there is an exception on the integration machine which can't be reproduced on anyone's machine. Multiple developers then spend days on this issue only finally realising that the error would have easy to uncover if they knew that Hibernate had been upgraded from 2.5 to 3.0.5.

Versions in jar names increase the expressiveness of your project and make them easier to maintain. This practice also reduces the potential for error.

23.2.2. Manage transitive dependencies

Transitive dependency management is a technique that enables your project to depend on libraries which, in turn, depend on other libraries. This recursive pattern of transitive dependencies results in a tree of dependencies including your project's first-level dependencies, second-level dependencies, and so on. If you don't model your dependencies as a hierarchical tree of first-level and second-level dependencies it is very easy to quickly lose control over an assembled mess of unstructured dependencies. Consider the Gradle project itself, while Gradle only has a few direct, first-level dependencies, when Gradle is compiled it needs more than one hundred dependencies on the classpath. On a far larger scale, Enterprise projects using Spring, Hibernate, and other libraries, alongside hundreds or thousands of internal projects, can result in very large dependency trees.

When these large dependency trees need to change, you'll often have to solve some dependency version conflicts. Say one open source library needs one version of a logging library and a another uses an alternative version. Gradle and other build tools all have the ability to resolve conflicts, but what differentiates Gradle is the control it gives you over transitive dependencies and conflict resolution.

While you could try to manage this problem manually, you will quickly find that this approach doesn't scale. If you want to get rid of a first level dependency you really can't be sure which other jars you should remove. A dependency of a first level dependency might also be a first level dependency itself, or it might be a transitive dependency of yet another first level dependency. If you try to manage transitive dependencies yourself, the end of the story is that your build becomes brittle: no one dares to change your dependencies because the risk of breaking the build is too high. The project classpath becomes a complete mess, and, if a classpath problem arises, hell on earth invites you for a ride.

NOTE:In one project, we found a mystery LDAP related jar in the classpath. No code referenced this jar and there was no connection to the project. No one could figure out what the jar was for, until it was removed from the build and the application suffered massive performance problems whenever it attempted to authenticate to LDAP. This mystery jar was a necessary transitive, fourth-level dependency that was easy to miss because no one had bothered to use managed transitive dependencies.

Gradle offers you different ways to express first-level and transitive dependencies. With Gradle you can mix and match approaches; for example, you could store your jars in an SCM without XML descriptor files and still use transitive dependency management.

23.2.3. Resolve version conflicts

Conflicting versions of the same jar should be detected and either resolved or cause an exception. If you don't use transitive dependency management, version conflicts are undetected and the often accidental order of the classpath will determine what version of a dependency will win. On a large project with many developers changing dependencies, successful builds will be few and far between as the order of dependencies may directly affect whether a build succeeds or fails (or whether a bug appears or disappears in production).

If you haven't had to deal with the curse of conflicting versions of jars on a classpath, here is a small anecdote of the fun that awaits you. In a large project with 30 submodules, adding a dependency to a subproject changed the order of a classpath, swapping Spring 2.5 for an older 2.4 version. While the build continued to work, developers were starting to notice all sorts of surprising (and surprisingly awful) bugs in production. Worse yet, this unintentional downgrade of Spring introduced several security vulnerabilities into the system, which now required a full security audit throughout the organization.

In short, version conflicts are bad, and you should manage your transitive dependencies to avoid them. You might also want to learn where conflicting versions are used and consolidate on a particular version of a dependency across your organization. With a good conflict reporting tool like Gradle, that information can be used to communicate with the entire organization and standardize on a single version. If you think version conflicts don't happen to you, think again. It is very common for different first-level dependencies to rely on a range of different overlapping versions for other dependencies, and the JVM doesn't yet offer an easy way to have different versions of the same jar in the classpath (see Section 23.1.2, “Dependency management and Java”).

Gradle offers the following conflict resolution strategies:

  • Newest: The newest version of the dependency is used. This is Gradle's default strategy, and is often an appropriate choice as long as versions are backwards-compatible.
  • Fail: A version conflict results in a build failure. This strategy requires all version conflicts to be resolved explicitly in the build script. See ResolutionStrategy for details on how to explicitly choose a particular version.

While the strategies introduced above are usually enough to solve most conflicts, Gradle provides more fine-grained mechanisms to resolve version conflicts:

  • Configuring a first level dependency as forced. This approach is useful if the dependency in conflict is already a first level dependency. See examples in DependencyHandler.
  • Configuring any dependency (transitive or not) as forced. This approach is useful if the dependency in conflict is a transitive dependency. It also can be used to force versions of first level dependencies. See examples in ResolutionStrategy
  • Dependency resolve rules are an incubating feature introduced in Gradle 1.4 which give you fine-grained control over the version selected for a particular dependency.

To deal with problems due to version conflicts, reports with dependency graphs are also very helpful. Such reports are another feature of dependency management.

23.2.4. Use Dynamic Versions and Changing Modules

There are many situations when you want to use the latest version of a particular dependency, or the latest in a range of versions. This can be a requirement during development, or you may be developing a library that is designed to work with a range of dependency versions. You can easily depend on these constantly changing dependencies by using a dynamic version. A dynamic version can be either a version range (e.g. 2.+) or it can be a placeholder for the latest version available (e.g. latest.integration).

Alternatively, sometimes the module you request can change over time, even for the same version. An example of this type of changing module is a Maven SNAPSHOT module, which always points at the latest artifact published. In other words, a standard Maven snapshot is a module that never stands still so to speak, it is a “changing module”.

The main difference between a dynamic version and a changing module is that when you resolve a dynamic version, you'll get the real, static version as the module name. When you resolve a changing module, the artifacts are named using the version you requested, but the underlying artifacts may change over time.

By default, Gradle caches dynamic versions and changing modules for 24 hours. You can override the default cache modes using command line options. You can change the cache expiry times in your build using the resolution strategy (see Section 23.9.3, “Fine-tuned control over dependency caching”).

23.3. Dependency configurations

In Gradle dependencies are grouped into configurations. Configurations have a name, a number of other properties, and they can extend each other. Many Gradle plugins add pre-defined configurations to your project. The Java plugin, for example, adds some configurations to represent the various classpaths it needs. see Section 45.5, “Dependency management” for details. Of course you can add custom configurations on top of that. There are many use cases for custom configurations. This is very handy for example for adding dependencies not needed for building or testing your software (e.g. additional JDBC drivers to be shipped with your distribution).

A project's configurations are managed by a configurations object. The closure you pass to the configurations object is applied against its API. To learn more about this API have a look at ConfigurationContainer.

To define a configuration:

Example 23.1. Definition of a configuration

build.gradle

configurations {
    compile
}

To access a configuration:

Example 23.2. Accessing a configuration

build.gradle

println configurations.compile.name
println configurations['compile'].name

To configure a configuration:

Example 23.3. Configuration of a configuration

build.gradle

configurations {
    compile {
        description = 'compile classpath'
        transitive = true
    }
    runtime {
        extendsFrom compile
    }
}
configurations.compile {
    description = 'compile classpath'
}

23.4. How to declare your dependencies

There are several different types of dependencies that you can declare:

Table 23.1. Dependency types

Type Description
External module dependency A dependency on an external module in some repository.
Project dependency A dependency on another project in the same build.
File dependency A dependency on a set of files on the local filesystem.
Client module dependency A dependency on an external module, where the artifacts are located in some repository but the module meta-data is specified by the local build. You use this kind of dependency when you want to override the meta-data for the module.
Gradle API dependency A dependency on the API of the current Gradle version. You use this kind of dependency when you are developing custom Gradle plugins and task types.
Local Groovy dependency A dependency on the Groovy version used by the current Gradle version. You use this kind of dependency when you are developing custom Gradle plugins and task types.

23.4.1. External module dependencies

External module dependencies are the most common dependencies. They refer to a module in an external repository.

Example 23.4. Module dependencies

build.gradle

dependencies {
    runtime group: 'org.springframework', name: 'spring-core', version: '2.5'
    runtime 'org.springframework:spring-core:2.5',
            'org.springframework:spring-aop:2.5'
    runtime(
        [group: 'org.springframework', name: 'spring-core', version: '2.5'],
        [group: 'org.springframework', name: 'spring-aop', version: '2.5']
    )
    runtime('org.hibernate:hibernate:3.0.5') {
        transitive = true
    }
    runtime group: 'org.hibernate', name: 'hibernate', version: '3.0.5', transitive: true
    runtime(group: 'org.hibernate', name: 'hibernate', version: '3.0.5') {
        transitive = true
    }
}

See the DependencyHandler class in the API documentation for more examples and a complete reference.

Gradle provides different notations for module dependencies. There is a string notation and a map notation. A module dependency has an API which allows further configuration. Have a look at ExternalModuleDependency to learn all about the API. This API provides properties and configuration methods. Via the string notation you can define a subset of the properties. With the map notation you can define all properties. To have access to the complete API, either with the map or with the string notation, you can assign a single dependency to a configuration together with a closure.

If you declare a module dependency, Gradle looks for a module descriptor file (pom.xml or ivy.xml) in the repositories. If such a module descriptor file exists, it is parsed and the artifacts of this module (e.g. hibernate-3.0.5.jar) as well as its dependencies (e.g. cglib) are downloaded. If no such module descriptor file exists, Gradle looks for a file called hibernate-3.0.5.jar to retrieve. In Maven, a module can have one and only one artifact. In Gradle and Ivy, a module can have multiple artifacts. Each artifact can have a different set of dependencies.

23.4.1.1. Depending on modules with multiple artifacts

As mentioned earlier, a Maven module has only one artifact. Hence, when your project depends on a Maven module, it's obvious what its artifact is. With Gradle or Ivy, the case is different. Ivy's dependency descriptor (ivy.xml) can declare multiple artifacts. For more information, see the Ivy reference for ivy.xml. In Gradle, when you declare a dependency on an Ivy module, you actually declare a dependency on the default configuration of that module. So the actual set of artifacts (typically jars) you depend on is the set of artifacts that are associated with the default configuration of that module. Here are some situations where this matters:
  • The default configuration of a module contains undesired artifacts. Rather than depending on the whole configuration, a dependency on just the desired artifacts is declared.
  • The desired artifact belongs to a configuration other than default. That configuration is explicitly named as part of the dependency declaration.
There are other situations where it is necessary to fine-tune dependency declarations. Please see the DependencyHandler class in the API documentation for examples and a complete reference for declaring dependencies.

23.4.1.2. Artifact only notation

As said above, if no module descriptor file can be found, Gradle by default downloads a jar with the name of the module. But sometimes, even if the repository contains module descriptors, you want to download only the artifact jar, without the dependencies. [10] And sometimes you want to download a zip from a repository, that does not have module descriptors. Gradle provides an artifact only notation for those use cases - simply prefix the extension that you want to be downloaded with '@' sign:

Example 23.5. Artifact only notation

build.gradle

dependencies {
    runtime "org.groovy:groovy:2.2.0@jar"
    runtime group: 'org.groovy', name: 'groovy', version: '2.2.0', ext: 'jar'
}


An artifact only notation creates a module dependency which downloads only the artifact file with the specified extension. Existing module descriptors are ignored.

23.4.1.3. Classifiers

The Maven dependency management has the notion of classifiers. [11] Gradle supports this. To retrieve classified dependencies from a Maven repository you can write:

Example 23.6. Dependency with classifier

build.gradle

compile "org.gradle.test.classifiers:service:1.0:jdk15@jar"
otherConf group: 'org.gradle.test.classifiers', name: 'service', version: '1.0', classifier: 'jdk14'

As can be seen in the first line above, classifiers can be used together with the artifact only notation.

It is easy to iterate over the dependency artifacts of a configuration:

Example 23.7. Iterating over a configuration

build.gradle

task listJars << {
    configurations.compile.each { File file -> println file.name }
}

Output of gradle -q listJars

> gradle -q listJars
hibernate-core-3.6.7.Final.jar
antlr-2.7.6.jar
commons-collections-3.1.jar
dom4j-1.6.1.jar
hibernate-commons-annotations-3.2.0.Final.jar
hibernate-jpa-2.0-api-1.0.1.Final.jar
jta-1.1.jar
slf4j-api-1.6.1.jar

23.4.2. Client module dependencies

Client module dependencies allow you to declare transitive dependencies directly in the build script. They are a replacement for a module descriptor in an external repository.

Example 23.8. Client module dependencies - transitive dependencies

build.gradle

dependencies {
    runtime module("org.codehaus.groovy:groovy:2.4.4") {
        dependency("commons-cli:commons-cli:1.0") {
            transitive = false
        }
        module(group: 'org.apache.ant', name: 'ant', version: '1.9.4') {
            dependencies "org.apache.ant:ant-launcher:1.9.4@jar",
                         "org.apache.ant:ant-junit:1.9.4"
        }
    }
}

This declares a dependency on Groovy. Groovy itself has dependencies. But Gradle does not look for an XML descriptor to figure them out but gets the information from the build file. The dependencies of a client module can be normal module dependencies or artifact dependencies or another client module. Also look at the API documentation for the ClientModule class.

In the current release client modules have one limitation. Let's say your project is a library and you want this library to be uploaded to your company's Maven or Ivy repository. Gradle uploads the jars of your project to the company repository together with the XML descriptor file of the dependencies. If you use client modules the dependency declaration in the XML descriptor file is not correct. We will improve this in a future release of Gradle.

23.4.3. Project dependencies

Gradle distinguishes between external dependencies and dependencies on projects which are part of the same multi-project build. For the latter you can declare Project Dependencies.

Example 23.9. Project dependencies

build.gradle

dependencies {
    compile project(':shared')
}

For more information see the API documentation for ProjectDependency.

Multi-project builds are discussed in Chapter 24, Multi-project Builds.

23.4.4. File dependencies

File dependencies allow you to directly add a set of files to a configuration, without first adding them to a repository. This can be useful if you cannot, or do not want to, place certain files in a repository. Or if you do not want to use any repositories at all for storing your dependencies.

To add some files as a dependency for a configuration, you simply pass a file collection as a dependency:

Example 23.10. File dependencies

build.gradle

dependencies {
    runtime files('libs/a.jar', 'libs/b.jar')
    runtime fileTree(dir: 'libs', include: '*.jar')
}

File dependencies are not included in the published dependency descriptor for your project. However, file dependencies are included in transitive project dependencies within the same build. This means they cannot be used outside the current build, but they can be used with the same build.

You can declare which tasks produce the files for a file dependency. You might do this when, for example, the files are generated by the build.

Example 23.11. Generated file dependencies

build.gradle

dependencies {
    compile files("$buildDir/classes") {
        builtBy 'compile'
    }
}

task compile << {
    println 'compiling classes'
}

task list(dependsOn: configurations.compile) << {
    println "classpath = ${configurations.compile.collect {File file -> file.name}}"
}

Output of gradle -q list

> gradle -q list
compiling classes
classpath = [classes]

23.4.5. Gradle API Dependency

You can declare a dependency on the API of the current version of Gradle by using the DependencyHandler.gradleApi() method. This is useful when you are developing custom Gradle tasks or plugins.

Example 23.12. Gradle API dependencies

build.gradle

dependencies {
    compile gradleApi()
}

23.4.6. Local Groovy Dependency

You can declare a dependency on the Groovy that is distributed with Gradle by using the DependencyHandler.localGroovy() method. This is useful when you are developing custom Gradle tasks or plugins in Groovy.

Example 23.13. Gradle's Groovy dependencies

build.gradle

dependencies {
    compile localGroovy()
}

23.4.7. Excluding transitive dependencies

You can exclude a transitive dependency either by configuration or by dependency:

Example 23.14. Excluding transitive dependencies

build.gradle

configurations {
    compile.exclude module: 'commons'
    all*.exclude group: 'org.gradle.test.excludes', module: 'reports'
}

dependencies {
    compile("org.gradle.test.excludes:api:1.0") {
        exclude module: 'shared'
    }
}

If you define an exclude for a particular configuration, the excluded transitive dependency will be filtered for all dependencies when resolving this configuration or any inheriting configuration. If you want to exclude a transitive dependency from all your configurations you can use the Groovy spread-dot operator to express this in a concise way, as shown in the example. When defining an exclude, you can specify either only the organization or only the module name or both. Also look at the API documentation of the Dependency and Configuration classes.

Not every transitive dependency can be excluded - some transitive dependencies might be essential for correct runtime behavior of the application. Generally, one can exclude transitive dependencies that are either not required by runtime or that are guaranteed to be available on the target environment/platform.

Should you exclude per-dependency or per-configuration? It turns out that in the majority of cases you want to use the per-configuration exclusion. Here are some typical reasons why one might want to exclude a transitive dependency. Bear in mind that for some of these use cases there are better solutions than exclusions!

  • The dependency is undesired due to licensing reasons.
  • The dependency is not available in any remote repositories.
  • The dependency is not needed for runtime.
  • The dependency has a version that conflicts with a desired version. For that use case please refer to Section 23.2.3, “Resolve version conflicts” and the documentation on ResolutionStrategy for a potentially better solution to the problem.

Basically, in most of the cases excluding the transitive dependency should be done per configuration. This way the dependency declaration is more explicit. It is also more accurate because a per-dependency exclude rule does not guarantee the given transitive dependency does not show up in the configuration. For example, some other dependency, which does not have any exclude rules, might pull in that unwanted transitive dependency.

Other examples of dependency exclusions can be found in the reference for the ModuleDependency or DependencyHandler classes.

23.4.8. Optional attributes

All attributes for a dependency are optional, except the name. Which attributes are required for actually finding dependencies in the repository will depend on the repository type. See Section 23.6, “Repositories”. For example, if you work with Maven repositories, you need to define the group, name and version. If you work with filesystem repositories you might only need the name or the name and the version.

Example 23.15. Optional attributes of dependencies

build.gradle

dependencies {
    runtime ":junit:4.12", ":testng"
    runtime name: 'testng'
}

You can also assign collections or arrays of dependency notations to a configuration:

Example 23.16. Collections and arrays of dependencies

build.gradle

List groovy = ["org.codehaus.groovy:groovy-all:2.4.4@jar",
               "commons-cli:commons-cli:1.0@jar",
               "org.apache.ant:ant:1.9.4@jar"]
List hibernate = ['org.hibernate:hibernate:3.0.5@jar',
                  'somegroup:someorg:1.0@jar']
dependencies {
    runtime groovy, hibernate
}

23.4.9. Dependency configurations

In Gradle a dependency can have different configurations (as your project can have different configurations). If you don't specify anything explicitly, Gradle uses the default configuration of the dependency. For dependencies from a Maven repository, the default configuration is the only possibility anyway. If you work with Ivy repositories and want to declare a non-default configuration for your dependency you have to use the map notation and declare:

Example 23.17. Dependency configurations

build.gradle

dependencies {
    runtime group: 'org.somegroup', name: 'somedependency', version: '1.0', configuration: 'someConfiguration'
}

To do the same for project dependencies you need to declare:

Example 23.18. Dependency configurations for project

build.gradle

dependencies {
    compile project(path: ':api', configuration: 'spi')
}

23.4.10. Dependency reports

You can generate dependency reports from the command line (see Section 4.6.4, “Listing project dependencies”). With the help of the Project report plugin (see Chapter 27, The Project Report Plugin) such a report can be created by your build.

Since Gradle 1.2 there is also a new programmatic API to access the resolved dependency information. The dependency reports (see the previous paragraph) are using this API under the covers. The API lets you walk the resolved dependency graph and provides information about the dependencies. In future releases the API will grow to provide more information about the resolution result. For more information about the API please refer to the Javadocs on ResolvableDependencies.getResolutionResult(). Potential usages of the ResolutionResult API:

  • Creation of advanced dependency reports tailored to your use case.
  • Enabling the build logic to make decisions based on the content of the dependency graph.

23.5. Working with dependencies

For the examples below we have the following dependencies setup:

Example 23.19. Configuration.copy

build.gradle

configurations {
    sealife
    alllife
}

dependencies {
    sealife "sea.mammals:orca:1.0", "sea.fish:shark:1.0", "sea.fish:tuna:1.0"
    alllife configurations.sealife
    alllife "air.birds:albatross:1.0"
}

The dependencies have the following transitive dependencies:

shark-1.0 -> seal-2.0, tuna-1.0

orca-1.0 -> seal-1.0

tuna-1.0 -> herring-1.0

You can use the configuration to access the declared dependencies or a subset of those:

Example 23.20. Accessing declared dependencies

build.gradle

task dependencies << {
    configurations.alllife.dependencies.each { dep -> println dep.name }
    println()
    configurations.alllife.allDependencies.each { dep -> println dep.name }
    println()
    configurations.alllife.allDependencies.findAll { dep -> dep.name != 'orca' }
        .each { dep -> println dep.name }
}

Output of gradle -q dependencies

> gradle -q dependencies
albatross

albatross
orca
shark
tuna

albatross
shark
tuna

The dependencies task returns only the dependencies belonging explicitly to the configuration. The allDependencies task includes the dependencies from extended configurations.

To get the library files of the configuration dependencies you can do:

Example 23.21. Configuration.files

build.gradle

task allFiles << {
    configurations.sealife.files.each { file ->
        println file.name
    }
}

Output of gradle -q allFiles

> gradle -q allFiles
orca-1.0.jar
shark-1.0.jar
tuna-1.0.jar
herring-1.0.jar
seal-2.0.jar

Sometimes you want the library files of a subset of the configuration dependencies (e.g. of a single dependency).

Example 23.22. Configuration.files with spec

build.gradle

task files << {
    configurations.sealife.files { dep -> dep.name == 'orca' }.each { file ->
        println file.name
    }
}

Output of gradle -q files

> gradle -q files
orca-1.0.jar
seal-2.0.jar

The Configuration.files method always retrieves all artifacts of the whole configuration. It then filters the retrieved files by specified dependencies. As you can see in the example, transitive dependencies are included.

You can also copy a configuration. You can optionally specify that only a subset of dependencies from the original configuration should be copied. The copying methods come in two flavors. The copy method copies only the dependencies belonging explicitly to the configuration. The copyRecursive method copies all the dependencies, including the dependencies from extended configurations.

Example 23.23. Configuration.copy

build.gradle

task copy << {
    configurations.alllife.copyRecursive { dep -> dep.name != 'orca' }
        .allDependencies.each { dep -> println dep.name }
    println()
    configurations.alllife.copy().allDependencies
        .each { dep -> println dep.name }
}

Output of gradle -q copy

> gradle -q copy
albatross
shark
tuna

albatross

It is important to note that the returned files of the copied configuration are often but not always the same than the returned files of the dependency subset of the original configuration. In case of version conflicts between dependencies of the subset and dependencies not belonging to the subset the resolve result might be different.

Example 23.24. Configuration.copy vs. Configuration.files

build.gradle

task copyVsFiles << {
    configurations.sealife.copyRecursive { dep -> dep.name == 'orca' }
        .each { file -> println file.name }
    println()
    configurations.sealife.files { dep -> dep.name == 'orca' }
        .each { file -> println file.name }
}

Output of gradle -q copyVsFiles

> gradle -q copyVsFiles
orca-1.0.jar
seal-1.0.jar

orca-1.0.jar
seal-2.0.jar

In the example above, orca has a dependency on seal-1.0 whereas shark has a dependency onseal-2.0. The original configuration has therefore a version conflict which is resolved to the newer seal-2.0 version. The files method therefore returns seal-2.0 as a transitive dependency oforca. The copied configuration only has orca as a dependency and therefore there is no version conflict and seal-1.0 is returned as a transitive dependency.

Once a configuration is resolved it is immutable. Changing its state or the state of one of its dependencies will cause an exception. You can always copy a resolved configuration. The copied configuration is in the unresolved state and can be freshly resolved.

To learn more about the API of the configuration class see the API documentation: Configuration.

23.6. Repositories

Gradle repository management, based on Apache Ivy, gives you a lot of freedom regarding repository layout and retrieval policies. Additionally Gradle provides various convenience method to add pre-configured repositories.

You may configure any number of repositories, each of which is treated independently by Gradle. If Gradle finds a module descriptor in a particular repository, it will attempt to download all of the artifacts for that module from the same repository. Although module meta-data and module artifacts must be located in the same repository, it is possible to compose a single repository of multiple URLs, giving multiple locations to search for meta-data files and jar files.

There are several different types of repositories you can declare:

Table 23.2. Repository types

Type Description
Maven central repository A pre-configured repository that looks for dependencies in Maven Central.
Maven JCenter repository A pre-configured repository that looks for dependencies in Bintray's JCenter.
Maven local repository A pre-configured repository that looks for dependencies in the local Maven repository.
Maven repository A Maven repository. Can be located on the local filesystem or at some remote location.
Ivy repository An Ivy repository. Can be located on the local filesystem or at some remote location.
Flat directory repository A simple repository on the local filesystem. Does not support any meta-data formats.

23.6.1. Maven central repository

To add the central Maven 2 repository (http://repo1.maven.org/maven2) simply add this to your build script:

Example 23.25. Adding central Maven repository

build.gradle

repositories {
    mavenCentral()
}

Now Gradle will look for your dependencies in this repository.

Warning: Be aware that the central Maven 2 repository is HTTP only and HTTPS is not supported. If you need a public HTTPS enabled central repository, you can use the JCenter public repository (see Section 23.6.2, “Maven JCenter repository”).

23.6.2. Maven JCenter repository

Bintray's JCenter is an up-to-date collection of all popular Maven OSS artifacts, including artifacts published directly to Bintray.

To add the JCenter Maven repository (https://jcenter.bintray.com) simply add this to your build script:

Example 23.26. Adding Bintray's JCenter Maven repository

build.gradle

repositories {
    jcenter()
}

Now Gradle will look for your dependencies in the JCenter repository. jcenter() uses HTTPS to connect to the repository. If you want to use HTTP you can configure jcenter():

Example 23.27. Using Bintrays's JCenter with HTTP

build.gradle

repositories {
    jcenter {
        url "http://jcenter.bintray.com/"
    }
}

23.6.3. Local Maven repository

To use the local Maven cache as a repository you can do:

Example 23.28. Adding the local Maven cache as a repository

build.gradle

repositories {
    mavenLocal()
}

Gradle uses the same logic as Maven to identify the location of your local Maven cache. If a local repository location is defined in a settings.xml, this location will be used. The settings.xml in USER_HOME/.m2 takes precedence over the settings.xml in M2_HOME/conf. If no settings.xml is available, Gradle uses the default location USER_HOME/.m2/repository.

23.6.4. Maven repositories

For adding a custom Maven repository you can do:

Example 23.29. Adding custom Maven repository

build.gradle

repositories {
    maven {
        url "http://repo.mycompany.com/maven2"
    }
}

Sometimes a repository will have the POMs published to one location, and the JARs and other artifacts published at another location. To define such a repository, you can do:

Example 23.30. Adding additional Maven repositories for JAR files

build.gradle

repositories {
    maven {
        // Look for POMs and artifacts, such as JARs, here
        url "http://repo2.mycompany.com/maven2"
        // Look for artifacts here if not found at the above location
        artifactUrls "http://repo.mycompany.com/jars"
        artifactUrls "http://repo.mycompany.com/jars2"
    }
}

Gradle will look at the first URL for the POM and the JAR. If the JAR can't be found there, the artifact URLs are used to look for JARs.

23.6.4.1. Accessing password protected Maven repositories

To access a Maven repository which uses basic authentication, you specify the username and password to use when you define the repository:

Example 23.31. Accessing password protected Maven repository

build.gradle

repositories {
    maven {
        credentials {
            username 'user'
            password 'password'
        }
        url "http://repo.mycompany.com/maven2"
    }
}

It is advisable to keep your username and password in gradle.properties rather than directly in the build file.

23.6.5. Flat directory repository

If you want to use a (flat) filesystem directory as a repository, simply type:

Example 23.32. Flat repository resolver

build.gradle

repositories {
    flatDir {
        dirs 'lib'
    }
    flatDir {
        dirs 'lib1', 'lib2'
    }
}

This adds repositories which look into one or more directories for finding dependencies. Note that this type of repository does not support any meta-data formats like Ivy XML or Maven POM files. Instead, Gradle will dynamically generate a module descriptor (without any dependency information) based on the presence of artifacts. However, as Gradle prefers to use modules whose descriptor has been created from real meta-data rather than being generated, flat directory repositories cannot be used to override artifacts with real meta-data from other repositories. So, for example, if Gradle finds only jmxri-1.2.1.jar in a flat directory repository, but jmxri-1.2.1.pom in another repository that supports meta-data, it will use the second repository to provide the module. For the use case of overriding remote artifacts with local ones consider using an Ivy or Maven repository instead whose URL points to a local directory. If you only work with flat directory repositories you don't need to set all attributes of a dependency. See Section 23.4.8, “Optional attributes”.

23.6.6. Ivy repositories

23.6.6.1. Defining an Ivy repository with a standard layout

Example 23.33. Ivy repository

build.gradle

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
    }
}

23.6.6.2. Defining a named layout for an Ivy repository

You can specify that your repository conforms to the Ivy or Maven default layout by using a named layout.

Example 23.34. Ivy repository with named layout

build.gradle

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout "maven"
    }
}

Valid named layout values are 'gradle' (the default), 'maven', 'ivy' and 'pattern'. See IvyArtifactRepository.layout() in the API documentation for details of these named layouts.

23.6.6.3. Defining custom pattern layout for an Ivy repository

To define an Ivy repository with a non-standard layout, you can define a 'pattern' layout for the repository:

Example 23.35. Ivy repository with pattern layout

build.gradle

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout "pattern", {
            artifact "[module]/[revision]/[type]/[artifact].[ext]"
        }
    }
}

To define an Ivy repository which fetches Ivy files and artifacts from different locations, you can define separate patterns to use to locate the Ivy files and artifacts:

Each artifact or ivy specified for a repository adds an additional pattern to use. The patterns are used in the order that they are defined.

Example 23.36. Ivy repository with multiple custom patterns

build.gradle

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout "pattern", {
            artifact "3rd-party-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
            artifact "company-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
            ivy "ivy-files/[organisation]/[module]/[revision]/ivy.xml"
        }
    }
}

Optionally, a repository with pattern layout can have its 'organisation' part laid out in Maven style, with forward slashes replacing dots as separators. For example, the organisation my.company would then be represented as my/company.

Example 23.37. Ivy repository with Maven compatible layout

build.gradle

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout "pattern", {
            artifact "[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
            m2compatible = true
        }
    }
}

23.6.6.4. Accessing password protected Ivy repositories

To access an Ivy repository which uses basic authentication, you specify the username and password to use when you define the repository:

Example 23.38. Ivy repository

build.gradle

repositories {
    ivy {
        url 'http://repo.mycompany.com'
        credentials {
            username 'user'
            password 'password'
        }
    }
}

23.6.7. Supported repository transport protocols

Maven and Ivy repositories support the use of various transport protocols. At the moment the following protocols are supported:

Table 23.3. Repository transport protocols

Type Credential types
file none
http username/password
https username/password
sftp username/password
s3 access key/secret key

To define a repository use the repositories configuration block. Within the repositories closure, a Maven repository is declared with maven. An Ivy repository is declared with ivy. The transport protocol is part of the URL definition for a repository. The following build script demonstrates how to create a HTTP-based Maven and Ivy repository:

Example 23.39. Declaring a Maven and Ivy repository

build.gradle

repositories {
    maven {
        url "http://repo.mycompany.com/maven2"
    }

    ivy {
        url "http://repo.mycompany.com/repo"
    }
}

If authentication is required for a repository, the relevant credentials can be provided. The following example shows how to provide username/password-based authentication for SFTP repositories:

Example 23.40. Providing credentials to a Maven and Ivy repository

build.gradle

repositories {
    maven {
        url "sftp://repo.mycompany.com:22/maven2"
        credentials {
            username 'user'
            password 'password'
        }
    }

    ivy {
        url "sftp://repo.mycompany.com:22/repo"
        credentials {
            username 'user'
            password 'password'
        }
    }
}

When using an AWS S3 backed repository you need to authenticate using AwsCredentials, providing access-key and a private-key. The following example shows how to declare a S3 backed repository and providing AWS credentials:

Example 23.41. Declaring a S3 backed Maven and Ivy repository

build.gradle

repositories {
    maven {
        url "s3://myCompanyBucket/maven2"
        credentials(AwsCredentials) {
            accessKey "someKey"
            secretKey "someSecret"
        }
    }

    ivy {
        url "s3://myCompanyBucket/ivyrepo"
        credentials(AwsCredentials) {
            accessKey "someKey"
            secretKey "someSecret"
        }
    }
}

23.6.7.1. S3 configuration properties

The following system properties can be used to configure the interactions with s3 repositories:

Table 23.4. S3 Configuration Properties

Property Description
org.gradle.s3.endpoint Used to override the AWS S3 endpoint when using a non AWS, S3 API compatible, storage service.
org.gradle.s3.maxErrorRetry Specifies the maximum number of times to retry a request in the event that the S3 server responds with a HTTP 5xx status code. When not specified a default value of 3 is used.

23.6.7.2. S3 URL formats

S3 URL's are 'virtual-hosted-style' and must be in the following format s3://<bucketName>[.<regionSpecificEndpoint>]/<s3Key>

e.g. s3://myBucket.s3.eu-central-1.amazonaws.com/maven/release

  • myBucket is the AWS S3 bucket name.
  • s3.eu-central-1.amazonaws.com is the optional region specific endpoint.
  • /maven/release is the AWS S3 key (unique identifier for an object within a bucket)

23.6.7.3. S3 proxy settings

A proxy for S3 can be configured using the following system properties:

  • https.proxyHost
  • https.proxyPort
  • https.proxyUser
  • https.proxyPassword
  • https.nonProxyHosts

If the 'org.gradle.s3.endpoint' property has been specified with a http (not https) URI the following system proxy settings can be used:

  • http.proxyHost
  • http.proxyPort
  • http.proxyUser
  • http.proxyPassword
  • http.nonProxyHosts

23.6.7.4. AWS S3 V4 Signatures (AWS4-HMAC-SHA256)

Some of the AWS S3 regions (eu-central-1 - Frankfurt) require that all HTTP requests are signed in accordance with AWS's signature version 4. It is recommended to specify S3 URL's containing the region specific endpoint when using buckets that require V4 signatures. e.g. s3://somebucket.s3.eu-central-1.amazonaws.com/maven/release

NOTE: When a region-specific endpoint is not specified for buckets requiring V4 Signatures, Gradle will use the default AWS region (us-east-1) and the following warning will appear on the console:

Attempting to re-send the request to .... with AWS V4 authentication. To avoid this warning in the future, please use region-specific endpoint to access buckets located in regions that require V4 signing.

Failing to specify the region-specific endpoint for buckets requiring V4 signatures means:

  • 3 round-trips to AWS, as opposed to one, for every file upload and download.
  • Depending on location - increased network latencies and slower builds.
  • Increased likelihood of transmission failures.

23.6.7.5. Configuring HTTP authentication schemes

When configuring a repository using HTTP or HTTPS transport protocols, multiple authentication schemes are available. By default, Gradle will attempt to use all schemes that are supported by the Apache HttpClient library, documented here. In some cases, it may be preferable to explicitly specify which authentication schemes should be used when exchanging credentials with a remote server. When explicitly declared, only those schemes are used when authenticating to a remote repository. The following example show how to configure a repository to use only digest authentication:

Example 23.42. Configure repository to use only digest authentication

build.gradle

repositories {
    maven {
        url 'https://repo.mycompany.com/maven2'
        credentials {
            username 'user'
            password 'password'
        }
        authentication {
            digest(DigestAuthentication)
        }
    }
}

Currently supported authentication schemes are:

Table 23.5. Authentication schemes

Type Description
BasicAuthentication Basic access authentication over HTTP. When using this scheme, credentials are sent preemptively.
DigestAuthentication Digest access authentication over HTTP.

23.6.7.6. Using preemptive authentication

Gradle's default behavior is to only submit credentials when a server responds with an authentication challenge in the form of a HTTP 401 response. In some cases, the server will respond with a different code (ex. for repositories hosted on GitHub a 404 is returned) causing dependency resolution to fail. To get around this behavior, credentials may be sent to the server preemptively. To enable preemptive authentication simply configure your repository to explicitly use the BasicAuthentication scheme:

Example 23.43. Configure repository to use preemptive authentication

build.gradle

repositories {
    maven {
        url 'https://repo.mycompany.com/maven2'
        credentials {
            username 'user'
            password 'password'
        }
        authentication {
            basic(BasicAuthentication)
        }
    }
}

23.6.8. Working with repositories

To access a repository:

Example 23.44. Accessing a repository

build.gradle

println repositories.localRepository.name
println repositories['localRepository'].name

To configure a repository:

Example 23.45. Configuration of a repository

build.gradle

repositories {
    flatDir {
        name 'localRepository'
    }
}
repositories {
    localRepository {
        dirs 'lib'
    }
}
repositories.localRepository {
    dirs 'lib'
}

23.6.9. More about Ivy resolvers

Gradle is extremely flexible regarding repositories:

  • There are many options for the protocol to communicate with the repository (e.g. filesystem, http, ssh, sftp ...)

  • The protocol sftp currently only supports username/password-based authentication.

  • Each repository can have its own layout.

Let's say, you declare a dependency on the junit:junit:3.8.2 library. Now how does Gradle find it in the repositories? Somehow the dependency information has to be mapped to a path. In contrast to Maven, where this path is fixed, with Gradle you can define a pattern that defines what the path will look like. Here are some examples: [12]

// Maven2 layout (if a repository is marked as Maven2 compatible, the organization (group) is split into subfolders according to the dots.)
someroot/[organisation]/[module]/[revision]/[module]-[revision].[ext]

// Typical layout for an Ivy repository (the organization is not split into subfolder)
someroot/[organisation]/[module]/[revision]/[type]s/[artifact].[ext]

// Simple layout (the organization is not used, no nested folders.)
someroot/[artifact]-[revision].[ext]

To add any kind of repository (you can pretty easy write your own ones) you can do:

Example 23.46. Definition of a custom repository

build.gradle

repositories {
    ivy {
        ivyPattern "$projectDir/repo/[organisation]/[module]-ivy-[revision].xml"
        artifactPattern "$projectDir/repo/[organisation]/[module]-[revision](-[classifier]).[ext]"
    }
}

An overview of which Resolvers are offered by Ivy and thus also by Gradle can be found here. With Gradle you just don't configure them via XML but directly via their API.

23.7. How dependency resolution works

Gradle takes your dependency declarations and repository definitions and attempts to download all of your dependencies by a process called dependency resolution. Below is a brief outline of how this process works.

  • Given a required dependency, Gradle first attempts to resolve the module for that dependency. Each repository is inspected in order, searching first for a module descriptor file (POM or Ivy file) that indicates the presence of that module. If no module descriptor is found, Gradle will search for the presence of the primary module artifact file indicating that the module exists in the repository.

    • If the dependency is declared as a dynamic version (like 1.+), Gradle will resolve this to the newest available static version (like 1.2) in the repository. For Maven repositories, this is done using the maven-metadata.xml file, while for Ivy repositories this is done by directory listing.

    • If the module descriptor is a POM file that has a parent POM declared, Gradle will recursively attempt to resolve each of the parent modules for the POM.

  • Once each repository has been inspected for the module, Gradle will choose the 'best' one to use. This is done using the following criteria:

    • For a dynamic version, a 'higher' static version is preferred over a 'lower' version.
    • Modules declared by a module descriptor file (Ivy or POM file) are preferred over modules that have an artifact file only.
    • Modules from earlier repositories are preferred over modules in later repositories.

    When the dependency is declared by a static version and a module descriptor file is found in a repository, there is no need to continue searching later repositories and the remainder of the process is short-circuited.

  • All of the artifacts for the module are then requested from the same repository that was chosen in the process above.

23.8. Fine-tuning the dependency resolution process

In most cases, Gradle's default dependency management will resolve the dependencies that you want in your build. In some cases, however, it can be necessary to tweak dependency resolution to ensure that your build receives exactly the right dependencies.

There are a number of ways that you can influence how Gradle resolves dependencies.

23.8.1. Forcing a particular module version

Forcing a module version tells Gradle to always use a specific version for given dependency (transitive or not), overriding any version specified in a published module descriptor. This can be very useful when tackling version conflicts - for more information see Section 23.2.3, “Resolve version conflicts”.

Force versions can also be used to deal with rogue metadata of transitive dependencies. If a transitive dependency has poor quality metadata that leads to problems at dependency resolution time, you can force Gradle to use a newer, fixed version of this dependency. For an example, see the ResolutionStrategy class in the API documentation. Note that 'dependency resolve rules' (outlined below) provide a more powerful mechanism for replacing a broken module dependency. See Section 23.8.2.3, “Blacklisting a particular version with a replacement”.

23.8.2. Using dependency resolve rules

A dependency resolve rule is executed for each resolved dependency, and offers a powerful api for manipulating a requested dependency prior to that dependency being resolved. This feature is incubating, but currently offers the ability to change the group, name and/or version of a requested dependency, allowing a dependency to be substituted with a completely different module during resolution.

Dependency resolve rules provide a very powerful way to control the dependency resolution process, and can be used to implement all sorts of advanced patterns in dependency management. Some of these patterns are outlined below. For more information and code samples see the ResolutionStrategy class in the API documentation.

23.8.2.1. Modelling releaseable units

Often an organisation publishes a set of libraries with a single version; where the libraries are built, tested and published together. These libraries form a 'releasable unit', designed and intended to be used as a whole. It does not make sense to use libraries from different releasable units together.

But it is easy for transitive dependency resolution to violate this contract. For example:

  • module-a depends on releasable-unit:part-one:1.0
  • module-b depends on releasable-unit:part-two:1.1

A build depending on both module-a and module-b will obtain different versions of libraries within the releasable unit.

Dependency resolve rules give you the power to enforce releasable units in your build. Imagine a releasable unit defined by all libraries that have 'org.gradle' group. We can force all of these libraries to use a consistent version:

Example 23.47. Forcing consistent version for a group of libraries

build.gradle

configurations.all {
    resolutionStrategy.eachDependency { DependencyResolveDetails details ->
        if (details.requested.group == 'org.gradle') {
            details.useVersion '1.4'
        }
    }
}


23.8.2.2. Implement a custom versioning scheme

In some corporate environments, the list of module versions that can be declared in Gradle builds is maintained and audited externally. Dependency resolve rules provide a neat implementation of this pattern:

  • In the build script, the developer declares dependencies with the module group and name, but uses a placeholder version, for example: 'default'.
  • The 'default' version is resolved to a specific version via a dependency resolve rule, which looks up the version in a corporate catalog of approved modules.

This rule implementation can be neatly encapsulated in a corporate plugin, and shared across all builds within the organisation.

Example 23.48. Using a custom versioning scheme

build.gradle

configurations.all {
    resolutionStrategy.eachDependency { DependencyResolveDetails details ->
        if (details.requested.version == 'default') {
            def version = findDefaultVersionInCatalog(details.requested.group, details.requested.name)
            details.useVersion version
        }
    }
}

def findDefaultVersionInCatalog(String group, String name) {
    //some custom logic that resolves the default version into a specific version
    "1.0"
}


23.8.2.3. Blacklisting a particular version with a replacement

Dependency resolve rules provide a mechanism for blacklisting a particular version of a dependency and providing a replacement version. This can be useful if a certain dependency version is broken and should not be used, where a dependency resolve rule causes this version to be replaced with a known good version. One example of a broken module is one that declares a dependency on a library that cannot be found in any of the public repositories, but there are many other reasons why a particular module version is unwanted and a different version is preferred.

In example below, imagine that version 1.2.1 contains important fixes and should always be used in preference to 1.2. The rule provided will enforce just this: any time version 1.2 is encountered it will be replaced with 1.2.1. Note that this is different from a forced version as described above, in that any other versions of this module would not be affected. This means that the 'newest' conflict resolution strategy would still select version 1.3 if this version was also pulled transitively.

Example 23.49. Blacklisting a version with a replacement

build.gradle

configurations.all {
    resolutionStrategy.eachDependency { DependencyResolveDetails details ->
        if (details.requested.group == 'org.software' && details.requested.name == 'some-library' && details.requested.version == '1.2') {
            //prefer different version which contains some necessary fixes
            details.useVersion '1.2.1'
        }
    }
}


23.8.2.4. Substituting a dependency module with a compatible replacement

At times a completely different module can serve as a replacement for a requested module dependency. Examples include using 'groovy' in place of 'groovy-all', or using 'log4j-over-slf4j' instead of 'log4j'. Starting with Gradle 1.5 you can make these substitutions using dependency resolve rules:

Example 23.50. Changing dependency group and/or name at the resolution

build.gradle

configurations.all {
    resolutionStrategy.eachDependency { DependencyResolveDetails details ->
        if (details.requested.name == 'groovy-all') {
            //prefer 'groovy' over 'groovy-all':
            details.useTarget group: details.requested.group, name: 'groovy', version: details.requested.version
        }
        if (details.requested.name == 'log4j') {
            //prefer 'log4j-over-slf4j' over 'log4j', with fixed version:
            details.useTarget "org.slf4j:log4j-over-slf4j:1.7.10"
        }
    }
}


23.8.3. Dependency Substitution Rules

Dependency substitution rules work similarly to dependency resolve rules. In fact, many capabilities of dependency resolve rules can be implemented with dependency substitution rules. They allow project and module dependencies to be transparently substituted with specified replacements. Unlike dependency resolve rules, dependency substitution rules allow project and module dependencies to be substituted interchangeably.

NOTE: Adding a dependency substitution rule to a configuration changes the timing of when that configuration is resolved. Instead of being resolved on first use, the configuration is instead resolved when the task graph is being constructed. This can have unexpected consequences if the configuration is being further modified during task execution, or if the configuration relies on modules that are published during execution of another task.

To explain:

  • A Configuration can be declared as an input to any Task, and that configuration can include project dependencies when it is resolved.
  • If a project dependency is an input to a Task (via a configuration), then tasks to built the project artifacts must be added to the task dependencies.
  • In order to determine the project dependencies that are inputs to a task, Gradle needs to resolve the Configuration inputs.
  • Because the Gradle task graph is fixed once task execution has commenced, Gradle needs to perform this resolution prior to executing any tasks.

In the absence of dependency substitution rules, Gradle knows that an external module dependency will never transitively reference a project dependency. This makes it easy to determine the full set of project dependencies for a configuration through simple graph traversal. With this functionality, Gradle can no longer make this assumption, and must perform a full resolve in order to determine the project dependencies.

23.8.3.1. Substituting an external module dependency with a project dependency

One use case for dependency substitution is to use a locally developed version of a module in place of one that is downloaded from an external repository. This could be useful for testing a local, patched version of a dependency.

The module to be replaced can be declared with or without a version specified.

Example 23.51. Substituting a module with a project

build.gradle

configurations.all {
    resolutionStrategy.dependencySubstitution {
        substitute module("org.utils:api") with project(":api")
        substitute module("org.utils:util:2.5") with project(":util")
    }
}

Note that a project that is substituted must be included in the multi-project build (via settings.gradle). Dependency substitution rules take care of replacing the module dependency with the project dependency and wiring up any task dependencies, but do not implicitly include the project in the build.

23.8.3.2. Substituting a project dependency with a module replacement

Another way to use substitution rules is to replace a project dependency with a module in a multi-project build. This can be useful to speed up development with a large multi-project build, by allowing a subset of the project dependencies to be downloaded from a repository rather than being built.

The module to be used as a replacement must be declared with a version specified.

Example 23.52. Substituting a project with a module

build.gradle

configurations.all {
    resolutionStrategy.dependencySubstitution {
        substitute project(":api") with module("org.utils:api:1.3")
    }
}

When a project dependency has been replaced with a module dependency, that project is still included in the overall multi-project build. However, tasks to build the replaced dependency will not be executed in order to build the resolve the depending Configuration.

23.8.3.3. Conditionally substituting a dependency

A common use case for dependency substitution is to allow more flexible assembly of sub-projects within a multi-project build. This can be useful for developing a local, patched version of an external dependency or for building a subset of the modules within a large multi-project build.

The following example uses a dependency substitution rule to replace any module dependency with the group "org.example", but only if a local project matching the dependency name can be located.

Example 23.53. Conditionally substituting a dependency

build.gradle

configurations.all {
    resolutionStrategy.dependencySubstitution.all { DependencySubstitution dependency ->
        if (dependency.requested instanceof ModuleComponentSelector && dependency.requested.group == "org.example") {
            def targetProject = findProject(":${dependency.requested.module}")
            if (targetProject != null) {
                dependency.useTarget targetProject
            }
        }
    }
}

Note that a project that is substituted must be included in the multi-project build (via settings.gradle). Dependency substitution rules take care of replacing the module dependency with the project dependency, but do not implicitly include the project in the build.

23.8.4. Specifying default dependencies for a configuration

A configuration can be configured with default dependencies to be used if no dependencies are explicitly set for the configuration. A primary use case of this functionality is for developing plugins that make use of versioned tools that the user might override. By specifying default dependencies, the plugin can use a default version of the tool only if the user has not specified a particular version to use.

Example 23.54. Specifying default dependencies on a configuration

build.gradle

configurations {
    pluginTool {
        defaultDependencies { dependencies ->
            dependencies.add(this.project.dependencies.create("org.gradle:my-util:1.0"))
        }
    }
}

23.8.5. Enabling Ivy dynamic resolve mode

Gradle's Ivy repository implementations support the equivalent to Ivy's dynamic resolve mode. Normally, Gradle will use the rev attribute for each dependency definition included in an ivy.xml file. In dynamic resolve mode, Gradle will instead prefer the revConstraint attribute over the rev attribute for a given dependency definition. If the revConstraint attribute is not present, the rev attribute is used instead.

To enable dynamic resolve mode, you need to set the appropriate option on the repository definition. A couple of examples are shown below. Note that dynamic resolve mode is only available for Gradle's Ivy repositories. It is not available for Maven repositories, or custom Ivy DependencyResolver implementations.

Example 23.55. Enabling dynamic resolve mode

build.gradle

// Can enable dynamic resolve mode when you define the repository
repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        resolve.dynamicMode = true
    }
}

// Can use a rule instead to enable (or disable) dynamic resolve mode for all repositories
repositories.withType(IvyArtifactRepository) {
    resolve.dynamicMode = true
}

23.8.6. Component metadata rules

Each module (also called component) has metadata associated with it, such as its group, name, version, dependencies, and so on. This metadata typically originates in the module's descriptor. Metadata rules allow certain parts of a module's metadata to be manipulated from within the build script. They take effect after a module's descriptor has been downloaded, but before it has been selected among all candidate versions. This makes metadata rules another instrument for customizing dependency resolution.

One piece of module metadata that Gradle understands is a module's status scheme. This concept, also known from Ivy, models the different levels of maturity that a module transitions through over time. The default status scheme, ordered from least to most mature status, is integration, milestone, release. Apart from a status scheme, a module also has a (current) status, which must be one of the values in its status scheme. If not specified in the (Ivy) descriptor, the status defaults to integration for Ivy modules and Maven snapshot modules, and release for Maven modules that aren't snapshots.

A module's status and status scheme are taken into consideration when a latest version selector is resolved. Specifically, latest.someStatus will resolve to the highest module version that has status someStatus or a more mature status. For example, with the default status scheme in place, latest.integration will select the highest module version regardless of its status (because integration is the least mature status), whereas latest.release will select the highest module version with status release. Here is what this looks like in code:

Example 23.56. 'Latest' version selector

build.gradle

dependencies {
    config1 "org.sample:client:latest.integration"
    config2 "org.sample:client:latest.release"
}

task listConfigs << {
    configurations.config1.each { println it.name }
    println()
    configurations.config2.each { println it.name}
}

Output of gradle -q listConfigs

> gradle -q listConfigs
client-1.5.jar

client-1.4.jar

The next example demonstrates latest selectors based on a custom status scheme declared in a component metadata rule that applies to all modules:

Example 23.57. Custom status scheme

build.gradle

dependencies {
    config3 "org.sample:api:latest.silver"
    components {
        all { ComponentMetadataDetails details ->
            if (details.id.group == "org.sample" && details.id.name == "api") {
                details.statusScheme = ["bronze", "silver", "gold", "platinum"]
            }
        }
    }
}

Component metadata rules can be applied to a specified module. Modules must be specified in the form of "group:module".

Example 23.58. Custom status scheme by module

build.gradle

dependencies {
    config4 "org.sample:lib:latest.prod"
    components {
        withModule('org.sample:lib') { ComponentMetadataDetails details ->
            details.statusScheme = ["int", "rc", "prod"]
        }
    }
}

Gradle can also create component metadata rules utilizing Ivy-specific metadata for modules resolved from an Ivy repository. Values from the Ivy descriptor are made available via the IvyModuleDescriptor interface.

Example 23.59. Ivy component metadata rule

build.gradle

dependencies {
    config6 "org.sample:lib:latest.rc"
    components {
        withModule("org.sample:lib") { ComponentMetadataDetails details, IvyModuleDescriptor ivyModule ->
            if (ivyModule.branch == 'testing') {
                details.status = "rc"
            }
        }
    }
}

Note that any rule that declares specific arguments must always include a ComponentMetadataDetails argument as the first argument. The second Ivy metadata argument is optional.

Component metadata rules can also be defined using a rule source object. A rule source object is any object that contains exactly one method that defines the rule action and is annotated with @Mutate.

This method:

Example 23.60. Rule source component metadata rule

build.gradle

dependencies {
    config5 "org.sample:api:latest.gold"
    components {
        withModule('org.sample:api', new CustomStatusRule())
    }
}

class CustomStatusRule {
    @Mutate
    void setStatusScheme(ComponentMetadataDetails details) {
        details.statusScheme = ["bronze", "silver", "gold", "platinum"]
    }
}

23.8.7. Component Selection Rules

Component selection rules may influence which component instance should be selected when multiple versions are available that match a version selector. Rules are applied against every available version and allow the version to be explicitly rejected by rule. This allows Gradle to ignore any component instance that does not satisfy conditions set by the rule. Examples include:

  • For a dynamic version like '1.+' certain versions may be explicitly rejected from selection
  • For a static version like '1.4' an instance may be rejected based on extra component metadata such as the Ivy branch attribute, allowing an instance from a subsequent repository to be used.

Rules are configured via the ComponentSelectionRules object. Each rule configured will be called with a ComponentSelection object as an argument which contains information about the candidate version being considered. Calling ComponentSelection.reject() causes the given candidate version to be explicitly rejected, in which case the candidate will not be considered for the selector.

The following example shows a rule that disallows a particular version of a module but allows the dynamic version to choose the next best candidate.

Example 23.61. Component selection rule

build.gradle

configurations {
    rejectConfig {
        resolutionStrategy {
            componentSelection {
                // Accept the highest version matching the requested version that isn't '1.5'
                all { ComponentSelection selection ->
                    if (selection.candidate.group == 'org.sample' && selection.candidate.module == 'api' && selection.candidate.version == '1.5') {
                        selection.reject("version 1.5 is broken for 'org.sample:api'")
                    }
                }
            }
        }
    }
}

dependencies {
    rejectConfig "org.sample:api:1.+"
}

Note that version selection is applied starting with the highest version first. The version selected will be the first version found that all component selection rules accept. A version is considered accepted no rule explicitly rejects it.

Similarly, rules can be targeted at specific modules. Modules must be specified in the form of "group:module".

Example 23.62. Component selection rule with module target

build.gradle

configurations {
    targetConfig {
        resolutionStrategy {
            componentSelection {
                withModule("org.sample:api") { ComponentSelection selection ->
                    if (selection.candidate.version == "1.5") {
                        selection.reject("version 1.5 is broken for 'org.sample:api'")
                    }
                }
            }
        }
    }
}

Component selection rules can also consider component metadata when selecting a version. Possible metadata arguments that can be considered are ComponentMetadata and IvyModuleDescriptor.

Example 23.63. Component selection rule with metadata

build.gradle

configurations {
    metadataRulesConfig {
        resolutionStrategy {
            componentSelection {
                // Reject any versions with a status of 'experimental'
                all { ComponentSelection selection, ComponentMetadata metadata ->
                    if (selection.candidate.group == 'org.sample' && metadata.status == 'experimental') {
                        selection.reject("don't use experimental candidates from 'org.sample'")
                    }
                }
                // Accept the highest version with either a "release" branch or a status of 'milestone'
                withModule('org.sample:api') { ComponentSelection selection, IvyModuleDescriptor descriptor, ComponentMetadata metadata ->
                    if (descriptor.branch != "release" && metadata.status != 'milestone') {
                        selection.reject("'org.sample:api' must have testing branch or milestone status")
                    }
                }
            }
        }
    }
}

Note that a ComponentSelection argument is always required as the first parameter when declaring a component selection rule with additional Ivy metadata parameters, but the metadata parameters can be declared in any order.

Lastly, component selection rules can also be defined using a rule source object. A rule source object is any object that contains exactly one method that defines the rule action and is annotated with @Mutate.

This method:

Example 23.64. Component selection rule using a rule source object

build.gradle

class RejectTestBranch {
    @Mutate
    void evaluateRule(ComponentSelection selection, IvyModuleDescriptor ivy) {
        if (ivy.branch == "test") {
            selection.reject("reject test branch")
        }
    }
}

configurations {
    ruleSourceConfig {
        resolutionStrategy {
            componentSelection {
                all new RejectTestBranch()
            }
        }
    }
}

23.8.8. Module replacement rules

Module replacement rules allow a build to declare that a legacy library has been replaced by a new one. A good example when a new library replaced a legacy one is the "google-collections" -> "guava" migration. The team that created google-collections decided to change the module name from "com.google.collections:google-collections" into "com.google.guava:guava". This a legal scenario in the industry: teams need to be able to change the names of products they maintain, including the module coordinates. Renaming of the module coordinates has impact on conflict resolution.

To explain the impact on conflict resolution, let's consider the "google-collections" -> "guava" scenario. It may happen that both libraries are pulled into the same dependency graph. For example, "our" project depends on guava but some of our dependencies pull in a legacy version of google-collections. This can cause runtime errors, for example during test or application execution. Gradle does not automatically resolve the google-collections VS guava conflict because it is not considered as a "version conflict". It's because the module coordinates for both libraries are completely different and conflict resolution is activated when "group" and "name" coordinates are the same but there are different versions available in the dependency graph (for more info, please refer to the section on conflict resolution). Traditional remedies to this problem are:

  • Declare exclusion rule to avoid pulling in "google-collections" to graph. It is probably the most popular approach.
  • Avoid dependencies that pull in legacy libraries.
  • Upgrade the dependency version if the new version no longer pulls in a legacy library.
  • Downgrade to "google-collections". It's not recommended, just mentioned for completeness.

Traditional approaches work but they are not general enough. For example, an organisation wants to resolve the google-collections VS guava conflict resolution problem in all projects. Starting from Gradle 2.2 it is possible to declare that certain module was replaced by other. This enables organisations to include the information about module replacement in the corporate plugin suite and resolve the problem holistically for all Gradle-powered projects in the enterprise.

Example 23.65. Declaring module replacement

build.gradle

dependencies {
    modules {
        module("com.google.collections:google-collections") {
            replacedBy("com.google.guava:guava")
        }
    }
}


For more examples and detailed API, please refer to the DSL reference for ComponentMetadataHandler.

What happens when we declare that "google-collections" are replaced by "guava"? Gradle can use this information for conflict resolution. Gradle will consider every version of "guava" newer/better than any version of "google-collections". Also, Gradle will ensure that only guava jar is present in the classpath / resolved file list. Please note that if only "google-collections" appears in the dependency graph (e.g. no "guava") Gradle will not eagerly replace it with "guava". Module replacement is an information that Gradle uses for resolving conflicts. If there is no conflict (e.g. only "google-collections" or only "guava" in the graph) the replacement information is not used.

Currently it is not possible to declare that certain modules is replaced by a set of modules. However, it is possible to declare that multiple modules are replaced by a single module.

23.9. The dependency cache

Gradle contains a highly sophisticated dependency caching mechanism, which seeks to minimise the number of remote requests made in dependency resolution, while striving to guarantee that the results of dependency resolution are correct and reproducible.

The Gradle dependency cache consists of 2 key types of storage:

  • A file-based store of downloaded artifacts, including binaries like jars as well as raw downloaded meta-data like POM files and Ivy files. The storage path for a downloaded artifact includes the SHA1 checksum, meaning that 2 artifacts with the same name but different content can easily be cached.

  • A binary store of resolved module meta-data, including the results of resolving dynamic versions, module descriptors, and artifacts.

Separating the storage of downloaded artifacts from the cache metadata permits us to do some very powerful things with our cache that would be difficult with a transparent, file-only cache layout.

The Gradle cache does not allow the local cache to hide problems and create other mysterious and difficult to debug behavior that has been a challenge with many build tools. This new behavior is implemented in a bandwidth and storage efficient way. In doing so, Gradle enables reliable and reproducible enterprise builds.

23.9.1. Key features of the Gradle dependency cache

23.9.1.1. Separate metadata cache

Gradle keeps a record of various aspects of dependency resolution in binary format in the metadata cache. The information stored in the metadata cache includes:

  • The result of resolving a dynamic version (e.g. 1.+) to a concrete version (e.g. 1.2).
  • The resolved module metadata for a particular module, including module artifacts and module dependencies.
  • The resolved artifact metadata for a particular artifact, including a pointer to the downloaded artifact file.
  • The absence of a particular module or artifact in a particular repository, eliminating repeated attempts to access a resource that does not exist.

Every entry in the metadata cache includes a record of the repository that provided the information as well as a timestamp that can be used for cache expiry.

23.9.1.2. Repository caches are independent

As described above, for each repository there is a separate metadata cache. A repository is identified by its URL, type and layout. If a module or artifact has not been previously resolved from this repository, Gradle will attempt to resolve the module against the repository. This will always involve a remote lookup on the repository, however in many cases no download will be required (seeSection 23.9.1.3, “Artifact reuse”, below).

Dependency resolution will fail if the required artifacts are not available in any repository specified by the build, even if the local cache has a copy of this artifact which was retrieved from a different repository. Repository independence allows builds to be isolated from each other in an advanced way that no build tool has done before. This is a key feature to create builds that are reliable and reproducible in any environment.

23.9.1.3. Artifact reuse

Before downloading an artifact, Gradle tries to determine the checksum of the required artifact by downloading the sha file associated with that artifact. If the checksum can be retrieved, an artifact is not downloaded if an artifact already exists with the same id and checksum. If the checksum cannot be retrieved from the remote server, the artifact will be downloaded (and ignored if it matches an existing artifact).

As well as considering artifacts downloaded from a different repository, Gradle will also attempt to reuse artifacts found in the local Maven Repository. If a candidate artifact has been downloaded by Maven, Gradle will use this artifact if it can be verified to match the checksum declared by the remote server.

23.9.1.4. Checksum based storage

It is possible for different repositories to provide a different binary artifact in response to the same artifact identifier. This is often the case with Maven SNAPSHOT artifacts, but can also be true for any artifact which is republished without changing it's identifier. By caching artifacts based on their SHA1 checksum, Gradle is able to maintain multiple versions of the same artifact. This means that when resolving against one repository Gradle will never overwrite the cached artifact file from a different repository. This is done without requiring a separate artifact file store per repository.

23.9.1.5. Cache Locking

The Gradle dependency cache uses file-based locking to ensure that it can safely be used by multiple Gradle processes concurrently. The lock is held whenever the binary meta-data store is being read or written, but is released for slow operations such as downloading remote artifacts.

23.9.2. Command line options to override caching

23.9.2.1. Offline

The --offline command line switch tells Gradle to always use dependency modules from the cache, regardless if they are due to be checked again. When running with offline, Gradle will never attempt to access the network to perform dependency resolution. If required modules are not present in the dependency cache, build execution will fail.

23.9.2.2. Refresh

At times, the Gradle Dependency Cache can be out of sync with the actual state of the configured repositories. Perhaps a repository was initially misconfigured, or perhaps a “non-changing” module was published incorrectly. To refresh all dependencies in the dependency cache, use the --refresh-dependencies option on the command line.

The --refresh-dependencies option tells Gradle to ignore all cached entries for resolved modules and artifacts. A fresh resolve will be performed against all configured repositories, with dynamic versions recalculated, modules refreshed, and artifacts downloaded. However, where possible Gradle will check if the previously downloaded artifacts are valid before downloading again. This is done by comparing published SHA1 values in the repository with the SHA1 values for existing downloaded artifacts.

23.9.3. Fine-tuned control over dependency caching

You can fine-tune certain aspects of caching using the ResolutionStrategy for a configuration.

By default, Gradle caches dynamic versions for 24 hours. To change how long Gradle will cache the resolved version for a dynamic version, use:

Example 23.66. Dynamic version cache control

build.gradle

configurations.all {
    resolutionStrategy.cacheDynamicVersionsFor 10, 'minutes'
}

By default, Gradle caches changing modules for 24 hours. To change how long Gradle will cache the meta-data and artifacts for a changing module, use:

Example 23.67. Changing module cache control

build.gradle

configurations.all {
    resolutionStrategy.cacheChangingModulesFor 4, 'hours'
}

For more details, take a look at the API documentation for ResolutionStrategy.

23.10. Strategies for transitive dependency management

Many projects rely on the Maven Central repository. This is not without problems.

  • The Maven Central repository can be down or can be slow to respond.

  • The POM files of many popular projects specify dependencies or other configuration that are just plain wrong (for instance, the POM file of the “commons-httpclient-3.0” module declares JUnit as a runtime dependency).

  • For many projects there is not one right set of dependencies (as more or less imposed by the POM format).

If your project relies on the Maven Central repository you are likely to need an additional custom repository, because:

  • You might need dependencies that are not uploaded to Maven Central yet.

  • You want to deal properly with invalid metadata in a Maven Central POM file.

  • You don't want to expose people to the downtimes or slow response of Maven Central, if they just want to build your project.

It is not a big deal to set-up a custom repository, [13] but it can be tedious to keep it up to date. For a new version, you always have to create the new XML descriptor and the directories. Your custom repository is another infrastructure element which might have downtimes and needs to be updated. To enable historical builds, you need to keep all the past libraries, not to mention a backup of these. It is another layer of indirection. Another source of information you have to lookup. All this is not really a big deal but in its sum it has an impact. Repository managers like Artifactory or Nexus make this easier, but most open source projects don't usually have a host for those products. This is changing with new services like Bintray that let developers host and distribute their release binaries using a self-service repository platform. Bintray also supports sharing approved artifacts though the JCenter public repository to provide a single resolution address for all popular OSS Java artifacts (see Section 23.6.2, “Maven JCenter repository”).

This is a common reason why many projects prefer to store their libraries in their version control system. This approach is fully supported by Gradle. The libraries can be stored in a flat directory without any XML module descriptor files. Yet Gradle offers complete transitive dependency management. You can use either client module dependencies to express the dependency relations, or artifact dependencies in case a first level dependency has no transitive dependencies. People can check out such a project from your source code control system and have everything necessary to build it.

If you are working with a distributed version control system like Git you probably don't want to use the version control system to store libraries as people check out the whole history. But even here the flexibility of Gradle can make your life easier. For example, you can use a shared flat directory without XML descriptors and yet you can have full transitive dependency management, as described above.

You could also have a mixed strategy. If your main concern is bad metadata in the POM file and maintaining custom XML descriptors, then Client Modules offer an alternative. However, you can still use a Maven2 repo or your custom repository as a repository for jars only and still enjoy transitive dependency management. Or you can only provide client modules for POMs with bad metadata. For the jars and the correct POMs you still use the remote repository.

23.10.1. Implicit transitive dependencies

There is another way to deal with transitive dependencies without XML descriptor files. You can do this with Gradle, but we don't recommend it. We mention it for the sake of completeness and comparison with other build tools.

The trick is to use only artifact dependencies and group them in lists. This will directly express your first level dependencies and your transitive dependencies (see Section 23.4.8, “Optional attributes”). The problem with this is that Gradle dependency management will see this as specifying all dependencies as first level dependencies. The dependency reports won't show your real dependency graph and the compile task uses all dependencies, not just the first level dependencies. All in all, your build is less maintainable and reliable than it could be when using client modules, and you don't gain anything.



[10] Gradle supports partial multiproject builds (see Chapter 24, Multi-project Builds).

[12] At http://ant.apache.org/ivy/history/latest-milestone/concept.html you can learn more about ivy patterns.

[13] If you want to shield your project from the downtimes of Maven Central things get more complicated. You probably want to set-up a repository proxy for this. In an enterprise environment this is rather common. For an open source project it looks like overkill.