Almost every Gradle build interacts with files in some way: think source files, file dependencies, reports and so on. That’s why Gradle comes with a comprehensive API that makes it simple to perform the file operations you need.

The API has two parts to it:

  • Specifying which files and directories to process

  • Specifying what to do with them

The File paths in depth section covers the first of these in detail, while subsequent sections, like File copying in depth, cover the second. To begin with, we’ll show you examples of the most common scenarios that users encounter.

Copying a single file

You copy a file by creating an instance of Gradle’s builtin Copy task and configuring it with the location of the file and where you want to put it. This example mimics copying a generated report into a directory that will be packed into an archive, such as a ZIP or TAR:

Example 1. How to copy a single file
GroovyKotlin
build.gradle
task copyReport(type: Copy) {
    from file("$buildDir/reports/my-report.pdf")
    into file("$buildDir/toArchive")
}

The Project.file(java.lang.Object) method is used to create a file or directory path relative to the current project and is a common way to make build scripts work regardless of the project path. The file and directory paths are then used to specify what file to copy using Copy.from(java.lang.Object…​) and which directory to copy it to using Copy.into(java.lang.Object).

You can even use the path directly without the file() method, as explained early in the section File copying in depth:

Example 2. Using implicit string paths
GroovyKotlin
build.gradle
task copyReport2(type: Copy) {
    from "$buildDir/reports/my-report.pdf"
    into "$buildDir/toArchive"
}

Although hard-coded paths make for simple examples, they also make the build brittle. It’s better to use a reliable, single source of truth, such as a task or shared project property. In the following modified example, we use a report task defined elsewhere that has the report’s location stored in its outputFile property:

Example 3. Prefer task/project properties over hard-coded paths
GroovyKotlin
build.gradle
task copyReport3(type: Copy) {
    from myReportTask.outputFile
    into archiveReportsTask.dirToArchive
}

We have also assumed that the reports will be archived by archiveReportsTask, which provides us with the directory that will be archived and hence where we want to put the copies of the reports.

Copying multiple files

You can extend the previous examples to multiple files very easily by providing multiple arguments to from():

Example 4. Using multiple arguments with from()
GroovyKotlin
build.gradle
task copyReportsForArchiving(type: Copy) {
    from "$buildDir/reports/my-report.pdf", "src/docs/manual.pdf"
    into "$buildDir/toArchive"
}

Two files are now copied into the archive directory. You can also use multiple from() statements to do the same thing, as shown in the first example of the section File copying in depth.

Now consider another example: what if you want to copy all the PDFs in a directory without having to specify each one? To do this, attach inclusion and/or exclusion patterns to the copy specification. Here we use a string pattern to include PDFs only:

Example 5. Using a flat filter
GroovyKotlin
build.gradle
task copyPdfReportsForArchiving(type: Copy) {
    from "$buildDir/reports"
    include "*.pdf"
    into "$buildDir/toArchive"
}

One thing to note, as demonstrated in the following diagram, is that only the PDFs that reside directly in the reports directory are copied:

copy with flat filter example
Figure 1. The effect of a flat filter on copying

You can include files in subdirectories by using an Ant-style glob pattern (**/*), as done in this updated example:

Example 6. Using a deep filter
GroovyKotlin
build.gradle
task copyAllPdfReportsForArchiving(type: Copy) {
    from "$buildDir/reports"
    include "**/*.pdf"
    into "$buildDir/toArchive"
}

This task has the following effect:

copy with deep filter example
Figure 2. The effect of a deep filter on copying

One thing to bear in mind is that a deep filter like this has the side effect of copying the directory structure below reports as well as the files. If you just want to copy the files without the directory structure, you need to use an explicit fileTree(dir) { includes }.files expression. We talk more about the difference between file trees and file collections in the File trees section.

This is just one of the variations in behavior you’re likely to come across when dealing with file operations in Gradle builds. Fortunately, Gradle provides elegant solutions to almost all those use cases. Read the in-depth sections later in the chapter for more detail on how the file operations work in Gradle and what options you have for configuring them.

Copying directory hierarchies

You may have a need to copy not just files, but the directory structure they reside in as well. This is the default behavior when you specify a directory as the from() argument, as demonstrated by the following example that copies everything in the reports directory, including all its subdirectories, to the destination:

Example 7. Copying an entire directory
GroovyKotlin
build.gradle
task copyReportsDirForArchiving(type: Copy) {
    from "$buildDir/reports"
    into "$buildDir/toArchive"
}

The key aspect that users struggle with is controlling how much of the directory structure goes to the destination. In the above example, do you get a toArchive/reports directory or does everything in reports go straight into toArchive? The answer is the latter. If a directory is part of the from() path, then it won’t appear in the destination.

So how do you ensure that reports itself is copied across, but not any other directory in $buildDir? The answer is to add it as an include pattern:

Example 8. Copying an entire directory, including itself
GroovyKotlin
build.gradle
task copyReportsDirForArchiving2(type: Copy) {
    from("$buildDir") {
        include "reports/**"
    }
    into "$buildDir/toArchive"
}

You’ll get the same behavior as before except with one extra level of directory in the destination, i.e. toArchive/reports.

One thing to note is how the include() directive applies only to the from(), whereas the directive in the previous section applied to the whole task. These different levels of granularity in the copy specification allow you to easily handle most requirements that you will come across. You can learn more about this in the section on child specifications.

Creating archives (zip, tar, etc.)

From the perspective of Gradle, packing files into an archive is effectively a copy in which the destination is the archive file rather than a directory on the file system. This means that creating archives looks a lot like copying, with all of the same features!

The simplest case involves archiving the entire contents of a directory, which this example demonstrates by creating a ZIP of the toArchive directory:

Example 9. Archiving a directory as a ZIP
GroovyKotlin
build.gradle
task packageDistribution(type: Zip) {
    archiveFileName = "my-distribution.zip"
    destinationDirectory = file("$buildDir/dist")

    from "$buildDir/toArchive"
}

Notice how we specify the destination and name of the archive instead of an into(): both are required. You often won’t see them explicitly set, because most projects apply the Base Plugin. It provides some conventional values for those properties. The next example demonstrates this and you can learn more about the conventions in the archive naming section.

Each type of archive has its own task type, the most common ones being Zip, Tar and Jar. They all share most of the configuration options of Copy, including filtering and renaming.

One of the most common scenarios involves copying files into specified subdirectories of the archive. For example, let’s say you want to package all PDFs into a docs directory in the root of the archive. This docs directory doesn’t exist in the source location, so you have to create it as part of the archive. You do this by adding an into() declaration for just the PDFs:

Example 10. Using the Base Plugin for its archive name convention
GroovyKotlin
build.gradle
plugins {
    id 'base'
}

version = "1.0.0"

task packageDistribution(type: Zip) {
    from("$buildDir/toArchive") {
        exclude "**/*.pdf"
    }

    from("$buildDir/toArchive") {
        include "**/*.pdf"
        into "docs"
    }
}

As you can see, you can have multiple from() declarations in a copy specification, each with its own configuration. See Using child copy specifications for more information on this feature.

Unpacking archives

Archives are effectively self-contained file systems, so unpacking them is a case of copying the files from that file system onto the local file system — or even into another archive. Gradle enables this by providing some wrapper functions that make archives available as hierarchical collections of files (file trees).

The two functions of interest are Project.zipTree(java.lang.Object) and Project.tarTree(java.lang.Object), which produce a FileTree from a corresponding archive file. That file tree can then be used in a from() specification, like so:

Example 11. Unpacking a ZIP file
GroovyKotlin
build.gradle
task unpackFiles(type: Copy) {
    from zipTree("src/resources/thirdPartyResources.zip")
    into "$buildDir/resources"
}

As with a normal copy, you can control which files are unpacked via filters and even rename files as they are unpacked.

More advanced processing can be handled by the eachFile() method. For example, you might need to extract different subtrees of the archive into different paths within the destination directory. The following sample uses the method to extract the files within the archive’s libs directory into the root destination directory, rather than into a libs subdirectory:

Example 12. Unpacking a subset of a ZIP file
GroovyKotlin
build.gradle
task unpackLibsDirectory(type: Copy) {
    from(zipTree("src/resources/thirdPartyResources.zip")) {
        include "libs/**"  // (1)
        eachFile { fcd ->
            fcd.relativePath = new RelativePath(true, fcd.relativePath.segments.drop(1))  // (2)
        }
        includeEmptyDirs = false  // (3)
    }
    into "$buildDir/resources"
}

  1. Extracts only the subset of files that reside in the libs directory

  2. Remaps the path of the extracting files into the destination directory by dropping the libs segment from the file path

  3. Ignores the empty directories resulting from the remapping, see Caution note below

⚠

You can not change the destination path of empty directories with this technique. You can learn more in this issue.

If you’re a Java developer and are wondering why there is no jarTree() method, that’s because zipTree() works perfectly well for JARs, WARs and EARs.

Creating "uber" or "fat" JARs

In the Java space, applications and their dependencies typically used to be packaged as separate JARs within a single distribution archive. That still happens, but there is another approach that is now common: placing the classes and resources of the dependencies directly into the application JAR, creating what is known as an uber or fat JAR.

Gradle makes this approach easy to accomplish. Consider the aim: to copy the contents of other JAR files into the application JAR. All you need for this is the Project.zipTree(java.lang.Object) method and the Jar task, as demonstrated by the uberJar task in the following example:

Example 13. Creating a Java uber or fat JAR
GroovyKotlin
build.gradle
plugins {
    id 'java'
}

version = '1.0.0'

repositories {
    mavenCentral()
}

dependencies {
    implementation 'commons-io:commons-io:2.6'
}

task uberJar(type: Jar) {
    archiveClassifier = 'uber'

    from sourceSets.main.output

    dependsOn configurations.runtimeClasspath
    from {
        configurations.runtimeClasspath.findAll { it.name.endsWith('jar') }.collect { zipTree(it) }
    }
}

In this case, we’re taking the runtime dependencies of the project — configurations.runtimeClasspath.files — and wrapping each of the JAR files with the zipTree() method. The result is a collection of ZIP file trees, the contents of which are copied into the uber JAR alongside the application classes.

Creating directories

Many tasks need to create directories to store the files they generate, which is why Gradle automatically manages this aspect of tasks when they explicitly define file and directory outputs. You can learn about this feature in the incremental build section of the user manual. All core Gradle tasks ensure that any output directories they need are created if necessary using this mechanism.

In cases where you need to create a directory manually, you can use the Project.mkdir(java.lang.Object) method from within your build scripts or custom task implementations. Here’s a simple example that creates a single images directory in the project folder:

Example 14. Manually creating a directory
GroovyKotlin
build.gradle
task ensureDirectory {
    doLast {
        mkdir "images"
    }
}

As described in the Apache Ant manual, the mkdir task will automatically create all necessary directories in the given path and will do nothing if the directory already exists.

Moving files and directories

Gradle has no API for moving files and directories around, but you can use the Apache Ant integration to easily do that, as shown in this example:

Example 15. Moving a directory using the Ant task
GroovyKotlin
build.gradle
task moveReports {
    doLast {
        ant.move file: "${buildDir}/reports",
                 todir: "${buildDir}/toArchive"
    }
}

This is not a common requirement and should be used sparingly as you lose information and can easily break a build. It’s generally preferable to copy directories and files instead.

Renaming files on copy

The files used and generated by your builds sometimes don’t have names that suit, in which case you want to rename those files as you copy them. Gradle allows you to do this as part of a copy specification using the rename() configuration.

The following example removes the "-staging-" marker from the names of any files that have it:

Example 16. Renaming files as they are copied
GroovyKotlin
build.gradle
task copyFromStaging(type: Copy) {
    from "src/main/webapp"
    into "$buildDir/explodedWar"

    rename '(.+)-staging(.+)', '$1$2'
}

You can use regular expressions for this, as in the above example, or closures that use more complex logic to determine the target filename. For example, the following task truncates filenames:

Example 17. Truncating filenames as they are copied
GroovyKotlin
build.gradle
task copyWithTruncate(type: Copy) {
    from "$buildDir/reports"
    rename { String filename ->
        if (filename.size() > 10) {
            return filename[0..7] + "~" + filename.size()
        }
        else return filename
    }
    into "$buildDir/toArchive"
}

As with filtering, you can also apply renaming to a subset of files by configuring it as part of a child specification on a from().

Deleting files and directories

You can easily delete files and directories using either the Delete task or the Project.delete(org.gradle.api.Action) method. In both cases, you specify which files and directories to delete in a way supported by the Project.files(java.lang.Object…​) method.

For example, the following task deletes the entire contents of a build’s output directory:

Example 18. Deleting a directory
GroovyKotlin
build.gradle
task myClean(type: Delete) {
    delete buildDir
}

If you want more control over which files are deleted, you can’t use inclusions and exclusions in the same way as for copying files. Instead, you have to use the builtin filtering mechanisms of FileCollection and FileTree. The following example does just that to clear out temporary files from a source directory:

Example 19. Deleting files matching a specific pattern
GroovyKotlin
build.gradle
task cleanTempFiles(type: Delete) {
    delete fileTree("src").matching {
        include "**/*.tmp"
    }
}

You’ll learn more about file collections and file trees in the next section.

File paths in depth

In order to perform some action on a file, you need to know where it is, and that’s the information provided by file paths. Gradle builds on the standard Java File class, which represents the location of a single file, and provides new APIs for dealing with collections of paths. This section shows you how to use the Gradle APIs to specify file paths for use in tasks and file operations.

But first, an important note on using hard-coded file paths in your builds.

On hard-coded file paths

Many examples in this chapter use hard-coded paths as string literals. This makes them easy to understand, but it’s not good practice for real builds. The problem is that paths often change and the more places you need to change them, the more likely you are to miss one and break the build.

Where possible, you should use tasks, task properties, and project properties — in that order of preference — to configure file paths. For example, if you were to create a task that packages the compiled classes of a Java application, you should aim for something like this:

Example 20. How to minimize the number of hard-coded paths in your build
GroovyKotlin
build.gradle
ext {
    archivesDirPath = "$buildDir/archives"
}

task packageClasses(type: Zip) {
    archiveAppendix = "classes"
    destinationDirectory = file(archivesDirPath)

    from compileJava
}

See how we’re using the compileJava task as the source of the files to package and we’ve created a project property archivesDirPath to store the location where we put archives, on the basis we’re likely to use it elsewhere in the build.

Using a task directly as an argument like this relies on it having defined outputs, so it won’t always be possible. In addition, this example could be improved further by relying on the Java plugin’s convention for destinationDirectory rather than overriding it, but it does demonstrate the use of project properties.

Single files and directories

Gradle provides the Project.file(java.lang.Object) method for specifying the location of a single file or directory. Relative paths are resolved relative to the project directory, while absolute paths remain unchanged.

⚠

Never use new File(relative path) because this creates a path relative to the current working directory (CWD). Gradle can make no guarantees about the location of the CWD, which means builds that rely on it may break at any time.

Here are some examples of using the file() method with different types of argument:

Example 21. Locating files
GroovyKotlin
build.gradle
// Using a relative path
File configFile = file('src/config.xml')

// Using an absolute path
configFile = file(configFile.absolutePath)

// Using a File object with a relative path
configFile = file(new File('src/config.xml'))

// Using a java.nio.file.Path object with a relative path
configFile = file(Paths.get('src', 'config.xml'))

// Using an absolute java.nio.file.Path object
configFile = file(Paths.get(System.getProperty('user.home')).resolve('global-config.xml'))

As you can see, you can pass strings, File instances and Path instances to the file() method, all of which result in an absolute File object. You can find other options for argument types in the reference guide, linked in the previous paragraph.

What happens in the case of multi-project builds? The file() method will always turn relative paths into paths that are relative to the current project directory, which may be a child project. If you want to use a path that’s relative to the root project directory, then you need to use the special Project.getRootDir() property to construct an absolute path, like so:

Example 22. Creating a path relative to a parent project
GroovyKotlin
build.gradle
File configFile = file("$rootDir/shared/config.xml")

Let’s say you’re working on a multi-project build in a dev/projects/AcmeHealth directory. You use the above example in the build of the library you’re fixing — at AcmeHealth/subprojects/AcmePatientRecordLib/build.gradle. The file path will resolve to the absolute version of dev/projects/AcmeHealth/shared/config.xml.

The file() method can be used to configure any task that has a property of type File. Many tasks, though, work on multiple files, so we look at how to specify sets of files next.

File collections

A file collection is simply a set of file paths that’s represented by the FileCollection interface. Any file paths. It’s important to understand that the file paths don’t have to be related in any way, so they don’t have to be in the same directory or even have a shared parent directory. You will also find that many parts of the Gradle API use FileCollection, such as the copying API discussed later in this chapter and dependency configurations.

The recommended way to specify a collection of files is to use the ProjectLayout.files(java.lang.Object...) method, which returns a FileCollection instance. This method is very flexible and allows you to pass multiple strings, File instances, collections of strings, collections of Files, and more. You can even pass in tasks as arguments if they have defined outputs. Learn about all the supported argument types in the reference guide.

⚠

Although the files() method accepts File instances, never use new File(relative path) with it because this creates a path relative to the current working directory (CWD). Gradle can make no guarantees about the location of the CWD, which means builds that rely on it may break at any time.

As with the Project.file(java.lang.Object) method covered in the previous section, all relative paths are evaluated relative to the current project directory. The following example demonstrates some of the variety of argument types you can use — strings, File instances, a list and a Path:

Example 23. Creating a file collection
GroovyKotlin
build.gradle
FileCollection collection = layout.files('src/file1.txt',
                                  new File('src/file2.txt'),
                                  ['src/file3.csv', 'src/file4.csv'],
                                  Paths.get('src', 'file5.txt'))

File collections have some important attributes in Gradle. They can be:

  • created lazily

  • iterated over

  • filtered

  • combined

Lazy creation of a file collection is useful when you need to evaluate the files that make up a collection at the time a build runs. In the following example, we query the file system to find out what files exist in a particular directory and then make those into a file collection:

Example 24. Implementing a file collection
GroovyKotlin
build.gradle
task list {
    doLast {
        File srcDir

        // Create a file collection using a closure
        collection = layout.files { srcDir.listFiles() }

        srcDir = file('src')
        println "Contents of $srcDir.name"
        collection.collect { relativePath(it) }.sort().each { println it }

        srcDir = file('src2')
        println "Contents of $srcDir.name"
        collection.collect { relativePath(it) }.sort().each { println it }
    }
}
Output of gradle -q list
> gradle -q list
Contents of src
src/dir1
src/file1.txt
Contents of src2
src2/dir1
src2/dir2

The key to lazy creation is passing a closure (in Groovy) or a Provider (in Kotlin) to the files() method. Your closure/provider simply needs to return a value of a type accepted by files(), such as List<File>, String, FileCollection, etc.

Iterating over a file collection can be done through the each() method (in Groovy) of forEach method (in Kotlin) on the collection or using the collection in a for loop. In both approaches, the file collection is treated as a set of File instances, i.e. your iteration variable will be of type File.

The following example demonstrates such iteration as well as how you can convert file collections to other types using the as operator or supported properties:

Example 25. Using a file collection
GroovyKotlin
build.gradle
        // Iterate over the files in the collection
        collection.each { File file ->
            println file.name
        }

        // Convert the collection to various types
        Set set = collection.files
        Set set2 = collection as Set
        List list = collection as List
        String path = collection.asPath
        File file = collection.singleFile

        // Add and subtract collections
        def union = collection + layout.files('src/file2.txt')
        def difference = collection - layout.files('src/file2.txt')

You can also see at the end of the example how to combine file collections using the + and - operators to merge and subtract them. An important feature of the resulting file collections is that they are live. In other words, when you combine file collections in this way, the result always reflects what’s currently in the source file collections, even if they change during the build.

For example, imagine collection in the above example gains an extra file or two after union is created. As long as you use union after those files are added to collection, union will also contain those additional files. The same goes for the different file collection.

Live collections are also important when it comes to filtering. If you want to use a subset of a file collection, you can take advantage of the FileCollection.filter(org.gradle.api.specs.Spec) method to determine which files to "keep". In the following example, we create a new collection that consists of only the files that end with .txt in the source collection:

Example 26. Filtering a file collection
GroovyKotlin
build.gradle
        FileCollection textFiles = collection.filter { File f ->
            f.name.endsWith(".txt")
        }
Output of gradle -q filterTextFiles
> gradle -q filterTextFiles
src/file1.txt
src/file2.txt
src/file5.txt

If collection changes at any time, either by adding or removing files from itself, then textFiles will immediately reflect the change because it is also a live collection. Note that the closure you pass to filter() takes a File as an argument and should return a boolean.

File trees

A file tree is a file collection that retains the directory structure of the files it contains and has the type FileTree. This means that all the paths in a file tree must have a shared parent directory. The following diagram highlights the distinction between file trees and file collections in the common case of copying files:

file collection vs file tree
Figure 3. The differences in how file trees and file collections behave when copying files
✨
 Although FileTree extends FileCollection (an is-a relationship), their behaviors do differ. In other words, you can use a file tree wherever a file collection is required, but remember: a file collection is a flat list/set of files, while a file tree is a file and directory hierarchy. To convert a file tree to a flat collection, use the FileTree.getFiles() property.

The simplest way to create a file tree is to pass a file or directory path to the Project.fileTree(java.lang.Object) method. This will create a tree of all the files and directories in that base directory (but not the base directory itself). The following example demonstrates how to use the basic method and, in addition, how to filter the files and directories using Ant-style patterns:

Example 27. Creating a file tree
GroovyKotlin
build.gradle
// Create a file tree with a base directory
ConfigurableFileTree tree = fileTree(dir: 'src/main')

// Add include and exclude patterns to the tree
tree.include '**/*.java'
tree.exclude '**/Abstract*'

// Create a tree using closure
tree = fileTree('src') {
    include '**/*.java'
}

// Create a tree using a map
tree = fileTree(dir: 'src', include: '**/*.java')
tree = fileTree(dir: 'src', includes: ['**/*.java', '**/*.xml'])
tree = fileTree(dir: 'src', include: '**/*.java', exclude: '**/*test*/**')

You can see more examples of supported patterns in the API docs for PatternFilterable. Also, see the API documentation for fileTree() to see what types you can pass as the base directory.

By default, fileTree() returns a FileTree instance that applies some default exclusion patterns for convenience — the same defaults as Ant in fact. For the complete default exclusion list, see the Ant manual.

If those default exclusions prove problematic, you can workaround the issue by using the defaultexcludes Ant task, as demonstrated in this example:

Example 28. Changing Ant default exclusions for a copy task
GroovyKotlin
build.gradle
task forcedCopy (type: Copy) {
    into "$buildDir/inPlaceApp"
    from 'src/main/webapp'

    doFirst {
        ant.defaultexcludes remove: "**/.git"
        ant.defaultexcludes remove: "**/.git/**"
        ant.defaultexcludes remove: "**/*~"
    }

    doLast {
        ant.defaultexcludes default: true
    }
}

In general, it’s best to ensure that the default exclusions are reset whenever you change them as modifications are visible to the entire build. The above example is performing such a reset in its doLast action.

You can do many of the same things with file trees that you can with file collections:

You can also traverse file trees using the FileTree.visit(org.gradle.api.Action) method. All of these techniques are demonstrated in the following example:

Example 29. Using a file tree
GroovyKotlin
build.gradle
// Iterate over the contents of a tree
tree.each {File file ->
    println file
}

// Filter a tree
FileTree filtered = tree.matching {
    include 'org/gradle/api/**'
}

// Add trees together
FileTree sum = tree + fileTree(dir: 'src/test')

// Visit the elements of the tree
tree.visit {element ->
    println "$element.relativePath => $element.file"
}

We’ve discussed how to create your own file trees and file collections, but it’s also worth bearing in mind that many Gradle plugins provide their own instances of file trees, such as Java’s source sets. These can be used and manipulated in exactly the same way as the file trees you create yourself.

Another specific type of file tree that users commonly need is the archive, i.e. ZIP files, TAR files, etc. We look at those next.

Using archives as file trees

An archive is a directory and file hierarchy packed into a single file. In other words, it’s a special case of a file tree, and that’s exactly how Gradle treats archives. Instead of using the fileTree() method, which only works on normal file systems, you use the Project.zipTree(java.lang.Object) and Project.tarTree(java.lang.Object) methods to wrap archive files of the corresponding type (note that JAR, WAR and EAR files are ZIPs). Both methods return FileTree instances that you can then use in the same way as normal file trees. For example, you can extract some or all of the files of an archive by copying its contents to some directory on the file system. Or you can merge one archive into another.

Here are some simple examples of creating archive-based file trees:

Example 30. Using an archive as a file tree
GroovyKotlin
build.gradle
// Create a ZIP file tree using path
FileTree zip = zipTree('someFile.zip')

// Create a TAR file tree using path
FileTree tar = tarTree('someFile.tar')

//tar tree attempts to guess the compression based on the file extension
//however if you must specify the compression explicitly you can:
FileTree someTar = tarTree(resources.gzip('someTar.ext'))

You can see a practical example of extracting an archive file in among the common scenarios we cover.

Understanding implicit conversion to file collections

Many objects in Gradle have properties which accept a set of input files. For example, the JavaCompile task has a source property that defines the source files to compile. You can set the value of this property using any of the types supported by the files() method, as mentioned in the api docs. This means you can, for example, set the property to a File, String, collection, FileCollection or even a closure or `Provider.

This is a feature of specific tasks! That means implicit conversion will not happen for just any task that has a FileCollection or FileTree property. If you want to know whether implicit conversion happens in a particular situation, you will need to read the relevant documentation, such as the corresponding task’s API docs. Alternatively, you can remove all doubt by explicitly using ProjectLayout.files(java.lang.Object...) in your build.

Here are some examples of the different types of arguments that the source property can take:

Example 31. Specifying a set of files
GroovyKotlin
build.gradle
task compile(type: JavaCompile) {

    // Use a File object to specify the source directory
    source = file('src/main/java')

    // Use a String path to specify the source directory
    source = 'src/main/java'

    // Use a collection to specify multiple source directories
    source = ['src/main/java', '../shared/java']

    // Use a FileCollection (or FileTree in this case) to specify the source files
    source = fileTree(dir: 'src/main/java').matching { include 'org/gradle/api/**' }

    // Using a closure to specify the source files.
    source = {
        // Use the contents of each zip file in the src dir
        file('src').listFiles().findAll {it.name.endsWith('.zip')}.collect { zipTree(it) }
    }
}

One other thing to note is that properties like source have corresponding methods in core Gradle tasks. Those methods follow the convention of appending to collections of values rather than replacing them. Again, this method accepts any of the types supported by the files() method, as shown here:

Example 32. Appending a set of files
GroovyKotlin
build.gradle
compile {
    // Add some source directories use String paths
    source 'src/main/java', 'src/main/groovy'

    // Add a source directory using a File object
    source file('../shared/java')

    // Add some source directories using a closure
    source { file('src/test/').listFiles() }
}

As this is a common convention, we recommend that you follow it in your own custom tasks. Specifically, if you plan to add a method to configure a collection-based property, make sure the method appends rather than replaces values.

File copying in depth

The basic process of copying files in Gradle is a simple one:

  • Define a task of type Copy

  • Specify which files (and potentially directories) to copy

  • Specify a destination for the copied files

But this apparent simplicity hides a rich API that allows fine-grained control of which files are copied, where they go, and what happens to them as they are copied — renaming of the files and token substitution of file content are both possibilities, for example.

Let’s start with the last two items on the list, which form what is known as a copy specification. This is formally based on the CopySpec interface, which the Copy task implements, and offers:

CopySpec has several additional methods that allow you to control the copying process, but these two are the only required ones. into() is straightforward, requiring a directory path as its argument in any form supported by the Project.file(java.lang.Object) method. The from() configuration is far more flexible.

Not only does from() accept multiple arguments, it also allows several different types of argument. For example, some of the most common types are:

  • A String — treated as a file path or, if it starts with "file://", a file URI

  • A File — used as a file path

  • A FileCollection or FileTree — all files in the collection are included in the copy

  • A task — the files or directories that form a task’s defined outputs are included

In fact, from() accepts all the same arguments as Project.files(java.lang.Object…​) so see that method for a more detailed list of acceptable types.

Something else to consider is what type of thing a file path refers to:

  • A file — the file is copied as is

  • A directory — this is effectively treated as a file tree: everything in it, including subdirectories, is copied. However, the directory itself is not included in the copy.

  • A non-existent file — the path is ignored

Here is an example that uses multiple from() specifications, each with a different argument type. You will probably also notice that into() is configured lazily using a closure (in Groovy) or a Provider (in Kotlin) — a technique that also works with from():

Example 33. Specifying copy task source files and destination directory
GroovyKotlin
build.gradle
task anotherCopyTask (type: Copy) {
    // Copy everything under src/main/webapp
    from 'src/main/webapp'
    // Copy a single file
    from 'src/staging/index.html'
    // Copy the output of a task
    from copyTask
    // Copy the output of a task using Task outputs explicitly.
    from copyTaskWithPatterns.outputs
    // Copy the contents of a Zip file
    from zipTree('src/main/assets.zip')
    // Determine the destination directory later
    into { getDestDir() }
}

Note that the lazy configuration of into() is different from a child specification, even though the syntax is similar. Keep an eye on the number of arguments to distinguish between them.

Filtering files

You’ve already seen that you can filter file collections and file trees directly in a Copy task, but you can also apply filtering in any copy specification through the CopySpec.include(java.lang.String…​) and CopySpec.exclude(java.lang.String…​) methods.

Both of these methods are normally used with Ant-style include or exclude patterns, as described in PatternFilterable. You can also perform more complex logic by using a closure that takes a FileTreeElement and returns true if the file should be included or false otherwise. The following example demonstrates both forms, ensuring that only .html and .jsp files are copied, except for those .html files with the word "DRAFT" in their content:

Example 34. Selecting the files to copy
GroovyKotlin
build.gradle
task copyTaskWithPatterns (type: Copy) {
    from 'src/main/webapp'
    into "$buildDir/explodedWar"
    include '**/*.html'
    include '**/*.jsp'
    exclude { FileTreeElement details ->
        details.file.name.endsWith('.html') &&
            details.file.text.contains('DRAFT')
    }
}

A question you may ask yourself at this point is what happens when inclusion and exclusion patterns overlap? Which pattern wins? Here are the basic rules:

  • If there are no explicit inclusions or exclusions, everything is included

  • If at least one inclusion is specified, only files and directories matching the patterns are included

  • Any exclusion pattern overrides any inclusions, so if a file or directory matches at least one exclusion pattern, it won’t be included, regardless of the inclusion patterns

Bear these rules in mind when creating combined inclusion and exclusion specifications so that you end up with the exact behavior you want.

Note that the inclusions and exclusions in the above example will apply to all from() configurations. If you want to apply filtering to a subset of the copied files, you’ll need to use child specifications.

Renaming files

The example of how to rename files on copy gives you most of the information you need to perform this operation. It demonstrates the two options for renaming:

  • Using a regular expression

  • Using a closure

Regular expressions are a flexible approach to renaming, particularly as Gradle supports regex groups that allow you to remove and replaces parts of the source filename. The following example shows how you can remove the string "-staging-" from any filename that contains it using a simple regular expression:

Example 35. Renaming files as they are copied
GroovyKotlin
build.gradle
task rename (type: Copy) {
    from 'src/main/webapp'
    into "$buildDir/explodedWar"
    // Use a closure to convert all file names to upper case
    rename { String fileName ->
        fileName.toUpperCase()
    }
    // Use a regular expression to map the file name
    rename '(.+)-staging-(.+)', '$1$2'
    rename(/(.+)-staging-(.+)/, '$1$2')
}

You can use any regular expression supported by the Java Pattern class and the substitution string (the second argument of rename() works on the same principles as the Matcher.appendReplacement() method.

✨
Regular expressions in Groovy build scripts

There are two common issues people come across when using regular expressions in this context:

  1. If you use a slashy string (those delimited by '/') for the first argument, you must include the parentheses for rename() as shown in the above example.

  2. It’s safest to use single quotes for the second argument, otherwise you need to escape the '$' in group substitutions, i.e. "\$1\$2"

The first is a minor inconvenience, but slashy strings have the advantage that you don’t have to escape backslash ('\') characters in the regular expression. The second issue stems from Groovy’s support for embedded expressions using ${ } syntax in double-quoted and slashy strings.

The closure syntax for rename() is straightforward and can be used for any requirements that simple regular expressions can’t handle. You’re given the name of a file and you return a new name for that file, or null if you don’t want to change the name. Do be aware that the closure will be executed for every file that’s copied, so try to avoid expensive operations where possible.

Filtering file content (token substitution, templating, etc.)

Not to be confused with filtering which files are copied, file content filtering allows you to transform the content of files while they are being copied. This can involve basic templating that uses token substitution, removal of lines of text, or even more complex filtering using a full-blown template engine.

The following example demonstrates several forms of filtering, including token substitution using the CopySpec.expand(java.util.Map) method and another using CopySpec.filter(java.lang.Class) with an Ant filter:

Example 36. Filtering files as they are copied
GroovyKotlin
build.gradle
import org.apache.tools.ant.filters.FixCrLfFilter
import org.apache.tools.ant.filters.ReplaceTokens

task filter(type: Copy) {
    from 'src/main/webapp'
    into "$buildDir/explodedWar"
    // Substitute property tokens in files
    expand(copyright: '2009', version: '2.3.1')
    expand(project.properties)
    // Use some of the filters provided by Ant
    filter(FixCrLfFilter)
    filter(ReplaceTokens, tokens: [copyright: '2009', version: '2.3.1'])
    // Use a closure to filter each line
    filter { String line ->
        "[$line]"
    }
    // Use a closure to remove lines
    filter { String line ->
        line.startsWith('-') ? null : line
    }
    filteringCharset = 'UTF-8'
}

The filter() method has two variants, which behave differently:

  • one takes a FilterReader and is designed to work with Ant filters, such as ReplaceTokens

  • one takes a closure or Transformer that defines the transformation for each line of the source file

Note that both variants assume the source files are text based. When you use the ReplaceTokens class with filter(), the result is a template engine that replaces tokens of the form @tokenName@ (the Ant-style token) with values that you define.

The expand() method treats the source files as Groovy templates, which evaluate and expand expressions of the form ${expression}. You can pass in property names and values that are then expanded in the source files. expand() allows for more than basic token substitution as the embedded expressions are full-blown Groovy expressions.

✨
 It’s good practice to specify the character set when reading and writing the file, otherwise the transformations won’t work properly for non-ASCII text. You configure the character set with the CopySpec.getFilteringCharset() property. If it’s not specified, the JVM default character set is used, which is likely to be different from the one you want.

Using the CopySpec class

A copy specification (or copy spec for short) determines what gets copied to where, and what happens to files during the copy. You’ve alread seen many examples in the form of configuration for Copy and archiving tasks. But copy specs have two attributes that are worth covering in more detail:

  1. They can be independent of tasks

  2. They are hierarchical

The first of these attributes allows you to share copy specs within a build. The second provides fine-grained control within the overall copy specification.

Sharing copy specs

Consider a build that has several tasks that copy a project’s static website resources or add them to an archive. One task might copy the resources to a folder for a local HTTP server and another might package them into a distribution. You could manually specify the file locations and appropriate inclusions each time they are needed, but human error is more likely to creep in, resulting in inconsistencies between tasks.

One solution Gradle provides is the Project.copySpec(org.gradle.api.Action) method. This allows you to create a copy spec outside of a task, which can then be attached to an appropriate task using the CopySpec.with(org.gradle.api.file.CopySpec…​) method. The following example demonstrates how this is done:

Example 37. Sharing copy specifications
GroovyKotlin
build.gradle
CopySpec webAssetsSpec = copySpec {
    from 'src/main/webapp'
    include '**/*.html', '**/*.png', '**/*.jpg'
    rename '(.+)-staging(.+)', '$1$2'
}

task copyAssets (type: Copy) {
    into "$buildDir/inPlaceApp"
    with webAssetsSpec
}

task distApp(type: Zip) {
    archiveFileName = 'my-app-dist.zip'
    destinationDirectory = file("$buildDir/dists")

    from appClasses
    with webAssetsSpec
}

Both the copyAssets and distApp tasks will process the static resources under src/main/webapp, as specified by webAssetsSpec.

✨

The configuration defined by webAssetsSpec will not apply to the app classes included by the distApp task. That’s because from appClasses is its own child specification independent of with webAssetsSpec.

This can be confusing to understand, so it’s probably best to treat with() as an extra from() specification in the task. Hence it doesn’t make sense to define a standalone copy spec without at least one from() defined.

If you encounter a scenario in which you want to apply the same copy configuration to different sets of files, then you can share the configuration block directly without using copySpec(). Here’s an example that has two independent tasks that happen to want to process image files only:

Example 38. Sharing copy patterns only
GroovyKotlin
build.gradle
def webAssetPatterns = {
    include '**/*.html', '**/*.png', '**/*.jpg'
}

task copyAppAssets(type: Copy) {
    into "$buildDir/inPlaceApp"
    from 'src/main/webapp', webAssetPatterns
}

task archiveDistAssets(type: Zip) {
    archiveFileName = 'distribution-assets.zip'
    destinationDirectory = file("$buildDir/dists")

    from 'distResources', webAssetPatterns
}

In this case, we assign the copy configuration to its own variable and apply it to whatever from() specification we want. This doesn’t just work for inclusions, but also exclusions, file renaming, and file content filtering.

Using child specifications

If you only use a single copy spec, the file filtering and renaming will apply to all the files that are copied. Sometimes this is what you want, but not always. Consider the following example that copies files into a directory structure that can be used by a Java Servlet container to deliver a website:

exploded war child copy spec example
Figure 4. Creating an exploded WAR for a Servlet container

This is not a straightforward copy as the WEB-INF directory and its subdirectories don’t exist within the project, so they must be created during the copy. In addition, we only want HTML and image files going directly into the root folder — build/explodedWar — and only JavaScript files going into the js directory. So we need separate filter patterns for those two sets of files.

The solution is to use child specifications, which can be applied to both from() and into() declarations. The following task definition does the necessary work:

Example 39. Nested copy specs
GroovyKotlin
build.gradle
task nestedSpecs(type: Copy) {
    into "$buildDir/explodedWar"
    exclude '**/*staging*'
    from('src/dist') {
        include '**/*.html', '**/*.png', '**/*.jpg'
    }
    from(sourceSets.main.output) {
        into 'WEB-INF/classes'
    }
    into('WEB-INF/lib') {
        from configurations.runtimeClasspath
    }
}

Notice how the src/dist configuration has a nested inclusion specification: that’s the child copy spec. You can of course add content filtering and renaming here as required. A child copy spec is still a copy spec.

The above example also demonstrates how you can copy files into a subdirectory of the destination either by using a child into() on a from() or a child from() on an into(). Both approaches are acceptable, but you may want to create and follow a convention to ensure consistency across your build files.

✨
 Don’t get your into() specifications mixed up! For a normal copy — one to the filesystem rather than an archive — there should always be one "root" into() that simply specifies the overall destination directory of the copy. Any other into() should have a child spec attached and its path will be relative to the root into().

One final thing to be aware of is that a child copy spec inherits its destination path, include patterns, exclude patterns, copy actions, name mappings and filters from its parent. So be careful where you place your configuration.

Copying files in your own tasks

There might be occasions when you want to copy files or directories as part of a task. For example, a custom archiving task based on an unsupported archive format might want to copy files to a temporary directory before they are then archived. You still want to take advantage of Gradle’s copy API, but without introducing an extra Copy task.

The solution is to use the Project.copy(org.gradle.api.Action) method. It works the same way as the Copy task by configuring it with a copy spec. Here’s a trivial example:

Example 40. Copying files using the copy() method without up-to-date check
GroovyKotlin
build.gradle
task copyMethod {
    doLast {
        copy {
            from 'src/main/webapp'
            into "$buildDir/explodedWar"
            include '**/*.html'
            include '**/*.jsp'
        }
    }
}

The above example demonstrates the basic syntax and also highlights two major limitations of using the copy() method:

  1. The copy() method is not incremental. The example’s copyMethod task will always execute because it has no information about what files make up the task’s inputs. You have to manually define the task inputs and outputs.

  2. Using a task as a copy source, i.e. as an argument to from(), won’t set up an automatic task dependency between your task and that copy source. As such, if you are using the copy() method as part of a task action, you must explicitly declare all inputs and outputs in order to get the correct behavior.

The following example shows you how to workaround these limitations by using the dynamic API for task inputs and outputs:

Example 41. Copying files using the copy() method with up-to-date check
GroovyKotlin
build.gradle
task copyMethodWithExplicitDependencies {
    // up-to-date check for inputs, plus add copyTask as dependency
    inputs.files copyTask
    outputs.dir 'some-dir' // up-to-date check for outputs
    doLast{
        copy {
            // Copy the output of copyTask
            from copyTask
            into 'some-dir'
        }
    }
}

These limitations make it preferable to use the Copy task wherever possible, because of its builtin support for incremental building and task dependency inference. That is why the copy() method is intended for use by custom tasks that need to copy files as part of their function. Custom tasks that use the copy() method should declare the necessary inputs and outputs relevant to the copy action.

Mirroring directories and file collections with the Sync task

The Sync task, which extends the Copy task, copies the source files into the destination directory and then removes any files from the destination directory which it did not copy. In other words, it synchronizes the contents of a directory with its source. This can be useful for doing things such as installing your application, creating an exploded copy of your archives, or maintaining a copy of the project’s dependencies.

Here is an example which maintains a copy of the project’s runtime dependencies in the build/libs directory.

Example 42. Using the Sync task to copy dependencies
GroovyKotlin
build.gradle
task libs(type: Sync) {
    from configurations.runtime
    into "$buildDir/libs"
}

You can also perform the same function in your own tasks with the Project.sync(org.gradle.api.Action) method.

Archive creation in depth

Archives are essentially self-contained file systems and Gradle treats them as such. This is why working with archives is very similar to working with files and directories, including such things as file permissions.

Out of the box, Gradle supports creation of both ZIP and TAR archives, and by extension Java’s JAR, WAR and EAR formats — Java’s archive formats are all ZIPs. Each of these formats has a corresponding task type to create them: Zip, Tar, Jar, War, and Ear. These all work the same way and are based on copy specifications, just like the Copy task.

Creating an archive file is essentially a file copy in which the destination is implicit, i.e. the archive file itself. Here’s a basic example that specifies the path and name of the target archive file:

Example 43. Archiving a directory as a ZIP
GroovyKotlin
build.gradle
task packageDistribution(type: Zip) {
    archiveFileName = "my-distribution.zip"
    destinationDirectory = file("$buildDir/dist")

    from "$buildDir/toArchive"
}

In the next section you’ll learn about convention-based archive names, which can save you from always configuring the destination directory and archive name.

The full power of copy specifications are available to you when creating archives, which means you can do content filtering, file renaming or anything else that is covered in the previous section. A particularly common requirement is copying files into subdirectories of the archive that don’t exist in the source folders, something that can be achieved with into() child specifications.

Gradle does of course allow you create as many archive tasks as you want, but it’s worth bearing in mind that many convention-based plugins provide their own. For example, the Java plugin adds a jar task for packaging a project’s compiled classes and resources in a JAR. Many of these plugins provide sensible conventions for the names of archives as well as the copy specifications used. We recommend you use these tasks wherever you can, rather than overriding them with your own.

Archive naming

Gradle has several conventions around the naming of archives and where they are created based on the plugins your project uses. The main convention is provided by the Base Plugin, which defaults to creating archives in the $buildDir/distributions directory and typically uses archive names of the form [projectName]-[version].[type].

The following example comes from a project named zipProject, hence the myZip task creates an archive named zipProject-1.0.zip:

Example 44. Creation of ZIP archive
GroovyKotlin
build.gradle
plugins {
    id 'base'
}

version = 1.0

task myZip(type: Zip) {
    from 'somedir'

    doLast {
        println archiveFileName.get()
        println relativePath(destinationDirectory)
        println relativePath(archiveFile)
    }
}
Output of gradle -q myZip
> gradle -q myZip
zipProject-1.0.zip
build/distributions
build/distributions/zipProject-1.0.zip

Note that the name of the archive does not derive from the name of the task that creates it.

If you want to change the name and location of a generated archive file, you can provide values for the archiveFileName and destinationDirectory properties of the corresponding task. These override any conventions that would otherwise apply.

Alternatively, you can make use of the default archive name pattern provided by AbstractArchiveTask.getArchiveFileName(): [archiveBaseName]-[archiveAppendix]-[archiveVersion]-[archiveClassifier].[archiveExtension]. You can set each of these properties on the task separately if you wish. Note that the Base Plugin uses the convention of project name for archiveBaseName, project version for archiveVersion and the archive type for archiveExtension. It does not provide values for the other properties.

This example — from the same project as the one above — configures just the archiveBaseName property, overriding the default value of the project name:

Example 45. Configuration of archive task - custom archive name
GroovyKotlin
build.gradle
task myCustomZip(type: Zip) {
    archiveBaseName = 'customName'
    from 'somedir'

    doLast {
        println archiveFileName.get()
    }
}
Output of gradle -q myCustomZip
> gradle -q myCustomZip
customName-1.0.zip

You can also override the default archiveBaseName value for all the archive tasks in your build by using the project property archivesBaseName, as demonstrated by the following example:

Example 46. Configuration of archive task - appendix & classifier
GroovyKotlin
build.gradle
plugins {
    id 'base'
}

version = 1.0
archivesBaseName = "gradle"

task myZip(type: Zip) {
    from 'somedir'
}

task myOtherZip(type: Zip) {
    archiveAppendix = 'wrapper'
    archiveClassifier = 'src'
    from 'somedir'
}

task echoNames {
    doLast {
        println "Project name: ${project.name}"
        println myZip.archiveFileName.get()
        println myOtherZip.archiveFileName.get()
    }
}
Output of gradle -q echoNames
> gradle -q echoNames
Project name: zipProject
gradle-1.0.zip
gradle-wrapper-1.0-src.zip

You can find all the possible archive task properties in the API documentation for AbstractArchiveTask, but we have also summarized the main ones here:

archiveFileName — Property<String>, default: archiveBaseName-archiveAppendix-archiveVersion-archiveClassifier.archiveExtension

The complete file name of the generated archive. If any of the properties in the default value are empty, their '-' separator is dropped.

archiveFile — Provider<RegularFile>, read-only, default: destinationDirectory/archiveFileName

The absolute file path of the generated archive.

destinationDirectory — DirectoryProperty, default: depends on archive type

The target directory in which to put the generated archive. By default, JARs and WARs go into $buildDir/libs. ZIPs and TARs go into $buildDir/distributions.

archiveBaseName — Property<String>, default: project.name

The base name portion of the archive file name, typically a project name or some other descriptive name for what it contains.

archiveAppendix — Property<String>, default: null

The appendix portion of the archive file name that comes immediately after the base name. It is typically used to distinguish between different forms of content, such as code and docs, or a minimal distribution versus a full or complete one.

archiveVersion — Property<String>, default: project.version

The version portion of the archive file name, typically in the form of a normal project or product version.

archiveClassifier — Property<String>, default: null

The classifier portion of the archive file name. Often used to distinguish between archives that target different platforms.

archiveExtension — Property<String>, default: depends on archive type and compression type

The filename extension for the archive. By default, this is set based on the archive task type and the compression type (if you’re creating a TAR). Will be one of: zip, jar, war, tar, tgz or tbz2. You can of course set this to a custom extension if you wish.

Sharing content between multiple archives

As described earlier, you can use the Project.copySpec(org.gradle.api.Action) method to share content between archives.

Reproducible archives

Sometimes it’s desirable to recreate archives exactly the same, byte for byte, on different machines. You want to be sure that building an artifact from source code produces the same result no matter when and where it is built. This is necessary for projects like reproducible-builds.org.

Reproducing the same byte-for-byte archive poses some challenges since the order of the files in an archive is influenced by the underlying file system. Each time a ZIP, TAR, JAR, WAR or EAR is built from source, the order of the files inside the archive may change. Files that only have a different timestamp also causes differences in archives from build to build. All AbstractArchiveTask (e.g. Jar, Zip) tasks shipped with Gradle include support for producing reproducible archives.

For example, to make a Zip task reproducible you need to set Zip.isReproducibleFileOrder() to true and Zip.isPreserveFileTimestamps() to false. In order to make all archive tasks in your build reproducible, consider adding the following configuration to your build file:

Example 47. Activating reproducible archives
GroovyKotlin
build.gradle
tasks.withType(AbstractArchiveTask) {
    preserveFileTimestamps = false
    reproducibleFileOrder = true
}

Often you will want to publish an archive, so that it is usable from another project. This process is described in Legacy Publishing.