A simple Groovy issue tracker using file system

Zemian DengDecember 24th, 2012Last Updated: December 24th, 2012

1 35 7 minutes read

It will be a chaos not to track bugs and feature requests when you developing software. Having a simple issue tracker would make managing the project much more successful. Now I like simple stuff, and I think for small project, having this tracker right inside the source control (especially with DSVC like Mercurial/Git etc) repository is not only doable, but very convenient as well. You don’t have to go crazy with all the fancy features, but just enough to track issues are fine. I would like to propose this layout for you.

Let’s say you have a project that looks like this

project
 +- src/main/java/Hello.java
 +- issues/issue-001.md
 +- pom.xml

All I need is a simple directory issues to get going. Now I have a place to track my issue! First issue issue-000.md should be what your project is about. For example:

/id=issue-001
/createdon=2012-12-16 18:07:08
/type=bug
/status=new
/resolution=
/from=Zemian
/to=
/found=
/fixed=
/subject=A simple Java Hello program
 
# Updated on 2012-12-16 18:07:08
 
We want to create a Maven based Hello world program. It should print 'Hello World.'

I choose .md as file extension for intending to write comments in Markdown format. Since it’s a text file, you do what you want. To be more structured, I have added some headers metadata for issue tracking. Let’s define some here. I would propose to use these and formatting:

/id=issue-<NUM>
 /createdon=<TIMESTAMP>
 /type=feature|bug|question
 /status=new|reviewing|working|onhold|testing|resolved
 /resolution=fixed|rejected|duplicated
 /from=<REPORTER_FROM_NAME>
 /to=<ASSIGNEE_TO_NAME>
 /found=<VERSION_FOUND>
 /fixed=<VERSION_FIXED>

That should cover most of the bug and feature development issues. It’s not cool to write software without a history of changes, including these issues created. So let’s use a source control. I highly recommend you to use Mercurial hg. You can create and initialize a new repository like this.

bash> cd project
bash> hg init
bash> hg add
bash> hg commit -m 'My hello world project'

Now your project is created and we have a place to track your issues. Now it’s simple text file, so use your favorite text editor and edit away. However, creating new issue with those header tags is boring. It will be nice to have a script that manage it a little. I have a Groovy script issue.groovy (see at the end of this article) that let you run reports and create new issues. You can add this script into your project/issues directory and you can instantly creating new issue and querying reports! Here is an example output on my PC:

bash> cd project
bash> groovy scripts/issue.groovy
 
Searching for issues with /status!=resolved
Issue: /id=issue-001 /status=new /subject=A simple Java Hello program
1 issues found.
 
bash> groovy scripts/issue.groovy --new /type=feature /subject='Add a unit test.'
 
project/issues/issue-002.md created.
/id=issue-002
/createdon=2012-12-16 19:10:00
/type=feature
/status=new
/resolution=
/from=Zemian
/to=
/found=
/fixed=
/subject=Add a unit test.
 
bash> groovy scripts/issue.groovy
 
Searching for issues with /status!=resolved
Issue: /id=issue-000 /status=new /subject=A simple Java Hello program
Issue: /id=issue-002 /status=new /subject=Add a unit test.
2 issues found.
 
bash> groovy scripts/issue.groovy --details /id=002
 
Searching for issues with /id=002
Issue: /id=issue-002
  /createdon=2012-12-16 19:10:00 /found= /from=Zemian /resolution= /status=new /type=feature
  /subject=Add a unit test.
1 issues found.
 
bash> groovy scripts/issue.groovy --update /id=001 /status=resolved /resolution=fixed 'I fixed this thang.'
Updating issue /id=issue-001
Updating /status=resolved
Updating /resolution=fixed
 
Update issue-001 completed.

The script give you some quick and consistent way to create/update/search issues. But they are just plain text files! You can just as well fire up your favorite text editor and change any any thing you want. Save and even commit it into your source repository. All will not lost.
Here is my issue.groovy script:

#!/usr/bin/env groovy
//
// A groovy script to manage issue files and its metadata/headers.
//
// Created by Zemian Deng <saltnlight5@gmail.com> 12/2012 v1.0.1
//
// Usage:
//  bash> groovy [java_opts] issue.groovy [option] [/header_name=value...] [arguments]
//
// Examples:
//   # Report all issues that match headers (we support RegEx!)
//   bash> groovy issue /resolution=fixed
//   bash> groovy issue /status!=onhold
//   bash> groovy issue '/subject=Improve UI|service'
//   bash> groovy issue --details /status=resolved
//
//   # Create a new bug issue file.
//   bash> groovy issue --new /type=bug /to=zemian /found=v1.0.1 /subject='I found some problem.' 'More details here.'
//
//   # Update an issue
//   bash> groovy issue --update /id=issue-001 /status=resolved /resolution=fixed 'I fixed this issue with Z algorithm.'
//
// Becareful on the following notes:
//  * Ensure your filename issue id match to the /id or your search may not work!
//  * You need to use quote the entire header such as these 'key=space value'
//
class issue {
    def ISSUES_HEADERS = ['/id', '/createdon', '/type', '/status', '/resolution', '/from', '/to', '/found', '/fixed', '/subject']
    def ISSUES_HEADERS_VALS = [
        '/type' : ['feature', 'bug', 'question'] as Set, 
        '/status' : ['new', 'reviewing', 'working', 'onhold', 'testing', 'resolved'] as Set, 
        '/resolution' : ['fixed', 'rejected', 'duplicated'] as Set
        ]
    def issuesDir = new File(System.getProperty("issuesDir", getDefaultIssuesDir()))
    def issuePrefix = System.getProperty("issuePrefix", 'issue')
    def arguments = [] // script arguments after parsing
    def options = [:]  // script options after parsing
    def headers = [:]  // user input issue headers
 
    static void main(String[] args) {
        new issue().run(args)
    }
 
    // Method declarations
    def run(String[] args) {
        // Parse and save options, arguments and headers vars
        args.each { arg ->
            def append = true
            if (arg =~ /^--{0,1}\w+/) {
                options[arg] = true
                append = false
            } else {
                def pos = arg.indexOf('=')
                if (pos >= 1 && arg.length() > pos) {
                    def name = arg.substring(0, pos)
                    def value = arg.substring(pos + 1)              
                    headers.put(name, value)
                    append = false
                }
            }
 
            if (append) {
                arguments << arg
            }
        }
 
        // support short option flag
        if (options['-d'])
            options['--details'] = true
 
        // Run script depending on options passed
        if (options['--help'] || options['-h']) {
            printHelp()
        } else if (options['--new'] || options['-n']) {
            createIssue()
        } else if (options['--update'] || options['-u']) {
            updateIssue()
        } else {
            reportIssues()
        }
    }
 
    def printHelp() {
        new File(getClass().protectionDomain.codeSource.location.path).withReader{ reader ->
            def done = false
            def line = null
            while (!done && (line = reader.readLine()) != null) {
                line = line.trim()
                if (line.startsWith("#") || line.startsWith("//"))
                    println(line)
                else
                    done = true
            }
        }
    }
 
    def validateHeaders() {
        def headersSet = ISSUES_HEADERS.toSet()
        headers.each{ name, value ->
            if (!headersSet.contains(name)) 
                throw new Exception("ERROR: Unkown header name $name.")
            if (ISSUES_HEADERS_VALS[name] != null && !(ISSUES_HEADERS_VALS[name].contains(value)))
                throw new Exception("ERROR: Unkown header $name=$value. Allowed: ${ISSUES_HEADERS_VALS[name].join(', ')}")
        }
    }
 
    def getDefaultIssuesDir() { 
        return new File(getClass().protectionDomain.codeSource.location.path).parentFile.path 
    }
 
    def getIssueIds() {
        def issueIds = []
        def files = issuesDir.listFiles()
        if (files == null)
            return issueIds
        files.each{ f ->
            def m = f.name =~ /^(\w+-\d+)\.md$/
            if (m)
                issueIds << m[0][1]
        }
        return issueIds
    }
 
    def getIssueFile(String issueid) {
        return new File(issuesDir, "${issueid}.md")
    }
 
    def reportIssues() {
        def userHeaders = new HashMap(headers)
        if (userHeaders.size() ==0)
            userHeaders['/status!'] = 'resolved'
        def headersLine = userHeaders.sort{ a,b -> a.key <=> b.key }.collect{ k,v -> "$k=$v" }.join(', ')   
        println "Searching for issues with $headersLine"
        def count = 0
        getIssueIds().each { issueid ->
            def file = getIssueFile(issueid)
            def issueHeaders = [:]
            file.withReader{ reader ->
                def done = false
                def line = null
                while (!done && (line = reader.readLine()) != null) {
                    if (line =~ /^\/\w+=.*$/) {
                        def words = line.split('=')
                        if (words.length >= 2) {
                            issueHeaders.put(words[0], words[1..-1].join('='))
                        }
                    } else if (issueHeaders.size() > 0) {
                        done = true
                    }
                }
            }
            def match = userHeaders.findAll{ k,v ->
                if (k.endsWith("!"))
                    (issueHeaders[k.substring(0, k.length() - 1)] =~ /${v}/) ? false : true
                else
                    (issueHeaders[k] =~ /${v}/) ? true : false
            }
            if (match.size() == userHeaders.size()) {
                def line = "Issue: /id=${issueHeaders['/id']}"
                if (options['--details']) {
                    def col = 4
                    def issueHeadersKeys = issueHeaders.keySet().sort() - ['/id', '/subject']
                    issueHeadersKeys.collate(col).each { set ->
                        line += "\n  " + set.collect{ k -> "$k=${issueHeaders[k]}" }.join(" ")
                    }
                    line += "\n  /subject=${issueHeaders['/subject']}"
                } else {
                    line +=
                        " /status=${issueHeaders['/status']}" +
                        " /subject=${issueHeaders['/subject']}"
                }
                println line
                count += 1
            }
        }
        println "$count issues found."
    }
 
    def createIssue() {
        validateHeaders()
        if (headers['/status'] == 'resolved' && headers['/resolution'] == null)
            throw new Exception("You must provide /resolution after resolved an issue.")
 
        def ids = getIssueIds().collect{ issueid -> issueid.split('-')[1].toInteger() }
        def nextid = ids.size() > 0 ? ids.max() + 1 : 1
        def issueid = String.format("${issuePrefix}-%03d", nextid)
        def file = getIssueFile(issueid)
        def createdon = new Date().format('yyyy-MM-dd HH:mm:ss')
        def newHeaders = [
            '/id' :  issueid,
            '/createdon' :  createdon,
            '/type' : 'bug',
            '/status' : 'new',
            '/resolution' : '',
            '/from' : System.properties['user.name'],
            '/to' :  '',
            '/found' : '',
            '/fixed' : '',
            '/subject' : 'A bug report'
        ]
        // Override newHeaders from user inputs
        headers.each { k,v -> newHeaders.put(k, v) }
 
        //Output to file
        file.withWriter{ writer ->
            ISSUES_HEADERS.each{ k -> writer.println("$k=${newHeaders[k]}")  }
            writer.println()
            writer.println("# Updated on ${createdon}")
            writer.println()
            arguments.each { 
                writer.println(it) 
                writer.println()
            }
            writer.println()
        }
 
        // Output issue headers to STDOUT
        println "$file created."
        ISSUES_HEADERS.each{ k -> println("$k=${newHeaders[k]}") }
    }
 
    def updateIssue() {
        validateHeaders()
        if (headers['/status'] == 'resolved' && headers['/resolution'] == null)
            throw new Exception("You must provide /resolution after resolved an issue.")
 
        def userHeaders = new HashMap(headers)
        userHeaders.remove('/createdon') // we should not update this field
 
        def issueid = userHeaders.remove('/id') // We will not re-update /id
        if (issueid == null)
            throw new Exception("Failed to update issue: missing /id value.")
        if (!issueid.startsWith(issuePrefix))
            issueid = "${issuePrefix}-${issueid}"
        println("Updating issue /id=${issueid}")
 
        def file = getIssueFile(issueid)
        def newFile = new File(file.parentFile, "${file.name}.update.tmp")
        def hasUpdate = false
        def issueHeaders = [:]
 
        if (!file.exists())
            throw new Exception("Failed to update issue: file not found for /id=${issueid}")
 
        // Read and update issue headers
        file.withReader{ reader ->
            // Read all issue headers first
            def done = false
            def line = null
            while (!done && (line = reader.readLine()) != null) {
                if (line =~ /^\/\w+=.*$/) {
                    def words = line.split('=')
                    if (words.length >= 2) {
                        issueHeaders.put(words[0], words[1..-1].join('='))
                    }
                } else if (issueHeaders.size() > 0) {
                    done = true
                }
            }
 
            // Find issue headers differences
            userHeaders.each{ k,v -> 
                if (issueHeaders[k] != v) {
                    println("Updating $k=$v")
                    issueHeaders[k] = v
                    if (!hasUpdate)
                        hasUpdate = true
                }
            }
 
            // Update issue file
            if (hasUpdate) {
                newFile.withWriter{ writer ->
                    ISSUES_HEADERS.each{ k -> writer.println("${k}=${issueHeaders[k] ?: ''}") }
                    writer.println()
 
                    // Write/copy the rest of the file.
                    done = false
                    while (!done && (line = reader.readLine()) != null) {
                        writer.println(line)
                    }
                    writer.println()
                }
            }
        } // reader
 
        if (hasUpdate) {
            // Rename the new file back to orig
            file.delete()
            newFile.renameTo(file)
        }
 
        // Append any arguments as user comments
        if (arguments.size() > 0) {
            file.withWriterAppend{ writer ->
                writer.println()
                writer.println("# Updated on ${new Date().format('yyyy-MM-dd HH:mm:ss')}")
                writer.println()
                arguments.each{ text -> 
                    writer.println(text)
                    writer.println()
                }
                println()
            }
        }
 
        println("Update $issueid completed.")
    }
}