A simple Groovy issue tracker using file system

Zemian DengDecember 24th, 2012Last Updated: December 24th, 2012

1 32 7 minutes read

It will be a chaos not to track bugs and feature requests when you developing software. Having a simple issue tracker would make managing the project much more successful. Now I like simple stuff, and I think for small project, having this tracker right inside the source control (especially with DSVC like Mercurial/Git etc) repository is not only doable, but very convenient as well. You don’t have to go crazy with all the fancy features, but just enough to track issues are fine. I would like to propose this layout for you.

Let’s say you have a project that looks like this

project
 +- src/main/java/Hello.java
 +- issues/issue-001.md
 +- pom.xml

All I need is a simple directory issues to get going. Now I have a place to track my issue! First issue issue-000.md should be what your project is about. For example:

/id=issue-001
/createdon=2012-12-16 18:07:08
/type=bug
/status=new
/resolution=
/from=Zemian
/to=
/found=
/fixed=
/subject=A simple Java Hello program

# Updated on 2012-12-16 18:07:08

We want to create a Maven based Hello world program. It should print 'Hello World.'

I choose .md as file extension for intending to write comments in Markdown format. Since it’s a text file, you do what you want. To be more structured, I have added some headers metadata for issue tracking. Let’s define some here. I would propose to use these and formatting:

/id=issue-<NUM>
 /createdon=<TIMESTAMP>
 /type=feature|bug|question
 /status=new|reviewing|working|onhold|testing|resolved
 /resolution=fixed|rejected|duplicated
 /from=<REPORTER_FROM_NAME>
 /to=<ASSIGNEE_TO_NAME>
 /found=<VERSION_FOUND>
 /fixed=<VERSION_FIXED>

That should cover most of the bug and feature development issues. It’s not cool to write software without a history of changes, including these issues created. So let’s use a source control. I highly recommend you to use Mercurial hg. You can create and initialize a new repository like this.

bash> cd project
bash> hg init
bash> hg add
bash> hg commit -m 'My hello world project'

Now your project is created and we have a place to track your issues. Now it’s simple text file, so use your favorite text editor and edit away. However, creating new issue with those header tags is boring. It will be nice to have a script that manage it a little. I have a Groovy script issue.groovy (see at the end of this article) that let you run reports and create new issues. You can add this script into your project/issues directory and you can instantly creating new issue and querying reports! Here is an example output on my PC:

bash> cd project
bash> groovy scripts/issue.groovy

Searching for issues with /status!=resolved
Issue: /id=issue-001 /status=new /subject=A simple Java Hello program
1 issues found.

bash> groovy scripts/issue.groovy --new /type=feature /subject='Add a unit test.'

project/issues/issue-002.md created.
/id=issue-002
/createdon=2012-12-16 19:10:00
/type=feature
/status=new
/resolution=
/from=Zemian
/to=
/found=
/fixed=
/subject=Add a unit test.

bash> groovy scripts/issue.groovy

Searching for issues with /status!=resolved
Issue: /id=issue-000 /status=new /subject=A simple Java Hello program
Issue: /id=issue-002 /status=new /subject=Add a unit test.
2 issues found.

bash> groovy scripts/issue.groovy --details /id=002

Searching for issues with /id=002
Issue: /id=issue-002
  /createdon=2012-12-16 19:10:00 /found= /from=Zemian /resolution= /status=new /type=feature
  /subject=Add a unit test.
1 issues found.

bash> groovy scripts/issue.groovy --update /id=001 /status=resolved /resolution=fixed 'I fixed this thang.'
Updating issue /id=issue-001
Updating /status=resolved
Updating /resolution=fixed

Update issue-001 completed.

The script give you some quick and consistent way to create/update/search issues. But they are just plain text files! You can just as well fire up your favorite text editor and change any any thing you want. Save and even commit it into your source repository. All will not lost.
Here is my issue.groovy script:

#!/usr/bin/env groovy
//
// A groovy script to manage issue files and its metadata/headers.
//
// Created by Zemian Deng <saltnlight5@gmail.com> 12/2012 v1.0.1
//
// Usage:
//  bash> groovy [java_opts] issue.groovy [option] [/header_name=value...] [arguments]
//
// Examples:
//   # Report all issues that match headers (we support RegEx!)
//   bash> groovy issue /resolution=fixed
//   bash> groovy issue /status!=onhold
//   bash> groovy issue '/subject=Improve UI|service'
//   bash> groovy issue --details /status=resolved
//
//   # Create a new bug issue file.
//   bash> groovy issue --new /type=bug /to=zemian /found=v1.0.1 /subject='I found some problem.' 'More details here.'
//
//   # Update an issue
//   bash> groovy issue --update /id=issue-001 /status=resolved /resolution=fixed 'I fixed this issue with Z algorithm.'
//
// Becareful on the following notes:
//  * Ensure your filename issue id match to the /id or your search may not work!
//  * You need to use quote the entire header such as these 'key=space value'
//
class issue {
	def ISSUES_HEADERS = ['/id', '/createdon', '/type', '/status', '/resolution', '/from', '/to', '/found', '/fixed', '/subject']
	def ISSUES_HEADERS_VALS = [
		'/type' : ['feature', 'bug', 'question'] as Set, 
		'/status' : ['new', 'reviewing', 'working', 'onhold', 'testing', 'resolved'] as Set, 
		'/resolution' : ['fixed', 'rejected', 'duplicated'] as Set
		]
	def issuesDir = new File(System.getProperty("issuesDir", getDefaultIssuesDir()))
	def issuePrefix = System.getProperty("issuePrefix", 'issue')
	def arguments = [] // script arguments after parsing
	def options = [:]  // script options after parsing
	def headers = [:]  // user input issue headers

	static void main(String[] args) {
		new issue().run(args)
	}

	// Method declarations
	def run(String[] args) {
		// Parse and save options, arguments and headers vars
		args.each { arg ->
			def append = true
			if (arg =~ /^--{0,1}\w+/) {
				options[arg] = true
				append = false
			} else {
				def pos = arg.indexOf('=')
				if (pos >= 1 && arg.length() > pos) {
					def name = arg.substring(0, pos)
					def value = arg.substring(pos + 1)				
					headers.put(name, value)
					append = false
				}
			}

			if (append) {
				arguments << arg
			}
		}

		// support short option flag
		if (options['-d'])
			options['--details'] = true

		// Run script depending on options passed
		if (options['--help'] || options['-h']) {
			printHelp()
		} else if (options['--new'] || options['-n']) {
			createIssue()
		} else if (options['--update'] || options['-u']) {
			updateIssue()
		} else {
			reportIssues()
		}
	}

	def printHelp() {
		new File(getClass().protectionDomain.codeSource.location.path).withReader{ reader ->
			def done = false
			def line = null
			while (!done && (line = reader.readLine()) != null) {
				line = line.trim()
				if (line.startsWith("#") || line.startsWith("//"))
					println(line)
				else
					done = true
			}
		}
	}

	def validateHeaders() {
		def headersSet = ISSUES_HEADERS.toSet()
		headers.each{ name, value ->
			if (!headersSet.contains(name)) 
				throw new Exception("ERROR: Unkown header name $name.")
			if (ISSUES_HEADERS_VALS[name] != null && !(ISSUES_HEADERS_VALS[name].contains(value)))
				throw new Exception("ERROR: Unkown header $name=$value. Allowed: ${ISSUES_HEADERS_VALS[name].join(', ')}")
		}
	}

	def getDefaultIssuesDir() {	
		return new File(getClass().protectionDomain.codeSource.location.path).parentFile.path 
	}

	def getIssueIds() {
		def issueIds = []
		def files = issuesDir.listFiles()
		if (files == null)
			return issueIds
		files.each{ f ->
			def m = f.name =~ /^(\w+-\d+)\.md$/
			if (m)
				issueIds << m[0][1]
		}
		return issueIds
	}

	def getIssueFile(String issueid) {
		return new File(issuesDir, "${issueid}.md")
	}

	def reportIssues() {
		def userHeaders = new HashMap(headers)
		if (userHeaders.size() ==0)
			userHeaders['/status!'] = 'resolved'
		def headersLine = userHeaders.sort{ a,b -> a.key <=> b.key }.collect{ k,v -> "$k=$v" }.join(', ')	
		println "Searching for issues with $headersLine"
		def count = 0
		getIssueIds().each { issueid ->
			def file = getIssueFile(issueid)
			def issueHeaders = [:]
			file.withReader{ reader ->
				def done = false
				def line = null
				while (!done && (line = reader.readLine()) != null) {
					if (line =~ /^\/\w+=.*$/) {
						def words = line.split('=')
						if (words.length >= 2) {
							issueHeaders.put(words[0], words[1..-1].join('='))
						}
					} else if (issueHeaders.size() > 0) {
						done = true
					}
				}
			}
			def match = userHeaders.findAll{ k,v ->
				if (k.endsWith("!"))
					(issueHeaders[k.substring(0, k.length() - 1)] =~ /${v}/) ? false : true
				else
					(issueHeaders[k] =~ /${v}/) ? true : false
			}
			if (match.size() == userHeaders.size()) {
				def line = "Issue: /id=${issueHeaders['/id']}"
				if (options['--details']) {
					def col = 4
					def issueHeadersKeys = issueHeaders.keySet().sort() - ['/id', '/subject']
					issueHeadersKeys.collate(col).each { set ->
						line += "\n  " + set.collect{ k -> "$k=${issueHeaders[k]}" }.join(" ")
					}
					line += "\n  /subject=${issueHeaders['/subject']}"
				} else {
					line +=
						" /status=${issueHeaders['/status']}" +
						" /subject=${issueHeaders['/subject']}"
				}
				println line
				count += 1
			}
		}
		println "$count issues found."
	}

	def createIssue() {
		validateHeaders()
		if (headers['/status'] == 'resolved' && headers['/resolution'] == null)
			throw new Exception("You must provide /resolution after resolved an issue.")

		def ids = getIssueIds().collect{ issueid -> issueid.split('-')[1].toInteger() }
		def nextid = ids.size() > 0 ? ids.max() + 1 : 1
		def issueid = String.format("${issuePrefix}-%03d", nextid)
		def file = getIssueFile(issueid)
		def createdon = new Date().format('yyyy-MM-dd HH:mm:ss')
		def newHeaders = [
			'/id' :  issueid,
			'/createdon' :  createdon,
			'/type' : 'bug',
			'/status' : 'new',
			'/resolution' : '',
			'/from' : System.properties['user.name'],
			'/to' :  '',
			'/found' : '',
			'/fixed' : '',
			'/subject' : 'A bug report'
		]
		// Override newHeaders from user inputs
		headers.each { k,v -> newHeaders.put(k, v) }

		//Output to file
		file.withWriter{ writer ->
			ISSUES_HEADERS.each{ k -> writer.println("$k=${newHeaders[k]}")	}
			writer.println()
			writer.println("# Updated on ${createdon}")
			writer.println()
			arguments.each { 
				writer.println(it) 
				writer.println()
			}
			writer.println()
		}

		// Output issue headers to STDOUT
		println "$file created."
		ISSUES_HEADERS.each{ k -> println("$k=${newHeaders[k]}") }
	}

	def updateIssue() {
		validateHeaders()
		if (headers['/status'] == 'resolved' && headers['/resolution'] == null)
			throw new Exception("You must provide /resolution after resolved an issue.")

		def userHeaders = new HashMap(headers)
		userHeaders.remove('/createdon') // we should not update this field

		def issueid = userHeaders.remove('/id') // We will not re-update /id
		if (issueid == null)
			throw new Exception("Failed to update issue: missing /id value.")
		if (!issueid.startsWith(issuePrefix))
			issueid = "${issuePrefix}-${issueid}"
		println("Updating issue /id=${issueid}")

		def file = getIssueFile(issueid)
		def newFile = new File(file.parentFile, "${file.name}.update.tmp")
		def hasUpdate = false
		def issueHeaders = [:]

		if (!file.exists())
			throw new Exception("Failed to update issue: file not found for /id=${issueid}")

		// Read and update issue headers
		file.withReader{ reader ->
			// Read all issue headers first
			def done = false
			def line = null
			while (!done && (line = reader.readLine()) != null) {
				if (line =~ /^\/\w+=.*$/) {
					def words = line.split('=')
					if (words.length >= 2) {
						issueHeaders.put(words[0], words[1..-1].join('='))
					}
				} else if (issueHeaders.size() > 0) {
					done = true
				}
			}

			// Find issue headers differences
			userHeaders.each{ k,v -> 
				if (issueHeaders[k] != v) {
					println("Updating $k=$v")
					issueHeaders[k] = v
					if (!hasUpdate)
						hasUpdate = true
				}
			}

			// Update issue file
			if (hasUpdate) {
				newFile.withWriter{ writer ->
					ISSUES_HEADERS.each{ k -> writer.println("${k}=${issueHeaders[k] ?: ''}") }
					writer.println()

					// Write/copy the rest of the file.
					done = false
					while (!done && (line = reader.readLine()) != null) {
						writer.println(line)
					}
					writer.println()
				}
			}
		} // reader

		if (hasUpdate) {
			// Rename the new file back to orig
			file.delete()
			newFile.renameTo(file)
		}

		// Append any arguments as user comments
		if (arguments.size() > 0) {
			file.withWriterAppend{ writer ->
				writer.println()
				writer.println("# Updated on ${new Date().format('yyyy-MM-dd HH:mm:ss')}")
				writer.println()
				arguments.each{ text -> 
					writer.println(text)
					writer.println()
				}
				println()
			}
		}

		println("Update $issueid completed.")
	}
}

Reference: A simple Groovy issue tracker using file system from our JCG partner Zemian Deng at the A Programmer’s Journal blog.