In a recent project, we had to do something I had personally never really had to look at; Compression. We needed to take a couple files and images, zip them up and make them available for FTP, and yes somedays it does feel like we are back in the 90’s. Besides the FTP trip into the past its was good opportunity to spend a little bit of time on the subject.
Compressing Files
So above the usual IO classes BufferedInputStream, FileOutputStream and File there are:
- ZipInputStream – An input stream for reading files in the ZIP file format. Zip entries are not cached, unlike ZipFile.
- ZipOutputStream – An output stream for writing files in the ZIP file format. This has a default internal buffer of 512, a BufferedOutputStream can be used to increase this.
- ZipEntry – Represents an entry int a zip file.
- ZipFile – Used to read entries from a zip file. The entries are cached.
- CRC32 – Used to compute the CRC-32 of a data stream.
Below is an example showing how to compress and decompress files in a folder, with and without a checksum:
package javaitzen.blog;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
import java.util.zip.CRC32;
import java.util.zip.CheckedInputStream;
import java.util.zip.CheckedOutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
/**
* The Class FileCompressionUtil.
*/
public class FileCompressionUtil {
private static final String PATH_SEP = "\\";
public static final int BUFFER = 2048;
private FileCompressionUtil() {}
/**
* Zip files in path.
*
* @param zipFileName the zip file name
* @param filePath the file path
* @throws IOException Signals that an I/O exception has occurred.
*/
public static void zipFilesInPath(final String zipFileName, final String filePath) throws IOException {
final FileOutputStream dest = new FileOutputStream(zipFileName);
final ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(dest));
try {
byte[] data = new byte[BUFFER];
final File folder = new File(filePath);
final List< String > files = Arrays.asList(folder.list());
for (String file : files) {
final FileInputStream fi = new FileInputStream(filePath + PATH_SEP + file);
final BufferedInputStream origin = new BufferedInputStream(fi, BUFFER);
out.putNextEntry(new ZipEntry(file));
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
fi.close();
}
} finally {
out.close();
dest.close();
}
}
/**
* Zip with checksum. CRC32
*
* @param zipFileName the zip file name
* @param folderPath the folder path
* @return the checksum
* @throws IOException Signals that an I/O exception has occurred.
*/
public static long zipFilesInPathWithChecksum(final String zipFileName, final String folderPath) throws IOException {
final FileOutputStream dest = new FileOutputStream(zipFileName);
final CheckedOutputStream checkStream = new CheckedOutputStream(dest, new CRC32());
final ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(checkStream));
try {
byte[] data = new byte[BUFFER];
final File folder = new File(folderPath);
final List< String > files = Arrays.asList(folder.list());
for (String file : files) {
final FileInputStream fi = new FileInputStream(folderPath + PATH_SEP + file);
final BufferedInputStream origin = new BufferedInputStream(fi, BUFFER);
out.putNextEntry(new ZipEntry(file));
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
}
} finally {
out.close();
checkStream.close();
dest.flush();
dest.close();
}
return checkStream.getChecksum().getValue();
}
/**
* Unzip files to path.
*
* @param zipFileName the zip file name
* @param fileExtractPath the file extract path
* @throws IOException Signals that an I/O exception has occurred.
*/
public static void unzipFilesToPath(final String zipFileName, final String fileExtractPath) throws IOException {
final FileInputStream fis = new FileInputStream(zipFileName);
final ZipInputStream zis = new ZipInputStream(new BufferedInputStream(fis));
try {
ZipEntry entry;
while ((entry = zis.getNextEntry()) != null) {
int count;
byte[] data = new byte[BUFFER];
final FileOutputStream fos = new FileOutputStream(fileExtractPath + PATH_SEP + entry.getName());
final BufferedOutputStream dest = new BufferedOutputStream(fos, BUFFER);
while ((count = zis.read(data, 0, BUFFER)) != -1) {
dest.write(data, 0, count);
}
dest.flush();
dest.close();
}
} finally {
fis.close();
zis.close();
}
}
/**
* Unzip files to path with checksum. CRC32
*
* @param zipFileName the zip file name
* @param fileExtractPath the file extract path
* @param checksum the checksum
* @return true, if checksum matches;
* @throws IOException Signals that an I/O exception has occurred.
*/
public static boolean unzipFilesToPathWithChecksum(final String zipFileName, final String fileExtractPath, final long checksum) throws IOException {
boolean checksumMatches = false;
final FileInputStream fis = new FileInputStream(zipFileName);
final CheckedInputStream checkStream = new CheckedInputStream(fis, new CRC32());
final ZipInputStream zis = new ZipInputStream(new BufferedInputStream(checkStream));
try {
ZipEntry entry = null;
while ((entry = zis.getNextEntry()) != null) {
int count;
byte[] data = new byte[BUFFER];
final FileOutputStream fos = new FileOutputStream(fileExtractPath + PATH_SEP + entry.getName());
final BufferedOutputStream dest = new BufferedOutputStream(fos, BUFFER);
while ((count = zis.read(data, 0, BUFFER)) != -1) {
dest.write(data, 0, count);
}
dest.flush();
dest.close();
}
} finally {
zis.close();
fis.close();
checkStream.close();
}
if(checkStream.getChecksum().getValue() == checksum) {
checksumMatches = true;
}
return checksumMatches;
}
}
Compressing Objects
We didn’t end up using object compression but I had a look at it anyways. I did a little generic compress / expand util, don’t know if it will ever be useful. I left the input params as OutputStream and InputStream as this could theoretically be used with any stream implementation from socket communication to string manipulation.
The compression related classes being used here:
- GZIPInputStream – An input stream filter for reading compressed data in the GZIP file format.
- GZIPOutputStream – An output stream filter for writing compressed data in the GZIP file format.
- Default internal buffer of 512, use BufferedOutputStream if you require more.
package javaitzen.blog;
import java.io.IOException;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.io.Serializable;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
/**
* The Class ObjectCompressionUtil.
*
* @param <T> the generic type of the serializable object to be compressed
*/
public class ObjectCompressionUtil<T extends Serializable> {
/**
* Compress object.
*
* @param objectToCompress the object to compress
* @param outstream the outstream
* @return the compressed object
* @throws IOException Signals that an I/O exception has occurred.
*/
public T compressObject(final T objectToCompress, final OutputStream outstream) throws IOException {
final GZIPOutputStream gz = new GZIPOutputStream(outstream);
final ObjectOutputStream oos = new ObjectOutputStream(gz);
try {
oos.writeObject(objectToCompress);
oos.flush();
return objectToCompress;
}finally {
oos.close();
outstream.close();
}
}
/**
* Expand object.
*
* @param objectToExpand the object to expand
* @param instream the instream
* @return the expanded object
* @throws IOException Signals that an I/O exception has occurred.
* @throws ClassNotFoundException the class not found exception
*/
public T expandObject(final T objectToExpand, final InputStream instream) throws IOException,
ClassNotFoundException {
final GZIPInputStream gs = new GZIPInputStream(instream);
final ObjectInputStream ois = new ObjectInputStream(gs);
try {
@SuppressWarnings("unchecked")
T expandedObject = (T) ois.readObject();
return expandedObject;
} finally {
gs.close();
ois.close();
}
}
}
Reference: Java Compression from our JCG partner Brian at Zen in the art of IT.
Happy coding!
Byron
Related Articles :
