Eric Bergman-Terrell's Blog

Previous Blog Post Java Tip: How to read a file and automatically specify the correct encoding Next Blog Post
November 15, 2008

By default, Java reads text files using the default encoding. If you know a file's encoding, it can be specified in the constructor of the FileInputStream object that's used to read the file. The following class automates the process of reading files, using the correct encoding, provided the encoding of the file is either UTF-8, UTF-16 (big or little endian), or the system default.

package mainPackage;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;

public class FileUtils {
/***
 * Determines the encoding of the specified file. If a UTF16 Byte Order Mark (BOM) is found an encoding of "UTF16" is returned.
 * If a UTF8 BOM is found an encoding of "UTF8" is returned. Otherwise the default encoding is returned.
 * @param filePath file path
 * @return "UTF8", "UTF16", or default encoding.
 */
private static String getEncoding(String filePath) {
String encoding = System.getProperty("file.encoding");  

BufferedReader bufferedReader = null;

try {
// In order to read files with non-default encoding, specify an encoding in the FileInputStream constructor.
bufferedReader = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));

char buffer[] = new char[3];
int length = bufferedReader.read(buffer);

if (length >= 2) {
if ((buffer[0] == (char) 0xff && buffer[1] == (char) 0xfe) /* UTF-16, little endian */ || 
    (buffer[0] == (char) 0xfe && buffer[1] == (char) 0xff) /* UTF-16, big endian */) {
encoding = "UTF16";
}
}
if (length >= 3) {
if (buffer[0] == (char) 0xef && buffer[1] == (char) 0xbb && buffer[2] == (char) 0xbf) /* UTF-8 */  {
encoding = "UTF8";
}
}
}
catch (IOException ex) {
}
finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
}
catch (IOException ex) {
}
}
}
 
return encoding;
}

/***
 * Returns the text of the specified file. If a Unicode Byte Order Mark (BOM) is found, the file is read with the corresponding encoding.
 * Otherwise the file is read using the default encoding.
 * @param filePath file path
 * @return text of file
 * @throws IOException
 */
public static String readFile(String filePath) throws IOException {
String encoding = getEncoding(filePath);

BufferedReader bufferedReader = null;

StringBuffer text = new StringBuffer();

try {
bufferedReader = new BufferedReader(new InputStreamReader(new FileInputStream(filePath), encoding));

char[] buffer = new char[1024 * 16];
int length;

while ((length = bufferedReader.read(buffer)) != -1) {
text.append(buffer, 0, length);
}
}
finally {
if (bufferedReader != null) {
bufferedReader.close();
}
}

return text.toString();
}
}
Keywords: Java, file, encoding, Unicode, UTF8, UTF-8, UTF16, UTF-16, file I/O, FileInputStream, InputStreamReader, BufferedReader

Reader Comments

Name Comment URL Date/Time
Victor This is not helpful, because only very few files do have a BOM. And UTF-8 with BOM is discouraged. Better use a library like GuessEncoding or juniversalchardet.
  October 22, 2013

Comment on this Blog Post

Recent Posts

Title Date
PowerShell Tip: Using PowerShell to change the Power Plan June 26, 2014
Vault 0.53 Released March 29, 2013
Vault 3 (Desktop Version) Updated March 23, 2013
EBTCalc for Android v. 1.12 - Now Runs on Phones and Smaller Tablets February 09, 2013
Vault 3 (PC Version) v. 0.50 Released January 07, 2013
Android Programming Tip: Recycle Views in Adapters December 07, 2012
Android Programming Tip: How to Embed EditText Views in a ListView (Don't!) December 07, 2012