在多线程中从文件生成哈希[关闭]

问题描述 投票:0回答:0

我想在 java 中从大文件(50G 或更多...)生成哈希,但这需要很多时间。

我尝试

MessageDigest
(java核心类)和一些使用它但不能在多线程中实现这个问题的库。

import java.io.FileInputStream;
import java.security.MessageDigest;
import java.util.concurrent.*;

public class HashFile {
    private static final int THREAD_COUNT = 4; // number of threads to use
    private static final int CHUNK_SIZE = 10 * 1024 * 1024; // chunk size in bytes
    private static final ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);

    public static void main(String[] args) throws Exception {
        StringBuilder sb = new StringBuilder();
        for (byte b : hashFile("D:\\big.rar")) {
            sb.append(String.format("%02x", b));
        }
        String hashedString = sb.toString();

        System.out.println("SHA-512 hash of file is " + hashedString);
    }
    public static byte[] hashFile(String filePath) throws Exception {
        MessageDigest md = MessageDigest.getInstance("SHA-512");
        byte[] buffer = new byte[CHUNK_SIZE];
        long fileSize = new java.io.File(filePath).length();
        int chunks = (int) Math.ceil((double) fileSize / CHUNK_SIZE);
        Future<byte[]>[] futures = new Future[chunks];

        try (FileInputStream fis = new FileInputStream(filePath)) {
            for (int i = 0; i < chunks; i++) {
                int bytesRead = fis.read(buffer);
                if (bytesRead < CHUNK_SIZE) {
                    byte[] lastChunk = new byte[bytesRead];
                    System.arraycopy(buffer, 0, lastChunk, 0, bytesRead);
                    futures[i] = executor.submit(() -> md.digest(lastChunk));
                } else {
                    byte[] chunk = new byte[CHUNK_SIZE];
                    System.arraycopy(buffer, 0, chunk, 0, bytesRead);
                    futures[i] = executor.submit(() -> md.digest(chunk));
                }
            }
        }

        byte[] result = md.digest();
        for (int i = 0; i < chunks; i++) {
            result = md.digest(futures[i].get());
        }

        executor.shutdown();
        return result;
    }
}

上面的代码每次运行都返回不同的结果

我想在多线程中实现它以获得更好的性能和时间。

java multithreading hash
© www.soinside.com 2019 - 2024. All rights reserved.