下载文件时内存不足

Question

我想在android下载一个CSV文件。下载后，我使用Jackson进行映射，然后继续使用sqlite3 db。我面临的问题是当我拨打下载文件的电话时，我得到了一个OutOfMemoryError。文件大小在12MB附近。当我在浏览器中尝试相同时，我能够下载该文件，因此我不认为它与端点有关。

要下载，我正在使用AsyncTask。

class DownloadCSVTask extends AsyncTask<Void, String, Void> {

    private final Call<ResponseBody> csvDump;
    private final Context context;

    public DownloadCSVTask(Call<ResponseBody> csvDump, Context context) {
        this.csvDump = csvDump;
        this.context = context;
    }

    @Override
    protected Void doInBackground(Void... voids) {
        try {
            Response<ResponseBody> response = csvDump.execute();
            File fileFromResponse = createFileFromResponse(response);
            mapFileContentsToModelAndPersist(fileFromResponse);
        } catch (IOException e) {
            Log.e(getClass().getSimpleName(), "CSV pull failed");
        }
        return null;
    }

    private File createFileFromResponse(Response<ResponseBody> response) throws IOException {
        File path = context.getFilesDir();
        File csvFile = File.createTempFile(TEMP_PREFIX, CSV_EXTENSION, path);
        FileOutputStream fileOutputStream = context.openFileOutput(csvFile.getName(), Context.MODE_PRIVATE);
        fileOutputStream.write(response.body().bytes());
        Log.i(getClass().getCanonicalName(), "Writing file completed" + csvFile.getAbsolutePath());
        fileOutputStream.close();
        return csvFile;
    }

    private void mapFileContentsToModelAndPersist(File fileFromResponse) {
        try {
            MappingIterator<City> cityIter = new CsvMapper().readerWithTypedSchemaFor(City.class).readValues(fileFromResponse);
            List<City> cities = cityIter.readAll();
            Stream.of(cities).forEach(City::persist);
        } catch (IOException e) {
            Log.e(getClass().getSimpleName(), "Failed to map contents of file to City Class");
        }
    }
}

我得到的错误日志：

 java.lang.RuntimeException: An error occured while executing doInBackground()
                         E      at android.os.AsyncTask$3.done(AsyncTask.java:300)
                         E      at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
                         E      at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
                         E      at java.util.concurrent.FutureTask.run(FutureTask.java:242)
                         E      at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
                         E      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
                         E      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
                         E      at java.lang.Thread.run(Thread.java:841)
                         E  Caused by: java.lang.OutOfMemoryError
                         E      at java.lang.String.<init>(String.java:354)
                         E      at java.lang.String.<init>(String.java:393)
                         E      at okio.Buffer.readString(Buffer.java:616)
                         E      at okio.Buffer.readString(Buffer.java:599)
                         E      at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:263)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
                         E      at com.someclass.somepath.network.util.CustomHeadersInterceptor.intercept(CustomHeadersInterceptor.java:42)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
                         E      at com.someclass.somepath.network.util.NetworkInterceptor.intercept(NetworkInterceptor.java:23)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
                         E      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
                         E      at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:170)
                         E      at okhttp3.RealCall.execute(RealCall.java:60)
                         E      at retrofit2.OkHttpCall.execute(OkHttpCall.java:174)
                         E      at retrofit2.ExecutorCallAdapterFactory$ExecutorCallbackCall.execute(ExecutorCallAdapterFactory.java:89)
                         E      at com.someclass.somepath.service.DownloadCSVTask.doInBackground(CsvPullHandler.java:60)
                         E      at com.someclass.somepath.service.DownloadCSVTask.doInBackground(CsvPullHandler.java:47)
                         E      at android.os.AsyncTask$2.call(AsyncTask.java:288)
                         E      at java.util.concurrent.FutureTask.run(FutureTask.java:237)

我在这里做错了吗？还有另一种方法吗？

Answer 1

Android上Java代码的内存量限制为16 Mb（因设备而异）。因此，您需要将此部分移动到本机NDK代码，其中malloc（）可以与设备一样多的内存，或者将文件直接保存到磁盘，然后处理它而不将整个文件加载到Java缓冲区中。在带有16Kb字节数组的循环中使用URLConnection.getInputStream()或URLConnection.read()，而不是将整个文件复制到response数组中。或者，您可以使用BufferedReader.readLine()直接从HTTP流中逐行读取，并逐行填充数据库。