如何将多个avro对象写入ByteArrayOutputStream

Question

Avro 文档页面有一个将多个 avro 对象写入文件的示例：

DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
dataFileWriter.create(user1.getSchema(), new File("users.avro"));
dataFileWriter.append(user1);
dataFileWriter.append(user2);
dataFileWriter.append(user3);
dataFileWriter.close();

我想实现类似的目标，但写入 ByteArrayOutputStream。

ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
Encoder encoder = EncoderFactory.get().binaryEncoder(outputStream, null);
DatumWriter<MyData> myDataDatumWriter = new SpecificDatumWriter<>(MyData.getClassSchema());

myDataDatumWriter.write(myData1, encoder);
encoder.flush();
myDataDatumWriter.write(myData2, encoder);
encoder.flush();

我将输出流中的数据作为http响应发送，但数据无法在客户端反序列化，因为它不是可识别的avro格式。

这段代码有什么问题？

Answer 1

您只需从字节数组创建一个

SeekableByteArrayInput

，然后使用

DataFileReader

读取它，例如：

public static void main(String[] args) throws IOException {
    final ByteArrayOutputStream baos = new ByteArrayOutputStream();

    // First write the records to the ByteArrayOutputStream.

    final DatumWriter<User> dataWriter = new SpecificDatumWriter<>(User.class);
    try (final DataFileWriter<User> dataFileWriter =
                 new DataFileWriter<>(dataWriter).create(User.SCHEMA$, baos)
    ) {
        dataFileWriter.append(myData1);
        dataFileWriter.append(myData2);
        dataFileWriter.append(myData3);
    }

    // Now read the records back in from the byte array.

    final DatumReader<User> dataReader = new SpecificDatumReader<>(User.class);

    final List<User> recs = new ArrayList<>();
    try (
            final SeekableInput seekIn = new SeekableByteArrayInput(baos.toByteArray());
            final DataFileReader<User> dataFileReader = new DataFileReader<>(seekIn, dataReader)
    ) {
        while (dataFileReader.hasNext()) {
            recs.add(dataFileReader.next());
        }
    }

    assert(recs.size() == 3);
}