Ignite Cache - 将外部化对象作为二进制存储在服务器上

Question

我们正在尝试以客户端服务器模式将 ignite 缓存集成到我们的解决方案中。我们不希望将所有类复制到所有 ignite 实例，而且在某些情况下不可能预定义这些类（例如客户自定义类）。

缓存中存储的一些类是可序列化的，一些是可外部化的。

根据文档（和日志），我们看到 Serialized 类在服务器上以二进制形式存储，而Externalized 则尝试在服务器上反序列化。由于这些类在服务器上不可用，我们收到错误（见下文）。

我们尝试将 PeerClassLoadingEnabled 设置为 true ，并测试了所有部署模式，但发现远程对等类加载不适用于缓存。也在文档中：

https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading

存在以下评论：

对等类加载功能不会部署存储在缓存中的条目的键和对象类。

我们还尝试使用 JdkMarshaller（在客户端和服务器上都配置了它），但是看起来问题仍然存在，并且对象正在尝试在服务器上反序列化。

因此：是否可以强制 ignite 在服务器上以二进制形式存储 Serialized 和Externalized 类，而不是尝试反序列化它们（我们不能在所有Externalized 类上实现 Binarylized ，因为其中一些类不在我们的控制之下）？如果没有，是否有一个选项可以强制在节点之间加载对等类以进行缓存？

10:42:34,690 WARNING [org.apache.ignite.internal.binary.BinaryContext] (RMI TCP Connection(4)-127.0.0.1) Class "org.restcomm.cluster.ClusteredUUID" cannot be serialized using BinaryMarshaller because it either implements Externalizable interface or have writeObject/readObject methods. OptimizedMarshaller will be used instead and class instances will be deserialized on the server. Please ensure that all nodes have this class in classpath. To enable binary serialization either implement Binarylizable interface or set explicit serializer using BinaryTypeConfiguration.setSerializer() method.
10:42:34,696 WARNING [org.apache.ignite.internal.binary.BinaryContext] (RMI TCP Connection(4)-127.0.0.1) Class "org.restcomm.cluster.ClusteredUUID" cannot be serialized using BinaryMarshaller because it either implements Externalizable interface or have writeObject/readObject methods. OptimizedMarshaller will be used instead and class instances will be deserialized on the server. Please ensure that all nodes have this class in classpath. To enable binary serialization either implement Binarylizable interface or set explicit serializer using BinaryTypeConfiguration.setSerializer() method.
10:42:34,713 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class org.apache.ignite.IgniteCheckedException
10:42:34,713 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class org.apache.ignite.binary.BinaryObjectException
10:42:34,714 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class org.apache.ignite.IgniteException
10:42:34,717 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class [Ljava.lang.StackTraceElement;
10:42:34,718 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1) class org.apache.ignite.IgniteException: Failed to unmarshal object with optimized marshaller
10:42:34,719 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1)   at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKey(GridCacheAdapter.java:697)
10:42:34,719 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1)   at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.containsKey(IgniteCacheProxyImpl.java:1252)
10:42:34,719 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1)   at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.containsKey(GatewayProtectedCacheProxy.java:796)

Answer 1

To enable binary serialization either implement Binarylizable interface or set explicit serializer using BinaryTypeConfiguration.setSerializer() method.

你尝试过后者吗？

Answer 2

目前我已经通过添加外部化类的包装器解决了这个问题。我不确定性能以及这是否是最佳选择，但它有效。

public class BinaryExternalizable implements Binarylizable
{
private Externalizable wrappedObject;

public BinaryExternalizable() {
    //empty class for reading
}

public BinaryExternalizable(Externalizable wrappedObject) {
    this.wrappedObject=wrappedObject;
}

@Override 
public void writeBinary(BinaryWriter writer) throws BinaryObjectException {
    ByteBuf buffer = Unpooled.buffer();
    ByteBufOutputStream bos = new ByteBufOutputStream(buffer);
    try {
        ObjectOutputStream out = new ObjectOutputStream(bos);
        out.writeUTF(wrappedObject.getClass().getCanonicalName());
        wrappedObject.writeExternal(out);
        out.close();
        bos.close();
    }
    catch(IOException ex) {
        throw new BinaryObjectException(ex.getMessage(), ex);
    }
    
    byte[] data=new byte[buffer.readableBytes()];
    buffer.readBytes(data);
    writer.rawWriter().writeByteArray(data);
}
 
@Override 
public void readBinary(BinaryReader reader) throws BinaryObjectException {
    byte[] data = reader.rawReader().readByteArray();
    ByteBufInputStream bis=new ByteBufInputStream(Unpooled.wrappedBuffer(data));
    try {
        ObjectInputStream ins=new ObjectInputStream(bis);       
        String className=ins.readUTF();         
        wrappedObject = (Externalizable)Class.forName(className).newInstance();
        wrappedObject.readExternal(ins);
        ins.close();
        bis.close();
    } catch(IOException | ClassNotFoundException | IllegalAccessException | InstantiationException ex) {
        throw new BinaryObjectException(ex.getMessage(),ex);
    }
}

public Externalizable getWrappedObject() {
    return this.wrappedObject;
}
}