如何将大型XML文件转换为对象的流式java8

Question

Hy,我有一个(非常)大的XML文件(100GB)和一个foo列表,我想将它转换成一个流,就像它们引入对象的java 8一样:

任何lib或代码示例的想法？

在开始时:

<foos> 
  <foo>...</foo>
  <foo>...</foo>
</foos>

在末尾:

Stream<Foo>  foosStream = ????("foo.xml")
streamFoos.forEach(foo->foo.doFooStuffs());

编辑:@Pierre谢谢,这是您的解决方案的实现:

  try {
            XMLEventReader reader = XMLInputFactory.newInstance().
                    createXMLEventReader(stream);
            final Unmarshaller unmarshaller = JAXBContext.newInstance(XXXXX.class).createUnmarshaller();

            Iterator<XXXXX> it = new XmlIterator<>(reader, unmarshaller, "xxxxxx");
            return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, Spliterator.ORDERED), false);
        } catch (XMLStreamException e1) {
            logger.error("XMLStreamException", e1);
        } catch (JAXBException e) {
            logger.error("JAXBException", e);
        }

和

public class XmlIterator<T> implements Iterator<T> {

    private final Logger logger = LoggerFactory.getLogger(this.getClass());

    XMLEventReader reader;

    XMLEvent event;

    Unmarshaller unmarshaller;
    String name;

    public XmlIterator(XMLEventReader reader, Unmarshaller unmarshaller, String name) {
        this.reader = reader;
        this.unmarshaller = unmarshaller;
        this.name = name;
        try {
            reader.next();
            this.event = reader.peek();
        } catch (XMLStreamException e) {
            logger.error("", e);
            event = null;
        }
    }

    @Override
    public boolean hasNext() {
        try {
            while (event != null && !(event.isStartElement() && name.equals(event.asStartElement().getName().getLocalPart()))) {
                Object a = reader.next();
                event = reader.peek();
            }
            return event != null;

        } catch (XMLStreamException e) {
            logger.error("", e);
            event = null;
        }
        return event != null;
    }

    @Override
    public T next() {
        try {
            T next = ((JAXBElement<T>) unmarshaller.unmarshal(reader)).getValue();
            event = reader.peek();
            return next;
        } catch (JAXBException e) {
            logger.error("error during unmarshalling ", e);
            return null;
        } catch (XMLStreamException e) {
            logger.error("error during stream ", e);
            return null;
        }
    }
}

Answer 1

• 创建一个Stax事件阅读器https://docs.oracle.com/javase/tutorial/jaxp/stax/example.html读取您的xml文件

• 每次看到标签'foo'(使用https://docs.oracle.com/javase/7/docs/api/javax/xml/stream/XMLEventReader.html#peek())时,请使用Stax阅读器进行解析并构建您的对象Foo

  Foo readFoo(XMLEventReader xmlIn) throws XMLStreamException {
     (...)
     return foo; 
  }
  

• 实现一个java.util.Iterator,它将使用前一个函数返回'next()'Foo

• 将此迭代器转换为流如何从迭代器创建Java 8流？