当我使用“arq”查询本地 rdf 文件时，为什么会出现“超出 GC 开销限制”的问题

Question

我正在使用ARQ来查询本地 RDF 文件。我正在使用的命令如下：

./arq --data /home/datasets/a-m-00027.nt --results CSV --query myQuery.sparql

---

myQuery.sparql 包含查询：

PREFIX basekb:<http://rdf.basekb.com/ns/>
PREFIX rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>

SELECT ?x
FROM  </home/data/a-m-00027.nt>
WHERE {?x rdf:type basekb:music.release} 
LIMIT 10

---

例外

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.concurrent.CopyOnWriteArrayList.iterator(CopyOnWriteArrayList.java:959)
        at com.hp.hpl.jena.graph.impl.SimpleEventManager.notifyAddTriple(SimpleEventManager.java:97)
        at com.hp.hpl.jena.graph.impl.GraphBase.notifyAdd(GraphBase.java:124)
        at com.hp.hpl.jena.graph.impl.GraphBase.add(GraphBase.java:203)
        at com.hp.hpl.jena.sparql.core.DatasetGraphCollection.add(DatasetGraphCollection.java:43)
        at com.hp.hpl.jena.sparql.core.DatasetGraphBase.add(DatasetGraphBase.java:82)
        at org.apache.jena.riot.system.StreamRDFLib$ParserOutputDataset.triple(StreamRDFLib.java:206)
        at org.apache.jena.riot.lang.LangNTriples.runParser(LangNTriples.java:61)
        at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:42)
        at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTLang.read(RDFParserRegistry.java:185)
        at org.apache.jena.riot.RDFDataMgr.process(RDFDataMgr.java:906)
        at org.apache.jena.riot.RDFDataMgr.parse(RDFDataMgr.java:687)
        at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:534)
        at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:501)
        at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:454)
        at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:432)
        at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:422)
        at arq.cmdline.ModDatasetGeneral.addGraphs(ModDatasetGeneral.java:98)
        at arq.cmdline.ModDatasetGeneral.createDataset(ModDatasetGeneral.java:87)
        at arq.cmdline.ModDatasetGeneralAssembler.createDataset(ModDatasetGeneralAssembler.java:35)
        at arq.cmdline.ModDataset.getDataset(ModDataset.java:34)
        at arq.query.getDataset(query.java:176)
        at arq.query.queryExec(query.java:198)
        at arq.query.exec(query.java:159)
        at arq.cmdline.CmdMain.mainMethod(CmdMain.java:102)
        at arq.cmdline.CmdMain.mainRun(CmdMain.java:63)
        at arq.cmdline.CmdMain.mainRun(CmdMain.java:50)
        at arq.arq.main(arq.java:28)

---

事实例子

<http://rdf.basekb.com/ns/architecture.building_complex>        <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>       <http://rdf.basekb.com/ns/type.type> 

---

是否将整个文件加载到内存中？

Answer 1

是否将整个文件加载到内存中？

没错，这就是你的问题。如前所述，您可能能够碰撞 Java 堆并使其适合。

但作为替代方案，或者对于您根本没有足够内存的情况，请尝试使用 TDB 来存储和索引文件，然后查询它：

$ tdbloader --loc my_tdb_store /home/datasets/a-m-00027.nt
$ tdbquery --loc my_tdb_store --results CSV --query myQuery.sparql

（完成后可以删除商店，它只是一个名为 的目录my_tdb_store）

作为第三种选择，您可以完全跳过 sparql。您查询只会找到带有 type 的前十件事basekb:music.release，您可以这样找到：

$ riot /home/datasets/a-m-00027.nt | \
  grep '<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdf.basekb.com/ns/music.release> .' | \
  cut -d ' ' -f 1 | \
  head -10

它使用最少的内存。