无法运行Giraph SimpleInDegreeCountComputation

Question

我正在尝试运行Giraph附带的SimpleInDegreeCountComputation示例.我的方法如下:

SimpleInDegreeCountComputation.java:

    public class SimpleInDegreeCountComputation extends BasicComputation
              <LongWritable, LongWritable, DoubleWritable, DoubleWritable> {
    .......

然后我尝试运行它:

    hadoop jar /path-to-giraph-folder/giraph-examples/target/giraph-examples-1.1.0- 
    SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar 
    org.apache.giraph.GiraphRunner  
    org.apache.giraph.examples.SimpleInDegreeCountComputation 
    -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat 
    -vip /path-to-input-file 
    -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat 
    -op /path-to-output-file -w 1 

结果如下:

    14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge input format specified. 
    Ensure your InputFormat does not require one.
    14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge output format specified.  
    Ensure your OutputFormat does not require one.
    Exception in thread "main" java.lang.IllegalArgumentException: checkClassTypes: vertex  
    value types not assignable, computation - class org.apache.hadoop.io.LongWritable,   
    VertexInputFormat - class org.apache.hadoop.io.DoubleWritable
at org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

我不太确定我做错了什么.如果有人可以指出我正确的方向,或链接到一个资源,解释一个更容易的方式,我正在尝试做,我会非常感激!我认为问题可能是错误的格式(-vif).我正在使用的输入文件如下:

    [0,0,[[1,5],[2,9]]]
    [1,0,[[0,5],[3,3]]]
    [2,0,[[0,9],[3,3],[4,3]]]
    [3,0,[[1,3],[2,3],[4,2]]]
    [4,0,[[2,3],[3,3]]]

Answer 1

看一下compute和vertex-input类的定义,似乎与JsonLongDoubleFloatDoubleVertexInputFormat它不兼容SimpleInDegreeCountComputation

SimpleInDegreeCountComputation:

public class SimpleInDegreeCountComputation extends BasicComputation<
    LongWritable, LongWritable, DoubleWritable, DoubleWritable> {

BasicComputation:

/**
 * Computation in which both incoming and outgoing message types are the same.
 *
 * @param <I> Vertex id
 * @param <V> Vertex data
 * @param <E> Edge data
 * @param <M> Message type
 */
public abstract class BasicComputation<I extends WritableComparable,
    V extends Writable, E extends Writable, M extends Writable>
    extends AbstractComputation<I, V, E, M, M> {
}

你可以看到:

• 顶点id是Type LongWritable
• 顶点数据是Type LongWritable
• 边缘数据属于Type DoubleWritable

...另一方面,您尝试使用的InputFormat ...

JsonLongDoubleFloatDoubleVertexInputFormat:

public class JsonLongDoubleFloatDoubleVertexInputFormat extends
    TextVertexInputFormat<LongWritable, DoubleWritable, FloatWritable> {

TextVertexInputFormat:

/**
 * Abstract class that users should subclass to use their own text based
 * vertex input format.
 *
 * @param <I> Vertex index value
 * @param <V> Vertex value
 * @param <E> Edge value
 */
@SuppressWarnings("rawtypes")
public abstract class TextVertexInputFormat<I extends WritableComparable,
    V extends Writable, E extends Writable>
    extends VertexInputFormat<I, V, E> {

你可以看到:

• 顶点id是Type LongWritable
• 顶点数据是Type DoubleWritable
• 边缘数据属于Type FloatWritable

因为它是LongWritable,DoubleWritable而FloatWritable不是Long,Double和Float- 这些类型无法自动转换.

我找不到您可以使用的任何InputFormat,因此您需要修改现有JsonLongDoubleFloatDoubleVertexInputFormat或修改NullWritable用于Edge数据类型的算法.我没有看到任何可以使用的Edge数据,所以它也可以为null.在这种情况下,您可以使用LongLongNullTextInputFormat.