小编mat*_*usr的帖子

我有一个使用Spark SQL的Java应用程序(Spark 1.5.2使用本地模式),但是我不能在没有错误的情况下执行任何SQL命令.

这是我正在执行的代码:

//confs
SparkConf sparkConf = new SparkConf();  
sparkConf.set("spark.master","local");
sparkConf.set("spark.app.name","application01");
sparkConf.set("spark.driver.host","10.1.1.36");
sparkConf.set("spark.driver.port", "51810");
sparkConf.set("spark.executor.port", "51815");
sparkConf.set("spark.repl.class.uri","http://10.1.1.36:46146");
sparkConf.set("spark.executor.instances","2");
sparkConf.set("spark.jars","");
sparkConf.set("spark.executor.id","driver");
sparkConf.set("spark.submit.deployMode","client");
sparkConf.set("spark.fileserver.uri","http://10.1.1.36:47314");
sparkConf.set("spark.localProperties.clone","true");
sparkConf.set("spark.app.id","app-45631207172715-0002");

//Initialize contexts
JavaSparkContext sparkContext = new JavaSparkContext(sparkConf);
SQLContext sqlContext = new SQLContext(sparkContext);           

//execute command
sqlContext.sql("show tables").show();

pom.xml中的 Spark依赖关系如下所示:

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.10</artifactId>
  <version>1.5.2</version>
</dependency>

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-sql_2.10</artifactId>
  <version>1.5.2</version>
</dependency>

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-hive_2.10</artifactId>
  <version>1.5.2</version>
</dependency>

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-repl_2.10</artifactId>
  <version>1.5.2</version>
</dependency>

这是我得到的错误:

java.lang.NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;

该堆栈跟踪是在这里.

我的应用程序是在Tomcat 7上运行的Web应用程序.我没有任何其他配置文件.我能做错什么？可能是一些依赖冲突,因为我能够在一个干净的项目中运行相同的代码？

编辑 …

我想将一个R脚本运行到eclipse中.

• R 3.1.1
• Ubuntu 12.04
• Eclipse Kepler

• 罐子:

  • JRI.jar
  • REngine.jar
  • RserveEngine.jar

• 环境(运行 - >运行配置 - >环境)

  • R_HOME:/ usr/local/lib/R.

码:

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import org.rosuda.REngine.REXP;
import org.rosuda.REngine.REXPMismatchException;
import org.rosuda.REngine.Rserve.RConnection;
import org.rosuda.REngine.Rserve.RserveException;
import org.rosuda.JRI.Rengine;

public class HelloWorldApp {

    public static void main(String[] args) throws RserveException, REXPMismatchException, FileNotFoundException, IOException {
        RConnection c = new RConnection("localhost",6311);
        if(c.isConnected()) {
            System.out.println("Connected to RServe.");
            if(c.needLogin()) {
                System.out.println("Providing Login");
                c.login("username", "password");
            }

            REXP x = c.eval("1:10");
            for(int i=0;i < x.length();i++)
            { …