由于StaticLoggerBinder.class中的重复数据删除错误,Scala SBT程序集无法合并

Question

我的问题是我不能再使用sbt-assembly插件了,因为在这个项目的几个人之间徘徊了某种依赖合并问题.

运行'sbt assembly'时的问题:

[error]在合并期间遇到3个错误java.lang.RuntimeException:deduplicate:在以下位置找到不同的文件内容:/Users/aris.vlasakakis/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback -classic-1.1.2.jar:org/slf4j/impl/StaticLoggerBinder.class /Users/aris.vlasakakis/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.6.1.jar: org/slf4j/impl/StaticLoggerBinder.class deduplicate:以下内容中找到的不同文件内容:/Users/aris.vlasakakis/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.1.2 .jar:org/slf4j/impl/StaticMDCBinder.class /Users/aris.vlasakakis/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.6.1.jar:org/slf4j/impl/ StaticMDCBinder.class deduplicate:以下内容中找到的不同文件内容:/Users/aris.vlasakakis/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.1.2.jar:org/slf4j /impl/StaticMarkerBinder.class /Users/aris.vlasakakis/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.6 .jar:org/slf4j/impl/StaticMarkerBinder.class at sbtassembly.Assembly $ .applyStrategies(Assembly.scala:140)at sbtassembly.Assembly $ .x $ 1 $ lzycompute $ 1(Assembly.scala:25)at sbtassembly.Assembly $ .x $ 1 $ 1(Assembly.scala:23)at sbtassembly.Assembly $ .stratMapping $ lzycompute $ 1(Assembly.scala:23)at sbtassembly.Assembly $ .stratMapping $ 1(Assembly.scala:23)at sbtassembly.Assembly $.输入$ lzycompute $ 1(Assembly.scala:67)at sbtassembly.Assembly $ .inputs $ 1(Assembly.scala:57)

... 等等

I am using SBT-assembly 0.13.0, and here is the build.sbt

name := "metamorphosis"

version := "0.10.0"

scalaVersion := "2.10.4"

lazy val common = RootProject(file("../"))

val main = Project(id = "bridge", base = file(".")).dependsOn(common)

//excludeDependencies += "org.apache.logging.log4j"

resolvers ++= Seq(
//  "Akka Snapshot Repository" at "http://repo.akka.io/snapshots/",
  "Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases/",
  "Sonatype OSS Releases"  at "http://oss.sonatype.org/content/repositories/releases/"
)

libraryDependencies ++= Seq(
  "com.github.nscala-time" %% "nscala-time" % "1.8.0",
  "com.sclasen" %% "akka-kafka" % "0.0.10" % "compile",
  "com.typesafe.akka" %% "akka-actor" % "2.3.2",
  "org.codehaus.groovy" % "groovy" % "2.3.7",
  "com.typesafe.scala-logging" %% "scala-logging-slf4j" % "2.1.2",
  "ch.qos.logback" % "logback-classic" % "1.1.2",
  "ch.qos.logback.contrib" % "logback-json-classic" % "0.1.2",
  "ch.qos.logback.contrib" % "logback-jackson" % "0.1.2",
  "org.slf4j" % "slf4j-api" % "1.7.7",
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.2.2",
  "org.clapper" %% "argot" % "1.0.3",
  "com.typesafe" % "config" % "1.2.1",
  "net.ceedubs" %% "ficus" % "1.0.1",
  "com.typesafe.play" %% "anorm" % "2.3.6",
  "org.json4s" %% "json4s-native" % "3.2.10",
  "org.json4s" %% "json4s-jackson" % "3.2.10",
  "com.github.tototoshi" %% "scala-csv" % "1.1.2",
  "org.scalatest" %% "scalatest" % "2.2.2",
  "junit" % "junit" % "4.11",
  "org.apache.kafka" %% "kafka" % "0.8.1.1"
    exclude("javax.jms", "jms")
    exclude("com.sun.jdmk", "jmxtools")
    exclude("com.sun.jmx", "jmxri")
    exclude("org.slf4j", "slf4j-simple")
)

mainClass in assembly := Some("com.company.kafka.agent.MetamorphosisActor")

有关如何解决如何修复此合并问题的任何线索？

谢谢!

Answer 1

以前的答案提供了所有必需的反馈,但实际上没有提供任何解决方案或步骤来尝试.

我的错误:

[error] (project/*:assembly) deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:org/slf4j/impl/StaticLoggerBinder.class
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticLoggerBinder.class
[error] deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:org/slf4j/impl/StaticMDCBinder.class
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMDCBinder.class
[error] deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:org/slf4j/impl/StaticMarkerBinder.class
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMarkerBinder.class

推荐方式

追踪冲突的依赖.例如,我不想要那些类:

org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticLoggerBinder.class
org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMDCBinder.class
org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.10.jar:org/slf4j/impl/StaticMarkerBinder.class

调用sbt(并确保使用sbt-dependency-graph插件):

whatDependsOn org.slf4j slf4j-log4j12 1.7.10

返回一个列表:

[info] org.slf4j:slf4j-log4j12:1.7.10
[info]   +-org.apache.hadoop:hadoop-auth:2.8.0
[info]   | +-org.apache.hadoop:hadoop-common:2.8.0
[info]   |   +-com.github.atais:test_2.11:0.0.3 [S]
[info]   |   
[info]   +-org.apache.hadoop:hadoop-common:2.8.0
[info]   | +com.github.atais:test_2.11:0.0.3 [S]
[info]   | 
[info]   +-org.apache.zookeeper:zookeeper:3.4.6
[info]     +-org.apache.curator:curator-client:2.7.1
[info]     | +-org.apache.curator:curator-framework:2.7.1
[info]     | | +-org.apache.curator:curator-recipes:2.7.1
[info]     | | | +-org.apache.hadoop:hadoop-common:2.8.0
[info]     | | |   +-com.github.atais:test_2.11:0.0.3 [S]
[info]     | | |   
[info]     | | +-org.apache.hadoop:hadoop-auth:2.8.0
[info]     | |   +-org.apache.hadoop:hadoop-common:2.8.0
[info]     | |     +-com.github.atais:test_2.11:0.0.3 [S]
[info]     | |     
[info]     | +-org.apache.hadoop:hadoop-common:2.8.0
[info]     |   +-com.github.atais:test_2.11:0.0.3 [S]
[info]     |   
[info]     +-org.apache.curator:curator-framework:2.7.1
[info]     | +-org.apache.curator:curator-recipes:2.7.1
[info]     | | +-org.apache.hadoop:hadoop-common:2.8.0
[info]     | |   +-com.github.atais:test_2.11:0.0.3 [S]
[info]     | |   
[info]     | +-org.apache.hadoop:hadoop-auth:2.8.0
[info]     |   +-org.apache.hadoop:hadoop-common:2.8.0
[info]     |     +-com.github.atais:test_2.11:0.0.3 [S]
[info]     |     
[info]     +-org.apache.curator:curator-recipes:2.7.1
[info]     | +-org.apache.hadoop:hadoop-common:2.8.0
[info]     |   +-com.github.atais:test_2.11:0.0.3 [S]
[info]     |   
[info]     +-org.apache.hadoop:hadoop-auth:2.8.0
[info]     | +-org.apache.hadoop:hadoop-common:2.8.0
[info]     |   +-com.github.atais:test_2.11:0.0.3 [S]
[info]     |   
[info]     +-org.apache.hadoop:hadoop-common:2.8.0
[info]       +-com.github.atais:test_2.11:0.0.3 [S]

我需要追踪hadoop-common依赖关系,因为它是连接com.github.atais:test_2.11:0.0.3到不需要的依赖关系org.slf4j:slf4j-log4j12:1.7.10

并修改它:

libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.8.1" exclude("org.slf4j", "slf4j-log4j12")

简单的方法

只需为冲突路径添加合并策略标志:

assemblyMergeStrategy in assembly := {
  ...
  case PathList("org", "slf4j", xs@_*) => MergeStrategy.first
  case x => (assemblyMergeStrategy in assembly).value(x)
}

Answer 2

问题是不止一个依赖会带来slf4j传递依赖,从而导致冲突.最好的方法是排除传递依赖,而不是使用SBT程序集的合并策略来解决冲突.

首先,您需要可视化slf4j的来源(适用于所有顶级依赖项).您可以使用此sbt-dependency-graph之类的工具来获取所有依赖项的可视化图表.或者,您可以从SBT命令提示符中探索SBT设置,或使用类似这一个或此一个的maven站点.

找到重复的依赖项后exclude,请使用与在构建文件中使用的方法相同的方法.只留下单一的slf4j传递依赖.您还可以排除所有slf4j传递依赖项(请参阅此处),并slf4j在顶层手动添加该依赖项.

请注意,如果您的依赖项需要多个不同版本,slf4j那么选择适用于所有情况的单个版本可能会有问题.取决于二进制兼容性.