小编Jac*_*Jac的帖子

我试图在Flink中使用Scala XML库来解析XML,但我无法使其工作.请注意,我需要在相同的处理函数中对我的代码使用序列化和非序列化(字符串)版本.

我尝试过不同的解决方案,它们总是在IntelliJ中工作,但是当我在Flink集群上运行时却没有.他们总是回归不同java.lang.LinkageError: com/sun/org/apache/xerces/internal/jaxp/SAXParserImpl$JAXPSAXParser; 我尝试了很多东西,但我仍然得到类似于这个的错误.

这是我的Flink Job的样子:

object StreamingJob {
  import org.apache.flink.streaming.api.scala._

  val l = List(
    """<ciao>ciao</ciao>""",
  )

  def main(args: Array[String]): Unit = {
    val env = StreamExecutionEnvironment.getExecutionEnvironment
    // set up kafka section excluded
    env.setParallelism(10)

    val stream = env.fromCollection(l)

    stream
      .uid("process")
      .map(new Processor)
      .print

    env.execute("Flink-TEST")
  }
}

这是我的处理功能的一个例子:

import javax.xml.parsers.{SAXParser, SAXParserFactory}
import org.apache.flink.api.common.functions.MapFunction
import scala.xml.{Elem, XML}
import scala.xml.factory.XMLLoader

class Processor extends MapFunction[String, String] {
  override def map(translatedMessage: String): String = {
    val xml = Processor.xmlLoader.loadString(translatedMessage)
    xml.toString
  }
} …

我正在努力将Google Cloud PubSub集成到一个示例c#项目中,我是c#的新手,因为这可能是我在公司工作的唯一c#项目,因为有一些要求与c#编写的游戏集成.我使用NuGet安装Google.Cloud.PubSub.V1.0.0-beta13并且安装成功,但是当我尝试运行使用docs创建的示例代码时,我收到以下错误:

C:/Users/MyUser/RiderProjects/TestConsole/TestConsole/bin/Debug/TestConsole.exe

Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'Google.Apis.Auth, Version=1.21.0.0, Culture=neutral, PublicKeyToken=4b01fa6e34db77ab' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at Google.Api.Gax.TaskExtensions.WaitWithUnwrappedExceptions(Task task) in C:\Users\jon\Test\Projects\gax-dotnet\releasebuild\src\Google.Api.Gax\TaskExtensions.cs:line 48
   at Google.Api.Gax.Grpc.ChannelPool.GetChannel(ServiceEndpoint endpoint) in C:\Users\jon\Test\Projects\gax-dotnet\releasebuild\src\Google.Api.Gax.Grpc\ChannelPool.cs:line 92
   at Google.Cloud.PubSub.V1.PublisherClient.Create(ServiceEndpoint endpoint, PublisherSettings settings) in C:\Users\jon\Test\Projects\google-cloud-dotnet\releasebuild\apis\Google.Cloud.PubSub.V1\Google.Cloud.PubSub.V1\PublisherClient.cs:line 558
   at TestConsole.Program.CreateTopic(String projectId, String topicId) in C:\Users\MyUser\RiderProjects\TestConsole\TestConsole\Program.cs:line 11
   at TestConsole.Program.Main(String[] args) in C:\Users\MyUser\RiderProjects\TestConsole\TestConsole\Program.cs:line 32

然后,我尝试将Google.Apis.Auth降级为1.21.0但问题仍然转移到"无法加载Google.Api.Gax,版本= 1.0.1.0",然后(如果我继续降级依赖关系)在Google.Protobuf 3.2上. 0.0,然后在Google.Apis.Core 1.24.1然后回到"无法加载Google.Apis.Auth 1.21.0",所以我猜问题是在其他地方.

是什么导致这种依赖性问题 …

我试图使用Apache Beam执行管道但是在尝试放置一些输出标签时出现错误:

import com.google.cloud.Tuple;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.io.gcp.pubsub.PubsubIO;
import org.apache.beam.sdk.io.gcp.pubsub.PubsubMessage;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.windowing.FixedWindows;
import org.apache.beam.sdk.transforms.windowing.Window;
import org.apache.beam.sdk.values.TupleTag;
import org.apache.beam.sdk.values.TupleTagList;
import org.joda.time.Duration;

import java.lang.reflect.Type;
import java.util.Map;
import java.util.stream.Collectors;

/**
 * The Transformer.
 */
class Transformer {
    final static TupleTag<Map<String, String>> successfulTransformation = new TupleTag<>();
    final static TupleTag<Tuple<String, String>> failedTransformation = new TupleTag<>();

    /**
     * The entry point of the application.
     *
     * @param args the input arguments
     */
    public static void main(String... …

当创建 Play 框架项目并用于WSClient进行 REST 调用时，官方 Play 框架文档建议添加ws到build.sbt管理依赖项。如果使用 Maven，则 ws 依赖项包含在：

<dependency>
  <groupId>com.typesafe.play</groupId>
  <artifactId>play-ws_2.12</artifactId>
  <version>${play2.version}</version>
</dependency>

但是，当尝试使用如下所示的片段调用 Web 服务时：

@Singleton
class Controller @Inject()(
  ws: WSClient,
  controllerComponents: ControllerComponents
)(implicit ec: ExecutionContext)
  extends AbstractController(controllerComponents) {
  def callApi(): Action[AnyContent] = Action.async { _ =>
    ws
      .url("https://mywebservice.com/api/bla")
      .get()
      .map(response => Ok(response.body.toString))
  }
}

然后出现如下错误：

CreationException: Unable to create injector, see the following errors:

1) No implementation for play.api.libs.ws.WSClient was bound.
  while locating play.api.libs.ws.WSClient
    for the 1st parameter of controllers.MyController.<init>(MyController.scala:13) …