我正在尝试设置一个基本的Java使用者来接收来自Kafka主题的消息.我已经按照以下示例访问了 - https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example - 并且拥有以下代码:
package org.example.kafka.client;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import kafka.consumer.ConsumerConfig;
import kafka.consumer.KafkaStream;
import kafka.javaapi.consumer.ConsumerConnector;
public class KafkaClientMain
{
private final ConsumerConnector consumer;
private final String topic;
private ExecutorService executor;
public KafkaClientMain(String a_zookeeper, String a_groupId, String a_topic)
{
this.consumer = kafka.consumer.Consumer.createJavaConsumerConnector(
createConsumerConfig(a_zookeeper, a_groupId));
this.topic = a_topic;
}
private static ConsumerConfig createConsumerConfig(String a_zookeeper, String a_groupId) {
Properties props = new Properties();
props.put("zookeeper.connect", a_zookeeper);
props.put("group.id", a_groupId);
props.put("zookeeper.session.timeout.ms", "1000");
props.put("zookeeper.sync.time.ms", "1000");
props.put("auto.commit.interval.ms", "1000"); …Run Code Online (Sandbox Code Playgroud) 我正在尝试让Giraph在YARN集群上运行(Hadoop 2.5.2),但我遇到了这个错误:
Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster
Run Code Online (Sandbox Code Playgroud)
我已经尝试了以前关于这个主题的消息中找到的所有内容,但无济于事.我的命令行是这样的:
hadoop jar /home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/prhodes/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/prhodes/giraph_output/shortestpaths -w 4 -yj /home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar,/home/prhodes/giraph/giraph-core/target/giraph-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar
Run Code Online (Sandbox Code Playgroud)
我已经检查过这些jar是在群集中每个节点上的那些位置.另外,我已经验证了jar文件确实包含GiraphApplicationMaster.class.然而它仍然抱怨没有找到GiraphApplicationMaster.
有什么想法可能导致问题,或任何解决方法?我考虑过将Giraph jar分发到每个节点并将它们物理地放在Hadoop Classpath上,但默认情况下Maven build会构建fat jar,这可能会导致问题.还有其他建议或想法吗?
配置:
MacOS 文图拉,13.5
JDK:Liberica JDK 11 和/或 Liberica JDK 17
系统 Gradle:Gradle 8.2.1
Grails:Grails 版本:6.0.0
问题:
我可以跑
$> grails create-app foobar
Run Code Online (Sandbox Code Playgroud)
创建一个新的空 Grails 应用程序。那部分工作正常。
然后我可以 cd 进入“foobar”目录并运行
$> ./gradlew clean build
Run Code Online (Sandbox Code Playgroud)
我在使用上述 JDK 11 时遇到以下错误:
FAILURE: Build failed with an exception.
* What went wrong:
Could not determine the dependencies of task ':integrationTest'.
> Could not create task ':mergeTestReports'.
> 'org.gradle.api.file.DirectoryProperty org.gradle.api.tasks.testing.TestReport.getDestinationDirectory()'
Run Code Online (Sandbox Code Playgroud)
如果我尝试使用系统 Gradle 进行构建:
$> gradle clean build
Run Code Online (Sandbox Code Playgroud)
我得到了这个(再次使用 JDK 11):
> Configure project :
Cannot resolve …Run Code Online (Sandbox Code Playgroud)