如何使用IDE在Storm生产集群中提交拓扑

Question

我在Must submit topologies using the 'storm' client script so that StormSubmitter knows which jar to upload使用IDE向生产集群提交拓扑时遇到了问题,而如果我使用storm jar命令在命令行中执行,它就像天堂一样运行.我从githublink看过相同的例子.

对于提交拓扑,我使用这些行

conf.put(Config.NIMBUS_HOST, NIMBUS_NODE);
conf.put(Config.NIMBUS_THRIFT_PORT,6627);
conf.put(Config.STORM_ZOOKEEPER_PORT,2181);
conf.put(Config.STORM_ZOOKEEPER_SERVERS,ZOOKEEPER_ID);
conf.setNumWorkers(20);
conf.setMaxSpoutPending(5000);
StormSubmitter submitter = new StormSubmitter();
submitter.submitTopology("test", conf, builder.createTopology());

如果这是正确的运行方法,请建议我？

Answer 1

很好找到解决方案.当我们运行"暴风罐"时,它会在提交的jar中触发storm.jar的属性标志.因此,如果我们想以编程方式提交jar,那么只需这样设置标志即可

System.setProperty("storm.jar", <path-to-jar>);

例如:

System.setProperty("storm.jar", "/Users/programming/apache-storm-1.0.1/lib/storm-core-1.0.1.jar");
StormSubmitter.submitTopology("myTopology", config, builder.createTopology());

Answer 2

要将拓扑提交到远程Storm集群,您需要将该jar上传到nimbus机器,然后使用NimbusClient将该jar提交到Cluster.
你可以这样做:

Map storm_conf = Utils.readStormConfig();
storm_conf.put("nimbus.host", "<Nimbus Machine IP>");
Client client = NimbusClient.getConfiguredClient(storm_conf)
                                .getClient();
String inputJar = "C:\\workspace\\TestStormRunner\\target\\TestStormRunner-0.0.1-SNAPSHOT-jar-with-dependencies.jar";
NimbusClient nimbus = new NimbusClient(storm_conf, "<Nimbus Machine IP>",
                                <Nimbus Machine Port>);
 // upload topology jar to Cluster using StormSubmitter
String uploadedJarLocation = StormSubmitter.submitJar(storm_conf,
                                inputJar);

String jsonConf = JSONValue.toJSONString(storm_conf);
nimbus.getClient().submitTopology("testtopology",
                      <uploadedJarLocation>, jsonConf, builder.createTopology());

以下是工作示例:将拓扑提交到Remote Storm Cluster