我已经配置了flume来读取日志文件并写入HDFS.当我启动水槽时,会读取日志文件,但不会写入HDFS.flume.log有警告信息 - could not configure sink - no channel configured for sink但我已经在conf文件中分配了一个信道.
下面给出了conf-file和错误消息:
File: spool-to-hdfs.properties
# List all components.
agent1.sources = source1
agent1.sinks = sink1
agent1.channels = channel1
# Describe source.
agent1.sources.source1.type = spooldir
agent1.sources.source1.spoolDir =/Suriya/flume/input_files
# Describe channel
#agent1.channels.channel1.type = file
#agent1.channels.channel1.checkpointDir = /Suriya/flume/checkpointDir
#agent1.channels.channel1.dataDirs =/Suriya/flume/dataDirs
agent1.channels.channel1.type = memory
# Describe sink
agent1.sinks.sink1.type = hdfs
#agent1.sinks.sink1.hdfs.path = hdfs://sandbox.hortonworks.com:8020/hdfs/Suriya/flume
agent1.sinks.sink1.hdfs.path = hdfs://localhost/hdfs/Suriya/flume
agent1.sinks.sink1.hdfs.fileType= DataStream
agent1.sinks.sink1.hdfs.writeFormat = Text
# Bind source and sink to channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channels = channel1
**-- starting the agent**
flume-ng agent --conf-file spool-to-hdfs.properties --conf /etc/flume/conf --name agent1;
Run Code Online (Sandbox Code Playgroud)
flume.log
03 Aug 2015 23:37:16,699 WARN [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks:697) - Could not configure sink sink1 due to: No channel configured for sink: sink1
org.apache.flume.conf.ConfigurationException: No channel configured for sink: sink1 at org.apache.flume.conf.sink.SinkConfiguration.configure(SinkConfiguration.java:51)
Run Code Online (Sandbox Code Playgroud)
Replace the bind part with.
# Bind source and sink to channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channels = channel1
This bind config
# Bind source and sink to channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channel = channel1
agent1.sources.source1.channels = channel1
Looks okay
BUT
agent1.sinks.sink1.channels = channel1
Should be
agent1.sinks.sink1.channel = channel1
Run Code Online (Sandbox Code Playgroud)
让我们知道怎么回事.
| 归档时间: |
|
| 查看次数: |
3170 次 |
| 最近记录: |