假设有2个进程P1和P2,它们访问共享文件Foo.txt.
假设P2正在读取Foo.txt.Foo.txt当P2正在读取时,我不希望P1写入.
所以我认为我可以让P1写入Foo.tmp并作为最后一步重命名Foo.tmp为Foo.txt.我的编程语言是Java
所以我的问题是,这会确保P2从中读取正确的数据Foo.txt吗？P2完成读取文件后,是否会提交重命名操作？
编辑
我尝试重新创建此方案如下:
我的P1代码是这样的:
File tempFile = new File(path1);
File realFile = new File(path2);
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
for(int i=0;i<10000;i++)
writer.write("Hello World\n");
writer.flush();
writer.close();
tempFile.renameTo(realFile);
我的P2代码是:
BufferedReader br = new BufferedReader(new FileReader(file));
String line = null;
while(true) {
while((line=br.readLine())!=null){
System.out.println(line);
Thread.sleep(1000);
}
br.close();
}
我的示例共享文件:
Test Input
Test Input
Test Input
我几乎同时开始P1和P2(P2首先开始).
所以根据我的理解,即使P1编写了一个新的Foo.txt,由于P2已经在读它,它应该读取旧的Foo.txt内容,直到它重新打开BufferedReader到Foo.txt.
但实际发生的事情是P2读取Test Input三次,正如输入所预期的那样,但之后它会读取由P1写入的新内容.
P2的输出:
Test Input
Test …我正在阅读一些用Google的Go语言编写的代码,我遇到了这个:
func Statusln(a ...interface{})
func Statusf(format string, a ...interface{})
我不明白是什么...意思.有人知道吗？
我在我的Ubuntu 10.04 x86_64机器上安装了python 2.5.1.
当我尝试导入hashlib/md5时,我收到此错误
>>> import hashlib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.5/hashlib.py", line 133, in <module>
md5 = __get_builtin_constructor('md5')
File "/usr/local/lib/python2.5/hashlib.py", line 60, in __get_builtin_constructor
import _md5
ImportError: No module named _md5
我尝试过在谷歌上找到的大部分解决方案,但对我来说没什么用.有谁知道如何解决这个问题？谢谢!
我想知道OutputCollector的'实例'输出是否在map函数中使用:output.collect(key,value)this -output-在某处存储键值对？即使它发送到reducer函数,它们也必须是一个中间文件,对吧？那些文件是什么？它们是否可见并由程序员决定？我们在main函数中指定的OutputKeyClass和OutputValueClasses这些存储位置是什么？[Text.class和IntWritable.class]
我在MapReduce中给出了Word Count示例的标准代码,我们可以在网络的许多地方找到它.
public class WordCount {
public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
word.set(tokenizer.nextToken());
output.collect(word, one);
}
}
}
public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
public void …You use the hadoop fs –put command to write a 300 MB file using and HDFS block size of 64 MB. Just after this command has finished writing 200 MB of this file, what would another user see when trying to access this file?
a.) They would see Hadoop throw an ConcurrentFileAccessException when they try to access this file.
b.) They would see the current state of the file, up to the last bit written by the command.
c.) They …我写了一个MapReduce程序,代码如下:
import java.io.IOException;
import java.util.Iterator;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
public class MaxTemperature {
public class MaxTemperatureMapper extends Mapper <LongWritable, Text, Text, IntWritable>{
private static final int MISSING = 9999;
public void map(LongWritable key, Text Value, Context context, Reporter reporter) throws IOException, InterruptedException{
String line = Value.toString();
String year = line.substring(15,19);
int airTemperature;
if(line.charAt(87)=='+')
airTemperature = Integer.parseInt(line.substring(88,92));
else
airTemperature = Integer.parseInt(line.substring(87,91)); …我的示例JSON输入如下:
"JobName":"Test Job 1",
"events":[
{ "features":[],
"InputHiveTable":"uilog_uiclientlogdata",
"eventColumn":"command",
"name":"edu.apollogrp.classroom.discussion.client.events.CreateDiscussionEvent"
},
考虑该字段"InputHiveTable",它可以是全部大写INPUTHIVETABLE,全部小写inputhivetable,或两者的混合,就像现在一样.
目前,我正在阅读以下字段(在Java中):
JSONObject jsonObject = (JSONObject) obj;
JSONArray events = (JSONArray) jsonObject.get("events");
String InputHiveTable = (String)event.get("InputHiveTable");
所以我的问题是如何"InputHiveTable"在忽略案例的同时搜索字段.我正在使用JSON Simple库.
我正在关注Web应用程序中Log4j日志记录的教程.
这是我的log4j.properties档案:
#log to console
log4j.rootLogger=INFO, , demoappender
log4j.appender.demoappender=org.apache.log4j.ConsoleAppender
log4j.appender.demoappender=System.out
log4j.appender.demoappender.layout=org.apache.log4j.PatternLayout
log4j.appender.demoappender.layout.ConversionPattern=%d{ABSOLUTE} %5p %c:%L - %m%n
我使用init()方法实例化记录器:
private Logger logger = null;
public void init(ServletConfig config) throws ServletException {
logger = Logger.getRootLogger();
}
这是我尝试记录一些示例消息的地方:
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
logger.info("logging INFO message");
logger.error("logging ERROR message");
logger.warn("loggging WARNING message");
logger.fatal("logging FATAL message");
response.setContentType("text/html");
当我在Tomcat上运行servlet时,我在控制台上收到此错误:
log4j:ERROR Could not find value for key log4j.appender.
log4j:ERROR Could not instantiate appender named "".
log4j:ERROR Could not instantiate class …我正在运行位于http://www.norstad.org/matrix-multiply/index.html的MapReduce矩阵乘法程序.我发现当输入矩阵中有0时,此实现无法正常工作.但是我不明白为什么,以及如何修改程序以使其工作？MapReduce操作完成,但输出始终是包含所有元素0的矩阵.
我输入的矩阵A和B是:
Matrix A Matrix B
0 0 0 6 7 4
0 1 6 9 1 3
7 8 9 7 6 2
输出矩阵:
0 0 0
0 0 0
0 0 0
以下内容取自作业的日志文件:
matrixB的地图输出:
##### Map setup: matrixA = false for hdfs://localhost/user/hadoop-user/B
strategy = 4
R1 = 4
I = 3
K = 3
J = 3
IB = 3
KB = 3
JB = 3
##### Map input: (0,0) 6
##### Map output: (0,0,0,1) (0,0,6) …我尝试使用java程序将文件写入我的本地HDFS设置,我正在使用Hadoop 2.3.0分发和hadoop-client 2.3.0 hadoop-hdfs 2.3.0库.
在HDFS日志中,它显示以下错误:
2014-04-07 18:40:44,479 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: prabhathp:50010:DataXceiver error processing unknown operation src: /127.0.0.1:38572 dest: /127.0.0.1:50010
java.io.IOException: Version Mismatch (Expected: 28, Received: 26738 )
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:54)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:198)
at java.lang.Thread.run(Thread.java:744)
有人可以解释一下吗？