在阅读下面的文本时,fread()无法检测第8列和第9列中的缺失值.这仅适用于默认选项integer64="integer64".设置integer64="double"或"character"正确检测NAs.请注意,文件有三种类型的V8和V9--可能来港的,,; , ,; 和NA.附加na.strings=c("NA","N/A",""," "), sep=","选项无效.
使用read.csv()方式与使用相同fread(integer="double").
要读取的文本(也可用作文件integer64_and_NA.csv):
2012,276,,0,"S1","001",1,,724135215,1590915056,
2012,276,2,8,"S1","001",1, ,,154598,0
2012,276,2,12,"S1","001",1,NA,5118863,21819477,
2012,276,2,0,"S1","011",8,3127133583,3127133583,9003982501,0
这是以下输出fread():
DT <- fread(input="integer64_and_NA.csv", verbose=TRUE, integer64="integer64", na.strings=c("NA","N/A",""," "), sep=",")
Input contains no \n. Taking this to be a filename to open
Detected eol as \r\n (CRLF) in that order, the Windows standard.
Looking for supplied sep ',' on line 4 (the last non blank line in the first 'autostart') ... found ok
Found 11 columns
First row with 11 fields occurs on line 1 (either column names or first row of data)
Some fields on line 1 are not type character (or are empty). Treating as a data row and using default column names.
Count of eol after first data row: 5
Subtracted 1 for last eol and any trailing empty lines, leaving 4 data rows
Type codes: 11114412221 (first 5 rows)
Type codes: 11114412221 (after applying colClasses and integer64)
Type codes: 11114412221 (after applying drop or select (if supplied)
Allocating 11 column slots (11 - 0 NULL)
0.000s ( 0%) Memory map (rerun may be quicker)
0.000s ( 0%) sep and header detection
0.000s ( 0%) Count rows (wc -l)
0.000s ( 0%) Column type detection (first, middle and last 5 rows)
0.000s ( 0%) Allocation of 4x11 result (xMB) in RAM
0.000s ( 0%) Reading data
0.000s ( 0%) Allocation for type bumps (if any), including gc time if triggered
0.000s ( 0%) Coercing data already read in type bumps (if any)
0.000s ( 0%) Changing na.strings to NA
0.001s Total
结果data.table是:
DT
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11
1: 2012 276 NA 0 S1 001 1 9218868437227407266 724135215 1590915056 NA
2: 2012 276 2 8 S1 001 1 9218868437227407266 9218868437227407266 154598 0
3: 2012 276 2 12 S1 001 1 9218868437227407266 5118863 21819477 NA
4: 2012 276 2 0 S1 011 8 3127133583 3127133583 9003982501 0
NA在没有的列中正确检测到值integer64.对于V8和V9,它fread()标记为整数64,而不是NA,我们有"9218868437227407266".有趣的是,str()将V8和V9的相应值返回为NA:
str(DT)
Classes ‘data.table’ and 'data.frame': 4 obs. of 11 variables:
$ V1 : int 2012 2012 2012 2012
$ V2 : int 276 276 276 276
$ V3 : int NA 2 2 2
$ V4 : int 0 8 12 0
$ V5 : chr "S1" "S1" "S1" "S1"
$ V6 : chr "001" "001" "001" "011"
$ V7 : int 1 1 1 8
$ V8 :Class 'integer64' num [1:4] NA NA NA 1.55e-314
$ V9 :Class 'integer64' num [1:4] 3.58e-315 NA 2.53e-317 1.55e-314
$ V10:Class 'integer64' num [1:4] 7.86e-315 7.64e-319 1.08e-316 4.45e-314
$ V11: int NA 0 NA 0
- attr(*, ".internal.selfref")=<externalptr>
......但没有别的人认为他们NA:
is.na(DT$V8)
[1] FALSE FALSE FALSE FALSE
max(DT$V8)
integer64
[1] 9218868437227407266
> max(DT$V8, na.rm=TRUE)
integer64
[1] 9218868437227407266
> class(DT$V8)
[1] "integer64"
> typeof(DT$V8)
[1] "double"
它似乎只是一个打印/屏幕问题,data.table将它们视为巨大的整数:
DT[, V12:=as.numeric(V8)]
Warning message:
In as.double.integer64(V8) :
integer precision lost while converting to double
> DT
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12
1: 2012 276 NA 0 S1 001 1 9218868437227407266 724135215 1590915056 NA 9.218868e+18
2: 2012 276 2 8 S1 001 1 9218868437227407266 9218868437227407266 154598 0 9.218868e+18
3: 2012 276 2 12 S1 001 1 9218868437227407266 5118863 21819477 NA 9.218868e+18
4: 2012 276 2 0 S1 011 8 3127133583 3127133583 9003982501 0 3.127134e+09
我错过了什么integer64,或者这是一个错误？如上所述,我可以绕过使用integer64="double",可能会失去一些精度,如帮助文件中所述.但意外的行为是默认的integer64......
这是在运行Revolution R 3.0.2的Windows 8.1 64位计算机上运行的,也是在运行kubuntu 13.10,CRAN-R 3.0.2的虚拟机上完成的.使用CRAN(2014年2月7日1.8.10)和1.8.11(rev.1110,2014-02-04 02:43:19)的最新稳定数据表进行测试,从拉链手动安装为r-forge在Windows上只有稳定的1.8.10,而且只有稳定的1.8.10.在两台机器上安装并加载了bit64.
> sessionInfo()
R version 3.0.2 (2013-09-25)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] grid stats graphics grDevices utils datasets methods base
other attached packages:
[1] bit64_0.9-3 bit_1.1-11 gdata_2.13.2 xts_0.9-7 zoo_1.7-10 nlme_3.1-113 hexbin_1.26.3 lattice_0.20-24 ggplot2_0.9.3.1
[10] plyr_1.8 reshape2_1.2.2 data.table_1.8.11 Revobase_7.0.0 RevoMods_7.0.0 RevoScaleR_7.0.0
loaded via a namespace (and not attached):
[1] codetools_0.2-8 colorspace_1.2-4 dichromat_2.0-0 digest_0.6.4 foreach_1.4.1 gtable_0.1.2 gtools_3.2.1 iterators_1.0.6
[9] labeling_0.2 MASS_7.3-29 munsell_0.4.2 proto_0.3-10 RColorBrewer_1.0-5 reshape_0.8.4 scales_0.2.3 stringr_0.6.2
[17] tools_3.0.2
这个bug,#488,现在是固定的这次提交的data.table在开发版本v1.9.5,和值分配的(和显示)正确的NA,如果bit64被加载.
require(data.table) # v1.9.5
require(bit64)
ans = fread("test.csv")
# V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11
# 1: 2012 276 NA 0 S1 001 1 NA 724135215 1590915056 NA
# 2: 2012 276 2 8 S1 001 1 NA NA 154598 0
# 3: 2012 276 2 12 S1 001 1 NA 5118863 21819477 NA
# 4: 2012 276 2 0 S1 011 8 3127133583 3127133583 9003982501 0
这显然是 bit64 包的问题，而不是fread()或data.table。来自bit64文档http://cran.r-project.org/web/packages/bit64/bit64.pdf
“当前不支持对不存在的元素进行下标以及使用 NA 进行下标。此类下标当前返回 9218868437227407266 而不是 NA（底层双代码的 NA 值）。在此遵循完整的 R 行为会破坏性能或需要大量 C -编码。”
我尝试将 9218868437227407266 值重新分配给 NA 认为它会起作用
前任。
DT[V8==9218868437227407266, ]
#actually returns nothing, but
DT[V8==max(V8), ]
#returns the rows with 9218868437227407266 in V8
#but this does not reassign the value
DT[V8==max(V8), V8:=NA]
#not that this makes sense, but I tried just in case...
DT[V8==max(V8), V8:=NA_character_]
因此，正如文档非常清楚地指出的那样，如果向量是整数64类，它将无法识别 NA 或缺失值。我将避免使用 bit64 只是为了不必处理这个问题......