python netcdf:制作所有变量和属性的副本,但只有一个

Question

我需要在netcdf文件中处理一个实际包含许多属性和变量的变量.我认为无法更新netcdf文件(请参阅如何删除Scientific.IO.NetCDF.NetCDFFile中的变量的问题？)

我的方法如下:

1. 获取要从原始文件处理的变量
2. 处理变量
3. 从原始netcdf复制所有数据但将处理过的变量复制到最终文件
4. 将处理过的变量复制到最终文件中

我的问题是编码第3步.我开始使用以下内容:

def  processing(infile, variable, outfile):
        data = fileH.variables[variable][:]

        # do processing on data...

        # and now save the result
        fileH = NetCDFFile(infile, mode="r")
        outfile = NetCDFFile(outfile, mode='w')
        # build a list of variables without the processed variable
        listOfVariables = list( itertools.ifilter( lamdba x:x!=variable , fileH.variables.keys() ) )
        for ivar in listOfVariables:
             # here I need to write each variable and each attribute

如何在不需要重建整个数据结构的情况下将所有数据和属性保存在一小撮代码中？

Answer 1

这就是我刚刚使用和工作的内容。@arne的答案已针对Python 3更新，并包括复制变量属性：

import netCDF4 as nc
toexclude = ['ExcludeVar1', 'ExcludeVar2']

with netCDF4.Dataset("in.nc") as src, netCDF4.Dataset("out.nc", "w") as dst:
    # copy global attributes all at once via dictionary
    dst.setncatts(src.__dict__)
    # copy dimensions
    for name, dimension in src.dimensions.items():
        dst.createDimension(
            name, (len(dimension) if not dimension.isunlimited() else None))
    # copy all file data except for the excluded
    for name, variable in src.variables.items():
        if name not in toexclude:
            x = dst.createVariable(name, variable.datatype, variable.dimensions)
            dst[name][:] = src[name][:]
            # copy variable attributes all at once via dictionary
            dst[name].setncatts(src[name].__dict__)

Answer 2

如果您只想复制挑选变量的文件,nccopy则是@rewfuss提交的一个很棒的工具.

这是一个Pythonic(和更灵活)的解决方案python-netcdf4.这允许您在写入文件之前将其打开以进行处理和其他计算.

with netCDF4.Dataset(file1) as src, netCDF4.Dataset(file2) as dst:

  for name, dimension in src.dimensions.iteritems():
    dst.createDimension(name, len(dimension) if not dimension.isunlimited() else None)

  for name, variable in src.variables.iteritems():

    # take out the variable you don't want
    if name == 'some_variable': 
      continue

    x = dst.createVariable(name, variable.datatype, variable.dimensions)
    dst.variables[x][:] = src.variables[x][:]

这没有考虑变量属性,例如fill_values.您可以轻松地按照文档进行操作.

要小心,netCDF4文件一旦写入/创建这种方式就无法撤消.修改变量的那一刻,它会在with语句结束时写入文件,或者如果你调用.close()了Dataset.

当然,如果您希望在编写变量之前处理变量,则必须注意要创建的维度.在新文件中,从不写入变量而不创建变量.而且,如果不具有限定尺寸从未创建变量,如所指出python-netcdf4的文档.

Answer 3

这个答案建立在 Xavier Ho ( /sf/answers/2240168101/ ) 的答案之上，但是我需要完成它的修复：

import netCDF4 as nc
import numpy as np
toexclude = ["TO_REMOVE"]
with nc.Dataset("orig.nc") as src, nc.Dataset("filtered.nc", "w") as dst:
    # copy attributes
    for name in src.ncattrs():
        dst.setncattr(name, src.getncattr(name))
    # copy dimensions
    for name, dimension in src.dimensions.iteritems():
        dst.createDimension(
            name, (len(dimension) if not dimension.isunlimited else None))
    # copy all file data except for the excluded
    for name, variable in src.variables.iteritems():
        if name not in toexclude:
            x = dst.createVariable(name, variable.datatype, variable.dimensions)
            dst.variables[name][:] = src.variables[name][:]