使用python pandas使用新数据框附加现有excel表

Question

我目前有这个代码.它完美地运作.

它循环遍历文件夹中的excel文件,删除前两行,然后将它们保存为单独的excel文件,并将文件作为附加文件保存在循环中.

目前,每次运行代码时附加的文件都会覆盖现有文件.

我需要将新数据附加到已经存在的Excel工作表的底部('master_data.xlsx)

dfList = []
path = 'C:\\Test\\TestRawFile' 
newpath = 'C:\\Path\\To\\New\\Folder'

for fn in os.listdir(path): 
  # Absolute file path
  file = os.path.join(path, fn)
  if os.path.isfile(file): 
    # Import the excel file and call it xlsx_file 
    xlsx_file = pd.ExcelFile(file) 
    # View the excel files sheet names 
    xlsx_file.sheet_names 
    # Load the xlsx files Data sheet as a dataframe 
    df = xlsx_file.parse('Sheet1',header= None) 
    df_NoHeader = df[2:] 
    data = df_NoHeader 
    # Save individual dataframe
    data.to_excel(os.path.join(newpath, fn))

    dfList.append(data) 

appended_data = pd.concat(dfList)
appended_data.to_excel(os.path.join(newpath, 'master_data.xlsx'))

我认为这将是一项简单的任务,但我猜不是.我想我需要将master_data.xlsx文件作为数据帧引入,然后将索引与新的附加数据匹配,并将其保存回来.或者也许有一种更简单的方法.任何帮助表示赞赏.

Answer 1

用于将DataFrame附加到现有 Excel文件的辅助函数:

def append_df_to_excel(filename, df, sheet_name='Sheet1', startrow=None,
                       truncate_sheet=False, 
                       **to_excel_kwargs):
    """
    Append a DataFrame [df] to existing Excel file [filename]
    into [sheet_name] Sheet.
    If [filename] doesn't exist, then this function will create it.

    Parameters:
      filename : File path or existing ExcelWriter
                 (Example: '/path/to/file.xlsx')
      df : dataframe to save to workbook
      sheet_name : Name of sheet which will contain DataFrame.
                   (default: 'Sheet1')
      startrow : upper left cell row to dump data frame.
                 Per default (startrow=None) calculate the last row
                 in the existing DF and write to the next row...
      truncate_sheet : truncate (remove and recreate) [sheet_name]
                       before writing DataFrame to Excel file
      to_excel_kwargs : arguments which will be passed to `DataFrame.to_excel()`
                        [can be dictionary]

    Returns: None
    """
    from openpyxl import load_workbook

    import pandas as pd

    # ignore [engine] parameter if it was passed
    if 'engine' in to_excel_kwargs:
        to_excel_kwargs.pop('engine')

    writer = pd.ExcelWriter(filename, engine='openpyxl')

    # Python 2.x: define [FileNotFoundError] exception if it doesn't exist 
    try:
        FileNotFoundError
    except NameError:
        FileNotFoundError = IOError


    try:
        # try to open an existing workbook
        writer.book = load_workbook(filename)

        # get the last row in the existing Excel sheet
        # if it was not specified explicitly
        if startrow is None and sheet_name in writer.book.sheetnames:
            startrow = writer.book[sheet_name].max_row

        # truncate sheet
        if truncate_sheet and sheet_name in writer.book.sheetnames:
            # index of [sheet_name] sheet
            idx = writer.book.sheetnames.index(sheet_name)
            # remove [sheet_name]
            writer.book.remove(writer.book.worksheets[idx])
            # create an empty sheet [sheet_name] using old index
            writer.book.create_sheet(sheet_name, idx)

        # copy existing sheets
        writer.sheets = {ws.title:ws for ws in writer.book.worksheets}
    except FileNotFoundError:
        # file does not exist yet, we will create it
        pass

    if startrow is None:
        startrow = 0

    # write out the new sheet
    df.to_excel(writer, sheet_name, startrow=startrow, **to_excel_kwargs)

    # save the workbook
    writer.save()

用法示例......

旧答案:它允许您将多个 DataFrame 写入新的Excel文件.

您可以将openpyxl引擎与startrow参数结合使用:

In [48]: writer = pd.ExcelWriter('c:/temp/test.xlsx', engine='openpyxl')

In [49]: df.to_excel(writer, index=False)

In [50]: df.to_excel(writer, startrow=len(df)+2, index=False)

In [51]: writer.save()

C:/temp/test.xlsx:

PS你可能还想指定header=None你是否不想复制列名...

更新:您可能还想检查此解决方案

Answer 2

如果您不是严格寻找 excel 文件，则将输出作为 csv 文件，然后将 csv 复制到新的 excel 文件

df.to_csv('filepath', mode='a', index = False, header=None)

模式 = 'a'

一种手段追加

这是一种迂回的方式，但工作得很好！

Answer 3

基于 MaxU 和其他人的代码和注释，但简化为仅修复 pandas ExcelWriter 的错误，该错误导致 to_excel 创建新工作表而不是在附加模式下附加到现有工作表。

正如其他人所指出的，to_excel 使用 ExcelWriter.sheets 属性，并且由 ExcelWriter 时不会填充该属性。

修复是单行代码，否则代码是标准 pandas 方法，如 to_excel 中记录的那样。

    # xl_path is destination xlsx spreadsheet
    with pd.ExcelWriter(xl_path, 'openpyxl', mode='a') as writer:
        # fix line
        writer.sheets = dict((ws.title, ws) for ws in writer.book.worksheets)
        df.to_excel(writer, sheet_name)

Answer 4

这对我有用

import os
import openpyxl
import pandas as pd
from openpyxl.utils.dataframe import dataframe_to_rows

file = r"myfile.xlsx"

df = pd.DataFrame({'A': 1, 'B': 2})

# create excel file
if os.path.isfile(file):  # if file already exists append to existing file
    workbook = openpyxl.load_workbook(file)  # load workbook if already exists
    sheet = workbook['my_sheet_name']  # declare the active sheet 

    # append the dataframe results to the current excel file
    for row in dataframe_to_rows(df, header = False, index = False):
        sheet.append(row)
    workbook.save(file)  # save workbook
    workbook.close()  # close workbook
else:  # create the excel file if doesn't already exist
    with pd.ExcelWriter(path = file, engine = 'openpyxl') as writer:
        df.to_excel(writer, index = False, sheet_name = 'my_sheet_name')