小编per*_*lmq的帖子

在ETL过程中,我需要从一个Postgres数据库中提取并加载JSON列到另一个.我们使用Pandas,因为它有很多方法可以从不同的源/目的地读取和写入数据,所有转换都可以使用Python和Pandas编写.我们对诚实的做法感到非常满意..但我们遇到了问题.

通常,读取和写入数据非常容易.您只需使用pandas.read_sql_table从源和pandas.to_sql读取数据以将其写入目标.但是,由于其中一个源表有一个JSON类型的列(来自Postgres),该to_sql函数崩溃时出现以下错误消息.

    df.to_sql(table_name, analytics_db)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/pandas/core/generic.py", line 1201, in to_sql
    chunksize=chunksize, dtype=dtype)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/pandas/io/sql.py", line 470, in to_sql
    chunksize=chunksize, dtype=dtype)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/pandas/io/sql.py", line 1147, in to_sql
    table.insert(chunksize)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/pandas/io/sql.py", line 663, in insert
    self._execute_insert(conn, keys, chunk_iter)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/pandas/io/sql.py", line 638, in _execute_insert
    conn.execute(self.insert_statement(), data)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/ec2-user/python-virtual-environments/etl/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params …

我真的很喜欢mosh(http://mosh.mit.edu/)但是在brew upgrade它停止工作之后,产生了以下错误.

> mosh example.com
dyld: Library not loaded: /usr/local/lib/libprotobuf.7.dylib
  Referenced from: /usr/local/bin/mosh-client
  Reason: image not found
Died at /usr/local/bin/mosh line 201.