我有一个hive/hbase集成表,定义如下.
create table user_c(user_id int, c_name string, c_kind string, c_industry string,
c_jobtitle string, c_workyear int, c_title string, c_company string)
stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:c_name,cf1:c_kind,cf1:c_industry,cf1:c_jobtitle,cf1:c_workyear,cf1:c_title,cf1:c_company")
TBLPROPERTIES ("hbase.table.name" = "user_c");
在我的java代码中,我创建了一个Put并用db读取的值填充它.代码如下所示:
final Put to = new Put(getByte(from, keyColumn));
for (final IColumn column : table.getColumns()) {
if (column.equals(keyColumn)) continue;
to.add(Bytes.toBytes(column.getColumnFamily()), Bytes.toBytes(column.getDestName()), getByte(from, column));
}
return to;
这getByte是一种将值转换为的方法byte[].看起来像
byte[] getByte(final Map<String, Object> map, IColumn column) {
final Object val = map.get(column.getName());
if (val instanceof Integer) …我配置Hive使用mysql Metastore.它工作正常.
但是当我试图放下一张桌子时,我会收到如下错误.
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.
NestedThrowables:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
我做了一些研究,谷歌和官方faq告诉我,更改mysql字符集设置将解决这个问题.
我的mysql设置如下
mysql> show variables like '%char%';
+--------------------------+----------------------------------------+
| Variable_name | Value |
+--------------------------+----------------------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | …