如何使用蜂巢中的结构数组

Question

我有一个下面给出的json文件,我想通过使用jsonserde创建外部表来使用hive访问它.

{"ResponseCode":"1","响应":"找到的数据","数据":[{"季节":"RABI","扇区":"HORTICULTURE","类别":"水果","裁剪" ":"Mango","QueryType":"文化习俗","QueryText":"如何控制芒果花落？","KCCAns":"推荐用于喷洒到planofix 5-7 mili/pump","StateName" ":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName":"BALAYAPALLE","SubmitDate":"11/1/2016 9:05:27 AM"},{"Season":"RABI" ,"部门":"农业","类别":"其他","作物":"其他","查询类型":"杂草管理","查询文本":"桉树中的杂草管理","KCCAns":"建议喷洒GLYPOSATE @ 2 LITERS PER ACRE","StateName":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName":"MARRIPADU","SubmitDate":"2016/1/11 9:07: 04 AM"},{"Season":"RABI","Sector":"HORTICULTURE","Category":"Vegetables","Crop":"Bhindi(Okra/Ladysfinger)","QueryType":"\ tPlant保护\ t","QueryText":"OKRA MITE DAMAGE","KCCAns":"建议喷洒DICOFOL 1 LITER/200 LATERERS OF WATER/ACRE","StateName":"ANDHRA PRADESH","DistrictNam e":"NELLORE","BlockName":"KOVUR","SubmitDate":"11/1/2016 3:11:59 PM"},{"季节":"RABI","部门":"农业" ,"类别":"谷物","作物":"稻田(Dhan)","QueryType":"杂草管理","QueryText":"15-20天作物的稻田杂草管理","KCCAns":"建议喷洒CYHALOPOP-P-BUTYL 250 ML PER ACRE","StateName":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName":"BALAYAPALLE","SubmitDate":"2016年11月3日2 :11:17 PM"},{"季节":"KHARIF","扇区":"农业","类别":"其他","作物":"其他","查询类型":"天气"," QueryText":"天气报告","KCCAns":"天气报告联系18004253141,08912543031","StateName":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName":"KALIGIRI","SubmitDate": "11/3/2016 5:22:22 PM"},{"季节":"RABI","扇区":"农业","类别":"豆类","作物":"木豆(红豆)/arhar/tur)","QueryType":"营养素管理","查询文字":"红色食物营养管理","KCCAns":"建议每次喷洒13-045一公斤","州名":"ANDHRA邦 "" DistrictNa 我":"NELLORE","BlockName":"ANUMASAMUDRAMPETA","SubmitDate":"2016年11月3日下午7:25:10"},{"季节":"RABI","部门":"农业" ,"类别":"豆类","作物":"木豆(红克/ arhar/tur)","QueryType":"营养素管理","QueryText":"红色食物营养管理","KCCAns": "推荐喷涂13-0-45 ONE KG/ACRE","StateName":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName":"ANUMASAMUDRAMPETA","SubmitDate":"2016/11/3 7:30:02 PM"},{"Season":"RABI","Sector":"HORTICULTURE","Category":"Fruits","Crop":"Citrus","QueryType":"\ tPlant Protection\t","QueryText":"CITRUS SUCKING PEST","KCCAns":"推荐用于DIMETHOATE 400 ML/ACRE 200 LITER WATER","StateName":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName" ":"KODAVALUR","SubmitDate":"11/4/2016 8:48:03 AM"},{"Season":"RABI","Sector":"HORTICULTURE","Category":"Fruits", "Crop":"Citrus","QueryType":"\ tPlant Protection\t","QueryText":"CITRUS SUCKING PEST","KCCAns":"推荐用于DIMETHOATE 400 ML/ACRE 200 LITER WATER","StateName" :"ANDHR PRADESH","区域名称":"NELLORE","BlockName":"KODAVALUR","SubmitDate":"2016年11月4日上午8:48:04"},{"季节":"KHARIF","扇区":""农业","类别":"其他","作物":"其他","查询类型":"天气","查询文字":"天气报告","KCCAns":"淋浴时有相当多云或雷暴在今天下午","StateName":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName":"VINJAMUR","SubmitDate":"2016年11月4日3:14:03 PM"}, {"Season":"KHARIF","Sector":"AGRICULTURE","Category":"Others","Crop":"Others","QueryType":"Weather","QueryText":"WEATHER RTEPORT", "KCCAns":"今天下午有阵雨或雷阵雨","StateName":"ANDHRA PRADESH","DistrictName":"NELLORE","BlockName":"MARRIPADU","SubmitDate":"11/4/2016 5:18:50 PM"},{"季节":"KHARIF","扇区":"农业","类别":"其他","作物":"其他","查询类型":" tPlant Protection\t","QueryText":"BENGAL GRAM BEST VARIETY","KCCAns":"BENGAL GRAM BEST VARIETY:JG-11,LAM SENAGA(LBEG-7),ICCV-10","StateName":" ANDHRA PRADESH","区名":"NELLORE","BlockName":"PELLAKUR","SubmitDate":"2016年11月4日7:57:44 PM"}}}

我创建了下表来访问上面的json数据.

create EXTERNAL table kccdata1 (
Response string,
data array<struct<
Season:STRING,
Sector:STRING,
Category:STRING,
Crop:STRING,
QueryType:STRING,
QueryText:string,
KCCAns:string,
StateName:string,
DistrictName:string,
BlockName:string,
SubmitDate:string>>) 
row format serde "org.apache.hive.hcatalog.data.JsonSerDe" 
location "/user/hduser/jsondata";

此表将每个结构作为每个记录的值.我可以使用下面的查询从中访问元素.

select data[0].querytype from kccdata1;

select data.season[1] from kccdata1;

但是,如果我想找到像crop = others的所有部分,我无法使用此表.对此有任何建议都会有很大帮助.谢谢.

Answer 1

侧视图外部内联

• 内联(内置表生成函数(UDTF))
• LateralView

select  k.response
       ,d.* 

from    kccdata1 k 
        lateral view outer inline (k.data) d
;

+------------+----------+--------------+------------+---------------------------------+---------------------+------------------------------------------+-----------------------------------------------------------------------------+----------------+----------------+--------------------+----------------------+
| k.response | d.season |   d.sector   | d.category |             d.crop              |     d.querytype     |               d.querytext                |                                  d.kccans                                   |  d.statename   | d.districtname |    d.blockname     |     d.submitdate     |
+------------+----------+--------------+------------+---------------------------------+---------------------+------------------------------------------+-----------------------------------------------------------------------------+----------------+----------------+--------------------+----------------------+
| Data Found | RABI     | HORTICULTURE | Fruits     | Mango                           | Cultural Practices  | how to control flower drop in mango ?    | recommended for spray to planofix 5-7 mili/pump                             | ANDHRA PRADESH | NELLORE        | BALAYAPALLE        | 11/1/2016 9:05:27 AM |
| Data Found | RABI     | AGRICULTURE  | Others     | Others                          | Weed Management     | weed management in eucalyptus            | RECOMMENDED TO SPRAY GLYPOSATE @ 2 LITRES PER ACRE                          | ANDHRA PRADESH | NELLORE        | MARRIPADU          | 11/1/2016 9:07:04 AM |
| Data Found | RABI     | HORTICULTURE | Vegetables | Bhindi(Okra/Ladysfinger)        | Plant Protection    | OKRA MITE DAMAGE                         | RECOMMENDED TO SPRAY DICOFOL 1 LITER /200 LITRES OF WATER /ACRE             | ANDHRA PRADESH | NELLORE        | KOVUR              | 11/1/2016 3:11:59 PM |
| Data Found | RABI     | AGRICULTURE  | Cereals    | Paddy (Dhan)                    | Weed Management     | paddy weed management at 15-20 days crop | RECOMMENDED TO SPRAY CYHALOPOP-P- BUTYL 250 ML PER ACRE                     | ANDHRA PRADESH | NELLORE        | BALAYAPALLE        | 11/3/2016 2:11:17 PM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Weather             | WEATHER REPORT                           | WEATHER REPORT CONTACT 18004253141 ,08912543031                             | ANDHRA PRADESH | NELLORE        | KALIGIRI           | 11/3/2016 5:22:22 PM |
| Data Found | RABI     | AGRICULTURE  | Pulses     | Pigeon pea (red gram/arhar/tur) | Nutrient Management | RED GRAM NUTRIENT MANAGEMENT             | RECOMMENDED TO SPRAY 13-045 ONE KG PER ACRE                                 | ANDHRA PRADESH | NELLORE        | ANUMASAMUDRAMPETA  | 11/3/2016 7:25:10 PM |
| Data Found | RABI     | AGRICULTURE  | Pulses     | Pigeon pea (red gram/arhar/tur) | Nutrient Management | RED GRAM NUTRIENT MANAGEMENT             | RECOMMENDED TO SPRAY 13-0-45 ONE KG/ACRE                                    | ANDHRA PRADESH | NELLORE        | ANUMASAMUDRAMPETA  | 11/3/2016 7:30:02 PM |
| Data Found | RABI     | HORTICULTURE | Fruits     | Citrus                          | Plant Protection    | CITRUS SUCKING PEST                      | RECOMMENDED FOR DIMETHOATE 400 ML / ACRE 200 LITER WATER                    | ANDHRA PRADESH | NELLORE        | KODAVALUR          | 11/4/2016 8:48:03 AM |
| Data Found | RABI     | HORTICULTURE | Fruits     | Citrus                          | Plant Protection    | CITRUS SUCKING PEST                      | RECOMMENDED FOR DIMETHOATE 400 ML / ACRE 200 LITER WATER                    | ANDHRA PRADESH | NELLORE        | KODAVALUR          | 11/4/2016 8:48:04 AM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Weather             | weather report                           | Considerable cloudiness with a shower or thunderstorm around this afternoon | ANDHRA PRADESH | NELLORE        | VINJAMUR           | 11/4/2016 3:14:03 PM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Weather             | WEATHER RTEPORT                          | Considerable cloudiness with a shower or thunderstorm around this afternoon | ANDHRA PRADESH | NELLORE        | MARRIPADU          | 11/4/2016 5:18:50 PM |
| Data Found | KHARIF   | AGRICULTURE  | Others     | Others                          | Plant Protection    | BENGAL GRAM BEST VARIETY                 | BENGAL GRAM BEST VARIETY : J.G-11, LAM SENAGA (LBEG-7),ICCV-10              | ANDHRA PRADESH | NELLORE        | PELLAKUR           | 11/4/2016 7:57:44 PM |
+------------+----------+--------------+------------+---------------------------------+---------------------+------------------------------------------+-----------------------------------------------------------------------------+----------------+----------------+--------------------+----------------------+