tl;dr:为了能够在列出的路径中使用通配符(globs),只需使用globStatus(...)而不是listStatus(...).
我的HDFS集群上的文件按分区进行组织,日期为"根"分区.文件结构的简化示例如下所示:
/schemas_folder
??? date=20140101
? ??? A-schema.avsc
? ??? B-schema.avsc
??? date=20140102
? ??? A-schema.avsc
? ??? B-schema.avsc
? ??? C-schema.avsc
??? date=20140103
??? B-schema.avsc
??? C-schema.avsc
就我而言,该目录在不同日期存储不同类型数据(本例中为A,B和C)的Avro模式.随着时间的推移,模式可能会开始存在,发展并停止存在.
我需要能够尽快获得给定类型的所有模式.在我希望得到类型A存在的所有模式的示例中,我想执行以下操作:
hdfs dfs -ls /schemas_folder/date=*/A-schema.avsc
那会给我
Found 1 items
-rw-r--r-- 3 user group 1234 2014-01-01 12:34 /schemas_folder/date=20140101/A-schema.avsc
Found 1 items
-rw-r--r-- 3 user group 2345 2014-01-02 23:45 /schemas_folder/date=20140102/A-schema.avsc
我不想使用shell命令,似乎无法在Java API中找到与上面的命令等效的东西.当我尝试自己实现循环时,我的表现非常糟糕.我至少想要命令行的性能(在我的情况下大约3秒)......
人们可以注意到它 …
Let's say that I have a simple data class in Java:
public class Person {
private final String name;
private final int age;
Person(String name, int age) {
this.name = name;
this.age = age;
}
public String getName() {
return name;
}
int String getAge() {
return age;
}
}
Note: in practice, I use Immutables to generate this, but am showing a POJO here for the sake of simplicity.
To document the model of the GET response, even …
我一直在为我公司的不同服务创建多个小型Java RESTful客户端库.大多数时候,我无法在服务器端更改任何内容,我需要编写Jersey代码片段以与现有RESTful API进行交互.
据我所知,我一直在使用Jersey和Jackson来使用JSON:当我查询POJO时,我从JSON反序列化它,当我需要发送POJO时,我将它序列化为JSON体.到目前为止,这两种片段一直在为我做这个工作......
ClientResponse response = webResource
.path("/path/to/resource")
.queryParam("key", "value")
.accept(Mediatype.APPLICATION_JSON)
.get(ClientResponse.class);
// (...) Check response status code
MyClassPojo pojo = response.getEntity(MyClassPojo.class);
ClientResponse response = webResource
.path("/path/to/resource")
.type(Mediatype.APPLICATION_JSON_TYPE)
.accept(Mediatype.APPLICATION_JSON)
.post(ClientResponse.class, pojo)
// (...) Check response status code
我现在面临一个RESTful服务器,它不接受JSON主体发送我的POJO.似乎唯一有用的是使用查询参数.
例如,如果我想发送对象
public MyClassPojo {
public int attr1;
public String attr2;
}
MyClassPojo pojo = new MyClassPojo();
pojo.attr1 = 42;
pojo.attr2 = "Foo bar";
我本来喜欢用JSON序列化它:
{
"attr1": 42,
"attr2": "Foo bar"
}
但是这个特定的RESTful服务器期待查询参数:
?attr1=42&attr2=Foo+bar …我想尝试一个简单的程序,但得到以下错误.我正在使用Eclipse Kelper和Tomcat 7.0.尝试过很多东西,但没有什么对我有用.
HTTP Status 500 - Servlet.init() for servlet appServlet threw exception
type Exception report
message Servlet.init() for servlet appServlet threw exception
description The server encountered an internal error that prevented it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet.init() for servlet appServlet threw exception
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)
root cause
java.lang.IllegalArgumentException
org.springframework.asm.ClassReader.<init>(Unknown Source)
org.springframework.asm.ClassReader.<init>(Unknown Source)
org.springframework.asm.ClassReader.<init>(Unknown Source)
org.springframework.core.type.classreading.SimpleMetadataReader.<init>(SimpleMetadataReader.java:52)
org.springframework.core.type.classreading.SimpleMetadataReaderFactory.getMetadataReader(SimpleMetadataReaderFactory.java:80)
org.springframework.core.type.classreading.CachingMetadataReaderFactory.getMetadataReader(CachingMetadataReaderFactory.java:102)
org.springframework.core.type.classreading.SimpleMetadataReaderFactory.getMetadataReader(SimpleMetadataReaderFactory.java:76)
org.springframework.context.annotation.ConfigurationClassParser.getImports(ConfigurationClassParser.java:298)
org.springframework.context.annotation.ConfigurationClassParser.getImports(ConfigurationClassParser.java:300)
org.springframework.context.annotation.ConfigurationClassParser.getImports(ConfigurationClassParser.java:300)
org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:230)
org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:153)
org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:130)
org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:285)
org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:223)
org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:630) …