那些遇到过的问题

Spring Batch:忽略与指定模式不匹配的任何行

one*_*rld 4 java annotations spring-batch

我有一个要求,我在那里读取具有不同类型输入的文件,如下所示:

*JAMBEG,APP=000007,123456
AC,654321,“ABCD12121212121212”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC,TREX,000000002
AC,654321,“ABCD12121212121213”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC, TREX,000000002
...
AC,654321,“ABCD12121212121214”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC, TREX,000000002
*JAMEND,APP=000007,123456
EOF
Run Code Online (Sandbox Code Playgroud)

我只需要Header行和后面的记录,忽略以TREX开头的行,*JAMEND,EOF.

以下是我的行映射器:

public LineMapper<Customer> lineMapper(){

    DelimitedLineTokenizer lineTokenizerHeader = new DelimitedLineTokenizer();
    lineTokenizerHeader.setNames(new String[]{"association","companyNumber","fileDate"});
    lineTokenizerHeader.setIncludedFields(new int[]{0,1,2});
    lineTokenizerHeader.setStrict(false);

    DelimitedLineTokenizer lineTokenizerBody = new DelimitedLineTokenizer();
    lineTokenizerBody.setNames(new String[]{"type","acNumber","orderNumber"});
    lineTokenizerBody.setIncludedFields(new int[]{0,1,2});
    lineTokenizerBody.setStrict(false);


    HashMap<String, DelimitedLineTokenizer> tokenizers = new HashMap<String, DelimitedLineTokenizer>();
    tokenizers.put("*BEG*", lineTokenizerHeader);
    tokenizers.put("AC*", lineTokenizerBody);

    BeanWrapperFieldSetMapper<Customer> beanWrapperFieldSetMapper = new BeanWrapperFieldSetMapper<Customer>();
    beanWrapperFieldSetMapper.setTargetType(Customer.class);
    beanWrapperFieldSetMapper.setStrict(false);

    HashMap<String, BeanWrapperFieldSetMapper<Customer>> fieldSetMappers = new HashMap<String, BeanWrapperFieldSetMapper<Customer>>();
    fieldSetMappers.put("*BEG*", beanWrapperFieldSetMapper);
    fieldSetMappers.put("AC*", beanWrapperFieldSetMapper);

    PatternMatchingCompositeLineMapper patternMatchingCompositeLineMapper = new PatternMatchingCompositeLineMapper();
    patternMatchingCompositeLineMapper.setTokenizers(tokenizers);
    patternMatchingCompositeLineMapper.setFieldSetMappers(fieldSetMappers);

    return patternMatchingCompositeLineMapper;
}
Run Code Online (Sandbox Code Playgroud)

这是我明显的错误,我没有TREX,*JAMEND,EOF模式的任何映射.因此它抛出以下异常:

2014年6月16日16:49:34746 [主要] DEBUG org.springframework.batch.core.step.item.FaultTolerantChunkProvider - 在线路解析错误:5在资源= [类路径资源[0000123456.csv],输入= [EOF]:org.springframework.batch.item.file.FlatFileParseException 2014年6月16日16:49:34746 [主要] DEBUG org.springframework.batch.core.step.item.FaultTolerantChunkProvider - 跳过失败输入org.springframework. batch.item.file.FlatFileParseException:分析在线路错误:在资源5 = [类路径资源[0000123456.csv],输入= [EOF]在org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader. java:183)org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:83)org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91)at org org.springframewor上的.springframework.batch.core.step.item.FaultTolerantChunkProvider.read(FaultTolerantChunkProvider.java:87)位于org.springframework.batch.repeat的org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)的k.batch.core.step.item.SimpleChunkProvider $ 1.doInIteration(SimpleChunkProvider.java:114) .support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)org.springframework.batch.core.step.item.SimpleChunkProvider.provide (SimpleChunkProvider.java:108)org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69)at org.springframework.batch.core.step.tasklet.TaskletStep $ ChunkTransactionCallback.doInTransaction(TaskletStep的.java:402)在org.springframework.batch.core.step.tasklet.TaskletStep $ ChunkTransactionCallback.doInTransaction(TaskletStep.java:326)在org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)在org.springframework.batch.core.step.tasklet.TaskletStep $ 2.doInChunkC attext.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:)上的文本(TaskletStep.java:267): 368)在org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)在org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)在org.springframework.batch .core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)在org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:198)在org.springframework.batch.core.job.SimpleStepHandler .handleStep(SimpleStepHandler.java:148)在org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:386)在org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135 )org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:304)at org.springframework.batch.core.launch.support .simpleJobLauncher $ 1.run(SimpleJobLauncher.java:135)org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher. java:128)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)at java.lang atref.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)中的.reflect.Method.invoke(Method.java:597)org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) )org.springframe上的Org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration $ PassthruAdvice.invoke(SimpleBatchConfiguration.java:117)com.un.proxy上的org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)中的work.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172).$ Proxy17.run(未知来源)在com.chofac.pm.batch.CustomerFileToDBJobTest.testLaunchJob(CustomerFileToDBJobTest.java:48)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)at sun.reflect .在org.junit.runners.model.FrameworkMethod $ 1.runReflectiveCall(FrameworkMethod.java:45)的orggatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)处于java.lang.refun.Modeod.invoke(Method.java:597) .junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)在org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)在org.junit.internal.runners.statements.InvokeMethod .evaluate(InvokeMethod.java:20)org.junit.internal.runners.statements.RunBe fores.evaluate(RunBefores.java:28)org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate( RunAfterTestMethodCallbacks.java:83)org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)org.junit.runners.ParentRunner $ 3.run(ParentRunner.java:231)at org.junit.runners.ParentRunner $ 1.schedule(ParentRunner.java) :60)在org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)在org.junit.runners.ParentRunner.access $ 000(ParentRunner.java:50)在org.junit.runners.ParentRunner $ 2.evaluate( ParentRunner.java:222)org.springframework.test.context.junit4.statements.RunBeforeTestClassCallback s评估(RunBeforeTestClassCallbacks.java:61)org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)org.junit.runners.ParentRunner.run(ParentRunner.java:300)在Org.eclipse.jdt的org.eclipse.jandt.Retner.run上的org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)的org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174) .internal.junit.runner.TestExecution.run(TestExecution.java:38)在org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)在org.eclipse.jdt.internal.junit .runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)位于org.eclipse.jdt.internal.junit.runner.RemoteTestRunner的org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) .主要(RemoteTestRunner.java:197)引起的:java.lang.IllegalStateException:找不到在org.springframework.b关键= [EOF]匹配的图案 atch.suternMatcher.match(PatternMatcher.java:226)位于org.springframework.batch.item.file.FlatFileItemReader的org.springframework.batch.item.file.mapping.PatternMatchingCompositeLineMapper.mapLine(PatternMatchingCompositeLineMapper.java:62). doRead(FlatFileItemReader.java:180)... 67更多

我看了很多例子,这个例子匹配得很近,并改变了我的步骤如下,但仍然是同样的问题.

@Bean
public Step step(){
    return stepBuilders.get("step")
    .<Customer,Customer>chunk(1)
    .reader(CustomerAUFileReader())
    .faultTolerant()
    .skipLimit(3)
    .skip(Exception.class)
    .processor(CustomerRecordProcessor())
    .writer(CustomerDBWriter())
    .listener(logProcessListener())
    .build();
}
Run Code Online (Sandbox Code Playgroud)

看一下这里的Spring.io文档跳过记录(5.1.5配置跳过逻辑),也不起作用.

请让我知道解决此问题的理想方法.是否有一种简单的方法来指定与特定情况不匹配的跳过记录？请指教.谢谢.

---

我有一个'*'模式映射器,我用虚拟类映射.我在进程阶段返回null但它抛出nullpointerexception.

堆栈跟踪:

2014-06-17 10:03:01,690 [main] DEBUG org.springframework.batch.core.step.item.FaultTolerantChunkProcessor - Skipping after failed process
org.springframework.batch.core.listener.StepListenerFailedException: Error in afterProcess.
    at org.springframework.batch.core.listener.MulticasterBatchListener.afterProcess(MulticasterBatchListener.java:136)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.doProcess(SimpleChunkProcessor.java:127)
    at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$1.doWithRetry(FaultTolerantChunkProcessor.java:225)
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:263)
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:193)
    at org.springframework.batch.core.step.item.BatchRetryTemplate.execute(BatchRetryTemplate.java:217)
    at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor.transform(FaultTolerantChunkProcessor.java:290)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:192)
    at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:402)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:326)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:267)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)
    at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
    at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
    at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
    at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:198)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:386)
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:304)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:117)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at com.sun.proxy.$Proxy17.run(Unknown Source)
    at com.chofac.pl.batch.CustomerFileToDBJobTest.testLaunchJob(CustomerFileToDBJobTest.java:48)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
    at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
    at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
    at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.NullPointerException
    at com.chofac.pl.batch.CustomerItemProcessListener.afterProcess(CustomerItemProcessListener.java:13)
    at org.springframework.batch.core.listener.CompositeItemProcessListener.afterProcess(CompositeItemProcessListener.java:60)
    at org.springframework.batch.core.listener.MulticasterBatchListener.afterProcess(MulticasterBatchListener.java:133)
Run Code Online (Sandbox Code Playgroud)

Adr*_*hum 5

一种方法:

您ItemReader只需阅读一行并按原样返回.因此,读者给出的项目将是一个简单的字符串.

写一个简单的ItemProcessor,主要是你的工作LineMapper,基于一个模式,例如:如果项目与模式匹配,然后将输入字符串转换为您的Customer返回.如果模式不匹配,只需返回null并跳过该项.

项目处理器的伪代码:

class CustomPatternMatchingItemProcessor<String, Customer> 
        implements ItemProcessor<String, Customer> {
    private String pattern;

    public Customer process(String s) {
        if (s matches pattern) {
            construct Customer object base on s
            return customer
        } else {
            return null;
        }
    }
}
Run Code Online (Sandbox Code Playgroud)

甚至更干净:让一个处理器完成从String到Customer的映射工作,另一个处理器在regex上进行字符串基础验证.

只需使用a链接您的处理器CompositeItemProcessor.这样可以更好地分离每个处理器的关注点.

归档时间：

查看次数：

12456 次

最近记录：

8 年，6 月 前
相关归档
Hashset与Treeset 482
根据键名从HashMap获取字符串值 120
如何在RESTful POST方法中访问参数 118
如何在Android应用程序中激活"分享"按钮？ 103
多个主要功能 68
哪里是我的无效角色(ORA-00911) 65
无需使用弹簧靴即可创建弹簧休息服务 65
我可以使用Java EE webapp中的Windows身份验证连接到SQL Server吗？ 53
神秘的Eclipse JSP验证错误 49
尝试通过使用TFS查看项目中的给定文件名及其行号来编写代码 4
难疑归档
为什么HTML认为"chucknorris"是一种颜色？ 7264
你如何存放未跟踪的文件？ 1287
Subversion存储库中"分支","标记"和"主干"的含义是什么？ 1181
我是否提交了由npm 5创建的package-lock.json文件？ 1164
迭代集合,在循环中删除对象时避免使用ConcurrentModificationException 1158
Vim最有效的捷径是什么？ 1127
如何以CSV格式输出MySQL查询结果？ 1118
如何配置Visual Studio代码以始终在新选项卡中打开文件？ 1078
获取Oracle中所有表的列表？ 1073
自我的目的是什么？ 1061