标签: spring-batch-admin

我有后端Spring经验，但是我没有在Web容器（例如Tomcat）上部署应用程序的丰富经验。我能够使用Eclipse STS工具运行Spring Batch Admin。然而，随着后这个我莫名其妙地被混淆关于如何从Spring Batch的管理用户界面创建自己的春天批处理应用程序，并在Tomcat上部署它们，并进行管理。

在IntelliJ上创建的Spring Batch项目中，我根本找不到任何war文件。我也阅读了上面的文档，它使您了解到这一点，以便您可以下载war文件。但是所有文件都是jar文件。我有点困惑，关于如何创建Spring Batch作业以及如何在Spring Batch Admin上部署它们的逐步指南会很棒。我很感激。

我正在使用Spring启动和Spring批处理.我已定义了多个工作.

我正在尝试构建junit来测试作业中的特定任务.

因此我使用JobLauncherTestUtils库.

当我运行我的测试用例时,我总是得到NoUniqueBeanDefinitionException.

这是我的测试类:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = {BatchConfiguration.class})
public class ProcessFileJobTest  {

    @Configuration
    @EnableBatchProcessing
    static class TestConfig {
        @Autowired

        private JobBuilderFactory jobBuilder;
        @Autowired
        private StepBuilderFactory stepBuilder;

        @Bean
        public JobLauncherTestUtils jobLauncherTestUtils() {
            JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils();
            jobLauncherTestUtils.setJob(jobUnderTest());
            return jobLauncherTestUtils;
        }


        @Bean
        public Job jobUnderTest() {
            return jobBuilder.get("job-under-test")
                    .start(processIdFileStep())
                    .build();
        }


        @Bean
        public Step processIdFileStep() {
            return stepBuilder.get("processIdFileStep")
                    .<PushItemDTO, PushItemDTO>chunk(1) //important to be one in this case to commit after every line read
                    .reader(reader(null))

                    .processor(processor(null, null, null, null))
                    .writer(writer())

                            // …

我在Tomcat 7中使用Spring Batch Admin 1.2.1.我正在尝试读取663 MB文件并收到以下错误.我也增加了Tomcat的堆大小但没有白费.这项工作非常直接.它读取平面文件并将其保存到DB,只需很少的处理.请帮忙.

15:05:10,535  INFO http-apr-8181-exec-4 SimpleStepHandler:133 - Executing step: [load]
    15:05:10,981 ERROR http-apr-8181-exec-4 AbstractStep:212 - Encountered an error executing the step
java.lang.OutOfMemoryError: Java heap space
    at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:3098)
    at org.apache.catalina.loader.WebappClassLoader.findResource(WebappClassLoader.java:1244)
    at org.apache.catalina.loader.WebappClassLoader.getResource(WebappClassLoader.java:1407)
    at org.springframework.core.io.ClassPathResource.exists(ClassPathResource.java:139)
    at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:248)
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:134)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy30.open(Unknown Source)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:93)
    at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:105)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:93)
    at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:301)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:192)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:135) …

我正在尝试使用循环从数据库中读取一些记录,然后对记录进行一些计算(更新名为total的字段).

但我是春季批次的新手所以请任何人都可以提供一些提示.

我有一个 Spring Batch (2.2.2) 应用程序，由于某种原因无法使作业参数增量工作。该步骤声明如下：

<job id="job1" xmlns="http://www.springframework.org/schema/batch" incrementer="incrementer">
    <step id="step1" parent="step" />
</job>

<bean id="incrementer" class="org.springframework.batch.core.launch.support.RunIdIncrementer" />

当我将断点放入增量器时，它甚至没有被调用。

使用相同的参数两次调用作业会出现以下异常：

A job instance already exists and is complete for parameters={fail=false}.  If you want to run this job again, change the parameters.

我在这里检查了官方样品

https://github.com/spring-projects/spring-batch-admin/tree/master/spring-batch-admin-sample

它有同样的问题

我很难使用 Spring-boot 将 Spring-batch 与 Spring-batch-admin 结合起来。

因为我使用的是 Spring-batch-admin，所以我必须禁用 @EnableBatchProcessing 并手动配置它提供的两个构建器。

这来自我的问题：

集成 spring-batch 和 Spring-Batch-admin 时出错

现在，当我想使用 @StepScope 和属性注入将参数从 jobContext 参数传递到 ItemReader 时，我尝试做简单的场景

我按照这里的示例（该示例没有 spring-batch-admin 和 spring boot）

使用@StepScope解释

我得到这个例外：

2015-01-06 18:22:09.852 ERROR 10260 --- [nio-8080-exec-1] o.s.batch.core.step.AbstractStep         : Exception while closing step execution resources in step step2 in job MyTestJob

java.lang.ClassCastException: com.sun.proxy.$Proxy74 cannot be cast to org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader$$FastClassBySpringCGLIB$$ebb633d0.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653)
    at org.springframework.batch.item.file.FlatFileItemReader$$EnhancerBySpringCGLIB$$e99a9c6b.close(<generated>)
    at org.springframework.batch.item.support.CompositeItemStream.close(CompositeItemStream.java:85)
    at org.springframework.batch.core.step.tasklet.TaskletStep.close(TaskletStep.java:305) …

我是Spring Batch的新手,我有一个以下主程序,我想将它转换为测试用例jobLauncherTestUtils.我们怎么做到这一点？

我按照http://docs.spring.io/spring-batch/reference/html/testing.html,但我没有看到任何指针.请指导.

private void run() {
        String[] springConfig = { "spring/batch/jobs/job-extract-users.xml" };
        ApplicationContext context = new ClassPathXmlApplicationContext(springConfig);

        JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
        Job job = (Job) context.getBean("testActualJob");

        try {
            JobParameters param = new JobParametersBuilder().addString("age", "20").toJobParameters();

            JobExecution execution = jobLauncher.run(job, param);
            System.out.println("----------------------------------------------");
            System.out.println("Exit Status : " + execution.getStatus());
            System.out.println("Exit Status : " + execution.getAllFailureExceptions());
            System.out.println("-----------------------------------------------");
        } catch (Exception e) {
            e.printStackTrace();
        }

        System.out.println("Done !!");
    } 

根据这个答案，您可以使用 Spring Batch 类org.springframework.batch.support.SystemPropertyInitializer在 Spring Context 启动期间设置系统属性。

特别是，我希望能够使用它来设置，ENVIRONMENT因为 Spring Batch 配置的一部分如下：

<bean id="placeholderProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="locations">
        <list>
            <value>classpath:/org/springframework/batch/admin/bootstrap/batch.properties</value>
            <value>classpath:batch-default.properties</value>
            <value>classpath:batch-${ENVIRONMENT:hsql}.properties</value>
        </list>
    </property>
    <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
    <property name="ignoreResourceNotFound" value="true" />
    <property name="ignoreUnresolvablePlaceholders" value="false" />
    <property name="order" value="1" />
</bean>

但SystemPropertyInitializer用于afterPropertiesSet()设置系统属性，显然这是在配置后PropertyPlaceholderConfigurer发生的。

有可能实现这一目标吗？

我在春季批处理步骤中为阅读器设置了ItemRepositoryReader。我想使用findByScenarioBetween方法，该方法带有两个参数min和max。

我的回购

public interface FuneralPricingRepository extends JpaRepository<FuneralPricing, Long> {

    @Query("select p from FuneralPricing p where p.scenario between ?1 and ?2")
    List<FuneralPricing> findByScenarioBetween(Long min, Long max, Pageable pageable);


}

我的读者

  @Bean
  @StepScope
  public ItemReader<FuneralPricing> funeralPricingRepositoryReader(
          @Value("#{jobParameters[start]}") Long start,
          @Value("#{jobParameters[end]}") Long end) {
    RepositoryItemReader<FuneralPricing> repositoryItemReader = new RepositoryItemReader<>();
    repositoryItemReader.setPageSize(10);
    repositoryItemReader.setSort(funeralPricingSort());
    repositoryItemReader.setRepository(funeralPricingRepository);
    repositoryItemReader.setMaxItemCount(100);
    repositoryItemReader.setMethodName("findByScenarioBetween");
    List parameters = new ArrayList();
    parameters.add(start);
    parameters.add(end);
    repositoryItemReader.setArguments(parameters);
    return repositoryItemReader;
  }

当我使用findAll时，它工作正常。但是，我不想每次都阅读所有行。我不知道我的阅读器是否缺少任何内容！，谢谢。

java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.springframework.data.domain.Page at org.springframework.batch.item.data.RepositoryItemReader.doPageRead(RepositoryItemReader.java:208) at org.springframework.batch.item.data.RepositoryItemReader.doRead(RepositoryItemReader.java:153) at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:88) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at …

我们计划从spring-batch-admin迁移到spring-cloud-dataflow,因为批处理管理员将进入Spring Attic,生命结束日期为2017年12月31日.

根据Migration Doc中的文档," 在Spring Batch用例中,每个批处理作业都打包为一个在Spring Cloud Data Flow中注册的独立SpringBootüberjar "

在这种情况下,如果有50个批次作业,必须创建50个超级罐？这样做将是一场维护噩梦,创造许多罐子并且难以维护它们.

任何相同的解决方法？所以我们可以将所有批量作业打包在单个超级罐中,我们可以在需要时随机启动任何工作吗？

任何帮助将非常感谢,提前感谢

BackGround -

I am trying to deploy two spring batch applications as .war in same cluster in a single Weblogic Domain & each of them have spring batch admin console configured in servlet.xml like below -

<?xml version="1.0" encoding="UTF-8" ?>
<beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

  <!-- Spring Batch Admin -->  
  <import resource="classpath*:/org/springframework/batch/admin/web/resources/servlet-config.xml"/>
  <import resource="classpath*:/org/springframework/batch/admin/web/resources/webapp-config.xml"/>

  <bean id="resourceService" class="org.springframework.batch.admin.web.resources.DefaultResourceService">
    <property name="servletPath" value="/batch" />
  </bean>  

</beans>

Its a maven project & spring-batch-admin-manager 1.2.2 is pulled in as dependency.

Problem -
I am struggling …

我正在开发Spring Batch- MultiResourceItemReader & HibernateItemWriter示例.我能够成功编译代码,但在尝试运行它时,我看到以下错误即将发生,不知道这里出了什么问题.

Exception in thread "main" org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Failed to import bean definitions from URL location [classpath:context-model.xml]
Offending resource: class path resource [spring-batch-context.xml]; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected exception parsing XML document from class path resource [context-model.xml]; nested exception is java.lang.NoClassDefFoundError: org/springframework/context/event/EventListenerFactory
    at org.springframework.beans.factory.parsing.FailFastProblemReporter.error(FailFastProblemReporter.java:70)
    at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:85)
    at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:76)
    at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.importBeanDefinitionResource(DefaultBeanDefinitionDocumentReader.java:229)
    at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.parseDefaultElement(DefaultBeanDefinitionDocumentReader.java:180)
    at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.parseBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:165)
    at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.doRegisterBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:138)
    at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.registerBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:94)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.registerBeanDefinitions(XmlBeanDefinitionReader.java:508)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:392)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:181)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:217)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:188)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:252) …

我正在使用spring-boot-starter-web和spring-boot-starter-data-jpa以及spring-batch-admin(版本1.3.0)将现有的Spring-Batch命令行应用程序迁移到webapp.

我尝试配置应用程序,以便它可以使用多个数据库.经过大量的尝试和错误后,我终于遵循了基于JTA的示例stackoverflow.com/questions/22779155/.

但我仍然BeanCreationException一次又一次地面对同样的事情(见帖子末尾的StackTrace).

由于spring-batch-admin集成可能会出现问题？

以下是ServletInitializer的外观:

@Configuration
@EnableAutoConfiguration(exclude = { BatchAutoConfiguration.class,
    DataSourceAutoConfiguration.class, WebMvcAutoConfiguration.class })
@Import(MainConfiguration.class)
public class BatchAdmin extends SpringBootServletInitializer {
      public static void main(String[] args) {
          SpringApplication.run(BatchAdmin.class, args);
      }
      // ...
}

这里是主要的配置类,如在提到的SO文章中所建议的:

@Configuration
@ComponentScan("com.company.*")
@Import({ ServletConfiguration.class, WebappConfiguration.class, Db1Configuration.class,
    Db2Configuration.class })
@EnableTransactionManagement
public class MainConfiguration {

  @Bean
  public JpaVendorAdapter jpaVendorAdapter() {
    HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter();
    hibernateJpaVendorAdapter.setShowSql(true);
    hibernateJpaVendorAdapter.setGenerateDdl(true);
    hibernateJpaVendorAdapter.setDatabase(Database.POSTGRESQL);
    hibernateJpaVendorAdapter.setDatabasePlatform(MyPGDialect.class.getName());
    return hibernateJpaVendorAdapter;
  }

  @Bean(name = "userTransaction")
  public UserTransaction userTransaction() throws Throwable {
    UserTransactionImp userTransactionImp = new …