停止/重新部署Tomcat 7+内存泄漏.Spring Data,JPA,Hibernate,MySQL

Question

停止/重新部署应用程序时,我有tomcat内存泄漏问题.它说以下Web应用程序已停止(重新加载,取消部署),但它们之前运行的类仍然在内存中加载,从而导致内存泄漏(使用分析器确认):/ test-1.0-SNAPSHOT

MySQL连接器驱动程序位于Tomcat/lib文件夹中.我可以在两者中重现这个问题:Tomcat 7/8.还尝试使用"net.sourceforge.jtds.*"驱动程序的MS SQL数据库,但没有帮助.

请在下面找到项目文件.Project仅在DB中创建1个表.

的build.gradle

group 'com.test'
version '1.0-SNAPSHOT'

apply plugin: 'java'
apply plugin: 'war'
sourceCompatibility = 1.8
repositories {
    mavenCentral()
}
dependencies {
    compile group: 'org.hibernate', name: 'hibernate-entitymanager', version: '5.2.10.Final'
    compile group: 'org.springframework.data', name: 'spring-data-jpa', version: '1.11.4.RELEASE'
    compile group: 'org.springframework', name: 'spring-webmvc', version: '4.3.9.RELEASE'
    providedCompile 'javax.servlet:javax.servlet-api:3.1.0'
    providedCompile group: 'mysql', name: 'mysql-connector-java', version: '5.1.6'
    compile group: 'commons-dbcp', name: 'commons-dbcp', version: '1.4'
}

ApplicationConfig.java

@Configuration
@Import({JPAConfiguration.class})
@EnableWebMvc
public class ApplicationConfig {}

JPAConfiguration.java

@Configuration
@EnableJpaRepositories("com.test.dao")
@EnableTransactionManagement
public class JPAConfiguration {

    @Bean
    public EntityManagerFactory entityManagerFactory() {
        LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
        factory.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
        factory.setPackagesToScan("com.test.model");
        factory.setDataSource(restDataSource());
        factory.setJpaPropertyMap(getPropertyMap());
        factory.afterPropertiesSet();
        return factory.getObject();
    }

    @Bean(destroyMethod = "close")
    public DataSource restDataSource() {
        BasicDataSource dataSource = new BasicDataSource();
        dataSource.setDriverClassName("com.mysql.jdbc.Driver");
        dataSource.setUrl("jdbc:mysql://localhost:3306/test");
        dataSource.setUsername("test");
        dataSource.setPassword("test");
        return dataSource;
    }

    private Map<String, String> getPropertyMap() {
        Map<String, String> hibernateProperties = new HashMap<>();
        hibernateProperties.put("hibernate.hbm2ddl.auto", "update");
        hibernateProperties.put("hibernate.show_sql", "true");
        hibernateProperties.put("hibernate.format_sql", "true");
        hibernateProperties.put("hibernate.dialect", "org.hibernate.dialect.MySQL5InnoDBDialect");
        return hibernateProperties;
    }

    @Bean
    public PlatformTransactionManager transactionManager() {
        JpaTransactionManager txManager = new JpaTransactionManager();
        txManager.setEntityManagerFactory(entityManagerFactory());
        return txManager;
    }

}

TestRepository.java

@Repository
public interface TestRepository extends JpaRepository<TestEntity, Long> {}

TestEntity.java

@Entity
@Table(name = "ent")
public class TestEntity {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String descript;
    //equals, hashcode, toString, getters, setters
}

AppInitializer.java

public class AppInitializer extends AbstractAnnotationConfigDispatcherServletInitializer {
    private WebApplicationContext rootContext;

    @Override
    protected Class<?>[] getRootConfigClasses() {
        return new Class[]{ApplicationConfig.class};
    }

    @Override
    protected Class<?>[] getServletConfigClasses() {
        return null;
    }

    @Override
    protected String[] getServletMappings() {
        return new String[]{"/"};
    }

}

命令

jmap -histo <tomcat_pid>

在tomcat停止后,只显示项目结构中的2个项目:

com.test.config.dao.JPAConfiguration$$EnhancerBySpringCGLIB$$792cb231$$FastClassBySpringCGLIB$$45ff499c
com.test.config.dao.JPAConfiguration$$FastClassBySpringCGLIB$$10104c1e

任何人都有想法或建议来解决这个问题？

Answer 1

这个小项目中有2个内存泄漏:

1. MySQL jdbc驱动程序的问题.

我们必须添加ContextLoaderListener注销jdbc驱动程序:

监听器:

@WebListener
public class ContextListener extends ContextLoaderListener {

    private final Logger log = LoggerFactory.getLogger(this.getClass());

    @Override
    public void contextInitialized(ServletContextEvent sce) {
        log.info("-= Context started =-");

    }

    @Override
    public void contextDestroyed(ServletContextEvent sce) {
        super.contextDestroyed(sce);
        log.info("-= Context destroyed =-");
        try {
            log.info("Calling MySQL AbandonedConnectionCleanupThread checkedShutdown");
            com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.uncheckedShutdown();

        } catch (Exception e) {
            log.error("Error calling MySQL AbandonedConnectionCleanupThread checkedShutdown {}", e);
        }

        ClassLoader cl = Thread.currentThread().getContextClassLoader();

        Enumeration<Driver> drivers = DriverManager.getDrivers();
        while (drivers.hasMoreElements()) {
            Driver driver = drivers.nextElement();

            if (driver.getClass().getClassLoader() == cl) {

                try {
                    log.info("Deregistering JDBC driver {}", driver);
                    DriverManager.deregisterDriver(driver);

                } catch (SQLException ex) {
                    log.error("Error deregistering JDBC driver {}", driver, ex);
                }

            } else {
                log.info("Not deregistering JDBC driver {} as it does not belong to this webapp's ClassLoader", driver);
            }
        }
    }
}

或者,如果您有权访问tomcat服务器,则可以在tomcat/conf/server.xml 示例中修改侦听器.

2. 第二个问题是jboss-logging库(链接)中已知的内存泄漏.

在我们从hibernate依赖项中排除这个库后,内存泄漏已经消失:

的build.gradle:

group 'com.test'
version '1.0-SNAPSHOT'

apply plugin: 'java'
apply plugin: 'war'
sourceCompatibility = 1.8
repositories {
    mavenCentral()
}
dependencies {
    compile(group: 'org.hibernate', name: 'hibernate-entitymanager', version: '5.2.10.Final') {
        exclude group: 'org.jboss.logging', module: 'jboss-logging'
    }

    compile group: 'org.springframework.data', name: 'spring-data-jpa', version: '1.11.4.RELEASE'

    compile group: 'org.springframework', name: 'spring-webmvc', version: '4.3.9.RELEASE'

    providedCompile 'javax.servlet:javax.servlet-api:3.1.0'
    providedCompile group: 'mysql', name: 'mysql-connector-java', version: '8.0.11'
    compile group: 'org.slf4j', name: 'slf4j-api', version: '1.7.25'
    compile group: 'org.slf4j', name: 'slf4j-simple', version: '1.7.25'
}

然后从repo构建jar 并添加到tomcat / lib文件夹.

jboss-logging的问题可能在Java 9(pull request link)中得到修复.

Answer 2

简短的回答 - 希望你也遇到同样的问题......

这两个com.test.config.dao.JPAConfiguration$$...CGLIB$$...类被Abandoned connection cleanup threadMySQL 间接引用：

20-Jun-2018 21:25:22.987 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [test-1.0-SNAPSHOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
 com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)

以下答案使我能够解决该问题。例如tomcat/conf/server.xml，在 中，查找该JreMemoryLeakPreventionListener行并将其替换为：

<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" 
    classesToInitialize="com.mysql.jdbc.Driver" />

这会强制 MySQL JDBC 驱动程序及其清理线程在 Web 应用程序的类加载器之外加载。这意味着清理线程不会将 webapp 类加载器的引用作为其上下文类加载器。

扩展答案 - 如何追踪环境中的泄漏......

希望以上内容就是您所需要的 - 这足以重现并解决https://github.com/egotovko/tomcat-leak的问题

然而，还有许多其他原因会导致 Web 应用程序引用泄露，从而阻止其取消部署。例如，其他线程仍在运行（Tomcat 擅长对这些线程发出警告）或来自 Web 应用程序外部的引用。

要正确追踪原因，您可以在堆转储中追踪引用。如果这不熟悉，您可以从 获取堆转储jmap -dump:file=dump.hprof <pid>，或者直接从诸如jvisualvm（也包含在 JDK 中）连接。

打开堆转储jvisualvm：

• 选择Classes堆转储按钮
• 按名称对类列表进行排序
• 在 Web 应用程序中查找类 - 例如com.test.config.dao.JPAConfiguration$$EnhancerBySpringCGLIB$$在此示例中
• 显示的实例数应该为 2 左右
• 双击以在Instances View
• 在References这些实例之一的窗格中，右键单击并Show Nearest GC Root
• 例如Abandoned connection cleanup threadMySQL 中的：

请注意 是如何AbandonedConnectionCleanupThread有一个 的contextClassLoader，它是ParallelWebappClassLoaderWeb 应用程序的 。Tomcat 需要能够释放类加载器才能取消部署 Web 应用程序。

一旦您找到了保存引用的内容，通常就需要研究如何更好地在 Tomcat 中配置该库，或者也许其他人已经看到了内存泄漏。当有多个参考文献需要清理时，必须重复该练习的情况也并不罕见。