Jenkins run selenium UI tests using chromedriver not work as expected, driver.quit sometimes doesn't work

We struggled with a similar situation with our testing efforts. We also wanted to leverage the TestNG parallelization along with Selenium web driver. Unfortunately, managing the web driver life cycle within each test class or trying to use a static class holding the web driver wasn't effective when enabling any of the TestNG parallel modes. ie. parallel="tests|classes|methods". Matching classes and methods under test with the active TestNG thread and the instantiated web driver was unwieldy when it worked at all. We were spending more time figuring out the web driver lifecycles than getting to the real web application tests. We found a better way. We have an open source project that abstracts away the web driver lifecycle management for TestNG/Selenium testing. It properly supports each of the TestNG parallel modes providing the correct and active web driver for each test class method under test. It's helped us greatly in our testing efforts. Perhaps it may be of use in your case as well. Or if nothing else give you an idea of how you might tackle it differently. We haven't yet integrated with Jenkins, but this may help. The project is over on GitHub as Concurrent Selenium TestNG (COSENG). Kind regards.