2

Background:

  • I'm working with (for me) a reasonably large codebase (eg: I've only got a few of the related projects checked out at the moment, and its > 11000 classes).
  • Build is ant, Tests are JUnit, CI is Jenkins.
  • Running all tests before checkin is not an option, it takes Jenkins hours. Even for some of the individual apps it can be 45 minutes.
  • There are some tests that don't reference individual methods based on reflection, and in some cases don't even directly reference the class of the tested methods, as they interrogate an aggregator class, and are aware of the patterns of pass-through methods in use here. As it's a big codebase, > 10 developers, and I'm not in charge, this is something I can not change for now.

What I want, is the ability to before check-in print out a list of all test classes that are two degrees away (Kevin-Bacon-wise) from any class in the git diff list. This way I can run them all and cut down on angry emails from Jenkins when something I missed eventually gets run and has an error.

The easiest way I can think of to achieve this is to code it myself with a Ruby script or similar, which allows me to account for some of the patterns we're using, but to do it I need to be able to query "which classes reference class X?"

I could parse .java or (easier) .class files to get this info, but I'd rather not :) Is there a way I can make Javac export it in a simple format as it compiles?

Improve this question
2
  • More info: I need to run this on my workstation, to catch errors I've made before they go upstream, at least until I can demonstrate it's useful. Switching to Git, Maven, and improving the Jenkins process would be nice, but it's not something I can make happen. Commented Apr 18, 2013 at 5:50
  • There are many tools that analyze dependencies between classes. YOu might be able to use on that has textual output and build a dependency graph. Also look at Sonar, which I don't know anything about but is supposed to be a great tool for any source analysis Commented Apr 19, 2013 at 0:02

1 Answer 1

Reset to default
3

Is there a way I can make Javac export it in a simple format as it compiles?

AFAIK, no.

However, there are other ways to get a list of the dependencies:

(Note however that you are unlikely to get a static tool to extract dependencies resulting from Class.forName(), etcetera. Also note that you cannot infer the complete set of dependencies from bytecode files because of the way that "compile time constants" are handled.)

It strikes me that there are a few problems here:

  • It sounds to me like your build, and indeed your project structure is monolithic. If you could restructure the code base into large-scale modules that build separately (according to their dependencies), and version controlled separately, then you only need to do a full build and run all unit tests when there is a change high up ... in a module that everything else depends on. (Can I suggest the "Maven" word. It really helps for a large codebase, and 11,000 classes is large.)

  • It sounds like you may be suffering from the "branches are hard" problem of classic VCS systems.

  • It sounds like you may need a beefier CI system. If you've got more cores and the build framework is right, you should be able to get faster CI builds. (And if you modularize so that you rebuild less ...)

I think it might be easier to address your slow build/test cycle that way rather than via extra (possibly bespoke) tooling to do dependency analysis.

But I recognize that it may not be up to you to make those decisions.

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

I'm trying to better catch errors before they get to Jenkins.
Hmmm ... have you considered using Jenkins to build your development branch? A beefier CI might be needed, but hardware is cheaper than developer time, especially if the hardware is shared by 10 developers.
The other point is that you can do parallel Maven builds on your desktop, assuming it has a bit of grunt. (You can get a 4 core / 16Gb machine for around $1K these days ... assuming you are in the US of A.)
I've actually been considering setting up set of POMs, but I haven't tackled it as I know how much trouble mvn gives you when you don't follow its demands wrt directory layout, and when looking for how to do so most answers tend to be along the lines of "f*** you, do it Maven's way"
A better characterization would be "Don't make life hell for yourself. Do it Maven's way". A codebase restructure is a once-off cost, and if you are using a decent VCS it won't disrupt your history. Yes it is all extra work, but think of the pay-off.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.