An early decision I had to make was whether I should programmatically determine which reports were available to test and programmatically determine which of their parameters were required, etc… or if I should track my own expectations for the reports I expected and the parameters owned by each. I went with the later because I don’t trust the devs to keep the former stable. JART needs the ability to determine when the wrong reports get released; a common dev mistake.

Since I have about 150 distinct reports, each with their own combinations of shared filter controls and possible filter values, I made a matrix in MS Excel. The matrix rows represent each report, the columns represent each filter control, and the intersections are the filter values I use to pass into JART for each report’s filter criteria controls. This single spreadsheet controls all the tests JART will execute.

Another advantage, for me, to controlling the tests via an Excel spreadsheet is that my BL already maintains an Excel spreadsheet that specifies which of the 150 reports should be available in each build. The BL’s list can control which reports JART tests, just like the BL's list controlled which reports I tested.

JART simply loops through each report in said matrix and provides standard verifications for each. Verifications are important, and tricky for report AUTs, so I’ll save those for the next post.