An hour of googling discovered that Dell had nothing decent, HP had something kind of decent but crippled with UEFI so as not to be able to boot Linux, and in general, it is getting hard to find boxes with room for a second let alone third hard drive.
Looking at Amazon's top sellers in desktops, the next tower-ish computer I saw was Lenovo's H530. A little checking showed that one version of the H530 was on Ubuntu's approved list. The H530 only provides a single VGA output connector, and no PS/2 keyboard/mouse ports, but that's ok for me. After hesitating between an i3-4150 6GB version from a small vendor for $426 and an i5-4460 8GB version for $550 from Amazon themselves, I let myself be lured in by the extra 3MB of L3 cache and the extra 2 cores, and went for the i5.
After reading Techreport's SSD Endurance Experiment and Luke's SSD test, and briefly considering hybrid disk drives (but shying away after seeing bad reports), went with the 250 GB Samsung 840 EVO MZ-7TE250BW SSD, $113 (Newegg) / $129 (Amazon).
And bada bing bada boom, I went from "I need a new PC" to having one on the way in four hours, including all waffling over cpu and ssd types. Things are easier than they used to be... at least for now. Next time around I might not be able to find one that meets all my requirements.
The box arrived promptly. Here's a picture of the inside:
The case does have room for a second hard drive (either desktop or laptop size) -- vertically along the side.
The software included with the SSD let me clone the windows drive onto the SSD on the first try. Ubuntu 14.04 installed uneventfully, or at least, any problems I had were easily solved, and the system dual boots nicely.
The computer is actually a gift for my wife; now that I've given it to her, I can drop the cover story :-) Her old Microsoft Windows 7 computer was an e7400 with a tiny SSD and a second hard drive; I'd planned to just move the 2nd hard drive, but it turns out to be old and small. Time to buy a new one.
Shopping around a bit, the sweet spot at the moment appears to be $75 for a 2TB WD Green drive at NewEgg ($82 including tax and shipping). (Yeah, WD Green drives aren't speed demons, but given the OS is on an SSD, performance should be fine.)
I also realized that, given the world political situation, we probably can't trust foreign antivirus products anymore. So this time we'll probably get a McAfee subscription, which adds another $30/year to the cost. Sigh.
This drove my wife crazy until I put a USB ethernet adapter in parallel with the main one. Now they both have to fail before she's left without internet.