I’m a happy user of vagrant, and over Christmas I was setting up a new set of instances for work, using Ubuntu under KVM. It’s always desirable to have a target VM that exactly matches your production environment, so your testing matches where you’re going to use your work.
Sure enough, it failed.
I know this works under Ubuntu, but my dev machine is running Fedora 25, and that probably hasn’t been tested. I can run VMs via Virtual Box, that’s fine, but it’s not exactly the same as production.
What has that to do with modems?
Well, in a previous life, I used to work with a company that did v32bis components for modems, and they called any “A doesn’t work with B” problems modem testing problems.
When modems were fairly new and there were dozens of manufacturers, each manufacturer tested their newest release against their previous release, and then perhaps against an old copy of a well-known competitor’s product, and released them into the wild.
At which point their customer-support line lit up with new customers complaining that their newly bought modem didn’t work with their ISP any more.
The usual answer was “tell your ISP to use our product instead of brand X”.
This was not particularly helpful: their ISP usually said “don’t use anything we haven’t tested, and we’ve only tested brand X”.
That’s exactly what happened here: the combination of vagrant, KVM, Ubuntu and Fedora hasn’t been tested, and, depending on the Ubuntu version, may have a failing NFS or may not have networking at all.
This is by no means unusual: tons of products only work if you get the exact configuration that the developers selected. That’s part of why I always have an exact copy of production in the first place!
What do you do about it?
If you’re an individual, file bugs. That’s the best thing you can do to help the implementors. Giving them a test sequence that they can add to their automated regression tests is a friendly act, so they won’t ever make the same mistake again. Be open to testing their fixes, and don’t just label them bozos (;-))
If you’re a contributor, have a matrix of stuff you need to at least smoke-test against. Making friends with beta-testers and bug reporters can give you a huge list of platforms that have been debugged, and you might talk them into running a cron job that makes them part of your nightly build-and-test farm.
You’re not going to have to solve the “every possible brand of modems” problem if you can find a community of people who care enough to report bugs and test fixes. You don’t actually ever have to test against everything, just the things people actually use. And if the people depend on your software, they’re motivated to want to have it run well. Recruit them!
