Tuesday, February 21, 2012

Admitting defeat and victory

tl;dr: bad data for testing can waste a lot of time.

I spent the last week or so trying to solve an issue.
The original goal was to reimplement an integration with a black-box system.
And it wasn't working.
Not only was it failing, but each testing request attempt crashed the system, preventing further testing without someone to restart it.
And of course there was no useful response or error info with the failure.

So I investigated and found an issue with the protocol and fixed it. No change.
As a baseline and comparison, I brought the old code into the new project. Same outcome. Failure without a cause, system down.
At this point it is getting too late into the development cycle and time to prepare the backup plan. This involved the new code calling the old project (not ideal since goal was to replace/remove it).

This is a difficult process: to admit defeat.
To admit that the world does not work as you expect it to, especially as a developer. To abandon your grand idea and accept your least ideal path. To hope one day to be able to come back and fix it given the time.
And in the end, the results were the same! Still failing.

Now this pointed to the request data being bad.
I had suspicions since it was one of the few differences (sending a remove command for something that didn't exist).
My guess was partially correct, but the root cause was bad test data. The ID being used was not valid in this dev environment. Normally this fails with a useful error message for other commands, but for remove it crashes the system.

So after a painful slow stretch of testing (having only 1 chance to test before killing it and waiting for restart), it wasn't even an issue with my code.
After using a valid ID, all implementations worked as expected.

Yes, this issue with bad test data had come up recently here.

No comments:

Post a Comment