But testing exhaustively means the full range of inputs are used, and therefore, if a bug arises as inputs not considered by the dev initially, it would show up as a failure.
What you're actually saying is that reaching full exhaustive testing is near impossible.
I think what they're saying is that both the code and test will contain the bug, since the bug is in the programmer's understanding rather than just in their coding.
If I understood func(5, 6) should return 9, but the client actually wants 10, no amount of tests I write will reveal the error.
I don't think that any form of formal specification will solve that issue. The only way to minimize the impact of it is to work on small iterations (agile) and be close to the client. One of the weakness in agile methodologies today is that the client representation (Product owner) is not aware of the client needs or too far from the client point of view.
Testing for regressions is a a valuable thing to do, but your point is also valid. We can prove that it is impossible to test every scenario in general (for example, does this program end ;-) ). In my mind, unit tests exist for documenting what the code is doing now -- not to determine if it is correct (acceptance tests are different). Your goal for a unit test is to have it fail when the behaviour of the production code fails. This way you can verify that the changes you are making are having the effect you expected. Similarly, you can be warned when changes you make affect part of the code you didn't expect to be affected.
Of course, you will never get full coverage, but tests are a programmer's tool. It allows you to reduce the amount of time you spend inspecting parts of the code you aren't working on. For example, when I'm writing code with good test coverage, I can find out what happens if I change an attribute in a structure simply by doing it and watching the tests fail. Each failure tells me where in the code I'm likely to have to make a change.
If you make a small viewpoint change that tests are not to find bugs, but rather are a tool to help you understand the code base, they make a lot more sense. (IMHO)
What you're actually saying is that reaching full exhaustive testing is near impossible.