What I mean by "tests can be slop"
A fellow reader writes in to say "pls no" in response to my comment that nobody reads tests.
That's [coming up with test cases] much more fun! And rewarding. And it's fine if the tests are slop. Nobody reads those.
Which sounds more controversial than you'd think until you've seen tests people write in the wild. Your fellow reader was right to complain. It can be bad out there.
The tests were slop and hard to understand. I don't know if they were AI-generated or just written by someone who didn't really know how to do it correctly. I ended up deleting them. It was a complete waste of time and the tests provided no additional value whatsoever, because when they did fail, I couldn't tell if it was my fault because I couldn't understand the test's intention (or its interdependencies with tests ran before it... because they built upon previously ran tests with the data changing across them...)
PS: you can read and share this online
Slop does not mean ignore good practices
Okay lots to unpack here. This sounds like an atrocious test suite.
When you write tests, AI slop or not, they should follow best practices:
- test the outcome, not the implementation
- name tests based on intent, not what happens
- make tests independent
- use shared fixtures or factories to set up data/state before tests run
- typical code needs integration tests – unit tests are for small algorithmic leetcodey functions
- use the type system, don't try to use testing to do the job of types (super common)
- avoid mocks and let your system do its thing. Mock the current timestamp, not the response of a function you wrote.
Name your test "Given X, Y, Z, when user does B, then Blah". Then test that Blah happened.
Think about test cases, not the implementation
Coming up with combinations of X, Y, Z, B, and Blah is interesting and important engineering work. It's okay if those tests are repetitive and follow the same pattern:
-
set up X
-
set up Y
-
set up Z
-
call B
-
test Blah
-
set up X
-
set up Y
-
set up Z
-
call B
-
test Blah
...
You're going to have dozens of tests that look almost the same. For different combinations of X, Y, Z, different values of B, and Blah. Use AI to help you write these. Make them even more obvious and repetitive than you think is reasonable. It's going to look like slop. That's okay.
The worst you can do is look at the repetition, get annoyed, and write an abstract hammer factory factory method that iterates through the combinations and runs a super awesome abstract test that can test almost anything. I've done this. It's a trap.
It's better to have more tests that are straightforward and easy to understand, than fewer tests that use magic. Focus on intent and make it obvious.
Be clever in your code. Keep tests simple. And if you're really worried about how much set up your tests take, write code that relies less on state and more on inputs. 😉
Cheers,
~Swizec
PS: don't be too clever in your code. You have to be twice as smart to debug your code than you were writing it.