A subject line test is only useful if you can explain what won and why you will reuse it. Testing a witty line against a completely different offer, preview text, and From name at the same time teaches you nothing.
Test one thing
Hold the body, From name, send time, and audience constant. Change only the subject (or only the preview text). Common splits: curiosity vs. clarity, question vs. statement, specific number vs. vague benefit, and branded vs. personal From names on a separate test.
Sample size and timing
Do not call a winner after 40 opens. For most lists you need enough unique opens that a few random clicks cannot flip the result. Send both variants into the same window so a news cycle does not favor one side. Leave a holdout if you also care about unsubscribes, not just opens.
Opens are a noisy metric
Apple Mail Privacy Protection and image-blocking inflate or hide opens. Pair the test with clicks and replies when the mail asks for a click. If the goal is a reply, measure replies. Optimizing only for opens trains you to write clickbait that tanks trust.
What to do with a winner
Roll the winner to the remainder of the send, log the hypothesis, and retire the loser. Do not keep running the same curiosity-gap trick until the list learns to ignore you. Rotate patterns the way you rotate offers.
