This is one of those, "I don't know why it broke but here's what fixed it". Safari started opening without any windows—private or otherwise.

Using AppCleaner¹, search for "Safari". I removed everything except what's remaining in this screenshot. If there's permission issues, use a terminal or Finder.

NB: Be wise in what you delete.