Answer: It depends.
This is because it is actually two privileges: the spousal testimonial privilege, and the confidential marital communications privilege.
The spousal testimonial privilege means that your current spouse can refuse to testify in a court proceeding against you. But if your spouse chooses to testify, you cannot stop him/her.
The confidential marital communications privilege, on the other hand, means that your spouse cannot testify about information you shared with him/her in confidence while you were married.
And even if your spouse wants to testify about confidential marital communications, you CAN prevent him/her from doing so.
Plus, the confidential marital communications privilege applies to ex-spouses as well as current ones. But it also applies only to confidential communications that took place while you were married (not before or after).