Kansas...I previously posted a link on Voice mail..however, it may not address your issue.
Is your showing of the 2 new messages appearing on the Phones answering machine window, but you are getting No actual messages left when you try a phone playback?
Login to the Cox online Phone Voice Tools and see if the Voice Mail there was activated. If you see entries here, then maybe the Cox Setting/pickup is set lower than the actual Phones Voice Mail/Answering. It took me a bit to wade thru this 2 Voice mail storage locations.... You can also check the Call History to try and pin down when calls are received and pair them with when you think messages are left. Do testing with your cell phone calling and leaving a message.
You can also check the Phones CallerID history, looking to sync up the calling times. I'm not sure by your post what is actually missing....looks like the indicator on the physical phone answering machine is incorrect and not getting updated. Maybe pulling the phone plug may work.