You only have to bypass the router for enough time to rule it out, and how long that is depends on how frequent the problem occurs. You said multiple times a day, so you should be able to see if the drop occurs within a few hours. I suggest testing overnight or doing down time. That's where pingplotter comes in. You can leave it running overnight and not only will it tell you if/when the problem happened, but if when it happens, if the packet is still making to the destination. If so, doesn't matter that your not getting the return packet. You can also try just running pingplotter without bypassing the router and only bypass if/when you have confirmed that the error you are seeing is relevant and not caused by something else.
To that end, can you give some context? Like, what model modem and router do you have? Anything new/different a few weeks ago? Has the problem happened before that? What is the nature of the problem? As in, what are you trying to do or fix? What lead you to do the ping test to begin with? Gaming? VOIP? WFH? Streaming?