The only way to know is if you can see a pattern, or are loosing it form the traditional bald spot or receding hairline. If it is male pattern baldness you will also be loosing some shorter and thinner hairs. Depending on the length of your hair and extent of your loss you my or may not notice these. If you do then it is a clear sign that you have male pattern baldness.
Chances are it is male pattern baldness. You didn't mention your age, but even if you are young you can get hit with male pattern baldness. It is normal to get 4-5 hairs on a pull, with male pattern baldness you have to worry about them growing back thinner and shorter the next time. If you truly suspect it is male pattern baldness then you should start thinking about treatments. Most will recommend getiing on finasteride.
Just a note. My hair is about 3-4 inches long. If I run a beard comb ( really fine and small) repeatedly thorugh my thinning area I get about 15-30 small male pattern baldness afflicted hairs each day. The hairs are thin and much shorter than my other hairs. They also have a tapered tip which proves they were never cut.