Nope. Not strange at all.
I think it’s likely that you’re too shy with your boyfriend and not communicating your needs. YOU know how to get yourself off, you know what your body likes, you know what you like, you what works. So now you just have to find a way to incorporate that into your sex life with your boyfriend.
If you find yourself picturing a girl or wishing that your boyfriend was a girl, then that’s a different situation and the answer for that goes without saying…
But just because you think it’s hot watching two women together does not make you a lesbian. Maybe you like girls more than you thought, maybe seeing something different turns you on more, maybe you actually like girls more than guys and you’re not that into your boyfriend after all. I don’t know, you’re the only one who knows that.
My advice: relax, speak up, walk him through it, make sure he knows exactly what you want and need. You can’t be timid or shy when it comes to your sexual satisfaction, and you can’t expect him to be a mind reader. If there’s something you want and he’s not doing it, tell him! He would want to know.
If things don’t start improving, even with the extra work you put into it, then maybe it’s not meant to work out at all. Sometimes the chemistry just isn’t there.