My boyfriend has never ever told he doesn't want me to date other men, but I genuinely feel selfish for wanting another guy when I have such a wonderful one. When I've expressed this to him he always tells me he just wants me to be happy. And the only girl I've dated since we've been together was just casual, but I don't think my feelings will stay casual for this second guy.
I think there are two issues here: one is that things with your boyfriend aren’t spelled out enough, and there’s a lot of anxiety in ambiguity. “I just want you to be happy” is not enough communication - you need to know what your boyfriend’s boundaries and desires are for this relationship arrangement, what his best- and worst-case scenarios are, etc. This is a situation where “the absence of a no” is not the same as a “yes” and it sounds like you would be on much more secure footing if you had some more specific signoff from your boyfriend than “whatever you want.” So ask him!
Second, it sounds like you have a lot of internalized shame and worry about what “sex with a man” is and means. That’s normal - our culture teaches us from birth that male sexuality is wrapped up in ideas about purity (you should only be with one man or else you’re wrong, sinful, dirty, etc.) and ownership (it somehow violates inherent male property rights if you have sex with more than one) and things like that. Fortunately, you can often overcome those feelings by pulling them out into the light, articulating and examining them. I discuss that here!