Is it a "normal poly thing" for me as a female to have a gf and a bf but the two of them don't have a relationship? What are some tips to making this work and comfortable for everyone?
Yes, that is called “V-shaped” polyamory and is one of the more common polyamorous arrangements. It’s actually pretty rare for there to be three or more people where all parties are dating. Even though media depictions, news stories, fanfiction, etc. tend to focus on triads or polycules where everyone is in a relationship with everyone, that’s not as common.
Tips for making it work are going to be the same as in all other healthy polyamorous relationships. Open, honest communication. Everyone taking responsibility for identifying and articulating their own needs and feelings. Since this is a very common polyamorous relationship configuration, most of the advice you’ll find in my FAQ resources will be helpful.
Also, the idea of a “normal poly thing” is useless. Even if you three were the only people in the world doing this, that wouldn’t really matter. And plenty of things that are “normal” or common in polyamorous relationships are not healthy and happy. Normalcy is not a measure of anything meaningful. You don’t need to find out if something is “normal” before trying it out. If it’s healthy and happy for you, it’s fine.
Your girlfriend and boyfriend are what’s called “metamours,” linked not by their own relationship but because they’re dating the same person. It’s wise not to complain to one of your partners about the other one, or involve them in relationship issues. If you want to introduce them, keep things low-key and low-pressure. Let their relationship be what it’s going to be, don’t try to push for more intimacy than is naturally present. I wrote about introducing metamours to each other here.