I don’t think you will find any rules to detect a set because there are just too many ways to play them. I sometimes bet out like I hit a split pair. I sometimes check and let someone else bet for me. Some people will jam the pot looking like two pair. It all depends on the board and the players at the table. There are too many variables.
Id say the best way is to raise them up and hope they come over the top of you. Paying for info is the best way to find out what they have.