Both buttons sit beside the key signature in the Editor, and they do different jobs.
Set changes only the song's default key signature. The chords stay exactly as you entered them. Set and Transpose transposes every chord in the chart into the key you chose, and sets the key signature to match.
Always make sure a song's default key is the key its chords are in as they appear in the Editor. If it is not, see Troubleshooting - Key signature doesn't match the chords shown.
For more, see Transpose a song on the iReal Pro website.