It is true, without having to bring Fallout 3 in to this, in the previous fallout games, all races have default start values and a default acceptable range for each attribute. If you look you'll notice really common similarities with states, i.e. Super Mutants will almost always have low cha and often times low int, whereas they almost all have incredibly high str. In fallout 2 if you go to broken hills and talk to the super mutants, you'll realise that they all have abnormally high str.
Then of course there's Fallout Tactics by interplay (a horrible game at that but the MP was nice), which had playable ghouls and super mutants which fell in line with the standards already set by its predacessors (str is 6 min 13 max or something like that). Let's face it, a ghoul just isn't going to have 10 cha 1 end.
Super mutants, ghouls, humans, and whatever else and biologically completely different, and their highly-contrasted SPECIAL stats are constantly brought up in dialogue and plot development. You can't say it falls in to lore to have all characters start with 5 in every stat with 5 more points, every stat having 1-10 as the min/max.
Edit: Also, I'd like to point out with the ghoul/radiation thing, go to Gecko plant in fallout and talk to the ghouls there. They love the radiation- It makes them feel spectacular, and the radiation is what caused them to have lifespans of 400 years.
Lastly, Fallout 3 is dumb. I mean, seriously, a ghoul was created by having a nuclear bomb drop directly on top of her. That's BS.