rmarcant
Hi
The invalid key error is only due to wrong Key added, the same issue is reported just 2 days back from another users and he also fixed it by create a new Captcha API key for his site.
Please regenerate the key for this domain and check.
About the 2nd question JomSocial use own registration, if you want to use any other registration you can use that as well. No need to tweak code as you have to publish your component registration menu for that.