My experience with this is that invites don't work however if your PS3 friend goes into one of their worlds playing online you can join (Invite only can't be selected) was just doing it the other day with my girl, however our issue was that we were unable to communicate via mic.
Best of luck!