As a Canadian born Chinese who grew up with English as my first language, I can safely say that learning Cantonese is harder than Mandarin, but neither are really that hard to learn to speak. I see non-Chinese doing it fluently all the time. Writing/reading however, is pretty much impossible.