It's important to realize that non-pet people (of which there are apparently many) don't think the same way about animals that we do. My husband likes animals, but didn't have any of his own when he met me. In order to help him understand how I feel about my animals (rats and reptiles) I compared it to "hobbies" that he has: tennis and chess. Non animal hobbies take both time and money, but the key is not letting them overwhelm you or your budget. i.e. if you can't afford medicine for yourself but MUST go out and buy the latest video game etc. Or you spend hours a day playing video games but don't spend time with your partner, etc. It works the same way with pets. I make sure that the ratties and my other pets have their needs met and I spend time with them every day, but I don't go out and spend money like crazy buying them all the toys I would want them to have, especially if we have other expenses we must meet. I also try and schedule my critter time for when my husband is doing other things so that I can spend time with him when he's around. He interacts with the animals some too, which helps.
Being with anyone is an act of balancing both persons' needs and often budgets (in the case that you're living together sharing expenses). If both parties involved don't realize this often things don't work out. I hope you made the right decision for yourself, but with your next relationship, hopefully my advice will help.
Good luck!