Both games are rather long and share many similar themes and gameplay concepts so I wouldn't say it's necessary to play both. Personally, I prefer much of the gameplay, characters, and themes in Persona 4, but the game has some stupidly hard fights and at times it's painful to watch the characters play Japan's dumbest team of detectives. Persona 3's gameplay is more fun in some of the expansions, bringing it up to P4's level, and it still has a good cast of characters, but instead of the theme of identity, Persona 3 focuses on a suicide theme that's pretty morbid, if well handled.
One of the coolest accomplishments of the Persona series is that it makes playing an RPG of high school and everyday life fun. The fact that you skip past most of the boring stuff and the homework is a definite bonus, but the game focuses on developing certain attributes and creating social links between the main character and his teammates, classmates, friends, and family in order to create more powerful Persona. The gameplay reward is a nice carrot but in most of these cases I became addicted to the story. Each Social Link has its own tale to tell and most of them are really good. These ministories added to the overall plot and the development of the main character's teammates over the course of the game creates a strong bond between all of these characters not just between them and the main character, but with the player. I really cared about Junpei, Naoto, Kanji, and the rest of them, and whenever I've talked with another Persona player they likewise talk about these characters in a familiar fashion.
The gameplay in Persona follows a relatively simple rock-paper-scissors like system, but unlike, say, Pokemon, there are potentially several combatants on each side with their own specific strengths and weaknesses. The strategy in Persona is not the most complex but it is immensely satisfying to work out how to cover teammates' weaknesses, make full use of their strengths, and win the day. Also, there are fights in both games that are hard as hell to beat and will require some serious strategy and patience to get through. The act of expanding the main character's array of Persona is probably my least favorite part of the gameplay but I know others who have had a ton of fun mashing Persona together in order to get just the right kind of creature they need.
Damn good, if slow-paced storytelling. Trying out at least one of the Persona series is a must, to experience the full breadth of characters, situations, and the way a game can take a singular theme and run forward with it for 70+ hours of gameplay and still leave you wanting more.