It’s a hard choice, but I’m with the majority here – Prisoner of Azkaban first, with Order of the Phoenix a close second. I think it’s because in both of those books, you could feel the whole world of the books shift a little – each time it got a little deeper, a little darker, and so much richer than it had been before.
Books 1 and 2 were really just Harry and his friends having adventures that happened to be related to Voldemort, but there wasn’t a sense of an overarching plot that went beyond the immediate stories. Like Trumi says, with book 3 you get the rich history of the Marauders and you start to see Harry’s place in the world a little better – and then with book 5, Harry’s thrown headfirst into a world at war, and it almost becomes an epic story.
I love epics – but maybe that’s because when I find a good book, I really just don’t want it to end… =)