I personally believe that Phantom was the book that started to bring everything together. It is in this book that Goodkind starts tying in everything from the first book, making the entire series seem goal-oriented and truly brilliant in the story compilation. So many unanswered questions begin to unravel in this book (i.e., the true story of Baraccus and the Temple of the Winds, the link between the chimes, orden, and the Sword of Truth).
Despite what others have said, this was a pivotal book for the whole series by directing itself towards the ultimate showdown.