It's an architecture that depends on long pipelining (20 stages initially, 31 starting with the Prescott). Long pipelines inherently decrease the number of instructions per cycle but counters that by allowing the CPU to run at a higher clock rate.
One of the side-effects of this was that some instructions wound up faster than previous generations, others wound up being slower. A particularly curious set of common instructions were slowed down by the new architecture -- rotate and shift. You know how you could do a certain shift and wind up with a very quick integer double/half operation? It wasn't as quick with the P4. Changes like that made it hard to target multiple processors because, given that particular example, what's fast on the P4 might be slow on an Athlon or P3, and vice versa.
The situation improved a bit with the Northwood core, which sported a doubled L2 cache, a smaller fabrication process, and hyper-threading technology enabled on certain chips. Eventually Intel did a major rehashing and wound up with the Prescott.
The Prescott, to put it lamely, is too hot to handle. It supported all of the same features, in addition to SSE3, a fancy new 90nm fab process, and, on later models, 64-bit and virtualization. It also doubled the cache of the Northwood, to 1MB (and eventually 2MB), per what would appear to be NetBurst tradition. Sadly, that's where the nice things end. Prescotts were found to perform worse in many situations when compared to a similarly-clocked Northwood chip. Not to mention that it runs bloody HOT. Remember how hot I mentioned that my Prescott runs? Mine is only a 2.8GHz -- imagine how hot a a 3.4GHz+ one must run, let alone one of those dual-core monstrosities that Intel released (the Pentium D).
Given the overall design of the NetBurst architecture, Intel clearly designed it simply to beat AMD in clock speed alone. In a way, they succeeded, claiming firsts for the 2GHz and, I think, 3GHz marks. However, by that point, the techies knew better and the soccer moms and grannies didn't seem to care anymore than they used to.
