Normally, when the subject of e-readers comes up in conversation, I point to the display as the main roadblock. Everything else in the design of the product is pretty well-understood and a lot cheaper than you expect. The key is how good the display is - not just in resolution and contrast but how it hits the battery.

The one thing I normally leave out is how the books themselves get onto the e-reader, something that Kat Hannaford has (rightly) slammed at Gizmodo. However, none of these problems are technological: they are purely about the way in which the publishing industry works and what level of protection they will attempt to impose to maximise revenue and lock out competitors.
The display, which is the key component in an e-reader after all, is critical. How that technology evolves will determine how big the e-reader market, what the devices will look like and how they will behave. The first thing to get across is that the e-book market is not synonymous with the e-reader market. The success of the e-book market is, despite the draconian DRM measures publishers and distributors have imposed, assured. Give people a light, long-lasting, high-resolution display, they will read books on an electronic device.
Take the CrunchPad/JooJoo, for example. If you show someone a colour tablet, they will probably agree that it's something they'd happily sit on a couch reading – although they would be right to worry about eyestrain with today's displays. Tell them it costs $500, barely lasts a single evening on one charge and isn't very flexible, and their interest will quickly wane. They will make do with a laptop. They probably already have one and, although it's a bit heavier, the overall experience of web-surfing on one is about the same.
