Before I migrated my entire library to my computer, I sometimes made carbon copies and cut the theorems out just to reach this sort of effect. Then I could reshuffle and have what I want in front of me instead of flipping around in a book (or across several books) which I hated for the sheer physicality of it.

Nowadays, when I read mostly papers in PDF or books I have scanned in or ebooks, I do screenshots and copy/paste to OmniOutliner, an outliner program.

It's all still a hack though. I'm still waiting for a clean way from within standard ebook reading apps. For example, an API by iBooks that would allow any 3rd party flashcard app to interact with my notes/highlights.

I used to think this use case is so obvious that it'd be a matter of months that someone would come up with a way to make it more easy to both read and write/organize content this way. As it is, now I'm increasingly asking myself whether I'm an exception or whether this is not the way non-literary books _should_ be read after all.

Having some structure is useful when you don't know what you're looking for, or even what kind of thing you're looking for. And I find that's true of a lot of the technical documents I read. I also quite enjoy reading a physical book with a dozen bookmarks and fingers, being able to see several full pages of text at once.

I could see your way of doing things being useful, though. I suspect it hasn't been partly because it's harder than it sounds (look at the design process for HTML, and all of the mistakes that were made and alternatives that were proposed) and partly because so much of the relevant IP (and thus market) is already locked up pretty tightly.

I agree about the structure. The thing about most technical, or how to call them, non-literary? books is that clearly you are not expected to read everything. In a sense, you start with the given structure but make the book your own as you go along. You skim, see that you already know about this, that you are not interested in that, and are _very_ interested in that. What I'd like to see is a format that makes it easy and gives us new possibilities to make a book our own. What I didn't see is your second point. Perhaps I imagined it too easy, maybe it is actually a tough nut to crack. After all, the format must accomodate all sorts of books and styles. And the IP issue, oh my...

Allowing 3rd party apps to access iBook notes and highlights would be useful, but will probably never happen. It's one of the weak points of the app model that there's not any good way to work with other programs' data. On the desktop, you could probably write a script to read annotations from a PDF and dump them into something like SuperMemo, but on tablets you need everyone to agree on a format to export so you can email them to yourself. Maybe Android is better about this.