Moore's Law and disk sizes

I got into computers back when 4K was a lot of memory; the first real computer I owned came with 1K of RAM expanded to 16K thanks to an expander. We didn't have disks then, only tapes. I remember my first floppy disk drive which stored 360K which seemed like a nearly unlimited amount; the computer I had at the time only had 64K of RAM, and only about half of that was usable for code, so you could write ten programs that filled every single bit of available space and save all of them on one disk with room to spare! My first hard drive ever was 20M and it was hard not to be astonished by it being over 50 floppy disks in size. I remember hearing about a coworker getting a 120M hard drive at the time and being amazed at the idea that it would take a hundred of the 1.44M floppy disks we had at the time to back it up, and wondering what you could fill such a thing with.

Not to sound too naïve, even back when I had my first 1K computer I could see how someday we'd want to store not just text but maybe pictures. When I had 360K floppies I knew someday we'd want to store music. When I had a 20M hard drive I could see how we'd want to have data-heavy programs like GPS that would take gigs of space. When I had a 1G drive I could see how we'd want to store video. At each step I thought how great it would be to have a data library that had hundreds, or thousands, of items.

It's easy to start imagining that every step along the way, the multiplying ten-fold of storage space that fits into the same volume and price is inevitable and will continue indefinitely, and that we will always find more needs for all that space. And it's not hard to project it into the future. Right now I can have a bunch of video on a hard drive, but I can't just have all the video I'd like; even with heavy compression, video takes up too much space. It'll be a long time before I can be as blasé about video as I am now with text files (and isn't it amazing that not that long ago we had to be as cautious about accumulating text files as we do now with video?). When I have terabytes, I can have a huge video library of stuff available at the tip of my finger. When my computers have petabytes, I'll have terabytes on devices in my pocket. And when my computer has exabytes, I'll be able to keep Wikipedia on my cell phone.

But maybe there's an end. If you'd asked me in 1988 about what kinds of files took up lots of space, right then I could have listed the text/database/picture/music/video hierarchy where each step is ten times bigger than the previous. But after video, what's next? When you can fit all the video you might ever want to watch on a chip the size of your fingertip, what will you store next? We can go a couple more orders of magnitude just undoing all our lossy compression and going back to native formats. We can go a few more with having huger archives of everything in the universe in one spot. But "more" can only take us so far before we need "bigger", the way video is bigger than music, to have our demand keep up with supply. And while, to this point in history, there's always been a "bigger" over the horizon, right now there isn't really a bigger waiting in the wings after video.

We could probably imagine one or two more iterations of "bigger" by talking about having interactive three-dimensional pre-rendered worlds, for instance. But I think we may, within the next couple of decades, see the point where more disk space stops being really useful. Where the endless ten-folding finally lets up just because the demand isn't there, and maybe storage manufacturing will turn to other factors. Speed has always been a priority because bigger size needs bigger speed just to break even, but what about speed without size increases, so it's real speed? How about more reliability -- pervasive redundancy, integral backup, more internal integrity checking, ubiquitous encryption at the hardware level? And there's always size.

Or maybe there'll be another five generations of ten-times-bigger kinds of data that haven't occurred to me. Maybe by then, we'll be able to use that space to record a human mind, or backups of your DNA, or all the preprocessing required to implant the Library of Congress in your brain in such a way that you can pull it up as fast as your own memories, or store a complete simulation of every particle in a solar system. But if disk sizes increase ten-fold every five years, there will be a point in my lifetime when my cell phone can store the position and momentum of every particle in the universe (except of course Heisenberg says it can't have both, but you know what I mean), so there's got to be an end.