-written by Jason Caulkins, Dataram Storage Blog Team Member

For the last 50+ years, computing has been out of balance. What I mean by that is that there are three fundamental pillars of computing: CPU, Memory, and Storage. You don’t have a very interesting computer without these fundamental bits. Yes, I know you need an IO system as well, but let’s call that a given. While the CPU and memory have enjoyed the benefits of Moore’s Law, the Storage Pillar has languished. It’s a slave to physics, due to the mechanical wonders of the rotating disk and moving head.

Thus, you have two skinny (fast) pillars and one fat (slow) one. This yields an imbalanced system. The software and systems folks have worked very hard to hide this imbalance to the user. They have, by and large, succeeded. However, in the era of MySpace, Facebook, Twitter, and Zynga, the fat pillar can no longer be ignored, like the elephant in the server room that it is. Nifty caching schemes have been developed on the server side to compensate for the relatively poor storage performance. These schemes are great, but are costly and just move the burden of storage performance into the server, which is an unfortunate price to pay for what should be a stand-alone aspect of the compute process (for now).

Long ago, someone figured out that you could use solid state core memory to store data long-term. They also figured out that in order for such a device to be useful, they had to wrap it in electronics to make it look like a disk interface, so as to be compatible with all the existing systems out there.

Now, fast forward a few decades and we have DRAM and flash technology far denser and cheaper than ever before. All that has to be done is to figure out what kind of electronics we should wrap it in so that it can be compatible with all the very numerous existing systems out there. Easy, right? Make it look like a disk, just like the old days. This works really well for a number of reasons. First, disks are modular, and designed to be replaced if required. This is handy for replacing failed units, expansion, and upgrading. This is fitting for the technology available today (flash). Soon, however, a new memory technology will become available and replace flash as well as DRAM. Once this occurs, there will no longer need to be a disk type interface at all. Long term and short term storage will be handled by the same technology.

This means that the storage technology will no longer need to be wrapped in drive (or other legacy bus) technology in order for it to be useful. It will all reside as directly accessible by the CPU (or bridge chip, depending on how technology flip-flops in the next 10 years). The other cool thing is that you will have a common storage/memory space. This means that the three pillars will effectively become two. There will probably remain a need for mechanical disks for archive, but online and even near-line storage will just occupy some of your main memory.

Which brings us back around to the concept of storage living in the PC or server. The exact thing that I said was unfortunate earlier. What will make all this manageable is software. Storage managed properly can be located anywhere in the connected world.

More on that later.