Thursday, November 30, 2006

Google File System

The Google File System is a scalable distributed file system for large distributed data-intensive applications, being used within Google. The paper, published in the 19th ACM Symposium on Operating Systems Principles, is a well written computer engineering paper. System builders should read it.

Quoted from its conclusion:

"We started by reexamining traditional file system assumptions in light of our current and anticipated application workloads and technological environment. Our observations have led to radically different points in the design space. We treat component failures as the norm rather than the exception, optimize for huge files that are mostly appended to (perhaps concurrently) and then read (usually sequentially), and both extend and relax the standard file system interface to improve the overall system."

Lesson learnt:
  • Application driven.
  • Base on the technological environment. Base on the facts of performance characteristics. Performance benchmark is the foundation for system design.
  • Application/file system (or OS, or underlying infrastructure, etc.) co-design.
  • Optimize for what should be optimized.
"Our system provides fault tolerance ..."

Lesson learnt:
  • Reliability is a must for any system. System must work. Never forget reliability, persistence, fault tolerance during system design.
"... This makes possible a simple, centralized master that does not become a bottleneck. ..."

Lesson learnt:
  • Down to earth.
  • ...I had already come to the conclusion that in the practise of computing, where we have so much latitude for making a mess of it, mathematical elegance is not a dispensible luxury, but a matter of life and death. - Edsger Wybe Dijkstra ["My hopes of computing science" (EWD 709)]
elegant: [...]ingeniously simple and effective. - Concise Oxford Dictionary

This elegance makes perfect sense in the domain of system design. May we say in system design, elegance is not a dispensible luxury, but a matter of life and death.

1 comment:

Kanye Co Jamila said...

Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Java developer learn from Java Training in Chennai. or learn thru Dot Net Training in Chennai . Nowadays Java has tons of job opportunities on various vertical industry.

or Javascript Training in Chennai. Nowadays JavaScript has tons of job opportunities on various vertical industry.