Building Scalable Web Sites - Cal Henderson

What This Book Is About This book is primarily about web application design: the design of software and hardware systems for web applications. We’ll be looking at application architecture, development practices, technologies, Unicode, and general infrastructural work. Perhaps as importantly, this book is about the development of web applications: the practice of building the hardware and implementing the software systems that we design. While the theory of application design is all well and good (and an essential part of the wh ole process), we need to recognize that the implementation plays a very important part in the construction of large applications and needs to be borne in mind during the design process. If we’re designing things that we can’t build, then we can’t know if we’re designing the right thing. This book is not about programming. At least, not really. Rather than talking about snippets of code, function names, and so forth, we’ll be looking at generalized techniques and approaches for building web applications. While the book does contain some snippets of example code, they are just that: examples. Most of the code examples in this book can be used only in the context of a larger application or infrastructure. A lot of what we’ll be looking at relates to designing application architectures and building application infrastructures. In the field of web applications, infrastructures tend to mean a combination of hardware platform, software platform, and maintenance and development practices. We’ll consider how all of these fit together to build a seamless infrastructure for large-scale applications. The largest chapter in this book (Chapter 9) deals solely with scaling applications: architectural approaches to design for scalability as well as technologies and techniques that can be used to help scale existing systems. While we can hardly cover the whole field in a single chapter (we could barely cover the basics in an entire book), we’ve picked a couple of the most useful approaches for applications with common requirements. It should be noted, however, that this is hardly an exhaustive guide to scaling, and there’s plenty more to learn. For an introduction to the wider world of scalable infrastructures, you might want to pick up a copy of Performance by Design: Computer Capacity Planning by Example (Prentice Hall). Toward the end of the book (Chapters 10 and 11), we look at techniques for keeping web applications running with event monitoring and long-term statistical tracking for capacity planning. Monitoring and alerting are core skills for anyone looking to create an application and then manage it for any length of time. For applications with custom components, or even just many components, the task of designing and building the probes and monitors often falls to the application designers, since they should best know what needs to be tracked and what constitutes an alertable state. For every component of our system, we need to design some way to check that it’s both working and working correctly. In the last chapter, we’ll look at techniques for sharing data and allowing other applications to integrate with our own via data feeds and read/write APIs. While we’ll be looking at the design of component APIs throughout the book as we deal with different components in our application, the final chapter deals with ways to present those interfaces to the outside world in a safe and accessible manner. We’ll also look at the various standards that have evolved for data export and interaction and look at approaches for presenting them from our application.