YouTube Architecture - High Scalability -

3. Use commodity hardware. 10. For high CPU intensive activities like encryption, they use C extensions. Use MySQL to store meta data like users, tags, and descriptions. 4. Use simple common tools. They went through a common evolution: single server, went to a single master with multiple read slaves, then partitioned the database, youtube shorts and then settled on a sharding approach. It's true that nobody really knows what simplicity is, but if you aren't afraid to make changes then that's a good sign simplicity is happening. Simplicity allows you to rearchitect more quickly so you can respond to problems. 7. Python allows rapid flexible development and deployment. 6. The Python web code is usually NOT the bottleneck, it spends most of its time blocked on RPCs. 1. Stall for time. It was time for wireless networks to evolve once again, video shorts this time to provide speedier mobile broadband service. The fastest cache is in your application server and it doesn't take much time to send precalculated data to all your servers. 3. Requests are routed for handling by a Python application server. 4. Application server talks to various databases and other informations sources to get all the data and formats the html page.  Post has  been cre​at ed with the he​lp of　G SA C ontent Gen᠎er at or  Demov ersion!

7. For images latency matters, especially when you have 60 images on a page. 8. Usually less than 100 ms page service times. This loop runs many times a day. The master is multi-threaded and runs on a large machine so it can handle a lot of work. 5. Handle random seeks well (SATA, tweaks). When they needed more hardware to handle load it took a few days to order and get delivered. 5. Can usually scale web tier by adding more machines. 2. Managed hosting can't scale with you. Can now scale database almost arbitrarily. 12. Row level caching in the database. Went to database partitioning. 3. So they went to a colocation arrangement. Slaves are single threaded and usually run on lesser machines and replication is asynchronous, so the slaves can lag significantly behind the master. Suffered from replica lag. Can come from any colo. 5. Videos come out of any data center. 1. Supports the delivery of over 100 million videos per day. YouTube grew incredibly fast, to over 100 million video views per day, with only a handful of people responsible for scaling the site. Upload, edit, watch, search, and comment on video from your own site without visiting YouTube.

Compose your site internally from APIs because you'll need to expose them later anyway. YouTube adds a new rich set of APIs in order to become your video platform leader--all for free. People who can set up printers, machines, install networks, and so on. The idea is that people want to watch video so that function should get the most resources. 2. Prioritize. Know what's essential to your service and prioritize your resources and efforts around those priorities. Don't be afraid to outsource some essential services. 1. NetScalar is used for load balancing and caching static content. Routers, switches, and other appliances may not be able to keep up with so much load. Much better cache locality which means less IO. Creating their own network would have taken too long and cost too much. 2. Keep a simple network path. 4. Keep it simple! 1. Keep it simple and cheap. 11. Some pre-generated cached HTML for expensive to render blocks.

13. Fully formed Python objects are cached. 14. Some data are calculated and sent to each application so the values are cached in local memory. You are also less likely find help on the net. Creative and risky tricks can help you cope in the short term while you work out longer term solutions. One of their solutions was prioritize traffic by splitting the data into two clusters: a video watch pool and a general cluster. Served data off a monolithic RAID 10 Volume with 10 disks. 8. Images are replicated to different data centers using BigTable. 4. Use 5 or 6 data centers plus the CDN. They use most tools build into Linux and layer on top of those. The social networking features of YouTube are less important so they can be routed to a less capable cluster. With a good team all things are possible. 7. You succeed as a team.

Location:

Share :