I was recently in a quandary over the choice of technology. I started RoR and I really like it. However I was concerned about long term implications of that choice. The thing that I am taking from this talk is that I shouldn't worry about that, right now. If and when I need to scale, I will have enough resources to make a better choice. Resources that I don't have right now.
Well, correct me if I'm wring but Shopfiy is a completely different scaling problem from Twitter. As I understand it Shopify's individual hosted stores are pretty much self-contained. So you can pretty much stick each one on it's own server with it's own database and it'll be fine. Twitter accounts all have to be able to talk to eachother in real time so you can't do that.
My current startup has a Shopify-like architecture which is what I'm counting on to help me if I ever need to scale fast.
So I think the first question you have to ask yourself when considering scaling is: what is my architecture like?
> So you can pretty much stick each one on it's own server with it's own database and it'll be fine.
That is not a scalable solution. Yes, it'll get you up and running out of the box, but as you keep spinning up servers to host each store and it's database you'll have to keep adding exponential resources (hardware, software, meatware) to the problem and keep you from achieving economies of scale.
>Yes, it'll get you up and running out of the box, but as you keep spinning up servers to host each store and it's database you'll have to keep adding exponential resources (hardware, software, meatware) to the problem and keep you from achieving economies of scale.
No, you'd be adding resources at a _linear_ rate relative to the growth of the customer base. The point about economies of scale is true enough but has nothing to do with a lack of exponential growth in costs.
Yes, its not an ideal solution for the long term, but its something easy you can do to scale quickly if you have this architecture. It'll make sure your site stays up.
Afterwards, you can spend the time doing it properly when things have calmed down a little.
That's a lot easier than scaling something like Twitter.
No, sites go down all the time and need to rebooted, reconfigured, redeployed, strangled, etc. by a human who has to watch the servers 24 hours a day. I'm ignorant to the number of customers that Shopify is hosting these stores for, but let's say for example that one human can monitor 100 virtual machines during an eight-hour shift and you have 500 vm's running at the same time over a series of physical servers. That means, in HR terms, you'll need five people per shift or fifteen per day to monitor and act upon the vm's, along with at least two operations people, one of whom will be carrying the pager for the entire 24-hour period. That's seventeen people needed to run your operation not including sick/vacation/leave time that needs to be covered. In salary costs alone, that's over $1million/yr., not including recruiting and management salary, benefits and compensation. And does not take into effect any development, real estate, office essentials, hardware and software costs.
> Afterwards, you can spend the time doing it properly when things have calmed down a little.
That'll cost you an extra million dollars to develop and deploy while simultaneously running your existing operations and migrating your clients over to the new solution.
I doubt that every single customer has to be on their own VM. It's more likely that many people share the same VM, and only larger customers need their own VM.
Let's run some numbers though, using your assumptions:
500 VMs @ $179 per customer (business class licence) means a little over $1,000,000. At that point, you'd be right---it wouldn't scale.
However, this is a business and not an HR exercise. So you could easily hire less people, and have less reliability. For a business like Shopify, I imagine their best customers are those who pay the $99 plan, accept a transaction cost, and other limitations. These customers can be piled onto the same VM together, and probably don't have such a huge throughput that a few minutes of downtime means money lost.
> you could easily hire less people, and have less reliability.
If your business has to be up and running 100% of the time, those are the minimal numbers needed to run the operation I've described. I cite a real word example, with real people and real money, the afterwards part if you will, which I'm currently working on to replace, not an hr exercise. And yes, it doesn't scale and it's very costly to replace, and one of the main reasons it was built in such a manner is because the original architect never thought it would have to scale and there are licensing considerations, something rarely mentioned around these parts. Licensing can really fuck things up.
I'm actually really glad to hear it. Though, I wonder about the "rails doesn't scale" mantra, is that really more for active record? In your experience is active record the biggest out of the box bottleneck?
The "rails doesn't scale" mantra was discredited 5 years ago, when people realised that it scales exactly the same way as PHP. Remember, scalability != performance.
We handle a lot less traffic than Shopify (~15-20 million reqs a day) but ActiveRecord isn't a bottleneck and I wouldn't expect to to be [...unless you are talking about something other than the performance of AR as a body of code?]
Instead of "Rails doesn't scale" we should say "Rails runs on Ruby which means that it will consume significantly more CPU and more memory* compared to something else"
In my case, 1 extra server (my estimate) was a small price to
pay for developer happiness.
The issue they have doesn't seem to be scaling, in that RoR is scaling linearly. But if you have hundreds of machines, raw performance saves real money, which means RoR is maybe not ideal for massive deployments and the JVM languages give you more performance on the same hardware. Though unless you're working on a top 50 site I wouldn't worry so much.
From what I read, "rails doesn't scale" is really a misnomer, because there are number of things that can be done to scale rails. I think Haiping Zhao of Facebook made it clear, that it's really just an efficiency problem. Bottom line, rails requires more computing power than jvm for sites like twitter. Which are only a handle of sites.
That seems to be the case. The view/controller layers can be scaled via more machines. Its the database layer that requires synchronization. Thats a universal problem, though.
I'm working at a company right now that have a very big Rails stack. Hundreds of models, probably thousands of files (not including plugins and gems) across the whole app. They are experiencing scaling issues but not for requests per second, but for developer productivity.
I'm reading a great book called Service-Oriented Design with Ruby and Rails, by Paul Dix. He states a lot of the issues that our team is running into with a monolithic app. Seems also that these issues can apply to other frameworks outside of Rails (I've run into some of these same issues years ago with Java web apps).
The main point of the book is figuring out how to abstract out various layers of the app into standalone services. I've talked to people about this and a lot of the time they recoil at the fact that you would increase the complexity of the app by adding yet more parts (services) that require separate machines, deployment dependencies, and their own data stores. However, looking at big sites like Amazon or Twitter shows how one needs and can break apart their systems to make it easier for developers to focus on particular pieces as well as increase system performance.
I'm going to be joining a new startup this year as the technology lead and all these things are on the table. I'm evaluating node, some NoSQL dbs, some cloud paas', etc. I'm also going to be designing from the beginning fast fail and services - or at least an architecture that makes it fairly easy to start migrating parts of the system over to their own standalone services should the need arise. I don't think you can plan for everything, but with the way technology is moving forward and the tools available today, you need to keep SOA at least in the back of your mind.
Like Matz, creator of Ruby, said [1] - and if I didn't paraphrase I think it would lose part of its charm - :
"If you can make up a website that has a higher traffic than Twitter ... it's a great success of business ... so you have money ... so you're safe to hire the Java programmer to replace it. [laughs]"
Excellent. This is exactly what all developers need to realize about technology choices. Make decisions based on what will help you to deliver value quickly now, you won't really know what needs, or how it needs, to scale untill much later anyways.
When that day arrived, the next question will be whether you have the guts to do it. How many companies dare to rewrite (even with the assumption you can do it peacemeal style)?
True that. Even LOLCODE can scale :) Twitter's problems are very specific to twitter - Mega throughput (~7000 Tweets per second) in realtime, sharded DBs requiring multiple connections etc.
Ruby will give you a significant time to market advantage especially if you are a startup.
My understanding is that the particular scaling problem of twitter is high fanout through subscriptions. Receiving 7k messages per second and storing them in a database is actually fairly straightforward.
