tag:blogger.com,1999:blog-8614766039092762057.post1130784329641309411..comments2022-11-29T22:35:13.098-08:00Comments on Sergei Tsarev: MongoDB vs. Clustrix Comparison: Part 1 -- PerformanceSergeihttp://www.blogger.com/profile/12505242855655371767noreply@blogger.comBlogger27125tag:blogger.com,1999:blog-8614766039092762057.post-21531075245331783712011-04-23T08:42:42.316-07:002011-04-23T08:42:42.316-07:00To quote you: "Once the database size exceede...To quote you: "Once the database size exceeded that node's available memory, everything went to shit."<br /><br />This is a limit that *most* NoSQL solutions have today. They work great until data fits in memory.<br /><br />-Darpan<br />http://darpanetwork.blogspot.com/Darpan Dinkerhttps://www.blogger.com/profile/11632107912253415119noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-36003817507281369802011-03-13T12:28:57.618-07:002011-03-13T12:28:57.618-07:00I think it was actually a good comparison. Mongo c...I think it was actually a good comparison. Mongo collapsed under certain situations. This is something the mongo team should address.<br />I wonder how cassandra compares.<br />I have no idea what clustrix is, but I'm sure it's a nice sqlproxy with some memcached in between :-)<br />My motivations for using "nosql (hate the word)" is because of the flexibility of the datamodel / containment, and the new "join-models", and the built-in map-reduce<br />Also, I've always thought it's weird to write sql for most cases. ORMs are here for a reason.<br /><br />What I would also like to know is what happens on all three platforms (mongo, cassandra, clustrix) when something goes wrong (master or slave dies, a disk failure happens, etc).<br /><br />Just read the clustrix site.. My assumptions were correct. I think it's a viable product, which reduces complexity you see often when trying to scale mysql (replication, sharding, and eventually moving to memcached)Jorishttps://www.blogger.com/profile/05395441592036173495noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-63512259256667996952011-02-14T07:09:41.981-08:002011-02-14T07:09:41.981-08:00Hi Sergei
I checked out your test harness and ra...Hi Sergei <br /><br />I checked out your test harness and ran the tests myself in my lab. This is what I found:<br /><br />You have just a single shard per host on the mongodb side. It seems unlikely that this single process will be able to make the best possible use of all that iron.<br /><br />I don't have the budget you got, so I ran on:<br /><br />3x DC Xeon 2.7Ghz 2GB RAM 1x40GB scsi hdd 1x 1GbE<br />Ubuntu 10.10 32bit<br />MongoDB 1.6.5 GA, same as you<br /><br />One of my systems is running Ganglia and is running the C++ test harness, in addition to MongoDB.<br /><br />I ran your tests with two shards per system (one for each core :D) for around 8 hours.<br /><br />I got a steady 2000 reads/sec and 400 writes/sec per shard. Aggregated, that's 12000 reads/sec and 2400 writes/sec on my $200 setup.<br /><br />I am quite happy with that, even if it is artificial workloads.<br /><br />I wonder what the Mongod performance profile would look like if you set up you 8-way SSD systems with eight shards per host instead of one?<br /><br />TIAUnknownhttps://www.blogger.com/profile/04739457849948317787noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-53676636503519431182011-02-12T13:32:33.139-08:002011-02-12T13:32:33.139-08:00Sergei, I'm actually in the market for a midra...Sergei, I'm actually in the market for a midrange MPP DBMS system and MySQL language compliance is appealing. <br /><br />I'd love to get something for free; but I haven't yet. IBM DB2 has the PureScale option for scale out OLTP workloads and Netezza is actually under the ownership of IBM since the last two months.<br /><br />I have actually had to deal with maxed out 50 node RAC infrastructure in the past, so you're preaching to the converted, thanks.<br /><br />Happy to take the discussion offline - convince me.Unknownhttps://www.blogger.com/profile/04739457849948317787noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-69161409493656877542011-02-12T13:16:22.551-08:002011-02-12T13:16:22.551-08:00Rob, if you want something for free, then Clustrix...Rob, if you want something for free, then Clustrix is not for you.<br /><br />IBM DB2 MPP is a data warehousing solution for analytics -- not OLTP. It competes with the likes of Teradata and Netezza.<br /><br />Oracle RAC costs at least 10x more than Clustrix, and its shared disk architecture prevents them from scaling out. Clustrix wins at scale. You don't have take my word for it. Go talk to folks who have actually tried to scale RAC for OLTP.<br /><br />And 10gen (maker of mongo) raised over $11M so far.Sergeihttps://www.blogger.com/profile/12505242855655371767noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-36033117108466163742011-02-12T12:39:39.801-08:002011-02-12T12:39:39.801-08:00@Sergei, That's great that you found some bugs...@Sergei, That's great that you found some bugs in Mongod and Clustrix sounds fantastic, but I don't see any published pricing for your setup and it seems to be a closed source, proprietary appliance. <br /><br />I don't have a big budget (actually I don't want to spend anything on software unless I get into trouble) and I want to run on commodity, general purpose boxes, with the option of cloud bursting out onto EC2 if it becomes necessary.<br /><br />Could you explain to me how your Clustrix product will make it possible for ME to run an MPP OLTP db? <br /><br />With respect, Clustrix looks like a system that can't pick on IBM DB2 MPP or Oracle 11g RAC so its beating up the little guys instead.Unknownhttps://www.blogger.com/profile/04739457849948317787noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-38403636407220572582011-02-01T15:07:01.938-08:002011-02-01T15:07:01.938-08:00The performance and scalability question is certai...The performance and scalability question is certainly interesting and I don't think this comparison is trying to say either way of approaching the probably is fundamentally running into scalability issues (obviously MongoDB has been doing work and has more work to do to remove performance bottlenecks). And setting aside the question of free vs. paid products, which is probably the major reason why MySQL et al. are so much more dominant than Oracle/DB2/SQL Server in the web space. It seems like a decision between a traditional RDBMS (like MySQL or Clustrix), a document store (like MongoDB or CouchDB) or a key value store (like Redis and the Memcache DB types) depends on your data set and usage characteristics.<br /><br />One thing the document stores provide are schema-less data stores. Depending on your data set, sometimes this sucks (eg. no typing of your field types and thus your ID field sometimes being a string and sometimes being a number because your client didn't cast the field appropriately; and having to waste space storing your field names in the document), but sometimes this is great (eg. adding a new field to your user object doesn't require altering your table structure which involves sync'ing code and DB ops; plus performance issues when expanding your table; plus having an unwieldy schema when you have limitless different types of fields).<br /><br />If you need to do joins, obviously key value stores and document stores suck, unless their performance is so great that multiple lookups is fast enough. However most web applications try to stay away from joins because it's easier to scale up the number of client machines to scale up the database machines..<br /><br />If you're using your data store to store transactional data, then obviously you're going to want a RDBMS. But again, a number of usage scenarios for web applications are not going to care about transactions or ACID compliance..<br /><br />I think the proliferation of NoSQL is really just an extension of the database movement (which even SQL database vendors like VoltDB are following) that making specialized data store systems for specialized purposes can open up a realm of undiscovered possibilities and applications..Ganonhttps://www.blogger.com/profile/09050837431358679069noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-31033268685058256352011-02-01T14:02:16.966-08:002011-02-01T14:02:16.966-08:00I look forward to the next post. Thanks for provid...I look forward to the next post. Thanks for providing so many details.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-54484402512711923832011-02-01T11:51:47.489-08:002011-02-01T11:51:47.489-08:00@robo: the system scaled to 10 nodes with a query ...@robo: the system scaled to 10 nodes with a query throughput rate that's higher than you can get on any single node system. That's scalability. Furthermore, we performed better than Mongo in many aspects of the test. That's performance.<br /><br />This is part 1 of a series of posts that focused on performance. I'm going to address the other aspects. Stay tuned.Sergeihttps://www.blogger.com/profile/12505242855655371767noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-65190155330137907822011-02-01T11:47:20.447-08:002011-02-01T11:47:20.447-08:00So how does this test answer the features for any ...So how does this test answer the features for any modern system?<br /><br />"Over and over, the following has always come up a list of must have features for any modern system:<br /><br />Incrementally Scalable Performance<br />High Availability and Fault Tolerance<br />Ease of Overall System Management<br />"robohttps://www.blogger.com/profile/02369895252691903216noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-72419989496781717972011-02-01T11:19:18.019-08:002011-02-01T11:19:18.019-08:00Thanks for such a Benchmark.
Despite I'm a Mon...Thanks for such a Benchmark.<br />Despite I'm a MongoDB fan, I have to admit I meet sometimes your issues, mostly about R/W.<br /><br />BUT, I'm wondering something : did you preallocate your files ? That's really matter when performing these tests...<br /><br />Thanks for answering :)Anonymoushttps://www.blogger.com/profile/13887310572024815894noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-43033438249416865752011-01-31T16:57:31.618-08:002011-01-31T16:57:31.618-08:00Your whole premise misses the point. mostly, see J...Your whole premise misses the point. mostly, see Jared's complaint above, but I'll elaborate.<br /><br />No one with a brain thinks SQL, at its core, is a bad idea. However, one can easily shoot themselves in the foot. The promise of NoSQL is that you can scale your cluster without having a database admin of 10 years experience and multiple certifications around to audit your developers' poor schema choices and badly written queries. You can put a limited SQL language in front of such a system if you want, but why? No one wants to learn a separate language to query their database anyway. None of this means all the great ideas of the last 25 years should be thrown out. NoSQL developers should still be writing good concurrent code.<br /><br />Your results are still interesting. Probably most interesting to the MongoDB team. But you shouldn't pretend for a second this has anything to do with NoSQL vs SQL. In fact, realize that what you're doing here is adding fuel to the fire and stoking confusion. You, sir, are contributing to the misconception that there's a rabid, anti-SQL camp. There isn't. Stop, please.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-28474558853707022692011-01-31T14:27:31.842-08:002011-01-31T14:27:31.842-08:00This is not a fair comparison. First of all, you ...This is not a fair comparison. First of all, you are the founder of Clustrix. Second, from what you write, you were clearly learning how MongoDB works on the fly, which leads me to believe that you have not optimized MongoDB for your particular situation. Also, you did not bother to utilize any relational features of Clustrix vs the document oriented structure of MongoDB. Since the initial data import is one of your major concerns, did you bother to compare the mongoimport tool to the Clustrix initial data import?Danhttps://www.blogger.com/profile/10743133809889248457noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-44398418606236339182011-01-31T12:34:27.018-08:002011-01-31T12:34:27.018-08:00Wouldn't you need to include at least one join...Wouldn't you need to include at least one join to have a reasonable comparision? If you choose MongoDB, then you have a document oriented data model. I'm thinking of situations like line-items on an invoice. They are totally owned by the invoice. When the invoice goes away, the line items go away. Using a traditional DB, you'd have two tables and use a join in your queries. In MongoDB, you'd include the invoice line items in the invoice document. <br /><br />To be fair, you would NOT store the customer information in the invoice document even if you were using MongoDB. It would take two round trips to get it in MongoDB but you could get it in one with a join using SQL.Anonymoushttps://www.blogger.com/profile/14086901997903372470noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-21752898366605590522011-01-31T12:07:43.524-08:002011-01-31T12:07:43.524-08:00Sergei,
1.) Yes, I did miss that you said that t...Sergei, <br /><br />1.) Yes, I did miss that you said that things balanced after the data load, but you had to help. Did you (want to) test with pre-splitting, before you did the initial data load?<br /><br />2.) That might be true, but initial data loading is a small part of the data lifetime, in most systems.<br />3.) You can specify a shard key as more than one field, just like an index can be compound.Scotthttps://www.blogger.com/profile/15950961019628738714noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-69297907076309305562011-01-31T11:37:35.338-08:002011-01-31T11:37:35.338-08:00How can one come to the conclusion, "The SQL ...How can one come to the conclusion, "The SQL relational model can clearly scale" when the presented experiment doesn't exercise any relational features of the Clustrix database?Unknownhttps://www.blogger.com/profile/08418874903890679413noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-19191441864439294242011-01-31T10:49:55.948-08:002011-01-31T10:49:55.948-08:00Scott,
I think if you reread the article, then yo...Scott,<br /><br />I think if you reread the article, then you will see that:<br /><br />1. I did manually pre-split the shard space because if I didn't do that, mongo would not actually rebalance the data under heavy load.<br /><br />2. The very first interaction that our customers have with clustrix is the initial data import. We've spent a lot of time optimizing for that case because while it's a rare workload, it's extremely important.<br /><br />3. I partitioned the space by user_id because you can see how that would be the dominant constraint on queries. Mongo allows only one shard key. If I split by say server_id (which would help the second query), then the user_id query would become a broadcast and performance would suffer.Sergeihttps://www.blogger.com/profile/12505242855655371767noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-82335390157085860572011-01-31T10:41:57.279-08:002011-01-31T10:41:57.279-08:00I would suggest contacting MongoDB and telling the...I would suggest contacting MongoDB and telling them about the benchmarks so they can review your configuration to make sure it was properly setup. I can guarantee your Clustrix configuration is setup properly since you are an original founder. Pehraps they can shed some light on a 775% write margin.Unknownhttps://www.blogger.com/profile/17834552172169007608noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-11681284532407358502011-01-31T10:19:30.446-08:002011-01-31T10:19:30.446-08:00This comment has been removed by the author.Scotthttps://www.blogger.com/profile/15950961019628738714noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-76242559888945992452011-01-31T10:08:26.674-08:002011-01-31T10:08:26.674-08:00This comment has been removed by the author.Scotthttps://www.blogger.com/profile/15950961019628738714noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-27814227408744312312011-01-31T10:04:09.634-08:002011-01-31T10:04:09.634-08:00Ok, I updated the post with benchmark source.Ok, I updated the post with benchmark source.Sergeihttps://www.blogger.com/profile/12505242855655371767noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-33686805905651894732011-01-31T08:19:19.428-08:002011-01-31T08:19:19.428-08:00Also, what version of MongoDB are you running? Th...Also, what version of MongoDB are you running? They are making huge improvements all the time.<br /><br />Sorry about minute vs second.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-37011563309258075852011-01-31T08:18:09.388-08:002011-01-31T08:18:09.388-08:00This comment has been removed by the author.Unknownhttps://www.blogger.com/profile/16960344012837618777noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-24727592007137533762011-01-31T08:14:44.970-08:002011-01-31T08:14:44.970-08:00I'm going to post the benchmarks later today.
...I'm going to post the benchmarks later today.<br /><br />Sammy, it's 37 minutes, not seconds.Sergeihttps://www.blogger.com/profile/12505242855655371767noreply@blogger.comtag:blogger.com,1999:blog-8614766039092762057.post-26976000752947157612011-01-31T08:13:20.551-08:002011-01-31T08:13:20.551-08:00Your math doesn't add up.
Using the clustrix ...Your math doesn't add up.<br /><br />Using the clustrix numbers, at 140k insets/sec for 37 seconds you get 5.1 M records. Even if that is per server, you get 51M records.<br /><br />Pleas explain why you cannot add?Anonymousnoreply@blogger.com