Harish Mallipeddi

Nov
20th

Fri

Riak’s dets backend - too many files open error

I was playing with Riak yesterday and ran into this error on my Macbook running Snow Leopard:


{
 {badmatch,{error,{{badmatch,{error,
 {file_error,"./dets-store/1392993748081016843912887106182707253109560705024",emfile}}},
                    [   {riak_vnode,init,1},
                        {gen_server2,init_it,6},
                        {proc_lib,init_p_do_apply,3}    ]}}
 },
 [{riak_vnode_master,get_vnode,2},
  {riak_vnode_master,handle_cast,2},
  {gen_server,handle_msg,5},
  {proc_lib,init_p_do_apply,3}]
}

When using the dets storage backend, Riak seems to create/open a dets database file for each vnode (partition) in your ring. When there’s only one node in your cluster, I’m guessing all the vnodes/partitions are owned by this node which results in a whole bunch of files being opened (under the dets-store folder you configured). In Snow Leopard, I’d to do ulimit -n 8192 to increase the limit on the no. of fds a process could take. You probably won’t notice this normally - I increased the partition size in the config file and hence ran into this problem.

Comments

Nov
3rd

Tue

When Linux Runs Out of Memory - O'Reilly Media

Good article explaining Linux’s default overcommit-ing behavior (/proc/sys/vm/overcommit_memory) and the OOM killer.

Comments

Oct
30th

Fri

Hadoop at Yahoo!

Slides from Eric14’s talk at UIUC.

Comments

Oct
22nd

Thu

GitHub's new architecture

Very well written article highlighting the new architecture of GitHub (post-Rackspace move). Every company/team should at least have a similar internal document describing their current architecture at a high-level but unfortunately doesn’t happen in most companies.

GitHub seems to be using a lot of familiar Opensource projects in their stack - Nginx, Ruby/Rails/Unicorn, HAProxy, MySQL, memcached, redis, bunch of RPC services speaking the new BERT/BERT-RPC protocol. The only thing I’ve not really played with yet is DRBD and I’ve heard lots of good things about DRBD from others.

So looks like now you’ve Facebook’s Thrift, Google’s Protocol Buffers, Hadoop/Doug Cutting’s Avro, just plain JSON/XML and GitHub’s BERT for your (de)serialization needs!

Comments

Oct
20th

Tue

Speaking at FOSS.MY 2009

I’ll be speaking about Hadoop at FOSS.MY 2009 in KL. They’ve an interesting bunch of speakers this year including Brain Aker, David Axmark, RMS. This will be my first time to FOSS.MY!

If you’re attending, do drop by my talk on Sunday afternoon :)

Comments

Log-analysis software

There were a couple of interesting articles on DBMS2 on log analysis software vendors, Splunk and SenSage. I’m amazed that both companies have around 150+ employees and are profitable. There’s also a good summary of who their main customers are and how they’re using their products.

Comments

Sep
15th

Tue

Kickfire's SQL chip

Daniel Abadi blogs about how Kickfire achieves parallelism in their FPGA-based SQL chip. Also watch Kickfire CEO, Raj Cherabuddi’s talk at MySQLConf 2008.

Btw if you’re interested in next-gen DBMSes, subscribing to Daniel Abadi is highly recommended; he’s a member of the Yale DB group which released HadoopDB.

Comments

Sep
14th

Mon

Effective A/B Testing

In-depth presentation on how to do A/B testing right.

Comments

MySQL and the Linux swap problem

Even though your box might have lot of physical memory left, Linux would prefer to swap out your process so it can allocate more space for the filesystem cache. Read Smugmug and Kevin Burton’s blog posts and the comments.

Comments

Aug
31st

Mon

In-depth look at Clojure's STM implementation

Insightful article. I read most of the Clojure book last weekend and it has been lot of fun. This will be my first proper attempt at learning a Lisp (I ditched CommonLISP for Python in the AI class back in school). I’m using Clojure to write simple Hadoop jobs (turns out Hadoop is the only use-case currently where I rely on the JVM runtime).

Comments

Aug
30th

Sun

Home is where the brain is | San Francisco online

Good article on the status of Indian immigrants in the US (specifically those in Silicon Valley).

Comments

Aug
27th

Thu

Amazon launches Virtual Private Cloud (VPC)

On the 3rd anniversary of EC2, AWS launches VPC. The pace at which AWS seems to launch new features is very impressive (the other cloud providers are severely lagging). Anyone has any idea how many people work for the AWS division at Amazon?

Comments

Aug
25th

Tue

Google AppEngine: scaling across multiple datacenters | High Scalability

Todd Hoff of HighScalability does a good job of summarizing a talk by Google engineer Ryan Barrett from Google I/O 2009.

Comments

Aug
19th

Wed

LZO compression for Hadoop-0.20+

Starting from Hadoop-0.20 onwards, any code related to LZO compression has been removed from the Hadoop source tree. This is because the LZO code is licensed under GPL and hence incompatible with Hadoop’s Apache license. One more thing you should know is LZO compression is only supported via a native library (AFAIK there’s no pure Java implementation of it). LzoCodec and LzopCodec are almost the same (LzopCodec is compatible with the output from the lzop unix utility).

Here are the steps to get LzopCodec working with Hadoop-0.20 (see the gist embed below). I’m assuming you’ve already downloaded and installed the Hadoop-0.20 release tarball. We’ll be adding the compiled library to Hadoop-0.20’s lib/ folder. Repackage it into a tarball and push it to your cluster using whatever magic you use and you should have LZO compression working.

Comments

Aug
5th

Wed

Cool hacks used to build SUSE Studio

Nat blogged about some really cool hacks they’d to resort to build a challenging product like SUSE Studio.

The hack to get a list of all the files changed in the filesystem during the “testdrive” is particularly interesting - they used QEMU’s COW feature which essentially produces a small delta image if you do any writes in your VM while keeping the original image pristine (aka copy-on-write). They then use libext2fs to just parse through this delta image to get a list of all the files which were modified and display them in a web-page.

Comments