Amazon announced SDB late last week. I have been using it under NDA for a bit and have written a Ruby API for it. I have opened up a RubyForge project to deposit the code and will be doing so in the upcoming days.
There has been a lot of chatter on the internets about SDB over the last few days. Some of it correct. Some of it not. Some say it “takes on the relational database” while others say it “removes the database scaling wall” or it “lets you run some calculations on your data.” TechCrunch, of all places, says it “takes on Oracle and IBM” (/me chuckles). …….Plll-e-e-ease, people. You are missing it.
SDB is NOT a real-time or transaction db (that’s got to be 95% of the DBs out there). SDB doesn’t have transactions and has “eventual consistency”. This means if you query for a record that was updated, it may or may not give you back the updated version you just stored. Eventually, they promise to give back the updated version - they promise. SDB, today, would be a really poor choice for data store where a user interacts with a UI. ( IE: “What do you mean my order can’t be found?! I just paid you on the last screen with my credit card!” ) SDB doesn’t do joins (across domains) or ANSI SQL.
SDB is still a very, very useful tool to put in your belt for your EC2 cluster. Here’s the real-deal. Amazon is giving their users Google’s ‘BigTable’ for their EC2 cluster. This is a massive read-mostly hash database, a tuplespace, a directory, warehousing, reporting, search indices or metadata system that can be layered on S3. It fits well where the data is write-once-in-while but read-mostly.
If you really want an eye-opener on what Amazon intends this for then read up on Google’s MapReduce and BigTable. Then go back and read the docs for EC2 / SDB and your brain will :click => :bingo
Posted on December 17th, 2007 by dysinger
Filed under: @work
I have been following up on SimpleDB for a while now and just tried your aws-sdb gem (0.1.1), mostly trying to exercise the examples given by Amazon for other languages. While it was instantly gratifying to talk to SimpleDB from Ruby, I had limited luck with the query method. Everytime I used more than one comparision on either a single attribute or intersect/union on multiple, SimpleDB came back with i.e.
AwsSdb::InvalidQueryExpressionError: Query expression ‘['Year' = '2000' or 'Year = '2001']‘ is not valid.
Do you have any insights of what might go wrong here? You do not seem to process the query string in any way before handing it over to SDB, so that should not be a problem. Maybe it’s in the request signing? Just guessing…
Sorry, the above example contained a typo, a better non-working one would be:
['Make' = 'Audi'] intersection ['Model' = 'S4']
So only intersection/union seems to be problematic…
Cheers,
Martin
After some further investigation I found the cause of the problem with using intersection/union in SimpleDB queries. It’s an API change by Amazon- the API version used in the aws_sdb gem does not yet support the syntax! Doh!
Changing line 121 of service.rb (at the beginning of the call method) to
'Version' => (params['Action'] == ‘Query’ ? ‘2007-11-07′ : ‘2007-02-09′),
works around the problem by using the current API version for Query calls while leaving the older version in for all other calls, so it does not break anything else. (There is at least one API change breaking full backward compatibility: the replace option in put_attributes has to be specified per attribute instead of per request).
It would be cool if you could get this change into your released gem.
Cheers,
Martin
[...] As Lars Schenk outlines on his Blog at least three different projects on RubyForge are addressing problem number one - only one of them has actual code, though. And that is aws-sdb by Tim Dysinger from Hawaii. [...]
Hey Tom, any closer to posting a ruby library for SimpleDB?
The new aws-sdb 0.1.2 was released and it’s updated for Amazons 2007-11-07 API