Yahoo’s Open Data Tables and the YQL Console – An Overview

I’ve been looking into the Yahoo Query Language (YQL) specification available on the Yahoo Developer Network (YDN). The scope of the functionality exposed is quite impressive and I would certainly suggest checking this out to anyone serious about Web-based programming.

What is YQL?

Yahoo has allied itself with a number of websites and online services, and provides a uniform API to these using YQL. YQL is is a well-documented fourth generation programming language (4GL) with typed syntax that is very close to traditional SQL.

Through the YQL API Yahoo exposes a standardized way to query and manipulate data from various authenticated Web services including Netflix and Twitter. The method of exposing this information is through what they call Open Data Tables. These Open Data Tables are XML files that describe how YQL can be mapped to a Web Service provided by a third party system (such as Twitter for example).

YQL and Open Data Tables point to Yahoo’s cloud storage infrastructure called ‘Sherpa’, and Yahoo implements security features for YQL using OAuth 1.0 and Access-Control-Allow-Origins.

How Does YQL Work?

To get familiar with the YQL specification and the functionalities offered by third parties (or even by Yahoo itself) as Open Data Tables one can use Yahoo’s YQL Console. I am including a screen capture of the YQL Console below.

I really like the YQL Console’s interface and ease of use.

You can prototype your YQL statement in the topmost text-area and see the XML or JSON formatted results in the text area below. Furthermore it is easy to explore the available Open Data Tables using the scrolling list at the bottom right hand side of the YQL Console. When you click on an Open Data Table that you are interested in querying, then a meta-query is created for you in the YQL statement text area, and you are provided with the API XML/JSON in the results Test area.

All-in-all I like using this system and find it very intuitive, and most importantly self-documenting. I can certainly see myself incorporating functionality from the Yahoo Open Data Tables in my future development work.

Yahoo also offers extensive documentation on the YDN that is full of helpful examples and tutorials.

Cool, but How do I Use this with my App?

Once you have become familiar with the YQL Console and the Open Data Tables available you can add results from here to your home-grown applications (I plan to cover this in a future article soon to come).  Yahoo supports direct YQL queries sent using HTTP GET, PUT, or DELETE  to handle SELECT, INSERT, UPDATE, and DELETE YQL functions to Open Data Tables. Users can even create their own Open Data Tables.

Advanced users of YQL can even use  EXECUTE to run custom JavaScript in Open Data Tables.

Ok, This is Great, but What’s the Cost?

Yahoo provides YQL and Open Data Table access free of charge for commercial and non-commercial use, which is great. The free access is limited to 100,000 calls per day per Access Key, or by IP on a per hour basis to 1,000 public calls and 10,000 YQL calls. These limits seem very viable for most applications, but if you are writing an application that will require more calls then Yahoo offers extended functionality at reasonable rates.