Selling Data Through Web Service

StrikeIron, "data as a service", provides a marketplace where people can put web services or purchase hits on web services that make available data which value is dependent on time, for instance, foreign currency exchange rates.

The key of business model is to sell valuable real time streamed data. And WS is adopted to simplify the data service implementation. The idea is cool and simple. And seems it works. I mean StrikeIron does have quite a few customers.

By googling "web serivces marketplace", I am aware Amazon has filed a patent of web services marketplace on 2005.

If someone is able to build up valuable data by doing data mining on Internet with the help of tools like google, and use StrikeIron to host a WS to provide the data, then (s)he can make money!

AJAX Enabled GrimoiresScript Interpreter

I implemented a toy language to handle Grimoires registry, which now I renamed as GrimoiresScript. Now I created a web-based GrimoiresScript interpreter, using AJAX to communicate between HTML page that takes in user input, and server-side JSP that invokes GrimoiresScript interpretation engine. The implementation follows the example in w3schools' AJAX tutorial.

AJAX is nothing but, or mostly a JavaScript RMI, which enables HTML pages to invoke remote methods.

Web Services Explorer in Eclipse

Web services explorer in eclipse can be used to browse UDDI registries, to parse services' WSDLs, and even to invoke service operations by sending them SOAP messages. It is implemented as a web application running on eclipse' embeded application server. So I am thinking to turn it into a standalone web application to act as Grimoires' web-based interface. As Chris Brealey pointed out, the WTP team also has this idea, but it is not easy to do this and the WTP team simply has no resource to be allocated on this. I played web services explorer today. It relies on quite a lot eclipse plug-ins. Even it could be turned into a standalone web application, it would be of a large footprint.

Monitor Memory Usage in Java

There is no "sizeof" in Java. Originally there was no way to check or estimate how much memory has been used in Java. But since Java 1.5, we can use the Monitoring and Management facility to monitor memory usage. The following code

MemoryMXBean mbean = ManagementFactory.getMemoryMXBean();         
System.out.println(mbean.getHeapMemoryUsage().toString();
System.out.println(mbean.getNonHeapMemoryUsage().toString();

will print out

init = 0(0K) used = 319360(311K) committed = 2031616(1984K) max = 66650112(65088K)
init = 8585216(8384K) used = 2379416(2323K) committed = 8880128(8672K) max = 100663296(98304K)

MemoryMXBean is also able to send out notification when some memory usage threshold is met.

Call Google AJAX Search Service in Java

Google has both a SOAP style service and an AJAX style for search. But it seems google focuses on the AJAX style rather than the SOAP style. For instance, the AJAX style has more functionalities than the SOAP style.

At the client side, the AJAX style is implemented as Javascripts embeded in a HTML page, which makes it impossible to be called from JDK script engine, at least not in a straightforward way.

Davanum Srinivas has published an interesting blog on how to call google ajax search service in Java. He used an reverse engineering approach to capture the url that the javascript sent to the google server, by some TCP/IP monitor. By manipulate the captured url, it is able to call google search in Java. The url looks like:

"http://www.google.com/uds/GwebSearch?"callback=GwebSearch.RawCompletion&context=0&
lstkp=0&
rsz=small&
hl=en&
sig=8656f49c146c5220e273d16b4b6978b2&
q=Axis2&
key=xxxxxxxxxxxxxxxxxx&
v=1.0"

When rsz is set to small, four records are returned; when large, eight are returned.

The search result contains result objects in JSON format, which can be parsed using some JSON parser.

It is easy to call google AJAX search service in Java. The challenge is when we have the capability to google repeatedly, untiredly, in brute force, or in a massive way, anything interesting can we achieve then?

Bridge GLUE LDAP schema and GLUE XML schema

GLUE is an information model to describe Grid resources. It has both an LDAP schema and an XML schema as its concrete data models. Usually GLUE data is in LDAP DIT in Grids. I need to read the LDAP data, convert to XML data according to GLUE schema, and then publish into Grimoires registry. What I am doing is:

1. Use JAXB to generate Java bean classes according to GLUE XML schema;
2. Parse GLUE LDAP schema with the help of an ANTLR-based LDAP schema parser, and generate Java code that converts LDAP entries to Java beans using the classes generated in step 1;
3. Use JNDI to read GLUE LDAP DIT, call the code generated in step 2 to create beans that are further serialized to XML (in GLUE schema) by JAXB;
4. Publish GLUE XML to Grimoires through the XMLView interface, which takes the configuration of XSLT between GLUE schema and Grimoires schema.
   

A Toy Language to Handle Grimoires Registry

I created a toy language to handle Grimoires registry using ANTLR. It also has a simple GUI to type and run the script. So I can type something like:

publish business (name="BusinessAA", desc="a business") -> $bk.

publish service (name="ServiceAA", desc="a service",bkey=$bk, wsdl="http://a/b/c.wsdl", address="http://a/b/") -> $sk.

attach metadata (type="http://www.grimoires.org/QoS", value="very good") to service (skey=$sk).

If run this script, it will first publish a business with name and description, and save the business key to a variable $bk, which will later be used in publishing service. It then will publish a service and attach a metadata to the published service.