Two Useful Eclipse Plugins

FindBugs uses static analysis to look for bugs in Java code.

ANTLR IDE provides support for ANTLR parser generator.

1<<32 = ?

Let 1 be a 32 bit integer in Java, what should 1<<32 be?

0? Because the "1" bit will be shifted to the far left?

In fact, it is 1. See in the Java programming spec, "If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator & with the mask value 0x1f. The shift distance actually used is therefore always in the range 0 to 31, inclusive." So << can be considered as a cyclic left shift operation, the bits shifted on the left end will reappear on the right end.

Ways of Implementing DSL

DSL, domain-specific language, or simply our very own language intended to solve our own specific problem, may not be limited to a programming or scripting language, it can also be, for instance, a file format, or a network protocol. In any case, the language has a syntax which defines the legal vocabulary in the language, and a grammar which defines rules of composing words in the vocabulary to meaningful sentences and paragraphs. So in order to understand the language and to take the right action based on the language, what we need are a lexer that translates language stream into token stream, and a parser that maps tokens to the grammar rules.

There are many ways of implementing DSL.

First, if the language is really simple, we can just code the lexer and parser manually. I.e., manually separate character stream into tokens, and manually construct an LL recursive descent parser. Obviously this method won't work well if the language becomes complex.

Second, make the DSL an XML language. For instance, BPEL is a web service workflow language. Though BPEL is nothing more than an XML language, it surely can fulfill complex tasks. The advantages of making DSL as XML are: first we can use XML schema (XSD) to design the language, so we can also use XML schema validation to validate if something is valid according to the schema; second there are mature and popular tools for validating and parsing XML possibly in every progamming languages.

Third, we can go to the conventional way to design a complex language, i.e., to use a parser generator, such as Antlr. Antlr is implemented in Java, but it can be used to generate code in many other languages than Java, such as C++, Python, C# ... One of the reasons for Antlr is quite popular is that Antlr has a very sophisticated tooling for designing, visualising and even debugging grammar: AntlrWorks, which is quite impressive.

We can even combine the above two methods together when designing a DSL. For instance, XSLT, XSL Transformation, is an XML-based language for transforming XML documents into other documents. XSLT makes use of XPath, which can be implemented using Antlr.

Fourth, nowadays, modern dynamic languages like Ruby and Groovy can be used to implement many DSLs. Because such DSLs will live inside Groovy, they have to obey Groovy's grammar rules. Compared to languages implemented using Antlr, their flexibility is somehow restricted. See here for an introduction to designing DSL with Groovy.

Protocol Verification Using Tcpdump and Wireshark

It is a must to use Tcpdump (to capture) and/or Wireshark (to analyse) to verify protocol implementation when doing TCP/IP network programming.

First, it can verify if the message sent by me is as intended, thus it can expose any bug in my protocol implementation. Today a bug has been found in this sense: I use Java DataOutputStream.write(int) to send a short, so only the lowest byte of the short has been sent.

Second, it can detect any inconsistency between the protocol specification, i.e., what I think the message should look like, and the actual messages that are sent by the other party, i.e., out of my control. Today, several inconsistencies have been found in this way. For instance, the length of a message (a TCP packet) is different from that defined in the spec. Clearly the spec in my hand is out of dated.

Without Tcpdump and Wireshark, I would be like walking in the dark when implementing my protocol.

Capture and Analyse Network Packets

tcpdump is the standard packet capturing facility available on most Linux systems, which is based on command line. Wireshark, formerly called Ethereal, is another popular packet capturing facility, free and GUI-based. Both tcpdump and Wireshark are based on pcap, so it is possible to combine them in capturing and analysing network packets, to take advantages of both.

For instance, I use the following tcpdump command to capture the traffic to and from www.google.com using http protocol:

sudo tcpdump -i wlan0 -w td.dat -nnvvXSs 1514 host www.google.com

Note:
sudo: It may require root privilege to capture packets.
-i wlan0: By default tcpdump captures packets on the eth0 interface. Since I am using wireless, I need to specify the wireless interface wlan0. When using VPN, the interface should be ppp0 instead usually.
-w td.dat: write all captured packets to the file td.dat.
-nn: no hostname and port resolving.
-vv: very verbose.
-X: print in both hex and ascii.
-S: absolute sequence.
-s 1514: tcpdump takes the first 68 bytes of data from a packet by default. Here the first 1514 bytes are taken.
host www.google.com: this is the expression which says capturing packets whose dst host or src host is www.google.com.

See this tcpdump tutorial for more info about tcpdump usage.

Now Wireshark can be used to analyse the captured packets by tcpdump. Here Wireshark's GUI is exploited.

Use Wireshark to open td.dat, and apply the preset http filter. The http traffic can be easily browsed.

Buzzwords in Job Descriptions

In these days, I am looking at job descriptions for senior Java developer position. Here is the list of buzzwords appearing inside. The list will grow when I come across more buzzwords.

JMeter: a Java framework for measuring server performance. Server types include Web, Web service, database (via JDBC), LDAP, JMS, mail (POP3 ...

Selenium: a Firefox extension that allows composing web tests inside Firefox, replaying tests and generating tests in many different programming languages such as Java, C#, Ruby, Groovy ... This is a good example for what extra functionalities Firefox extension can bring to the browser.

Memory Overhead of Java Objects

First, each Java object has two implicit references: one to its monitor (lock), the other to its method dispatch table. Each reference occupies 4 bytes, so that is 8 bytes overhead.

Second, byte alignment needs to be taken into consideration. On a 32-bit machine, object needs to be aligned at 4-byte boundary. On a 64-bit machine, object needs to be aligned at 8-byte boundary. Nowadays, 64-bit machines become popular. For instance, my laptop has a 64-bit Intel Core 2 Duo CPU T7500.

So on my laptop if I create an object that has only one byte field, then the actual size of the object will be 16 bytes. That is 93.75% overhead. If I create an object that has three int fields, then the actual size of the object will be 24 bytes. That is 50% overhead.

So be very careful when creating a huge number of small objects, because significant extra memory will be required for object overhead.

How to measure the size of an object

Write a simple program consisting of an infinite loop. Inside the loop, create the object whose size is to be measured. Then use "jmap -histo pid" to measure the size. pid is the process id of the Java program. Because the loop is infinite, it gives plenty of time for jmap to connect to the Java process.

Adjustable Timer Task

Java provides TimerTask for tasks that can be scheduled for one-time or repeated execution by a Timer. But it has some serious drawbacks.

First, TimerTask can only be scheduled either for one-time or for repeated execution of a roughly fixed period. It can not be scheduled with changing periods, for example, at the following moments since its start: 0, 5, 6, 12, 100, 101, 102 seconds ... If it is scheduled twice, for example, first scheduled after 1 seconds, then 3 seconds after the first execution, no matter in which thread the second schedule is requested, inside or outside the Timer thread, a java.lang.IllegalStateException: Task already scheduled or cancelled will be thrown.

Second, each TimerTask is served with an individual background thread. Thus it makes TimerTask not a very scalable solution if a lot of TimerTasks are needed but each of them is not very heavy weighted.

Since 1.5, Java provides the java.util.concurrent package, which includes an Executor Framework. The basic idea of Executor Framework is to separate the concerns of tasks and the mechanism to execute tasks. So programmers define tasks and then leave tasks to be executed by the Executor Framework, which can be configurable to use a single thread, or a thread pool to execute the tasks.

The following Java code illustrates how to use ScheduledExecutorService to implement adjustable timer task.

public class Test {

 static private ScheduledExecutorService scheduler =
     Executors.newSingleThreadScheduledExecutor();

 static private Runnable pig = new Runnable() {
     public void run() {
         System.out.println("This is pig");
         scheduler.schedule(this, 1,
            TimeUnit.SECONDS);
     };
 };

 static private Runnable bear = new Runnable() {
     Random random = new Random();
     public void run() {
         System.out.println("This is bear");
         scheduler.schedule(this,
            2 + random.nextInt(3),
            TimeUnit.SECONDS);
     };
 };

 /**
  * @param args
  */
 public static void main(String[] args) {
     scheduler.schedule(pig, 0, TimeUnit.SECONDS);
     scheduler.schedule(bear, 0, TimeUnit.SECONDS);
 }

}

Globus Toolkit's Future

Today Ian Foster sent out an email about Globus' future plans, which contains many interesting points.

First, a GRAM5 (for job submission) will come out as a replacement for both GRAM2 (old fashion) and GRAM4 (web service based). It confirms my feeling that web service based job submission such as GridSAM does not pick up enough users. The old fashion job submission just can not be retired at this moment.

Second, the Java web service core will be re-implemented to leverage state-of-the-art technologies. Apache CXF is selected as the web service development kit. See here for a very informative comparison between several popular latest the web service development kits including Apache CXF, Apache Axis2 and Metro (the JAX-WS RI). Apache CXF is favored over Apache Axis2 because Axis2 uses proprietary deployment model and lacks the support for IoC containers such as Spring Framework.

Third, globus.org will provide RFT (reliable file transfer) online service, which provides reliable, high-performance, end-to-end, fire-and-forget data transfer. globus.org has moved into the era of cloud computing!

Fourth, MDS (monitoring and discovery service) will be refactored to separate monitoring and service/resource discovery. Monitoring will be left to mature dedicated monitor softwares. MDS itself will focus on acting as a service registry. I certainly agree this decision since it favors the idea of separating concerns. The recently published GLUE2 spec still mixes up monitoring data and service metadata for discovery. It is interesting to see if the future development of GLUE will reflect the decision made on MDS.

Jar Service Provider

Jar File Spec provides a simple service provider mechanism: a provider configuration file is located inside META-INF/services/, with the service interface or abstract class name as the configuration file name, which contains the list of implementing class names. Java 1.6 has a ServiceLoader class for looking up service providers.

It will be handy to define a well known interface, and to use the provider configuration file to list all implementations. Thus at runtime, ServiceLoader can be used to iterate all available service providers.

Set up HTTP Cookie in Java

In my Java programming, I need to set up some pre-defined HTTP cookie so that I can tell the website what content I prefer on using HttpURLConnection, for example. This is what I do:

// Create my cookie
HttpCookie cookie = new HttpCookie("cookie_name", "cookie_value");
// So that the cookie applies to all pages
cookie.setPath("/");

CookieManager cookieManager = new CookieManager();
cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);

CookieStore cookieStore = cookieManager.getCookieStore();
cookieStore.add(uri, cookie);

// Set my cookie manager that contains my cookie to be used system-wide
CookieHandler.setDefault(cookieManager);

// From now on my cookie will be used for all connections to the web site denoted by uri.

Web Scraping in Java

There are at least three ways to do web scraping in Java.

First, "manually" use string matching and regular expression to extract information from downloaded HTML.

Second, use JTidy to transform HTML to XHTML, and then use XQuery (e. g., Saxon, ...) over XHTML to extract required information.

Third, which is what I prefer:

1. Create a TagSoup HTML parser, which provides an SAX interface;
2. Use XOM to build a DOM from HTML using the TagSoup SAX parser;
3. Use the built-in XPath query facility inside XOM (i.e., Jaxen) to parse the XOM DOM document.
   

A sample code skeleton looks like:

// Create a TagSoup SAX parser.
XMLReader parser = new org.ccil.cowan.tagsoup.Parser();

// Use the TagSoup parser to build an XOM document from HTML.
Document doc = new Builder(parser).build(new File("index.html"));

// Do some XPath query: find all "table" elements.
Nodes nodes = doc.query("//*[local-name()='table']");

Tomcat Admin Web App and JMX

Tomcat has provided an Admin webapp, which sits inside $CATALINA_HOME/server/webapps in order to access classes contained in Tomcat jars, to make it easy to configure webapps, for instance, to add a DataSource to a webapp.

As seen from its soource code, the Admin webapp simply creates JMX MBeans (managed bean), and save them to a MBean server. Tomcat MBean server then rewrites Tomcat server.xml and webapps/webapp/META-INF/context.xml.

The JMX Remote API specification details how an LDAP server can be used to store and retrieve information about JMX connectors exposed by JMX agents. JNDI is used to talk to an LDAP server.

MBeans can be viewed in the MBeans tab of jconsole.

Further digging on MBeans in Tomcat

• All key constructs in Tomcat, such as Server, Engine, Host and Context, are implemented as MBeans, see package org.apache.catalina.core.
  
• Since Tomcat makes use of Apache Commons Modeler to deliver the Model MBean support, mbeans-descriptors.xml (read by Apache Commons Modeler) appears in many packages that contain MBeans.
• Tomcat uses the MBean server implementation provided by JVM.
• The Server and Context MBeans support operations to store their configurations, which are delegated to the StoreConfig MBean that implements the logic of rewriting various Tomcat configuration files, see package org.apache.catalina.storeconfig in container/modules/storeconfig.
• org.apache.catalina.storeconfig implements a StoreConfigLifecycleListener that registers the StoreConfig MBean right after Tomcat is started. StoreConfigLifecycleListener is configured in Tomcat server.xml.

The information above is based on Tomcat 5.5.27.

Troubleshooting Remote Java Application

Previously, I was using SSH+VPN to reduce the task of troubleshooting a remote Java application to the task of troubleshooting a local Java application. This is a generic approach, but has its own disadvantages. One of them is that sometimes the proper debugging/profiling/monitoring tools can not be run in remote machine due to the restraints in the deployment environment. In that case, we have to run tools locally and perform a genuine remote troubleshooting.

Remote jconsole

To start a Java application that supports remote jconsole is easy: just add the following into Java command line arguments:

-Dcom.sun.management.jmxremote.port=portNum -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

Then we can ask jconsole to simply connect to hostname:portNum, given that a firewall is not set up on the remote machine.

jconsole uses JMX which is built on RMI. The portNum we specify is the port number used by the RMI registry. The actural RMI connection will be opened using another port that can not be specified as a Java command line argument. When the firewall on the remote machine is disabled, and the local jconsole is successfully connected to the remote Java application, we can use "lsof -p pid | grep TCP" to check which port is used by RMI.

See this and that for a programmatic approach to get jconsole through the firewall. It basically starts up a customised RMI registry that open a RMI channel on a pre-defined port, which is implemented as a Java Agent. Here is an almost "official" tutorial to achieve that.

Remote jvisualvm

First of all, according to VisualVM's document, VisualVM can retrieve monitoring information on remote applications but it cannot profile remote applications. VisualVM requires jstatd running on the remote machine. Since jstatd is based on RMI, so VisualVM suffers from the same issue as jconsole when facing firewall.

Remote YourKit

Maybe it is easier to set up YourKit Java profiler to troubleshoot a remote Java application. And YourKit is claimed to be so lightweight that it can be used when the application is running in the production mode.

YourKit provides some means to integrate with a remote JEE/servlet container, such as Tomcat, JBoss, WebSphere, WebLogic ... For Tomcat, the integration creates a startup_with_yjp.sh based on startup.sh, which simply adds the following magic Java options:

"-agentpath:YJP_HOME/bin/linux-x86-32/libyjpagent.so=disablestacktelemetry,
disableexceptiontelemetry,delay=10000,
port=16666,sessionname=Tomcat"

"port=portNum" can be used to specified a number instead of the default 10001. To connect the remote Java application with YourKit agent turned on, simply ask to connect to serverName:portNum in the local YourKit UI. Since all profiling data go in the specified port number, it is easy to set up an SSH tunnel if that port is blocked by firewall.

More investigation needs to be done to understand the profiling overhead given that setting.

More on Java Troubleshooting

JDK has provided better and better troubleshooting facilities, among other tools. Obviously they are extremely important to develop serious Java applications, which usually have to offer specified performance under resource restraint.

These tools include

• jinfo pid : check Java command line options and system properties.
• jstack pid : print thread's stack the their status (e.g., blocking on some object). This is great for knowing what's going on inside the Java application at runtime. And there is no need to set up JVM specially for this purpse.
• jvisualvm : profile and monitor Java process. It seems JDK on Linux has jvisualvm by default. But you need to install jvisualvm separately on Windows.

as well as some other tools already mentioned before:

• jconsole
• jmap
• jhat

Clean up Harddisk before Dispose of Computer

After 3 years' working as a heater, my Sony VAIO laptop is finally retiring. There is one critical task that must be done before it can be disposed of: cleaning up its harddisk. Since I graduated from a computer lab that researched and developed one of the very first harddisks in China, I long know not only deleted files but also overwritten files can be recovered. Nevertheless, this blog as well as its comments still give me a lot of information. There are some useful points:

• On Linux, shred can be used to clean up disk. By default it overwrites the harddisk for 25 times. Call it like this: shred -vz /dev/hda (-z: finally overwrite with all zero to disguise the shred process.)
• Or, use DBAN, which is used by many governments. DBAN stands for "Darik's Boot And Nuke". The software is a boot CD/DVD image that is used to boot the computer and then do the cleaning up.
• Here is the seminal paper describing the theory behind securely deleting harddisk data.

It will be useful to know how long it takes for shred or DBAN to process a harddisk of a certain size.

On my Sony VAIO VGN-SZ3XWP/C, shred takes about 3.5 hours to 4 hours to run a random pass of a 39 GiB disk partition! A random pass overwrites the disk with random bits. Not sure how random it is. The 1st and the 13th (possibly the last one, the 25th pass) passes are random passes. The other passes use different short fixed pattern to overwrite. Each costs about half an hour.

Further updates for running "shred -vz /dev/sda" on a 94GiB harddisk. The 1st, 13th and 25th passes are using random data generated from /dev/urandom. The other passes write fixed but different bit patterns. It takes 49h33m26s to finish.

IDE for C++

I may work on a C++ project in future. So I am looking for an IDE for C++. I know A good IDE can double the productivity of programming.

Unlike Java, C++ is not cross platform and has many variants though it does have a standard. Thus which IDE to choose is dependent on which C++ to use.

On Windows, Microsoft Visual Studio is the choice for Visual C++ :)

I would like to choose Eclipse CDT for GNU C++. CDT is only IDE. To build and debug C++ code, CDT requires external toolchain, e.g., GNU toolchain. CDT on Liunx will automatically pick up GNU toolchain, which usually is there. But to make CDT on Windows work, we need to install MinGW (Minimalist GNU for Windows). Here is a good tutorial on installing MinGW. Once MinGW is properly installed and added into PATH, CDT can pick up MinGW as a toolchain automatically.

Troubleshooting OutOfMemoryError

When a large running Java application throws out an OutOfMemoryError, it indicates either the existence of a memory leak bug, or simply the fact that the maximum heap size has been reached. Don't panic, it is straightforward to do troubleshooting.

Most importantly, DO NOT SHUTDOWN the problematic JVM. Keep the crime scene.

1. Use jmap to dump the heap. Under JDK 1.5, the dumped heap, in the format of hprof, is always put under the home directory and under the name "heap.bin". In addition, jmap can also be used to show the heap object histogram, a quick way to see the classes occupying the most space.
2. Use jhat to analyze the heap dumped by jmap. If the heap is dumped by a JDK 1.5 jmap, invoke jhat (that is only available in JDK6+) in this way: jhat -J-mxNm -stack false heap_dump_file. "-stack false" turns off tracking object allocation call stack because the allocation site information is not available in the heap dump. N should be larger than the maximum heap size used when running the problematic Java process. jhat takes quite some time to analyze the heap dump.
   
3. jhat starts up a web server after finishes analyzing the heap dump. Now we can use a browser to point to the jhat web server, and find out who use up all the memory.
4. At the very end of the front page, click the link "Show heap histogram". it takes quite some time to generate the histogram.
5. In the histogram, the classes whose instances occupy most of the heap can be easily identified. Obviously they are suspects of the crime.
6. Clicking one of the suspects brings us to the page showing the referers of the suspect. In this way, we can track down which part of our code hold references to objects that use up heap. Now by using our knowledge of the program logic, the problem cause can be finally located.

See, the method is staightforward. All we need is patience and a good understanding of the source code in order to find the problem cause.

Direction+

I am working on a web app, Direction+, running on Google App Engine for Java. Yes, it is another direction service that suggests a driving route based on your chosen source and destination. But it has some add-on features as its name suggests: Direction Plus.

First, you can personalise Direction+ by providing your car's information, in particular, the fuel consumption measurement of your car, for urban cycle, extra-urban cycle and combined cycle. Thus, when calculating the route, Direction+ is also able to calculate how much fuel you will use and how much it costs (based on the fuel price you set). The fuel consumption calculation is measured on each individual step of the route. For instance, driving on a motor way will be much more fuel-efficient than driving in the city center. Isn't it good to know the cost of the driving before hand? In addition, the calculation is done in your own browser and your personalisation is stored in your browser as a cookie. So there is no privacy concern at all.

Second, Direction+ is a mashup that combines Google Maps Service and BBC travel news service together. When a driving route is calculated, the travel news, e. g., accidents news, along the route are displayed as well on the map. Isn't it good to know the traffic situation before driving? Currently too many travel news are returned especially when the route is very long, efforts are being made to improve the algorithm of filtering the not-so-relevant news.

How quick is Direction+ updated with BBC travel news? Please see its status page. A complete update takes about 30 minutes. For instance, at the time of writing, the last update took 26 minutes to finish. It read 2297 news from BBC, among which only 50 happened after the previous update, and among the 50 news that newly happened, 12 have cached coordinate in the data store. So though the number of news is large, Direction+ manages to reduce the communication to a small amount.

One of the tricky things about GAE for java is that every servlet request must be served within 30 seconds. So the update of news is implemented in an incremental and on-the-fly way:

• Incremental: the complete update is divided into many small steps, and each step can be finished in a controlled time slot;
• On-the-fly: the news updated in each small step is available immediately, long before the complete update is done.

Direction+ is designed for UK users because I live in UK. Enjoy it and let me know what you think!

Monitoring/Profiling/Debugging Java Process Remotely

Usually there are two difficulties involved:

1. How to communicate between the remote (target) machine and the local (console) machine? This varies per monitoring/profiling/debugging method. E.g., the built-in JMX console (jconsole) uses remote RMI.
   
2. How to penetrate the firewall if existing?

The generic approach is:

1. Know how to do it locally;
2. Run a vncserver on the remote machine (In fact, vncserver is lightweight, and easy to install in case there is no vncserver installed on the remote machine);
3. Use ssh to set up a tunnel between the remote machine and the local machine, e.g., ssh -L5901:remote_machine:5901 userid@remote_machine (ssh is very user-friendly w.r.t. firewall);
   
4. Start a vnc verview locally, pointing to localhost:1, and follow the procedure established in step 1.

In this way, remote monitoring/profiling/debugging tasks are doable following a step-by-step procedure.

Of course, sometimes, there is no need to set up a vncserver as long as the communication between the remote machine and the local machine is done in TCP and the port number is known.