Can you design a good benchmark to compare Apache Mina and Python Twisted?

There is an interesting blog entry Mina and Twisted Matrix benchmark with update, it claimed that Twisted out-perform Apache Mina with the blogger's benchmark.

It actually raised a very interesting question: can we design a good and fair benchmark to evaluate Apache Mina and Python Twisted?

The following are some of my rough thoughts on a might-be good benchmark to evaluate Apache Mina and Twisted for TCP based Networking Application:

1. The benchmark should run on a platform which both Mina and Twisted can utilize its advanced Non-blocking Networking feature, such as Linux with epoll, otherwise we are not really benchmark two frameworks on a fair playground

2. Since both Apache Mina and Twisted have protocol processing component built in, we can define a simple Text Protocol with a simple format like "[Header][PayLoadLength][PayLoad]" and a binary protocol sending image for protocol processing. Based on these, we can create a group of sample protocol messages with different size/length for benchmark

3. We can create a test driver which will open a TCP Connection to the server, send a sample protocol messages to the server, the server will decode the message, construct a response with a different format like "[Header][PayLoad][PayLoadLength]", and echo back the payload received and close TCP Connection, the driver can process the response and make sure that the payload is correctly returned by the server.

We can increase concurrent connections to compare total connections established, the processing time and total payload size measured at driver side

4. We can create a test driver which will open a persistent TCP Connection to the server, and keep sending protocol messages to the server and server keep echoing back the payload received in a similar fashion as 3.

We can increase concurrent connections to compare total connections established, the processing time and total payload size measured at driver side

5. Since both Apache Mina and Twisted support building Networking Client and Server, the test actually can be done in the following combination:
-- Apache Mina Client against Twisted Server
-- Twisted Client against Apache Mina Server
-- Apache Mina Client against Apache Mina Server
-- Twisted Client against Twisted Server