Yesterday, one of our engineers here at SnapLogic pointed me to this page here. That’s a simple speed comparison between Python and Java, done by Glyph Lefkowitz some time ago. The tests consist of small code snippets – written first in Python and then in Java – each exercising a specific area such as object creation, loop performance, etc. It’s clearly subjective, and some of the test will depend on many other factors. Still, it’s an interesting set of tests.
Two things are remarkable about those tests:
- The Python code is significantly more compact than the Java code. Completely independent of the actual run time performance, this means that the programmer’s productivity should be higher, with less lines of code to debug and maintain later on. I have not replicated the code snippets here on this page, but do take a look at them on Glyph’s page. It is truly impressive.
- The tests indicated that Python was significantly faster than Java in some cases (and also much slower in others). I was assuming that Java should be able to run faster than Pyhton in most cases, though.
The comparison was done with Python 1.5.2 and JDK 1.1. Clearly not the latest and greatest version of each. Glyph mentions on this page that Java has improved in speed since then, but he feels that the basic conclusions still hold.
I decided to redo several of the tests with updated versions of Python (2.5) and the JDK (Java 6). And indeed, my suspicions were confirmed: Java has made huge speed improvements, and is now faster than Python in almost all cases.
Also, I ran the Python tests with the help of the Psyco Python compiler. This resulted in speed improvements in all cases, and yielded a significant speedup for some tests. The only changes in the Python source were to import psyco, move the test itself into a function and call
psyco.full() as the first statement of the program.
Update (July 11th): The timing was done in such a way that startup cost and interpreter initialization time was NOT taken into account. This means that contrary to the original article by Glyph I am not trying to comment on the suitability of a language for piped, small scripts. Considering that we are working here at SnapLogic on a data integration server, we are naturally more interested in server performance. Therefore, I don’t mind the startup cost at all. This should be kept in mind. The actual timing itself was done quite naively: In the code I take a high-resolution time stamp just before the loop starts, and right after it ends. Then I print the difference. That’s all. Since such a timing is clearly influenced by other activities of the system, I repeated each run several times. I am aware that the results are not 100% accurate. However, they are indeed close enough to give us a decent impression of the run time performance. Note how the conclusion describes the speed comparison in approximate terms because of this.
Below now are the details for the tests on my Dell D820 laptop, Intel Core Duo, 2.16 GHz, 2 GBytes, Ubuntu 7.04:
|Test||Java 6||Python 2.5||Psyco||Comparison|
|Standard Output||29.47||27.96||25.65||Python and Psyco slightly faster|
|Hashtable*||0.22||0.46||0.12||Psyco 2X faster than Java,
Java 2X faster than Python
|I/O||0.49||1.12||0.68||Psyco 1.4X slower than Java,
Java 2X faster than Python
|List||0.10||0.45||0.06||Psyco almost 2X faster than Java,
Java 4.5X faster than Python
|Object Allocation||0.14||6.75||3.17||Psyco 20X slower than Java,
Java 50x faster than Python
|Interpreter Speed**||0.004||0.37||0.12||Psyco 30X slower than Java,
Java 90X faster than Python
* Please see the update at the bottom of this post regarding the use of more
modern Java features, such as HashMaps with generics, that can result in significant speed improvements.
** Apparently, Java optimizes away a loop without side effects. Therefore, I added
a simple counter calculation to this test. Even so, the speed difference is significant.
A few observations:
- It can be seen here that the speedup one can obtain through Psyco is truly considerable, and can even be noticed on IO bound operations – probably due to the optimized loops around the test statements.
- Java’s performance in tight loops is excellent. However, one of the reasons for this may be the arbitrary precision integers used by Python vs. the more ‘native’ types used in Java. If the Java data type for the for-loop variable in the SpeedTest example is changed from integer to long, the Java performance already drops significantly (about twice), indicating the impact that the counter type has on performance. Python’s arbitrary size integers are convenient, but they are definitely not helping its performance.
- Object allocation in Python remains much slower than in Java. This is not at all helped by changing to new-style classes (the original code for the test used old-style classes). In fact, the new-style classes are even marginally slower.
- The Hashtable test for Python seems to be mostly limited by Python’s slower loop performance. Java apparently has increased its loop performance significantly, since this test originally ran slower in Java when conducted by Glyph a while ago. But when the Python test is now run with Psyco, the speed is better than that of Java. Psyco would only have an impact of the surrounding loops that are constructed for the test, not on the core Hash object, which is written in C anyway. Therefore, we can see that the build-in Hash object for Python genuinely performs better than Java’s version, while Java shines with its loop performance. Update: Please see the update at the bottom of this post regarding newer Java features, such as HashMaps with generics that can result in performance improvements.
- An interesting observation: For the Psyco tests the test loops were moved into a function (Psyco only optimizes functions). When I ran those tests even without Psyco, the performance was already improved – just because the code was now located inside of a function. The speedup differed from a few percent to 100% depending on the test. Apparently, we can see that the definition of a function object allows the interpreter to make some optimizing assumptions. Java’s code always runs inside of a function, and thus these optimizations can always be made. In the future, a realistic comparison of the Python performance with the Java performance would always see the Python test code inside of a function.
Python is really compact, and allows you to write code more quickly than Java. Therefore, it can improve programmer efficiency, which is often more of a scarce resource than CPU cycles.
For IO bound operations, the speed difference is much less important, which means that in most real world applications one does not need to be too concerned with the Python performance penalty.
Psyco is really good! It should be used more often. If you don’t use it you might want to take a look at it.
Update (July 11th): Several people have pointed out that Java now provides different ways to do the same thing, which may result in better performance. For example, the use of HashMaps and generics. Or buffered IO for the console output test. I made a few comparisons and in general those changes do result in some significant performance improvements. However, the code used here for comparison is roughly the same as the code in Glyph’s original paper, without those modern additions. I may be mistaken, but frankly, I don’t think that the use of buffered IO for a simple console output test is what would intuitively come to mind. Mind you, if I think about buffered IO under Java for performance improvement then I should also consider that for the Python cases, which I am not doing here. Simple IO operations are used instead in both cases, and that is what is compared.
However, the use of HashMaps under Java does provide a very significant speedup over the naive Hashtable example. Using HashMaps makes Java around 4 times faster than Python with Psyco in that test. I would agree with those who have submitted this suggestion that it is indeed the way to go these days in Java. In that case, the original HashTest here compares Python’s performance to a somewhat outdated feature in Java. This is unfortunate, but I don’t want to start changing the sources around now and redo the test. I won’t optimize the Python code to use new or better features, and also won’t change the Java code. The goal of the original test sources was to compare the code that initially comes to mind, even if it is somewhat naive. The code that comes to mind of course depends on the person that writes it, that’s understood.