15.4 JPython ( Jython): Python for Java

JPython (recently renamed "Jython") is an entirely distinct implementation of the Python programming language that allows programmers to use Python as a scripting component in Java-based applications. In short, JPython makes Python code look like Java, and consequently offers a variety of technology options inherited from the Java world. With JPython, Python code may be run as client-side applets in web browsers, as server-side scripts, and in a variety of other roles. JPython is distinct from other systems mentioned in this section in terms of its scope: while it is based on the core Python language we've seen in this book, it actually replaces the underlying implementation of that language rather than augmenting it.^[2]

^[2] At this writing, JPython is the second implementation of the Python language. By contrast, the standard, original implementation of Python is sometimes now referred to as "CPython," because it is implemented in ANSI C. Among other things, the JPython implementation is driving a clearer definition of the Python language itself, independent of a particular implementation's effects. A new Python implementation for Microsoft's C#/.NET environment is also on the way (see later in this chapter) and may further drive a definition of what it means to be Python.

This section briefly explores JPython and highlights some of the reasons you may or may not want to use it instead of the standard Python implementation. Although JPython is primarily of interest to programmers writing Java-based applications, it underscores integration possibilities and language definition issues that merit the attention of all Python users. Because JPython is Java-centric, you need to know something about Java development to make the most sense of JPython, and this book doesn't pretend to teach that in the next few pages. For more details, interested readers should consult other materials, including JPython documentation at http://www.jython.org.

The JPython port is now called "Jython." Although you are likely to still see it called by its original JPython name on the Net (and in this book) for some time, the new Jython title will become more common as time goes by.

15.4.1 A Quick Introduction to JPython

Functionally speaking, JPython is a collection of Java classes that run Python code. It consists of a Python compiler, written in Java, that translates Python scripts to Java bytecodes so they can be executed by a Java virtual machine -- the runtime component that executes Java programs and is used by major web browsers. Moreover, JPython automatically exposes all Java class libraries for use in Python scripts. In a nutshell, here's what comes with the JPython system:

Python-to-Java-bytecode compiler

JPython always compiles Python source code into Java bytecode and passes it to a Java virtual machine (JVM) runtime engine to be executed. A command-line compiler program, jpythonc, is also able to translate Python source code files into Java .class and .jar files, which can then be used as Java applets, beans, servlets, and so on. To the JVM, Python code run through JPython looks the same as Java code. Besides making Python code work on a JVM, JPython code also inherits all aspects of the Java runtime system, including Java's garbage collection and security models. jpythonc also imposes Java source file class rules.

Access to Java class libraries (extending )

JPython uses Java's reflection API (runtime type information) to expose all available Java class libraries to Python scripts. That is, Python programs written for the JPython system can call out to any resident Java class automatically simply by importing it. The Python-to-Java interface is completely automatic and remarkably seamless -- Java class libraries appear as though they are coded in Python. Import statements in JPython scripts may refer to either JPython modules or Java class libraries. For instance, when a JPython script imports java.awt, it gains access to all the tools available in the awt library. JPython internally creates a "dummy" Python module object to serve as an interface to awt at import time. This dummy module consists of hooks for dispatching calls from JPython code to Java class methods and automatically converting datatypes between Java and Python representations as needed. To JPython scripts, Java class libraries look and feel exactly like normal Python modules (albeit with interfaces defined by the Java world).

Unified object model

JPython objects are actually Java objects internally. In fact, JPython implements Python types as instances of a Java PyObject class. By contrast, C Python classes and types are still distinct in the current release. For instance, in JPython, the number 123 is an instance of the PyInteger Java class, and you can specify things like [].__class__ since all objects are class instances. That makes data mapping between languages simple: Java can process Python objects automatically, because they are Java objects. JPython automatically converts types between languages according to a standard type map as needed to call out to Java libraries, and selects among overloaded Java method signatures.

API for running Python from Java (embedding )

JPython also provides interfaces that allow Java programs to execute JPython code. As for embedding in C and C++, this allows Java applications to be customized by bits of dynamically written JPython code. For instance, JPython ships with a Java PythonInterpreter class, which allows Java programs to create objects that function as Python namespaces for running Python code. Each PythonInterpreter object is roughly a Python module, with methods such as exec(a string of Python code), execfile(a Python filename), and get and set methods for assigning Python global variables. Because Python objects are really instances of a Java PyObject class, an enclosing Java layer can access and process data created by Python code naturally.

Interactive Python command line

Like the standard Python implementation, JPython comes with an interactive command line that runs code immediately after it is typed. JPython's jpython program is equivalent to the python executable we've been using in this book; without arguments, it starts an interactive session. Among other things, this allows JPython programmers to import and test class components actually written in Java. This ability alone is compelling enough to interest many Java programmers.

Interface automations

Java libraries are somewhat easier to use in JPython code than in Java. That's because JPython automates some of the coding steps Java implies. For instance, callback handlers for Java GUI libraries may be simple Python functions, even though Java coders need to provide methods in fully specified classes (Java does not have first-class function objects). JPython also makes Java class data members accessible as both Python attribute names (object.name) and object constructor keyword arguments (name=value); such Python syntax is translated into calls to getName and setName accessor methods in Java classes. We'll see these automation tricks in action in the following examples. You don't have to use any of these (and they may confuse Java programmers at first glance), but they further simplify coding in JPython, and give Java class libraries a more Python-like flavor.

The net effect of all this is that JPython allows us to write Python programs that can run on any Java-aware machine -- in particular, in the context of most web browsers. More importantly, because Python programs are translated into Java bytecodes, JPython provides an incredibly seamless and natural integration between the two languages. Both walk and talk in terms of the Java model, so calls across language boundaries are trivial. With JPython's approach, it's even possible to subclass a Java class in Python and vice versa.

So why go to all this trouble to mix Python into Java environments? The most obvious answer is that JPython makes Java components easier to use: JPython scripts are typically a fraction of the size of their Java equivalents, and much less complex. More generally, the answer is really the same as it is for C and C++ environments: Python, as an easy-to-use, object-oriented scripting language, naturally complements the Java programming language.

By now, it is clear to most people that Java is too complex to serve as a scripting or rapid-development tool. But this is exactly where Python excels; by adding Python to the mix with JPython, we add a scripting component to Java systems, exactly as we do when integrating Python with C or C++. For instance, we can use JPython to quickly prototype Java systems, test Java classes interactively, and open up Java systems for end-user customization. In general, adding Python to Java development can significantly boost programmer productivity, just as it does for C and C++ systems.

JPython Versus the Python C API Functionally, JPython is primarily an integration system: it allows us to mix Python with Java components. We also study ways to integrate Python with C and C++ components in the next part of this book. It's worth noting that we need different techniques to integrate Python with Java (such as the JPython compiler), because Java is a somewhat closed system: it prefers an all-Java mix. The C and C++ integration tools are generally less restrictive in terms of language assumptions, and any C-compatible language components will do. Java's strictness is partly due to its security goals, but the net effect is to foster integration techniques that are specific to Java alone. On the other hand, because Java exposes runtime type information through its reflection API, JPython can largely automate the conversions and dispatching needed to access Java components from Python scripts; Python code simply imports and calls Java components. When mixing Python with C or C++, we must provide a "glue" code layer that integrates the two languages explicitly. Some of this can be automated (with the SWIG system we'll meet later in this text). No glue code is required in JPython, however, because JPython's (and Java's) developers have done all the linkage work already, in a generic fashion. It is also possible to mix in C/C++ components with Java via its native call interface ( JNI), but this can be cumbersome and may cancel out Java's reported portability and security benefits.

15.4.2 A Simple JPython Example

Once a Python program is compiled with JPython, it is all Java: the program is translated to Java bytecodes, it uses Java classes to do its work, and there is no Python left except for the original source code. Because the compiler tool itself is also written in Java, JPython is sometimes called "100% pure Java." That label may be more profound to marketeers than programmers, though, because JPython scripts are still written using standard Python syntax. For instance, Example 15-4 is a legal JPython program, derived from an example originally written by Guido van Rossum.

Example 15-4. PP2E\Internet\Other\jpython.py

############################################
# implement a simple calculator in JPython;
# evaluation runs a full expression all at 
# once using the Python eval(  ) built-in-- 
# JPython's compiler is present at run-time
############################################
 
from java import awt                   # get access to Java class libraries
from pawt import swing                 # they look like Python modules here 

labels = ['0', '1', '2', '+',          # labels for calculator buttons
          '3', '4', '5', '-',          # will be used for a 4x4 grid
          '6', '7', '8', '*',
          '9', '.', '=', '/' ]

keys = swing.JPanel(awt.GridLayout(4, 4))     # do Java class library magic
display = swing.JTextField(  )                  # Python data auto-mapped to Java

def push(event):                              # callback for regular keys
    display.replaceSelection(event.actionCommand)

def enter(event):                             # callback for the '=' key
    display.text = str(eval(display.text))    # use Python eval(  ) to run expr
    display.selectAll(  )

for label in labels:                          # build up button widget grid
    key = swing.JButton(label)                # on press, invoke Python funcs
    if label == '=':
        key.actionPerformed = enter
    else:
        key.actionPerformed = push
    keys.add(key)

panel = swing.JPanel(awt.BorderLayout(  ))      # make a swing panel
panel.add("North", display)                   # text plus key grid in middle
panel.add("Center", keys)
swing.test(panel)                             # start in a GUI viewer

The first thing you should notice is that this is genuine Python code -- JPython scripts use the same core language that we've been using all along in this book. That's good news, both because Python is such an easy language to use and because you don't need to learn a new, proprietary scripting language to use JPython. It also means that all of Python's high-level language syntax and tools are available. For example, in this script, the Python eval built-in function is used to parse and evaluate constructed expressions all at once, saving us from having to write an expression evaluator from scratch.

15.4.3 Interface Automation Tricks

The previous calculator example also illustrates two interface automations performed by JPython: function callback and attribute mappings. Java programmers may have already noticed that this example doesn't use classes. Like standard Python and unlike Java, JPython supports but does not impose OOP. Simple Python functions work fine as callback handlers. In Example 15-4, assigning key.actionPerformed to a Python function object has the same effect as registering an instance of a class that defines a callback handler method:

def push(event):
    ...
key = swing.JButton(label)
key.actionPerformed = push

This is noticeably simpler than the more Java-like:

class handler(awt.event.ActionListener):
    def actionPerformed(self, event):
        ...
key = swing.JButton(label)
key.addActionListener(handler(  ))

JPython automatically maps Python functions to the Java class method callback model. Java programmers may now be wondering why we can assign to something named key.actionPerformed in the first place. JPython's second magic feat is to make Java data members look like simple object attributes in Python code. In abstract terms, JPython code of the form:

X = Object(argument)
X.property = value + X.property

is equivalent to the more traditional and complex Java style:

X = Object(argument)
X.setProperty(value + X.getProperty(  ))

That is, JPython automatically maps attribute assignments and references to Java accessor method calls by inspecting Java class signatures (and possibly Java BeanInfo files if used). Moreover, properties can be assigned with keyword arguments in object constructor calls, such that:

X = Object(argument, property=value)

is equivalent to both this more traditional form:

X = Object(argument)
X.setProperty(value)

as well as the following, which relies on attribute name mapping:

X = Object(argument)
X.property = value

We can combine both callback and property automation for an even simpler version of the callback code snippet:

def push(event):
    ...
key = swing.JButton(label, actionPerformed=push)

You don't need to use these automation tricks, but again, they make JPython scripts simpler, and that's most of the point behind mixing Python with Java.

15.4.4 Writing Java Applets in JPython

I would be remiss if I didn't include a brief example of JPython code that more directly masquerades as a Java applet: code that lives on a server machine but is downloaded to and run on the client machine when its Internet address is referenced. Most of the magic behind this is subclassing the appropriate Java class in a JPython script, demonstrated in Example 15-5.

Example 15-5. PP2E\Internet\Other\jpython-applet.py

#######################################
# a simple java applet coded in Python
#######################################

from java.applet import Applet                            # get java superclass

class Hello(Applet):
    def paint(self, gc):                                  # on paint callback
        gc.drawString("Hello applet world", 20, 30)       # draw text message

if __name__ == '__main__':                                # if run standalone
    import pawt                                           # get java awt lib
    pawt.test(Hello(  ))                                    # run under awt loop

The Python class in this code inherits all the necessary applet protocol from the standard Java Applet superclass, so there is not much new to see here. Under JPython, Python classes can always subclass Java classes, because Python objects really are Java objects when compiled and run. The Python-coded paint method in this script will be automatically run from the Java AWT event loop as needed; it simply uses the passed-in gc user-interface handle object to draw a text message.

If we use JPython's jpythonc command-line tool to compile this into a Java .class file and properly store that file on a web server, it can then be used exactly like applets written in Java. Because most web browsers include a JVM, this means that such Python scripts may be used as client-side programs that create sophisticated user-interface devices within the browser, and so on.

15.4.5 JPython Trade-offs

Depending on your background, though, the somewhat less good news about JPython is that even though the calculator and applet scripts discussed here are straight Python code, the libraries they use are different than what we've seen so far. In fact, the library calls employed are radically different. The calculator, for example, relies primarily on imported Java class libraries, not standard Python libraries. You really need to understand Java's awt and swing libraries to make sense of its code, and this library skew between language implementations becomes more acute as programs grow larger. The applet example is even more Java-bound: it depends both on Java user-interface libraries and Java applet protocols.

If you are already familiar with Java libraries, this isn't an issue at all, of course. But because most of the work performed by realistic programs is done by using libraries, the fact that most JPython code relies on very different libraries makes compatibility with standard Python less potent than it may seem at first glance. To put that more strongly, apart from very trivial core language examples, many JPython programs won't run on the standard Python interpreter, and many standard Python programs won't work under JPython.

Generally, JPython presents a number of trade-offs, partly due to its relative immaturity as of this writing. I want to point out up front that JPython is indeed an excellent Java scripting tool -- arguably the best one available, and most of its trade-offs are probably of little or no concern to Java developers. For instance, if you are coming to JPython from the Java world, the fact that Java libraries are at the heart of JPython scripts may be more asset than downside. But if you are presented with a choice between the standard and Java-based Python language implementations, some of JPython's implications are worth knowing about:

JPython is not yet fully compatible with the standard Python language

At this writing, JPython is not yet totally compatible with the standard Python language, as defined by the original C implementation. In subtle ways, the core Python language itself works differently in JPython. For example, until very recently, assigning file-like objects to the standard input sys.stdin failed, and exceptions were still strings, not class objects. The list of incompatibilities (viewable at http://www.jython.org) will likely shrink over time, but will probably never go away completely. Moreover, new language features are likely to show up later in JPython than in the standard C-based implementation.

JPython requires programmers to learn Java development too

Language syntax is only one aspect of programming. The library skew mentioned previously is just one example of JPython's dependence on the Java system. Not only do you need to learn Java libraries to get real work done in JPython, but you also must come to grips with the Java programming environment in general. Many standard Python libraries have been ported to JPython, and others are being adopted regularly. But major Python tools such as Tkinter GUIs may show up late or never in JPython (and instead are replaced with Java tools).^[3] In addition, many core Python library features cannot be supported in JPython, because they would violate Java's security constraints. For example, the os.system call for running shell commands may never become available in JPython.

^[3] But see the note at the end of the later section on Grail; an early port of Tkinter for JPython is already available on the Net.

JPython applies only where a JVM is installed or shipped

You need the Java runtime to run JPython code. This may sound like a non-issue given the pervasiveness of the Internet, but I have very recently worked in more than one company for which delivering applications to be run on JVMs was not an option. Simply put, there was no JVM to be found at the customer's site. In such scenarios, JPython is either not an option, or will require you to ship a JVM with your application just to run your compiled JPython code. Shipping the standard Python system with your products is completely free; shipping a JVM may require licensing and fees. This may become less of a concern as robust open source JVMs appear. But if you wish to use JPython today and can't be sure that your clients will be able to run your systems in Java-aware browsers (or other JVM components), you should consider the potential costs of shipping a Java runtime system with your products.^[4]

^[4] Be sure you can get a JVM to develop those products too! Installing JPython on Windows 98 while writing this book proved painful, not because of JPython, but because I also had to come to grips with Java commands to run during installation, and track down and install a JVM other than the one provided by Microsoft. Depending on your platform, you may be faced with JPython's Java-dependence even before you type your first line of code.

JPython doesn't support Python extension modules written in C or C++

At present, no C or C++ extension modules written to work with the C Python implementation will work with JPython. This is a major impediment to deploying JPython outside the scope of applications run in a browser. To date, the half-million-strong Python user community has developed thousands of extensions written for C Python, and these constitute much of the substance of the Python development world. JPython's current alternative is to instead expose Java class libraries and ask programmers to write new extensions in Java. But this dismisses a vast library of prior and future Python art. In principle, C extensions could be supported by Java's native call interface, but it's complex, has not been done, and can negate Java portability and security.

JPython is noticeably slower than C Python

Today, Python code generally runs slower under the JPython implementation. How much slower depends on what you test, which JVM you use to run your test, whether a just-in-time ( JIT) compiler is available, and which tester you cite. Posted benchmarks have run the gamut from 1.7 times slower than C Python, to 10 times slower, and up to 100 times slower. Regardless of the exact number, the extra layer of logic JPython requires to map Python to the Java execution model adds speed overheads to an already slow JVM and makes it unlikely that JPython will ever be as fast as the C Python implementation. Given that C Python is already slower than compiled languages like C, the additional slowness of JPython makes it less useful outside the realm of Java scripting. Furthermore, the Swing GUI library used by JPython scripts is powerful, but generally considered to be the slowest and largest of all Python GUI options. Given that Python's Tkinter library is a portable and standard GUI solution, Java's proprietary user-interface tools by themselves are probably not reason enough to use the JPython implementation.

JPython is less robust than C Python

At this writing, JPython is substantially more buggy than the standard C implementation of the language. This is certainly due to its younger age and smaller user base and varies from JVM to JVM, but you are more likely to hit snags in JPython. In contrast, C Python has been amazingly bug-free since its introduction in 1990.

JPython may be less portable than C Python

It's also worth noting that as of this writing, the core Python language is far more portable than Java (despite marketing statements to the contrary). Because of that, deploying standard Python code with the Java-based JPython implementation may actually lessen its portability. Naturally, this depends on the set of extensions you use, but standard Python runs today on everything from handheld PDAs and PCs to Cray supercomputers and IBM mainframes.

Some incompatibilities between JPython and standard Python can be very subtle. For instance, JPython inherits all of the Java runtime engine's behavior, including Java security constraints and garbage collection. Java garbage collection is not based on standard Python's reference count scheme, and therefore can automatically collect cyclic objects.^[5] It also means that some common Python programming idioms won't work. For example, it's typical in Python to code file-processing loops in this form:

^[5] But Python 2.0's garbage collector can now collect cyclic objects too. See the 2.0 release notes and Appendix A.

for filename in bigfilenamelist:
    text = open(filename).read(  )
    dostuffwith(text)  

That works because files are automatically closed when garbage-collected in standard Python, and we can be sure that the file object returned by the open call will be immediately garbage collected (it's a temporary, so there are no more references as soon as we call read). It won't work in JPython, though, because we can't be sure when the temporary file object will be reclaimed. To avoid running out of file descriptors, we usually need to code this differently for JPython:

for filename in bigfilenamelist:
    file = open(filename)
    text = file.read(  )
    dostuffwith(text)
    file.close(  )  

You may face a similar implementation mismatch if you assume that output files are immediately closed: open(name,'w').write(bytes) collects and closes the temporary file object and hence flushes the bytes out to the file under the standard C implementation of Python only, while JPython instead collects the file object at some arbitrary time in the future. In addition to such file-closing concerns, Python __del__ class destructors are never called in JPython, due to complications associated with object termination.

15.4.6 Picking Your Python

Because of concerns such as those just mentioned, the JPython implementation of the Python language is probably best used only in contexts where Java integration or web browser interoperability are crucial design concerns. You should always be the judge, of course, but the standard C implementation seems better suited to most other Python applications. Still, that leaves a very substantial domain to JPython -- almost all Java systems and programmers can benefit from adding JPython to their tool sets.

JPython allows programmers to write programs that use Java class libraries in a fraction of the code and complexity required by Java-coded equivalents. Hence, JPython excels as an extension language for Java-based systems, especially those that will run in the context of web browsers. Because Java is a standard component of most web browsers, JPython scripts will often run automatically without extra install steps on client machines. Furthermore, even Java-coded applications that have nothing to do with the Web can benefit from JPython's ease of use; its seamless integration with Java class libraries makes JPython simply the best Java scripting and testing tool available today.

For most other applications, though, the standard Python implementation, possibly integrated with C and C++ components, is probably a better design choice. The resulting system will likely run faster, cost less to ship, have access to all Python extension modules, be more robust and portable, and be more easily maintained by people familiar with standard Python.

On the other hand, I want to point out again that the trade-offs listed here are mostly written from the Python perspective; if you are a Java developer looking for a scripting tool for Java-based systems, many of these detriments may be of minor concern. And to be fair, some of JPython's problems may be addressed in future releases; for instance, its speed will probably improve over time. Yet even as it exists today, JPython clearly makes an ideal extension-language solution for Java-based applications, and offers a much more complete Java scripting solution than those currently available for other scripting languages.^[6]

^[6] Other scripting languages have addressed Java integration by reimplementing a Java virtual machine in the underlying scripting language or by integrating their original C implementations with Java using the Java native call interface. Neither approach is anywhere near as seamless and powerful as generating real Java bytecode.

For more details, see the JPython package included on this book's CD (see http://examples.oreilly.com/python2), and consult the JPython home page, currently maintained at http://www.jython.org. At least one rumor has leaked concerning an upcoming JPython book as well, so check http://www.python.org for developments on this front. See also the sidebar later in this chapter about the new Python implementation for the C#/.NET environment on Windows. It seems likely that there will be three Pythons to choose from very soon (not just two), and perhaps more in the future. All will likely implement the same core Python language we've used in this text, but may emphasize alternative integration schemes, application domains, development environments, and so on.